Posts in category Papers

Can Learners be Earners? Investigating a Design to Enable MOOC Learners to Apply their Skills and Earn Money in an Online Market Place

This article is published in the journal IEEE Transactions on Learning Technologies.

Abstract

Massive Open Online Courses (MOOCs) aim to educate the world. More often than not, however, MOOCs fall short of this goal — a majority of learners are already highly educated (with a Bachelor degree or more) and come from specific parts of the (developed) world. Learners from developing countries without a higher degree are underrepresented, though desired, in MOOCs. One reason for those learners to drop out of a course can be found in their financial realities and the subsequent limited amount of time they can dedicate to a course besides earning a living. If we could pay learners to take a MOOC, this hurdle would largely disappear. With MOOCS, this leads to the following fundamental challenge: How can learners be paid at scale? Ultimately, we envision a recommendation engine that recommends tasks from online market places such as Upwork or witmart to learners, that are relevant to the course content of the MOOC. In this manner, the learners learn and earn money. To investigate the feasibility of this vision, in this paper we explored to what extent (1) online market places contain tasks relevant to a specific MOOC, and (2) learners are able to solve real-world tasks correctly and with sufficient quality. Finally, based on our experimental design, we were also able to investigate the impact of real-world bonus tasks in a MOOC on the general learner population.

Reference

Guanliang Chen, Dan Davis, Markus Krause, Efthimia Aivaloglou, Claudia Hauff, Geert-Jan Houben, “Can Learners be Earners? Investigating a Design to Enable MOOC Learners to Apply their Skills and Earn Money in an Online Market Place”, IEEE Transactions on Learning Technologies, vol. , no. , pp. 1, 5555, doi:10.1109/TLT.2016.2614302

Download

Push or Pull Students into Blended Education: a Case Study at Delft University of Technology

Paper is published in the International Journal of Engineering Education (ISSN 0949-149X) Volume 32 Number 5(A).

Abstract

Blended education, or ‘‘flipping the classroom’’ is rapidly becoming a mainstream form of teaching within universities. Within Engineering Education, it is popular as it allows more time in-class to focus on hands on activities such as demonstrations and solving complex problems. This paper discusses the effort conducted to re-structure, according to the blended learning principles, the ‘‘Propulsion and Power’’ course of the Aerospace Engineering Bachelor degree programme at Delft University of Technology (TU Delft). The redesigned course was supported by a dedicated online & blended education unit within the university, and is characterized by a very peculiar structure due to the different approach chosen by the two involved lecturers. The first lecturer decided to ‘‘pull’’ the students, by proposing a number of additional videos available in the World Wide Web as a support and complement to the material taught in class. Conversely, the second lecturer opted for a ‘‘push’’ approach, self-recording theory videos to be watched by the students at home and devoting the in-class hours to exercises and applications of the theory. This format resulted in a clear improvement of the average exam grades and pass rates. The student feedback showed enthusiasm about the new blended course, with only a very small minority still preferring the previous, more traditional approach. Although there seems to be a slight preference of students towards the ‘‘push’’ strategy, the ‘‘pull’’ approach has also been widely appreciated.
However, the objective to re-attract students to the contact hours in class was only partially achieved, since just a slight improvement in the number of attending students was observed. This paper clearly shows that the efforts to implement a blended teaching strategy has great benefits for both students and staff alike.

Keywords

blended learning; online education; student engagement; electronic assessment; aerospace engineering

Reference

Cervone, A., Melkert, J.A., Mebus, L.F.M., Saunders-Smits, G.N. (2016). Push or Pull Students into Blended Education. A Case Study at Delft University of Technology. In International Journal of Engineering Education. Volume 32 Number 5A. ISSN 0949-149X (page 1911-1921)

Download

Download paper from researchgate

MOOCs and their Effect on the Institution

Paper was published in journal Foro de Educación.

Abstract

Four years after the introduction of MOOCs – which were proclaimed to be «the end of education as we know it» in 2012 – the role and effect of these free, online courses is becoming clearer. The online means of delivery to the heterogeneous audiences of MOOCs have enabled and compelled instructors and course teams to develop innovative and flexible learning materials. We can analyse the data on the study behaviour of learners to identify which course elements are effective. In addition, the integration of elements of MOOCs in campus education has resulted in promising outcomes and positive reactions from both students and teachers. On the level of the institution, we also see the effect of MOOCs: ranging from new possibilities in communication and branding, to new needs for faculty development and the support organisation. Furthermore, MOOCs play a role in the unbundling of education, e.g. the learning experience and the assessment tasks now can be uncoupled and may be delivered by different institutions and by different means: the learning experience may be in the form of a MOOC and the assessment may be a written exam at an institution.

Reference

Kiers, J. (2016). MOOCs and their Effect on the Institution: Experiences in Course Design, Delivery and Evaluation; Research; Faculty Development; Unbundling and Credits for MOOCs. Foro de Educación, 14(21), 133-149. doi:http://dx.doi.org/10.14516/fde.2016.014.021.007

Download Paper

Augmented Learning Environment for wound care simulation

Paper presented at the EDEN Annual Global Meeting 2016 in Budapest, Hungary from 15-17 June 2016.

Introduction

Emerging technologies for teaching and learning have made it possible to create environments, scenarios and virtual patients that simulate clinical practices in order to promote the development of skills and knowledge in healthcare education (Lewis et al., 2005; Hogan, Sabri, & Kapralos, 2007). These simulations are seen as educational techniques that bring interactivity and immersion into the learning process, allowing the recreation of clinical experiences without the risk of causing harm to patients (Maran & Glavin, 2003). Other known advantage is the possibility learners have to practice an unlimited number of times a procedure or technique until correct realization, before applying it in real-world scenarios (Rey et al., 2006).
Virtual reality (VR) and Augmented reality (AR) are examples of emerging technologies for teaching and learning that allow the creation of digitally enhanced learning environments. These technologies are expected to have an impact in education, as highlighted by the Horizon report for Higher Education in 2010, 2011, and most recently in 2016 (Johnson et al., 2010, 2011, 2016). Regarding healthcare education, several studies indicate the positive effect of AR and VR in developing decision making skills and practical procedures using virtual simulators, with a higher impact on non-experienced participants (Zhu et al., 2014).
However, using VR in healthcare education can be a debatable approach since it immerses learners in a synthetic environment, enabling them to see the surrounding real-world. Acting in a different environment from which learners will have to act in real life scenarios is another concern to take into account.

Reference

Jorge, Nelson; Morgado, Lina; Gaspar, Pedro (2016). Augmented Learning Environment for wound care simulation in Teixeira, Szucs and Mazar (2016). Conference Proceedings EDEN 2016 Annual Conference. Page 256- 264. ISBN 978-615-5511-10-3. License CC-BY 4.0.

Download

The TU Delft Online Learning Experience: From Theory to Practice

Best Practice Initiative AwardPaper presented at the EDEN Annual Global Meeting 2016 in Budapest, Hungary from 15-17 June 2016. At the conference Gala Dinner the paper received the EDEN 2016 Best Practice Initiative Award.

Introduction

In 2014 TU Delft started an innovation programme to educate the world and improve quality of education based on online learning. The programme included open (OpenCourseWare and MOOCs) and online (Professional Education, Bachelor and Master) courses. Lecturers of TU Delft have shared their knowledge in MOOCs with more than 750,000 learners around the world. Next to the open courses, more than 800 learners enrolled in our online courses. Adopting new instructional strategies based on online learning elements has had a very positive impact in TU Delft’s overall education, benefiting traditional on campus education and contributing to the changing need in educating engineers (Kamp, 2014). Naturally, blended learning has arisen on campus, where online learning materials are reused in a flipped classroom approach with very positive results: higher completion rates, higher average grade, more flexibility for students to interact with the course material and more flexibility for teachers in choosing which elements to include in the interactive classroom sessions (van Valkenburg, 2015).

The development of TU Delft online courses is based on the Online Learning Experience (OLE), a pedagogical model that supports the development of our courses and strives for increasing quality. The creation of the OLE was an important step for TU Delft, contributing to the development of online courses in a more systematic and consistent way, guiding all course development teams through the realisation of several shared educational principles.

Last year, when we presented the OLE at the 2015 EDEN Conference in Barcelona, we were still at an early stage of its development, collecting fundamental background to support it and feedback from online learning experts. Although we only had a collection of ideas translated into 8 principles, it was clear that the model needed to be flexible in order to accommodate many educational scenarios that coexist among TU Delft’s Faculties, but with a clear and useful purpose to help improve the quality of our online education (Jorge, Dopper & van Valkenburg, 2015). This paper describes how the OLE is applied in practice.The main goal of the OLE is to improve the quality of our online education by setting course design and development principles to support course teams. At the same time, the OLE can be used as a tool to promote reflection before the course starts to set expectations, and in the end to evaluate and plan improvements for the next run. In the next sections we’ll describe the OLE in both ways – as course design principles (guidelines) and as a tool (the radar graph).

Reference

Jorge, Nelson; Van Valkenburg, Willem; Dopper, Sofia (2016). The TU Delft Online Learning Experience: From Theory to Practice in Teixeira, Szucs and Mazar (2016). Conference Proceedings EDEN 2016 Annual Conference. ISBN 978-615-5511-10-3. License CC-BY 4.0

Download

video of teacher’s perspective of Online Learning Experience:

Beyond the MOOC platform: gaining insights about learners from the social web

Paper presented at the 8th ACM Conference on Web Science in Hanover, Germany.

Abstract

Massive Open Online Courses (MOOCs) have enabled millions of learners across the globe to increase their levels of expertise in a wide variety of subjects. Research efforts surrounding MOOCs are typically focused on improving the learning experience, as the current retention rates (less than 7% of registered learners complete a MOOC) show a large gap between vision and reality in MOOC learning.

Current data-driven approaches to MOOC adaptations rely on data traces learners generate within a MOOC platform such as edX or Coursera. As a MOOC typically lasts between five and eight weeks and with many MOOC learners being rather passive consumers of the learning material, this exclusive use of MOOC platform data traces limits the insights that can be gained from them.

The Social Web potentially offers a rich source of data to supplement the MOOC platform data traces, as many learners are also likely to be active on one or more Social Web platforms. In this work, we present a first exploratory analysis of the Social Web platforms MOOC learners are active on — we consider more than 320,000 learners that registered for 18 MOOCs on the edX platform and explore their user profiles and activities on StackExchange, GitHub, Twitter and LinkedIn.

Reference

Guanliang Chen, Dan Davis, Jun Lin, Claudia Hauff, and Geert-Jan Houben. 2016. Beyond the MOOC platform: gaining insights about learners from the social web. In Proceedings of the 8th ACM Conference on Web Science (WebSci ’16). ACM, New York, NY, USA, 15-24. DOI=http://dx.doi.org/10.1145/2908131.2908145

Download

Download paper

Learning Transfer: Does It Take Place in MOOCs? An Investigation into the Uptake of Functional Programming in Practice

Paper presented at the Learning @ Scale 2016 conference in Edinburgh.

Abstract

The rising number of Massive Open Online Courses (MOOCs) enable people to advance their knowledge and competencies in a wide range of fields. Learning though is only the first step, the transfer of the taught concepts into practice is equally important and often neglected in the investigation of MOOCs. In this paper, we consider the specific case of FP101x (a functional programming MOOC on edX) and the extent to which learners alter their programming behaviour after having taken the course. We are able to link about one third of all FP101x learners to GitHub, the most popular social coding platform to date and contribute a first exploratory analysis of learner behaviour beyond the MOOC platform. A detailed longitudinal analysis of GitHub log traces reveals that (i) more than 8% of engaged learners transfer, and that (ii) most existing transfer learning findings from the classroom setting are indeed applicable in the MOOC setting as well.

Reference

Guanliang Chen, Dan Davis, Claudia Hauff, and Geert-Jan Houben. 2016. Learning Transfer: Does It Take Place in MOOCs? An Investigation into the Uptake of Functional Programming in Practice. In Proceedings of the Third (2016) ACM Conference on Learning @ Scale (L@S ’16). ACM, New York, NY, USA, 409-418. DOI=http://dx.doi.org/10.1145/2876034.2876035

Paper

Download paper

EMOOCS2016: Guidelines for Evaluating the Teaching and Learning in MOOCs: a TU Delft approach

Paper presented at the EMOOCS 2016 conference in Graz, Austria.

Abstract

What does it mean to qualify as a ‘successful’ MOOC? This question haunts policy makers and educators alike, and is at the core of the continued development and funding for Massive Open Online Courses. Because MOOCs can serve many purposes, their value lies in more than just their short-term educational role. A ‘successful’ MOOC can do more than just teach; it can provide institutional brand recognition, address global challenges, improve the quality of campus education, and generate data for educational research. In this paper, we examine the
methods and tools TU Delft uses to evaluate the teaching and learning within its own MOOCs in particular. Recommendations are provided for the use of a set of qualitatitve tools in addition to the more common quantitative tools used to evaluate the ‘success’ of a MOOC.

Reference

Marquis, Danika; Kiers, Janine; Meijerink, Leonie (2016). Guidelines for Evaluating the Teaching and  Learning in MOOCs: a TU Delft approach. In Proceedings of the EUROPEAN STAKEHOLDER SUMMIT on experiences and best practices in and around MOOCs (EMOOCS 2016). http://emoocs2016.eu/wp-content/uploads/2016/02/proceedings-emoocs2016.pdf. ISBN 9783739237107 (page 447-459)

Slides

Paper

Paper is part of the conference proceedings (page 447-459)

EMOOCS2016: Carpe Diem: a new day for flexible MOOC design

Paper presented at the EMOOCs 2016 conference in Graz, Austria. The paper was awarded the outstanding paper & video award.

eMOOCs2016 award certificate

Abstract

Is there one approach to course design that can be recommended in engineering education? At Delft University of Technology (TU Delft), we designed and developed 25 MOOCs, and our experience and expertise in course design is advancing.
One of the frequently used approaches for supporting course teams was inspired by the Carpe Diem approach (Salmon 2014). This paper discusses the experience of TU Delft in implementing this methodology in an engineering setting in MOOCs. The reason for choosing the Carpe Diem approach is that it is simple to use, supports constructive alignment and is a team based approach. In the approach a more activity-based design of MOOCs is promoted through developing e-tivities. In this article the experiences of e-learning developers are described in supporting ten course teams (5 online courses and 5 MOOCs) using the Carpe Diem approach. Two main challenges in supporting course teams are highlighted;
I. How to introduce course teams to the value of the methodology and
II. How to ensure that the specific characteristics of a MOOC are embedded in the design.
For both challenges a range of ‘proposed solutions’ is suggested based on the experiences of the e-learning developers. This results in lessons learned that can be applied by anyone who would like to make use of the Carpe Diem approach for flexible MOOC design. This paper argues that the Carpe Diem approach needs to be used in an interactive and flexible way, taking into account the diversity of the course teams and course leaders as well as the special characteristics of a MOOC.

Reference

Meijerink, Leonie; Kiers, Janine; Marquis, Danika (2016). Carpe Diem: a new day for flexible  MOOC design. In Proceedings of the EUROPEAN STAKEHOLDER SUMMIT on experiences and best practices in and around MOOCs (EMOOCS 2016). http://emoocs2016.eu/wp-content/uploads/2016/02/proceedings-emoocs2016.pdf. ISBN 9783739237107 (page 425-438)

Video

Slides

Paper

Paper is part of the conference proceedings (page 425-438)

EDM2015: Modeling Learners’ Social Centrality and Performance through Language and Discourse

This paper is presented at the Educational Data Mining Conference 2015 in Madrid, Spain.

Abstract

There is an emerging trend in higher education for the adoption of massive open online courses (MOOCs). However, despite this interest in learning at scale, there has been limited work investigating the impact MOOCs can play on student learning. In this study, we adopt a novel approach, using language and discourse as a tool to explore its association with two established measures of learning: traditional academic performance and social centrality. We demonstrate how characteristics of language diagnostically reveal the performance and social position of learners as they interact in a MOOC. We use Coh-Metrix, a theoretically grounded, computational linguistic modeling tool, to explore students’ forum postings across five potent discourse dimensions. Using a Social Network Analysis (SNA) methodology, we determine learners’ social centrality. Linear mixed-effect modeling is used for all other analyses to control for individual learner and text characteristics. The results indicate that learners performed significantly better when they engaged in more expository style discourse, with surface and deep level cohesive integration, abstract language, and simple syntactic structures. However, measures of social centrality revealed a different picture. Learners garnered a more significant and central position in their social network when they engaged with more narrative style discourse with less overlap between words and ideas, simpler syntactic structures and abstract words. Implications for further research and practice are discussed regarding the misalignment between these two learning-related outcomes.

Paper

Download from researchgate

© 2011 TU Delft