Posts in category Papers
EMOOCS2016: Guidelines for Evaluating the Teaching and Learning in MOOCs: a TU Delft approach
Paper presented at the EMOOCS 2016 conference in Graz, Austria.
Abstract
What does it mean to qualify as a ‘successful’ MOOC? This question haunts policy makers and educators alike, and is at the core of the continued development and funding for Massive Open Online Courses. Because MOOCs can serve many purposes, their value lies in more than just their short-term educational role. A ‘successful’ MOOC can do more than just teach; it can provide institutional brand recognition, address global challenges, improve the quality of campus education, and generate data for educational research. In this paper, we examine the
methods and tools TU Delft uses to evaluate the teaching and learning within its own MOOCs in particular. Recommendations are provided for the use of a set of qualitatitve tools in addition to the more common quantitative tools used to evaluate the ‘success’ of a MOOC.
Reference
Marquis, Danika; Kiers, Janine; Meijerink, Leonie (2016). Guidelines for Evaluating the Teaching and Learning in MOOCs: a TU Delft approach. In Proceedings of the EUROPEAN STAKEHOLDER SUMMIT on experiences and best practices in and around MOOCs (EMOOCS 2016). http://emoocs2016.eu/wp-content/uploads/2016/02/proceedings-emoocs2016.pdf. ISBN 9783739237107 (page 447-459)
Slides
Paper
Paper is part of the conference proceedings (page 447-459)
EMOOCS2016: Carpe Diem: a new day for flexible MOOC design
Paper presented at the EMOOCs 2016 conference in Graz, Austria. The paper was awarded the outstanding paper & video award.
Abstract
Is there one approach to course design that can be recommended in engineering education? At Delft University of Technology (TU Delft), we designed and developed 25 MOOCs, and our experience and expertise in course design is advancing.
One of the frequently used approaches for supporting course teams was inspired by the Carpe Diem approach (Salmon 2014). This paper discusses the experience of TU Delft in implementing this methodology in an engineering setting in MOOCs. The reason for choosing the Carpe Diem approach is that it is simple to use, supports constructive alignment and is a team based approach. In the approach a more activity-based design of MOOCs is promoted through developing e-tivities. In this article the experiences of e-learning developers are described in supporting ten course teams (5 online courses and 5 MOOCs) using the Carpe Diem approach. Two main challenges in supporting course teams are highlighted;
I. How to introduce course teams to the value of the methodology and
II. How to ensure that the specific characteristics of a MOOC are embedded in the design.
For both challenges a range of ‘proposed solutions’ is suggested based on the experiences of the e-learning developers. This results in lessons learned that can be applied by anyone who would like to make use of the Carpe Diem approach for flexible MOOC design. This paper argues that the Carpe Diem approach needs to be used in an interactive and flexible way, taking into account the diversity of the course teams and course leaders as well as the special characteristics of a MOOC.
Reference
Meijerink, Leonie; Kiers, Janine; Marquis, Danika (2016). Carpe Diem: a new day for flexible MOOC design. In Proceedings of the EUROPEAN STAKEHOLDER SUMMIT on experiences and best practices in and around MOOCs (EMOOCS 2016). http://emoocs2016.eu/wp-content/uploads/2016/02/proceedings-emoocs2016.pdf. ISBN 9783739237107 (page 425-438)
Video
Slides
Paper
Paper is part of the conference proceedings (page 425-438)
EDM2015: Modeling Learners’ Social Centrality and Performance through Language and Discourse
This paper is presented at the Educational Data Mining Conference 2015 in Madrid, Spain.
Abstract
There is an emerging trend in higher education for the adoption of massive open online courses (MOOCs). However, despite this interest in learning at scale, there has been limited work investigating the impact MOOCs can play on student learning. In this study, we adopt a novel approach, using language and discourse as a tool to explore its association with two established measures of learning: traditional academic performance and social centrality. We demonstrate how characteristics of language diagnostically reveal the performance and social position of learners as they interact in a MOOC. We use Coh-Metrix, a theoretically grounded, computational linguistic modeling tool, to explore students’ forum postings across five potent discourse dimensions. Using a Social Network Analysis (SNA) methodology, we determine learners’ social centrality. Linear mixed-effect modeling is used for all other analyses to control for individual learner and text characteristics. The results indicate that learners performed significantly better when they engaged in more expository style discourse, with surface and deep level cohesive integration, abstract language, and simple syntactic structures. However, measures of social centrality revealed a different picture. Learners garnered a more significant and central position in their social network when they engaged with more narrative style discourse with less overlap between words and ideas, simpler syntactic structures and abstract words. Implications for further research and practice are discussed regarding the misalignment between these two learning-related outcomes.
Paper
SEFI2015: Gender and Diversity in Engineering MOOCs, a first Appraisal
This paper was presented at the 43rd Annual SEFI Conference June 29 – July 2, 2015 in Orléans, France
Abstract
This paper addresses the participation and performance of MOOC students in relation to gender and diversity. It is a first appraisal based on the data collected from the five engineering MOOCs executed in 2013-2014 at the Delft University of Technology (TUD) on the edX platform, which is part of the edX consortium (www.edx.org). section 2 gives an overview about previous research outcomes. In section 3 the data collection from TUD is summarized under the special focus on gender and diversity. In section 4 the authors of this paper present the outcomes of the data analysis with a focus on gender and diversity of student population in relation to participation and performance. Section 5 summarizes the outcomes and gives an outlook to further educational and research questions in this field.
Paper
SEFI2015: The Value of Engineering MOOCs from a Learner’s Perspective
This paper was presented at the 43rd Annual SEFI Conference June 29 – July 2, 2015 in Orléans, France
Abstract
This paper looks at the perceived value students adhere to the DelftX MOOC engineering courses they have taken, in other words what is the course worth in the context of their learning needs? Are you doing the course because you are curious, you want to get more knowledgeable on the topic, you need to know something related to your work, you do it for other professional reasons. This research will not cover all arguments as it is a first endeavor to get to know the learner better from the value perspective.
Paper
IJCLEE2015: Who is the Learner in the DelftX Engineering MOOCs?
This paper was presented at International Joint Conference on the Learner in Engineering Education (IJCLEE 2015) in San Sebastian.
Abstract
The Delft University of Technology (TUD) deployed her first generation of massive open online courses (MOOCs) in 2013-2014 delivered through the edX platform. These DelftX MOOCs were engineering courses designed at the level equivalent to that of a bachelor-program entry level. Almost 140 thousand students registered, around 3,7% received certificates of completion, and the rest participated to a degree reflective of their needs. To better understand and ultimately enhance the MOOCs, TUD conducted the collection and analysis of data about learners and their contexts. This exploratory paper focuses on the specific analyses pertinent to describing the demographics of an Engineering MOOC participant, as observed in the first generation of TUD MOOCs. The implications of the observed participant demographics are analysed and discussed.
Paper
Defining a Pedagogical Model: The TU Delft Online Learning Experience
This paper was presented at the EDEN 2015 conference in Barcelona from 9-12 June 2015 by Nelson Jorge, Sofia Dopper and Willem van Valkenburg.
Abstract
In early 2014, the Delft University of Technology (TU Delft) started an innovation program with the aim to respond even more effectively to recent developments in open and online education. Drawing on the fields of Distance Education research and the university’s vision of the “engineer of the future”, TU Delft’s Extension School created a unified pedagogical model – the Online Learning Experience (OLE) – contributing to greater consistency in the development of online courses. Continue reading
EMOOCs 2015: Experiences from 18 DelftX MOOCs
Paper presented at the European Stakeholders summit on experiences and best practices in and around MOOCs in Mons, Belgium (18-20 May 2015).
Abstract
This paper describes experiences and lessons learned from developing, building and running 18 MOOCs (including 4 reruns) at the Delft University of Technology, namely the selection of courses, the design, development and delivery of courses, innovative educational elements for use in online and in residential teaching, and the generation of data. Overall the development and offering of MOOCs has augmented the awareness, expertise, and innovation regarding online and residential learning, and has contributed to an active and positive spirit of educational innovation. In addition, we saw indications of a positive effect on quality and quantity of enrolment for on-campus programs, gathered data for research and were able to share the university’s knowledge with the world.
Reference
Kiers, J., Jorge, N., (2015). Experiences from 18 DelftX MOOCs in Proceedings Paper EMOOCs 2015 (page 65-70): http://www.emoocs2015.eu/sites/default/files/Papers.pdf
EMOOCs2015: Reconsidering Retention in MOOCs: the Relevance of Formal Assessment and Pedagogy
Paper presented at the European Stakeholders summit on experiences and best practices in and around MOOCs in Mons, Belgium (18-20 May 2015).
Abstract
The motivation to enrol in a MOOC is more diverse than the motivation for a conventional course. This diversity requires re-conceptualization of the terms for enrolment, participation, and achievement. The paper addresses the concept of retention and focuses on engagement relative to assessment. Student retention is often used to determine the value of higher education. In this paper we argue that retention data about specific groups of students
can supply valuable insights to improve MOOC design and align expectations. The paper reports three short studies conducted to gain insights into disengagement from assessment, based on the data gathered in the first five DelftX MOOCs. The empirical part of the paper demonstrates that retention rates in relation to formal assessment vary from course to course. In the analysed case, fewer learners disengaged from the formal assessment in the course with highest degree of student autonomy, high learning support and scaffolds. Consistently across courses, learners who received lower grades on the first assessment task, tend to disengage from further assessment.
Reference
De Vries, P., Hennis, T., Skrypnyk, A., (2015). Reconsidering Retention in MOOCs: the Relevance of Formal Assessment and Pedagogy in Proceedings Paper EMOOCs 2015 (page 168-173): http://www.emoocs2015.eu/sites/default/files/Papers.pdf
LAK15: Scalability and Flexibility through Open Research
Abstract
Delft University of Technology (TUD) is the largest university of engineering sciences in the Netherlands. TUD has been an active member of the Open Resources Movement since 2007 and in 2012 it has joined the EdX consortium. Openness in research and education is central in the TUD mission. It was the first edX partner to issue all its MOOCs under a creative commons license. In this presentation, we present our open research approach and toolbox. This toolbox facilitates sharing of data and the process of collaborative research in the context of MOOCs/online education. The presentation explains our rationale for open research, and our experiences with the toolbox, including research highlights.