What metrics are used to define and measure learner engagement in MOOCs?

Engagement in MOOCs can be measured through various activities beyond just enrollment, such as participating by clicking into course materials, posting to forums, interacting with problems, or watching videos. A participant is considered engaged if they perform at least one of these actions.

How did learner behavior change in MOOCs during the pandemic compared to pre-pandemic times?

During the pandemic, forum engagement significantly increased. However, there was a decrease in video views, and those who did watch videos viewed fewer of them. Verification rates for paid tracks increased, but certification completion rates dipped, suggesting a disconnect between initial commitment and final completion.

Are students from other universities using MIT's MOOCs like the Introduction to Biology course?

Yes, during the pandemic, there was a notable increase in verified and certified learners who were affiliated with universities, indicated by .edu email addresses. This suggests that many students were using the MOOC as a supplement to their own university courses.

What is the completion rate for MOOCs, and why do students not finish?

It's difficult to define 'dropout' precisely in MOOCs, as students often stop interacting rather than formally unenrolling. Motivations for not completing can include a lack of self-regulation, changed priorities, or using the MOOC for purposes other than certification, such as for personal interest or as a teaching tool.

How are MIT's MOOC competency exams designed, and are they as difficult as on-campus MIT exams?

MIT's MOOCs, like the Introduction to Biology, use competency exams designed to test mastery and application of knowledge, similar to the rigor of on-campus MIT courses. These exams feature randomization and no feedback to ensure integrity and assess genuine understanding rather than memorization.

How has the pandemic influenced the future of online learning and teaching at MIT?

The pandemic accelerated the adoption of online learning tools and approaches at MIT. Many instructors have integrated digital tools and embraced new attitudes towards remote collaboration, leading to a diversification of teaching methods for both online and in-person formats.

Key Moments

Open Learning Talks | Research on MOOC Engagement During the Pandemic

MIT OpenCourseWare

Education4 min read58 min video

Nov 16, 2021|330 views|8

Save to Pod

Key Moments

On this page

TL;DR

Research analyzes pandemic MOOC engagement, finding increased enrollment but varied participation and engagement patterns.

Key Insights

The pandemic significantly increased MOOC enrollment, with one course seeing nearly triple the usual sign-ups.

While enrollment surged, the proportion of enrolled students actively participating (clicking into course materials) increased initially but then decreased.

Forums saw a significant boost in activity during the pandemic, indicating increased student interaction.

Video engagement showed a decline during the pandemic, with fewer students watching videos and those who did watching fewer videos overall.

Despite initial enthusiasm for the verified track, completion rates (certification) saw a dip during the pandemic.

A notable increase in verified and certified learners affiliated with universities suggests MOOCs were used as supplementary learning tools.

THE DIGITAL LEARNING LAB AND THE BIOLOGY MOOC TEAM

The MIT Open Learning initiative supports groups like the Digital Learning Lab, which comprises individuals with academic appointments and discipline expertise. These teams collaborate with faculty on digital learning projects, including the creation of Massive Open Online Courses (MOOCs). The MITx Biology group, a part of this initiative, focuses on developing MOOCs for the biology department, with a notable early contribution being the '7.00x Introduction to Biology' course launched in 2013, which has since evolved to include a competency-based exam model.

RESEARCH ON PANDEMIC-INDUCED CHANGES IN MOOC ENGAGEMENT

Recognizing a surge in MOOC activity during the COVID-19 pandemic, researchers investigated engagement levels in MITx MOOCs, particularly '7.00x Introduction to Biology'. The study compared pre-pandemic course runs with those during the initial six months of the pandemic. The primary challenge was defining 'engagement' beyond mere enrollment, leading to the analysis of various metrics to understand how learner behavior evolved under pandemic conditions.

ENROLLMENT AND PARTICIPATION TRENDS

During the pandemic, '7.00x Introduction to Biology' experienced a substantial increase in enrollment, nearly tripling compared to pre-pandemic runs. However, when analyzed as a proportion of enrolled students, initial participation (defined as actively clicking into course materials) showed an increase in March 2020 but declined in July 2020. This suggested that while more people were signing up, the proportion engaging with the content beyond the initial enrollment step was not sustained.

MEASURING LEARNER ENGAGEMENT WITH COURSE MATERIALS

Engagement was further broken down into specific actions: posting to forums, interacting with problems, or viewing videos. Forum engagement saw a significant increase during the pandemic. Conversely, video viewing decreased, with fewer students watching videos and those who did watching fewer of them. This shift in video consumption patterns raised questions about how learners were accessing and interacting with core course content.

VERIFIED TRACK ENROLLMENT AND CERTIFICATION RATES

The study observed a notable increase in students opting for the verified track during the pandemic, indicating a higher dedication to completing the course. However, this enthusiasm did not fully translate into certification. While a larger proportion of participants chose the verified track, the certification rate, especially when considered out of those who attempted the competency exam, showed a dip in July 2020. This suggested that many learners were engaging with the course more deeply but struggled to complete it.

THE ROLE OF AFFILIATION AND SUPPLEMENTARY LEARNING

Analysis revealed a significant increase in verified and certified learners who were affiliated with universities (using .edu email addresses). This finding suggests that many students and potentially instructors were using the MOOC as a supplementary resource for their own university courses, possibly due to the shift to remote learning and the need for additional educational materials during the pandemic.

PERSONAL PERSPECTIVES AND LESSONS FROM THE PANDEMIC

The researchers shared personal experiences, highlighting how the pandemic shifted educational norms and reinforced the importance of digital learning. The transition to remote learning for students and educators mirrored some of the early challenges and debates surrounding MOOCs. The experience underscored the value of digital tools and flexible learning approaches, some of which are likely to persist post-pandemic, influencing future teaching and learning strategies.

UNDERSTANDING DROPOUT AND REPEAT LEARNER BEHAVIORS

Defining 'dropout' in MOOCs is complex due to their self-paced nature and the competency exam model. Many students stop interacting but do not formally unenroll, potentially re-engaging later for exams. The research also noted a surprising number of repeat learners, whose motivations (e.g., for professional development, teaching support, or struggling with content) are varied and difficult to ascertain without direct surveys or interviews. Self-regulation emerged as a key factor for completion.

COURSE DESIGN AND CONTENT EFFECTIVENESS

The design of MOOCs, including video length and assessment strategies, significantly impacts engagement. Research suggests that content relevance and engagement are more critical than video duration alone. The use of shorter video segments followed by formative assessments, like 'test yourself' questions, is a pedagogical practice that encourages active learning and checks understanding, aligning with the goals of applying knowledge rather than just memorizing facts.

COMPARISON WITH ON-CAMPUS COURSES AND FUTURE IMPLICATIONS

MOOCs, designed for asynchronous, independent learning with computer-graded assessments, differ from traditional on-campus courses. While MIT traditionally focused on residential experiences, the pandemic blurred lines, leading to more blended and emergency remote teaching. Insights gained from MOOCs and pandemic-induced online learning are expected to influence future pedagogical approaches, with an increased acceptance and integration of digital tools and flexible learning attitudes within the academic community.

Mentioned in This Episode

●Software & Apps

●Organizations

●Concepts

●People Referenced

Best Practices for MOOC Engagement and Design

Practical takeaways from this episode

Do This

Break videos into smaller chunks with checks for understanding every 5-10 minutes.

Incorporate formative assessments like 'test yourself' questions after instruction.

Design assessments (like the competency exam) to focus on applying knowledge rather than memorization.

Consider offering financial aid or discounts for verified tracks to improve accessibility.

Leverage online tools and attitudes that proved beneficial during the pandemic for future teaching and learning.

Avoid This

Rely solely on enrollment numbers as a measure of engagement.

Use lengthy, uninterrupted video lectures without intermediate checks.

Design MOOCs as watered-down versions of on-campus courses; maintain rigor.

Assume that simply paying for a verified track guarantees course completion.

Ignore the potential benefits of asynchronous learning and digital tools.

Common Questions

The pandemic led to a significant surge in MOOC enrollment, with the Introduction to Biology course seeing nearly triple the usual number of learners in early 2020. While enrollment remained higher than pre-pandemic levels in mid-2020, it showed a slight decrease from the initial pandemic high.

Topics

Technology & Innovation Science & Mathematics Higher Education Online Learning Course Design Learner Engagement Pandemic Impact Digital Education

Mentioned in this video

Software & Apps

700x Introduction to Biology

The first MOOC created by the MITx Biology group, launched in March 2013, focusing on 'The Secret of Life'. It evolved to include a competency exam model.

Python

A programming language introduced to Katie Blackwell through the Quantitative Biology Workshop MOOC, which she later used for data analysis.

MATLAB

A programming language used in scientific computing, which Katie Blackwell was introduced to through the Quantitative Biology Workshop MOOC.

Canvas

The learning management system used at MIT that faculty transitioned to during the pandemic, with assistance from the digital learning lab.

edx

The platform on which the MITx Biology MOOCs are hosted.

705x Biochemistry

A MOOC developed by the MITx Biology team, studied by Martha Castro for engagement trends.

706 Cell Biology

A MOOC developed by the MITx Biology team, with Katie Blackwell contributing by writing test questions.

Quantitative Biology Workshop

A MOOC offered by MITx Biology that provides instruction in programming languages like Python and Matlab, useful for biologists and data analysis.

People

Eric Lander

Led the creation of the 700x Introduction to Biology MOOC.

Tanya Baker

Instrumental in creating the digital learning lab and supporting digital learning early on at MIT Biology.

Katie Blackwell

Presented research on MOOC engagement during the pandemic, was an undergraduate researcher at UT Dallas, and is now a graduate student at MIT.

Organizations

University of Texas at Dallas

Katie Blackwell's undergraduate alma mater.

Digital Learning Lab

A group within MIT Biology that works on MOOCs and digital learning projects, composed of individuals with academic appointments and discipline expertise.

MITx Biology Group

The group within the MIT Biology department that develops MOOCs and digital learning initiatives.

Bernie and Sophie Gould MIT Summer Research Program

A summer research program that Katie Blackwell participated in, which was conducted virtually due to the pandemic.

MIT Open Learning

An organization at MIT that collaborated with the biology department and supported the digital learning lab.

Concepts

Systems Biology

A joint department course at MIT that Katie Blackwell is taking, enabled by her prior completion of the Quantitative Biology Workshop.