際際滷

際際滷Share a Scribd company logo
UNDERSTANDING LEARNERS
DROP-OUT IN MOOCS
Alya Itani, Laurent Brisson, Serge Garlatti

Massive Open Online Course (Premium members)
Send feedback to learners
to prevent drop-out
Provide personalized
feedback to teachers
Contributions
A predictive system that can detect at-risk droppers at different instants of the
learners interaction with the course
New features associated with learners trajectory of engagement with the
course.
Suitable intervention strategies for both teachers and learners.
UNDERSTANDING LEARNERS DROP-OUT IN MOOCS
Alya Itani, Laurent Brisson, Serge Garlatti
The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018
UNDERSTANDING LEARNERS DROP-OUT IN MOOCS
Context
2
 The structure of a course may not be helpful
to all participants and supporting different patterns of
engagement may be beneficial [Onah et. al, 2014]
Deficiency in some skills
(digital skills or prerequisite)
Bad MOOC design
(inefficient material, high workload)
Lack of intention to complete
(material hunter, curious explorers)
Inaccurate expectations
(misunderstanding about the objectives or
content)
UNDERSTANDING LEARNERS DROP-OUT IN MOOCS
Alya Itani, Laurent Brisson, Serge Garlatti
The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018
UNDERSTANDING LEARNERS DROP-OUT IN MOOCS
Problems
3
Create your Website with HTML (HTML5) 2015-2016
11,520 active learners
Completers: 38%
(4,333)
Droppers: 62%
(7,187)
Understanding the Web (Web) 2015-2016
7,160 active learners
Completers: 71%
(5,085)
Droppers: 29%
(2,075)
Features
Completed activities
Time spent on each activity
Grades obtained
UNDERSTANDING LEARNERS DROP-OUT IN MOOCS
Alya Itani, Laurent Brisson, Serge Garlatti
The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018
CASE STUDY
Openclassroom Dataset
4
Creation of behavioral features
3 new attributes are created for each pair
of activities
Back jumps
(a step back from the scenario planned by
the teacher)
Forward jumps
(intended transition between 2 activities or
skipped activities)
Inactivity time
(for each jump, inactivity time between
activities)
UNDERSTANDING LEARNERS DROP-OUT IN MOOCS
Alya Itani, Laurent Brisson, Serge Garlatti
The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018
CASE STUDY
Features creation
5
CRISP-DM Process Data Preparation
New behavior features
Modeling
Stratified training and test set
Algorithms : random forest, gradient
boosting, decision tree, logistic
regression
For each algorithm:
Hyperparameter tuning
K-fold cross-validation
Performance metrics : F-measure,
AUC
Evaluation
Provide personalized feedback to
learners and teachers
UNDERSTANDING LEARNERS DROP-OUT IN MOOCS
Alya Itani, Laurent Brisson, Serge Garlatti
The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018
CASE STUDY
Predictive process
6
Experiments goals
Evaluate algorithms performance
considering the 3 following contexts
Impact of the prediction at 25% or 50% of
the course progression
  Impact of Behavorial features
Impact of the course
UNDERSTANDING LEARNERS DROP-OUT IN MOOCS
Alya Itani, Laurent Brisson, Serge Garlatti
The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018
CASE STUDY
Experiments
7
Percentage of completed activities 25% 50%
Behavorial features with without with without
Create your site with HTML5 RF
0.76 賊 .02 0.85 賊 .01
GB
DT
LR
Understanding the web RF
0.25 賊 .01
0.91 賊 .01
GB
DT
LR 0.46
UNDERSTANDING LEARNERS DROP-OUT IN MOOCS
Alya Itani, Laurent Brisson, Serge Garlatti
The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018
RESULTS
Overview (F-measure metric)
8
Logistic Regression (6 most influential features in absolute value)
Decision Tree (pruned)
UNDERSTANDING LEARNERS DROP-OUT IN MOOCS
Alya Itani, Laurent Brisson, Serge Garlatti
The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018
RESULTS
Models readability
9
The most discriminating feature is a descriptive indicator
(completed activity, grades, time spent)
  
Diagnostic:
Is the activity motivating? [Viau, 2000]
Does the activity respect constructive alignment?
The activity is a lesson or an exercise:
Identify difficulties (unannounced prerequisites, ...)
Check the target audience of the course
The activity is an assessment :
Identify difficulties (complex statements, ...)
Check the activity (workload, kind of assessment, ...)
UNDERSTANDING LEARNERS DROP-OUT IN MOOCS
Alya Itani, Laurent Brisson, Serge Garlatti
The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018
RESULTS
Situation 1 Doing an activity increases the probability of drop-out
10
The most discriminating feature is a behavioral indicator
(back jump, forward jumps, inactivity time)
  
The feature is a forward-jump that respects the recommended progression
Check activity to identify a wrong constructive alignment
Suggest new routes to students and change course progression
The feature is a back-jump or a forward jump skipping the recommended
progression
Remind students of the importance of following recommended progression
Add self-assessment activities to help to identify relevant jumps (back and
forward)
UNDERSTANDING LEARNERS DROP-OUT IN MOOCS
Alya Itani, Laurent Brisson, Serge Garlatti
The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018
RESULTS
Situation 2 Doing an jump increases the probability of drop-out
11
At a given course progression the prediction is bad and so no feature
is really relevant
None of the activity over the period allow to anticipate successes and failures :
Activities could be designed to build student self-confidence
... but we could check a wrong constructive alignment!
Prediction performance increases at a latter instant
The most predictive activities are often those that require some commitment
(exercices, assemnents):
No such activities during the first 25% of course progression
The course lack activities to help student to assess their progress
UNDERSTANDING LEARNERS DROP-OUT IN MOOCS
Alya Itani, Laurent Brisson, Serge Garlatti
The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018
RESULTS
Situation 3 Prediction performance increases a lot between two different instants
12
Contributions
A predictive system that can detect at-risk droppers at different instants of the
learners interaction with the course
New features associated with learners trajectory of engagement with the
course
Suitable intervention strategies for both teachers and learners
Perspectives
Use business knowledge: learner model, course structure, learning outcomes
(e-education)
Focus on social interactions and group dynamics in the learning mechanism
(social network analysis)
UNDERSTANDING LEARNERS DROP-OUT IN MOOCS
Alya Itani, Laurent Brisson, Serge Garlatti
The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018
CONCLUSION & PERSPECTIVES 13
[Onah et al., 2014] Onah, D. F. O., Sinclair, J., & Boyatt, R. (2014). Dropout rates of
massive open online courses behavioural patterns. 6th International Conference on
Education and New Learning Technologies, 58255834.
https://doi.org/10.13140/RG.2.1.2402.0009
[Viau, 2000] Viau, R. (2000). Conditions to be respected to encourage student
motivation. La revue web sur la valorisation du fran巽ais en milieu coll辿gial. Retrieved
from https://goo.gl/Cva6KH
UNDERSTANDING LEARNERS DROP-OUT IN MOOCS
Alya Itani, Laurent Brisson, Serge Garlatti
The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018
REFERENCES 14
Ad

Recommended

Across System Learning Environment and Dashboard Design for K12 Teachers and ...
Across System Learning Environment and Dashboard Design for K12 Teachers and ...
Jessie Chuang
Open Learning Analytics panel presentation - LAK 15
Open Learning Analytics panel presentation - LAK 15
Sandeep M. Jayaprakash
The UTS Connected Intelligence Centre
The UTS Connected Intelligence Centre
Simon Buckingham Shum
Deploying Open Learning Analytics at a National Scale
Deploying Open Learning Analytics at a National Scale
michaeldwebb
SNS messages Recommendation for Learning Motivation (AIED 2015)
SNS messages Recommendation for Learning Motivation (AIED 2015)
Sebastien Louvigne
Who are you and makes you special?
Who are you and makes you special?
Simon Buckingham Shum
Personalized Online Practice Systems for Learning Programming
Personalized Online Practice Systems for Learning Programming
Peter Brusilovsky
Learning Design
Learning Design
Scot Aldred
LAK19 - Towards Value-Sensitive Learning Analytics Design
LAK19 - Towards Value-Sensitive Learning Analytics Design
Bodong Chen
Intro to jisc Learning Analytics March 16
Intro to jisc Learning Analytics March 16
Paul Bailey
Tools and Evaluation Techniques to Support Social Awareness in CSCeL: The AVA...
Tools and Evaluation Techniques to Support Social Awareness in CSCeL: The AVA...
EuroCAT CSCL
Tools and Evaluation Techniques to Support Social Awareness in CSCeL: The AV...
Tools and Evaluation Techniques to Support Social Awareness in CSCeL: The AV...
Niki Lambropoulos PhD
Public PhD defense
Public PhD defense
Jose Luis Santos Odriozola
Learning Analytics for MOOCs: EMMA case
Learning Analytics for MOOCs: EMMA case
Universit degli Studi di Modena e Reggio Emilia/Tallinn University
Learning Analytics
Learning Analytics
Viplav Baxi
insight-centre-galway-learning-analytics
insight-centre-galway-learning-analytics
Simon Buckingham Shum
VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...
VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...
eMadrid network
Data-Driven Learning Strategy
Data-Driven Learning Strategy
Jessie Chuang
FACT2 Learning Analytics Task Group (LATG) SCOA briefing
FACT2 Learning Analytics Task Group (LATG) SCOA briefing
gketcham
Learning Design meets Learning Analytics and Artificial Intelligence: Potent...
Learning Design meets Learning Analytics and Artificial Intelligence: Potent...
Yannis
Learning Analytics in Education: Using Students Big Data to Improve Teaching
Learning Analytics in Education: Using Students Big Data to Improve Teaching
Rafael Scapin, Ph.D.
Phase 1 Learning Analytics Intro 際際滷s
Phase 1 Learning Analytics Intro 際際滷s
Paul Bailey
Blended e-learning, evaluation and emerging scenarios
Blended e-learning, evaluation and emerging scenarios
Grundtvig Multilateral Project Quality in Blended Learning
Exploring learning analytics
Exploring learning analytics
Jisc
Social Learning Analytics
Social Learning Analytics
SocialLearn, Open U
IRJET- Evaluation Technique of Student Performance in various Courses
IRJET- Evaluation Technique of Student Performance in various Courses
IRJET Journal
IRJET- Predicting Academic Performance based on Social Activities
IRJET- Predicting Academic Performance based on Social Activities
IRJET Journal
Jisc learning analytics service oct 2016
Jisc learning analytics service oct 2016
Paul Bailey
Q1_ENGLISH_PPT_WEEK 1 power point grade 3 Quarter 1 week 1
Q1_ENGLISH_PPT_WEEK 1 power point grade 3 Quarter 1 week 1
jutaydeonne
GREAT QUIZ EXCHANGE 2025 - GENERAL QUIZ.pptx
GREAT QUIZ EXCHANGE 2025 - GENERAL QUIZ.pptx
Ronisha Das

More Related Content

Similar to Understanding Learner's Drop-out in MOOCs (20)

LAK19 - Towards Value-Sensitive Learning Analytics Design
LAK19 - Towards Value-Sensitive Learning Analytics Design
Bodong Chen
Intro to jisc Learning Analytics March 16
Intro to jisc Learning Analytics March 16
Paul Bailey
Tools and Evaluation Techniques to Support Social Awareness in CSCeL: The AVA...
Tools and Evaluation Techniques to Support Social Awareness in CSCeL: The AVA...
EuroCAT CSCL
Tools and Evaluation Techniques to Support Social Awareness in CSCeL: The AV...
Tools and Evaluation Techniques to Support Social Awareness in CSCeL: The AV...
Niki Lambropoulos PhD
Public PhD defense
Public PhD defense
Jose Luis Santos Odriozola
Learning Analytics for MOOCs: EMMA case
Learning Analytics for MOOCs: EMMA case
Universit degli Studi di Modena e Reggio Emilia/Tallinn University
Learning Analytics
Learning Analytics
Viplav Baxi
insight-centre-galway-learning-analytics
insight-centre-galway-learning-analytics
Simon Buckingham Shum
VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...
VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...
eMadrid network
Data-Driven Learning Strategy
Data-Driven Learning Strategy
Jessie Chuang
FACT2 Learning Analytics Task Group (LATG) SCOA briefing
FACT2 Learning Analytics Task Group (LATG) SCOA briefing
gketcham
Learning Design meets Learning Analytics and Artificial Intelligence: Potent...
Learning Design meets Learning Analytics and Artificial Intelligence: Potent...
Yannis
Learning Analytics in Education: Using Students Big Data to Improve Teaching
Learning Analytics in Education: Using Students Big Data to Improve Teaching
Rafael Scapin, Ph.D.
Phase 1 Learning Analytics Intro 際際滷s
Phase 1 Learning Analytics Intro 際際滷s
Paul Bailey
Blended e-learning, evaluation and emerging scenarios
Blended e-learning, evaluation and emerging scenarios
Grundtvig Multilateral Project Quality in Blended Learning
Exploring learning analytics
Exploring learning analytics
Jisc
Social Learning Analytics
Social Learning Analytics
SocialLearn, Open U
IRJET- Evaluation Technique of Student Performance in various Courses
IRJET- Evaluation Technique of Student Performance in various Courses
IRJET Journal
IRJET- Predicting Academic Performance based on Social Activities
IRJET- Predicting Academic Performance based on Social Activities
IRJET Journal
Jisc learning analytics service oct 2016
Jisc learning analytics service oct 2016
Paul Bailey
LAK19 - Towards Value-Sensitive Learning Analytics Design
LAK19 - Towards Value-Sensitive Learning Analytics Design
Bodong Chen
Intro to jisc Learning Analytics March 16
Intro to jisc Learning Analytics March 16
Paul Bailey
Tools and Evaluation Techniques to Support Social Awareness in CSCeL: The AVA...
Tools and Evaluation Techniques to Support Social Awareness in CSCeL: The AVA...
EuroCAT CSCL
Tools and Evaluation Techniques to Support Social Awareness in CSCeL: The AV...
Tools and Evaluation Techniques to Support Social Awareness in CSCeL: The AV...
Niki Lambropoulos PhD
Learning Analytics
Learning Analytics
Viplav Baxi
insight-centre-galway-learning-analytics
insight-centre-galway-learning-analytics
Simon Buckingham Shum
VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...
VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...
eMadrid network
Data-Driven Learning Strategy
Data-Driven Learning Strategy
Jessie Chuang
FACT2 Learning Analytics Task Group (LATG) SCOA briefing
FACT2 Learning Analytics Task Group (LATG) SCOA briefing
gketcham
Learning Design meets Learning Analytics and Artificial Intelligence: Potent...
Learning Design meets Learning Analytics and Artificial Intelligence: Potent...
Yannis
Learning Analytics in Education: Using Students Big Data to Improve Teaching
Learning Analytics in Education: Using Students Big Data to Improve Teaching
Rafael Scapin, Ph.D.
Phase 1 Learning Analytics Intro 際際滷s
Phase 1 Learning Analytics Intro 際際滷s
Paul Bailey
Exploring learning analytics
Exploring learning analytics
Jisc
IRJET- Evaluation Technique of Student Performance in various Courses
IRJET- Evaluation Technique of Student Performance in various Courses
IRJET Journal
IRJET- Predicting Academic Performance based on Social Activities
IRJET- Predicting Academic Performance based on Social Activities
IRJET Journal
Jisc learning analytics service oct 2016
Jisc learning analytics service oct 2016
Paul Bailey

Recently uploaded (20)

Q1_ENGLISH_PPT_WEEK 1 power point grade 3 Quarter 1 week 1
Q1_ENGLISH_PPT_WEEK 1 power point grade 3 Quarter 1 week 1
jutaydeonne
GREAT QUIZ EXCHANGE 2025 - GENERAL QUIZ.pptx
GREAT QUIZ EXCHANGE 2025 - GENERAL QUIZ.pptx
Ronisha Das
English 3 Quarter 1_LEwithLAS_Week 1.pdf
English 3 Quarter 1_LEwithLAS_Week 1.pdf
DeAsisAlyanajaneH
Communicable Diseases and National Health Programs Unit 9 | B.Sc Nursing 5t...
Communicable Diseases and National Health Programs Unit 9 | B.Sc Nursing 5t...
RAKESH SAJJAN
YSPH VMOC Special Report - Measles Outbreak Southwest US 6-14-2025.pptx
YSPH VMOC Special Report - Measles Outbreak Southwest US 6-14-2025.pptx
Yale School of Public Health - The Virtual Medical Operations Center (VMOC)
Code Profiling in Odoo 18 - Odoo 18 際際滷s
Code Profiling in Odoo 18 - Odoo 18 際際滷s
Celine George
2025 June Year 9 Presentation: Subject selection.pptx
2025 June Year 9 Presentation: Subject selection.pptx
mansk2
How payment terms are configured in Odoo 18
How payment terms are configured in Odoo 18
Celine George
ECONOMICS, DISASTER MANAGEMENT, ROAD SAFETY - STUDY MATERIAL [10TH]
ECONOMICS, DISASTER MANAGEMENT, ROAD SAFETY - STUDY MATERIAL [10TH]
SHERAZ AHMAD LONE
SCHIZOPHRENIA OTHER PSYCHOTIC DISORDER LIKE Persistent delusion/Capgras syndr...
SCHIZOPHRENIA OTHER PSYCHOTIC DISORDER LIKE Persistent delusion/Capgras syndr...
parmarjuli1412
ENGLISH-5 Q1 Lesson 1.pptx - Story Elements
ENGLISH-5 Q1 Lesson 1.pptx - Story Elements
Mayvel Nadal
Nutrition Assessment and Nutrition Education Unit 4 | B.Sc Nursing 5th Seme...
Nutrition Assessment and Nutrition Education Unit 4 | B.Sc Nursing 5th Seme...
RAKESH SAJJAN
SPENT QUIZ NQL JR FEST 5.0 BY SOURAV.pptx
SPENT QUIZ NQL JR FEST 5.0 BY SOURAV.pptx
Sourav Kr Podder
GEOGRAPHY-Study Material [ Class 10th] .pdf
GEOGRAPHY-Study Material [ Class 10th] .pdf
SHERAZ AHMAD LONE
Publishing Your Memoir with Brooke Warner
Publishing Your Memoir with Brooke Warner
Brooke Warner
THE PSYCHOANALYTIC OF THE BLACK CAT BY EDGAR ALLAN POE (1).pdf
THE PSYCHOANALYTIC OF THE BLACK CAT BY EDGAR ALLAN POE (1).pdf
nabilahk908
Assisting Individuals and Families to Promote and Maintain Health Unit 7 | ...
Assisting Individuals and Families to Promote and Maintain Health Unit 7 | ...
RAKESH SAJJAN
Health Care Planning and Organization of Health Care at Various Levels Unit...
Health Care Planning and Organization of Health Care at Various Levels Unit...
RAKESH SAJJAN
Chalukyas of Gujrat, Solanki Dynasty NEP.pptx
Chalukyas of Gujrat, Solanki Dynasty NEP.pptx
Dr. Ravi Shankar Arya Mahila P. G. College, Banaras Hindu University, Varanasi, India.
Community Health Nursing Approaches, Concepts, Roles & Responsibilities Uni...
Community Health Nursing Approaches, Concepts, Roles & Responsibilities Uni...
RAKESH SAJJAN
Q1_ENGLISH_PPT_WEEK 1 power point grade 3 Quarter 1 week 1
Q1_ENGLISH_PPT_WEEK 1 power point grade 3 Quarter 1 week 1
jutaydeonne
GREAT QUIZ EXCHANGE 2025 - GENERAL QUIZ.pptx
GREAT QUIZ EXCHANGE 2025 - GENERAL QUIZ.pptx
Ronisha Das
English 3 Quarter 1_LEwithLAS_Week 1.pdf
English 3 Quarter 1_LEwithLAS_Week 1.pdf
DeAsisAlyanajaneH
Communicable Diseases and National Health Programs Unit 9 | B.Sc Nursing 5t...
Communicable Diseases and National Health Programs Unit 9 | B.Sc Nursing 5t...
RAKESH SAJJAN
Code Profiling in Odoo 18 - Odoo 18 際際滷s
Code Profiling in Odoo 18 - Odoo 18 際際滷s
Celine George
2025 June Year 9 Presentation: Subject selection.pptx
2025 June Year 9 Presentation: Subject selection.pptx
mansk2
How payment terms are configured in Odoo 18
How payment terms are configured in Odoo 18
Celine George
ECONOMICS, DISASTER MANAGEMENT, ROAD SAFETY - STUDY MATERIAL [10TH]
ECONOMICS, DISASTER MANAGEMENT, ROAD SAFETY - STUDY MATERIAL [10TH]
SHERAZ AHMAD LONE
SCHIZOPHRENIA OTHER PSYCHOTIC DISORDER LIKE Persistent delusion/Capgras syndr...
SCHIZOPHRENIA OTHER PSYCHOTIC DISORDER LIKE Persistent delusion/Capgras syndr...
parmarjuli1412
ENGLISH-5 Q1 Lesson 1.pptx - Story Elements
ENGLISH-5 Q1 Lesson 1.pptx - Story Elements
Mayvel Nadal
Nutrition Assessment and Nutrition Education Unit 4 | B.Sc Nursing 5th Seme...
Nutrition Assessment and Nutrition Education Unit 4 | B.Sc Nursing 5th Seme...
RAKESH SAJJAN
SPENT QUIZ NQL JR FEST 5.0 BY SOURAV.pptx
SPENT QUIZ NQL JR FEST 5.0 BY SOURAV.pptx
Sourav Kr Podder
GEOGRAPHY-Study Material [ Class 10th] .pdf
GEOGRAPHY-Study Material [ Class 10th] .pdf
SHERAZ AHMAD LONE
Publishing Your Memoir with Brooke Warner
Publishing Your Memoir with Brooke Warner
Brooke Warner
THE PSYCHOANALYTIC OF THE BLACK CAT BY EDGAR ALLAN POE (1).pdf
THE PSYCHOANALYTIC OF THE BLACK CAT BY EDGAR ALLAN POE (1).pdf
nabilahk908
Assisting Individuals and Families to Promote and Maintain Health Unit 7 | ...
Assisting Individuals and Families to Promote and Maintain Health Unit 7 | ...
RAKESH SAJJAN
Health Care Planning and Organization of Health Care at Various Levels Unit...
Health Care Planning and Organization of Health Care at Various Levels Unit...
RAKESH SAJJAN
Community Health Nursing Approaches, Concepts, Roles & Responsibilities Uni...
Community Health Nursing Approaches, Concepts, Roles & Responsibilities Uni...
RAKESH SAJJAN
Ad

Understanding Learner's Drop-out in MOOCs

  • 1. UNDERSTANDING LEARNERS DROP-OUT IN MOOCS Alya Itani, Laurent Brisson, Serge Garlatti
  • 2. Massive Open Online Course (Premium members) Send feedback to learners to prevent drop-out Provide personalized feedback to teachers Contributions A predictive system that can detect at-risk droppers at different instants of the learners interaction with the course New features associated with learners trajectory of engagement with the course. Suitable intervention strategies for both teachers and learners. UNDERSTANDING LEARNERS DROP-OUT IN MOOCS Alya Itani, Laurent Brisson, Serge Garlatti The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018 UNDERSTANDING LEARNERS DROP-OUT IN MOOCS Context 2
  • 3. The structure of a course may not be helpful to all participants and supporting different patterns of engagement may be beneficial [Onah et. al, 2014] Deficiency in some skills (digital skills or prerequisite) Bad MOOC design (inefficient material, high workload) Lack of intention to complete (material hunter, curious explorers) Inaccurate expectations (misunderstanding about the objectives or content) UNDERSTANDING LEARNERS DROP-OUT IN MOOCS Alya Itani, Laurent Brisson, Serge Garlatti The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018 UNDERSTANDING LEARNERS DROP-OUT IN MOOCS Problems 3
  • 4. Create your Website with HTML (HTML5) 2015-2016 11,520 active learners Completers: 38% (4,333) Droppers: 62% (7,187) Understanding the Web (Web) 2015-2016 7,160 active learners Completers: 71% (5,085) Droppers: 29% (2,075) Features Completed activities Time spent on each activity Grades obtained UNDERSTANDING LEARNERS DROP-OUT IN MOOCS Alya Itani, Laurent Brisson, Serge Garlatti The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018 CASE STUDY Openclassroom Dataset 4
  • 5. Creation of behavioral features 3 new attributes are created for each pair of activities Back jumps (a step back from the scenario planned by the teacher) Forward jumps (intended transition between 2 activities or skipped activities) Inactivity time (for each jump, inactivity time between activities) UNDERSTANDING LEARNERS DROP-OUT IN MOOCS Alya Itani, Laurent Brisson, Serge Garlatti The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018 CASE STUDY Features creation 5
  • 6. CRISP-DM Process Data Preparation New behavior features Modeling Stratified training and test set Algorithms : random forest, gradient boosting, decision tree, logistic regression For each algorithm: Hyperparameter tuning K-fold cross-validation Performance metrics : F-measure, AUC Evaluation Provide personalized feedback to learners and teachers UNDERSTANDING LEARNERS DROP-OUT IN MOOCS Alya Itani, Laurent Brisson, Serge Garlatti The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018 CASE STUDY Predictive process 6
  • 7. Experiments goals Evaluate algorithms performance considering the 3 following contexts Impact of the prediction at 25% or 50% of the course progression Impact of Behavorial features Impact of the course UNDERSTANDING LEARNERS DROP-OUT IN MOOCS Alya Itani, Laurent Brisson, Serge Garlatti The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018 CASE STUDY Experiments 7
  • 8. Percentage of completed activities 25% 50% Behavorial features with without with without Create your site with HTML5 RF 0.76 賊 .02 0.85 賊 .01 GB DT LR Understanding the web RF 0.25 賊 .01 0.91 賊 .01 GB DT LR 0.46 UNDERSTANDING LEARNERS DROP-OUT IN MOOCS Alya Itani, Laurent Brisson, Serge Garlatti The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018 RESULTS Overview (F-measure metric) 8
  • 9. Logistic Regression (6 most influential features in absolute value) Decision Tree (pruned) UNDERSTANDING LEARNERS DROP-OUT IN MOOCS Alya Itani, Laurent Brisson, Serge Garlatti The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018 RESULTS Models readability 9
  • 10. The most discriminating feature is a descriptive indicator (completed activity, grades, time spent) Diagnostic: Is the activity motivating? [Viau, 2000] Does the activity respect constructive alignment? The activity is a lesson or an exercise: Identify difficulties (unannounced prerequisites, ...) Check the target audience of the course The activity is an assessment : Identify difficulties (complex statements, ...) Check the activity (workload, kind of assessment, ...) UNDERSTANDING LEARNERS DROP-OUT IN MOOCS Alya Itani, Laurent Brisson, Serge Garlatti The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018 RESULTS Situation 1 Doing an activity increases the probability of drop-out 10
  • 11. The most discriminating feature is a behavioral indicator (back jump, forward jumps, inactivity time) The feature is a forward-jump that respects the recommended progression Check activity to identify a wrong constructive alignment Suggest new routes to students and change course progression The feature is a back-jump or a forward jump skipping the recommended progression Remind students of the importance of following recommended progression Add self-assessment activities to help to identify relevant jumps (back and forward) UNDERSTANDING LEARNERS DROP-OUT IN MOOCS Alya Itani, Laurent Brisson, Serge Garlatti The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018 RESULTS Situation 2 Doing an jump increases the probability of drop-out 11
  • 12. At a given course progression the prediction is bad and so no feature is really relevant None of the activity over the period allow to anticipate successes and failures : Activities could be designed to build student self-confidence ... but we could check a wrong constructive alignment! Prediction performance increases at a latter instant The most predictive activities are often those that require some commitment (exercices, assemnents): No such activities during the first 25% of course progression The course lack activities to help student to assess their progress UNDERSTANDING LEARNERS DROP-OUT IN MOOCS Alya Itani, Laurent Brisson, Serge Garlatti The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018 RESULTS Situation 3 Prediction performance increases a lot between two different instants 12
  • 13. Contributions A predictive system that can detect at-risk droppers at different instants of the learners interaction with the course New features associated with learners trajectory of engagement with the course Suitable intervention strategies for both teachers and learners Perspectives Use business knowledge: learner model, course structure, learning outcomes (e-education) Focus on social interactions and group dynamics in the learning mechanism (social network analysis) UNDERSTANDING LEARNERS DROP-OUT IN MOOCS Alya Itani, Laurent Brisson, Serge Garlatti The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018 CONCLUSION & PERSPECTIVES 13
  • 14. [Onah et al., 2014] Onah, D. F. O., Sinclair, J., & Boyatt, R. (2014). Dropout rates of massive open online courses behavioural patterns. 6th International Conference on Education and New Learning Technologies, 58255834. https://doi.org/10.13140/RG.2.1.2402.0009 [Viau, 2000] Viau, R. (2000). Conditions to be respected to encourage student motivation. La revue web sur la valorisation du fran巽ais en milieu coll辿gial. Retrieved from https://goo.gl/Cva6KH UNDERSTANDING LEARNERS DROP-OUT IN MOOCS Alya Itani, Laurent Brisson, Serge Garlatti The 19th International Conference on Intelligent Data Engineering and Automated Learning - 21-23 November 2018 REFERENCES 14