Generating LADs that make sense
0 likes10 views
This document summarizes research on developing a methodology for generating Learning Analytics Dashboards (LADs) that effectively support sensemaking and decision-making. The methodology combines participatory design with stakeholders and generative design techniques. A tool called LADStudio was created to help designers rapidly prototype LADs based on specifications from co-design processes. An evaluation study found the tool had satisfactory usability and user experience. Future work includes collecting more user proposals and addressing barriers to adopting the LAD design methodology in practice.
More Related Content
Similar to Generating LADs that make sense (20)
More from Madjid Sadallah (7)
Recently uploaded (20)
Generating LADs that make sense
- 1. Generating LADs that make sense CSEDU 2023 22/04/2023 Madjid SADALLAH* & Jean-Marie GILLIOT MOTEL, IMT Atlantique / Lab-STICC. Brest, France *madjid.sadallah@imt-atlantique.fr | https://www.madjidsadallah.net/
- 2. M. Sadallah. Generating LADs that make sense. CSEDU 2023 Roadmap Context & Research questions 1 Theoretical framework 2 LADStudio 3 Evaluation study 4 Conclusion 5 2
- 3. M. Sadallah. Generating LADs that make sense. CSEDU 2023 Why Learning Analytics Dashboards (LADs) Design Matters LADs : visualize educational data resulting from LA process Well-designed LADs: effective to awareness & decision-making sustain the learning process & improve its outcomes but hard to design Limited LAD adoption little or no stakeholder involvement in the design process inadequate addressing of their needs and expectations insufficient support for their abilities e.g., visual literacy poor understanding of the associated decision-making process: how the sensemaking with LADs occurs 3 Our aim : A design methodology that addresses these factors for effective LADs
- 4. M. Sadallah. Generating LADs that make sense. CSEDU 2023 Research project 4 Guiding Design Design Space DEFLAD Ideation: Codesign toolkit PaDLAD Prototyping: Generating LADs that make sense LADStudio (Prieto et al., 2018)
- 5. M. Sadallah. Generating LADs that make sense. CSEDU 2023 Current stage, in other words From this: To this: 5
- 6. M. Sadallah. Generating LADs that make sense. CSEDU 2023 Research Questions 6 How can the decision-making process be reflected on a LAD? RQ1 How to support designers in translating design specifications into LADs, with explicit support for the decision-making process? RQ2
- 7. M. Sadallah. Generating LADs that make sense. CSEDU 2023 Making sense of LAD sensemaking Integrate the sensemaking dimension in LAD design Sensemaking = process of creating meaning from complex data SM with LADs = process of interpreting data presented on the LAD and using it to inform and support decision-making 7 How can the decision-making process be reflected on a LAD? RQ1
- 8. M. Sadallah. Generating LADs that make sense. CSEDU 2023 Making sense of LAD sensemaking Distributed cognition theory (Hutchins 1995) Sensemaking develops through perception and interaction Data/Frame theory of SM (Klein 2006) LAD displays = Data Gained insight = frame 8 How can the decision-making process be reflected on a LAD? RQ1 Interaction-Sensemaking Loop LAD design: adequate support needed for accurate frame construction (e.g., effective interaction and appropriate representations)
- 9. M. Sadallah. Generating LADs that make sense. CSEDU 2023 Making sense of LAD sensemaking DEFLAD : LAD Design Framework, defining a refined design space Context of use LAD goal, focus and targeted SA level Stakeholders, timing and info circulation Interaction & visualization Data & representation Types of Interactions Explicit SM features at different levels () 9 How can the decision-making process be reflected on a LAD? RQ1
- 10. M. Sadallah. Generating LADs that make sense. CSEDU 2023 Generative CoDesign Methodology A methodology to combine participatory design & generative design Participatory design high levels of stakeholder involvement stakeholders needs, requirements and abilities Generative design rapidly generate prototypes compliant with stakeholders reqs and designers specs leveraging the computational power and efficiency 10 Supporting designers in prototyping LADs with explicit SM processes? RQ2
- 11. M. Sadallah. Generating LADs that make sense. CSEDU 2023 Generative CoDesign Methodology Generative codesign methodology Clear and structured approach Encourage stakeholder involvement Comply with end-user needs Rapid prototyping 11 Supporting designers in prototyping LADs with explicit SM processes? RQ2
- 12. M. Sadallah. Generating LADs that make sense. CSEDU 2023 LADStudio, a tool for generating codesigned LADs Prototyping tool that implements the generative codesign approach: from needs capturing to their realization Specifications developed through co-design process Built within the proposed design space Instrumented with features to support end-user SM and DM processes A platform for stakeholder involvement and customization Enables designers to rapidly obtain functional prototypes. 12 4 modules 1. Library of components 2. Specification module 3. Generation module 4. Grafana instance
- 13. M. Sadallah. Generating LADs that make sense. CSEDU 2023 LADStudio : library of components
- 14. M. Sadallah. Generating LADs that make sense. CSEDU 2023 LADStudio : LAD specification & generation
- 15. M. Sadallah. Generating LADs that make sense. CSEDU 2023 Qualitative evaluation study Evaluate usability and user experience of LADStudio 13 Participants: purposive sampling Inclusion: experience with EdTools Exclusion: unwillingness to participate Evaluation procedure LADStudio demonstration Independent experimentation Collective codesign workshop Questionnaire and open-ended questions Duration: 2 hours 15
- 16. M. Sadallah. Generating LADs that make sense. CSEDU 2023 Evaluation study: instruments Usability evaluation System Usability Scale (SUS) questionnaire Reliable usability measure User Experience (UX) evaluation User Experience Questionnaire (UEQ) Valid tool for UX assessment 26 items, 6 scales: attractiveness, efficiency, perspicuity, dependability, stimulation, novelty 16
- 17. M. Sadallah. Generating LADs that make sense. CSEDU 2023 Results: usability Highest scores Q1: able to use it frequently Q3: easy to use Q5: components well integrated Lowest scores Q2: unnecessarily complex Q4: need support Q6: too much inconsistency 17 SUS score: 71.15 System usability rated as OK
- 18. M. Sadallah. Generating LADs that make sense. CSEDU 2023 Results: UX Highest score novelty efficiency stimulation Lowest score perspicuity 18 UEQ score: 1.99 Rating : Good!
- 19. M. Sadallah. Generating LADs that make sense. CSEDU 2023 Results: Participants' Feedback Challenging at first but effective with practice Capable of creating useful and well-designed LADs Reasoning mirroring, support and guidance, at design Improvement areas Challenging theoretical concepts Need design support ; lack of data/visual literacy 19
- 20. M. Sadallah. Generating LADs that make sense. CSEDU 2023 Discussion Usability & UX: satisfactory Encouraging feedback Some limitations self-selection bias small sample size restricts generalizability, further investigation needed for LAD quality 20
- 21. M. Sadallah. Generating LADs that make sense. CSEDU 2023 Conclusion & outlook Efficient and involved LAD design methodology participatory & generative LADStudio: functional prototypes from co-design specs Success in innovative proposals and LAD adoption Next steps: collect user and practitioner LAD proposals address design adoption issues, and share findings with the learning community. 21
- 22. Thank you for your attention! 22 Madjid SADALLAH madjid.sadallah@imt-atlantique.fr https://www.madjidsadallah.net/