際際滷

際際滷Share a Scribd company logo

Ectel sem_info_rec_learning_resources_v6.0_20120921_ma

Mojisola Erdt n辿e Anjorin
Mojisola Erdt n辿e Anjorin

This document summarizes a presentation on exploiting semantic information for graph-based recommendations of learning resources using the CROKODIL platform. It introduces CROKODIL's extended folksonomy model, which incorporates semantic tag types, activities, learner groups, and friendships. It describes two approaches for calculating resource scores using the extended graph model: AScore, which runs a graph-based ranking algorithm on the extended folksonomy graph; and AInheritScore, which leverages activity hierarchies to calculate scores. The presentation evaluates these approaches on a real-world dataset using leave-one-out cross-validation, finding that AScore and GFolkRank perform best. It concludes by discussing future work, including additional evaluation

1 of 35
Download to read offline
息 author(s) of these slides including research results from the KOM research network and TU Darmstadt; otherwise it is specified at the respective slide 21-Sep-12 Prof. Dr.-Ing. Ralf Steinmetz KOM - Multimedia Communications Lab ECTEL__Sem_Info_rec_learning_resources_v6.0_20120921_MA.pptx Exploiting Semantic Information for Graph-based Recommendations of Learning Resources Mojisola Anjorin Thomas Rodenhausen Renato Dom鱈nguez Garc鱈a Christoph Rensing EC-TEL 2012, Saarbr端cken Research Talk Ranking Algorithms 際際滷share Tags ResourcesUsers Prepare Talk Read-Up on Basics Activities Find Related Work Friends Friends Friends Blue Group
KOM Multimedia Communications Lab 2 Resource-Based Learning
KOM Multimedia Communications Lab 3 Application Scenario: CROKODIL CROKODIL is a platform offering support for resource-based learning 則рSemantic Tag Types 則рActivities 則рLearner Groups & Friendships 則рRecommendations [Anjorin et al, 2011] http://demo.crokodil.de
KOM Multimedia Communications Lab 4 則рMotivation: Resource-based Learning 則рApplication Scenario: CROKODIL 則рCROKODILs Extended Folksonomy Model 則рAscore and AInheritScore 則рEvaluation Methodology, Metrics and Results 則рConclusion & Future Work Overview
KOM Multimedia Communications Lab 5 A folksonomy is a quadruple F:= (U, T, R, Y), where U Users T Tags R Resources Y U T R - tag assignment Folksonomy Model Research Talk Ranking Algorithms 際際滷share Tags ResourcesUsers [Hotho et al. 2006]
KOM Multimedia Communications Lab 6 CROKODIL Extends the Folksonomy Model Research Talk Ranking Algorithms 際際滷share Tags ResourcesUsers
KOM Multimedia Communications Lab 7 with Semantic Tag Types [B旦hnstedt et al. 2009] Research Talk Ranking Algorithms 際際滷share Tags ResourcesUsers Genre Event Person Location Other Topic
KOM Multimedia Communications Lab 8 with Activities Research Talk Ranking Algorithms 際際滷share Tags ResourcesUsers Prepare Talk Read-Up on Basics Activities Find Related Work
KOM Multimedia Communications Lab 9 with Learner Groups and Friendships Research Talk Ranking Algorithms 際際滷share Tags ResourcesUsers Prepare Talk Read-Up on Basics Activities Find Related Work Friends Friends Friends Blue Group
KOM Multimedia Communications Lab 10 CROKODILs Extended Folksonomy FC:= (U, TTyped, R, YT, (A, <), YA, YU, G, friends) where U users TTyped typed tags R learning resources YT U TTyped R tag assignment (A, <) activities with sub-activities YA U A R activity assignment YU U A activity membership assignment G P(U) groups of learners friends U U friendship relation Research Talk Ranking Algorithms 際際滷share Tags ResourcesUsers Prepare Talk Read-Up on Basics Activities Find Related Work Friends Friends Friends Blue Group
KOM Multimedia Communications Lab 11 Resource Recommendations for CROKODIL http://demo.crokodil.de
KOM Multimedia Communications Lab 12 Graph-based recommender techniques can be classified as neighbourhood-based collaborative filtering approaches Graph-based Resource Recommendations Graph-based Ranking Algorithm Resource Score r1 0.9 r2 0.7 r3 0.5 r4 0.2 1 1 2 1 P1 P2 P4 P3 3 4 2 1 2 Folksonomy Graph e.g. FolkRank based on Random Walk of PageRank Recommendation List (ranked resources) [Desrosiers et al. 2011]
KOM Multimedia Communications Lab 13 則рMotivation: Resource-based Learning 則рApplication Scenario: CROKODIL 則рCROKODILs Extended Folksonomy Model 則рAscore and AInheritScore 則рEvaluation Methodology, Metrics and Results 則рConclusion & Future Work Overview
KOM Multimedia Communications Lab 14 1. Add activity nodes Vc = VF A 2. Add edges: 則рactivity assignments (u, r, a) 則рassignments of a user to an activity (u, a) 則рactivity hierarchies (asub , asuper) 4. Assign weights to edges: 則рw(r,a) = w(r,u) = w(u,a) = max(|Ut,r|) 則рw(u, a) = max(|Ru,t|) 則рw(asub,asuper) = max(|Ut,r|, |Ru,t|) 5. Run graph-based ranking algorithm e.g. FolkRank AScore [Abel et al, 2011]Inspired by GFolkRank Extend the Folksonomy Graph F = (V, E) with Activities Research Talk Ranking Algorithms 際際滷share Tags ResourcesUsers Prepare Talk Read-Up on Basics Activities Find Related Work
KOM Multimedia Communications Lab 15 則рDepending on the tags of a user, scores are inherited over the activity hierarchy 則рResources and users assigned to activities influence the scores as well 則рScores are attenuated depending on activity distance 則р Activity distance between two activities: the number of hops from one activity to the other AInheritScore [Abel et al, 2011]Inspired by GRank Leveraging Activity Hierarchies to Calculate Scores Research Talk Ranking Algorithms Research Talk Prepare Talk Read-Up on Basics Find Related Work ... ... ...
KOM Multimedia Communications Lab 16 則рMotivation: Resource-based Learning 則рApplication Scenario: CROKODIL 則рCROKODILs Extended Folksonomy Model 則рAscore and AInheritScore 則рEvaluation Methodology, Metrics and Results 則рConclusion & Future Work Overview
KOM Multimedia Communications Lab 17 GroupMe! dataset Evaluation Corpus and Evaluation Metrics [Abel et al, GroupMe!] Elements Count Users 649 Tags 2580 Resources 1789 Groups of Resources 1143 Posts 1865 Tag assignments 4366 The mean of the Average Precision over several queries Q Mean Normalized Precision: The mean of the Precision@k over several queries Q MAP(Q) = 1 |Q| |Q| j=1 1 mj mj k=1 Precision(Rjk) Mean Average Precision: MNP(Q, k) = 1 |Q| |Q| j=1 Precisionj(k) Precisionmax,j(k) [Manning et al 2008]
KOM Multimedia Communications Lab 18 Tango Buenos Aires Dancing Festival Tango Buenos Aires Dancing Festival A post is a Pu,r= {(u,r,t)|(u,r,t) Y} For LeavePostOut, the recommendation task with user as input is harder as with tag as input Evaluation Methodology: LeavePostOut [J辰schke et al. 2007]
KOM Multimedia Communications Lab 19 RTr,t= {(u,r,t)|(u,r,t) Y} For LeaveRTOut, the recommendation task with tag as input is harder as with user as input Evaluation Methodology: LeaveRTOut Tango Buenos Aires Dancing Festival Tango Buenos Aires Dancing Festival
KOM Multimedia Communications Lab 20 A violin plot is a combination of a box plot and a density trace Visualization of Results with Violin Plots [Hintze et al. 1998]
KOM Multimedia Communications Lab 21 A violin plot is a combination of a box plot and a density trace Visualization of Results with Violin Plots Median 3rd Quartile 1st Quartile [Hintze et al. 1998]
KOM Multimedia Communications Lab 22 Evaluation results with user as input Evaluation Results for LeavePostOut
KOM Multimedia Communications Lab 23 Evaluation results with user as input Evaluation Results for LeavePostOut
KOM Multimedia Communications Lab 24 Evaluation results with user as input Evaluation Results for LeavePostOut
KOM Multimedia Communications Lab 25 Evaluation results with user as input Evaluation Results for LeavePostOut
KOM Multimedia Communications Lab 26 Evaluation results with user as input Evaluation Results for LeavePostOut
KOM Multimedia Communications Lab 27 Evaluation results with user as input Evaluation Results for LeavePostOut
KOM Multimedia Communications Lab 28 Evaluation Results for LeavePostOut Approaches MAP GFolkRank 0.70 AScore 0.70 AInheritscore 0.47 GRank 0.38 FolkRank 0.19 Popularity 0.00
KOM Multimedia Communications Lab 29 Evaluation Results for LeaveRTOut Evaluation results with user as input
KOM Multimedia Communications Lab 30 Evaluation Results for LeaveRTOut Approaches MAP AScore 0.20 GFolkRank 0.20 FolkRank 0.18 GRank 0.14 AInheritscore 0.11 Popularity 0.02
KOM Multimedia Communications Lab 31 Exploiting hierarchical activity structures as found in CROKODIL can improve the ranking of resources for the purpose of recommending learning resources 則рAScore 則рAInheritscore Future Work 則рEvaluation using a data set from CROKODIL 則рUser Study 則рHybrid approaches Conclusion and Future Work www.crokodil.de
KOM Multimedia Communications Lab 32 Questions Contact
KOM Multimedia Communications Lab 33 Statistical Significance Tests LeavePostOut More effective than Popularity Folk Rank GFolk Rank AScore GRank AInheritScore Poularity FolkRank X GFolkRank X X X X X AScore X X X X GRank X X AInheritScore X X X Significance matrix of pair-wise comparisons of LeavePostOut results Based on Average Precision with a significance level of p = 0.05
KOM Multimedia Communications Lab 34 Statistical Significance Tests LeaveRTOut More effective than Popularity Folk Rank GFolk Rank AScore GRank AInheritScore Poularity FolkRank X X X GFolkRank X X X X AScore X X X X X GRank X X AInheritScore X Significance matrix of pair-wise comparisons of LeaveRTOut results Based on Average Precision with a significance level of p = 0.05
KOM Multimedia Communications Lab 35 Adapted PageRank ! ! ! # # $%'()*+, Tango 0 Buenos Aires 0 Buenos Aires 0 Dancing Festival 0 1 -. #-. #-. -. PageRanks intelligent surfer model The ranking of a node is determined by how often the surfer visits the node Adjoining edges are followed with a certain probability determined by the edge weights The query node acts as the starting point and focus i.e. the surfer returns to this node with a certain probability determined by the node weights [Hotho et al. 2006]

More Related Content

Ectel sem_info_rec_learning_resources_v6.0_20120921_ma

  • 1. 息 author(s) of these slides including research results from the KOM research network and TU Darmstadt; otherwise it is specified at the respective slide 21-Sep-12 Prof. Dr.-Ing. Ralf Steinmetz KOM - Multimedia Communications Lab ECTEL__Sem_Info_rec_learning_resources_v6.0_20120921_MA.pptx Exploiting Semantic Information for Graph-based Recommendations of Learning Resources Mojisola Anjorin Thomas Rodenhausen Renato Dom鱈nguez Garc鱈a Christoph Rensing EC-TEL 2012, Saarbr端cken Research Talk Ranking Algorithms 際際滷share Tags ResourcesUsers Prepare Talk Read-Up on Basics Activities Find Related Work Friends Friends Friends Blue Group
  • 2. KOM Multimedia Communications Lab 2 Resource-Based Learning
  • 3. KOM Multimedia Communications Lab 3 Application Scenario: CROKODIL CROKODIL is a platform offering support for resource-based learning 則рSemantic Tag Types 則рActivities 則рLearner Groups & Friendships 則рRecommendations [Anjorin et al, 2011] http://demo.crokodil.de
  • 4. KOM Multimedia Communications Lab 4 則рMotivation: Resource-based Learning 則рApplication Scenario: CROKODIL 則рCROKODILs Extended Folksonomy Model 則рAscore and AInheritScore 則рEvaluation Methodology, Metrics and Results 則рConclusion & Future Work Overview
  • 5. KOM Multimedia Communications Lab 5 A folksonomy is a quadruple F:= (U, T, R, Y), where U Users T Tags R Resources Y U T R - tag assignment Folksonomy Model Research Talk Ranking Algorithms 際際滷share Tags ResourcesUsers [Hotho et al. 2006]
  • 6. KOM Multimedia Communications Lab 6 CROKODIL Extends the Folksonomy Model Research Talk Ranking Algorithms 際際滷share Tags ResourcesUsers
  • 7. KOM Multimedia Communications Lab 7 with Semantic Tag Types [B旦hnstedt et al. 2009] Research Talk Ranking Algorithms 際際滷share Tags ResourcesUsers Genre Event Person Location Other Topic
  • 8. KOM Multimedia Communications Lab 8 with Activities Research Talk Ranking Algorithms 際際滷share Tags ResourcesUsers Prepare Talk Read-Up on Basics Activities Find Related Work
  • 9. KOM Multimedia Communications Lab 9 with Learner Groups and Friendships Research Talk Ranking Algorithms 際際滷share Tags ResourcesUsers Prepare Talk Read-Up on Basics Activities Find Related Work Friends Friends Friends Blue Group
  • 10. KOM Multimedia Communications Lab 10 CROKODILs Extended Folksonomy FC:= (U, TTyped, R, YT, (A, <), YA, YU, G, friends) where U users TTyped typed tags R learning resources YT U TTyped R tag assignment (A, <) activities with sub-activities YA U A R activity assignment YU U A activity membership assignment G P(U) groups of learners friends U U friendship relation Research Talk Ranking Algorithms 際際滷share Tags ResourcesUsers Prepare Talk Read-Up on Basics Activities Find Related Work Friends Friends Friends Blue Group
  • 11. KOM Multimedia Communications Lab 11 Resource Recommendations for CROKODIL http://demo.crokodil.de
  • 12. KOM Multimedia Communications Lab 12 Graph-based recommender techniques can be classified as neighbourhood-based collaborative filtering approaches Graph-based Resource Recommendations Graph-based Ranking Algorithm Resource Score r1 0.9 r2 0.7 r3 0.5 r4 0.2 1 1 2 1 P1 P2 P4 P3 3 4 2 1 2 Folksonomy Graph e.g. FolkRank based on Random Walk of PageRank Recommendation List (ranked resources) [Desrosiers et al. 2011]
  • 13. KOM Multimedia Communications Lab 13 則рMotivation: Resource-based Learning 則рApplication Scenario: CROKODIL 則рCROKODILs Extended Folksonomy Model 則рAscore and AInheritScore 則рEvaluation Methodology, Metrics and Results 則рConclusion & Future Work Overview
  • 14. KOM Multimedia Communications Lab 14 1. Add activity nodes Vc = VF A 2. Add edges: 則рactivity assignments (u, r, a) 則рassignments of a user to an activity (u, a) 則рactivity hierarchies (asub , asuper) 4. Assign weights to edges: 則рw(r,a) = w(r,u) = w(u,a) = max(|Ut,r|) 則рw(u, a) = max(|Ru,t|) 則рw(asub,asuper) = max(|Ut,r|, |Ru,t|) 5. Run graph-based ranking algorithm e.g. FolkRank AScore [Abel et al, 2011]Inspired by GFolkRank Extend the Folksonomy Graph F = (V, E) with Activities Research Talk Ranking Algorithms 際際滷share Tags ResourcesUsers Prepare Talk Read-Up on Basics Activities Find Related Work
  • 15. KOM Multimedia Communications Lab 15 則рDepending on the tags of a user, scores are inherited over the activity hierarchy 則рResources and users assigned to activities influence the scores as well 則рScores are attenuated depending on activity distance 則р Activity distance between two activities: the number of hops from one activity to the other AInheritScore [Abel et al, 2011]Inspired by GRank Leveraging Activity Hierarchies to Calculate Scores Research Talk Ranking Algorithms Research Talk Prepare Talk Read-Up on Basics Find Related Work ... ... ...
  • 16. KOM Multimedia Communications Lab 16 則рMotivation: Resource-based Learning 則рApplication Scenario: CROKODIL 則рCROKODILs Extended Folksonomy Model 則рAscore and AInheritScore 則рEvaluation Methodology, Metrics and Results 則рConclusion & Future Work Overview
  • 17. KOM Multimedia Communications Lab 17 GroupMe! dataset Evaluation Corpus and Evaluation Metrics [Abel et al, GroupMe!] Elements Count Users 649 Tags 2580 Resources 1789 Groups of Resources 1143 Posts 1865 Tag assignments 4366 The mean of the Average Precision over several queries Q Mean Normalized Precision: The mean of the Precision@k over several queries Q MAP(Q) = 1 |Q| |Q| j=1 1 mj mj k=1 Precision(Rjk) Mean Average Precision: MNP(Q, k) = 1 |Q| |Q| j=1 Precisionj(k) Precisionmax,j(k) [Manning et al 2008]
  • 18. KOM Multimedia Communications Lab 18 Tango Buenos Aires Dancing Festival Tango Buenos Aires Dancing Festival A post is a Pu,r= {(u,r,t)|(u,r,t) Y} For LeavePostOut, the recommendation task with user as input is harder as with tag as input Evaluation Methodology: LeavePostOut [J辰schke et al. 2007]
  • 19. KOM Multimedia Communications Lab 19 RTr,t= {(u,r,t)|(u,r,t) Y} For LeaveRTOut, the recommendation task with tag as input is harder as with user as input Evaluation Methodology: LeaveRTOut Tango Buenos Aires Dancing Festival Tango Buenos Aires Dancing Festival
  • 20. KOM Multimedia Communications Lab 20 A violin plot is a combination of a box plot and a density trace Visualization of Results with Violin Plots [Hintze et al. 1998]
  • 21. KOM Multimedia Communications Lab 21 A violin plot is a combination of a box plot and a density trace Visualization of Results with Violin Plots Median 3rd Quartile 1st Quartile [Hintze et al. 1998]
  • 22. KOM Multimedia Communications Lab 22 Evaluation results with user as input Evaluation Results for LeavePostOut
  • 23. KOM Multimedia Communications Lab 23 Evaluation results with user as input Evaluation Results for LeavePostOut
  • 24. KOM Multimedia Communications Lab 24 Evaluation results with user as input Evaluation Results for LeavePostOut
  • 25. KOM Multimedia Communications Lab 25 Evaluation results with user as input Evaluation Results for LeavePostOut
  • 26. KOM Multimedia Communications Lab 26 Evaluation results with user as input Evaluation Results for LeavePostOut
  • 27. KOM Multimedia Communications Lab 27 Evaluation results with user as input Evaluation Results for LeavePostOut
  • 28. KOM Multimedia Communications Lab 28 Evaluation Results for LeavePostOut Approaches MAP GFolkRank 0.70 AScore 0.70 AInheritscore 0.47 GRank 0.38 FolkRank 0.19 Popularity 0.00
  • 29. KOM Multimedia Communications Lab 29 Evaluation Results for LeaveRTOut Evaluation results with user as input
  • 30. KOM Multimedia Communications Lab 30 Evaluation Results for LeaveRTOut Approaches MAP AScore 0.20 GFolkRank 0.20 FolkRank 0.18 GRank 0.14 AInheritscore 0.11 Popularity 0.02
  • 31. KOM Multimedia Communications Lab 31 Exploiting hierarchical activity structures as found in CROKODIL can improve the ranking of resources for the purpose of recommending learning resources 則рAScore 則рAInheritscore Future Work 則рEvaluation using a data set from CROKODIL 則рUser Study 則рHybrid approaches Conclusion and Future Work www.crokodil.de
  • 32. KOM Multimedia Communications Lab 32 Questions Contact
  • 33. KOM Multimedia Communications Lab 33 Statistical Significance Tests LeavePostOut More effective than Popularity Folk Rank GFolk Rank AScore GRank AInheritScore Poularity FolkRank X GFolkRank X X X X X AScore X X X X GRank X X AInheritScore X X X Significance matrix of pair-wise comparisons of LeavePostOut results Based on Average Precision with a significance level of p = 0.05
  • 34. KOM Multimedia Communications Lab 34 Statistical Significance Tests LeaveRTOut More effective than Popularity Folk Rank GFolk Rank AScore GRank AInheritScore Poularity FolkRank X X X GFolkRank X X X X AScore X X X X X GRank X X AInheritScore X Significance matrix of pair-wise comparisons of LeaveRTOut results Based on Average Precision with a significance level of p = 0.05
  • 35. KOM Multimedia Communications Lab 35 Adapted PageRank ! ! ! # # $%'()*+, Tango 0 Buenos Aires 0 Buenos Aires 0 Dancing Festival 0 1 -. #-. #-. -. PageRanks intelligent surfer model The ranking of a node is determined by how often the surfer visits the node Adjoining edges are followed with a certain probability determined by the edge weights The query node acts as the starting point and focus i.e. the surfer returns to this node with a certain probability determined by the node weights [Hotho et al. 2006]