Crowdsourcing video annotation
Download as PPTX, PDF
1 like1,659 views
Presentation at the final showcase of the European project PrestoPRIME: Waisda video labeling game, the analysis of the collected tags and tag gardening.
Crowdsourcing video annotation
- 1. Final Showcase PrestoPRIME FP7-ICT-2007-3 231161 Crowdsourcing video annotation Michiel Hildebrand, VU University Amsterdam and Lora Aroyo, Riste Gligorov, Jacco van Ossenbruggen, Guus Schreiber and Maarten Brinkerink, Lotte Baltussen, Johan Oomen, Erwin Verbruggen
- 2. FINAL SHOWCASE 21.- 22.11.2012 VIENNA P Access C Planning/Checking- Enrichment Monitoring and R in/Ingest Running Preservation O Video Quality assessment Multivalent O framework N Preservation Planning Fingerprint content Managing Archive D Metadata Mapping and tracking S Services U Validating Rights Ontology and Access U C Storage and Master RightsDraw Policy based Long- for Master M Levels Crowdsourcing video term Data Management level E materials E annotation; Waisda R R Integration Platforms: Rosetta P4 Knowledge and Services: PrestoCentre Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 2
- 3. FINAL SHOWCASE 21.- 22.11.2012 Motivation VIENNA Goal 1: Collecttime-basedvideo annotations Goal 2: Improve access to video fragments Goal 3: Connect with the public Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 3
- 4. FINAL SHOWCASE 21.- 22.11.2012 VIENNA Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 4
- 5. FINAL SHOWCASE 21.- 22.11.2012 Results VIENNA 2009 TIMAF European Best Practices Award 2010 EuroITV Winner Grand Challenge 2010 ArchivesNext Best Archives on the Web Award 2012 Nominated for Dutch Game Award Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 5
- 6. FINAL SHOWCASE 21.- 22.11.2012 Results Goal 1 : annotation VIENNA Number of tags collected in 4 months by semi-professionals from NCRV and Waisda users Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 6
- 7. FINAL SHOWCASE 21.- 22.11.2012 Results Goal 1 : annotation VIENNA Percentage of tags found in GTAA (Professional), WordNet (Dutch Dictionary), Google Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 7
- 8. FINAL SHOWCASE 21.- 22.11.2012 Results Goal 1 : annotation VIENNA Panofsky-Shatford matrix for 1,313 Waisdatags from 1stWaisda pilot Abstract General Specific Total Who 10 166 177 31% 73 12 57% What 563 Where 0 68 8 7% When 4 31 6 5% Total 7% 74% 9% Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 8
- 9. FINAL SHOWCASE 21.- 22.11.2012 Results Goal 2 : search VIENNA Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 9
- 10. FINAL SHOWCASE 21.- 22.11.2012 Results Goal 2 : search VIENNA Mean Average Precision of fragment retrieval using 50 queries and 198 videos Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 10
- 11. FINAL SHOWCASE 21.- 22.11.2012 Results Goal 3: user engagment VIENNA Number of users 2nd pilot Waisda 735 1000s registered anonymous Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 11
- 12. FINAL SHOWCASE 21.- 22.11.2012 Results goal 3: user engagment VIENNA User visits for Waisda 1st pilot Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 12
- 13. FINAL SHOWCASE 21.- 22.11.2012 Lessons learned VIENNA Use short video fragments Expect tags: in a user vocabulary mainly general descriptions of What and Who Time-based annotations enable: within-video search high recall fragment retrieval Large number of contributors provide additional motivation (short term goal) Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 13
- 14. FINAL SHOWCASE 21.- 22.11.2012 Outcomes VIENNA Waisda white label available under GPL open source license http://github.com/beeldengeluid/waisda Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 14
- 15. FINAL SHOWCASE 21.- 22.11.2012 Outcomes : student projects VIENNA Detecting emotions in Waisda tags (Master project Wouter Meys) Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 15
- 16. FINAL SHOWCASE 21.- 22.11.2012 Outcomes : student projects VIENNA Short term goals within Waisda through missions (master project Olga Beza) Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 16
- 17. FINAL SHOWCASE 21.- 22.11.2012 Outcomes : student projects VIENNA Waisda for fictional movies (PhD research Liliana Melgar) Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 17
- 18. FINAL SHOWCASE 21.- 22.11.2012 Outcomes : new collections VIENNA Film institute Netherlands Dutch broadcaster Europeana V2 Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 18
- 19. FINAL SHOWCASE 21.- 22.11.2012 Install your own Waisda? VIENNA 1. git clone https://github.com/beeldengeluid/waisda.git 2. setup MySQL database 3. add videos identifier title duration URL video source URL key frame 4. style user interface Logo color scheme static pages 5. deploy in Jetty or Tomcat Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 19
- 20. FINAL SHOWCASE 21.- 22.11.2012 Integration : Waisda API VIENNA videos tags submit export update sync Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 20
- 21. FINAL SHOWCASE 21.- 22.11.2012 Integration : Tag Gardening VIENNA videos annotations tags Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 21
- 22. FINAL SHOWCASE 21.- 22.11.2012 Integration : Tag Gardening VIENNA Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 22
- 23. FINAL SHOWCASE 21.- 22.11.2012 challenges VIENNA Maintain Waisda? white label Improve quality of tags Waisda: collect specific types of tags Processing: filter tags to improve precision Michiel Hildebrand (VUA) FP7-ICT-2007-3 231161PrestoPRIME 23