Constructing Conceptual Knowledge Artefacts: Activity Patterns in the Ontology Authoring Process
•
0 likes•1,327 views
We present the results of a study that identifies common activity patterns through analysis of eye-tracking data and the event logs of the popular authoring tool, Protégé. Informed by the activity patterns discovered, we propose design guidelines for bulk editing, efficient reasoning and increased situational awareness. Methodological implications go beyond the remit of knowledge artefacts.
Constructing Conceptual Knowledge Artefacts: Activity Patterns in the Ontology Authoring Process
- 1. Constructing Conceptual Knowledge Artefacts: Activity Patterns in the Ontology Authoring Process Markel Vigo, Caroline Jay, Robert Stevens University of Manchester (UK) CHI 2015, Seoul (Korea) @markelvigo markel.vigo@manchester.ac.uk
- 2. What are ontologies? Mad cow Lion Giraffe Cow Animal Classes eats SubClassOf PropertiesAxioms Cow, Giraffe, Lion à SubClassOf Animal Mad cow à SubClassOf Cow Mad cow, lion à Animal eats some Animal
- 3. What are ontologies? Mad cow SubClassOf Lion Giraffe Cow Animal Classes PropertiesAxioms Cow, Giraffe, Lion à SubClassOf Animal Mad cow à SubClassOf Cow Mad cow, lion à Animal eats some Animal eats
- 4. What are ontologies? Mad cow SubClassOf Lion Giraffe Cow Animal Classes PropertiesAxioms Cow, Giraffe, Lion à SubClassOf Animal Mad cow à SubClassOf Cow Mad cow, lion à Animal eats some Animal Vegetarian à Animal and (eats only (not (Animal))) eats
- 5. What are ontologies? Mad cow SubClassOf Lion Giraffe Cow Animal Classes PropertiesAxioms Cow, Giraffe, Lion à SubClassOf Animal Mad cow à SubClassOf Cow Mad cow, lion à Animal eats some Animal Vegetarian à Animal and (eats only (not (Animal))) eats Vegetarian
- 6. What are ontologies? Mad cow SubClassOf Lion Giraffe Cow Animal Classes PropertiesAxioms Cow, Giraffe, Lion à SubClassOf Animal Mad cow à SubClassOf Cow Mad cow, lion à Animal eats some Animal Vegetarian à Animal and (eats only (not (Animal))) eats Vegetarian
- 7. Complex artefacts • Large size • Domain expertise required • Critical • Highly expressive
- 8. Problem We don’t know... § if existing ontology authoring tools are effective § how ontologist go about authoring § the authoring workflows
- 10. Authoring environment: Protégé File menu Class hierarchy Property hierarchy Annotations-Usage Description Mad cow SubClassOf Lion Giraffe Cow Animal Classes Properties Axioms Cow, Giraffe, Lion à SubClassOf Animal Mad cow à SubClassOf Cow Mad cow, lion à Animal eats some Animal Vegetarian à Animal and (eats only (not (Animal))) eats
- 11. File menu Class hierarchy Property hierarchy Annotations-Usage Description Pop up Explanation Edit entity Authoring environment: Protégé Mad cow SubClassOf Lion Giraffe Cow Animal Classes Properties Axioms Cow, Giraffe, Lion à SubClassOf Animal Mad cow à SubClassOf Cow Mad cow, lion à Animal eats some Animal Vegetarian à Animal and (eats only (not (Animal))) eats
- 12. Study • 16 ontology authors, 3 typical tasks • Protégé à Protégé4US • Eye-tracking • Sync and merge interaction + fixation data • N-gram analysis 1389973572771,eye,Class hierarchy! 1389973577038,eye,Description! 1389973584775,log,EntitySelected! 1389973586349,log,DescriptionSelected! 1389973598978,log,EntityModified! 1389973603166,log,EntitySelected! 1389973605053,log,DescriptionSelected! 1389973607847,log,EntityModified! 1389973616404,eye,Description! 1389973616754,eye,Class hierarchy! 1389973617221,eye,Description!
- 13. Findings: interaction log data • Interaction events account for 65% of events while authoring events are 30% • The top 3 events (entity selection, description selection and invocation of editing menu) account for 56% of events 6 12 19 23 28 39 47 61 82 113 139 142 182 199 267 314 332 426 617 960 1004 1405 2793 Back Undo Get explanation Entity renamed Set property Entity dragged Property addition Entity deleted Load ontology Hierarchy collapsed(i) Save Description selected(i) Run reasoner Hierarchy collapsed Convert into defined Hierarchy expanded(i) Class addition Entity selected(i) Hierarchy expanded Entity edited:finish Entity edited:start Description selected Entity selected 0 1000 2000
- 14. Findings: eye-tracking data The class hierarchy is the pivotal area • Index of the ontology • External memory Transitions between AOIs from to Ann−Usage Class hierarchy Description Explanation File menu Pop up Edit Entity Prop. hierarchy Ann−U sageC lass hierarchy D escription Explanation File m enu Pop up EditEntityProp.hierarchy 0 1000 2000 3000 4000 5000 6000
- 15. Findings: eye-tracking data The class hierarchy receives users’ attention 45% of the time 0 100 200 300 400 File m enu Ann.−U sage C lass hierarchy D escription Popup Editentity Prop.hierarchy Explanation AOI time(sec)
- 16. Findings: workflows Select description Select entity 0.29 Modify entity 0.37 0.63 0.59 Editing activity Run reasoner Convert into defined class Save Select description 0.16 0.15 0.17 0.40 Expand inferred hierarchy 0.30 Select entity 0.41 0.37 0.43 Select inferred entity 0.54 0.25 0.12 Reasoning activity Select entity Expand hierarchy 0.48 0.31 Select inferred entity Expand inferred hierarchy 0.25 0.43 0.12 0.54 Load ontology 0.52 0.31 Expand hierarchy Select description 0.29 0.37 Exploration activity Exploration workflow Editing workflow Reasoning workflow
- 17. Implications: from raw data to workflows Workflows can be automatically identified raw data cleaning merging filtering workflow detection ~7K rows ~200 rows • Different authoring styles • Time distribution per workflow • Identification of confounding variables
- 18. Implications for design • Support for bulk editing • Anticipation of reasoner invocation • Automatic detection of authoring problems • Make changes to the inferred hierarchy explicit
- 19. tl;dr • Identification of activity patterns when dealing with complex interactive artefacts • Interaction log data + eye gaze data • Data-driven • Application on knowledge artefacts
- 20. Markel Vigo, Caroline Jay, Robert Stevens University of Manchester (UK) @markelvigo markel.vigo@manchester.ac.uk