ijcnlp04.....................................................
- 1. Visual Semantics and Ontology of Eventive Verbs Minhua Eunice Ma and Paul Mc Kevitt School of Computing and Intelligent Systems Faculty of Engineering University of Ulster, Magee Derry/Londonderry, N. Ireland
- 2. IJCNLP-04 Sanya, China, 22 Mar 2004 Outline ? Background: CONFUCIUS ? Previous verb taxonomies ? Visual semantics & verb classes ? CONFUCIUS’ ontology of verbs ? Current status of implementation ? Relation to other work ? Conclusion & future work
- 3. IJCNLP-04 Sanya, China, 22 Mar 2004 Architecture of CONFUCIUS 3D authoring tools, existing 3D models & character models Visual knowledge (3D graphic library) Prefabricated objects (knowledge base) Surface transformer Media allocator Natural Language Processing Text To Speech Sound effects Animation engine Synchronising & fusion 3D world with audio in VRML Natural language sentences Language knowledge mapping LCS lexicon WordNet semantic representations visual knowledge
- 4. IJCNLP-04 Sanya, China, 22 Mar 2004 NLP in CONFUCIUS Coreference resolution Part-of-speech tagger Syntactic parser Morphological parser Semantic inference Pre-processing Connexor FDG parser WordNet LCS database FEATURES Disambiguation Temporal reasoning Lexical temporal relations Post-lexical temporal relations
- 5. IJCNLP-04 Sanya, China, 22 Mar 2004 ? Grammatical categorisation & valency ? Thematic roles (Fillmore, 1968; Jackendoff, 1990; Halliday, 1985; Dowty, 1991) ? Aspectual classes (Vendler, 1967; Stede, 1996) ? Semantic verb classes in WordNet (Fellbaum, 1998) ? Levin’s (1993) verb classes ? Dimension of causation (Asher & Lascarides, 1995) Previous verb taxonomies
- 6. IJCNLP-04 Sanya, China, 22 Mar 2004 Grammatical categorisation & valency ? Subcategorisation description of verb categories in LDOCE (Longman Dictionary of Contemporary English) ? D – ditransitive ? I – intransitive ? L – linking verb with complement ? T1 – transitive verb with NP object ? T3 – transitive verb with infinitival clause as object
- 7. IJCNLP-04 Sanya, China, 22 Mar 2004 ? Subcategorisation description of verb categories in LDOCE (Longman Dictionary of Contemporary English) ? Syntactic valency ? Obligatory valency fillers (complements) e.g. subject, object ? Optional valency fillers (adjuncts) e.g. temporal, locational adjuncts ? Semantic valency (Leech, 1981) Grammatical categorisation & valency
- 8. IJCNLP-04 Sanya, China, 22 Mar 2004 Thematic roles ? Other names: theta-role, case role, deep grammatical function, transitivity role, valency role, case frame ? Extend syntactic analysis into semantic domain to capture roles of participants surface case (nominative, accusative) surface function (subject, object) ? Classifying verbs based on thematic roles (Dixon, 1991) Thematic roles (e.g. agent, patient/theme, instrument, source, goal, place)
- 9. IJCNLP-04 Sanya, China, 22 Mar 2004 Aspectual (temporal) classification ? Vendler’s (1967) verb classes ? activities: run, swim, sleep, cry ? statives: love, hate, know ? achievements: arrive, win, find, die ? accomplishments: build (a house), write (a book) ? Stede’s (1996) MOOSE ontology ? Formal ontologies DOLCE, SUMO, and CYC assume traditional aspectual (temporal) classification for events
- 10. IJCNLP-04 Sanya, China, 22 Mar 2004 Aspectual (temporal) classification ? Vendler’s (1967) verb classes ? Stede’s (1996) ontology of MOOSE ? Formal ontologies DOLCE, SUMO, and CYC assume traditional aspectual (temporal) classification for events situation state event activity protracted activity moment activity culmination transition protracted culmination moment culmination
- 11. IJCNLP-04 Sanya, China, 22 Mar 2004 Aspectual (temporal) classification ? Vendler’s (1967) verb classes ? Stede’s (1996) ontology of MOOSE ? Formal ontologies DOLCE, SUMO, and CYC assume traditional aspectual (temporal) classification for events situation state event activity protracted activity moment activity culmination transition protracted culmination moment culmination
- 12. IJCNLP-04 Sanya, China, 22 Mar 2004 Semantic verb classes in WordNet Lexicographer file Contents verb.body grooming, dressing, bodily care verb.change size, temperature change verb.cognition thinking, judging, doubting verb.communication telling, asking, ordering, singing verb.competition fighting, athletic activities verb.consumption eating and drinking verb.contact touching, hitting, tying, digging verb.creation sewing, baking, painting verb.emotion feeling verb.motion walking, flying, swimming verb.perception seeing, hearing, feeling verb.possession buying, selling, owning verb.social political/social activities ? Taxonomic approach based on pure lexical semantics ? Reveal semantic organisation of lexicon in terms of lexical & semantic relations ? Top nodes of WordNet’s verb file
- 13. IJCNLP-04 Sanya, China, 22 Mar 2004 Levin’s (1993) verb classes ? Theoretic ground -- semantic/syntactic correlations: verbs with similar meaning (identical LCSs in terms of specific meaning components) show same syntactic behaviors Verbs of motion Verbs of inherently directed motion: arrive, come, enter Leave verbs: leave, abandon, desert Manner of motion verbs: roll, run, sneak, waddle Verbs of motion using a vehicle: bike, drive, fly Chase verbs: chase, follow, track Accompany verbs: accompany, escort, guide Waltz verbs: clog, polka, waltz
- 14. IJCNLP-04 Sanya, China, 22 Mar 2004 Dimension of causation ? Asher and Lascarides’ (1995) dimension of causation-change ? causation and change are specified along four dimensions: locative, formal, matter, intentional locative cause formal matter change intentional loc-change obj of put intent-cause sub of amuse fml-cause sub of build mtr-cause sub of paint fml-change obj of build mtr-change obj of paint intent-change obj of amuse loc-cause sub of put
- 15. IJCNLP-04 Sanya, China, 22 Mar 2004 Visual semantics & verb classes Visual factors concerning verb categorisation ? Visual valency ? Somatotopic factors in visualisation ? Level-of-detail of visual information Verbs belonging to same class in the classification ? Visual “synonyms” ? Substitutable in same set of animation keyframes Visualisation of action verbs is effective evaluation of the classification
- 16. IJCNLP-04 Sanya, China, 22 Mar 2004 Visual valency ? Capacity of verb to take specific number and type of visual arguments in language visualisation (3D animation) ? valency filler -- visual role ? 2 types of visual roles requiring different processes in visualisation ? human (biped articulated animate entity) ? object (inanimate entity) ? Visual valency overlaps with syntactic & semantic valency ? Visual modality requires more obligatory roles than surface grammar or semantics
- 17. IJCNLP-04 Sanya, China, 22 Mar 2004 Somatotopic effectors of action verbs ? Theoretical ground: execution/perception/visualisation of action verbs produced by same somatotopic effector activate same parts of cortex ? Distinguish facial expression (e.g. lip movement) & body posture (arm/leg/torso) in our ontological system ? Further divisions like distinction between upper/lower arm, hands, & even fingers are possible torso – bow facial expression – sing, laugh body posture Leg – run, kick Arm – wave, put
- 18. IJCNLP-04 Sanya, China, 22 Mar 2004 Level-Of-Detail (LOD) basic-level verbs & troponyms EVENT go run cause … event level verbs walk climb jump manner level verbs limp stride swagger trot … skip bounce hop jog romp troponym level verbs Levels Verbs Basic-level verb for ... event level go, move atomic object manner level walk, jump human/non-atomic object troponym level limp, stride human
- 19. IJCNLP-04 Sanya, China, 22 Mar 2004 CONFUCIUS’ verb taxonomy 2.2.1. Action verbs 2.2.1.1. One visual valency (the role is a human, (partial) movement) 2.2.1.1.1. Biped kinematics: arm actions (wave, scratch), leg actions (walk, jump, kick), torso actions (bow), combined actions (climb) 2.2.1.1.2. Facial expressions & lip movement, e.g. laugh, fear, say, sing, order 2.2.1.2. Two visual valency (at least one role is human) 2.2.1.2.1. One human and one object (vt. or vi.+instrument) e.g. throw, push, kick, open, eat, drink, bake, trolley 2.2.1.2.2. Two humans, e.g. fight, chase, guide 2.2.1.3. Visual valency ≥ 3 (at least one role is human) 2.2.1.3.1. Two humans and one object (inc. ditransitive verbs), e.g. give, show 2.2.1.3.2. One human and 2+ objects (vt. + object + implicit instr./goal/theme) e.g. cut, write, butter, pocket, dig, cook 2.2.1.4. Verbs without distinct visualisation when out of context: verbs of trying, helping, letting, creating/destroying 2.2.1.5. High level behaviours (routine events), political and social activities e.g. interview, eat out (go to restaurant), go shopping
- 20. IJCNLP-04 Sanya, China, 22 Mar 2004 Text-to-Animation of single sentences ? Collision detection example (contact verbs: hit, collide, scratch, touch), no human role involved “The car collided with a wall.” ? using ParallelGraphics’ VRML extension--object-to-object collision ? non-speech sound effects ? H-Anim examples: action verbs 3 visual valency verbs “John gave Nancy a loaf of bread.” “John put a cup of coffee on the table.” ? H-Anim Site node ? locative tags of object (on_table tag for table object) 2 visual valency verbs “John pushed the door.” “John ate the bread.” “Nancy sat on the chair.” For more demos, please visit http://www.infm.ulst.ac.uk/~eunice/3D_anim.html
- 21. IJCNLP-04 Sanya, China, 22 Mar 2004 Relation to other work ? Categorise verbs from visual semantics perspective ? Language visualisation in CONFUCIUS provides independent criteria for identifying classes of verbs sharing certain aspects of meaning, i.e. semantic/visual correlations ? Relation to Levin’s verb classes: Verbs of cutting 2.2.1.3.2, visual valency=3 2.1.2, visual valency=2 1 to N 2.2.1.3.1 visual valency=3 Verbs of sending & carrying Verbs of change of possession “Carol cut the whole wheat bread.” “Whole wheat bread cuts easily.” “Nancy brought the book to John.” “Nancy gave the book to John.” N to 1
- 22. IJCNLP-04 Sanya, China, 22 Mar 2004 Conclusion & future work ? Categorise verbs from visual semantic perspective ? Provides independent criteria for identifying classes of verbs based on semantic/visual correlations ? Visual semantic analysis of eventive verbs revealed striking influences in taxonomic verb tree ? Various criteria ranging from visual valency, somatotopic effectors, to LOD are proposed ? Evaluation issues using automatic animation generation & psychological experiments ? Discourse level interpretation