際際滷

際際滷Share a Scribd company logo
semantic construction grammar

+
how can NIPS help with deep reading

michael @witbrock
WHATS THE POINT OF CYC
3-Stage Research Program for AI
1. Slowly hand-code a very large and very broad
KB.
2. When enough knowledge is present, the system
should actively help with the KA process. It
should be faster to acquire more from texts,
databases, [websites], interactive dialogues,
etc.
3. To go beyond the frontier of human knowledge,
the system will have to rely on learning by
discovery, to expand its KB domain by domain.

(Doug Lenat)
CYC KNOWLEDGE BASE
Euro
isa
Education
Mathematics
Planet
Vehicles
Cat
Thing
Learning
Physics
Working
Driving
Human
Time Tree
isa
Hatred
isa
isa
Cat
isa
Words
Chemistry
Nature
Money
Time
located in Emotions
Celestial Earth
Euro
Event
Universe
Body
Stabbing someone Learning
Animal
Fear
Event
Vehicles
Words
subclass
Food
Euro
Tree Physics
Driving
subclas
School
Learning
Mathematics
Rain
Planet
Vehicles
Cat
s
Working
LanguageEducation
Driving
Time
Emotions
EventHuman for a
Going
isa Love
Rain
Education
Hatred
Words
walk
Chemistry
Fear
Money
Euro
Emotions Earth
Stabbing someone
Hatred
Going for a
School
Learning
Fear
Vehicles
walk
Food
Education
Driving
Event
Rain
Stabbing someone
Hatred
Language
Emotions
Going for a
Love
Fear
walk
Nature Physics
Death
Tree School

Death
Thing
Intangible Individual
Thing
Sets
Relations

Space
Physical
Objects
Living
Things
Ecology
Natural
Geography
Political
Geography
Weather
Earth &
Solar System

Human
Beings
Human
Artifacts

Human
Anatomy &
Physiology

Partially
Tangible
Thing

Time
Events
Scripts

Artifacts

Plans
Goals

Physical
Agents

Animals

Mechanical Software
Social
Language Relations,
& Electrical Literature
Devices
Works of Art
Culture

Organization

Organizational
Actions

Organizational
Plans

Agent
Organizations

Social
Behavior

Agents

Actors
Actions

Movement

State Change
Dynamics

Plants

Temporal
Thing

Logic
Math

Borders
Geometry

Emotion
Human
Products Conceptual
Perception Behavior &
Devices Works
Belief
Actions

Vehicles
Buildings
Weapons

Paths

Spatial
Paths

Materials
Parts
Statics

Life
Forms

Spatial
Thing

Social
Activities

Human
Activities

Business &
Commerce

Purchasing
Shopping

Types of
Organizations

Politics
Warfare

Sports
Recreation
Entertainment

Transportation
& Logistics

Human
Organizations
Nations
Governments
Geo-Politics

Professions
Occupations

Travel
Communication

Law

Everyday
Living

Business,
Military
Organizations

General Knowledge about Various Domains
Specific data, facts, and observations
Cycorp 息 2006
EVENT  TEMPORAL-THING  PARTIALLY-TANGIBLE-THING

Upper
Ontology

Core
Theories
Domain-Specific
Theories
Very specific information
(some indirect, via SKSI)

( a, b ) a  EVENT  b  EVENT 
causes( a, b )  precedes( a, b )

( m, a ) m  MAMMAL  a  ANTHRAX 
causes( exposed-to( m, a ), infected-by( m, a ) )



(ist FtLaudHolyCrossERCase#403921
(caused CutaneousAnthrax
(SkinLesions Ahmed_al-Haznawit)))

First Order Predicate Calculus: unambiguous; enable mechanical reasoning
Every NZr has a Queen.
Every NZr has a mother.

Higher Order Logic: contexts,
predicates as variables,
nested modals, reflection,
NZr(x)  ruler(x,y)&Queen(y)

y.x.
x.y. NZr(x)  mother(x,y)
First Order
(isa AVPR2-Human-GIS GeneTypeBySpeciesAndProductFamily)
(gene-GISTypeCodesForType AVPR2-Human-GIS VasopressinV2Receptor)

With Context
In Mt : VertebratePhysiologyMt
(relationExistsAll outputsCreated IntramembranousBoneGrowthAndDevelopment FlatBone)

Each vertebrate flat
bone was created by
intramembranous
bone development.

Rule
In Mt: MolecularBiologyMt
(implies
(isa ?MOLECULE-TYPE TranscriptionFactor)
(behaviorCapable ?MOLECULE-TYPE
(ChemicalBindingEventTypeWithTypesFn TranscriptionFactor DNAMolecule)
objectOfAttachment))

Transcription factors can
bind with DNA.

Exceptions
(implies (and (isa ?MUT GeneticMutationEvent-Cellular)
(locusOfCellularProcess-Cell ?MUT ?ANCESTOR)
(isa ?ANCESTOR Cell)
(subEvents ?REPRO ?MUT))
(abnormal (TheList ?REPRO ?PROGENY ?ANCESTOR)
(implies(and
(isa ?REPRO AsexualReproductionEvent)
(outputsCreated ?REPRO ?PROGENY)
(isa ?PROGENY BiologicalLivingObject)
(doneBy ?REPRO ?ANCESTOR))
(geneticallyIdentical ?ANCESTOR ?PROGENY))))

NIPS TASK 1:
DISTRIBUTED
REP OF THIS

Normally, the progeny of asexual
reproduction are genetically
identical to the parent; however,
if the parent is a cell in which a
mutation has occurred, this rule
doesnt apply.
#$TransportationEvent
#$ControllingATransportationDevice
#$TransportWithMotorizedLandVehicle
(#$SteeringFn #$RoadVehicle)
#$TransporterCrashEvent
#$VehicleAccident
#$CarAccident
#$Colliding
#$IncurringDamage
#$TippingOver
#$Navigating
#$EnteringAVehicle
Is it attached to the
inside of the outer object?
 Yes -- Try
#$connectedToInside

Can it be removed by pulling, if
enough force is used, without
damaging either object?
 No -- Try #$in-Snugly
or #$screwedIn
Does the inner object
stick into the outer object?
Yes  Try
#$sticksInto

Cycorp 息 2007


Does part of the inner object
stick out of the container?
 None of it.
#$in-ContCompletely
 Yes
#$in-ContPartially
 No

#$in-ContClosed

 If the container were
turned around could
the contained object
fall out?

Yes
#$in-ContOpen

NIPS TASK 2:
LEARN CONCEPTS
THIS FINELY
DISTINGUISED
NIPS TASK 3: LEARN TO
PRODUCE COHERENT NL
FROM REPRESENTATIONS

10

incyc.cyc.com
Semantic Construction Grammar
representational context: Cyc vocabulary and syntax
a sad realisation and an opportunity
logical representations good for inference are
sometimes very far from natural language
 in unpredictable ways
 and they need to be right in ways NL does not
So, syntactic mapping is (pretty much) hopeless
BUT, storage is pretty much free,
 and inference is getting there
 and ILP works at least some of the time
(EBMT  FrameNet  Cyc)  SCG
12
Renaissance Artists

Bronze Age Farmers

(SubcollectionOfWithRelationToFn
Artist activeDuringPeriod
TheRenaissance)
(SubcollectionOfWithRelationToFn
Farmer activeDuringPeriod
TheBronzeAge)

Kind of TimeInterval
Noun Form: not plural

Kind of Agent-Generic
Noun form
Attributive Adjective & Noun
(SubcollectionWithRelationToFn
Submarine
Yellow Submarine
mainColorOfObject
Yellow)
(SubcollectionWithRelationToFn
Sick Child
HumanChild
stateOfHealth
Sick)
Supporting Lexical Assertions:
(adjSemTrans Yellow-TheWord 0 RegularAdjFrame
(mainColorOfObject :NOUN YellowColor))
(adjSemTrans Sick-TheWord 0 RegularAdjFrame
(stateOfHealth :NOUN Sick))
NIPS TASK 1 again

15
(StuffFn Mushroom-Broadly)
FeedingPromotingSomething

Acre

a man

Mushroom-Broadly

FeedingEvent

feeds

a

Pepperoni

dog

AdultHumanMale

Dog

Man-TheProgram

pepperoni

EdibleMushroom

and

mushroom

pizza

HotDog

Pizza

ControllingSomething

16
17
NIPS TASK 5: evaluate (subtype ?X ?Y) in detail
50 Generalisations:

75 Generalisations:

Agent-Underspecified BiologicalLivingObject BoundaryUnderspecified EnduringThing-Localized

Agent-Underspecified Artifact-Generic BiologicalLivingObject BoundaryUnderspecified BusinessRelatedThing Container-Underspecified
ContainerIndependentShapedThing CulturalThing EdibleMushroom

EukaryoticOrganism

FiniteSpatialThing
FixedFunctionalSystem FunctionalSystem Fungus FungusBLO
Heterotroph Individual Landmark-Underspecified LocationUnderspecified Mushroom-Broadly NaturalTangibleStuff Opaque
OrganicMaterial Organism-Whole PartiallyTangible
PolyDimensionalThing PositiveDimensionalThing RegionUnderspecified SomethingExisting SpatialThing SpatialThingLocalized SpatialThing-NonSituational SpatiallyBoundedThing
SpatiallyContinuousThing SurfaceRegion-Underspecified System
TemporalThing TemporallyContinuousThing
TemporallyExistingThing Thing ThreeDimensionalThing TrajectorUnderspecified TrueFungus (CollectionDifferenceFn
SpatialThing Situation) (CollectionUnionFn (TheSet
BiologicalLivingObject FictionalCharacter)) (CollectionUnionFn
(TheSet BiologicalLivingObject (GroupFn BiologicalLivingObject)))
(CollectionUnionFn (TheSet Fungus Bacterium)) (CollectionUnionFn
(TheSet Organism-Whole Cell)) (CollectionUnionFn (TheSet
SpatialThing VectorInterval)) (CollectionUnionFn (TheSet
TemporalThing Collection)) (MeaningInSystemFn SENSUSInformation1997 "DECOMPOSABLE-OBJECT")

Mushroom-Broadly

EdibleStuff

EnduringThing-Localized EukaryoticOrganism
FiniteSpatialThing FixedFunctionalSystem Food FoodOrDrink

FoodOrDrinkOrIngredient

FunctionalSystem Fungus
FungusBLO GroceryMarketCategory Heterotroph HumanScaleObject
Individual Landmark-Underspecified Location-Underspecified MushroomBroadly NaturalTangibleStuff NonFluidlikeTangibleThing Opaque
OrganicMaterial Organism-Whole PartiallyTangible PartiallyTangibleProduct
PolyDimensionalThing PositiveDimensionalThing Product RegionUnderspecified SolidFood SolidTangibleThing SomethingExisting
SpatialThing SpatialThing-Localized SpatialThing-NonSituational
SpatiallyBoundedThing SpatiallyContinuousThing SurfaceRegionUnderspecified System TemporalThing TemporallyContinuousThing
TemporallyExistingThing Thing ThreeDimensionalThing TrajectorUnderspecified TrueFungus (CollectionDifferenceFn SomethingExisting
Organization) (CollectionDifferenceFn SpatialThing Situation)
(CollectionUnionFn (TheSet Action Artifact-Generic PropositionalConceptualWork))
(CollectionUnionFn (TheSet BiologicalLivingObject FictionalCharacter))
(CollectionUnionFn (TheSet BiologicalLivingObject (GroupFn
BiologicalLivingObject))) (CollectionUnionFn (TheSet DurableGood ServiceEvent
Product)) (CollectionUnionFn (TheSet FirstOrderCollection Artifact-Generic))
(CollectionUnionFn (TheSet Fungus Bacterium)) (CollectionUnionFn (TheSet
Organism-Whole Cell)) (CollectionUnionFn (TheSet SolidTangibleThing
LiquidTangibleThing)) (CollectionUnionFn (TheSet SpatialThing VectorInterval))
(CollectionUnionFn (TheSet TemporalThing Collection)) (EdibleByFn
HomoSapiens) (MeaningInSystemFn SENSUS-Information1997
"DECOMPOSABLE-OBJECT") (MeaningInSystemFn SENSUS-Information1997
"NONDECOMPOSABLE-OBJECT") (MeaningInSystemFn SENSUSInformation1997 "SEPARABLE-ENTITY") 

EdibleMushroom

18
110 Generalisations, including:

68 Generalisations, including:

AboveGround AerobicBLO AerobicOrganism Agent-Generic
Agent-NonArtifactual Agent-NonGeographical AgentPartiallyTangible Agent-Underspecified AirBreathingVertebrate

Agent-Underspecified Artifact Artifact-Generic ArtifactNonAgentive Boundary-Underspecified BusinessRelatedThing
Container-Underspecified ContainerIndependentShapedThing

Animal

Amniote
AnimalBLO AxisymmetricObject
BilateralObject BilaterallySymmetricObject BiologicalLivingObject
Boundary-Underspecified CanineAnimal CanisGenus Carnivore
CarnivoreOrder CellularTangibleThing ChordataPhylum
Coelomate CompositeTangibleAndIntangibleObject ContainerUnderspecified ContainerIndependentShapedThing
DiploidOrganism Diplont Dog DomesticatedAnimal
EmbodiedAgent EnduringThing-Localized EukaryoticOrganism
Eutheria FiniteSpatialThing FixedFunctionalSystem
FrontAndBackSidedObject FunctionalSystem Heterotroph
HexalateralObject Homeotherm HumanScaleObject Individual
IndividualAgent Landmark-Underspecified
LeftAndRightSidedObject Location-Underspecified Mammal
MulticellularOrganism NaturalTangibleStuff
NonFluidlikeTangibleThing NonHumanAnimal
NonPersonAnimal Opaque OrganicMaterial Organism-Whole
PartiallyIntangible PartiallyIntangibleIndividual PartiallyTangible
PerceptualAgent PerceptualAgent-Embodied
PolyDimensionalThing PositiveDimensionalThing Quadruped
Region-Underspecified Sentient SentientAnimal
SolidTangibleThing SomethingExisting SpatialThing SpatialThingLocalized SpatialThing-NonSituational SpatiallyBoundedThing
SpatiallyContinuousThing SurfaceRegion-Underspecified System
TameAnimal TemporalThing TemporallyContinuousThing
TemporallyExistingThing TerrestrialFunctioningObject
TerrestrialOrganism Thing ThreeDimensionalThing
TopAndBottomSidedObject Trajector-Underspecified Vertebrate
ViviparousAnimal 

CulturalThing

EdibleStuff

EdibleStuffComposite

EnduringThing-Localized FiniteSpatialThing Food
FoodComposite FoodOrDrink FoodOrDrinkOrIngredient
GroceryMarketCategory HotDog HumanScaleObject
InanimateObject InanimateObject-NonNatural Individual
InertArtifact Landmark-Underspecified Location-Underspecified
NonFluidlikeTangibleThing NonNaturalThing Opaque
OrganicMaterial PartiallyTangible PartiallyTangibleProduct
PolyDimensionalThing PositiveDimensionalThing PreparedFood
Product Region-Underspecified Sausage SolidFood
SolidFoodArtifact SolidTangibleArtifact SolidTangibleThing
SomethingExisting SpatialThing SpatialThing-Localized
SpatialThing-NonSituational SurfaceRegion-Underspecified
TemporalThing TemporallyExistingThing Thing
ThreeDimensionalThing Trajector-Underspecified
(CollectionDifferenceFn SomethingExisting Organization)
(CollectionDifferenceFn SpatialThing Situation)
(CollectionDifferenceFn TemporalThing BiologicalLivingObject)
(CollectionUnionFn (TheSet Action Artifact-Generic
PropositionalConceptualWork)) (CollectionUnionFn (TheSet
DurableGood ServiceEvent Product)) (CollectionUnionFn (TheSet
Event Artifact)) 

Dog

HotDog
19
$EdibleStuff#0 and $EdibleStuff#1 pizza
Logic: (#$equalSymbols ?PIZZA
(#$PizzaFn (#$TheSet $EdibleStuff#1 $EdibleStuff#0) ))
Var: ?PIZZA Type: #$Pizza

20
a $IntelligentAgent#0 feeds a $Animal#1 $EdibleStuff#2
Logic: (#$equalSymbols ?EVT (#$CollectionSubsetFn #$FeedingEvent (#$TheSetOf ?EVENT
(#$thereExists ?FEEDER (#$thereExists ?FED (#$thereExists ?FOOD (#$and (#$isa ?FOOD
$EdibleStuff#2) (#$consumedObject ?EVENT ?FOOD) (#$isa ?FED $Animal#1)
(#$recipientOfService ?EVENT ?FED) (#$isa ?FEEDER $IntelligentAgent#0) (#$providerOfService
?EVENT ?FEEDER))))))))
Var: ?EVT Type: #$FeedingEvent

21
Vision for KB-based Protection
(implies
(and
(programSubsumesInstruction ?PROG ?ALLOC-CALL)
(codingFunctionInvoked ?ALLOC-CALL ?ALLOC-FN)
(isa ?ALLOC-FN MemoryAllocationFunction)
(methodCallReturn ?ALLOC-CALL ?ALLOC-VAL)
(candidateNullValueInputTo ?ALLOC-VAL ?ADD-OP)
(programSubsumesInstruction ?PROG ?ADD-OP)
(arithmeticFunctionOfOperation ?ADD-OP PlusFn)
(oPHistOutputsToArgumentPosition
?ADD-OP ?ALLOC-VAL ?ARGN ?WRITE-OP)
(programSubsumesInstruction ?PROG ?WRITE-OP)
Induce high- (codingFunctionInvoked ?WRITE-OP ?WRITE-FN)
(isa ?WRITE-FN DataLocationAlteringFunction)
level
(mutableArgumentPosition ?WRITE-FN ?ARGN))
heuristics (programHasPropensityForActionOfTypeDueToStatement
?PROG ?WRITE-OP OutOfRangePointerOffsetWrite))

high-level syntax heuristic

Safely
Execute

Code
Analysis
DB

Cyc KB
and Inference

low-level heuristic

Connect
data

Specialize

Use reading
+ KA to
script
scripted attack
attack
profile
profiles
Cyc and Semantic Construction Grammar NIPS 2013 KET Workshop
Reading knowledge about a code pattern
Describes
weak code
Understood code description is directly
usable with SPICE analyzed binaries

26
Logical Description knows how to use the data

NIPS TASK 6: SUPPORT THIS
LEVEL (or more) OF INFERENCE
Part of read spec

27
Read description of weakness allows its ID in code
& human-understandable justification

28
Inference During Interpretation

Plausibility reasoning

29
9 Shades of Fail
Term interpretation fails lenient CycL truth test (WFF)
Arg required to be a collection but is not
Arg not an instance of all argument type constraints (strict)
Arg provably disjoint with a constraint (collections)
Arg is provably not-isa or not-genl a constraint (everything)
Argument is probably bad since it conflicts with implicit
argument constraints via #$relationNotExistsExists,
#$relationAllExists, or
#$someTypePlaysRoleInSituationType KB knowledge
Volume mismatch between types using
#$typicallyMoreVoluminousThan KB knowledge
Esoteric
Unlikely
30
Argument is probably bad since it conflicts with implicit
argument constraints via #$relationNotExistsExists,
#$relationAllExists, or
#$someTypePlaysRoleInSituationType KB knowledge

(#$SubcollectionOfWithRelationToTypeFn #$Fist
#$properPhysicalParts #$EthnicGroupOfRussians)
(relationAllExists #$Fist #$properPhysicalParts
#$AnimalBodyPart)
31
Volume mismatch between types
Can horses jump?: by analogy with Tahoe bars.
(#$SubcollectionOfWithRelationToTypeFn
#$SawHorse #$objectFoundInLocation #$Can)
This interpretation is blocked if we can prove:
(#$typicallyMoreVoluminousThan #$SawHorse #$Can)

32
Plausibility: Flat sheets of paper
Sheets that are part of an apartment unit that are part of some paper
(#$sentencePlausibilityScore
(#$Quote
(#$equalSymbols ?X
(#$SubcollectionOfWithRelationFromTypeFn
(#$SubcollectionOfWithRelationFromTypeFn #$BedSheet
#$physicalParts #$ApartmentUnit) #$physicalParts #$Paper)))
?SCORE) in #$PlausibilityQueryMt

?SCORE: (#$NumericLikelihoodFn 0.08)

33
Plausibility: Play chamber
music
(#$SubcollectionOfWithRelationToTypeFn
#$PlayingAMusicalInstrument #$objectActedOn #$ChamberMusic)
one can either call why-sentence-plausibility-factor:
(why-sentence-plausibility-factor
(#$SubcollectionOfWithRelationToTypeFn
#$PlayingAMusicalInstrument #$objectActedOn #$ChamberMusic))

0.06
(# # (:GENLS (#$genls #$PlayingAMusicalInstrument #$PhysicalEvent)
#$PlausibilityQueryMt :TRUE-DEF) (:DISJOINTWITH (#$disjointWith
#$MusicalInstrument #$ChamberMusicComposition)
#$PlausibilityQueryMt :TRUE-DEF))

34
35







Primacy of Semantics
Importance of mapping not to
logical form, but to logic
The frightening complexity of
human level knowledge
Importance of doing inference
during understanding
The time is now/nigh for uniting
the threads of AI

end
36
37

More Related Content

Cyc and Semantic Construction Grammar NIPS 2013 KET Workshop

  • 1. semantic construction grammar + how can NIPS help with deep reading michael @witbrock
  • 2. WHATS THE POINT OF CYC 3-Stage Research Program for AI 1. Slowly hand-code a very large and very broad KB. 2. When enough knowledge is present, the system should actively help with the KA process. It should be faster to acquire more from texts, databases, [websites], interactive dialogues, etc. 3. To go beyond the frontier of human knowledge, the system will have to rely on learning by discovery, to expand its KB domain by domain. (Doug Lenat)
  • 3. CYC KNOWLEDGE BASE Euro isa Education Mathematics Planet Vehicles Cat Thing Learning Physics Working Driving Human Time Tree isa Hatred isa isa Cat isa Words Chemistry Nature Money Time located in Emotions Celestial Earth Euro Event Universe Body Stabbing someone Learning Animal Fear Event Vehicles Words subclass Food Euro Tree Physics Driving subclas School Learning Mathematics Rain Planet Vehicles Cat s Working LanguageEducation Driving Time Emotions EventHuman for a Going isa Love Rain Education Hatred Words walk Chemistry Fear Money Euro Emotions Earth Stabbing someone Hatred Going for a School Learning Fear Vehicles walk Food Education Driving Event Rain Stabbing someone Hatred Language Emotions Going for a Love Fear walk Nature Physics Death Tree School Death
  • 4. Thing Intangible Individual Thing Sets Relations Space Physical Objects Living Things Ecology Natural Geography Political Geography Weather Earth & Solar System Human Beings Human Artifacts Human Anatomy & Physiology Partially Tangible Thing Time Events Scripts Artifacts Plans Goals Physical Agents Animals Mechanical Software Social Language Relations, & Electrical Literature Devices Works of Art Culture Organization Organizational Actions Organizational Plans Agent Organizations Social Behavior Agents Actors Actions Movement State Change Dynamics Plants Temporal Thing Logic Math Borders Geometry Emotion Human Products Conceptual Perception Behavior & Devices Works Belief Actions Vehicles Buildings Weapons Paths Spatial Paths Materials Parts Statics Life Forms Spatial Thing Social Activities Human Activities Business & Commerce Purchasing Shopping Types of Organizations Politics Warfare Sports Recreation Entertainment Transportation & Logistics Human Organizations Nations Governments Geo-Politics Professions Occupations Travel Communication Law Everyday Living Business, Military Organizations General Knowledge about Various Domains Specific data, facts, and observations Cycorp 息 2006
  • 5. EVENT TEMPORAL-THING PARTIALLY-TANGIBLE-THING Upper Ontology Core Theories Domain-Specific Theories Very specific information (some indirect, via SKSI) ( a, b ) a EVENT b EVENT causes( a, b ) precedes( a, b ) ( m, a ) m MAMMAL a ANTHRAX causes( exposed-to( m, a ), infected-by( m, a ) ) (ist FtLaudHolyCrossERCase#403921 (caused CutaneousAnthrax (SkinLesions Ahmed_al-Haznawit))) First Order Predicate Calculus: unambiguous; enable mechanical reasoning Every NZr has a Queen. Every NZr has a mother. Higher Order Logic: contexts, predicates as variables, nested modals, reflection, NZr(x) ruler(x,y)&Queen(y) y.x. x.y. NZr(x) mother(x,y)
  • 6. First Order (isa AVPR2-Human-GIS GeneTypeBySpeciesAndProductFamily) (gene-GISTypeCodesForType AVPR2-Human-GIS VasopressinV2Receptor) With Context In Mt : VertebratePhysiologyMt (relationExistsAll outputsCreated IntramembranousBoneGrowthAndDevelopment FlatBone) Each vertebrate flat bone was created by intramembranous bone development. Rule In Mt: MolecularBiologyMt (implies (isa ?MOLECULE-TYPE TranscriptionFactor) (behaviorCapable ?MOLECULE-TYPE (ChemicalBindingEventTypeWithTypesFn TranscriptionFactor DNAMolecule) objectOfAttachment)) Transcription factors can bind with DNA. Exceptions (implies (and (isa ?MUT GeneticMutationEvent-Cellular) (locusOfCellularProcess-Cell ?MUT ?ANCESTOR) (isa ?ANCESTOR Cell) (subEvents ?REPRO ?MUT)) (abnormal (TheList ?REPRO ?PROGENY ?ANCESTOR) (implies(and (isa ?REPRO AsexualReproductionEvent) (outputsCreated ?REPRO ?PROGENY) (isa ?PROGENY BiologicalLivingObject) (doneBy ?REPRO ?ANCESTOR)) (geneticallyIdentical ?ANCESTOR ?PROGENY)))) NIPS TASK 1: DISTRIBUTED REP OF THIS Normally, the progeny of asexual reproduction are genetically identical to the parent; however, if the parent is a cell in which a mutation has occurred, this rule doesnt apply.
  • 8. Is it attached to the inside of the outer object? Yes -- Try #$connectedToInside Can it be removed by pulling, if enough force is used, without damaging either object? No -- Try #$in-Snugly or #$screwedIn Does the inner object stick into the outer object? Yes Try #$sticksInto Cycorp 息 2007
  • 9. Does part of the inner object stick out of the container? None of it. #$in-ContCompletely Yes #$in-ContPartially No #$in-ContClosed If the container were turned around could the contained object fall out? Yes #$in-ContOpen NIPS TASK 2: LEARN CONCEPTS THIS FINELY DISTINGUISED
  • 10. NIPS TASK 3: LEARN TO PRODUCE COHERENT NL FROM REPRESENTATIONS 10 incyc.cyc.com
  • 11. Semantic Construction Grammar representational context: Cyc vocabulary and syntax
  • 12. a sad realisation and an opportunity logical representations good for inference are sometimes very far from natural language in unpredictable ways and they need to be right in ways NL does not So, syntactic mapping is (pretty much) hopeless BUT, storage is pretty much free, and inference is getting there and ILP works at least some of the time (EBMT FrameNet Cyc) SCG 12
  • 13. Renaissance Artists Bronze Age Farmers (SubcollectionOfWithRelationToFn Artist activeDuringPeriod TheRenaissance) (SubcollectionOfWithRelationToFn Farmer activeDuringPeriod TheBronzeAge) Kind of TimeInterval Noun Form: not plural Kind of Agent-Generic Noun form
  • 14. Attributive Adjective & Noun (SubcollectionWithRelationToFn Submarine Yellow Submarine mainColorOfObject Yellow) (SubcollectionWithRelationToFn Sick Child HumanChild stateOfHealth Sick) Supporting Lexical Assertions: (adjSemTrans Yellow-TheWord 0 RegularAdjFrame (mainColorOfObject :NOUN YellowColor)) (adjSemTrans Sick-TheWord 0 RegularAdjFrame (stateOfHealth :NOUN Sick))
  • 15. NIPS TASK 1 again 15
  • 17. 17
  • 18. NIPS TASK 5: evaluate (subtype ?X ?Y) in detail 50 Generalisations: 75 Generalisations: Agent-Underspecified BiologicalLivingObject BoundaryUnderspecified EnduringThing-Localized Agent-Underspecified Artifact-Generic BiologicalLivingObject BoundaryUnderspecified BusinessRelatedThing Container-Underspecified ContainerIndependentShapedThing CulturalThing EdibleMushroom EukaryoticOrganism FiniteSpatialThing FixedFunctionalSystem FunctionalSystem Fungus FungusBLO Heterotroph Individual Landmark-Underspecified LocationUnderspecified Mushroom-Broadly NaturalTangibleStuff Opaque OrganicMaterial Organism-Whole PartiallyTangible PolyDimensionalThing PositiveDimensionalThing RegionUnderspecified SomethingExisting SpatialThing SpatialThingLocalized SpatialThing-NonSituational SpatiallyBoundedThing SpatiallyContinuousThing SurfaceRegion-Underspecified System TemporalThing TemporallyContinuousThing TemporallyExistingThing Thing ThreeDimensionalThing TrajectorUnderspecified TrueFungus (CollectionDifferenceFn SpatialThing Situation) (CollectionUnionFn (TheSet BiologicalLivingObject FictionalCharacter)) (CollectionUnionFn (TheSet BiologicalLivingObject (GroupFn BiologicalLivingObject))) (CollectionUnionFn (TheSet Fungus Bacterium)) (CollectionUnionFn (TheSet Organism-Whole Cell)) (CollectionUnionFn (TheSet SpatialThing VectorInterval)) (CollectionUnionFn (TheSet TemporalThing Collection)) (MeaningInSystemFn SENSUSInformation1997 "DECOMPOSABLE-OBJECT") Mushroom-Broadly EdibleStuff EnduringThing-Localized EukaryoticOrganism FiniteSpatialThing FixedFunctionalSystem Food FoodOrDrink FoodOrDrinkOrIngredient FunctionalSystem Fungus FungusBLO GroceryMarketCategory Heterotroph HumanScaleObject Individual Landmark-Underspecified Location-Underspecified MushroomBroadly NaturalTangibleStuff NonFluidlikeTangibleThing Opaque OrganicMaterial Organism-Whole PartiallyTangible PartiallyTangibleProduct PolyDimensionalThing PositiveDimensionalThing Product RegionUnderspecified SolidFood SolidTangibleThing SomethingExisting SpatialThing SpatialThing-Localized SpatialThing-NonSituational SpatiallyBoundedThing SpatiallyContinuousThing SurfaceRegionUnderspecified System TemporalThing TemporallyContinuousThing TemporallyExistingThing Thing ThreeDimensionalThing TrajectorUnderspecified TrueFungus (CollectionDifferenceFn SomethingExisting Organization) (CollectionDifferenceFn SpatialThing Situation) (CollectionUnionFn (TheSet Action Artifact-Generic PropositionalConceptualWork)) (CollectionUnionFn (TheSet BiologicalLivingObject FictionalCharacter)) (CollectionUnionFn (TheSet BiologicalLivingObject (GroupFn BiologicalLivingObject))) (CollectionUnionFn (TheSet DurableGood ServiceEvent Product)) (CollectionUnionFn (TheSet FirstOrderCollection Artifact-Generic)) (CollectionUnionFn (TheSet Fungus Bacterium)) (CollectionUnionFn (TheSet Organism-Whole Cell)) (CollectionUnionFn (TheSet SolidTangibleThing LiquidTangibleThing)) (CollectionUnionFn (TheSet SpatialThing VectorInterval)) (CollectionUnionFn (TheSet TemporalThing Collection)) (EdibleByFn HomoSapiens) (MeaningInSystemFn SENSUS-Information1997 "DECOMPOSABLE-OBJECT") (MeaningInSystemFn SENSUS-Information1997 "NONDECOMPOSABLE-OBJECT") (MeaningInSystemFn SENSUSInformation1997 "SEPARABLE-ENTITY") EdibleMushroom 18
  • 19. 110 Generalisations, including: 68 Generalisations, including: AboveGround AerobicBLO AerobicOrganism Agent-Generic Agent-NonArtifactual Agent-NonGeographical AgentPartiallyTangible Agent-Underspecified AirBreathingVertebrate Agent-Underspecified Artifact Artifact-Generic ArtifactNonAgentive Boundary-Underspecified BusinessRelatedThing Container-Underspecified ContainerIndependentShapedThing Animal Amniote AnimalBLO AxisymmetricObject BilateralObject BilaterallySymmetricObject BiologicalLivingObject Boundary-Underspecified CanineAnimal CanisGenus Carnivore CarnivoreOrder CellularTangibleThing ChordataPhylum Coelomate CompositeTangibleAndIntangibleObject ContainerUnderspecified ContainerIndependentShapedThing DiploidOrganism Diplont Dog DomesticatedAnimal EmbodiedAgent EnduringThing-Localized EukaryoticOrganism Eutheria FiniteSpatialThing FixedFunctionalSystem FrontAndBackSidedObject FunctionalSystem Heterotroph HexalateralObject Homeotherm HumanScaleObject Individual IndividualAgent Landmark-Underspecified LeftAndRightSidedObject Location-Underspecified Mammal MulticellularOrganism NaturalTangibleStuff NonFluidlikeTangibleThing NonHumanAnimal NonPersonAnimal Opaque OrganicMaterial Organism-Whole PartiallyIntangible PartiallyIntangibleIndividual PartiallyTangible PerceptualAgent PerceptualAgent-Embodied PolyDimensionalThing PositiveDimensionalThing Quadruped Region-Underspecified Sentient SentientAnimal SolidTangibleThing SomethingExisting SpatialThing SpatialThingLocalized SpatialThing-NonSituational SpatiallyBoundedThing SpatiallyContinuousThing SurfaceRegion-Underspecified System TameAnimal TemporalThing TemporallyContinuousThing TemporallyExistingThing TerrestrialFunctioningObject TerrestrialOrganism Thing ThreeDimensionalThing TopAndBottomSidedObject Trajector-Underspecified Vertebrate ViviparousAnimal CulturalThing EdibleStuff EdibleStuffComposite EnduringThing-Localized FiniteSpatialThing Food FoodComposite FoodOrDrink FoodOrDrinkOrIngredient GroceryMarketCategory HotDog HumanScaleObject InanimateObject InanimateObject-NonNatural Individual InertArtifact Landmark-Underspecified Location-Underspecified NonFluidlikeTangibleThing NonNaturalThing Opaque OrganicMaterial PartiallyTangible PartiallyTangibleProduct PolyDimensionalThing PositiveDimensionalThing PreparedFood Product Region-Underspecified Sausage SolidFood SolidFoodArtifact SolidTangibleArtifact SolidTangibleThing SomethingExisting SpatialThing SpatialThing-Localized SpatialThing-NonSituational SurfaceRegion-Underspecified TemporalThing TemporallyExistingThing Thing ThreeDimensionalThing Trajector-Underspecified (CollectionDifferenceFn SomethingExisting Organization) (CollectionDifferenceFn SpatialThing Situation) (CollectionDifferenceFn TemporalThing BiologicalLivingObject) (CollectionUnionFn (TheSet Action Artifact-Generic PropositionalConceptualWork)) (CollectionUnionFn (TheSet DurableGood ServiceEvent Product)) (CollectionUnionFn (TheSet Event Artifact)) Dog HotDog 19
  • 20. $EdibleStuff#0 and $EdibleStuff#1 pizza Logic: (#$equalSymbols ?PIZZA (#$PizzaFn (#$TheSet $EdibleStuff#1 $EdibleStuff#0) )) Var: ?PIZZA Type: #$Pizza 20
  • 21. a $IntelligentAgent#0 feeds a $Animal#1 $EdibleStuff#2 Logic: (#$equalSymbols ?EVT (#$CollectionSubsetFn #$FeedingEvent (#$TheSetOf ?EVENT (#$thereExists ?FEEDER (#$thereExists ?FED (#$thereExists ?FOOD (#$and (#$isa ?FOOD $EdibleStuff#2) (#$consumedObject ?EVENT ?FOOD) (#$isa ?FED $Animal#1) (#$recipientOfService ?EVENT ?FED) (#$isa ?FEEDER $IntelligentAgent#0) (#$providerOfService ?EVENT ?FEEDER)))))))) Var: ?EVT Type: #$FeedingEvent 21
  • 22. Vision for KB-based Protection (implies (and (programSubsumesInstruction ?PROG ?ALLOC-CALL) (codingFunctionInvoked ?ALLOC-CALL ?ALLOC-FN) (isa ?ALLOC-FN MemoryAllocationFunction) (methodCallReturn ?ALLOC-CALL ?ALLOC-VAL) (candidateNullValueInputTo ?ALLOC-VAL ?ADD-OP) (programSubsumesInstruction ?PROG ?ADD-OP) (arithmeticFunctionOfOperation ?ADD-OP PlusFn) (oPHistOutputsToArgumentPosition ?ADD-OP ?ALLOC-VAL ?ARGN ?WRITE-OP) (programSubsumesInstruction ?PROG ?WRITE-OP) Induce high- (codingFunctionInvoked ?WRITE-OP ?WRITE-FN) (isa ?WRITE-FN DataLocationAlteringFunction) level (mutableArgumentPosition ?WRITE-FN ?ARGN)) heuristics (programHasPropensityForActionOfTypeDueToStatement ?PROG ?WRITE-OP OutOfRangePointerOffsetWrite)) high-level syntax heuristic Safely Execute Code Analysis DB Cyc KB and Inference low-level heuristic Connect data Specialize Use reading + KA to script scripted attack attack profile profiles
  • 24. Reading knowledge about a code pattern
  • 26. Understood code description is directly usable with SPICE analyzed binaries 26
  • 27. Logical Description knows how to use the data NIPS TASK 6: SUPPORT THIS LEVEL (or more) OF INFERENCE Part of read spec 27
  • 28. Read description of weakness allows its ID in code & human-understandable justification 28
  • 30. 9 Shades of Fail Term interpretation fails lenient CycL truth test (WFF) Arg required to be a collection but is not Arg not an instance of all argument type constraints (strict) Arg provably disjoint with a constraint (collections) Arg is provably not-isa or not-genl a constraint (everything) Argument is probably bad since it conflicts with implicit argument constraints via #$relationNotExistsExists, #$relationAllExists, or #$someTypePlaysRoleInSituationType KB knowledge Volume mismatch between types using #$typicallyMoreVoluminousThan KB knowledge Esoteric Unlikely 30
  • 31. Argument is probably bad since it conflicts with implicit argument constraints via #$relationNotExistsExists, #$relationAllExists, or #$someTypePlaysRoleInSituationType KB knowledge (#$SubcollectionOfWithRelationToTypeFn #$Fist #$properPhysicalParts #$EthnicGroupOfRussians) (relationAllExists #$Fist #$properPhysicalParts #$AnimalBodyPart) 31
  • 32. Volume mismatch between types Can horses jump?: by analogy with Tahoe bars. (#$SubcollectionOfWithRelationToTypeFn #$SawHorse #$objectFoundInLocation #$Can) This interpretation is blocked if we can prove: (#$typicallyMoreVoluminousThan #$SawHorse #$Can) 32
  • 33. Plausibility: Flat sheets of paper Sheets that are part of an apartment unit that are part of some paper (#$sentencePlausibilityScore (#$Quote (#$equalSymbols ?X (#$SubcollectionOfWithRelationFromTypeFn (#$SubcollectionOfWithRelationFromTypeFn #$BedSheet #$physicalParts #$ApartmentUnit) #$physicalParts #$Paper))) ?SCORE) in #$PlausibilityQueryMt ?SCORE: (#$NumericLikelihoodFn 0.08) 33
  • 34. Plausibility: Play chamber music (#$SubcollectionOfWithRelationToTypeFn #$PlayingAMusicalInstrument #$objectActedOn #$ChamberMusic) one can either call why-sentence-plausibility-factor: (why-sentence-plausibility-factor (#$SubcollectionOfWithRelationToTypeFn #$PlayingAMusicalInstrument #$objectActedOn #$ChamberMusic)) 0.06 (# # (:GENLS (#$genls #$PlayingAMusicalInstrument #$PhysicalEvent) #$PlausibilityQueryMt :TRUE-DEF) (:DISJOINTWITH (#$disjointWith #$MusicalInstrument #$ChamberMusicComposition) #$PlausibilityQueryMt :TRUE-DEF)) 34
  • 35. 35
  • 36. Primacy of Semantics Importance of mapping not to logical form, but to logic The frightening complexity of human level knowledge Importance of doing inference during understanding The time is now/nigh for uniting the threads of AI end 36
  • 37. 37

Editor's Notes

  • #7: The representation language of Cyc, and the inference engine that draws conclusions from the KB content and data, is sufficiently powerful to express and reason about biological processes. Some examples of the representation of biological objects, relations, and processes are given here.
  • #11: These pages synthesised using natural language generation from an underlying logical representation.