際際滷

際際滷Share a Scribd company logo

[Paper review] neural production system

Seonghoon Jung
Seonghoon Jung

Paper Review of DeepMind's Neural Production System https://arxiv.org/abs/2103.01937

1 of 15
Download to read offline
Neural Production Systems 朱 襴觀 2021.03 Seong Hoon Jung hoondori@gmail.com https://arxiv.org/abs/2103.01937
螻(80s) AI Production System Production system for planning, reasoning, prediction at 1980s, Modeling of high-level congnition, mathematical problem solving Condition-Action Rules Rule conditions are matched to entities in working memory Match can trigger computational actions that update working memory or external actions on the outside world
Our contributions We develop E2E deep learning model that construct object- centric representations of entities in videos Operate on these entities with differentiable and thus learnable production rules Disentangle of Knowledge Symbolic AI
れ 襦襦 螳♀鍵 MNIST Transformation Entity 1) image 2) ops Rule 覲 蠏豺 ex) Rotate Right朱 襭一 るジ讓曙朱 90 蠍一語碁. Rule rotate right when ops == 1 image exist then produce rotate 90 right image
Slots and Sparse Rules Image ops Step 1. parsing input image into slot-based entities (slot-wise representation of entities[1]) Step 2. select {rule, primary slot} pair by attention => 殊 襭磯Г豺 => MNIST ops=1 rule_rot_right Step3 select a contextual slot given primary slot/rule => MNIST image Step 4 apply the selected rule based on primary and context slots => 殊 Rule => MNIST 覲 image [1] https://arxiv.org/abs/1909.10893
Algorithm in details , ex) video frame Time t M螳 entityれ representation Ex) 覓朱Μ 蟆 覓語 M=3 embedding N螳 Rule, ex) MNIST 覲 N=4 Learned rule embedding vector 襷豺 旧 rule when 企 Rule then 襦 world襯 覦蠑碁 action 蟆一
覓語襷 伎狩 hyper-parameter 1) 襭一 螳 N 2) 襭 (stage) K 3) MLP 蟲譟 4) Contextual slot 螳 Entityれ 蟲. (updated or initial) K 覯 蟇語 轟 襭一 伎 蟆 覦覲牛. 螳覲 襭 覯 貎朱Μ襦 螻, 覈 Entntiyれ key襦 伎 attension . 螳 豕螻讌譴 螳讌 轟 襭郁骸 轟 entity(primary)襯 . 轟 襭郁骸 轟 primary entit襯 貎朱Μ襦 螻, るジ entityれ key襦 伎 attention 螳 豕螻讌譴 螳讌 轟 contextual entity襯 . c.f) 覓語 磯殊 覲旧 螳 襭一 action with primary/contextual Action world レ 殊 entntiy 覲. 襭 覯 一危碁 企? WHEN THEN
Experiments : 一 Task M = 3 Stage K=1 N = 4 ( +, -, * , /) T = 1 蟲 [feature, label] = (0.4, 0.5, +) , 0.9 (0.3, 0.2, - ) , 0.1 0.4 + 0.5 = 0.9 0.3 - 0.2 = 0.1 0.1 * 0.5 = 0.05 Test 豌 一朱 螻壱 task ル 觜蟲 螳 => contextual entites 螳 operation => primary entity り 蠍語伎覃 讌襦 NPS 覦覯 一 讀
Experiments : MNIST 覲 M = 2 (ops螳 primary, image螳 contextual) N = 4 (RR, RL, TU, TD) Stage K=1 T = 1 旧 [feature, label] = (企語, RR), RR覲企語 Image to entity repr Rule action : 蠍一 覲 企語
覓殊牡 豢 覯豺 覓朱Μ 覯豺 : 覓願碓 覓殊牡螳 讌企 蟆暑 螳覯殊 覓殊牡 讌碁. When Object move Another lighter object exists Then lighter object will be pushed Agent() 蟆曙 覓殊牡襯 讌 (action=U/D/L/R) 覓殊牡 豢 手鍵覃伎 企 蟆 豢 覦襴讌(pushed) 蟯谿壱朱 覓殊牡螳 覓願 蟆曙 .
覓殊牡 豢 覯豺 旧 殊 X = (x1, a1, x2, a2,..xT, aT) Image embedding 3螳 覓殊牡襦 覿危 embedding 轟 覓殊牡襯 覩碁 embedding 螳覩 NPS襯 , 殊 State Transition Model Primary 覿豎 覦襴/覦襴 螳豌 Contexual 覩碁 覓殊牡 襭 ′ => 覦れ 覯襴
覓殊牡 豢 覯豺 襦 譬 Dense interaction GNN覲企 Sparse interaction NPS螳 一 襭一 螳覦 蟯るΜ 覯 K>1 rule 覃 Dense蟆 焔レ . 覩碁 step 豸 (襾 覩碁 襦 豸° ) 企語 螳 襭 讀螳
Atari Game X = (x1, a1, x2, a2,..xT, aT) 襦 譬 覩碁 step 豸 (襾 覩碁 襦 豸° ) れ entity螳 覲旧″ 覓朱Μ覯豺(覦)襦 語覩襦 襷 襭一 朱 焔 譬讌
Rule 螳語 煙 NPS 螳讌. Modular Each rule is atomic (can be added/modified/deleted independently) Abstract Abstract that can match to a wide range of patterns Allow for transfer learning across different environments Sparse Involve only a subset of entities 覦覃 GNN 蟯覯蟆 involve螳 Causal and asymmetric knowledge
Conclusion We learn rules from only low-level observables like images Learn Action-Condition World Model and Extrapolation of knowledge in the form of learned rule in Atari games 蠍一 world model State transition model in Atari game 覩誤螻, learned rule implicit蟆 牛 襭(襯 れ 覦襭)れ 覩誤 Human seems to exploit the inductive bias in the sparcity of rules ... very efficient For such problems, exploration become bottleneck Using rules as a source of behavioral priors can drive necessary exploration

More Related Content

[Paper review] neural production system

  • 1. Neural Production Systems 朱 襴觀 2021.03 Seong Hoon Jung hoondori@gmail.com https://arxiv.org/abs/2103.01937
  • 2. 螻(80s) AI Production System Production system for planning, reasoning, prediction at 1980s, Modeling of high-level congnition, mathematical problem solving Condition-Action Rules Rule conditions are matched to entities in working memory Match can trigger computational actions that update working memory or external actions on the outside world
  • 3. Our contributions We develop E2E deep learning model that construct object- centric representations of entities in videos Operate on these entities with differentiable and thus learnable production rules Disentangle of Knowledge Symbolic AI
  • 4. れ 襦襦 螳♀鍵 MNIST Transformation Entity 1) image 2) ops Rule 覲 蠏豺 ex) Rotate Right朱 襭一 るジ讓曙朱 90 蠍一語碁. Rule rotate right when ops == 1 image exist then produce rotate 90 right image
  • 5. Slots and Sparse Rules Image ops Step 1. parsing input image into slot-based entities (slot-wise representation of entities[1]) Step 2. select {rule, primary slot} pair by attention => 殊 襭磯Г豺 => MNIST ops=1 rule_rot_right Step3 select a contextual slot given primary slot/rule => MNIST image Step 4 apply the selected rule based on primary and context slots => 殊 Rule => MNIST 覲 image [1] https://arxiv.org/abs/1909.10893
  • 6. Algorithm in details , ex) video frame Time t M螳 entityれ representation Ex) 覓朱Μ 蟆 覓語 M=3 embedding N螳 Rule, ex) MNIST 覲 N=4 Learned rule embedding vector 襷豺 旧 rule when 企 Rule then 襦 world襯 覦蠑碁 action 蟆一
  • 7. 覓語襷 伎狩 hyper-parameter 1) 襭一 螳 N 2) 襭 (stage) K 3) MLP 蟲譟 4) Contextual slot 螳 Entityれ 蟲. (updated or initial) K 覯 蟇語 轟 襭一 伎 蟆 覦覲牛. 螳覲 襭 覯 貎朱Μ襦 螻, 覈 Entntiyれ key襦 伎 attension . 螳 豕螻讌譴 螳讌 轟 襭郁骸 轟 entity(primary)襯 . 轟 襭郁骸 轟 primary entit襯 貎朱Μ襦 螻, るジ entityれ key襦 伎 attention 螳 豕螻讌譴 螳讌 轟 contextual entity襯 . c.f) 覓語 磯殊 覲旧 螳 襭一 action with primary/contextual Action world レ 殊 entntiy 覲. 襭 覯 一危碁 企? WHEN THEN
  • 8. Experiments : 一 Task M = 3 Stage K=1 N = 4 ( +, -, * , /) T = 1 蟲 [feature, label] = (0.4, 0.5, +) , 0.9 (0.3, 0.2, - ) , 0.1 0.4 + 0.5 = 0.9 0.3 - 0.2 = 0.1 0.1 * 0.5 = 0.05 Test 豌 一朱 螻壱 task ル 觜蟲 螳 => contextual entites 螳 operation => primary entity り 蠍語伎覃 讌襦 NPS 覦覯 一 讀
  • 9. Experiments : MNIST 覲 M = 2 (ops螳 primary, image螳 contextual) N = 4 (RR, RL, TU, TD) Stage K=1 T = 1 旧 [feature, label] = (企語, RR), RR覲企語 Image to entity repr Rule action : 蠍一 覲 企語
  • 10. 覓殊牡 豢 覯豺 覓朱Μ 覯豺 : 覓願碓 覓殊牡螳 讌企 蟆暑 螳覯殊 覓殊牡 讌碁. When Object move Another lighter object exists Then lighter object will be pushed Agent() 蟆曙 覓殊牡襯 讌 (action=U/D/L/R) 覓殊牡 豢 手鍵覃伎 企 蟆 豢 覦襴讌(pushed) 蟯谿壱朱 覓殊牡螳 覓願 蟆曙 .
  • 11. 覓殊牡 豢 覯豺 旧 殊 X = (x1, a1, x2, a2,..xT, aT) Image embedding 3螳 覓殊牡襦 覿危 embedding 轟 覓殊牡襯 覩碁 embedding 螳覩 NPS襯 , 殊 State Transition Model Primary 覿豎 覦襴/覦襴 螳豌 Contexual 覩碁 覓殊牡 襭 ′ => 覦れ 覯襴
  • 12. 覓殊牡 豢 覯豺 襦 譬 Dense interaction GNN覲企 Sparse interaction NPS螳 一 襭一 螳覦 蟯るΜ 覯 K>1 rule 覃 Dense蟆 焔レ . 覩碁 step 豸 (襾 覩碁 襦 豸° ) 企語 螳 襭 讀螳
  • 13. Atari Game X = (x1, a1, x2, a2,..xT, aT) 襦 譬 覩碁 step 豸 (襾 覩碁 襦 豸° ) れ entity螳 覲旧″ 覓朱Μ覯豺(覦)襦 語覩襦 襷 襭一 朱 焔 譬讌
  • 14. Rule 螳語 煙 NPS 螳讌. Modular Each rule is atomic (can be added/modified/deleted independently) Abstract Abstract that can match to a wide range of patterns Allow for transfer learning across different environments Sparse Involve only a subset of entities 覦覃 GNN 蟯覯蟆 involve螳 Causal and asymmetric knowledge
  • 15. Conclusion We learn rules from only low-level observables like images Learn Action-Condition World Model and Extrapolation of knowledge in the form of learned rule in Atari games 蠍一 world model State transition model in Atari game 覩誤螻, learned rule implicit蟆 牛 襭(襯 れ 覦襭)れ 覩誤 Human seems to exploit the inductive bias in the sparcity of rules ... very efficient For such problems, exploration become bottleneck Using rules as a source of behavioral priors can drive necessary exploration