ºÝºÝߣ

ºÝºÝߣShare a Scribd company logo

Natural User Interfaces using YVision

?Download as PPTX, PDF?
?
Ant?o Almada
Ant?o Almada

For the past 10 years, YDreams has been researching natural user interfaces to allow more natural human-computer interaction beyond traditional interfaces. Their framework called YVision uses camera sensors and computer vision techniques to enable gesture-based and touchless interaction. It has a component-based architecture and uses behavior trees to model how virtual objects respond to input, supporting realistic simulations. YDreams has applied this framework to augmented reality, virtual characters, large-scale interactive installations, and interactive assistants.

1 of 26
Natural User Interfaces usingAnt¨®nioC?maraCEOAnt?o AlmadaDirector, Strategic Sofware Development
For the past ten years, YDreams has been researching and developing new ways for humans to communicate with machines.
The goal is to transcend traditional interfaces and create experiences that would allow a more natural interaction with computational systems.
We take inspiration from the natural everyday environments that surround us, the world of objects which can be touched, pulled and pushed, and which are imbued with meaning in an appropriate context.
What is YVision?A development framework for NUI applications.Developed in YDreams¡¯ R&D department.Integrates 10 years of experience on NUI apps.Used internaly by YDreams¡¯ production in more than 300 projects.
Camera-based interaction
Camera-based interactionVideo ¡°Virtual Garden¡± (http://www.youtube.com/watch?v=YCuPU6fVGCo)
Camera-based interactionVideo camera is a very rich sensor.Video is a stream of pictures.¡°A picture is worth a thousand words¡±.How to extract usefull information?ColorsShapesMovementsFacesSensor fusionGestures
Sensor data processing pipelineDetectionInterpretationBrokerSimulationCapture
Natural User InterfacesShort duration interactions.No time/willingness to learn new paradigms.Interaction must be based on prior knowledge.Shape, Color, Texture, etc.BehaviorNUI applications should support:Realistic real-time renderingPhysics simulation Artificial intelligence
Component-based architectureEach object is composed of several components.Components expose services.Services allow the change of state.Objects are structured into hierarchies.Objects have access to its ancestors services.Facilitates decoupling => extensibility.
Object BehaviorsNUI raises ¡°new¡± challenges for object behaviorsThe ¡°user¡± should not be a collection of ¡°cursors¡±.The ¡°user¡± is a blob (2D or 3D).Steering behaviors should be usedInterpolated paths (motion tweens) -> Path findingHard connections -> Flocking
Behavior TreeActions are encapsulated into behaviors.Behaviors can have children.Behaviors have access to services of the associated object and ancestor behaviors.Behaviors return on each iteration:RunningSuccededFailedStandard behaviorsSequenceStep SequenceParallelStocastic SelectorSuspendStatus Control
Behavior Trees¡°Behavior trees replace the often intangible growing mess of state transitions of finite state machines (FSMs) with a more restrictive but also more structured traversal defining approach.¡± BjoernKnaflahttp://bjoernknafla.com/introduction-to-behavior-trees
Behavior Tree of a bubble
Natural User Interfaces using YVision
Cases
Augmented Realityusing depth-sensing cameras
Video ¡°3D Augmented Reality experience with depth-sensing cameras¡± (http://www.vimeo.com/11160661)
ChameleonVirtual intelligent characters as tour guides in an augmented reality environment
Video ¡°VicTour, Virtual Interactive Character Tour Guide¡± (http://vimeo.com/22415896)
Audience Entertainment20,000 people playing at Lyon Festival of Lights
Video ¡°AE, Orange and 20,000 People play at Lyon Festival of Lights Dec 5 2009¡± (http://www.youtube.com/watch?v=U33BqjtRJ58)
SantanderInteractive Guest Assistants
Video ¡°The making-of a visitors center using advanced robotics and interactive interface design¡± (http://www.vimeo.com/11655898)
Free download athttp://www.yvision.com/

More Related Content

Natural User Interfaces using YVision

  • 1. Natural User Interfaces usingAnt¨®nioC?maraCEOAnt?o AlmadaDirector, Strategic Sofware Development
  • 2. For the past ten years, YDreams has been researching and developing new ways for humans to communicate with machines.
  • 3. The goal is to transcend traditional interfaces and create experiences that would allow a more natural interaction with computational systems.
  • 4. We take inspiration from the natural everyday environments that surround us, the world of objects which can be touched, pulled and pushed, and which are imbued with meaning in an appropriate context.
  • 5. What is YVision?A development framework for NUI applications.Developed in YDreams¡¯ R&D department.Integrates 10 years of experience on NUI apps.Used internaly by YDreams¡¯ production in more than 300 projects.
  • 7. Camera-based interactionVideo ¡°Virtual Garden¡± (http://www.youtube.com/watch?v=YCuPU6fVGCo)
  • 8. Camera-based interactionVideo camera is a very rich sensor.Video is a stream of pictures.¡°A picture is worth a thousand words¡±.How to extract usefull information?ColorsShapesMovementsFacesSensor fusionGestures
  • 9. Sensor data processing pipelineDetectionInterpretationBrokerSimulationCapture
  • 10. Natural User InterfacesShort duration interactions.No time/willingness to learn new paradigms.Interaction must be based on prior knowledge.Shape, Color, Texture, etc.BehaviorNUI applications should support:Realistic real-time renderingPhysics simulation Artificial intelligence
  • 11. Component-based architectureEach object is composed of several components.Components expose services.Services allow the change of state.Objects are structured into hierarchies.Objects have access to its ancestors services.Facilitates decoupling => extensibility.
  • 12. Object BehaviorsNUI raises ¡°new¡± challenges for object behaviorsThe ¡°user¡± should not be a collection of ¡°cursors¡±.The ¡°user¡± is a blob (2D or 3D).Steering behaviors should be usedInterpolated paths (motion tweens) -> Path findingHard connections -> Flocking
  • 13. Behavior TreeActions are encapsulated into behaviors.Behaviors can have children.Behaviors have access to services of the associated object and ancestor behaviors.Behaviors return on each iteration:RunningSuccededFailedStandard behaviorsSequenceStep SequenceParallelStocastic SelectorSuspendStatus Control
  • 14. Behavior Trees¡°Behavior trees replace the often intangible growing mess of state transitions of finite state machines (FSMs) with a more restrictive but also more structured traversal defining approach.¡± BjoernKnaflahttp://bjoernknafla.com/introduction-to-behavior-trees
  • 15. Behavior Tree of a bubble
  • 17. Cases
  • 19. Video ¡°3D Augmented Reality experience with depth-sensing cameras¡± (http://www.vimeo.com/11160661)
  • 20. ChameleonVirtual intelligent characters as tour guides in an augmented reality environment
  • 21. Video ¡°VicTour, Virtual Interactive Character Tour Guide¡± (http://vimeo.com/22415896)
  • 22. Audience Entertainment20,000 people playing at Lyon Festival of Lights
  • 23. Video ¡°AE, Orange and 20,000 People play at Lyon Festival of Lights Dec 5 2009¡± (http://www.youtube.com/watch?v=U33BqjtRJ58)
  • 25. Video ¡°The making-of a visitors center using advanced robotics and interactive interface design¡± (http://www.vimeo.com/11655898)

Editor's Notes

  1. Example of people working on camera-based interaction:Myron KreugerZack Booth SimpsonGolan LevinZach Liberman
  2. Computer vision
  3. Cognitive PsychologyArtificial IntelligenceAffective ComputingUser ModelingIntelligentAgentsMachine LearningDataMiningMultimodal Interfaces