ºÝºÝߣ

ºÝºÝߣShare a Scribd company logo
Multimodal Interfaces November, 2008 Thomas Winningham
Multi ah-wha? Computer vision, mostly The Wiimote 100hz camera Capacitive surfaces (or lots of buttons)? IPhone & Mac touchpad HP TouchSmart Tin Foil & an Arduino Force transducers, the Wii Fit Teaching new dogs old tricks
Regions of interest At 30 fps or faster: Find a finger smudge (blobs)? Dots of light Triangulate based on closest data Color keying LEDs as two-way Use IR for spectrum separation Isolate features with laser Map in 3D using Lidar
Better Software & Hardware DARPA Challenge Intel OpenCV Multiprocessor Machines Home clusters of machines GPU processors on videocards Better web cams Cheaper, more widely available materials Easier access to information
Fresh Ways to Interface with Computers It is not a panacea, but at least everyone is mostly invited.
Leave it to the big boys? IPhone's  And knock-offs Microsoft Surface & Windows 7 & Tablets Microsoft Research HP TouchSmart Perceptive Pixel NUI Group  step 1. cut a hole in a box
Gesturing Combos! Changes over time Inferring about the placement of input in orientation to the real world Surface Wii fit You're holding a nail near tinfoil
Simple! Or at least, reasonably simple given the complexity of what is going on. Small blocks of code in repition Again, faster computers
Web Cam in a Box Step #2 Put your junk in that box
Double true! Stackable cubes Large walls Small hobby projects Multiple cameras GPUs Networked machines Store Windows, floors , buildings
Mostly flash (needs a lot of help from c)? But also: Python Processing Max VVVV Pure Data .NET Firefox / Javascript (Lily)?
Rethinking a lot of things Gesture navigation User Interface Feedback
Guerrilla Arm
All kinds of things I tell ya! Sphere Gesturing with your hands  above the home row Or just using your hands Or moving your head, or eyes, or thoughts This is an extra bullet point. Ha!
Meanwhile, back at the bat cave... Home clustering Semistructured web and public data HDTV's, projectors, err... iphones Social connectivity Except in the workplace where we'd generally like everyone to politely sod off Advanced modeling, free software for fluid dynamics and physics simulation
Lack of standards? lack of patent free ideas, maybe TUIO / OSC for ROI or Fiducials
Romper room I see: NUI Group Open Frameworks Processing Apple Developer forums IPhone jailbreakers Multi-pointer X-Windows, Compiz, and the whole open-source world.
Feedback loop Proliferation? Adoption? Social implications This will all spawn something else Everybody's coming to the net Voluntary mass hallucination (Gibson)?
Come out and play! If yer under eighteen ya won't be doin any ty-ee-eye-ee-ime
There's a place for us! More people playing! == better software Not as hard as it sounds, just steal code ;p Who's getting laid anyway? It is amazing what you can get done if you don't care who gets the credit. (jets ftw!)?
Ad

Recommended

Does Design Matter?
Does Design Matter?
PetroITLtd
?
History of Human-Computer Interaction
History of Human-Computer Interaction
Erik Duval
?
Speaking assessment 2 final
Speaking assessment 2 final
Gon?alo Duarte
?
A keyboard
A keyboard
Jeddie Bere
?
CSS Cascade [Òë]
CSS Cascade [Òë]
chencheng ÔÆÇ«
?
Using Google and Social Networks to Grow Your Business
Using Google and Social Networks to Grow Your Business
Alvin Borromeo
?
Let's Hug It Out Columbus
Let's Hug It Out Columbus
Alvin Borromeo
?
Artie Isaac
Alvin Borromeo
?
The parable of the totaled saab
The parable of the totaled saab
Alvin Borromeo
?
The Time to Grow Your Network is Before You Need It
The Time to Grow Your Network is Before You Need It
Alvin Borromeo
?
Backcasting with Japanese Animation
Backcasting with Japanese Animation
Alvin Borromeo
?
Coliformes
guestd65d
?
Six Degrees Of Dan
Six Degrees Of Dan
Alvin Borromeo
?
Default01
Default01
venzz
?
±Ê°ù±ð²õ±ð²Ô³Ù²¹³¦¾±¨®±·1
SopaAguiar
?
ǰ¶Ë²âÊÔÖ®ÌÔ±¦Êµ¼ù
ǰ¶Ë²âÊÔÖ®ÌÔ±¦Êµ¼ù
chencheng ÔÆÇ«
?
Promoting Art + Technology for Fun and (non)Profit
Promoting Art + Technology for Fun and (non)Profit
Alvin Borromeo
?
±Ê°ù±ð²õ±ð²Ô³Ù²¹³¦¾±¨®±·1
SopaAguiar
?
±Ê°ù±ð²õ±ð²Ô³Ù²¹³¦¾±¨®±·1
SopaAguiar
?
Temparate Broadleaf Deciduous Forest
Temparate Broadleaf Deciduous Forest
Alvin Borromeo
?
Peri¨®dico El Socialista, 4 de octubre de 1934.
Alfredo Garc¨ªa
?
Embedded & Tangible Interaction Design
Embedded & Tangible Interaction Design
Dave Shaw
?
Multi-Touch Tangible Interface; HCI trends, projects, and development
Multi-Touch Tangible Interface; HCI trends, projects, and development
Jazmi Jamal
?
Graphics in mobile terminals
Graphics in mobile terminals
Tomi Aarnio
?
ICS3211 lecture 08
ICS3211 lecture 08
Vanessa Camilleri
?
Digital natives: freedom and hackability in a mobile future
Digital natives: freedom and hackability in a mobile future
Tristan Nitot
?
Is the Browser a Transitional Technology?
Is the Browser a Transitional Technology?
Allen Wirfs-Brock
?
Future of interface design 2010
Future of interface design 2010
Pavel R??i?ka
?
The computer
The computer
pallaviskb
?
Gesture Based Interaction
Gesture Based Interaction
lanesk8er
?

More Related Content

Viewers also liked (13)

The parable of the totaled saab
The parable of the totaled saab
Alvin Borromeo
?
The Time to Grow Your Network is Before You Need It
The Time to Grow Your Network is Before You Need It
Alvin Borromeo
?
Backcasting with Japanese Animation
Backcasting with Japanese Animation
Alvin Borromeo
?
Coliformes
guestd65d
?
Six Degrees Of Dan
Six Degrees Of Dan
Alvin Borromeo
?
Default01
Default01
venzz
?
±Ê°ù±ð²õ±ð²Ô³Ù²¹³¦¾±¨®±·1
SopaAguiar
?
ǰ¶Ë²âÊÔÖ®ÌÔ±¦Êµ¼ù
ǰ¶Ë²âÊÔÖ®ÌÔ±¦Êµ¼ù
chencheng ÔÆÇ«
?
Promoting Art + Technology for Fun and (non)Profit
Promoting Art + Technology for Fun and (non)Profit
Alvin Borromeo
?
±Ê°ù±ð²õ±ð²Ô³Ù²¹³¦¾±¨®±·1
SopaAguiar
?
±Ê°ù±ð²õ±ð²Ô³Ù²¹³¦¾±¨®±·1
SopaAguiar
?
Temparate Broadleaf Deciduous Forest
Temparate Broadleaf Deciduous Forest
Alvin Borromeo
?
Peri¨®dico El Socialista, 4 de octubre de 1934.
Alfredo Garc¨ªa
?
The parable of the totaled saab
The parable of the totaled saab
Alvin Borromeo
?
The Time to Grow Your Network is Before You Need It
The Time to Grow Your Network is Before You Need It
Alvin Borromeo
?
Backcasting with Japanese Animation
Backcasting with Japanese Animation
Alvin Borromeo
?
Coliformes
guestd65d
?
Default01
Default01
venzz
?
±Ê°ù±ð²õ±ð²Ô³Ù²¹³¦¾±¨®±·1
SopaAguiar
?
ǰ¶Ë²âÊÔÖ®ÌÔ±¦Êµ¼ù
ǰ¶Ë²âÊÔÖ®ÌÔ±¦Êµ¼ù
chencheng ÔÆÇ«
?
Promoting Art + Technology for Fun and (non)Profit
Promoting Art + Technology for Fun and (non)Profit
Alvin Borromeo
?
±Ê°ù±ð²õ±ð²Ô³Ù²¹³¦¾±¨®±·1
SopaAguiar
?
±Ê°ù±ð²õ±ð²Ô³Ù²¹³¦¾±¨®±·1
SopaAguiar
?
Temparate Broadleaf Deciduous Forest
Temparate Broadleaf Deciduous Forest
Alvin Borromeo
?
Peri¨®dico El Socialista, 4 de octubre de 1934.
Alfredo Garc¨ªa
?

Similar to Multimodal Interfaces (20)

Embedded & Tangible Interaction Design
Embedded & Tangible Interaction Design
Dave Shaw
?
Multi-Touch Tangible Interface; HCI trends, projects, and development
Multi-Touch Tangible Interface; HCI trends, projects, and development
Jazmi Jamal
?
Graphics in mobile terminals
Graphics in mobile terminals
Tomi Aarnio
?
ICS3211 lecture 08
ICS3211 lecture 08
Vanessa Camilleri
?
Digital natives: freedom and hackability in a mobile future
Digital natives: freedom and hackability in a mobile future
Tristan Nitot
?
Is the Browser a Transitional Technology?
Is the Browser a Transitional Technology?
Allen Wirfs-Brock
?
Future of interface design 2010
Future of interface design 2010
Pavel R??i?ka
?
The computer
The computer
pallaviskb
?
Gesture Based Interaction
Gesture Based Interaction
lanesk8er
?
Chapter 9 - Past, Present, Future, User interface
Chapter 9 - Past, Present, Future, User interface
Muhammad Najib
?
The iPad is 40 years old
The iPad is 40 years old
Arnold Wasserman
?
Digital Meetup - How Not to be Shit!
Digital Meetup - How Not to be Shit!
Stuart Neale
?
426 Lecture 9: Research Directions in AR
426 Lecture 9: Research Directions in AR
Mark Billinghurst
?
Ubiquitous computing applications
Ubiquitous computing applications
Hamza Malik
?
18 applied architectures_part_2
18 applied architectures_part_2
Majong DevJfu
?
Operating Systems
Operating Systems
Leon Lei
?
Prototyping in code
Prototyping in code
Marcin Ignac
?
HCI 3e - Ch 20: Ubiquitous computing and augmented realities
HCI 3e - Ch 20: Ubiquitous computing and augmented realities
Alan Dix
?
Internet trends and ICT knowledge necessary in the next years - 2013 ed.
Internet trends and ICT knowledge necessary in the next years - 2013 ed.
Antonio Ciccarelli, PMP
?
Natural Interfaces for Augmented Reality
Natural Interfaces for Augmented Reality
Mark Billinghurst
?
Embedded & Tangible Interaction Design
Embedded & Tangible Interaction Design
Dave Shaw
?
Multi-Touch Tangible Interface; HCI trends, projects, and development
Multi-Touch Tangible Interface; HCI trends, projects, and development
Jazmi Jamal
?
Graphics in mobile terminals
Graphics in mobile terminals
Tomi Aarnio
?
Digital natives: freedom and hackability in a mobile future
Digital natives: freedom and hackability in a mobile future
Tristan Nitot
?
Is the Browser a Transitional Technology?
Is the Browser a Transitional Technology?
Allen Wirfs-Brock
?
Future of interface design 2010
Future of interface design 2010
Pavel R??i?ka
?
Gesture Based Interaction
Gesture Based Interaction
lanesk8er
?
Chapter 9 - Past, Present, Future, User interface
Chapter 9 - Past, Present, Future, User interface
Muhammad Najib
?
Digital Meetup - How Not to be Shit!
Digital Meetup - How Not to be Shit!
Stuart Neale
?
426 Lecture 9: Research Directions in AR
426 Lecture 9: Research Directions in AR
Mark Billinghurst
?
Ubiquitous computing applications
Ubiquitous computing applications
Hamza Malik
?
18 applied architectures_part_2
18 applied architectures_part_2
Majong DevJfu
?
Operating Systems
Operating Systems
Leon Lei
?
HCI 3e - Ch 20: Ubiquitous computing and augmented realities
HCI 3e - Ch 20: Ubiquitous computing and augmented realities
Alan Dix
?
Internet trends and ICT knowledge necessary in the next years - 2013 ed.
Internet trends and ICT knowledge necessary in the next years - 2013 ed.
Antonio Ciccarelli, PMP
?
Natural Interfaces for Augmented Reality
Natural Interfaces for Augmented Reality
Mark Billinghurst
?
Ad

Recently uploaded (20)

Coordinated Disclosure for ML - What's Different and What's the Same.pdf
Coordinated Disclosure for ML - What's Different and What's the Same.pdf
Priyanka Aash
?
Tech-ASan: Two-stage check for Address Sanitizer - Yixuan Cao.pdf
Tech-ASan: Two-stage check for Address Sanitizer - Yixuan Cao.pdf
caoyixuan2019
?
Cyber Defense Matrix Workshop - RSA Conference
Cyber Defense Matrix Workshop - RSA Conference
Priyanka Aash
?
" How to survive with 1 billion vectors and not sell a kidney: our low-cost c...
" How to survive with 1 billion vectors and not sell a kidney: our low-cost c...
Fwdays
?
"Database isolation: how we deal with hundreds of direct connections to the d...
"Database isolation: how we deal with hundreds of direct connections to the d...
Fwdays
?
AI Agents and FME: A How-to Guide on Generating Synthetic Metadata
AI Agents and FME: A How-to Guide on Generating Synthetic Metadata
Safe Software
?
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik
?
OWASP Barcelona 2025 Threat Model Library
OWASP Barcelona 2025 Threat Model Library
PetraVukmirovic
?
MuleSoft for AgentForce : Topic Center and API Catalog
MuleSoft for AgentForce : Topic Center and API Catalog
shyamraj55
?
A Constitutional Quagmire - Ethical Minefields of AI, Cyber, and Privacy.pdf
A Constitutional Quagmire - Ethical Minefields of AI, Cyber, and Privacy.pdf
Priyanka Aash
?
Cracking the Code - Unveiling Synergies Between Open Source Security and AI.pdf
Cracking the Code - Unveiling Synergies Between Open Source Security and AI.pdf
Priyanka Aash
?
GenAI Opportunities and Challenges - Where 370 Enterprises Are Focusing Now.pdf
GenAI Opportunities and Challenges - Where 370 Enterprises Are Focusing Now.pdf
Priyanka Aash
?
OpenPOWER Foundation & Open-Source Core Innovations
OpenPOWER Foundation & Open-Source Core Innovations
IBM
?
EIS-Webinar-Engineering-Retail-Infrastructure-06-16-2025.pdf
EIS-Webinar-Engineering-Retail-Infrastructure-06-16-2025.pdf
Earley Information Science
?
CapCut Pro Crack For PC Latest Version {Fully Unlocked} 2025
CapCut Pro Crack For PC Latest Version {Fully Unlocked} 2025
pcprocore
?
UserCon Belgium: Honey, VMware increased my bill
UserCon Belgium: Honey, VMware increased my bill
stijn40
?
2025_06_18 - OpenMetadata Community Meeting.pdf
2025_06_18 - OpenMetadata Community Meeting.pdf
OpenMetadata
?
Mastering AI Workflows with FME by Mark Do?ring
Mastering AI Workflows with FME by Mark Do?ring
Safe Software
?
Oh, the Possibilities - Balancing Innovation and Risk with Generative AI.pdf
Oh, the Possibilities - Balancing Innovation and Risk with Generative AI.pdf
Priyanka Aash
?
Quantum AI: Where Impossible Becomes Probable
Quantum AI: Where Impossible Becomes Probable
Saikat Basu
?
Coordinated Disclosure for ML - What's Different and What's the Same.pdf
Coordinated Disclosure for ML - What's Different and What's the Same.pdf
Priyanka Aash
?
Tech-ASan: Two-stage check for Address Sanitizer - Yixuan Cao.pdf
Tech-ASan: Two-stage check for Address Sanitizer - Yixuan Cao.pdf
caoyixuan2019
?
Cyber Defense Matrix Workshop - RSA Conference
Cyber Defense Matrix Workshop - RSA Conference
Priyanka Aash
?
" How to survive with 1 billion vectors and not sell a kidney: our low-cost c...
" How to survive with 1 billion vectors and not sell a kidney: our low-cost c...
Fwdays
?
"Database isolation: how we deal with hundreds of direct connections to the d...
"Database isolation: how we deal with hundreds of direct connections to the d...
Fwdays
?
AI Agents and FME: A How-to Guide on Generating Synthetic Metadata
AI Agents and FME: A How-to Guide on Generating Synthetic Metadata
Safe Software
?
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik
?
OWASP Barcelona 2025 Threat Model Library
OWASP Barcelona 2025 Threat Model Library
PetraVukmirovic
?
MuleSoft for AgentForce : Topic Center and API Catalog
MuleSoft for AgentForce : Topic Center and API Catalog
shyamraj55
?
A Constitutional Quagmire - Ethical Minefields of AI, Cyber, and Privacy.pdf
A Constitutional Quagmire - Ethical Minefields of AI, Cyber, and Privacy.pdf
Priyanka Aash
?
Cracking the Code - Unveiling Synergies Between Open Source Security and AI.pdf
Cracking the Code - Unveiling Synergies Between Open Source Security and AI.pdf
Priyanka Aash
?
GenAI Opportunities and Challenges - Where 370 Enterprises Are Focusing Now.pdf
GenAI Opportunities and Challenges - Where 370 Enterprises Are Focusing Now.pdf
Priyanka Aash
?
OpenPOWER Foundation & Open-Source Core Innovations
OpenPOWER Foundation & Open-Source Core Innovations
IBM
?
EIS-Webinar-Engineering-Retail-Infrastructure-06-16-2025.pdf
EIS-Webinar-Engineering-Retail-Infrastructure-06-16-2025.pdf
Earley Information Science
?
CapCut Pro Crack For PC Latest Version {Fully Unlocked} 2025
CapCut Pro Crack For PC Latest Version {Fully Unlocked} 2025
pcprocore
?
UserCon Belgium: Honey, VMware increased my bill
UserCon Belgium: Honey, VMware increased my bill
stijn40
?
2025_06_18 - OpenMetadata Community Meeting.pdf
2025_06_18 - OpenMetadata Community Meeting.pdf
OpenMetadata
?
Mastering AI Workflows with FME by Mark Do?ring
Mastering AI Workflows with FME by Mark Do?ring
Safe Software
?
Oh, the Possibilities - Balancing Innovation and Risk with Generative AI.pdf
Oh, the Possibilities - Balancing Innovation and Risk with Generative AI.pdf
Priyanka Aash
?
Quantum AI: Where Impossible Becomes Probable
Quantum AI: Where Impossible Becomes Probable
Saikat Basu
?
Ad

Multimodal Interfaces

  • 1. Multimodal Interfaces November, 2008 Thomas Winningham
  • 2. Multi ah-wha? Computer vision, mostly The Wiimote 100hz camera Capacitive surfaces (or lots of buttons)? IPhone & Mac touchpad HP TouchSmart Tin Foil & an Arduino Force transducers, the Wii Fit Teaching new dogs old tricks
  • 3. Regions of interest At 30 fps or faster: Find a finger smudge (blobs)? Dots of light Triangulate based on closest data Color keying LEDs as two-way Use IR for spectrum separation Isolate features with laser Map in 3D using Lidar
  • 4. Better Software & Hardware DARPA Challenge Intel OpenCV Multiprocessor Machines Home clusters of machines GPU processors on videocards Better web cams Cheaper, more widely available materials Easier access to information
  • 5. Fresh Ways to Interface with Computers It is not a panacea, but at least everyone is mostly invited.
  • 6. Leave it to the big boys? IPhone's And knock-offs Microsoft Surface & Windows 7 & Tablets Microsoft Research HP TouchSmart Perceptive Pixel NUI Group step 1. cut a hole in a box
  • 7. Gesturing Combos! Changes over time Inferring about the placement of input in orientation to the real world Surface Wii fit You're holding a nail near tinfoil
  • 8. Simple! Or at least, reasonably simple given the complexity of what is going on. Small blocks of code in repition Again, faster computers
  • 9. Web Cam in a Box Step #2 Put your junk in that box
  • 10. Double true! Stackable cubes Large walls Small hobby projects Multiple cameras GPUs Networked machines Store Windows, floors , buildings
  • 11. Mostly flash (needs a lot of help from c)? But also: Python Processing Max VVVV Pure Data .NET Firefox / Javascript (Lily)?
  • 12. Rethinking a lot of things Gesture navigation User Interface Feedback
  • 14. All kinds of things I tell ya! Sphere Gesturing with your hands above the home row Or just using your hands Or moving your head, or eyes, or thoughts This is an extra bullet point. Ha!
  • 15. Meanwhile, back at the bat cave... Home clustering Semistructured web and public data HDTV's, projectors, err... iphones Social connectivity Except in the workplace where we'd generally like everyone to politely sod off Advanced modeling, free software for fluid dynamics and physics simulation
  • 16. Lack of standards? lack of patent free ideas, maybe TUIO / OSC for ROI or Fiducials
  • 17. Romper room I see: NUI Group Open Frameworks Processing Apple Developer forums IPhone jailbreakers Multi-pointer X-Windows, Compiz, and the whole open-source world.
  • 18. Feedback loop Proliferation? Adoption? Social implications This will all spawn something else Everybody's coming to the net Voluntary mass hallucination (Gibson)?
  • 19. Come out and play! If yer under eighteen ya won't be doin any ty-ee-eye-ee-ime
  • 20. There's a place for us! More people playing! == better software Not as hard as it sounds, just steal code ;p Who's getting laid anyway? It is amazing what you can get done if you don't care who gets the credit. (jets ftw!)?