狠狠撸

狠狠撸Share a Scribd company logo

Midi2013 presentation

?
?
Agata Cie?lar (Pasikowska)
Agata Cie?lar (Pasikowska)

This document describes a proposed multi-modal application to provide psychological support through dialogue with a virtual interlocutor. The application would use artificial intelligence to analyze verbal and non-verbal cues from users to create individualized conversations. It would integrate tools like brainwave readers, facial recognition, and questionnaires to detect mood and provide coping strategies for stressful situations through a chatbot interface. The goal is to study how internal dialogue benefits well-being and develop new technologies to support self-discovery and quality of life.

1 of 36
A Dialogue With a Virtual Imaginary Interlocutor as a Form of Psychological Support for Well-being Project of multi-modal application based on arti?cial intelligence software Agata Pasikowska - University of Social Sciences and Humanities Abolfazl Zaraki - Reaseach Center “E.Piaggio”, University of Pisa Nicole Lazzeri - Reaseach Center “E.Piaggio”, University of Pisa Warsaw, 25th June 2013 (c) Pasikowska, Zaraki, Lazerri
1. Introduction 2.The Application The Idea Structure of The Application User Flow 3.Technological Methodology Power of Mobile Devices Structure of The System Brainwave Reader Module Questionnaires Module Facial Expression Recognition Module Voice Analysis Module Interface - chatbot AIML Potential technology for Avatar 4.Summary 5.Questions&Answers (c) Pasikowska, Zaraki, Lazerri
Introduction (c) Pasikowska, Zaraki, Lazerri
Aim of the Project ? ? create a virtual reality application to sustain well-being and improve quality of life develop a solution for self-discovery and broadening techniques for coping with stressful situations ? build a multi-modal application based on storing and tracking over the time both verbal and non-verbal cues but also on the explicit declarations of the users through the questionnaires and the chat activity with a chatbot ? make a research tool for PhD study “Dialogue with an imaginary person and a coping with dif?cult situations” (c) Pasikowska, Zaraki, Lazerri
Importance of the Project ? The concept of interactive mind assumes different states, based on a combination of internal and external dialogue. Internal dialogue can be as effective in generating possible solutions to the problem situation (Baltes and Staudinger, 1996) ? Inner dialogue is a tool of support (Puchalska - Wasyl, Chmielecka - Kuter, Ole?, 2008) ? Inner dialogue with an imaginary person is a supportive coping technique in a stressful situation (Pasikowska, 2013) Staudinger U. M., Baltes P. P. (1996). Interactive Minds: A Facilitative Setting for Wisdom-Related Performance. Journal of Personality and Social Psychology 1996,Vol 71, No. 4, 746-762. Pasikowska, A (2013). Dialog z wirtualnym i wyobra?onym rozmówc? jako forma wsparcia i poprawy zdrowia psychicznego (w druku). Puchalska-Wasyl, M., Chmielnicka-Kuter, E. i Ole?, P. K. (2008). From internal interlocutors to psychological functions of dialogical activity. Journal of Constructivist Psychology, 21, (3), 239-269. (c) Pasikowska, Zaraki, Lazerri
Importance of the Project ? ? What is the function of internal dialogue? Is the internal dialogue is particularly effective way to bene?t from support in a stressful situation? ? ? Is a meeting with a person is needed? How to build a multi-modal application to support internal dialogues bene?ts? (c) Pasikowska, Zaraki, Lazerri
The Application (c) Pasikowska, Zaraki, Lazerri
The Idea ? born during Human Computer Con?uence Summer School 2012 in Milan ? uses the internal dialogue as a form of support in daily challenges and problems of life ? is based on arti?cial intelligence software that learns our internal dialogue and delivers it to us in the form of a virtual conversation with the imaginary interlocutor ! (c) Pasikowska, Zaraki, Lazerri
Structure of the System MOOD DETECTOR INTERFACE CHATBOT USER (c) Pasikowska, Zaraki, Lazerri
Structure of the System MOOD DETECTOR INTERFACE CHATBOT USER (c) Pasikowska, Zaraki, Lazerri
Structure of the System MOOD DETECTOR INTERFACE CHATBOT USER (c) Pasikowska, Zaraki, Lazerri
Structure of the System MOOD DETECTOR INTERFACE CHATBOT USER (c) Pasikowska, Zaraki, Lazerri
User Flow (c) Pasikowska, Zaraki, Lazerri
User Flow (c) Pasikowska, Zaraki, Lazerri
User Flow (c) Pasikowska, Zaraki, Lazerri
User Flow (c) Pasikowska, Zaraki, Lazerri
User Flow (c) Pasikowska, Zaraki, Lazerri
Technological Methodology (c) Pasikowska, Zaraki, Lazerri
Power of Mobile Devices Xpression Application allows users to record their mood through their speech acoustics Aurora App is fostering socially supportive behaviors by the sharing of affect on mobile devices. The INTERSTRESS project is a VR treatment of psychological stress. Aurora: http://edition.cnn.com/2013/02/26/tech/mood-tracking-app-xperience/index.html Xpression: http://idl.cornell.edu/projects/aurora/ INTERSTRESS: http://interstress.eu (c) Pasikowska, Zaraki, Lazerri
Structure of the System MOOD DETECTOR INTERFACE CHATBOT USER (c) Pasikowska, Zaraki, Lazerri
Structure of the System MOOD DETECTOR INTERFACE CHATBOT USER (c) Pasikowska, Zaraki, Lazerri
Structure of the System MOOD DETECTOR INTERFACE CHATBOT USER (c) Pasikowska, Zaraki, Lazerri
Structure of the System MOOD DETECTOR INTERFACE CHATBOT USER (c) Pasikowska, Zaraki, Lazerri
(c) Pasikowska, Zaraki, Lazerri
Brainwave Reader Module ? ? ? NeuroSky device to capture Brainwave signals It is not really an easy task. This part of the work is prototype and we haven't ?nalized it yet. The output of the system will be sent to the Mood Detector engine. http://www.neurosky.com/products/mindwavemobile.aspx (c) Pasikowska, Zaraki, Lazerri
Questionnaires Module ? effectiveness of the coping techniques controlled by achieved support. ? ? ? ? level of self-esteem anxiety emotional intelligence quality of life. (c) Pasikowska, Zaraki, Lazerri
Facial Expression Recognition Module ? ? ? ? Standard face detection and facial analyzer engine Strong feedback for the system Re?ect emotional mood of user. Angry, happy, sad, surprise also age and gender (c) Pasikowska, Zaraki, Lazerri
Facial Expression Recognition Module ? ? ? ? Standard face detection and facial analyzer engine Strong feedback for the system Re?ect emotional mood of user. Angry, happy, sad, surprise also age and gender (c) Pasikowska, Zaraki, Lazerri
Voice Analysis Module ? Analyses the user’s voice to obtain an in-depth view of her/his range of emotions inferring the stress and anxiety level. ? Voice analysis in real-time is a challenging issue. In this work we prefer to use voice signals as a communication tools between users and system. ? We integrate speech recognition engine which recognizes some keyword that are trained by user before experiments. (c) Pasikowska, Zaraki, Lazerri (c) Agata Pasikowska
(c) Pasikowska, Zaraki, Lazerri
Interface - chatbot For example: User says "Hello" <category> <pattern>HELLO</pattern> <template>Hi there!</template> = Chatbot answers "Hi there!" </category> ! (c) Pasikowska, Zaraki, Lazerri
AIML ? The AIML pattern language is simple, consisting only of words, spaces, and the wildcard symbols _ and *. The words may consist of letters and numerals, but no other characters. The pattern language is case invariant. Words are separated by a single space, and the wildcard characters function like words. Chatbot asks "Do you like movies?" Another example: <category> <pattern>YES</pattern> <that>DO YOU LIKE MOVIES</that> <template>What is your favorite movie?</template> </category> = User says "Yes" “What is your favorite movie?” ! (c) Pasikowska, Zaraki, Lazerri
Potential technology for Avatar ? Faceshift analyzes the face motions of an actor, and describes them as a mixture of basic expressions, plus head orientation and gaze. http://www.faceshift.com http://youtu.be/Bz4WSLUy0Wo (c) Pasikowska, Zaraki, Lazerri
Summary We will: ? Create an innovative e-service which aims to increase knowledge of the human phenomenon of internal dialogues ? ? ? Support people at risk of stress Develop a new technology for self-discovery and improve the quality of life Build an arti?cial intelligence in the form of virtual speaker for storing and tracking over the time both verbal (a voice analysis) and non-verbal (brainwave, facial emotion recognition) cues but also on the explicit declarations of the users through the questionnaires and the chat activity ! (c) Pasikowska, Zaraki, Lazerri
Q &A Any technical questions? Please contact: Abolfazl Zaraki (ab.zaraki@gmail.com) Nicole Lazzeri (nicole_lazzeri@hotmail.com) (c) Pasikowska, Zaraki, Lazerri
Thank You. Agata Pasikowska (agata.pasikowska@gmail.com) (c) Pasikowska, Zaraki, Lazerri

More Related Content

Midi2013 presentation

  • 1. A Dialogue With a Virtual Imaginary Interlocutor as a Form of Psychological Support for Well-being Project of multi-modal application based on arti?cial intelligence software Agata Pasikowska - University of Social Sciences and Humanities Abolfazl Zaraki - Reaseach Center “E.Piaggio”, University of Pisa Nicole Lazzeri - Reaseach Center “E.Piaggio”, University of Pisa Warsaw, 25th June 2013 (c) Pasikowska, Zaraki, Lazerri
  • 2. 1. Introduction 2.The Application The Idea Structure of The Application User Flow 3.Technological Methodology Power of Mobile Devices Structure of The System Brainwave Reader Module Questionnaires Module Facial Expression Recognition Module Voice Analysis Module Interface - chatbot AIML Potential technology for Avatar 4.Summary 5.Questions&Answers (c) Pasikowska, Zaraki, Lazerri
  • 4. Aim of the Project ? ? create a virtual reality application to sustain well-being and improve quality of life develop a solution for self-discovery and broadening techniques for coping with stressful situations ? build a multi-modal application based on storing and tracking over the time both verbal and non-verbal cues but also on the explicit declarations of the users through the questionnaires and the chat activity with a chatbot ? make a research tool for PhD study “Dialogue with an imaginary person and a coping with dif?cult situations” (c) Pasikowska, Zaraki, Lazerri
  • 5. Importance of the Project ? The concept of interactive mind assumes different states, based on a combination of internal and external dialogue. Internal dialogue can be as effective in generating possible solutions to the problem situation (Baltes and Staudinger, 1996) ? Inner dialogue is a tool of support (Puchalska - Wasyl, Chmielecka - Kuter, Ole?, 2008) ? Inner dialogue with an imaginary person is a supportive coping technique in a stressful situation (Pasikowska, 2013) Staudinger U. M., Baltes P. P. (1996). Interactive Minds: A Facilitative Setting for Wisdom-Related Performance. Journal of Personality and Social Psychology 1996,Vol 71, No. 4, 746-762. Pasikowska, A (2013). Dialog z wirtualnym i wyobra?onym rozmówc? jako forma wsparcia i poprawy zdrowia psychicznego (w druku). Puchalska-Wasyl, M., Chmielnicka-Kuter, E. i Ole?, P. K. (2008). From internal interlocutors to psychological functions of dialogical activity. Journal of Constructivist Psychology, 21, (3), 239-269. (c) Pasikowska, Zaraki, Lazerri
  • 6. Importance of the Project ? ? What is the function of internal dialogue? Is the internal dialogue is particularly effective way to bene?t from support in a stressful situation? ? ? Is a meeting with a person is needed? How to build a multi-modal application to support internal dialogues bene?ts? (c) Pasikowska, Zaraki, Lazerri
  • 8. The Idea ? born during Human Computer Con?uence Summer School 2012 in Milan ? uses the internal dialogue as a form of support in daily challenges and problems of life ? is based on arti?cial intelligence software that learns our internal dialogue and delivers it to us in the form of a virtual conversation with the imaginary interlocutor ! (c) Pasikowska, Zaraki, Lazerri
  • 9. Structure of the System MOOD DETECTOR INTERFACE CHATBOT USER (c) Pasikowska, Zaraki, Lazerri
  • 10. Structure of the System MOOD DETECTOR INTERFACE CHATBOT USER (c) Pasikowska, Zaraki, Lazerri
  • 11. Structure of the System MOOD DETECTOR INTERFACE CHATBOT USER (c) Pasikowska, Zaraki, Lazerri
  • 12. Structure of the System MOOD DETECTOR INTERFACE CHATBOT USER (c) Pasikowska, Zaraki, Lazerri
  • 13. User Flow (c) Pasikowska, Zaraki, Lazerri
  • 14. User Flow (c) Pasikowska, Zaraki, Lazerri
  • 15. User Flow (c) Pasikowska, Zaraki, Lazerri
  • 16. User Flow (c) Pasikowska, Zaraki, Lazerri
  • 17. User Flow (c) Pasikowska, Zaraki, Lazerri
  • 19. Power of Mobile Devices Xpression Application allows users to record their mood through their speech acoustics Aurora App is fostering socially supportive behaviors by the sharing of affect on mobile devices. The INTERSTRESS project is a VR treatment of psychological stress. Aurora: http://edition.cnn.com/2013/02/26/tech/mood-tracking-app-xperience/index.html Xpression: http://idl.cornell.edu/projects/aurora/ INTERSTRESS: http://interstress.eu (c) Pasikowska, Zaraki, Lazerri
  • 20. Structure of the System MOOD DETECTOR INTERFACE CHATBOT USER (c) Pasikowska, Zaraki, Lazerri
  • 21. Structure of the System MOOD DETECTOR INTERFACE CHATBOT USER (c) Pasikowska, Zaraki, Lazerri
  • 22. Structure of the System MOOD DETECTOR INTERFACE CHATBOT USER (c) Pasikowska, Zaraki, Lazerri
  • 23. Structure of the System MOOD DETECTOR INTERFACE CHATBOT USER (c) Pasikowska, Zaraki, Lazerri
  • 25. Brainwave Reader Module ? ? ? NeuroSky device to capture Brainwave signals It is not really an easy task. This part of the work is prototype and we haven't ?nalized it yet. The output of the system will be sent to the Mood Detector engine. http://www.neurosky.com/products/mindwavemobile.aspx (c) Pasikowska, Zaraki, Lazerri
  • 26. Questionnaires Module ? effectiveness of the coping techniques controlled by achieved support. ? ? ? ? level of self-esteem anxiety emotional intelligence quality of life. (c) Pasikowska, Zaraki, Lazerri
  • 27. Facial Expression Recognition Module ? ? ? ? Standard face detection and facial analyzer engine Strong feedback for the system Re?ect emotional mood of user. Angry, happy, sad, surprise also age and gender (c) Pasikowska, Zaraki, Lazerri
  • 28. Facial Expression Recognition Module ? ? ? ? Standard face detection and facial analyzer engine Strong feedback for the system Re?ect emotional mood of user. Angry, happy, sad, surprise also age and gender (c) Pasikowska, Zaraki, Lazerri
  • 29. Voice Analysis Module ? Analyses the user’s voice to obtain an in-depth view of her/his range of emotions inferring the stress and anxiety level. ? Voice analysis in real-time is a challenging issue. In this work we prefer to use voice signals as a communication tools between users and system. ? We integrate speech recognition engine which recognizes some keyword that are trained by user before experiments. (c) Pasikowska, Zaraki, Lazerri (c) Agata Pasikowska
  • 31. Interface - chatbot For example: User says "Hello" <category> <pattern>HELLO</pattern> <template>Hi there!</template> = Chatbot answers "Hi there!" </category> ! (c) Pasikowska, Zaraki, Lazerri
  • 32. AIML ? The AIML pattern language is simple, consisting only of words, spaces, and the wildcard symbols _ and *. The words may consist of letters and numerals, but no other characters. The pattern language is case invariant. Words are separated by a single space, and the wildcard characters function like words. Chatbot asks "Do you like movies?" Another example: <category> <pattern>YES</pattern> <that>DO YOU LIKE MOVIES</that> <template>What is your favorite movie?</template> </category> = User says "Yes" “What is your favorite movie?” ! (c) Pasikowska, Zaraki, Lazerri
  • 33. Potential technology for Avatar ? Faceshift analyzes the face motions of an actor, and describes them as a mixture of basic expressions, plus head orientation and gaze. http://www.faceshift.com http://youtu.be/Bz4WSLUy0Wo (c) Pasikowska, Zaraki, Lazerri
  • 34. Summary We will: ? Create an innovative e-service which aims to increase knowledge of the human phenomenon of internal dialogues ? ? ? Support people at risk of stress Develop a new technology for self-discovery and improve the quality of life Build an arti?cial intelligence in the form of virtual speaker for storing and tracking over the time both verbal (a voice analysis) and non-verbal (brainwave, facial emotion recognition) cues but also on the explicit declarations of the users through the questionnaires and the chat activity ! (c) Pasikowska, Zaraki, Lazerri
  • 35. Q &A Any technical questions? Please contact: Abolfazl Zaraki (ab.zaraki@gmail.com) Nicole Lazzeri (nicole_lazzeri@hotmail.com) (c) Pasikowska, Zaraki, Lazerri
  • 36. Thank You. Agata Pasikowska (agata.pasikowska@gmail.com) (c) Pasikowska, Zaraki, Lazerri