際際滷

際際滷Share a Scribd company logo

Collaborative Assistant/Virtual Assistant/ Conversational AI agents at AIISC

Download as PPTX, PDF
Artificial Intelligence Institute at UofSC
Artificial Intelligence Institute at UofSC

The document discusses collaborative assistants being developed at AIISC focused on health and education. Examples include a COVID-19 symptom checker, assistants for pediatric asthma, nutrition, mental health, autism, and more. Complex assistants for health domains require context using deep domain knowledge, personalization with personalized health graphs, sensing data from multiple modalities like text, voice and images. The virtual health assistant for asthma aims to do self-monitoring, self-appraisal, management, and disease tracking using sensors, social media, clinical data and knowledge graphs to have informed intelligent conversations. Developing such assistants is quite challenging due to big data issues, need for domain context, personalization, explainability, and ease of use.

1 of 6
Collaborative Assistant/Virtual Assistant/ Conversational AI agents at AIISC Panel @ Collaborative Assistants for the Society (CASY 2020) Amit Sheth Director, Artificial Intelligence Institute of UofSC (#AIISC, aiisc.ai)
Focus on two domains: Health & Education Broad variety: Simple chatbot (flexible interactions) to quite complex (manage chronic disease) Uniqueness of complex examples: a. Context with deep domain knowledge, b. Personalization with personalized health knowledge graph, c. active/passive sensing, d. multimodal (text, voice, image) Examples at various stages of development Health-e Gamecock COVID-10 daily symptom checker Pediatric Asthma Nutrition Mental Health Autism, After Cancer Exercise, Diabetes, and more in planning. http://wiki.aiisc.ai/index.php/Covid19
1. Self Monitoring 2. Self Appraisal 3. Self Management 4. Intervention 5. Disease Progression and Tracking Virtual Health Assistant for Augmented Personalized Health: example of Asthma Complex- multifactorial disease, personalized Access- challenges in working with patients (IRB, privacy, medical/clinical knowledge and lack of gold standards, etc.
4 Sensor, Social, Clinical Datastreams: Informed & Intelligent Questions Weather information (temperature, pollen, humidity, etc) Elasticsearch (ES) Database Query & Rule Abstract raw values into information Asthma Domain Knowledge https://bioportal.bioontology.org/ontologies/AO http://www.childhealthservicemodels.eu Patient Data from EMR & PGHD (Compliance score, prescribed medications, asthma control level) IoTs (Foobot & Fitbit) Conversation Rules & Scripts (DialogFlow) Sensor,Social,Clinical Architectural Framework for Intelligent & Informed Conversations: kBOT Asthma Sensor (IoTs) & Cyber Datastream Clinical (Baseline) Datastream Patient Consented Social Data (Facebook, Instagram, Twitter Activity) Social Datastream Knowledge Datastream Smarter & engaging agent Minimize active sensing (Questions to be asked) Ask only informed & intelligent questions Relevant & Contextualized conversations Personalized & Human-Like Human-Like Aspect
Contextualization and Personalization kBOT initiates greeting conversation. Understands the patients health condition (allergic reaction to high ragweed pollen level) via the personalized patients knowledge graph generated from EMR, PGHD, and prior interactions with the kBOT. Generates predictions or recommended course of actions. Inference based on patients historical records and background health knowledge graph containing contextualized (domain-specific) knowledge. Figure: Example kBOT conversation which utilizes background health knowledge graph and patients knowledge graph to infer and generate recommendation to patients. Conversing only information relevant to the patient 5
Huge need in several fields health and education among the important areas. Examples from health domain: not enough available clinical expertise to meet mental health needs; chronic disease need continuous care; massive growth in patient generate data with demonstrated value for making decisions on health, wellness and fitness. But, Quite challenging: big data challenges esp variety of data, need for context to interpret data (and relevant deep domain knowledge), personalization for each patient, need to be able to explain results before clinicians will take the technology seriously, ease of use/UX, incentives in validation studies (patients, clinicians, insurance companies), etc.

More Related Content

Collaborative Assistant/Virtual Assistant/ Conversational AI agents at AIISC

  • 1. Collaborative Assistant/Virtual Assistant/ Conversational AI agents at AIISC Panel @ Collaborative Assistants for the Society (CASY 2020) Amit Sheth Director, Artificial Intelligence Institute of UofSC (#AIISC, aiisc.ai)
  • 2. Focus on two domains: Health & Education Broad variety: Simple chatbot (flexible interactions) to quite complex (manage chronic disease) Uniqueness of complex examples: a. Context with deep domain knowledge, b. Personalization with personalized health knowledge graph, c. active/passive sensing, d. multimodal (text, voice, image) Examples at various stages of development Health-e Gamecock COVID-10 daily symptom checker Pediatric Asthma Nutrition Mental Health Autism, After Cancer Exercise, Diabetes, and more in planning. http://wiki.aiisc.ai/index.php/Covid19
  • 3. 1. Self Monitoring 2. Self Appraisal 3. Self Management 4. Intervention 5. Disease Progression and Tracking Virtual Health Assistant for Augmented Personalized Health: example of Asthma Complex- multifactorial disease, personalized Access- challenges in working with patients (IRB, privacy, medical/clinical knowledge and lack of gold standards, etc.
  • 4. 4 Sensor, Social, Clinical Datastreams: Informed & Intelligent Questions Weather information (temperature, pollen, humidity, etc) Elasticsearch (ES) Database Query & Rule Abstract raw values into information Asthma Domain Knowledge https://bioportal.bioontology.org/ontologies/AO http://www.childhealthservicemodels.eu Patient Data from EMR & PGHD (Compliance score, prescribed medications, asthma control level) IoTs (Foobot & Fitbit) Conversation Rules & Scripts (DialogFlow) Sensor,Social,Clinical Architectural Framework for Intelligent & Informed Conversations: kBOT Asthma Sensor (IoTs) & Cyber Datastream Clinical (Baseline) Datastream Patient Consented Social Data (Facebook, Instagram, Twitter Activity) Social Datastream Knowledge Datastream Smarter & engaging agent Minimize active sensing (Questions to be asked) Ask only informed & intelligent questions Relevant & Contextualized conversations Personalized & Human-Like Human-Like Aspect
  • 5. Contextualization and Personalization kBOT initiates greeting conversation. Understands the patients health condition (allergic reaction to high ragweed pollen level) via the personalized patients knowledge graph generated from EMR, PGHD, and prior interactions with the kBOT. Generates predictions or recommended course of actions. Inference based on patients historical records and background health knowledge graph containing contextualized (domain-specific) knowledge. Figure: Example kBOT conversation which utilizes background health knowledge graph and patients knowledge graph to infer and generate recommendation to patients. Conversing only information relevant to the patient 5
  • 6. Huge need in several fields health and education among the important areas. Examples from health domain: not enough available clinical expertise to meet mental health needs; chronic disease need continuous care; massive growth in patient generate data with demonstrated value for making decisions on health, wellness and fitness. But, Quite challenging: big data challenges esp variety of data, need for context to interpret data (and relevant deep domain knowledge), personalization for each patient, need to be able to explain results before clinicians will take the technology seriously, ease of use/UX, incentives in validation studies (patients, clinicians, insurance companies), etc.

Editor's Notes

  1. Architecture slide, sensor, social, clinical