際際滷

際際滷Share a Scribd company logo
4
Most read
6
Most read
Gregory S. Nelson
University of Georgia
PhD, ABD Social Psychology
Duke University  Fuqua
School of Business,
M.S., Clinical Informatics,
Healthcare Analytics
University of California,
Santa Cruz
B.A. Social Psychology,
Quantitative Methods
AVP Analytic Services:
Intermountain Healthcare
VP Analytics & Strategy:
Vidant Health
Adjunct Faculty:
Duke University
Author: The Analytics
Lifecycle Toolkit
Founder and CEO
ThotWave Technologies
Delivering on the Promise of
Responsible AI
Seizing the Opportunities and Managing the Risks of Use of AI in Health IT
An ONC Artificial Intelligence Showcase
Greg Nelson FACHE, MMCi, CHCO, CPHIMS
AVP, Analytics Services  Intermountain Healthcare
Meet Shakira, Age 34
 Shakira presents at an Urgent Care for
skin concern
 Physician findings indicate possible melanoma
 Patient needs dermatology consult ASAP
 Physician fills out referral paper
 Paper gets lost
 Patient care delayed by several months
 Patient finally seen by a dermatologist and
diagnosed with metastatic melanoma
Shakiras Referral Journey Map
Journey mapping is the process of illustrating a complete story about the
relationship  and resulting experience  that unfolds over time between an
individual and a system, service, product, organization
Sentiment
Referral
Specialist
Choice
Recommendation
Importance
Search
Scheduling
Insurance
Path forward
Email
Care Pathway Communication
Optimistic Puzzled Annoyed Forgetful Curiously obligated Irritated Satisfied Delighted
Happy Exasperated Frustrated
AI-Enabled Process
Sentiment
Referral
Specialist
Choice
Recommendation
Importance
Search
Scheduling
Insurance
Path forward
Email
Care Pathway Communication
Next Best
Action
Algorithm Patients-like me
Recommender
engine
AI Enabled
Patient
Matching
Virtual
Assistant
Virtual
Assistant
Follow up
Pre-authorization/
payor confidence
algorithm
Care pathway lane
departure warning
eConsult
Recommender
Generation next ought to be
systems of intention, insight and
cognition  systems that know what
we want, help us see what we
otherwise wouldn't see and help us
think.
Cris Ross
Mayo Clinic
ED Arrivals, Acuity,& Census
Predictions
LWBS Prediction
ED Flow
Optimization
Inpatient Admission Prediction
Short Stay Prediction
Inpatient LOS Prediction
Readmission Prediction
ICU High Cost/LOS
Prediction
ICU Readmission Prediction
Same Day Surgery
Cancellation Prediction
EOL Prediction
Opportunity Landscape
Pre-Diagnosis Acute Care
Well-Care Prevention Chronic Care Palliative Care
What is true what would have
to be true?
Inspired by A Simple Nuance that Produces Great Strategy Discussions by Roger L. Martin
Responsible might mean different things
Proven & Private
Patient
Fair & Ethical
Community
Trustworthy & Involved
Provider
Transparent & Validated
Caregiver
Responsible AI is a framework for deploying artificial intelligence in an ethical,
equitable, and transparent way. Using AI responsibly is the act of making sure we
always couple AI solutions with these key values in order to identify and avoid
unintended consequences.
AI Playbook: Building Trust and AI Literacy
Our AI XLA
Experience Level Agreement
How we are executing against that vision
Community of
Practice
Validation &
Monitoring
Model
Catalog &
Monitor
Vendor
Accountability
AI Literacy
Bias
Detection
Toolkit
Product
Management
Strategic
Development
Engineering A. Intake B. Feasibility Analysis C. Model Development D. Pilot E. Deployment & Sustainment
Perform Data
Identification
Problem
Identification/Req
uirement
Identify Impact
Cohort
Method Proposal &
Review:
 Adoption
Potential
 Plausibility
Is All
Data in
EDW?
Design Data Inflow
Process
Review Existing
Internal Solution
Yes No
Whats the impact
if we take this
action to the
cohort?
Industry & External
Research
Identify Actions
Could Be Taken in
Workflow
Identify Enterprise
Partnership
Conduct Method
Review
Build Baseline
Models and Select
Model
Identify Additional
Research Needs &
Weight Cost vs
Added Benefit
Perform Baseline
Model Validation
Evaluate Pilot KPIs
Conduct Pilot
Ethical AI & Bias
Review (SA-ME)
Design Experiment
Conduct Pilot
Analytics Benefit
Review
Perform Pilot
Model Validation
Deploy Model to
Production
Generate EDC
Documents
Integrate into
Workflow
Conduct Annual
Analytics Benefit
Review
Evaluate Project &
Team Fit
Develop UX/UI
Draft UX/UI Design
Iterative Process
Complete Ethical
AI & Bias Model
Review
Draft Key
Messaging and
Communication
Deploy Model into
Test Environment
Develop Test
Environment for
Workflow
Integration
Define Pilot KPIs
Refine Baseline
Model
Data Science Workflow Version 0.1
Automate Annual
Ethical AI & Bias
Review
Establish Feedback
Loop
Evaluate Workflow
Integration
Potential
Automate Model
Validation
Conduct Annual
Ethical AI & Bias
Review (SA-ME)
Conduct User
Trainings
Conduct User
Interviews
Monitor Key
Metrics
Complete ROI
Assessment
Determine Data
Availability &
Frequency
Compare Initial
Governance and
Compliance
Standard
Perform Data
Wrangling
Draft Pilot Test
Environment and
Workflow
Integration Plan
Build MVP
Propose ETL
Process and
Production
Environment
Evaluate Model
Speed &
Complexity
Conduct Product
Retirement
Assessment
F. Retirement
Sunset
Development
Work
Shut down
Production
Pipeline
Archive
Engineering
Pipeline for Future
Accessibility
Document Product
for Future
Reference
Complete Ethical
Review  intake
form
Sample Assessment
(SA) and Equity
Escalation
(Compliance/Chief
Equity officer)
Model Evaluation
(ME) and Equity
Escalation
Vended Solutions
Proven off-the-shelf solutions
In-House Developed
Improve our internal
capabilities and agility
Collaborations
Build community relationships
to accelerate learning
Our Approach to Responsible AI
14
Question and Answers
@gregorysnelson
linkedin.com/in/gregorysnelson/
greg.nelson@imail.org
919.931.4736
Contact
Ad

Recommended

Spinka pechakucha1
Spinka pechakucha1
Daniel Spinka
Eapp Evaluation
Eapp Evaluation
PGAEAPP
Artificial Intelligence for Societal Impact
Artificial Intelligence for Societal Impact
Amit Sharma
Digital Scholar Webinar: Breaking Down (Brick) Walls: Switching to Remote, Vi...
Digital Scholar Webinar: Breaking Down (Brick) Walls: Switching to Remote, Vi...
SC CTSI at USC and CHLA
Algorithmic Bias: Challenges and Opportunities for AI in Healthcare
Algorithmic Bias: Challenges and Opportunities for AI in Healthcare
Gregory Nelson
Pandemic Tech: How Emerging Technologies Can Mitigate the Impact of Covid-19 ...
Pandemic Tech: How Emerging Technologies Can Mitigate the Impact of Covid-19 ...
Tory Cenaj
Algorithms and bias: What lenders need to know
Algorithms and bias: What lenders need to know
White & Case
Full METL Research to Faster Miracles
Full METL Research to Faster Miracles
Sean Manion PhD
Pbrn digital social media 2
Pbrn digital social media 2
Amanda Ross
Reciprocity ROI, LLC Methods in Healthcare, Robotics and AI
Reciprocity ROI, LLC Methods in Healthcare, Robotics and AI
Paul Claxton
What healthcare executives should know about artificial intelligence
What healthcare executives should know about artificial intelligence
Arlen Meyers, MD, MBA
EY Drug R&D: Big DATA for big returns
EY Drug R&D: Big DATA for big returns
Thomas Wilckens
Ai in healthcare (3)
Ai in healthcare (3)
Nicholas Gormley
But Does It Work? The Critical Role of Evaluation in Digital Health
But Does It Work? The Critical Role of Evaluation in Digital Health
Dustin DiTommaso
Mobile Health Symposium #HIMSS15 Session Mh1
Mobile Health Symposium #HIMSS15 Session Mh1
3GDR
AI for operations CDAO fall 2023.pdf
AI for operations CDAO fall 2023.pdf
Dr. Niren Sirohi
TeraCrunch: Transforming Organizations with Machien Learning and Gen-AI Solut...
TeraCrunch: Transforming Organizations with Machien Learning and Gen-AI Solut...
KC Digital Drive
Sergio Carbone - Gartner MDM Sumit
Sergio Carbone - Gartner MDM Sumit
sccarbone
Strategies for Promoting Innovation in Healthcare Like Akiva Greenfield.pdf
Strategies for Promoting Innovation in Healthcare Like Akiva Greenfield.pdf
akivagreenfieldus
David Champeaux @ChampeauxDavid
David Champeaux @ChampeauxDavid
3GDR
Best AI Healthcare Solution Company - Bluebash
Best AI Healthcare Solution Company - Bluebash
Bluebash
IoHT Tatiana Abreu
IoHT Tatiana Abreu
Tatiana Abreu-Romu
What's your strategy? Getting tactical about being strategic.
What's your strategy? Getting tactical about being strategic.
Doug Hall
From Diagnosis to Delivery: How AI is Revolutionizing the Patient Experience
From Diagnosis to Delivery: How AI is Revolutionizing the Patient Experience
Aggregage
The Pros & Cons of Brexit for UKs Automotive & Healthcare Industries
The Pros & Cons of Brexit for UKs Automotive & Healthcare Industries
RNayak3
Healthcare BPM | Healthcare BPO Companies| Healthcare Outsourcing
Healthcare BPM | Healthcare BPO Companies| Healthcare Outsourcing
RNayak3
Healthcare BPM | Healthcare BPO Services| Healthcare Outsourcing
Healthcare BPM | Healthcare BPO Services| Healthcare Outsourcing
RNayak3
Health care analytics
Health care analytics
Rohit Bisht
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
Kikuyu Kiku
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
Meru Kilonzo

More Related Content

What's hot (7)

Pbrn digital social media 2
Pbrn digital social media 2
Amanda Ross
Reciprocity ROI, LLC Methods in Healthcare, Robotics and AI
Reciprocity ROI, LLC Methods in Healthcare, Robotics and AI
Paul Claxton
What healthcare executives should know about artificial intelligence
What healthcare executives should know about artificial intelligence
Arlen Meyers, MD, MBA
EY Drug R&D: Big DATA for big returns
EY Drug R&D: Big DATA for big returns
Thomas Wilckens
Ai in healthcare (3)
Ai in healthcare (3)
Nicholas Gormley
But Does It Work? The Critical Role of Evaluation in Digital Health
But Does It Work? The Critical Role of Evaluation in Digital Health
Dustin DiTommaso
Mobile Health Symposium #HIMSS15 Session Mh1
Mobile Health Symposium #HIMSS15 Session Mh1
3GDR
Pbrn digital social media 2
Pbrn digital social media 2
Amanda Ross
Reciprocity ROI, LLC Methods in Healthcare, Robotics and AI
Reciprocity ROI, LLC Methods in Healthcare, Robotics and AI
Paul Claxton
What healthcare executives should know about artificial intelligence
What healthcare executives should know about artificial intelligence
Arlen Meyers, MD, MBA
EY Drug R&D: Big DATA for big returns
EY Drug R&D: Big DATA for big returns
Thomas Wilckens
But Does It Work? The Critical Role of Evaluation in Digital Health
But Does It Work? The Critical Role of Evaluation in Digital Health
Dustin DiTommaso
Mobile Health Symposium #HIMSS15 Session Mh1
Mobile Health Symposium #HIMSS15 Session Mh1
3GDR

Similar to Delivering on Responsible AI in Healthcare (20)

AI for operations CDAO fall 2023.pdf
AI for operations CDAO fall 2023.pdf
Dr. Niren Sirohi
TeraCrunch: Transforming Organizations with Machien Learning and Gen-AI Solut...
TeraCrunch: Transforming Organizations with Machien Learning and Gen-AI Solut...
KC Digital Drive
Sergio Carbone - Gartner MDM Sumit
Sergio Carbone - Gartner MDM Sumit
sccarbone
Strategies for Promoting Innovation in Healthcare Like Akiva Greenfield.pdf
Strategies for Promoting Innovation in Healthcare Like Akiva Greenfield.pdf
akivagreenfieldus
David Champeaux @ChampeauxDavid
David Champeaux @ChampeauxDavid
3GDR
Best AI Healthcare Solution Company - Bluebash
Best AI Healthcare Solution Company - Bluebash
Bluebash
IoHT Tatiana Abreu
IoHT Tatiana Abreu
Tatiana Abreu-Romu
What's your strategy? Getting tactical about being strategic.
What's your strategy? Getting tactical about being strategic.
Doug Hall
From Diagnosis to Delivery: How AI is Revolutionizing the Patient Experience
From Diagnosis to Delivery: How AI is Revolutionizing the Patient Experience
Aggregage
The Pros & Cons of Brexit for UKs Automotive & Healthcare Industries
The Pros & Cons of Brexit for UKs Automotive & Healthcare Industries
RNayak3
Healthcare BPM | Healthcare BPO Companies| Healthcare Outsourcing
Healthcare BPM | Healthcare BPO Companies| Healthcare Outsourcing
RNayak3
Healthcare BPM | Healthcare BPO Services| Healthcare Outsourcing
Healthcare BPM | Healthcare BPO Services| Healthcare Outsourcing
RNayak3
Health care analytics
Health care analytics
Rohit Bisht
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
Kikuyu Kiku
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
Meru Kilonzo
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
Kikuyu Kiku
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
Meru Kilonzo
Structured solutions to wicked problems
Structured solutions to wicked problems
Nasir Hussain
2015 Houston CHIME Lead Forum
2015 Houston CHIME Lead Forum
Health IT Conference iHT2
Connecting the Healthcare Ecosystem - An Architecture for Improved Health
Connecting the Healthcare Ecosystem - An Architecture for Improved Health
Prolifics
AI for operations CDAO fall 2023.pdf
AI for operations CDAO fall 2023.pdf
Dr. Niren Sirohi
TeraCrunch: Transforming Organizations with Machien Learning and Gen-AI Solut...
TeraCrunch: Transforming Organizations with Machien Learning and Gen-AI Solut...
KC Digital Drive
Sergio Carbone - Gartner MDM Sumit
Sergio Carbone - Gartner MDM Sumit
sccarbone
Strategies for Promoting Innovation in Healthcare Like Akiva Greenfield.pdf
Strategies for Promoting Innovation in Healthcare Like Akiva Greenfield.pdf
akivagreenfieldus
David Champeaux @ChampeauxDavid
David Champeaux @ChampeauxDavid
3GDR
Best AI Healthcare Solution Company - Bluebash
Best AI Healthcare Solution Company - Bluebash
Bluebash
What's your strategy? Getting tactical about being strategic.
What's your strategy? Getting tactical about being strategic.
Doug Hall
From Diagnosis to Delivery: How AI is Revolutionizing the Patient Experience
From Diagnosis to Delivery: How AI is Revolutionizing the Patient Experience
Aggregage
The Pros & Cons of Brexit for UKs Automotive & Healthcare Industries
The Pros & Cons of Brexit for UKs Automotive & Healthcare Industries
RNayak3
Healthcare BPM | Healthcare BPO Companies| Healthcare Outsourcing
Healthcare BPM | Healthcare BPO Companies| Healthcare Outsourcing
RNayak3
Healthcare BPM | Healthcare BPO Services| Healthcare Outsourcing
Healthcare BPM | Healthcare BPO Services| Healthcare Outsourcing
RNayak3
Health care analytics
Health care analytics
Rohit Bisht
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
Kikuyu Kiku
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
Meru Kilonzo
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
Kikuyu Kiku
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
AI & ML Workshop_Career Launcher_Africare & Lifecare Group.pdf
Meru Kilonzo
Structured solutions to wicked problems
Structured solutions to wicked problems
Nasir Hussain
Connecting the Healthcare Ecosystem - An Architecture for Improved Health
Connecting the Healthcare Ecosystem - An Architecture for Improved Health
Prolifics
Ad

Recently uploaded (20)

Radiological diagnosis of Gastric carcinoma
Radiological diagnosis of Gastric carcinoma
abid hossain
VoyageHealing- ketamine and wellness clinic.pdf
VoyageHealing- ketamine and wellness clinic.pdf
Voyage Healing
palliative medicine palliative medicines
palliative medicine palliative medicines
dramrabdelghaffar87
Deep Vein Thrombosis Treatment in East Brunswick, NJ
Deep Vein Thrombosis Treatment in East Brunswick, NJ
Vein & Vascular Medicine Clinic
Public-Private Partnerships in Healthcare_ A Sustainable Model for African Na...
Public-Private Partnerships in Healthcare_ A Sustainable Model for African Na...
oliverwanyama96
Authorization and medicare_medicare adv plan ppt.pptx
Authorization and medicare_medicare adv plan ppt.pptx
avengurlekar228
Improving access to appropriately fitted equipment to support mothers express...
Improving access to appropriately fitted equipment to support mothers express...
Health Innovation Wessex
Soft Tissue Injury/Handling in orthopedics.
Soft Tissue Injury/Handling in orthopedics.
Dr. Prabhat Pandey
anxiety case presentation in soap format
anxiety case presentation in soap format
sanjaay047
Ayurveda: Ancient Wisdom for Modern Wellness
Ayurveda: Ancient Wisdom for Modern Wellness
Nutrazin
Common Medical Billing Errors.pppptttttx
Common Medical Billing Errors.pppptttttx
johnbutlermbc
LYMPHOMA IN CHILDREN -CHILD HEALTH NURSING.pptx
LYMPHOMA IN CHILDREN -CHILD HEALTH NURSING.pptx
Shibili Abraham
renal failure in children -child health nsg.pptx
renal failure in children -child health nsg.pptx
Shibili Abraham
"Why Am I Not Affected?" Understanding Emotional Detachment
"Why Am I Not Affected?" Understanding Emotional Detachment
Rangen A. Ghafur
Errors that Could be Deemed Fraudulent in Ambulance Billing.pptx
Errors that Could be Deemed Fraudulent in Ambulance Billing.pptx
johnbutlermbc
Dengue Basic Information_Mitigation & Response_Dr. Pankaj Singh
Dengue Basic Information_Mitigation & Response_Dr. Pankaj Singh
panpunpra
Congenital Vertical Talus: Clinical Presentation, Imaging, Etiology, and Mult...
Congenital Vertical Talus: Clinical Presentation, Imaging, Etiology, and Mult...
Dr. Prabhat Pandey
Burns types and its management .pptx
Burns types and its management .pptx
Rishika Rawat
Leading Mammography System Companies Revolutionizing Breast Cancer Detection ...
Leading Mammography System Companies Revolutionizing Breast Cancer Detection ...
ganeshdukare428
Trending-Now_-Ayurvedic-Bone-Regeneration-Therapies-for-AVN-_2025_.ppt
Trending-Now_-Ayurvedic-Bone-Regeneration-Therapies-for-AVN-_2025_.ppt
khephioisols
Radiological diagnosis of Gastric carcinoma
Radiological diagnosis of Gastric carcinoma
abid hossain
VoyageHealing- ketamine and wellness clinic.pdf
VoyageHealing- ketamine and wellness clinic.pdf
Voyage Healing
palliative medicine palliative medicines
palliative medicine palliative medicines
dramrabdelghaffar87
Public-Private Partnerships in Healthcare_ A Sustainable Model for African Na...
Public-Private Partnerships in Healthcare_ A Sustainable Model for African Na...
oliverwanyama96
Authorization and medicare_medicare adv plan ppt.pptx
Authorization and medicare_medicare adv plan ppt.pptx
avengurlekar228
Improving access to appropriately fitted equipment to support mothers express...
Improving access to appropriately fitted equipment to support mothers express...
Health Innovation Wessex
Soft Tissue Injury/Handling in orthopedics.
Soft Tissue Injury/Handling in orthopedics.
Dr. Prabhat Pandey
anxiety case presentation in soap format
anxiety case presentation in soap format
sanjaay047
Ayurveda: Ancient Wisdom for Modern Wellness
Ayurveda: Ancient Wisdom for Modern Wellness
Nutrazin
Common Medical Billing Errors.pppptttttx
Common Medical Billing Errors.pppptttttx
johnbutlermbc
LYMPHOMA IN CHILDREN -CHILD HEALTH NURSING.pptx
LYMPHOMA IN CHILDREN -CHILD HEALTH NURSING.pptx
Shibili Abraham
renal failure in children -child health nsg.pptx
renal failure in children -child health nsg.pptx
Shibili Abraham
"Why Am I Not Affected?" Understanding Emotional Detachment
"Why Am I Not Affected?" Understanding Emotional Detachment
Rangen A. Ghafur
Errors that Could be Deemed Fraudulent in Ambulance Billing.pptx
Errors that Could be Deemed Fraudulent in Ambulance Billing.pptx
johnbutlermbc
Dengue Basic Information_Mitigation & Response_Dr. Pankaj Singh
Dengue Basic Information_Mitigation & Response_Dr. Pankaj Singh
panpunpra
Congenital Vertical Talus: Clinical Presentation, Imaging, Etiology, and Mult...
Congenital Vertical Talus: Clinical Presentation, Imaging, Etiology, and Mult...
Dr. Prabhat Pandey
Burns types and its management .pptx
Burns types and its management .pptx
Rishika Rawat
Leading Mammography System Companies Revolutionizing Breast Cancer Detection ...
Leading Mammography System Companies Revolutionizing Breast Cancer Detection ...
ganeshdukare428
Trending-Now_-Ayurvedic-Bone-Regeneration-Therapies-for-AVN-_2025_.ppt
Trending-Now_-Ayurvedic-Bone-Regeneration-Therapies-for-AVN-_2025_.ppt
khephioisols
Ad

Delivering on Responsible AI in Healthcare

  • 1. Gregory S. Nelson University of Georgia PhD, ABD Social Psychology Duke University Fuqua School of Business, M.S., Clinical Informatics, Healthcare Analytics University of California, Santa Cruz B.A. Social Psychology, Quantitative Methods AVP Analytic Services: Intermountain Healthcare VP Analytics & Strategy: Vidant Health Adjunct Faculty: Duke University Author: The Analytics Lifecycle Toolkit Founder and CEO ThotWave Technologies
  • 2. Delivering on the Promise of Responsible AI Seizing the Opportunities and Managing the Risks of Use of AI in Health IT An ONC Artificial Intelligence Showcase Greg Nelson FACHE, MMCi, CHCO, CPHIMS AVP, Analytics Services Intermountain Healthcare
  • 3. Meet Shakira, Age 34 Shakira presents at an Urgent Care for skin concern Physician findings indicate possible melanoma Patient needs dermatology consult ASAP Physician fills out referral paper Paper gets lost Patient care delayed by several months Patient finally seen by a dermatologist and diagnosed with metastatic melanoma
  • 4. Shakiras Referral Journey Map Journey mapping is the process of illustrating a complete story about the relationship and resulting experience that unfolds over time between an individual and a system, service, product, organization Sentiment Referral Specialist Choice Recommendation Importance Search Scheduling Insurance Path forward Email Care Pathway Communication Optimistic Puzzled Annoyed Forgetful Curiously obligated Irritated Satisfied Delighted Happy Exasperated Frustrated
  • 5. AI-Enabled Process Sentiment Referral Specialist Choice Recommendation Importance Search Scheduling Insurance Path forward Email Care Pathway Communication Next Best Action Algorithm Patients-like me Recommender engine AI Enabled Patient Matching Virtual Assistant Virtual Assistant Follow up Pre-authorization/ payor confidence algorithm Care pathway lane departure warning eConsult Recommender
  • 6. Generation next ought to be systems of intention, insight and cognition systems that know what we want, help us see what we otherwise wouldn't see and help us think. Cris Ross Mayo Clinic
  • 7. ED Arrivals, Acuity,& Census Predictions LWBS Prediction ED Flow Optimization Inpatient Admission Prediction Short Stay Prediction Inpatient LOS Prediction Readmission Prediction ICU High Cost/LOS Prediction ICU Readmission Prediction Same Day Surgery Cancellation Prediction EOL Prediction Opportunity Landscape Pre-Diagnosis Acute Care Well-Care Prevention Chronic Care Palliative Care
  • 8. What is true what would have to be true? Inspired by A Simple Nuance that Produces Great Strategy Discussions by Roger L. Martin
  • 9. Responsible might mean different things Proven & Private Patient Fair & Ethical Community Trustworthy & Involved Provider Transparent & Validated Caregiver Responsible AI is a framework for deploying artificial intelligence in an ethical, equitable, and transparent way. Using AI responsibly is the act of making sure we always couple AI solutions with these key values in order to identify and avoid unintended consequences.
  • 10. AI Playbook: Building Trust and AI Literacy
  • 11. Our AI XLA Experience Level Agreement
  • 12. How we are executing against that vision Community of Practice Validation & Monitoring Model Catalog & Monitor Vendor Accountability AI Literacy Bias Detection Toolkit Product Management
  • 13. Strategic Development Engineering A. Intake B. Feasibility Analysis C. Model Development D. Pilot E. Deployment & Sustainment Perform Data Identification Problem Identification/Req uirement Identify Impact Cohort Method Proposal & Review: Adoption Potential Plausibility Is All Data in EDW? Design Data Inflow Process Review Existing Internal Solution Yes No Whats the impact if we take this action to the cohort? Industry & External Research Identify Actions Could Be Taken in Workflow Identify Enterprise Partnership Conduct Method Review Build Baseline Models and Select Model Identify Additional Research Needs & Weight Cost vs Added Benefit Perform Baseline Model Validation Evaluate Pilot KPIs Conduct Pilot Ethical AI & Bias Review (SA-ME) Design Experiment Conduct Pilot Analytics Benefit Review Perform Pilot Model Validation Deploy Model to Production Generate EDC Documents Integrate into Workflow Conduct Annual Analytics Benefit Review Evaluate Project & Team Fit Develop UX/UI Draft UX/UI Design Iterative Process Complete Ethical AI & Bias Model Review Draft Key Messaging and Communication Deploy Model into Test Environment Develop Test Environment for Workflow Integration Define Pilot KPIs Refine Baseline Model Data Science Workflow Version 0.1 Automate Annual Ethical AI & Bias Review Establish Feedback Loop Evaluate Workflow Integration Potential Automate Model Validation Conduct Annual Ethical AI & Bias Review (SA-ME) Conduct User Trainings Conduct User Interviews Monitor Key Metrics Complete ROI Assessment Determine Data Availability & Frequency Compare Initial Governance and Compliance Standard Perform Data Wrangling Draft Pilot Test Environment and Workflow Integration Plan Build MVP Propose ETL Process and Production Environment Evaluate Model Speed & Complexity Conduct Product Retirement Assessment F. Retirement Sunset Development Work Shut down Production Pipeline Archive Engineering Pipeline for Future Accessibility Document Product for Future Reference Complete Ethical Review intake form Sample Assessment (SA) and Equity Escalation (Compliance/Chief Equity officer) Model Evaluation (ME) and Equity Escalation
  • 14. Vended Solutions Proven off-the-shelf solutions In-House Developed Improve our internal capabilities and agility Collaborations Build community relationships to accelerate learning Our Approach to Responsible AI 14