This document introduces Tolstoy Targets, a visualization method using radial axes to provide a concise summary of multiple objectives or attributes. It discusses principles like using traffic light colors to indicate success or failure of predefined targets. Conventions are outlined, such as grouping attributes by direction and adding confidence ranges. Practical examples demonstrate comparing projects, mass screening of enzymes, transfusion risks for multiple patients, and assessment scores. The document concludes by providing contact information for the author.
1 of 31
Download to read offline
More Related Content
TolstoyTarget,AnimatedExpl,v5
1. Traffic Lights, Targets, & Tolstoy:
Introduction to Tolstoy Targets
Dennis Sweitzer
www.dennis-sweitzer.com
denswei@gmail.com
(Enterprise dashboard
with 4 dredging projects)
4. Testing Drug Izitwerkin: Traffic Light
The predefined objectives
of the study are:
# 1 , #2. #3.. #4.. #5..
For X$ within Y months
Cut to the chase: we know
what are the objectives.
Did we meet them?
We need to make plans
for lunch..
5. Testing Drug Izitwerkin: Green Light
Great, we can quit
early
& celebrate
over lunch
Met all predefined objectives
for efficacy, safety, etc.
And also budget, timelines,.
6. Testing Drug Izitwerkin: Red Light
Clearly failed on predefined
objectives for efficacy, safety,
budget, timeline, etc.
Which failed? .
If its critical, we cancel the rest.
Well commiserate over lunch.
7. Testing Drug Izitwerkin: Yellow Light
Mostly met objectives, but.
Neither clear success
nor clear failure
But what? .
Should we have had more patients?
A little over budget? A little late?
Some bad luck? Unusual circumstances?
What can we salvage?
What do we have to redo?
Get lunch delivered,
its going to be a long day..
8. Targets: Testing Drug Izitwerkin
Multiple Objectives, Multiple Shots:
Timelines Budget
Adverse Events
Abnormal Lab Changes
Pain Relief
Drug Stability
Speed of Relief
Ready..
9. Targets: Aiming for Multiple Objectives
Timelines Budget
Adverse Events
Abnormal Lab Changes
Pain Relief
Drug Stability
Speed of
Relief
FIRE!!!Aim..
10. Targets: Traffic light color code
Timelines Budget
Adverse Events
Abnormal Lab Changes
Pain Relief
Drug Stability
Speed of
Relief
11. Targets: 1 Radial Axis per Objective
Timelines Budget
Adverse Events
Abnormal Lab Changes
Pain Relief
Drug Stability
Speed of
Relief
14. Tolstoy?
Visual, rapid, high level understanding
without having to read & interpret
Click on each target to drill down for details
Happy
families
are all alike;
Every failed project
fails
in its own way.
--Not Tolstoy
Every unhappy
family
is unhappy
in its own way.
--Leo Tolstoy
(Anna Karenina)
Successful projects
are all alike;
16. Convention: The waterline
Above the waterline:
Known unknowns
Things we know
we dont know
Eg, efficacy
Things to get us off the ground
(MUST have some Green)
Below the waterline:
Unknown unknowns
Things we dont know
we dont know
Eg, safety
Things to sink us
(MUST NOT have any Red)
18. Adding Confidence/Certainty Ranges
Elicited opinions,
Statistical Calculations
Best Case
/Worst Case
Hi/Med/Low
Uncertainty
Best likely
/ Worst Likely
Optimistic
/ Pessimistic
Estimates
+ 95% Confidence Intervals
19. Piling on the Symbols
Circles
and pluses
and Xes,
oh my!
Perhaps:
Splats are point estimates
O Past Estimates
+ Optimal (nice to have)?
X The competition?
20. Extreme Example (24 axes x 6 values)
Complex but
Interpretable
Additional
Symbols take
some effort
Splats arent
bad
Use sparingly
Point estimates
Range
O Past Estimates
+ Optimal
X The competition
22. Comparing Projects in the ABC program
At a
glance,
can see
successes
& failures!
--And
Where!
Study ABC-OhNoStudy ABC-GoGo
Study ABC-GoSlo Study ABC-NoNo
23. Larger project with some issues
apparently including damaged
equipment
Big Picture: Multiple Project Dashboard
(Randomly generated dredging examples)
Small project
with many
problems, but on
schedule and in
budget
Larger project with a
couple of possible
problems, but overall
doing well
Metrics for Dredging Projects
& Summary over all Projects
Bad weather, so a
little behind
schedule,
but in budget
Bars indicate range of metric over all projects
24. Mass Screening of Evolving Enzymes
Criteria:
At each step:
Pick best candidates for each criteria
Recombine those to generate new candidates
Repeat until optimal
Alkalinity, Acidity, Yield, Salinity, Metal Tolerance,
Durability, Km (Michaelsons constant)
T.Targets provide comprehensive & visual
feedback on process
GenerationX
Generation1
25. Ex. Transfusion Risks
Whole plot
One Patient
Each Radial Axis
Blood type Group
Each Dot
Patient Risk
Green Dot Low Risk
Normal procedures
Red Dot High Risk
Clear & identified risk
Special Procedures
Yellow Dot Uncertain Risk
Moderate Risks
Caution
Further testing?
No Dot No Information
臓No confusion with Low Risk
!
26. Transfusion Risk of Multiple Patients
Risks for 16 patients
Each Legible
3 Redundancies:
Color Traffic-Light
coding
{Red, Yellow, Green}
Location Targets
Red on Rim, next to label
Easier to identify
Size Proportional to Risk
The Tolstoy principal:
"All happy families are alike;
Every unhappy family is unhappy
in it's own way. (Anna Karenina)
Low Risk patients look alike
High Risk patients are distinct!
What way & How much
27. Predicted Risks vs Outcomes
Unclear connections between
Risks & Outcomes
Add feedback on outcome
Some risks may have a
stronger connection with bad
outcomes
Circle the target
(Same conventions)
Good Outcome Thin Green
Mixed outcome Yellow
Bad outcome Thick Red
Unknown outcome No Circle
Example:
One Bad Outcome
Patient with 2 risk factors
Mostly Good Outcomes
Even with Risk Factors
28. Assessment Scores
For 12 patients
Factor scores from
3 assessments
Mania,
Depression,
Schizophrenia
3 Total Scores
+ 11 subscales
31. For more information, see my website:
www.dennis-sweitzer.com
My linked-In profile:
http://www.linkedin.com/in/dennissweitzer
Or email me:
denswei@gmail.com
Editor's Notes
#5: The History of every major Galactic Civilization tends to pass through three distinct and recognizable phases, those of Survival, Inquiry and Sophistication, otherwise known as the How, Why, and Where phases. For instance, the first phase is characterized by the question 'How can we eat?' the second by the question 'Why do we eat?' and the third by the question 'Where shall we have lunch?
#20: Mutated from WLGores management principals.
Plimsoll mark
Above & below the waterline is easy.
Is there any other ways to categorize? (say, left-right)
because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns - - the ones we don't know we don't know. And if one looks throughout the history of our country and other free countries, it is the latter category that tend to be the difficult ones. Donald Rumsfield
Plimsoll mark
Above & below the waterline is easy.
Is there any other ways to categorize? (say, left-right)
#23: These were just convenient to do in Excel.
Additional elements should be added sparingly
#24: Very sparingly, as appropriate:
Say, Os for last budget & Timelines, Xs for competitors in select outcomes +s for ????
For epidemiology:
Os might be observed parameters (infectivity, etc) for isolated cases,
Splats & bars might be
#26: Abc-GoGo is a happy family: objectives were met.
Abc-GoSlo is pretty average. Got some issues
Abc-OhNo has some problems in some areas
Abc-NoNo has lots of problems
#28: Evolving Enzymes
Must meet criteria for
Durability, Metal Tolerance, Salinity, Yield
Seek best criteria for
Acidity, Alkalinity, Km (Michaelsons constant)
#29: We had brainstormed about blood bank uses of the plot on my Business card, so I made a few to see how they would look. Attached are 3 4 5 figures, one with a single plot, two with 16, and one with 100. (here I explored about how to use them for transfusion risk, but they can be used in many ways)The basic plot for a transfusion candidate (below) has blood group typing systems on radial axes, with ABO and Rh on the horizontal axes for easy identifiably, less frequent ones below, and more frequent ones above. The interpretation of this is that: Green dots (toward the center) indicate normal procedures can be used with low risk to the patient, Red dots (outermost) indicate that special handling is required because of a clear & identified risk to the patients (say, known to be heterozygous), and Yellow dots (middle band) indicate there are risk factors (say, the patient is part of an at-risk ethnic group). NB: the plot would change for a patient as more information became known. An Rh- patient might automatically be in the yellow, and an Rh- patient who has been exposed to Rh antigens might automatically be classified in the red. There is also different levels within each band (3 risk bands x 3 levels each). For instance, on the ABO axis, an AB patient would be on the innermost level because they are "universal recipients", while an O patient would be on the outermost green level, an A or B type would be on the middle level, but an O patient exposed to A or B antigens might go into the Red. The 9 levels are relatively intuitive since people are conditioned to rate things on a scale of 0 to 10. I usually ask people to imagine the best possible outcome (score=0), the worst possible outcome (score = 0), where the subject falls among Good, Bad, and Mediocre (eg, Low, High, and Medium risk), and then how do they compare to others within the color category (so an extremely Low risk patient might score =1, while a relatively Low risk patient might score=3). Note that the size of the dot increases with the level of risk, so there is 3 redundancies in the ranking (color, location, and size) that make it very easy to rapidly interpret the plots: the color (Red, Yellow, Green, just like a traffic light); the distance from the center (like a target), and the size. Consequently, the highest risks cases are the most obvious, low risk cases are clearly seen as such, and there is little risk of confusing a lack of data (no dot) with a low risk case (small green dot). The graph also follows the Tolstoy principal: "All happy families are alike; Every unhappy family is unhappy in it's own way." (the 1st line of Anna Karenina). Panels of targets can present a rapid overview for many subjects, with the goal of identifying subjects that need special attention. The figure with 16 targets is fairly readable: you can still read off the blood group typing on the labels; in the figure with 100 targets, you can clearly see the category reds & yellows, and can still distinguish the Rh & ABO axes because they are horizontal. I envision a data display in which one clicks on a target out of 100's to drill down for a detailed examination. (It could also automatically select the high risk ones by various criteria).An attending physician might only need see the plot for an individual patient to get an immediate & intuitive grasp of there risks. The floor nurse might review a panel of all patients on the floor to identify those needing special attention; An administrator might review panels of dozens for QC purposes. The last figure includes the outcomes of the transfusion, in order to complete the picture of the represented cases. This might be used to refine the risk assessment process itself (perhaps a green dot should have been red or yellow), or to identify procedural problems (did the staff follow the proper procedures for a yellow dot case? Or was there a flaw in the procedures?). The figure follows the same conventions as the rest of the plot: a thin green line indicates a good outcome; a thick red one, bad; a yellow circle, a mixed outcome; no circle, no outcome. Additional information can be included on the plot, such a a solid line indicating the range of uncertainty in an estimate, circles for the last prior risk assessment (if it has changed), etc. This plot can be used in many other ways: For instance, in a plot of blood product supplies in the entire blood bank the axes could represent blood types & products, the dots represent current stocks, an overlaid circle could indicate predicted future stocks (perhaps several weeks ahead, taking into account expected needs & supplies), and confidence limits as an overlaid solid bar indicating the range of uncertainty in predicted future stocks (perhaps from simulations based on historical data. In this use, if a confidence limit enters a red band, contingency plans might be implemented to reduce that risk).
#30: I envision a data display in which one clicks on a target out of 100's to drill down for a detailed examination. (It could also automatically select the high risk ones by various criteria).An attending physician might only need see the plot for an individual patient to get an immediate & intuitive grasp of there risks. The floor nurse might review a panel of all patients on the floor to identify those needing special attention; An administrator might review panels of dozens for QC purposes. The last figure includes the outcomes of the transfusion, in order to complete the picture of the represented cases. This might be used to refine the risk assessment process itself (perhaps a green dot should have been red or yellow), or to identify procedural problems (did the staff follow the proper procedures for a yellow dot case? Or was there a flaw in the procedures?). The figure follows the same conventions as the rest of the plot: a thin green line indicates a good outcome; a thick red one, bad; a yellow circle, a mixed outcome; no circle, no outcome. Additional information can be included on the plot, such a a solid line indicating the range of uncertainty in an estimate, circles for the last prior risk assessment (if it has changed), etc. This plot can be used in many other ways: For instance, in a plot of blood product supplies in the entire blood bank the axes could represent blood types & products, the dots represent current stocks, an overlaid circle could indicate predicted future stocks (perhaps several weeks ahead, taking into account expected needs & supplies), and confidence limits as an overlaid solid bar indicating the range of uncertainty in predicted future stocks (perhaps from simulations based on historical data. In this use, if a confidence limit enters a red band, contingency plans might be implemented to reduce that risk).
#31: The last figure includes the outcomes of the transfusion, in order to complete the picture of the represented cases. This might be used to refine the risk assessment process itself (perhaps a green dot should have been red or yellow), or to identify procedural problems (did the staff follow the proper procedures for a yellow dot case? Or was there a flaw in the procedures?). The figure follows the same conventions as the rest of the plot: a thin green line indicates a good outcome; a thick red one, bad; a yellow circle, a mixed outcome; no circle, no outcome.
Row1, Col3: bad outcome in a patient with 2 red risks. Are either of those 2 risks particularly associated a bad outcome?* the upper right risk (red) also occurred in 2 other patients with good outcomes (row1, cols 1&2)* the upper left red risk occurred in 1 other patient with a mixed outcome (42, c2).
these 2 risk factors might be synergistic