際際滷

際際滷Share a Scribd company logo
Visualization of
Complex Systems
Chris Laffra
1989 - 2019
Who is Chris La鍖ra?
Chris works at Uber Amsterdam, since Jan 2018.
Chris has worked in Tech at IBM Research, Morgan Stanley,
IBM Rational, Bank of America, and Google. At Uber, Chris
works on tools that increase Ubers ability to roll out software
faster and more frequently.
Chris has spent the last 30 years of his life on a mission to
make other engineers more productive.
See chrislaffra.com for more details.
Goals
This is an overview of my career. Over the last 30 years,
I have spent most of my technical career towards
understanding, profiling, and visualizing complex systems.
System
Related entities forming a unified whole
First, what is a system?
Complex
No centrally defined interactions
Consisting of many parts
Complicated
Second, what is complexity?
Cynefin framework
Images, diagrams, or animations
communicating a message.
A picture is worth a thousand words
Visualization
Finally, what is visualization?
PROCOL
How I got started, by visualizing the execution of applications
written in PROCOL, my PhD thesis language.
Book #1 Book #2
Explore the
heap of objects,
starting with the
main object.
Understand the
shape of an object
See live calls to
object methods
And argument
values
1989
The goal: Understand
what is happening and
see surprises.
HotWire
My first job after my PhD. Developed a system for custom
visualizations for C++ and Smalltalk.
the
subject
Showing
live objects,
methods, &
patterns
Original source code
Custom, declarative
visualization script.
The result
1994
JVM
As soon as Java came out, I wanted to learn more about it.
So, I ended up writing book the fourth book that was
published on Java while working at Morgan Stanley.
Book #3
At IBM Research, I worked on Jax, which used JikesBT, that I developed, to analyze whole
program call graphs and optimize Java programs for size. For example, we reduced javac to
half its size. Aside from being smaller, the secondary benefit was that applets loaded faster. 1999
2001
I wrote the first AdBlock, 8 years before the actual AdBlock was launched. I instrumented the
entire IE Java runtime, using JikesBT to do bytecode instrumentation, and added specific
hooks for networking, general profiling, and ability to stop/start/kill any applet remotely.
Eclipse
Book #4
My IBM/OTI team in Amsterdam developed the first commercial IBM product
based on Eclipse, focusing on embedded Java. We constantly worried about size
and performance. The learnings from Jax were applied to make embedded Java
apps as small as possible. I also started looking into performance of larger Java
programs, such as Eclipse itself. To better understand how Eclipse works, I signed
up to co-author a book on Eclipse and wrote numerous profilers.
One experiment was a
heap-exploration tool, in
this case showing all the
objects in Eclipse
pointing to a given
editor.
2004
A custom CPU
and memory
analysis tool I
developed for
Eclipse was
XRay. The goal
was to show
patterns and for
engineers to
discover
surprising
patterns related
to performance.
2004
Other tools, such as Eclipse Universe, offered novel structural communication patterns
between Eclipse plugins, allowing engineers to dive deeper into specific scenarios. It showed
domain-specific issues, in addition to resource management issues and challenges. 2004
Finally, working with IBM Research, I devised novel way to package up Eclipse applications
using streaming delivery, combined with introducing memory-mapped shared classes using the
J9 JXE format. This allowed us to save about  in startup time and memory used. 2005
WebSphere VMs per Node
Normal JXEs
My impact.
2X scaling.
Python
Book #5
At Bank of America, I was the first hire on the Quartz project. My interest on this
project was to increase other peoples understanding in how the platform was
structured and implemented. I developed a live-coding training system and
contributed to coding workflows, training, and general developer advocacy.
Quartz
Academy
Chris Laffra
Bank of America
2012
At BofA, I developed a
live coding system
similar to Jupyter
notebooks.
A novel invention I
added was the ability
to do time-travel
debugging of generic
Python code, but also
algorithms that were
implemented using
Quartz features.
I also wrote all the
training materials,
published internally.
Still at BofA, I
developed repository of
>50 Computer Science
algorithms with
visualization.
This essentially was a
reimplementation of
Hotwire for Python.
This was implemented
using Google
AppEngine and cool
Python execution.
2013
code test
2016
The goal of Auger was to introspect a running system, record everything, and then produce
specific output that uses all the data being recorded. In this case I generate unit tests. But,
another application could be to automatically generate performance metrics, or derive types.
Pava
Pava takes Java
bytecodes and
converts them to
Python source code.
Not sure how useful
Pava really is, but for
me, this was really an
effort to study the
differences in
bytecodes for Java
and Python.
2017
Ikke
HappyMac
Tempo
At Google, I worked a lot on Java and Android tools and apps. I also developed an
interest in making the life of engineers better by providing them with tools to
understand how they fit into their organisation and how they can profile their life.
2016-2018
People also represent
a complex system and
I played a lot with
different visualizations
of my personal and
work graph to discover
human connections.
github: Ikke Graph.
While working at Uber, my MacBook was always struggling with the CPU usage of Intellij and
the Android Emulator. To understand better what was going on, I created HappyMac, which
allows its user to discover expensive processes and auto-suspend them using some rules. 2018
HappyWeb is like HappyMac, but focuses on slow network connections. 2018
Tempo constantly
watches your
MacBook and
records the
currently active
program and saves
an event in a local
SQL database.
The recorded
events can show
effectiveness of
workflows, how
much time is spent
in certain activities,
and how to avoid
context switches.
A human profiler?
2019
Summary
Complex systems are all around us, either as technical solutions or as connections between humans. In
order to improve these complex systems, we need to understand them better. To understand them better,
we need to measure and generate metrics. Custom tools will then provide insights.
Visualization of Complex Systems
The End
Be humble, we have small brains.
Ad

Recommended

犖÷顕犖犖犖犖迦犖犖迦犖朽犢犖ム鍵犖犖伍犖о幻犖犖巌硯犖巌犖迦犖朽 犖犖迦犖迦硯犖巌犖迦犖朽犖犖伍犖犖迦見犖犖犖犖÷犖迦牽犖犖ム鹸犖犢犖÷犖犖巌検犖犢 犖犖迦牽犖迦犢犖犖...
犖÷顕犖犖犖犖迦犖犖迦犖朽犢犖ム鍵犖犖伍犖о幻犖犖巌硯犖巌犖迦犖朽 犖犖迦犖迦硯犖巌犖迦犖朽犖犖伍犖犖迦見犖犖犖犖÷犖迦牽犖犖ム鹸犖犢犖÷犖犖巌検犖犢 犖犖迦牽犖迦犢犖犖...
Totsaporn Inthanin
閣喝鰻庄界看鉛温庄界庄顎界喝京温噛艶鉛艶喝掘稼糸看糸看稼釘庄艶庄喝永姻温界岳庄界艶喝檎或珂(1).沿糸韓
閣喝鰻庄界看鉛温庄界庄顎界喝京温噛艶鉛艶喝掘稼糸看糸看稼釘庄艶庄喝永姻温界岳庄界艶喝檎或珂(1).沿糸韓
miau27
犢犖犖犖о顕犖÷牽犖項犖犖朽 5 犖犖迦牽犢犖犖朽権犖犖犖園犖犖迦犢犖犖犢犖犖犖
犢犖犖犖о顕犖÷牽犖項犖犖朽 5 犖犖迦牽犢犖犖朽権犖犖犖園犖犖迦犢犖犖犢犖犖犖
dechathon
犖犖園姑┯犖伍犖犖犖
犖犖園姑┯犖伍犖犖犖
supreechafkk
curriculo-sandro-en
curriculo-sandro-en
Sandro Bihaiko
Javaland keynote final
Javaland keynote final
Marcus Lagergren
jlettvin.resume.20160922.STAR
jlettvin.resume.20160922.STAR
Jonathan Lettvin
Mastering AngularJS for NET Developers 1st Edition Majid Mohammad Wadood Mirz...
Mastering AngularJS for NET Developers 1st Edition Majid Mohammad Wadood Mirz...
kisoravalan2
Open Source Software & the Power of Free
Open Source Software & the Power of Free
Miguel Hernandez
Charlas CityJS, una por una, atestiguando el crecimiento tech
Charlas CityJS, una por una, atestiguando el crecimiento tech
Eligreg L坦pez
SOFTORIX Project TitlesList
SOFTORIX Project TitlesList
guest6071d14
Hand crafted resume (2-page PDF)
Hand crafted resume (2-page PDF)
cies010
Evolution of Operating System and its Role in Robotics.pptx
Evolution of Operating System and its Role in Robotics.pptx
avishek.choudhuri
How to move that pixel?
How to move that pixel?
Marcin Ignac
Data structures and_algorithms_in_java
Data structures and_algorithms_in_java
Prabhu vip
RESUME_RAVI
RESUME_RAVI
Ravi Godugu
Ron Newman Resume T
Ron Newman Resume T
ronman2
Shut Up And Eat Your Veg
Shut Up And Eat Your Veg
Garth Gilmour
Jason_Tiscione
Jason_Tiscione
Jason Tiscione
20100423sage
20100423sage
Jeff Hammerbacher
Curriculum Vitae - Dimitrios Panagiotou - Senior Software Engineer
Curriculum Vitae - Dimitrios Panagiotou - Senior Software Engineer
dpanag1979
Training report
Training report
Bhaskar Mangal
An Infographic on My 25 Articles in Open Source For You Magazine
An Infographic on My 25 Articles in Open Source For You Magazine
Dr. K.S. Kuppu Samy
Top 5 Software Development Jobs In Trending
Top 5 Software Development Jobs In Trending
Myjobspace
CV - Resume
CV - Resume
Francesco
Open source for_you__july_2017
Open source for_you__july_2017
B狸nh Phan
Essential programmer library
Essential programmer library
Andrew Hunt
Develer offering for Qt
Develer offering for Qt
QT-day
Java bytecode hacking
Java bytecode hacking
Chris Laffra
Project Cacophonia
Project Cacophonia
Chris Laffra

More Related Content

Similar to Visualization of Complex Systems (20)

Open Source Software & the Power of Free
Open Source Software & the Power of Free
Miguel Hernandez
Charlas CityJS, una por una, atestiguando el crecimiento tech
Charlas CityJS, una por una, atestiguando el crecimiento tech
Eligreg L坦pez
SOFTORIX Project TitlesList
SOFTORIX Project TitlesList
guest6071d14
Hand crafted resume (2-page PDF)
Hand crafted resume (2-page PDF)
cies010
Evolution of Operating System and its Role in Robotics.pptx
Evolution of Operating System and its Role in Robotics.pptx
avishek.choudhuri
How to move that pixel?
How to move that pixel?
Marcin Ignac
Data structures and_algorithms_in_java
Data structures and_algorithms_in_java
Prabhu vip
RESUME_RAVI
RESUME_RAVI
Ravi Godugu
Ron Newman Resume T
Ron Newman Resume T
ronman2
Shut Up And Eat Your Veg
Shut Up And Eat Your Veg
Garth Gilmour
Jason_Tiscione
Jason_Tiscione
Jason Tiscione
20100423sage
20100423sage
Jeff Hammerbacher
Curriculum Vitae - Dimitrios Panagiotou - Senior Software Engineer
Curriculum Vitae - Dimitrios Panagiotou - Senior Software Engineer
dpanag1979
Training report
Training report
Bhaskar Mangal
An Infographic on My 25 Articles in Open Source For You Magazine
An Infographic on My 25 Articles in Open Source For You Magazine
Dr. K.S. Kuppu Samy
Top 5 Software Development Jobs In Trending
Top 5 Software Development Jobs In Trending
Myjobspace
CV - Resume
CV - Resume
Francesco
Open source for_you__july_2017
Open source for_you__july_2017
B狸nh Phan
Essential programmer library
Essential programmer library
Andrew Hunt
Develer offering for Qt
Develer offering for Qt
QT-day
Open Source Software & the Power of Free
Open Source Software & the Power of Free
Miguel Hernandez
Charlas CityJS, una por una, atestiguando el crecimiento tech
Charlas CityJS, una por una, atestiguando el crecimiento tech
Eligreg L坦pez
SOFTORIX Project TitlesList
SOFTORIX Project TitlesList
guest6071d14
Hand crafted resume (2-page PDF)
Hand crafted resume (2-page PDF)
cies010
Evolution of Operating System and its Role in Robotics.pptx
Evolution of Operating System and its Role in Robotics.pptx
avishek.choudhuri
How to move that pixel?
How to move that pixel?
Marcin Ignac
Data structures and_algorithms_in_java
Data structures and_algorithms_in_java
Prabhu vip
Ron Newman Resume T
Ron Newman Resume T
ronman2
Shut Up And Eat Your Veg
Shut Up And Eat Your Veg
Garth Gilmour
Curriculum Vitae - Dimitrios Panagiotou - Senior Software Engineer
Curriculum Vitae - Dimitrios Panagiotou - Senior Software Engineer
dpanag1979
An Infographic on My 25 Articles in Open Source For You Magazine
An Infographic on My 25 Articles in Open Source For You Magazine
Dr. K.S. Kuppu Samy
Top 5 Software Development Jobs In Trending
Top 5 Software Development Jobs In Trending
Myjobspace
CV - Resume
CV - Resume
Francesco
Open source for_you__july_2017
Open source for_you__july_2017
B狸nh Phan
Essential programmer library
Essential programmer library
Andrew Hunt
Develer offering for Qt
Develer offering for Qt
QT-day

More from Chris Laffra (7)

Java bytecode hacking
Java bytecode hacking
Chris Laffra
Project Cacophonia
Project Cacophonia
Chris Laffra
Productivity and happiness
Productivity and happiness
Chris Laffra
Eclipse Visualization and Performance Monitoring
Eclipse Visualization and Performance Monitoring
Chris Laffra
Jax retrospective
Jax retrospective
Chris Laffra
Curry on/Ecoop/ISSTA 2018 report
Curry on/Ecoop/ISSTA 2018 report
Chris Laffra
Little languages
Little languages
Chris Laffra
Java bytecode hacking
Java bytecode hacking
Chris Laffra
Project Cacophonia
Project Cacophonia
Chris Laffra
Productivity and happiness
Productivity and happiness
Chris Laffra
Eclipse Visualization and Performance Monitoring
Eclipse Visualization and Performance Monitoring
Chris Laffra
Jax retrospective
Jax retrospective
Chris Laffra
Curry on/Ecoop/ISSTA 2018 report
Curry on/Ecoop/ISSTA 2018 report
Chris Laffra
Little languages
Little languages
Chris Laffra
Ad

Recently uploaded (20)

Who will create the languages of the future?
Who will create the languages of the future?
Jordi Cabot
The Anti-Masterclass Live - Peak of Data & AI 2025
The Anti-Masterclass Live - Peak of Data & AI 2025
Safe Software
Modern Platform Engineering with Choreo - The AI-Native Internal Developer Pl...
Modern Platform Engineering with Choreo - The AI-Native Internal Developer Pl...
WSO2
arctitecture application system design os dsa
arctitecture application system design os dsa
za241967
Azure AI Foundry: The AI app and agent factory
Azure AI Foundry: The AI app and agent factory
Maxim Salnikov
IDM Crack with Internet Download Manager 6.42 Build 41 [Latest 2025]
IDM Crack with Internet Download Manager 6.42 Build 41 [Latest 2025]
pcprocore
NVIDIA GPU Technologies for AI and High-Performance Computing
NVIDIA GPU Technologies for AI and High-Performance Computing
SandeepKS52
Key Challenges in Troubleshooting Customer On-Premise Applications
Key Challenges in Troubleshooting Customer On-Premise Applications
Tier1 app
Advance Doctor Appointment Booking App With Online Payment
Advance Doctor Appointment Booking App With Online Payment
AxisTechnolabs
Zonerankers Digital marketing solutions
Zonerankers Digital marketing solutions
reenashriee
Folding Cheat Sheet # 9 - List Unfolding as the Computational Dual of ...
Folding Cheat Sheet # 9 - List Unfolding as the Computational Dual of ...
Philip Schwarz
Porting Qt 5 QML Modules to Qt 6 Webinar
Porting Qt 5 QML Modules to Qt 6 Webinar
ICS
Making significant Software Architecture decisions
Making significant Software Architecture decisions
Bert Jan Schrijver
Streamlining CI/CD with FME Flow: A Practical Guide
Streamlining CI/CD with FME Flow: A Practical Guide
Safe Software
HPE Reseller in uae by numerosystom.pptx
HPE Reseller in uae by numerosystom.pptx
aadibva452
Milwaukee Marketo User Group June 2025 - Optimize and Enhance Efficiency - Sm...
Milwaukee Marketo User Group June 2025 - Optimize and Enhance Efficiency - Sm...
BradBedford3
Simplify Task, Team, and Project Management with Orangescrum Work
Simplify Task, Team, and Project Management with Orangescrum Work
Orangescrum
SAP PM Module Level-IV Training Complete.ppt
SAP PM Module Level-IV Training Complete.ppt
MuhammadShaheryar36
Download Adobe Illustrator Crack free for Windows 2025?
Download Adobe Illustrator Crack free for Windows 2025?
grete1122g
Heat Treatment Process Automation in India
Heat Treatment Process Automation in India
Reckers Mechatronics
Who will create the languages of the future?
Who will create the languages of the future?
Jordi Cabot
The Anti-Masterclass Live - Peak of Data & AI 2025
The Anti-Masterclass Live - Peak of Data & AI 2025
Safe Software
Modern Platform Engineering with Choreo - The AI-Native Internal Developer Pl...
Modern Platform Engineering with Choreo - The AI-Native Internal Developer Pl...
WSO2
arctitecture application system design os dsa
arctitecture application system design os dsa
za241967
Azure AI Foundry: The AI app and agent factory
Azure AI Foundry: The AI app and agent factory
Maxim Salnikov
IDM Crack with Internet Download Manager 6.42 Build 41 [Latest 2025]
IDM Crack with Internet Download Manager 6.42 Build 41 [Latest 2025]
pcprocore
NVIDIA GPU Technologies for AI and High-Performance Computing
NVIDIA GPU Technologies for AI and High-Performance Computing
SandeepKS52
Key Challenges in Troubleshooting Customer On-Premise Applications
Key Challenges in Troubleshooting Customer On-Premise Applications
Tier1 app
Advance Doctor Appointment Booking App With Online Payment
Advance Doctor Appointment Booking App With Online Payment
AxisTechnolabs
Zonerankers Digital marketing solutions
Zonerankers Digital marketing solutions
reenashriee
Folding Cheat Sheet # 9 - List Unfolding as the Computational Dual of ...
Folding Cheat Sheet # 9 - List Unfolding as the Computational Dual of ...
Philip Schwarz
Porting Qt 5 QML Modules to Qt 6 Webinar
Porting Qt 5 QML Modules to Qt 6 Webinar
ICS
Making significant Software Architecture decisions
Making significant Software Architecture decisions
Bert Jan Schrijver
Streamlining CI/CD with FME Flow: A Practical Guide
Streamlining CI/CD with FME Flow: A Practical Guide
Safe Software
HPE Reseller in uae by numerosystom.pptx
HPE Reseller in uae by numerosystom.pptx
aadibva452
Milwaukee Marketo User Group June 2025 - Optimize and Enhance Efficiency - Sm...
Milwaukee Marketo User Group June 2025 - Optimize and Enhance Efficiency - Sm...
BradBedford3
Simplify Task, Team, and Project Management with Orangescrum Work
Simplify Task, Team, and Project Management with Orangescrum Work
Orangescrum
SAP PM Module Level-IV Training Complete.ppt
SAP PM Module Level-IV Training Complete.ppt
MuhammadShaheryar36
Download Adobe Illustrator Crack free for Windows 2025?
Download Adobe Illustrator Crack free for Windows 2025?
grete1122g
Heat Treatment Process Automation in India
Heat Treatment Process Automation in India
Reckers Mechatronics
Ad

Visualization of Complex Systems

  • 2. Who is Chris La鍖ra? Chris works at Uber Amsterdam, since Jan 2018. Chris has worked in Tech at IBM Research, Morgan Stanley, IBM Rational, Bank of America, and Google. At Uber, Chris works on tools that increase Ubers ability to roll out software faster and more frequently. Chris has spent the last 30 years of his life on a mission to make other engineers more productive. See chrislaffra.com for more details.
  • 3. Goals This is an overview of my career. Over the last 30 years, I have spent most of my technical career towards understanding, profiling, and visualizing complex systems.
  • 4. System Related entities forming a unified whole First, what is a system?
  • 5. Complex No centrally defined interactions Consisting of many parts Complicated Second, what is complexity?
  • 7. Images, diagrams, or animations communicating a message. A picture is worth a thousand words Visualization Finally, what is visualization?
  • 8. PROCOL How I got started, by visualizing the execution of applications written in PROCOL, my PhD thesis language. Book #1 Book #2
  • 9. Explore the heap of objects, starting with the main object. Understand the shape of an object See live calls to object methods And argument values 1989 The goal: Understand what is happening and see surprises.
  • 10. HotWire My first job after my PhD. Developed a system for custom visualizations for C++ and Smalltalk.
  • 12. Original source code Custom, declarative visualization script. The result 1994
  • 13. JVM As soon as Java came out, I wanted to learn more about it. So, I ended up writing book the fourth book that was published on Java while working at Morgan Stanley. Book #3
  • 14. At IBM Research, I worked on Jax, which used JikesBT, that I developed, to analyze whole program call graphs and optimize Java programs for size. For example, we reduced javac to half its size. Aside from being smaller, the secondary benefit was that applets loaded faster. 1999
  • 15. 2001 I wrote the first AdBlock, 8 years before the actual AdBlock was launched. I instrumented the entire IE Java runtime, using JikesBT to do bytecode instrumentation, and added specific hooks for networking, general profiling, and ability to stop/start/kill any applet remotely.
  • 16. Eclipse Book #4 My IBM/OTI team in Amsterdam developed the first commercial IBM product based on Eclipse, focusing on embedded Java. We constantly worried about size and performance. The learnings from Jax were applied to make embedded Java apps as small as possible. I also started looking into performance of larger Java programs, such as Eclipse itself. To better understand how Eclipse works, I signed up to co-author a book on Eclipse and wrote numerous profilers.
  • 17. One experiment was a heap-exploration tool, in this case showing all the objects in Eclipse pointing to a given editor. 2004
  • 18. A custom CPU and memory analysis tool I developed for Eclipse was XRay. The goal was to show patterns and for engineers to discover surprising patterns related to performance. 2004
  • 19. Other tools, such as Eclipse Universe, offered novel structural communication patterns between Eclipse plugins, allowing engineers to dive deeper into specific scenarios. It showed domain-specific issues, in addition to resource management issues and challenges. 2004
  • 20. Finally, working with IBM Research, I devised novel way to package up Eclipse applications using streaming delivery, combined with introducing memory-mapped shared classes using the J9 JXE format. This allowed us to save about in startup time and memory used. 2005 WebSphere VMs per Node Normal JXEs My impact. 2X scaling.
  • 21. Python Book #5 At Bank of America, I was the first hire on the Quartz project. My interest on this project was to increase other peoples understanding in how the platform was structured and implemented. I developed a live-coding training system and contributed to coding workflows, training, and general developer advocacy. Quartz Academy Chris Laffra Bank of America
  • 22. 2012 At BofA, I developed a live coding system similar to Jupyter notebooks. A novel invention I added was the ability to do time-travel debugging of generic Python code, but also algorithms that were implemented using Quartz features. I also wrote all the training materials, published internally.
  • 23. Still at BofA, I developed repository of >50 Computer Science algorithms with visualization. This essentially was a reimplementation of Hotwire for Python. This was implemented using Google AppEngine and cool Python execution. 2013
  • 24. code test 2016 The goal of Auger was to introspect a running system, record everything, and then produce specific output that uses all the data being recorded. In this case I generate unit tests. But, another application could be to automatically generate performance metrics, or derive types.
  • 25. Pava Pava takes Java bytecodes and converts them to Python source code. Not sure how useful Pava really is, but for me, this was really an effort to study the differences in bytecodes for Java and Python. 2017
  • 26. Ikke HappyMac Tempo At Google, I worked a lot on Java and Android tools and apps. I also developed an interest in making the life of engineers better by providing them with tools to understand how they fit into their organisation and how they can profile their life.
  • 27. 2016-2018 People also represent a complex system and I played a lot with different visualizations of my personal and work graph to discover human connections. github: Ikke Graph.
  • 28. While working at Uber, my MacBook was always struggling with the CPU usage of Intellij and the Android Emulator. To understand better what was going on, I created HappyMac, which allows its user to discover expensive processes and auto-suspend them using some rules. 2018
  • 29. HappyWeb is like HappyMac, but focuses on slow network connections. 2018
  • 30. Tempo constantly watches your MacBook and records the currently active program and saves an event in a local SQL database. The recorded events can show effectiveness of workflows, how much time is spent in certain activities, and how to avoid context switches. A human profiler? 2019
  • 31. Summary Complex systems are all around us, either as technical solutions or as connections between humans. In order to improve these complex systems, we need to understand them better. To understand them better, we need to measure and generate metrics. Custom tools will then provide insights.
  • 33. The End Be humble, we have small brains.