際際滷

際際滷Share a Scribd company logo
Erlang Application Metrics
                       with Folsom
                       Joe Williams (@williamsjoe)

                       Boundary (@boundary / boundary.com)




Friday, March 30, 12
Who Am I?

                   Joe Williams

                        @williamsjoe

                        Ops at Boundary




Friday, March 30, 12
What is Boundary?

                   Realtime Monitoring and Analytics Platform

                   Were hiring!

                   TRY MY PRODUCT




Friday, March 30, 12
Agenda

                   Current and Future Monitoring Systems

                   Define Application Metrics

                   What is Folsom?

                   How do you use Folsom?



Friday, March 30, 12
Legacy Monitoring Tools
                   Consumers of application metrics

                        Munin, Ganglia, etc

                   Wide variety of protocols, interfaces and formats

                   Usually RRD, results in inflexible, infrequently refreshed PNGs on
                    HTML

                   Not what this talk is about.


Friday, March 30, 12
Current Monitoring Tools

                   Consumers of application metrics

                        OpenTSDB, statsd, Rienmann, etc are a steps in the right direction

                   Hopefully fewer protocols, interfaces and formats

                   Realtime, dynamic graphing

                   Not what this talk is about.


Friday, March 30, 12
Application Metrics



                        .. code generates business value when it runs. ~ Coda Hale




Friday, March 30, 12
Application Metrics

                   Code instrumentation

                        Write code to measure code

                        Verify code behaves as you expect

                   Exporting results of instrumentation

                   General VM health


Friday, March 30, 12
Folsom

                   Application-level metrics

                   Multiple metric types

                   Powerful statistical analysis

                   Easily extensible



Friday, March 30, 12
Folsom


                        DEMO!



Friday, March 30, 12
Metric Types
                   Counters

                   Gauges

                   Histograms

                   Histories

                   Meters

                   Erlang VM Metrics

Friday, March 30, 12
Counters


                   Counter metrics provide increment and decrement capabilities for
                    a single scalar value.




Friday, March 30, 12
Counters

                            > folsom_metrics:new_counter(Name).
                       > folsom_metrics:notify({Name, {inc, Value}}).
                       > folsom_metrics:notify({Name, {dec, Value}}).




Friday, March 30, 12
Gauges


                   Gauges are point-in-time single value metrics.




Friday, March 30, 12
Gauges

                          > folsom_metrics:new_gauge(Name).
                       > folsom_metrics:notify({Name, Value}).




Friday, March 30, 12
Histograms

                   Histograms are collections of values that have statistical analysis
                    done to them, such as mean, min, max, kurtosis and percentile.

                   The can be used like "timers" as well with the timed update
                    functions.




Friday, March 30, 12
Histograms

                                   > folsom_metrics:new_histogram(Name).
                       > folsom_metrics:histogram_timed_update(Name, Mod, Fun, Args).
                         > folsom_metrics:histogram_timed_update(Name, Fun, Args).
                            > folsom_metrics:histogram_timed_update(Name, Fun).
                                  > folsom_metrics:notify({Name, Value}).




Friday, March 30, 12
Histories


                   Histories are a collection of past events, such as errors or log
                    messages.




Friday, March 30, 12
Histories

                              > folsom_metrics:new_history(Name).
                       > folsom_metrics:get_history_values(Name, Count).
                         > folsom_metrics:notify({Name, Value}).




Friday, March 30, 12
Meters


                   Meters are increment only counters with mean rates and
                    exponentially weighted moving averages applied to them, similar
                    to a unix load average.




Friday, March 30, 12
Meters

                           > folsom_metrics:new_meter(Name).
                        > folsom_metrics:notify({Name, Value}).




Friday, March 30, 12
Erlang VM


                   General VM metrics provided by Erlang VM.




Friday, March 30, 12
Erlang VM

                            > folsom_vm_metrics:get_memory().
                        > folsom_vm_metrics:get_system_info().
                         > folsom_vm_metrics:get_statistics().
                          > folsom_vm_metrics:get_port_info().
                       > folsom_vm_metrics:get_process_info().




Friday, March 30, 12
Statistics
                          Statistical Analysis

                            Exponentially Weighted Moving Average (EWMA)

                            Rate / Acceleration

                            Avg, Mean, Min, Max, etc

                            Distribution

                            Percentiles / Histograms

                            Covariance / Correlation


Friday, March 30, 12
Sampling

                   Exponentially decaying

                   Uniform

                   None




Friday, March 30, 12
Extensibility
                   Clean Native API

                   Folsom Wrappers

                        REST+JSON

                         Munin plugins!

                        SNMP? Graphite? CollectD?


Friday, March 30, 12
Future Work

                   Metrics REST+JSON Spec

                   Performance Improvements

                   Better Querying, Slicing and etc of metric values

                   More metric types (meter reader, timer, etc)



Friday, March 30, 12
Similar Tools
                      Coda Hales Metrics (JVM) - github.com/coda/metrics

                      Ruby - github.com/johnewart/ruby-metrics

                      JavaScript - github.com/mikejihbe/metrics

                      .NET Metrics - github.com/danielcrenna/metrics-net

                      Ostrich - github.com/twitter/ostrich

                      Python Ostrich - github.com/wadey/python-ostrich

                      .Net Ostrich - github.com/ewhauser/OstrichNet



Friday, March 30, 12
Folsom


                        DEMO!



Friday, March 30, 12
Questions?
                            Joe Williams (@williamsjoe)

                       Boundary (@boundary / boundary.com)

                       https://github.com/boundary/folsom




Friday, March 30, 12
Ad

Recommended

Monitoring with exometer at AdRoll
Monitoring with exometer at AdRoll
Brian Troutwine
Modern Systems Development talk for TechStars Cloud 2015
Modern Systems Development talk for TechStars Cloud 2015
troytoman
StatsD Workshop Monitorama 2013
StatsD Workshop Monitorama 2013
Daniel Schauenberg
Measuring
Measuring
bostonrb
Librato's Joseph Ruscio at Heroku's 2013: Instrumenting 12-Factor Apps
Librato's Joseph Ruscio at Heroku's 2013: Instrumenting 12-Factor Apps
Heroku
Measure Everything
Measure Everything
Arik Fraimovich
StasD & Graphite - Measure anything, Measure Everything
StasD & Graphite - Measure anything, Measure Everything
Avi Revivo
Data opsdc clearspringmetrics
Data opsdc clearspringmetrics
abramsm
metrics - performance monitoring or business value optimization?
metrics - performance monitoring or business value optimization?
Jo達o Nelas
Winning the metrics battle
Winning the metrics battle
sihil
Trending with Purpose
Trending with Purpose
Jason Dixon
Nagios Conference 2012 - Dave Josephsen - 2002 called they want there rrd she...
Nagios Conference 2012 - Dave Josephsen - 2002 called they want there rrd she...
Nagios
Metrics
Metrics
Zach Cox
Code Inspection
Code Inspection
F叩ber D. Giraldo
Graph Analysis: New Algorithm Models, New Architectures
Graph Analysis: New Algorithm Models, New Architectures
Jason Riedy
STING: Spatio-Temporal Interaction Networks and Graphs for Intel Platforms
STING: Spatio-Temporal Interaction Networks and Graphs for Intel Platforms
Jason Riedy
OSDC 2014: Devdas Bhagat - Graphite: Graphs for the modern age
OSDC 2014: Devdas Bhagat - Graphite: Graphs for the modern age
NETWAYS
Timeseries data in Riak - Riak Meetup Stockholm 1/11/2012
Timeseries data in Riak - Riak Meetup Stockholm 1/11/2012
Bip Thelin
Crunching the numbers: Open Source Community Metrics at OSCON
Crunching the numbers: Open Source Community Metrics at OSCON
Dawn Foster
Crunching the numbers: Open Source Community Metrics
Crunching the numbers: Open Source Community Metrics
Dawn Foster
Changes and Bugs: Mining and Predicting Development Activities
Changes and Bugs: Mining and Predicting Development Activities
Thomas Zimmermann
Monitoring as Software Validation
Monitoring as Software Validation
BioDec
Software metrics sucess, failures and new directions
Software metrics sucess, failures and new directions
Andrws Vieira
Software metrics and estimation lecture 1
Software metrics and estimation lecture 1
Faiz Zeya
Software metrics lecture 1 from BUKC lecture
Software metrics lecture 1 from BUKC lecture
FaizUlHaqueZeya
Rethinking metrics: metrics 2.0 @ Lisa 2014
Rethinking metrics: metrics 2.0 @ Lisa 2014
Dieter Plaetinck
How Salesforce.com Uses Hadoop
How Salesforce.com Uses Hadoop
Salesforce Developers
Floss Metrics 2009
Floss Metrics 2009
Inria
AI VIDEO MAGAZINE - June 2025 - r/aivideo
AI VIDEO MAGAZINE - June 2025 - r/aivideo
1pcity Studios, Inc
OpenPOWER Foundation & Open-Source Core Innovations
OpenPOWER Foundation & Open-Source Core Innovations
IBM

More Related Content

Similar to Erlang Application Metrics with Folsom (20)

metrics - performance monitoring or business value optimization?
metrics - performance monitoring or business value optimization?
Jo達o Nelas
Winning the metrics battle
Winning the metrics battle
sihil
Trending with Purpose
Trending with Purpose
Jason Dixon
Nagios Conference 2012 - Dave Josephsen - 2002 called they want there rrd she...
Nagios Conference 2012 - Dave Josephsen - 2002 called they want there rrd she...
Nagios
Metrics
Metrics
Zach Cox
Code Inspection
Code Inspection
F叩ber D. Giraldo
Graph Analysis: New Algorithm Models, New Architectures
Graph Analysis: New Algorithm Models, New Architectures
Jason Riedy
STING: Spatio-Temporal Interaction Networks and Graphs for Intel Platforms
STING: Spatio-Temporal Interaction Networks and Graphs for Intel Platforms
Jason Riedy
OSDC 2014: Devdas Bhagat - Graphite: Graphs for the modern age
OSDC 2014: Devdas Bhagat - Graphite: Graphs for the modern age
NETWAYS
Timeseries data in Riak - Riak Meetup Stockholm 1/11/2012
Timeseries data in Riak - Riak Meetup Stockholm 1/11/2012
Bip Thelin
Crunching the numbers: Open Source Community Metrics at OSCON
Crunching the numbers: Open Source Community Metrics at OSCON
Dawn Foster
Crunching the numbers: Open Source Community Metrics
Crunching the numbers: Open Source Community Metrics
Dawn Foster
Changes and Bugs: Mining and Predicting Development Activities
Changes and Bugs: Mining and Predicting Development Activities
Thomas Zimmermann
Monitoring as Software Validation
Monitoring as Software Validation
BioDec
Software metrics sucess, failures and new directions
Software metrics sucess, failures and new directions
Andrws Vieira
Software metrics and estimation lecture 1
Software metrics and estimation lecture 1
Faiz Zeya
Software metrics lecture 1 from BUKC lecture
Software metrics lecture 1 from BUKC lecture
FaizUlHaqueZeya
Rethinking metrics: metrics 2.0 @ Lisa 2014
Rethinking metrics: metrics 2.0 @ Lisa 2014
Dieter Plaetinck
How Salesforce.com Uses Hadoop
How Salesforce.com Uses Hadoop
Salesforce Developers
Floss Metrics 2009
Floss Metrics 2009
Inria
metrics - performance monitoring or business value optimization?
metrics - performance monitoring or business value optimization?
Jo達o Nelas
Winning the metrics battle
Winning the metrics battle
sihil
Trending with Purpose
Trending with Purpose
Jason Dixon
Nagios Conference 2012 - Dave Josephsen - 2002 called they want there rrd she...
Nagios Conference 2012 - Dave Josephsen - 2002 called they want there rrd she...
Nagios
Metrics
Metrics
Zach Cox
Graph Analysis: New Algorithm Models, New Architectures
Graph Analysis: New Algorithm Models, New Architectures
Jason Riedy
STING: Spatio-Temporal Interaction Networks and Graphs for Intel Platforms
STING: Spatio-Temporal Interaction Networks and Graphs for Intel Platforms
Jason Riedy
OSDC 2014: Devdas Bhagat - Graphite: Graphs for the modern age
OSDC 2014: Devdas Bhagat - Graphite: Graphs for the modern age
NETWAYS
Timeseries data in Riak - Riak Meetup Stockholm 1/11/2012
Timeseries data in Riak - Riak Meetup Stockholm 1/11/2012
Bip Thelin
Crunching the numbers: Open Source Community Metrics at OSCON
Crunching the numbers: Open Source Community Metrics at OSCON
Dawn Foster
Crunching the numbers: Open Source Community Metrics
Crunching the numbers: Open Source Community Metrics
Dawn Foster
Changes and Bugs: Mining and Predicting Development Activities
Changes and Bugs: Mining and Predicting Development Activities
Thomas Zimmermann
Monitoring as Software Validation
Monitoring as Software Validation
BioDec
Software metrics sucess, failures and new directions
Software metrics sucess, failures and new directions
Andrws Vieira
Software metrics and estimation lecture 1
Software metrics and estimation lecture 1
Faiz Zeya
Software metrics lecture 1 from BUKC lecture
Software metrics lecture 1 from BUKC lecture
FaizUlHaqueZeya
Rethinking metrics: metrics 2.0 @ Lisa 2014
Rethinking metrics: metrics 2.0 @ Lisa 2014
Dieter Plaetinck
Floss Metrics 2009
Floss Metrics 2009
Inria

Recently uploaded (20)

AI VIDEO MAGAZINE - June 2025 - r/aivideo
AI VIDEO MAGAZINE - June 2025 - r/aivideo
1pcity Studios, Inc
OpenPOWER Foundation & Open-Source Core Innovations
OpenPOWER Foundation & Open-Source Core Innovations
IBM
FIDO Seminar: Perspectives on Passkeys & Consumer Adoption.pptx
FIDO Seminar: Perspectives on Passkeys & Consumer Adoption.pptx
FIDO Alliance
UserCon Belgium: Honey, VMware increased my bill
UserCon Belgium: Honey, VMware increased my bill
stijn40
FIDO Seminar: Evolving Landscape of Post-Quantum Cryptography.pptx
FIDO Seminar: Evolving Landscape of Post-Quantum Cryptography.pptx
FIDO Alliance
Techniques for Automatic Device Identification and Network Assignment.pdf
Techniques for Automatic Device Identification and Network Assignment.pdf
Priyanka Aash
10 Key Challenges for AI within the EU Data Protection Framework.pdf
10 Key Challenges for AI within the EU Data Protection Framework.pdf
Priyanka Aash
Lessons Learned from Developing Secure AI Workflows.pdf
Lessons Learned from Developing Secure AI Workflows.pdf
Priyanka Aash
2025_06_18 - OpenMetadata Community Meeting.pdf
2025_06_18 - OpenMetadata Community Meeting.pdf
OpenMetadata
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik
GenAI Opportunities and Challenges - Where 370 Enterprises Are Focusing Now.pdf
GenAI Opportunities and Challenges - Where 370 Enterprises Are Focusing Now.pdf
Priyanka Aash
From Manual to Auto Searching- FME in the Driver's Seat
From Manual to Auto Searching- FME in the Driver's Seat
Safe Software
OpenACC and Open Hackathons Monthly Highlights June 2025
OpenACC and Open Hackathons Monthly Highlights June 2025
OpenACC
Key Requirements to Successfully Implement Generative AI in Edge DevicesOpt...
Key Requirements to Successfully Implement Generative AI in Edge DevicesOpt...
Edge AI and Vision Alliance
Enhance GitHub Copilot using MCP - Enterprise version.pdf
Enhance GitHub Copilot using MCP - Enterprise version.pdf
Nilesh Gule
Securing AI - There Is No Try, Only Do!.pdf
Securing AI - There Is No Try, Only Do!.pdf
Priyanka Aash
"How to survive Black Friday: preparing e-commerce for a peak season", Yurii ...
"How to survive Black Friday: preparing e-commerce for a peak season", Yurii ...
Fwdays
Security Tips for Enterprise Azure Solutions
Security Tips for Enterprise Azure Solutions
Michele Leroux Bustamante
Improving Data Integrity: Synchronization between EAM and ArcGIS Utility Netw...
Improving Data Integrity: Synchronization between EAM and ArcGIS Utility Netw...
Safe Software
FIDO Seminar: Authentication for a Billion Consumers - Amazon.pptx
FIDO Seminar: Authentication for a Billion Consumers - Amazon.pptx
FIDO Alliance
AI VIDEO MAGAZINE - June 2025 - r/aivideo
AI VIDEO MAGAZINE - June 2025 - r/aivideo
1pcity Studios, Inc
OpenPOWER Foundation & Open-Source Core Innovations
OpenPOWER Foundation & Open-Source Core Innovations
IBM
FIDO Seminar: Perspectives on Passkeys & Consumer Adoption.pptx
FIDO Seminar: Perspectives on Passkeys & Consumer Adoption.pptx
FIDO Alliance
UserCon Belgium: Honey, VMware increased my bill
UserCon Belgium: Honey, VMware increased my bill
stijn40
FIDO Seminar: Evolving Landscape of Post-Quantum Cryptography.pptx
FIDO Seminar: Evolving Landscape of Post-Quantum Cryptography.pptx
FIDO Alliance
Techniques for Automatic Device Identification and Network Assignment.pdf
Techniques for Automatic Device Identification and Network Assignment.pdf
Priyanka Aash
10 Key Challenges for AI within the EU Data Protection Framework.pdf
10 Key Challenges for AI within the EU Data Protection Framework.pdf
Priyanka Aash
Lessons Learned from Developing Secure AI Workflows.pdf
Lessons Learned from Developing Secure AI Workflows.pdf
Priyanka Aash
2025_06_18 - OpenMetadata Community Meeting.pdf
2025_06_18 - OpenMetadata Community Meeting.pdf
OpenMetadata
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik
GenAI Opportunities and Challenges - Where 370 Enterprises Are Focusing Now.pdf
GenAI Opportunities and Challenges - Where 370 Enterprises Are Focusing Now.pdf
Priyanka Aash
From Manual to Auto Searching- FME in the Driver's Seat
From Manual to Auto Searching- FME in the Driver's Seat
Safe Software
OpenACC and Open Hackathons Monthly Highlights June 2025
OpenACC and Open Hackathons Monthly Highlights June 2025
OpenACC
Key Requirements to Successfully Implement Generative AI in Edge DevicesOpt...
Key Requirements to Successfully Implement Generative AI in Edge DevicesOpt...
Edge AI and Vision Alliance
Enhance GitHub Copilot using MCP - Enterprise version.pdf
Enhance GitHub Copilot using MCP - Enterprise version.pdf
Nilesh Gule
Securing AI - There Is No Try, Only Do!.pdf
Securing AI - There Is No Try, Only Do!.pdf
Priyanka Aash
"How to survive Black Friday: preparing e-commerce for a peak season", Yurii ...
"How to survive Black Friday: preparing e-commerce for a peak season", Yurii ...
Fwdays
Security Tips for Enterprise Azure Solutions
Security Tips for Enterprise Azure Solutions
Michele Leroux Bustamante
Improving Data Integrity: Synchronization between EAM and ArcGIS Utility Netw...
Improving Data Integrity: Synchronization between EAM and ArcGIS Utility Netw...
Safe Software
FIDO Seminar: Authentication for a Billion Consumers - Amazon.pptx
FIDO Seminar: Authentication for a Billion Consumers - Amazon.pptx
FIDO Alliance
Ad

Erlang Application Metrics with Folsom

  • 1. Erlang Application Metrics with Folsom Joe Williams (@williamsjoe) Boundary (@boundary / boundary.com) Friday, March 30, 12
  • 2. Who Am I? Joe Williams @williamsjoe Ops at Boundary Friday, March 30, 12
  • 3. What is Boundary? Realtime Monitoring and Analytics Platform Were hiring! TRY MY PRODUCT Friday, March 30, 12
  • 4. Agenda Current and Future Monitoring Systems Define Application Metrics What is Folsom? How do you use Folsom? Friday, March 30, 12
  • 5. Legacy Monitoring Tools Consumers of application metrics Munin, Ganglia, etc Wide variety of protocols, interfaces and formats Usually RRD, results in inflexible, infrequently refreshed PNGs on HTML Not what this talk is about. Friday, March 30, 12
  • 6. Current Monitoring Tools Consumers of application metrics OpenTSDB, statsd, Rienmann, etc are a steps in the right direction Hopefully fewer protocols, interfaces and formats Realtime, dynamic graphing Not what this talk is about. Friday, March 30, 12
  • 7. Application Metrics .. code generates business value when it runs. ~ Coda Hale Friday, March 30, 12
  • 8. Application Metrics Code instrumentation Write code to measure code Verify code behaves as you expect Exporting results of instrumentation General VM health Friday, March 30, 12
  • 9. Folsom Application-level metrics Multiple metric types Powerful statistical analysis Easily extensible Friday, March 30, 12
  • 10. Folsom DEMO! Friday, March 30, 12
  • 11. Metric Types Counters Gauges Histograms Histories Meters Erlang VM Metrics Friday, March 30, 12
  • 12. Counters Counter metrics provide increment and decrement capabilities for a single scalar value. Friday, March 30, 12
  • 13. Counters > folsom_metrics:new_counter(Name). > folsom_metrics:notify({Name, {inc, Value}}). > folsom_metrics:notify({Name, {dec, Value}}). Friday, March 30, 12
  • 14. Gauges Gauges are point-in-time single value metrics. Friday, March 30, 12
  • 15. Gauges > folsom_metrics:new_gauge(Name). > folsom_metrics:notify({Name, Value}). Friday, March 30, 12
  • 16. Histograms Histograms are collections of values that have statistical analysis done to them, such as mean, min, max, kurtosis and percentile. The can be used like "timers" as well with the timed update functions. Friday, March 30, 12
  • 17. Histograms > folsom_metrics:new_histogram(Name). > folsom_metrics:histogram_timed_update(Name, Mod, Fun, Args). > folsom_metrics:histogram_timed_update(Name, Fun, Args). > folsom_metrics:histogram_timed_update(Name, Fun). > folsom_metrics:notify({Name, Value}). Friday, March 30, 12
  • 18. Histories Histories are a collection of past events, such as errors or log messages. Friday, March 30, 12
  • 19. Histories > folsom_metrics:new_history(Name). > folsom_metrics:get_history_values(Name, Count). > folsom_metrics:notify({Name, Value}). Friday, March 30, 12
  • 20. Meters Meters are increment only counters with mean rates and exponentially weighted moving averages applied to them, similar to a unix load average. Friday, March 30, 12
  • 21. Meters > folsom_metrics:new_meter(Name). > folsom_metrics:notify({Name, Value}). Friday, March 30, 12
  • 22. Erlang VM General VM metrics provided by Erlang VM. Friday, March 30, 12
  • 23. Erlang VM > folsom_vm_metrics:get_memory(). > folsom_vm_metrics:get_system_info(). > folsom_vm_metrics:get_statistics(). > folsom_vm_metrics:get_port_info(). > folsom_vm_metrics:get_process_info(). Friday, March 30, 12
  • 24. Statistics Statistical Analysis Exponentially Weighted Moving Average (EWMA) Rate / Acceleration Avg, Mean, Min, Max, etc Distribution Percentiles / Histograms Covariance / Correlation Friday, March 30, 12
  • 25. Sampling Exponentially decaying Uniform None Friday, March 30, 12
  • 26. Extensibility Clean Native API Folsom Wrappers REST+JSON Munin plugins! SNMP? Graphite? CollectD? Friday, March 30, 12
  • 27. Future Work Metrics REST+JSON Spec Performance Improvements Better Querying, Slicing and etc of metric values More metric types (meter reader, timer, etc) Friday, March 30, 12
  • 28. Similar Tools Coda Hales Metrics (JVM) - github.com/coda/metrics Ruby - github.com/johnewart/ruby-metrics JavaScript - github.com/mikejihbe/metrics .NET Metrics - github.com/danielcrenna/metrics-net Ostrich - github.com/twitter/ostrich Python Ostrich - github.com/wadey/python-ostrich .Net Ostrich - github.com/ewhauser/OstrichNet Friday, March 30, 12
  • 29. Folsom DEMO! Friday, March 30, 12
  • 30. Questions? Joe Williams (@williamsjoe) Boundary (@boundary / boundary.com) https://github.com/boundary/folsom Friday, March 30, 12