際際滷

際際滷Share a Scribd company logo

DevTalks.ro - How do we Measure Software

Download as PPTX, PDF
Ruben Darius Moldovan
Ruben Darius Moldovan

The document discusses various ways to measure software metrics at different stages of the software development lifecycle. It defines key terms and outlines metrics for planning, development, testing, support and other areas. Examples of metrics include defects, requirements, automated test coverage, customer satisfaction, and financial metrics. The document also provides examples of how to present and analyze metrics over time to identify trends and room for improvement.

1 of 25
Download to read offline
How do we Measure Software? RUBEN-DARIUS MOLDOVAN 13 MAY 2015 DEVTALKS
HDWM Software Questions Who measures? Why measure? Where? When? How? What is ? Why would I invest in your product? To improve To express qualitative factor by a number To estimate costs, schedule, complexity, effort requirements
HDWM Software Agenda Definitions Software Definitions Questions Types Complexity Status Metrics Planning Developing Testing Support Summary & Questions
HDWM Software Type Price List Internal Shared
HDWM Software Complexity Business No. of People working on it No. of Customers No. of Supported Releases Size? Technology Lines of code Modules Languages Symbiosis Customization Size? Weighted Methods?
HDWM Software Status Value Sales Customers Quality Health Performance Direction Market Competitors
HDWM Software Life Calendar Months Weeks Fiscal Years Quarters Cycle Releases Planning Specifications Development Design Testing Validation Support Maintenance
HDWM Software Agenda Metrics Software Definitions Questions Types Complexity Status Metrics Planning Developing Testing Support Summary & Questions
HDWM Software Metrics Support Incoming Cases Days to Resolution Elevations Customer Satisfaction Index Solved without Help Development Requirements Defects Enhancements Escalations Patents Lines of Code Testing Automation Performance Code Coverage Defect Escape Rate Security Financial Sales / Licenses Costs Customers Net Promoter Score Other Ease to Run Time toValue Usage Statistics Maintainability Mismatch / Consistency
HDWM Software Meaningful Deliverables Process Effectiveness DevelopmentTime Product Quality
HDWM Software Reports
HDWM Software Indicator Model No. Meaning 1 This is the metric's value. 2 The background color of the cell indicates the status of the metric which can be: "red", "yellow" or "green" color, based on theThresholds 3 A blue rectangle means that the status changed from red to yellow/green or vice versa. 4 This is the trend of the metric.The arrows can be colored in red, or green if the difference was higher than 20% of the previous quarter.The trend is up if the product improved over previous quarter from a business perspective, not from a mathematical point of view. 5 This is the difference from the previous quarter's value in percent.The number is green if the metric improved, or red if it declined. 6 This is the absolute difference from the previous quarter's value.
HDWM Software Indicator Example Let's suppose a product has 8 Escalations for the current quarter and 5 Escalations for the previous quarter.The Incident Escalations cell would have the following values: 8 as the cell value (count), 2 is the grade Background color will be red, because Esc grade is 2. The blue border is present because last quarter the EscGrade was 3.0 and was yellow. The trend arrow is down (current quarter value is worse than previous quarter), but the difference is more than 20%, so red color is applied. The percentage difference rounded up to integer number is 60% and is colored in red because the metric is worse than previous quarter. Finally, the absolute difference is +3 Incident Escalations More detailed information is available on every number's mouse over tooltip: values for the selected timeframes, grades, including metric numerators and denominators and history charts.
HDWM Software Support Incoming Cases Increase / decrease why? Seasonal? Solved without help DaysTo Resolution What cases do we analyze? Average overTime? Median? Elevations Customer Satisfaction Index Top 2,Top x Overall, Product Quality, Support, Documentation, Ease to.. etc How an improvement looks? How do we improve? Whats the cost of improvement?
HDWM Software Support - DTR DaysTo Resolution (DTR) DTRAvg = (ｐ 倹＄ 倹＄) 駒 For Current Backlog ｐ 倹＄ = Considering the Urgent / High severity Customer Interaction What would you show in a current month? High volume vs. Low volume Is this seasonal?
HDWM Software Planning Agile Metrics Burn Up / Downs Velocity Feature Status Traceability Defects Features Tests
HDWM Software Development Cost per Defect Critical / Serious Defects CustomerVisible / Customer Encountered Backlog of Defects / Enhancements Escalations Defect Escape Rate (Leakage) Code Analysis Length (LOC) Cost per 1k / Errors per 1k / LOC per engineer Complexity Re-usability
HDWM Software Testing AutomatedTests 100% automated tests? Ease to (automated) test Analysis ofTest Results Performance HW/SW Requirements Security
HDWM Software Financial Sales Licenses Support Market Share Net Promotor Score Double blind surveys
HDWM Software Other Product Management Traceability (between Requirements,Test and Defects) Improvement Programs Time 2Value (T2 Demo, POC,T2 Install,T2 See Results,T2 Profit of Usage,T2Customization) Ease 2 Run (Upgrades, HC to Operate) HR Attrition Current (T) Shape or Engineers,Trainings
HDWM Software Usage Statistics Know the top Features / Modules and their performance Know the top Users their environment their roles Attrition Current (T) Shape or Engineers,Trainings
HDWM Software Agenda Summary Software Definitions Questions Types Complexity Status Metrics Planning Developing Testing Support Summary & Questions
HDWM Software Summary Pre-Release / Post-Release Metrics Process / Product - Project / Program Metrics Size / Function Oriented Metrics Customer Metrics Private and Public Metrics What is improvement? How do we simulate a 5%-20% improvement? Usage of the Product Logs Install base history Ecological (Predictive) Metrics The substance to anchor into the future
HDWM Software Questions ?
HDWM Software Thank you!

More Related Content

DevTalks.ro - How do we Measure Software

  • 1. How do we Measure Software? RUBEN-DARIUS MOLDOVAN 13 MAY 2015 DEVTALKS
  • 2. HDWM Software Questions Who measures? Why measure? Where? When? How? What is ? Why would I invest in your product? To improve To express qualitative factor by a number To estimate costs, schedule, complexity, effort requirements
  • 3. HDWM Software Agenda Definitions Software Definitions Questions Types Complexity Status Metrics Planning Developing Testing Support Summary & Questions
  • 5. HDWM Software Complexity Business No. of People working on it No. of Customers No. of Supported Releases Size? Technology Lines of code Modules Languages Symbiosis Customization Size? Weighted Methods?
  • 6. HDWM Software Status Value Sales Customers Quality Health Performance Direction Market Competitors
  • 7. HDWM Software Life Calendar Months Weeks Fiscal Years Quarters Cycle Releases Planning Specifications Development Design Testing Validation Support Maintenance
  • 8. HDWM Software Agenda Metrics Software Definitions Questions Types Complexity Status Metrics Planning Developing Testing Support Summary & Questions
  • 9. HDWM Software Metrics Support Incoming Cases Days to Resolution Elevations Customer Satisfaction Index Solved without Help Development Requirements Defects Enhancements Escalations Patents Lines of Code Testing Automation Performance Code Coverage Defect Escape Rate Security Financial Sales / Licenses Costs Customers Net Promoter Score Other Ease to Run Time toValue Usage Statistics Maintainability Mismatch / Consistency
  • 10. HDWM Software Meaningful Deliverables Process Effectiveness DevelopmentTime Product Quality
  • 12. HDWM Software Indicator Model No. Meaning 1 This is the metric's value. 2 The background color of the cell indicates the status of the metric which can be: "red", "yellow" or "green" color, based on theThresholds 3 A blue rectangle means that the status changed from red to yellow/green or vice versa. 4 This is the trend of the metric.The arrows can be colored in red, or green if the difference was higher than 20% of the previous quarter.The trend is up if the product improved over previous quarter from a business perspective, not from a mathematical point of view. 5 This is the difference from the previous quarter's value in percent.The number is green if the metric improved, or red if it declined. 6 This is the absolute difference from the previous quarter's value.
  • 13. HDWM Software Indicator Example Let's suppose a product has 8 Escalations for the current quarter and 5 Escalations for the previous quarter.The Incident Escalations cell would have the following values: 8 as the cell value (count), 2 is the grade Background color will be red, because Esc grade is 2. The blue border is present because last quarter the EscGrade was 3.0 and was yellow. The trend arrow is down (current quarter value is worse than previous quarter), but the difference is more than 20%, so red color is applied. The percentage difference rounded up to integer number is 60% and is colored in red because the metric is worse than previous quarter. Finally, the absolute difference is +3 Incident Escalations More detailed information is available on every number's mouse over tooltip: values for the selected timeframes, grades, including metric numerators and denominators and history charts.
  • 14. HDWM Software Support Incoming Cases Increase / decrease why? Seasonal? Solved without help DaysTo Resolution What cases do we analyze? Average overTime? Median? Elevations Customer Satisfaction Index Top 2,Top x Overall, Product Quality, Support, Documentation, Ease to.. etc How an improvement looks? How do we improve? Whats the cost of improvement?
  • 15. HDWM Software Support - DTR DaysTo Resolution (DTR) DTRAvg = (ｐ 倹＄ 倹＄) 駒 For Current Backlog ｐ 倹＄ = Considering the Urgent / High severity Customer Interaction What would you show in a current month? High volume vs. Low volume Is this seasonal?
  • 16. HDWM Software Planning Agile Metrics Burn Up / Downs Velocity Feature Status Traceability Defects Features Tests
  • 17. HDWM Software Development Cost per Defect Critical / Serious Defects CustomerVisible / Customer Encountered Backlog of Defects / Enhancements Escalations Defect Escape Rate (Leakage) Code Analysis Length (LOC) Cost per 1k / Errors per 1k / LOC per engineer Complexity Re-usability
  • 18. HDWM Software Testing AutomatedTests 100% automated tests? Ease to (automated) test Analysis ofTest Results Performance HW/SW Requirements Security
  • 19. HDWM Software Financial Sales Licenses Support Market Share Net Promotor Score Double blind surveys
  • 20. HDWM Software Other Product Management Traceability (between Requirements,Test and Defects) Improvement Programs Time 2Value (T2 Demo, POC,T2 Install,T2 See Results,T2 Profit of Usage,T2Customization) Ease 2 Run (Upgrades, HC to Operate) HR Attrition Current (T) Shape or Engineers,Trainings
  • 21. HDWM Software Usage Statistics Know the top Features / Modules and their performance Know the top Users their environment their roles Attrition Current (T) Shape or Engineers,Trainings
  • 22. HDWM Software Agenda Summary Software Definitions Questions Types Complexity Status Metrics Planning Developing Testing Support Summary & Questions
  • 23. HDWM Software Summary Pre-Release / Post-Release Metrics Process / Product - Project / Program Metrics Size / Function Oriented Metrics Customer Metrics Private and Public Metrics What is improvement? How do we simulate a 5%-20% improvement? Usage of the Product Logs Install base history Ecological (Predictive) Metrics The substance to anchor into the future