Works on my machine, your problem now? - QCon 2014
?Download as PPTX, PDF?
1 like?1,654 views
Can you get away with that answer after crashing your production website with a change you just deployed? Usually you can¡¯t, and instead you¡¯re tasked with figuring out and fixing the problem. In this session, we will talk about typical architectural, coding and deployment problems you might recognize, show what data you need to quickly identify them, and how to catch them before impacting the business.
Works on my machine, your problem now? - QCon 2014
- 1. 11 COMPANY CONFIDENTIAL ¨C DO NOT DISTRIBUTE Works on my machine ¨C your problem now? Wolfgang Gottesheim Compuware APM
- 2. 22 Business comes up with new features
- 3. 33 Testing?
- 4. 44 And this is what you end up with¡
- 6. 66 What Operations tells ¶Ù±ð±¹±ð±ô´Ç±è±ð°ù²õ¡
- 7. 77 ¡and what Devs would like to know
- 8. 88 ¡and what Devs would like to know Top Contributor is related to String handling 99% of that time comes from RegEx Pattern Matching Page Rendering is the main component
- 9. 99 Attitudes like this don¡¯t help either Image taken from https://www.scriptrock.com/blog/devops-whats-hype-about/
- 10. 1010 Very ¨Dexpensive¡¬ to work on these issues ~80% of problems caused by ~20% patterns YES we know this 80%Dev Time in Bug Fixing $60B Defect Costs BUT
- 11. 1111
- 12. 1212 #1: Exhausted Resource Pools
- 13. 1313 #2: Maxing out Worker Threads The timeline shows how these active worker threads are distributed across all JVMs At ~10:10 AM almost all JVMs max out their available worker threads Detailed information for every single JVM
- 14. 1414 Root Cause: Class Loading as Performance Hotspot Most of the time is spent in CLASSLOADING during Peak Load But the same is true for ¨Dnormal¡¬ load. Classloading seems to be a general problem that is not load related
- 15. 1515 Root Cause: Trying to Load a Missing Class Class Loading impacts ALL transactions (fast or slow) Class Loader tries to load a class ending in TransferValidatorBPBeanInfo It¡¯s a class that doesn¡¯t exist
- 17. 1717 Root Cause: Missing File
- 18. 1818 #4: Different settings in Test & Prod
- 19. 1919 #5: Real-world Data != Test Data
- 20. 2020 #6: N+1 Query Problem
- 21. 2121 #7: Misconfigured Caching Framework 798772 DB Calls in 30 minutes With NO TRAFFIC
- 22. 2222 #8: Memory Leaks Still crashes Problem fixed! Fixed Version Deployed
- 23. 2323 #9: Bloated Web Sites 17! JS Files ¨C 1.7MB in Size Useless Information! Even might be a security risk!
- 24. 2424 Recent example: Healthcare.gov 55 JS Files, 16 jQuery related! Merging files can reduce roundtrips by 95%
- 25. 2525 #10: Browser caches 62! Resources not cached 49! Resources with short expiration
- 26. 2626 Problems that could have been avoided BUT WHY are they still making it to Production? HOW can we catch them earlier?
- 27. 2727 Root Cause: Disconnected Teams
- 28. 28 Solution: DevOps + Performance Focus
- 29. 2929 Culture Become ONE Team
- 31. 3131 Automate & Measure ¡±Ê±ð°ù´Ú´Ç°ù³¾²¹²Ô³¦±ð
- 32. 3232 Automate & Measure ¡³§³¦²¹±ô²¹²ú¾±±ô¾±³Ù²â
- 34. 3434 Share Tools
- 35. 3535 How? Performance Focus in Test Automation 12 0 120ms 3 1 68ms Build 20 testPurchase OK testSearch OK Build 17 testPurchase OK testSearch OK Build 18 testPurchase FAILED testSearch OK Build 19 testPurchase OK testSearch OK Build # Test Case Status # SQL # Excep CPU 12 0 120ms 3 1 68ms 12 5 60ms 3 1 68ms 75 0 230ms 3 1 68ms Test Framework Results Architectural Data We identified a regresesion Problem solved Lets look behind the scenes Exceptions probably reason for failed tests Problem fixed but now we have an architectural regression Problem fixed but now we have an architectural regression Now we have the functional and architectural confidence
- 36. 3636 How? Performance Focus in Test Automation Embed your Architectural Results in Jenkins
- 37. 3737 Version Control System dynaTrace Server Developer CI Server Commit Trigger build Build and run tests Publish performance metrics Drilldown for further analysis Inform about build status Look beyond test pass/fail!
- 38. 3838 How? Performance Focus in Test Automation Analyzing All Unit / Performance Tests Analyze Perf Metrics Identify Regressions
- 39. 3939 How? Performance Focus in Test Automation Cross Impact of KPIs
- 41. 41 ? 2011 Compuware Corporation ¡ª All Rights Reserved Simply Smarter