際際滷

際際滷Share a Scribd company logo

Measuring mobile performance (@LDNWebPerf Version)

Download as PPTX, PDF
Stephen Thair
Stephen Thair

A presentation to the London Web Performance User Group covering the different ways of measuring Mobile web performance and some of the strength & weaknesses of each, depending on your needs.

1 of 49
Stephen Thair, Seriti Consulting, @TheOpsMgrLondon Web Performance Meetup Group, @LDNWebPerfMeasuring mobile performance14/09/2011息 Seriti Consulting1
Before we start3Things to keep in mind14/09/2011息 Seriti Consulting2
DesktopMobile14/09/2011息 Seriti Consulting3
A Java app calling a java app calling a C++ app using JNI to call a java app.14/09/2011息 Seriti Consulting4http://assets.en.oreilly.com/1/event/60/Analyzing%20the%20Performance%20of%20Mobile%20Web_%20Challenges%20and%20Techniques%20Presentation.pdf
WifiMobile14/09/2011息 Seriti Consulting5
WiFivs HSDPA Performance 14/09/2011息 Seriti Consulting6
MobileMobile14/09/2011息 Seriti Consulting7
Different Phones = Different Performance14/09/2011息 Seriti Consulting8http://www.anandtech.com/show/4144/lg-optimus-2x-nvidia-tegra-2-review-the-first-dual-core-smartphone/8
Different operating systems14/09/2011息 Seriti Consulting9
Different Browsers14/09/2011息 Seriti Consulting10http://www.webdevelopersnotes.com/articles/mobile-web-browsers-list.php
Different network types14/09/2011息 Seriti Consulting11http://developer.android.com/reference/android/telephony/TelephonyManager.html
Mobile Performance is location-dependent14/09/2011息 Seriti Consulting12http://mytests.3pmobile.com/context/?WyIyUFVDZHlWV2ErZHFISmxCSWFXRENnPT0iLCJEYnFYVVp3SEN4MD0iXQ%3D%3D
Page Load times (along the thames)14/09/2011息 Seriti Consulting13Off the chart (2m36s)
HSDPA on a #!%&$@*ing train14/09/2011息 Seriti Consulting14Max RTT = 20266mshttp://blog.davidsingleton.org/mobiletcp
How important is performance on mobile, anyway?14/09/2011息 Seriti Consulting15
You only have 1.5 seconds on mobile!!!14/09/2011息 Seriti Consulting16
How many ways are there to measure web performance?And how useful are they on Mobile?14/09/2011息 Seriti Consulting17
614/09/2011息 Seriti Consulting18
6Browser plug-ins14/09/2011息 Seriti Consulting19As used in HTTPWatch, DynatraceAjax Edition etcBut only some mobile browsers support plug-ins Android & iPhone = No, WinMo & Blackberry = Yes.
5Proxy Timing14/09/2011息 Seriti Consulting20E.g. Fiddler Proxy (http://www.fiddler2.com/Fiddler2/version.asp)There is no proxy settings in Android unless you root your phone (http://android-proxy.blogspot.com/)Only useful for WiFiBeware the observer effect
4Web Server Module14/09/2011息 Seriti Consulting21As used by many APM solutions e.g. Dynatrace, New RelicInstall mod_ or ISAPI filter Measure the RTT between web server and DB/Application tierGreat for measuring your back-end performanceNot much good for front end
3JavaScript Timing14/09/2011息 Seriti Consulting22As used in Boomerang, Episodes, GA, WebTuna etcOK for web pages, better with new WebTiming and NavigationTiming APIs not that mobile browsers support themRelies on JavaScript and Cookies which is problematic
2 Network level Timing14/09/2011息 Seriti Consulting23As used by PCAPPerf and by Atomic Labs Pion, Tealeaf, CoradiantPCAPPERF limited to over WiFi (otherwise you cant capture the packets)*Pion, Tealeaf etc require network tap
1 Custom browser14/09/2011息 Seriti Consulting24As used by 3PMobile.com (on the phone) and synthetic agents (by active monitoring providers e.g. Site Confidence)At present this is the only way to get real OTA performance data.
14/09/2011息 Seriti Consulting25So what to use?
14/09/2011息 Seriti Consulting26Depends on your needs
14/09/2011息 Seriti Consulting27Development vsReal World
14/09/2011息 Seriti Consulting28Network perspective vsBrowser Perspective?
PCAPPerf Testing 14/09/2011息 Seriti Consulting29WiFi Connection to PCPC acting as Wifi hotspot and capturing packets with WiresharkUpload pcap file to pcapperf.appspot.comAndView the waterfall Uploadhttp://code.google.com/p/pcaphar/wiki/CaptureMobileTraffics
Mobitest blaze.io/mobile 14/09/2011息 Seriti Consulting30PCAPPerfas SaaS + some on-device event capture + screenshotsWiFi onlyTethered machines in Ottawa, DC, Amsterdam
Mobitest results screen14/09/2011息 Seriti Consulting31Download timePage SizeWaterfallScreenshotLink to HAR
Mobitest / PCAPPerf waterfall14/09/2011息 Seriti Consulting32
3PMobile.com Browser214/09/2011息 Seriti Consulting33Android Native Browser code instrumented for performance measurementTest Anywhere you have signalFull Mobile context device & location & network. WiFi & 3G.Browser-level perspectiveScreenshotsAutomated TestingHTML, CSS, Script source code
3PMobile.com Browser2 Waterfall14/09/2011息 Seriti Consulting34Waterfall of www.ft.com (redirected to http://m.ft.com)
Screenshots14/09/2011息 Seriti Consulting35
Device Context14/09/2011息 Seriti Consulting36
14/09/2011息 Seriti Consulting37Location Context
14/09/2011息 Seriti Consulting38Network Context
Download the HTML, Script, CSS source14/09/2011息 Seriti Consulting39
Sign up for free Beta @ www.3pmobile.com14/09/2011息 Seriti Consulting40
14/09/2011息 Seriti Consulting41So what else is there?
TCP Dump on Rooted phone14/09/2011息 Seriti Consulting42http://www.vbsteven.be/blog/android-debugging-inspectin-network-traffic-with-tcpdump/
JavaScript Bookmarklets14/09/2011息 Seriti Consulting43Steve Souders Mobile Performance Bookmarlethttp://stevesouders.com/mobileperf/mobileperfbkm.phpNone of these measure performance per se, but they can give you more information about the page (but beware of a strong observer effect)
Keynote MITE (Emulation)14/09/2011息 Seriti Consulting44The free Keynote MITE emulatorWebkit-based emulation based on device profiles
Debugging tools14/09/2011息 Seriti Consulting45Most SDKs include debugging tools and profilers that can help you understand whats going onBut you are tethered to your PC (normally)
WatchmouseIAM library14/09/2011息 Seriti Consulting46In-App Monitoring (IAM) is a set of libraries to help instrument your web apps. Currently iPhone only. Replace your app WebView calls with IAMWebViewIt will gather the performance data and beacon it out for youhttps://bitbucket.org/watchmouse/iam/
BBC / Epitro Signal Strength Survey14/09/2011息 Seriti Consulting47http://www.bbc.co.uk/news/technology-14644507
14/09/2011息 Seriti Consulting48And thats all Ive got
14/09/2011息 Seriti Consulting49Questions?Stephen Thair, Seriti Consulting, @TheOpsMgrStephen.thair@seriticonsulting.comwww.seriticonsulting.com/blogLondon Web Performance Meetup Group, @LDNWebPerfMonthly Meetups in Central London (near Bank)http://www.meetup.com/London-Web-Performance-Group/

More Related Content

Measuring mobile performance (@LDNWebPerf Version)

Editor's Notes

  1. Forget what you thought you knew
  2. Here is one reason why a mobile browser isnt the same as a desktop browser.(on Android) the web browser is a Java App calling a C/C++ app which calls another C app
  3. Different bandwidthDifferent latencyDifferent network layer entirely (Backhaul probably isnt even IP-based)
  4. And if you throw in CyanogenMod you can get even better performance
  5. And if you throw in CyanogenMod you can get even better performance
  6. What type of network is it connected to?
  7. I was on the Thames Clipper on my way back from the O2 after a concert. So I ran some automated testing and this graphs my results Average32.859Median12.071Std Deviation45.691Min9.275Max156.520
  8. Test MarkerTime (Sec)Signal Strength (ASU)19.2753212.681339.68274156.52018512.071969.28513752.86814810.1849923.1658Average32.859Median12.071Std Deviation45.691
  9. Snakes on a motherf**king plane! Network latency varies enormously. What can we see here?The connection is relatively good at sending and receiving data.The green crosses represent packets that were actually lost which only happened for 75 of 1000.There are some crazy-high round trip times.The minimum round trip time was 107ms (which would put my home cable connection to shame) and even the median is pretty awesome at 239ms but the maximum was a whopping 20226 ms - that's more than 20 seconds!
  10. 1.JavaScript timing e.g. WebTuna, Boomerang, Episodes2.Browser plug-in e.g. HTTPwatch3.Custom browser variant e.g. A Webkit-build (often used as a monitoring agent)4.Proxy timing e.g. Fiddler proxy5.Web Server Module e.g. APM solutions like AppDynamics, Dynatraceetc6.Network-level e.g. Atomic Labs Pion, Tealeaf, Coradiant
  11. 2.Browser plug-in e.g. HTTPwatchBut only some mobile browsers have a plug-in architecture and exactly what timing info that would provide I have no idea
  12. 4.Proxy timing e.g. Fiddler proxy or Charles ProxyObserver Affect #1 = http://www.fiddler2.com/fiddler/perf/PerfMeasure.aspObserver Affect #2= http://insidehttp.blogspot.com/2005/06/using-fiddler-for-performance.html
  13. 5.Web Server Module e.g. APM solutions like AppDynamics, Dynatraceetc
  14. 1.JavaScript timing e.g. WebTuna, Boomerang, EpisodesYou can also use YUI Profiler I believe http://developer.yahoo.com/yui/profilerI have seen JavaScript warnings appear due to excessive time in the onBeforeUnload event so keep in mind that JavaScript is a lot slower in mobile devices and hence the observer effect is larger (and the timing more accurate).
  15. Network tap based solutions are great for measuring your own site, not so great for measuring 3rd party sites.PCAPPerf is limited to WiFi which isnt anything like the real mobile experience. The mobile network stack is NOT the same as the desktop stack, and the client side delay is larger also. *In theory you can use tcpdump on a rooted phone. More on that later
  16. The mobile network stack is NOT the same as the desktop stack, and the client side delay is larger also.
  17. Is a tethered, rooted phone over wifi adequate to your needs?
  18. Is knowing whats happening on the wire enough, or do you want to understand whats happening at the browser level?And the mobile context So lets look at a few in detail
  19. Flexible solution (in theory works with every WiFi enabled phone)But WiFi isnt mobileAnd its a faff
  20. Essentially the PCAPPerf approach as SaaSBut it does support iPhone and Android.
  21. Currently Android only but PoC code for Blackberry and iPhone exists. iPhone client in 90 to 120 days, all being well.
  22. Rooted and Tethered, again
  23. Even a PC-based version of Webkit isnt the same as webkit running on a mobile device. But useful none the less. Lots of other features and unsurprisingly there is a (expensive) Pro version The MITE Pro annual user license is $5,000 and includes 100 hours of free remote testing. To learn more about single- and multiple-license purchases, please contact us today.
  24. The mobile network stack is NOT the same as the desktop stack, and the client side delay is larger also.