際際滷

際際滷Share a Scribd company logo

Your Game Analytics Playbook

GameAnalytics
GameAnalytics

Topic: What is the role of analytics in running live games? Game studios generally know that they need to instrument and use analytics, but often don't know how to use the data they are presented with to improve their games. Allison presents a "playbook" with detailed instructions on how to analyze the health of a game, what data to be focusing on to get insights and direction, and a framework by which everyone can be involved in the analysis process. This presentation pulls on Allison's experience building the game analytics function at PopCap Games, as well as bring insights from other leaders in the game industry and analytics space. [These were the supporting slides for Allison's talk at Nordic Game 2014]. Allison Bilas Bio: Hailing from Seattle, Washington, Allison earned her MBA at the University of Washington and then joined mobile games giant PopCap in 2011, where she built the game analytics team from scratch and assisted on hit titles Bejeweled Blitz and Plants vs. Zombies 2, amongst others. Now with several years of high-level user behavior analysis under her belt, Bilas joined the GameAnalytics team as VP of Product to manage the product design team and align efforts in making the best game analytics tool for game developers of all sizes.

1 of 16
Downloaded 27 times
Your Game Analytics Playbook
Your Game Analytics Playbook Allison Bilas @allisonbilas / allison@gameanalytics.com 1. Introduction 1. Currently VP of Product @ GameAnalytics 2. Just moved to Denmark, loving hygge. 2. PopCap for last 3.5 years. 1. Hired when they didnt know what to do with data. 2. Built game analytics team from scratch. 3. Learned a LOT about how to infuse data into the decision making process behind developing and operating games. 4. Hope to share some of my experience with you.
Agenda What is a game analytics playbook? Why is it important? Components of a play Examples: pre-season to end season This presentation is about: ! 1. What a game analytics playbook is 2. Why its important to have one 3. Ill go into the component of a game analytics play 4. And Ill give you some concrete examples from the pre-season to the end season ! Lets get started
What is a game analytics playbook? 1. A playbook comes from American football. 1. Im not actually that into the sport. 2. But its a powerful tool. 2. I want you to imagine a football team on the 鍖eld. The QB calls his team together. 1. Competitors position 2. Health of his team 3. Time left on the clock 2. Makes a call for a speci鍖c play > team takes action > success/fail 3. If the game isnt over, they start again. ! ! 5. What allows the QB to make that decision is their playbook. 1. Put together at the beginning of season. 2. Anticipates every situation the team is likely to encounter. ! A game analytics playbook is your plan for using data to inform the development and operations of your game. ! We had one at PopCap that was explicit, but it doesnt have to be.
Information resolves uncertainty. 1. So, why is this important? 2. Assuming youre here because you understand the power of data. 1. Playbook can be used to inform AEM and live ops. 2. In short, it helps to resolve uncertainty and speed up your decision making and development. 3. Caution that analytics doesnt solve everything. Game still needs to be a great experience for your players.
Components of a Play Context Question 1. Lets get into the components of a play. 1. This is essentially the analysis process. 2. Most studios start with thinking about what to track, but there are a couple of steps that should happen before that 2. Like the QB on the football 鍖eld, 鍖rst step is CONTEXT. 1. Where your game is at in its lifecycle. (Well go over these in the next sections.) 2. Competitive context and industry trends 3. Goals for your game in terms of KPIs 3. By assessing the context, youll start to come up with analysis QUESTIONS. 1. Typically focused on AEM, but also can be more technical in nature around things like bugs and frame rates. 2. Likely to have a LOT of questions. 3. Prioritize based on: measureable / actionable / most impact.
Components of a Play Context Question Data / Tools Action 1. DATA AND ANALYSIS TOOLS 1. Here is where you need to start thinking about what to track in order to answer the questions you have 2. In deciding what to track you should also consider the type of analysis youre going to do, for example: 1. Segmentation, benchmarking, A/B testing and trend analysis. 2. Last component is ACTION. Heres is where the playbook is most powerful. 1. If youve done everything right: CONTEXT > QUESTION > DATA/TOOLS 2. Deciding what action should be very easy. Typically implied in the questions youre asking.
Pre-season (Dev) Early Season (Beta) Mid Season (Launch) Late Season (Ongoing Ops) Seasons 1. Next Im going to give examples that follow the life cycle of your game from: 1. Pre-season or development 2. Early season or beta period 3. Its not until the mid season that youve 鍖nally launched 4. And then the late season, or ongoing ops
Pre-Season (Development) Instrument tracking EARLY and TEST Be judicious in what you track Think through event hierarchies Consider multiple tracking systems 1. Pre-season is straight forward - SET UP TRACKING. 2. I recommend you start doing this very early. 1. This means youll track things initially that you dont need in the end. This is okay. 2. It lets you start to re鍖ne the questions you have and ensure that you have the data needed to answer them. 3. Also gives you time to test. Every game Ive every launched has found tracking bugs in the 鍖rst day after launch. 3. Lets you be smart about what you track. E.g. Game I was working with wanted to track every time a player turn on/off the sound. 4. Also in the early season, you should be thinking very closely about your event hierarchies. 1. These can dictate the type of analysis you can do, and its very nuanced. 2. It helps to start tracking and trying out analysis, and then making changes. 5. Multiple tracking systems can add some complexity, but allows validation of data.
Early Season (Beta) 1. Early season is BETA period. 1. Select markets, and you have data! YAY! 2. Exciting period. Easy to loose focus. 2. Most important = look for unexpected behaviors in gameplay and monetization. 3. HIDDEN AGENDA 1. Hidden object game launched on FB a few years ago. Few hundred Australians playing it every day. 2. One player was spending a lot of money in the game. WHALE! 3. What was she spending money on > gotcha boxes of consumables. 4. How was she progressing in the game > noticed she was stuck on Level 18. 5. Needed a 鍖nger print duster to move forward. But that item wasnt available until Level 20. BLESS HER!
Early Season (Beta) 1. Because it was the beta period, this wasnt terrible. We were able to identify the weird behavior, 鍖nd the problem and 鍖x it. 2. In terms of analysis in the beta period: 1. Segmentation is very important. In addition to the few hundred Australians, we had thousands of Philipine installs coming and going in the game in order to pressure test. 2. Important to 鍖lter and look at them separately. 3. I also want to mention that A/B testing isnt necessary. Lots of info just by observing.
Mid Season (Launch) 1. Finally at mid-season the game launches! This is when things start moving really quickly. 2. Analysis should focus on high level metrics > dive deeper when something isnt working. 3. One example from my experience at PopCap was when I was working with PVZ2. 1. Immediately after launch, millions of installs. Playing for a few days and then leaving. Retention was horrible. Everything else was good! 2. Digging in to engagement players spending less than 5 minutes per session in the game. BUT, time to complete a level took 15 to 20 minutes. 3. Players had to return 3 or 4 times to the game before they had a reward moment. Felt like a slog. 4. Team identi鍖ed this and set about 鍖nding ways to shorten the game lengths.
Mid Season (Launch) 1. Quite honestly, too late! Millions came and left and it was hard to get them to come back. 2. In launch analysis, youll be looking at data all the time. HOURLY, DAILY. 3. Intense period lasts probably 1 month THEN
Late Season (Ongoing Ops) 1. At some point you need to slow down and go deeper with your analysis. This is your end game, but it should never actually end! 1. Focus on re鍖nement and longer term game design. 2. Robust content pipeline and promotional program. 2. Start looking at trends in the data. 3. Bejeweled Blitz Rare Gems 1. Super interesting game. FB for 5yrs, mobile for 3yrs. 2. Success because of content and promotions. 3. Main content is Rare Gems, extra powerful boosts. 4. Had to launch a few of them look at trends in order to understand what made them successful. 1. First one was a big hit, just because it was 鍖rst. 2. 2nd and 3rd were also good 3. Wasnt until 6th and 7th that we started to get it. 5. Rare Gem experience is made up of: 1. In game visuals and explosions 2. Last hurrah with explosions 3. Slot machine mechanic at the end 6. Most successful had ALL of them, but with trend analysis we could understand the tradeoffs between including each of these components or not. 7. This informed our development and let us make explicit decisions around the tradeoffs in the content design.
Summary A game analytics playbook is your plan for using data to inform the decisions you make in operating your game. Context Question Data / Tools Action 1. Ive told you what a playbook is and how it relates to building and operating you game. 2. Your next step is to consider where your game is at in its lifecycle, and start developing your plays. ! 3. I would love to talk more about this with you! I love thinking through this process and how to make it both accessible and actionable.
Allison Bilas @allisonbilas / allison@gameanalytics.com THANK YOU!

More Related Content

Your Game Analytics Playbook

  • 2. Your Game Analytics Playbook Allison Bilas @allisonbilas / allison@gameanalytics.com 1. Introduction 1. Currently VP of Product @ GameAnalytics 2. Just moved to Denmark, loving hygge. 2. PopCap for last 3.5 years. 1. Hired when they didnt know what to do with data. 2. Built game analytics team from scratch. 3. Learned a LOT about how to infuse data into the decision making process behind developing and operating games. 4. Hope to share some of my experience with you.
  • 3. Agenda What is a game analytics playbook? Why is it important? Components of a play Examples: pre-season to end season This presentation is about: ! 1. What a game analytics playbook is 2. Why its important to have one 3. Ill go into the component of a game analytics play 4. And Ill give you some concrete examples from the pre-season to the end season ! Lets get started
  • 4. What is a game analytics playbook? 1. A playbook comes from American football. 1. Im not actually that into the sport. 2. But its a powerful tool. 2. I want you to imagine a football team on the 鍖eld. The QB calls his team together. 1. Competitors position 2. Health of his team 3. Time left on the clock 2. Makes a call for a speci鍖c play > team takes action > success/fail 3. If the game isnt over, they start again. ! ! 5. What allows the QB to make that decision is their playbook. 1. Put together at the beginning of season. 2. Anticipates every situation the team is likely to encounter. ! A game analytics playbook is your plan for using data to inform the development and operations of your game. ! We had one at PopCap that was explicit, but it doesnt have to be.
  • 5. Information resolves uncertainty. 1. So, why is this important? 2. Assuming youre here because you understand the power of data. 1. Playbook can be used to inform AEM and live ops. 2. In short, it helps to resolve uncertainty and speed up your decision making and development. 3. Caution that analytics doesnt solve everything. Game still needs to be a great experience for your players.
  • 6. Components of a Play Context Question 1. Lets get into the components of a play. 1. This is essentially the analysis process. 2. Most studios start with thinking about what to track, but there are a couple of steps that should happen before that 2. Like the QB on the football 鍖eld, 鍖rst step is CONTEXT. 1. Where your game is at in its lifecycle. (Well go over these in the next sections.) 2. Competitive context and industry trends 3. Goals for your game in terms of KPIs 3. By assessing the context, youll start to come up with analysis QUESTIONS. 1. Typically focused on AEM, but also can be more technical in nature around things like bugs and frame rates. 2. Likely to have a LOT of questions. 3. Prioritize based on: measureable / actionable / most impact.
  • 7. Components of a Play Context Question Data / Tools Action 1. DATA AND ANALYSIS TOOLS 1. Here is where you need to start thinking about what to track in order to answer the questions you have 2. In deciding what to track you should also consider the type of analysis youre going to do, for example: 1. Segmentation, benchmarking, A/B testing and trend analysis. 2. Last component is ACTION. Heres is where the playbook is most powerful. 1. If youve done everything right: CONTEXT > QUESTION > DATA/TOOLS 2. Deciding what action should be very easy. Typically implied in the questions youre asking.
  • 8. Pre-season (Dev) Early Season (Beta) Mid Season (Launch) Late Season (Ongoing Ops) Seasons 1. Next Im going to give examples that follow the life cycle of your game from: 1. Pre-season or development 2. Early season or beta period 3. Its not until the mid season that youve 鍖nally launched 4. And then the late season, or ongoing ops
  • 9. Pre-Season (Development) Instrument tracking EARLY and TEST Be judicious in what you track Think through event hierarchies Consider multiple tracking systems 1. Pre-season is straight forward - SET UP TRACKING. 2. I recommend you start doing this very early. 1. This means youll track things initially that you dont need in the end. This is okay. 2. It lets you start to re鍖ne the questions you have and ensure that you have the data needed to answer them. 3. Also gives you time to test. Every game Ive every launched has found tracking bugs in the 鍖rst day after launch. 3. Lets you be smart about what you track. E.g. Game I was working with wanted to track every time a player turn on/off the sound. 4. Also in the early season, you should be thinking very closely about your event hierarchies. 1. These can dictate the type of analysis you can do, and its very nuanced. 2. It helps to start tracking and trying out analysis, and then making changes. 5. Multiple tracking systems can add some complexity, but allows validation of data.
  • 10. Early Season (Beta) 1. Early season is BETA period. 1. Select markets, and you have data! YAY! 2. Exciting period. Easy to loose focus. 2. Most important = look for unexpected behaviors in gameplay and monetization. 3. HIDDEN AGENDA 1. Hidden object game launched on FB a few years ago. Few hundred Australians playing it every day. 2. One player was spending a lot of money in the game. WHALE! 3. What was she spending money on > gotcha boxes of consumables. 4. How was she progressing in the game > noticed she was stuck on Level 18. 5. Needed a 鍖nger print duster to move forward. But that item wasnt available until Level 20. BLESS HER!
  • 11. Early Season (Beta) 1. Because it was the beta period, this wasnt terrible. We were able to identify the weird behavior, 鍖nd the problem and 鍖x it. 2. In terms of analysis in the beta period: 1. Segmentation is very important. In addition to the few hundred Australians, we had thousands of Philipine installs coming and going in the game in order to pressure test. 2. Important to 鍖lter and look at them separately. 3. I also want to mention that A/B testing isnt necessary. Lots of info just by observing.
  • 12. Mid Season (Launch) 1. Finally at mid-season the game launches! This is when things start moving really quickly. 2. Analysis should focus on high level metrics > dive deeper when something isnt working. 3. One example from my experience at PopCap was when I was working with PVZ2. 1. Immediately after launch, millions of installs. Playing for a few days and then leaving. Retention was horrible. Everything else was good! 2. Digging in to engagement players spending less than 5 minutes per session in the game. BUT, time to complete a level took 15 to 20 minutes. 3. Players had to return 3 or 4 times to the game before they had a reward moment. Felt like a slog. 4. Team identi鍖ed this and set about 鍖nding ways to shorten the game lengths.
  • 13. Mid Season (Launch) 1. Quite honestly, too late! Millions came and left and it was hard to get them to come back. 2. In launch analysis, youll be looking at data all the time. HOURLY, DAILY. 3. Intense period lasts probably 1 month THEN
  • 14. Late Season (Ongoing Ops) 1. At some point you need to slow down and go deeper with your analysis. This is your end game, but it should never actually end! 1. Focus on re鍖nement and longer term game design. 2. Robust content pipeline and promotional program. 2. Start looking at trends in the data. 3. Bejeweled Blitz Rare Gems 1. Super interesting game. FB for 5yrs, mobile for 3yrs. 2. Success because of content and promotions. 3. Main content is Rare Gems, extra powerful boosts. 4. Had to launch a few of them look at trends in order to understand what made them successful. 1. First one was a big hit, just because it was 鍖rst. 2. 2nd and 3rd were also good 3. Wasnt until 6th and 7th that we started to get it. 5. Rare Gem experience is made up of: 1. In game visuals and explosions 2. Last hurrah with explosions 3. Slot machine mechanic at the end 6. Most successful had ALL of them, but with trend analysis we could understand the tradeoffs between including each of these components or not. 7. This informed our development and let us make explicit decisions around the tradeoffs in the content design.
  • 15. Summary A game analytics playbook is your plan for using data to inform the decisions you make in operating your game. Context Question Data / Tools Action 1. Ive told you what a playbook is and how it relates to building and operating you game. 2. Your next step is to consider where your game is at in its lifecycle, and start developing your plays. ! 3. I would love to talk more about this with you! I love thinking through this process and how to make it both accessible and actionable.
  • 16. Allison Bilas @allisonbilas / allison@gameanalytics.com THANK YOU!