ºÝºÝߣ

ºÝºÝߣShare a Scribd company logo

Big Data, Big Changes: Data-Driven Product Development at Etsy

?
?
Jason Davis
Jason Davis

This document summarizes a presentation by Dr. Jason Davis from Etsy about how they use big data and data-driven experiments to improve their products. It discusses how Etsy analyzes over 1.4 billion pageviews and $525 million in sales to conduct dozens of experiments monthly across search, advertising, and analytics. It emphasizes investing in tools for analysis, understanding tradeoffs, and appreciating the complexity of building big data products and algorithms at scale.

1 of 32
Downloaded 27 times
Big Data, Big Changes: Data-Driven Product Development at Etsy Strata London, 2012 Dr. Jason Davis Director of Search & Data jdavis@etsy.com / @jasondavis Monday, October 1, 12
(The change) Monday, October 1, 12
6 months from inception to launch 3 engineering teams 3 product launches Dozens of experiments Lots and lots of data analysis Monday, October 1, 12
The switch to relevancy Search, advertising, analytics, and more Building big data products at Etsy Invest in tooling Understand tradeo?s Appreciate complexity Monday, October 1, 12
Monday, October 1, 12
By the Numbers 1.4B pageviews/month $525MM sales in 2011 40MM unique visitors/month 800k shops / 150 countries Monday, October 1, 12
SELECT?*?FROM?listings? WHERE?title?LIKE? '%silver%' ORDER BY creation_tsz DESC; Monday, October 1, 12
Solving the recency problem Monday, October 1, 12
Better buyer experience Improved search relevancy Monday, October 1, 12
Improved advertising Search Ads Monday, October 1, 12
More search insight for sellers Shop Stats Monday, October 1, 12
Building big data products Monday, October 1, 12
Build lots of tools Monday, October 1, 12
Go to the source Raw data visit logs raw query logs Descriptive statistics search depth query conversion rates Monday, October 1, 12
Invest in tooling Lunr Monday, October 1, 12
Side by side testing Monday, October 1, 12
A/B Analyzer Monday, October 1, 12
Shop Stats with Referrals Monday, October 1, 12
Understand tradeo?s (most tradeo?s don¡¯t look like this) Monday, October 1, 12
The bike is 20% lighter! Monday, October 1, 12
Queries a, b, & c are better. Queries x & y are not. Monday, October 1, 12
?Hola! ???????! Guten Tag ÄãºÃ !"#$% ??*)'?? ? Hello ??#$ ½ñÈÕ¤Ï Monday, October 1, 12
Thinking short-term vs long-term How will the change a?ect behavior? A?ect expectations? User incentives? Improved data? Monday, October 1, 12
Appreciate complexity Monday, October 1, 12
Vs Algorithmic Complexity Monday, October 1, 12
Success / Complexity Tradeo? 4 months 30 experiments 11 wins Monday, October 1, 12
UI June 2005 Complexity November 2006 May 2008 Today Monday, October 1, 12
Communications & Support Complexity Monday, October 1, 12
(The change) Monday, October 1, 12
Etsy - Strata NYC Continous Experimentation with Continuous Deplyoment Steve Mardenfeld Scala + Cascading = Scalding Avi Bryant Case Study: What¡¯s a Customer Worth? Roberto Medri October 23-25 Monday, October 1, 12
Resources Code as Craft blog http://codeascraft.etsy.com/ ºÝºÝߣs (including these) http://www.slideshare.net/etsy Github http://github.com/etsy/ Join in the fun http://www.etsy.com/careers/ Monday, October 1, 12
Monday, October 1, 12

More Related Content

Big Data, Big Changes: Data-Driven Product Development at Etsy

  • 1. Big Data, Big Changes: Data-Driven Product Development at Etsy Strata London, 2012 Dr. Jason Davis Director of Search & Data jdavis@etsy.com / @jasondavis Monday, October 1, 12
  • 3. 6 months from inception to launch 3 engineering teams 3 product launches Dozens of experiments Lots and lots of data analysis Monday, October 1, 12
  • 4. The switch to relevancy Search, advertising, analytics, and more Building big data products at Etsy Invest in tooling Understand tradeo?s Appreciate complexity Monday, October 1, 12
  • 6. By the Numbers 1.4B pageviews/month $525MM sales in 2011 40MM unique visitors/month 800k shops / 150 countries Monday, October 1, 12
  • 7. SELECT?*?FROM?listings? WHERE?title?LIKE? '%silver%' ORDER BY creation_tsz DESC; Monday, October 1, 12
  • 8. Solving the recency problem Monday, October 1, 12
  • 9. Better buyer experience Improved search relevancy Monday, October 1, 12
  • 10. Improved advertising Search Ads Monday, October 1, 12
  • 11. More search insight for sellers Shop Stats Monday, October 1, 12
  • 12. Building big data products Monday, October 1, 12
  • 13. Build lots of tools Monday, October 1, 12
  • 14. Go to the source Raw data visit logs raw query logs Descriptive statistics search depth query conversion rates Monday, October 1, 12
  • 15. Invest in tooling Lunr Monday, October 1, 12
  • 16. Side by side testing Monday, October 1, 12
  • 18. Shop Stats with Referrals Monday, October 1, 12
  • 19. Understand tradeo?s (most tradeo?s don¡¯t look like this) Monday, October 1, 12
  • 20. The bike is 20% lighter! Monday, October 1, 12
  • 21. Queries a, b, & c are better. Queries x & y are not. Monday, October 1, 12
  • 22. ?Hola! ???????! Guten Tag ÄãºÃ !"#$% ??*)'?? ? Hello ??#$ ½ñÈÕ¤Ï Monday, October 1, 12
  • 23. Thinking short-term vs long-term How will the change a?ect behavior? A?ect expectations? User incentives? Improved data? Monday, October 1, 12
  • 25. Vs Algorithmic Complexity Monday, October 1, 12
  • 26. Success / Complexity Tradeo? 4 months 30 experiments 11 wins Monday, October 1, 12
  • 27. UI June 2005 Complexity November 2006 May 2008 Today Monday, October 1, 12
  • 28. Communications & Support Complexity Monday, October 1, 12
  • 30. Etsy - Strata NYC Continous Experimentation with Continuous Deplyoment Steve Mardenfeld Scala + Cascading = Scalding Avi Bryant Case Study: What¡¯s a Customer Worth? Roberto Medri October 23-25 Monday, October 1, 12
  • 31. Resources Code as Craft blog http://codeascraft.etsy.com/ ºÝºÝߣs (including these) http://www.slideshare.net/etsy Github http://github.com/etsy/ Join in the fun http://www.etsy.com/careers/ Monday, October 1, 12