Knowing Ranking Factors won't be enough!
0 likes55 views
The document discusses how understanding ranking factors in SEO has evolved since 2009, highlighting that while humans struggled with the algorithm, advancements in machine learning are complicating this further. It emphasizes the importance of adapting human methodologies and consulting skills to remain relevant in a world where machines excel in diagnostics. In conclusion, it suggests that despite the opaque nature of ranking factors, there's a necessity for humans to leverage expertise, avoid biases, and focus on effective debugging to maintain a competitive edge.
![We sell custom cigar humidors. Our
custom cigar humidors are handmade. If
youre thinking of buying a custom cigar
humidor, please contact our custom
cigar humidor specialists at
custom.cigar.humidors@example.com
What this needs is another mention of [cigar humidors]](https://image.slidesharecdn.com/willcritchlow-mkgo2018-beattherobotswidescreen-180714083348/85/Knowing-Ranking-Factors-won-t-be-enough-48-320.jpg)
![With no mentions of [cigar] or [humidor] this
page would be unlikely to rank
And yet you can clearly go too far, and have the effect turn negative.
This is called nonlinearity.
The cigar example is taken directly from Googles quality guidelines.](https://image.slidesharecdn.com/willcritchlow-mkgo2018-beattherobotswidescreen-180714083348/85/Knowing-Ranking-Factors-won-t-be-enough-49-320.jpg)
![Hughes sent his team an email. All
projects on the old system were to be
suspended immediately.
[Read the whole story ]](https://image.slidesharecdn.com/willcritchlow-mkgo2018-beattherobotswidescreen-180714083348/85/Knowing-Ranking-Factors-won-t-be-enough-83-320.jpg)
![Rules of ML [PDF] outlines engineering lessons
from getting ML into production at Google](https://image.slidesharecdn.com/willcritchlow-mkgo2018-beattherobotswidescreen-180714083348/85/Knowing-Ranking-Factors-won-t-be-enough-97-320.jpg)
Knowing Ranking Factors won't be enough!
- 1. Knowing ranking factors wont be enough How to avoid losing your job to a robot @willcritchlow - MKGO 2018
- 2. Im going to tell you about a robot that understands ranking factors better than any of you ...but before I get to that, lets look at a bit of history...
- 3. The other day I searched:
- 4. Unsurprisingly, I got an answer
- 5. But it got me thinking about how, in 2009, the results would have looked more like this.
- 6. In 2009, it would have looked more like this. With every title containing the keyphrase.
- 7. In 2009, it would have looked more like this. With every title containing the keyphrase. Most at the beginning.
- 8. OK. Maybe wikipedia would have been #1.
- 9. We used to have a pretty good understanding of ranking factors
- 10. My mental model for ~2009 ranking factors had three different modes:
- 11. One in the hyper-competitive head My mental model for ~2009 ranking factors had three different modes: One in the competitive mid-tail ...and one in the long-tail
- 13. Tons of perfectly on-topic pages to choose from One in the hyper-competitive head
- 14. So pick only perfectly-on-topic pages One in the hyper-competitive head
- 15. (*) Page authority, but the domain inevitably factors into that calculation. This is why so many homepages ranked One in the hyper-competitive head ...and rank by authority (*)
- 16. This resulted in a mix of homepages of mid-size sites, and inner pages on huge sites One in the hyper-competitive head
- 17. But the general way to move up was through increased authority One in the hyper-competitive head
- 18. Kind of search result Pages ranking To move up... Head Homepages of mid-size sites and inner pages of massive sites. All perfectly-targeted. Improve authority. Mid-tail Long-tail
- 19. One in the hyper-competitive head One in the competitive mid-tail
- 20. Wealth of ROUGHLY on-topic pages to choose from One in the competitive mid-tail
- 21. PERFECTLY on-topic could do well even on a relatively weak site One in the competitive mid-tail
- 22. Rank the roughly on-topic pages by authority x on-topicness One in the competitive mid-tail
- 23. Move up with better targeting or more authority One in the competitive mid-tail
- 24. Kind of search result Pages ranking To move up... Head Homepages of mid-size sites and inner pages of massive sites. All perfectly-targeted. Improve authority. Mid-tail Perfectly on-topic pages on relatively weak sites plus roughly on-topic on bigger sites. Improve targeting or authority. Long-tail
- 25. One in the hyper-competitive head One in the competitive mid-tail ...and one in the long-tail
- 26. In the long-tail, a site of arbitrary weakness could rank if it was the most relevant ...and one in the long-tail
- 27. Otherwise, massive sites rank with off-topic pages that mention something similar ...and one in the long-tail
- 28. Generally, move up with better targeting ...and one in the long-tail
- 29. Kind of search result Pages ranking To move up... Head Homepages of mid-size sites and inner pages of massive sites. All perfectly-targeted. Improve authority. Mid-tail Perfectly on-topic pages on relatively weak sites plus roughly on-topic on bigger sites. Improve targeting or authority. Long-tail Arbitrarily-weak on-topic pages and roughly-targeted deep pages on massive sites. Improve targeting.
- 30. Kind of search result Pages ranking To move up... Head Homepages of mid-size sites and inner pages of massive sites. All perfectly-targeted. Improve authority. Mid-tail Perfectly on-topic pages on relatively weak sites plus roughly on-topic on bigger sites. Improve targeting or authority. Long-tail Arbitrarily-weak on-topic pages and roughly-targeted deep pages on massive sites. Improve targeting. So that was ~2009
- 31. Its not so simple any more. Google is harder to understand these days.
- 32. PageRank (the first algorithm to use the link structure of the web) We know how we got to ~2009...
- 35. Information retrieval PageRank Original research TWEAKS ...with growing complexity in subsequent years
- 36. Particularly this comment from a user called Kevin Lacker (@lacker):
- 37. I was thinking about it like it was a math puzzle and if I just thought really hard it would all make sense. -- Kevin Lacker (@lacker)
- 38. Hey why don't you take the square root? -- Amit Singhal according to Kevin Lacker (@lacker)
- 39. oh... am I allowed to write code that doesn't make any sense? -- Kevin Lacker (@lacker)
- 40. -- Amit Singhal according to Kevin Lacker (@lacker) Multiply by 2 if it helps, add 5, whatever, just make things work and we can make it make sense later.
- 41. Why does this make the algorithm so hard to understand?
- 46. You might know what any one of the levers does, but they can interact with each other in complex ways This is what a high-dimensional function looks like
- 48. We sell custom cigar humidors. Our custom cigar humidors are handmade. If youre thinking of buying a custom cigar humidor, please contact our custom cigar humidor specialists at custom.cigar.humidors@example.com What this needs is another mention of [cigar humidors]
- 49. With no mentions of [cigar] or [humidor] this page would be unlikely to rank And yet you can clearly go too far, and have the effect turn negative. This is called nonlinearity. The cigar example is taken directly from Googles quality guidelines.
- 51. Discontinuities are steps in the function Think about so-called over-optimization tipping points
- 52. Lets put all this together into a practical example:
- 53. Think about category pages: Do you recommend removing SEO text? Weve tested it, so we know the answer.
- 54. If you said yes, congratulations (+3.1% organic sessions in a split-test)
- 55. Unless youre responsible for this site No effect / possible negative effect
- 56. No, but Im still pretty good at this Youre thinking this to yourself right now.
- 73. I promised to tell you about a robot that is better than even experienced SEOs... Well. It turns out all we needed was a coin to flip. Youre all fired.
- 76. Its only going to get worse under Sundar Pichai
- 77. Sundar takes over as CEO Singhal leaves Giannandrea takes over Jeff Dean does Jeff Dean things
- 78. The original Google Translate was the result of the work of hundreds of engineers over 10 years.
- 79. Director of Translate, Macduff Hughes said that it sounded to him as if maybe they could pull off a neural-network-based replacement in three years.
- 80. Jeff Dean said we can do it by the end of the year, if we put our minds to it.
- 81. Hughes: Im not going to be the one to say Jeff Dean cant deliver speed.
- 82. A month later, the work of a team of 3 engineers was tested against the existing system. The improvement was roughly equivalent to the improvement of the old system over the previous 10 years.
- 83. Hughes sent his team an email. All projects on the old system were to be suspended immediately. [Read the whole story ]
- 85. How to avoid losing your job to a robot This is what you promised, Will.
- 86. Lets start by understanding some robot weaknesses
- 87. Whats this?
- 88. Ooh. Ooh. I know this one. -- robot
- 89. Its a leopard. Im like 99% sure.
- 90. Computers are better than humans at classification, but struggle with adversaries Read more about this here -- Cheetah, Leopard, Jaguar
- 91. We dont fully understand all ML mistakes See: adversarial AI
- 92. And when youre trying to fool the machine... See: adversarial AI
- 93. And when youre trying to fool the machine... See: adversarial AI
- 94. You get some really wild examples See: adversarial AI
- 95. Lesson: We expect adversarial abilities to take a step backwards They will remain good at classifying bad links but will be likely to fall prey to weird outcomes in adversarial situations
- 96. Were going to see new kinds of bugs
- 97. Rules of ML [PDF] outlines engineering lessons from getting ML into production at Google
- 98. That document also has a section on trying to understand what the machines are doing
- 99. But human explainability may not even be possible Not every concept a neural network uses fits neatly into a concept for which we have a word. Its not clear this is a weakness per se, but...
- 100. ...this means that engineers wont always know more than we do about why a page does or doesnt rank The big knowledge gap of the future is data - clickthrough rates, bounce rates etc.
- 101. So how do we fight back?
- 102. Michael Lewis latest book is about Kahneman and Tversky spelling. It recounts a story about a piece of medical software that existed in the 1960s.
- 103. It was designed to encapsulate how a range of doctors diagnosed stomach cancer from x-rays.
- 104. It proceeded to outperform those same doctors despite only containing their expertise. Real people have biases, and fool themselves. Encapsulate your own expert knowledge.
- 105. At Distilled, we use a methodology we call the balanced digital scorecard. This encapsulates our beliefs about how to build a high-performing business. Applying it helps avoid our own biases.
- 106. Also, while we are talking about books, The Checklist Manifesto is an important part of avoiding the same cognitive biases.
- 107. Focus on consulting skills Ive written a few things about this (DistilledU module, writing better business documents, using split-tests to consult better). Use case studies and creativity. Computers are better at diagnosis than cure. This means: getting things done, convincing organizations, applying general knowledge, learning new things.
- 108. We are going to need to be better than ever at debugging things. I wrote about debugging skills for non-developers here. A lot of the story of enterprise consulting is going to be about figuring out why things have gone wrong in the face of sparse or incorrect information from Google.
- 110. Disregard expert surveys Firstly, there are all the problems outlined in the search result pairs study - both in the ability of experts to understand factors, and in your ability to use the information even if they do. Secondly, they are broken with another bias called the law of small numbers from Lewis book. PS - I say this as a participant in many of them Me
- 111. Equally, building your digital strategy on what Google tells you to do will become an even worse idea than it already is.
- 112. Check out Tom Cappers presentation on how engineers statements can be misleading
- 113. ...and remember the confounding split-tests Its already not always as simple as feature X is good Which all means we may need to be more independent-minded and do more of our own research
- 114. This is why we have been investing so much in split-testing Check out odn.distilled.net if you havent already. The team will be happy to demo for you. We served ~5 billion requests last quarter and recently published everything from response times to our +贈100k / month split test.
- 115. Lets recap 1. Even in a world of 200+ classical ranking factors, humans were bad at understanding the algorithm
- 116. Lets recap 1. Even in a world of 200+ classical ranking factors, humans were bad at understanding the algorithm 2. Machine learning will make this worse, and is accelerating under Sundar
- 117. Lets recap 1. Even in a world of 200+ classical ranking factors, humans were bad at understanding the algorithm 2. Machine learning will make this worse, and is accelerating under Sundar 3. There are things computers remain bad at, and rankings will become more opaque even to Google engineers
- 118. Lets recap 1. Even in a world of 200+ classical ranking factors, humans were bad at understanding the algorithm 2. Machine learning will make this worse, and is accelerating under Sundar 3. There are things computers remain bad at, and rankings will become more opaque even to Google engineers 4. We remain relevant by: a. Using methodologies and checklists to capture human capabilities and avoid our biases
- 119. Lets recap 1. Even in a world of 200+ classical ranking factors, humans were bad at understanding the algorithm 2. Machine learning will make this worse, and is accelerating under Sundar 3. There are things computers remain bad at, and rankings will become more opaque even to Google engineers 4. We remain relevant by: a. Using methodologies and checklists to capture human capabilities and avoid our biases b. Becoming great consultants and change agents
- 120. Lets recap 1. Even in a world of 200+ classical ranking factors, humans were bad at understanding the algorithm 2. Machine learning will make this worse, and is accelerating under Sundar 3. There are things computers remain bad at, and rankings will become more opaque even to Google engineers 4. We remain relevant by: a. Using methodologies and checklists to capture human capabilities and avoid our biases b. Becoming great consultants and change agents c. Debugging the heck out of everything
- 121. Lets recap 1. Even in a world of 200+ classical ranking factors, humans were bad at understanding the algorithm 2. Machine learning will make this worse, and is accelerating under Sundar 3. There are things computers remain bad at, and rankings will become more opaque even to Google engineers 4. We remain relevant by: a. Using methodologies and checklists to capture human capabilities and avoid our biases b. Becoming great consultants and change agents c. Debugging the heck out of everything d. Avoiding being misled by experts or Google
- 122. Lets recap 1. Even in a world of 200+ classical ranking factors, humans were bad at understanding the algorithm 2. Machine learning will make this worse, and is accelerating under Sundar 3. There are things computers remain bad at, and rankings will become more opaque even to Google engineers 4. We remain relevant by: a. Using methodologies and checklists to capture human capabilities and avoid our biases b. Becoming great consultants and change agents c. Debugging the heck out of everything d. Avoiding being misled by experts or Google Testing!
- 128. Mobius strip Confusion Signal box Cigar Discontinuity Confidence Burt Totaro Sundar Pichai John Giannandrea Chuck Norris Jeff Dean Fencing Keyboard Image credits Go Robot Leopard print sofa Leopard Bug Lego robots Iron Man Roundabout