�ݺ�ߣ

SOB 2022
Crawl capacity management
on Envato Elements
Gastón Riera - @gastonriera

Heads up..
�ݺ�ߣs in English 󰎉
Speako en Español 󰎆

At the end I'll share a 90%
discount for Elements! ☺

That's how I used to look,
all well dressed and all.
Gastón Riera

Everything you need to get your
creative projects done.
The big names:
Other very cool products:

The two things I like the most about working at envato:
- Being sustainable and caring about the community
- Fully remote (ANZ/MX) and working from abroad

Let's get into SEO 🔎

The problem:
A big part of the site was
not being indexed 👎

☹

It looked like Google was not recrawling a
large amount of pages!

And taking very long to recrawl
some pages

Was Google actually
crawling the site?🤔

Yes, it was👍

But, it wasn't getting to
crawl the entire site 😕

And, why was that?󰤈

No idea

Just kidding 😆

We came up with two
theories to work on 😎

- Content quality
- Internal linking
We needed to work on

We needed to work on
I'll get to them in a bit.
- Content quality
- Internal linking

What did we do? 🤔

The basics!

The basics!
- Noindex

The basics!
- Noindex
- Redirects

The basics!
- Noindex
- Redirects
- Nofollow

The basics!
- Noindex
- Redirects
- Nofollow
- Crawl paths (more/less)

Gastón, that's nothing new!
😡

I know! 😉

How are we using them?
Let's get to it

Battle_1:
Content quality

Content is not just text on the
page, but everything on it.
Every page is content.

Battle_1: Content quality
Two options:
1. Add content focussing on quality over
quantity.
2. Remove content from Google's index.
We already had +9M items!

Two options:
quantity. ❌
2. Remove content from Google's index.

Two options:
quantity. ❌
2. Remove content from Google's index. ✅

Do you know what reduces the
content quality of any site?

Do you know what reduces the
content quality of any site?
DUPLICATE CONTENT!

Noindex and remove duplicates,
RUTHLESSLY
Noindex a good part of
the items library.
-> Several million less
discoverable pages!
Why we decided to
noindex instead of
a fancier solution?
Ask me later
😉

A few tips on how to get what to noindex?
- Use google's crawled not indexed as a proxy
- Check duplicate titles/urls/content description
- Just a different image doesn't make it a different page to
the eyes of Google!

Noindex and remove duplicates,
RUTHLESSLY
Why the redirected path
had 15% of site's trafﬁc
and 20x the destination.
Ask me later
😉
Merged two translations that ended up being way
more similar that intended
-> A few millions pages removed from Google.

Other big things we did
● Turned Tag pages into Search pages
● Search pages are noindex by default
The overall result? Decreased the index size to a half
without impacting organic trafﬁc.
50%

Battle_2
Internal linking
Reference

Battle_2: Internal linking
Out of many tactics:
1. Reduce the number of crawl paths
2. Nofollow on links to low-value pages

Basically,
Be intentional and smart
about crawl paths.

Link to only valuable pages
Added links between related
search pages
10% Organic trafﬁc!
If it's a useful search page,
it will not have a noindex.
Note that

Remove hreﬂang when you're uncertain
of the quality on other languages
15% size of index!
hreﬂang are bidirectional,
remove them on every
language.
Remember 😉

What are valuable pages?
In short, pages we want Google to index.

As per nofollow:
● Nofollow on links to noindex pages
● Filters and facets, all nofollow
The overall result? Google re-crawled more pages.
60%

So, why crawl capacity
management?

Crawl budget stayed the
same.
*On average, over the last 2yrs.

BONUS TRACK
and unpopular opinion.

BONUS TRACK
and unpopular opinion.
We learnt that
● Sitemaps didn't help indexing AT ALL
󰤃
● Helpful only for debugging 🤓

1st month 1 USD🥳
Go to SOB22.com
Yeah no kidding, I did
register that domain to
share the discount ☺

Gracias! - Thank You!

�ݺ�ߣ

Crawl Capacity Management - SEontheBeach 2022

More Related Content

Crawl Capacity Management - SEontheBeach 2022