Google Has Two Types Of Crawling – Discovery And Refresh

google crawling
Share on facebook
Share on google
Share on twitter
Share on linkedin

Google uses two forms of web crawling: one for discovering new information and the other for renewing the publishing content.

Google utilizes two types of crawling methods while crawling websites: one to find a new information and the other to update existing content.

On January 7, Google’s Search Diplomat John Mueller discusses this over the Google Search Central SEO office hangout. Swyamdipta Chakraborty, an Optimization consultant, joins the Livestream to ask Mueller a series of questions, one being about how often Googlebot crawls Mueller’s website.

google bot crawler
Google bot crawler

See Also: Google On How To Improve SEO Audits

When he published more regularly, Googlebot visited his site every day, but when he published less frequently, Googlebot didn’t visit as much. He wonders Mueller if this is normal, perhaps fearful that a drop in crawl regularity is a bad sign.

Mueller assures him all is fine and then goes on to explain the two types of crawling that Googlebot performs.

You’ll learn a lot about how Google scans web pages in the section below.

Googlebot Crawling Is Divided Into Two Types

According to a report through Search Console, Googlebot searches your site regularly, and there could be instances when it gets crawled more regularly than others.

googlebot crawling
Googlebot crawling

When questioned about the report, Mueller believes the alterations are factual and describes the two types of crawling in detail:

“It’s possible. We don’t so much crawl a site as we do specific pages within a site. And while it relates to crawling, there are approximately two sorts of crawling. The first is discovering a crawl, in which we try to find new web pages on your site. The other one is a refresh crawl, in which we update pages that we already are aware about.”

Crawl frequency might be different for the entire website and for specific web pages.

You’ll see more Googlebot acts on your site, for example, if it’s modified more often than other sites.

Certain types of web pages are over likely to get more crawling than others.

A news site that modifies multiple times each day is crawled more often than the one modifying once a month.

Googlebot may notice these patterns and adjust its crawl rate as a result. Don’t be alarmed if Googlebot seems to be arriving at your website more often or less often.

web crawler
web crawler

Furthermore, if Googlebot is crawling your website and modifications to recent content aren’t coming up in search results, don’t worry.

It’s likely that Google is looking at your website to see whether there was any new information instead of updating what was already there.

If your website’s public material isn’t updating very often, Googlebot may go deeper to uncover such updates.

Once again, this has hardly anything to do with the article’s quality.

See Also: Yoast SEO Offers Keyword Position Tracking With Wincher Integration

Sign up for our Newsletter

Talk to Digital Expert Now!