Nov 25, 2020 16:00:00

Googlebot's various statistical information 'crawl statistics report' has been improved, more detailed information can be obtained

We announced that we have improved the '

Crawl Statistics Report ' for a better understanding of the crawler ' Googlebot ' that Google operates to collect various statistics on the website.

New and improved crawl stats for your site | Google Search Central Blog
https://developers.google.com/search/blog/2020/11/search-console-crawl-stats-report

Google operates a crawler 'Googlebot' that mechanically acquires documents and images on the web and creates a database in order to grasp the world's websites that exist innumerably. Googlebot continues to accumulate data about web pages in Google's database day and night by looking at web pages, following links on the pages, just as users browse content on the web.

How Google Search Works | Crawling and Indexing
https://www.google.com/intl/ja/search/howsearchworks/crawling-indexing/

If you run your own site, you've always been able to find out the stats Googlebot got from the Crawl Statistics Report. The announcement announced on November 24, 2020 is to improve this crawl statistics report, from now on the total number of requests grouped by response code, the type of file crawled, the purpose of the crawl, the type of Googlebot. In addition, detailed information on host status and examples of URLs for which requests have been made within the site can be referenced from the crawl statistics report, and domain properties that allow multiple subdomains to be grouped together are also supported.

Among these improvements, the items that are notable are as follows. In 'Over-time charts', you can see the total number of requests, the average size of downloads, and the transition of average response time.