Posted by goralewicz Crawling and indexing has been a hot topic over the last few years. As soon as Google launched Google Panda, people rushed to their server logs and crawling stats and began fixing their index bloat. All those problems didn’t exist in the “SEO = backlinks” era from a few years ago. With […]
Google Shares Details About the Technology Behind Googlebot Read More »
I recently started helping a site that was negatively impacted by the May 17 algorithm update. The site had been surfing the gray area of quality for a long time, surging with some quality updates, and sometimes dropping. So I started digging in via a crawl analysis and audit of the site. Once I started
The benefit and shock of analyzing Googlebot crawl spikes via log files [Case Study] Read More »
Recently, we’ve heard a number of definitions for “crawl budget”, however we don’t have a single term that would describe everything that “crawl budget” stands for externally. With this post we’ll clarify what we actually have and what it means for Googlebot. First, we’d like to emphasize that crawl budget, as described below, is not
