Friday, 16 December 2016

GoogleBot

Google's search engine spider.
Google has a shared crawl cache between their various spiders, including vertical search spiders and spiders associated with ad targeting.

No comments:

Post a Comment