If you haven’t been busy at SES in our neck of the woods this week, you might have noticed we’ve been rolling out fresh updates to our crawl, index and ranking systems — ones that are relatively significant for webmasters.
In response to feedback from the community, we’re reducing our crawling machines by a significant amount with this update, so you might have observed a significant reduction in crawl load from us along with fewer machine IP addresses hitting you over the last few days.
The new crawler continues to improve comprehensiveness and freshness of coverage while also enhancing crawl efficiency, which reduces spurious load on websites.
We had a brief bout of increased crawl load while testing and rolling out updates, like support for ‘Dynamic URL Rewriting’ in Site Explorer, announced yesterday. Don’t fret, though, if you’re concerned about seeing an increased load. We’ve initiated efforts and established policies internally to ensure this doesn’t happen, even temporarily, in the future.
Thanks and keep the feedback coming.
Priyank Garg
Yahoo! Search
Recent Comments