« Stingrays Are Terrorists
Using Textbooks As Shields »

Friday, October 20th, 2006Friday, October 20th, 2006

Google Crawl Rate

by Shawn @ 7:32 pm · filed under Tech Stuff

Here's something cool that Google has added to Google Sitemaps for site owners...

Let's you see how much Google's spider crawls your site in the last 90 days, and even gives you the option to instruct Googlebot to increase or decrease their crawl activity on your site.

Here's the stats for the Digital Point forum... Google crawling an average of 52,000+ pages per day (without putting it on "increase activity")... Getting close to a page per second 24/7. :)

10 Responses to “Google Crawl Rate”

  1. Tom C. Says:
    October 20th, 2006 at 7:38 pm

    Now your tracking… good stuff!!! http://www.capitalwindow.com :)

  2. alek Says:
    October 21st, 2006 at 6:58 am

    Figured DP would be huge - two orders of magnitude more than my site. BTW, I don’t have the “Faster” option listed for me - they say “At this time, crawl rate is not a factor in your site’s crawl. If it becomes a factor, the Faster option below will become available.”

  3. Matthew Says:
    October 21st, 2006 at 8:48 am

    Thats an interesting tool. I was suprised at the numbers shown on my site. I didnt realise google crawled it as much as that.

  4. Johan Says:
    October 22nd, 2006 at 2:06 pm

    Your average load time is pretty impressive… (Mine’s 4 times that!)

    A la Alek, “Faster” isn’t an option for me either. Maybe due to the high(er) avg load time?

  5. Matt Says:
    October 23rd, 2006 at 12:51 pm

    Same us as - 50,203 pages per day

  6. GoogleBot Says:
    January 24th, 2007 at 12:04 pm

    If the “Faster” option is disabled or greyed out for your website, you can easily activate it by following this webpage:
    http://www.nopower.org/how_to_make_google_sitemaps_faster_crawl_rate_option_available.php

  7. sanjeevkumar Says:
    June 30th, 2007 at 6:42 am

    good one… Thanks

  8. Coupon Codes Says:
    November 20th, 2007 at 8:36 am

    How can I enable fast crawl rate in my google sitemap tool for my website http://www.couponrefund.com/

  9. imi Says:
    August 26th, 2008 at 10:14 am

    Faster option is disabled for my website ( http://www.warezhouse.info )and i can’t enable it :(
    Come to my Blog too…

  10. Johnson Says:
    September 28th, 2008 at 10:58 pm

    That nopower link is not working…

Leave a Reply