So, how does this Googlebot actually work?Rolf Broer, who works at Onetomarket, just released a very interesting article about Googlebot and its crawling capabilities. Rolf created several tests for the so called superior bot of Google to see how it reacts on particular setups of links and pages. Biggest takeaway is that you can use a Google sitemap to boost the crawlrate. But there is more...
Google's statement about the fact that you shouldn't use more than 100 links per page doesn't make it. The test showed that Google will crawl way over 100 links if it has to. Also Matt Cutts statement that "the amount of pages being crawled is roughly proportional to your PageRank" seems to be a tad of course. In 31 days, Googlebot visited about 375.000 pages on a PageRank 0 website. Rolf declares that when the website would have had a PageRank of 1, Googlebot would crawl over 140.000.000.000 pages in 31 days. That simply means that PageRank doesn't matter at all to get your pages crawled. Breadcrumbs, JavaScript & Semantic Intelligence Rolf and his team at Onetomarket did a few more tests with the Googlebot. For instance, they checked how Googlebot handles breadcrumbs and JavaScript, or Semantic Intelligence of links. The outcomes are definitely worth the read. For Dutch readers, read the Dutch version of the story right here. CommentGoogle indexes SVG1 comment  According to the Google Webmaster Blog, Google is now indexing .SVG, (Scalable Vector Graphics) presented as an alternative to Flash. But who besides wikipedia actuall...... read more
Semetis develops its rich media ...0 comments
BRUSSELS,
1 September 2010 – Search and web analytics agency Semetis today announced the
launch of a new interactive marketing concept, a premiere on the Belgian
market.
Semetis channel on YouTube is fully dedicated to video
content related to search marketing and web analytics. In the past, Semetis had
already posted more than 20 videos with useful information and tips. As from
now, Semetis will post video tutorials on a weekly basis on its YouTube brand
channel.
Startin...... read more
Google combating cowboy agencies4 comments  Google today announced they are changing the rules on how their partner agencies are to be working with AdWords accounts/advertisers. In order to combat non-transparen...... read more
Google Grants Belgium looking fo...1 comment 
Brussels, Tuesday 20
July 2010 - David Gillain, Agency Manager at Google Belgium launched today an
appeal to all Belgian Search agencies looking for volunteers i...... read more
Related tweets
|
Last Comments
Bloggers
Latest Videos
  Columns
Tagcloudgoogle realtime privacy apple tools search engine images funny google earth a4uexpo spain sea ppc ses google maps europe russia seo matt cutts research search engine strategies social media analytics interview smx twitter google wave blogger yandex searchcowboys gmail news website linkbuilding youtube search maps streetview google tools advertisingMost Commented
AgendaMost Read
|
    Search
My BlogLogBlogroll |
© 2010 Searchcowboys.com - All Rights Reserved - All views and opinions expressed are those of the authors of Searchcowboys.
All trademarks, slogans, text or logo representation used or referred to in this website are the property of their respective owners. Sitemap
Comments (5)
Great article offering some real insight Dennis. I guess the 100 links per page myth is another case of Google leading us down the path they want us to go when the reality is very different. But then thats what makes it fun!
Di 20 jul 2010, 12:03
@Gareth Rees, yep, it's all part of the game!
Vr 23 jul 2010, 15:06
Funny to read but good to know. Its all part of the game.
Di 3 aug 2010, 18:18
I think this statement "PageRank doesn't matter at all to get your pages crawled." can be very easily clarified from a very simple things which we already know. They are, crawling is the first step designed to a bot, next comes indexing, next ranking. I have just given a brief process about crawlers behavior without going deep into that. So I would like to say that any search bot even before ranking a page should first crawl and index it and later decide its ranking.
Zo 8 aug 2010, 15:20
Googlebot has been a stranger to me but through your post I gained insightful ideas on how it works.
Vr 20 aug 2010, 05:37