Recently in Googlebot Category

Google hasn't indexed my site since April or so because it can't see my robots.txt file for some reason.

But yet Google Reader doesn't have any issue with pulling a feed from this site.

And MSN and Yahoo and the rest don't have any issue either.

It's really annoying.

I was recently checking with Google's Webmaster Tools to see if I was having any issues with this site. I discovered that every single page had an " robots.txt unreachable" error attached to it. Gotta admit that I was rather surprised when I saw that. I hadn't put in a robots.txt file when I moved over to Movable Type and really hadn't thought of it. I mean not having one in the past has never caused an issue in the past. Why should now be any different.

Imaging my surprise when I read the help file on the error:

Before we crawled the pages of your site, we tried to check your robots.txt file to ensure we didn't crawl any pages that you had roboted out. However, your robots.txt file was unreachable. To make sure we didn't crawl any pages listed in that file, we postponed our crawl. When this happens, we return to your site later and crawl it once we can reach your robots.txt file. Note that this is different from a 404 response when looking for a robots.txt file. If we receive a 404, we assume that a robots.txt file does not exist and we continue the crawl.

Now I state again that I didn't have a robots.txt file in place. Per that FAQ, a missing robots.txt file shouldn't be an issue but, for some reason, it was in this case. I quickly created an empty file within the backend and waited to see if it was resolved. It was.

Just wanted to pass that along. Hope it helps someone else with that issue.

Biggest Moron Ever

Worst Web designer ever

That's the idiot I had to deal with a few weeks ago over on the WordPress.com support forums.

*sigh* I really like it when folks listen to me. :)

edit: Feel free to add in your own links to those sites if you want to help out. :)

I'm flipping through some of the posts about Google's recent broken PR update (All of my new sites have PR0s still.) and came across this suggestion of removing the "rel='nofollow'" bit from the comments.

I've got to admit that I'm kind of torn. While I can understand the need to have the tag in the links, most bloggers monitor what appears in their comments. I know I do. Anything that's spam gets removed.

I'm going to remove it from my WordPress setup and see what happens.

Came across this wonderful tool that lets you compare results from the different Google datacenters. Find out if one datacenter sees your website different from another.

Simply makes a post about how MSN was the first search engine to crawl his or her blog and, in doing so, maybe the better search engine.

I've got to agree. When I ran my Daria site, the only search engine I didn't have issues with was MSN's. Both Yahoo and Google has thousands of bad links, they listed sites before my which had nothing to do with Daria, and were just a royal pain.

I went over to MarketLeap.com to compare the results for searches from the search engines for this site. Now, mind you, that Google's blog search is big on promoting using ping services and says that that's how it will pick up on the site.

Link Popularity Check:

Google: 0
Hotbot: 0
MSN: 8 (From 6 different sites)
Yahoo/ Altavista: 1 (But doesn't list it)

Search Engine Saturation

Google: 0
HotBot: 5 (Home page plus 4 articles)
MSN: 1 (Home page)
Yahoo: 0

Mind you that this is now after 2 months of blogging here. Google's blog index has three incoming links to this site but doesn't have any of the pages indexed though. Strange...

I'll take another look in a few days.