What's the Tailrank Robot?

Tailrank is a service that monitors blogs trying to find interesting memes and hot stories. We have a 'robot' which analyzes blogs periodically trying to find interesting stories. If we find that a story on your site is 'hot' we promoted it to our front page. This is a good thing and can drive a lot traffic to your website.

Why are you reading my site?

We're reading your site trying to find out how you're influencing the blogosphere. We then use this to drive the ranking system on our website.

Are you using wasting my bandwidth?

We shouldn't be. Tailrank uses very little bandwidth to monitor your site. We only request pages once and cache them once we've fetched them.

Can I tell Tailrank to stop reading my site?

We currently monitor the XML feeds syndicated from your weblog. If you want us to index your feeds let us know (as well as the HTML). Most people want us to index their site so this rarely happens.

Why is Tailrank requesting XML files that don't exist on my server?

We attempt to use web standards as much as possible to find the feeds which exist on your site. Unfortunately, there are many websites that break web standards in ways which can confuse robots. We attempt to assert that your weblog software is configured correctly by requesting additional files. We try to avoid downloading the entire file and only use conditional gets to avoid wasting bandwidth. The biggest problem with this approach though is that it generates 404 error messages but we only do this once per week.

Does Tailrank index my feed?

If your site offers an RSS feed we try to find it and index it by our service. If not we also try to analyze your HTML as well. If you want to influence Tailrank the best way possible is to use an RSS feed with a full content feed (including all HTML from your post).

How does Tailrank attempt to minimize my bandwidth usage?

1. Compression We use gzip compression to reduce the number of bytes between our servers and your servers. This can usually result in a significant savings in bandwidth.
2. Only fetch when your weblog has changed. We use the If-Modified-Since and ETag HTTP headers to prevent duplicate downloads. Not every weblog system supports these standards in all scenarios.