Why do I have references to robots txt in my web stat programs?

Many search engines will look for a robots.txt file before spidering your sites content. The robots.txt file contains specific rules a robot/spider should abide by when capturing data.

  • 524 Users Found This Useful

Was this answer helpful?

Related Articles

Convert PDF to HTML Service

To convert your PDF files to html try using the following converter tool from Adobe:...

What are some tools are available to optimize my website?

There are many great tools to optimize your website. There is an optimize tool in your hosting...

How do I cancel my account?

We are sorry to hear you would like to cancel your account! If there is anything we can do,...

Do you support TeamSpeak?

We do not support Teamspeak on our servers but allow you to create a subdomain such as...

Why is my account suspended?

Why is my account suspended? If you are receiving an account suspended page when trying to access...