Why do I have references to robots.txt in my web stat programs?

Many search engines will look for a robots.txt file before spidering your sites content. The robots.txt file contains specific rules a robot/spider should abide by when capturing data.
  • 522 Users Found This Useful

Was this answer helpful?

Related Articles

I need a module installed but I dont have rights.

No problem. Send us a trouble ticket and we will install the module as long as there are no...

Can I change my username?

Usernames are setup in the manner for security. Most programs will allow you to save the...

Enable Mod Rewrite

Create an .htaccess file with the following contents and upload it to your public_html directory....

Reseller: Unable to find an ip address in when creating an account.

When creating accounts be sure to select a package in the package field. You can create a package...

Sub-Domain times out

When you add a sub-domain and it times out make sure you do not have redirection set from that...