Many search engines will look for a robots.txt file before spidering your sites content. The robots.txt file contains specific rules a robot/spider should abide by when capturing data.
- 524 Users Found This Useful
Related Articles
Point my domain to my wix account
How can I point my domain name to my wix.com account?To set this up you will want to login to...
Can I setup SSL on my account?
Yes, you can purchase a secure certificate through us so you can use SSL. You can also give us...
What is a virtual Domain?
A virtual domain is the base URL of the domain name. Examples include yourdomain.com, and...
Mounting Samba Share in Linux for Startup
The following may be used to mount a samba share via your /etc/fstab file for bootup on...
Can I change my username?
Usernames are setup in the manner for security. Most programs will allow you to save the...
