InMotion Hosting Support Center

robots7

Category: Databases

daveengle
Asked:
2011-11-01 5:23 pm EST

Hits: 10,353
What can be done to block the robots and spiders from tracking my domain?

You must login before you can ask a follow up question.

You must login before you can submit an answer.

Best answer chosen by InMotion Hosting staff

0

Tim S.
Staff
9,967 Points
2011-11-02 7:10 am EST
Hi daveengle,

Thanks for posting your question. I'm more than happy to assist you today. There's two methods to keeping spiders from crawling your website. The first, involves creating a robots.txt file. This file will be read first, and specifies whether the bot can crawl the site. To learn more about robots.txt files please visit the site:

www.robotstxt.org

The second option is to place a specific "no index, no follow" tag in the header of each webpage you wish to exclude from being crawled. Here's the basic tag to do so:

‹META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">


Also, I'd suggest creating a Google Webmaster Tools account. This will give you the ability to decide how Google's bots crawl your website.

I hope this helps! If you need further assistance please feel free to contact us.

Thank you!

Tim S

You must login before you can post a comment about this answer.

Like this Question?

Support Center Login

Social Media Login

   
Social Login Joomla

Related Articles

It looks like there are no related articles.
Would you like to ask a question about this page? If so, click the button below!
Ask a Question

Need more Help?

Search

Ask the Community!

Get help with your questions from our community of like-minded hosting users and InMotion Hosting Staff.

Current Customers

Chat: Click to Chat Now E-mail: support@InMotionHosting.com
Call: 888-321-HOST (4678) Ticket: Submit a Support Ticket

Not a Customer?

Get web hosting from a company that is here to help. Sign up today!