You can use a specific <META> tag to tell search engine robots to not index  the content of a specific webpage. If you would like to find out how to stop bots from accessing your entire website please read our article on creating a Robots.txt file. You can also use the same tag to direct the bot to not scan the page for links to follow as well. This is a great tag to use if you are only trying to restrict one page on your website. Here's a basic 'no follow, no index' tag in HTML code:

<html>
<head>
<title>...</title>
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
</head>

There are three important peices of information you need to keep in mind when using the <META> tag:

  • Robots can ignore the tag if they wish. Typically, the bots that will ignore the <META> tag are rogue bots that are scanning the web for security vulnerablities to exploit or email harvesters used by spammers to grab email addresses.
  • the NOFOLLOW directive only applies to links on the page. If the bot finds a link on another page without a NOFOLLOW it will still arrive on the page
  • The NOFOLLOW in this tag is NOT the same as the rel="nofollow" attribute you can set on an <a href> tag

Writing a ROBOTS META Tag

Like other <META> tags used in HTML they should be placed in the HEAD section of your HTML code just like in the example above. 

<META NAME="ROBOTS" CONTENT="NOINDEX, FOLLOW">

So lets break the above example down further.

  • The "NAME" attribute will always be "ROBOTS". 
  • The "CONTENT" attribute can have four different values: "INDEX", "NOINDEX", "FOLLOW", "NOFOLLOW"

Multiple comma separated values are allowed but only certain combinations make sense. If there's no <META> tag for ROBOTS present the default is "INDEX, FOLLOW" so there's no need to specify that. You can have the following:

<META NAME="ROBOTS" CONTENT="NOINDEX, FOLLOW">
<META NAME="ROBOTS" CONTENT="INDEX, NOFOLLOW">
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">

If you need further assistance with blocking robots from indexing your webpages, please read our article on How to stop Search Engines from crawling your Website. If you need furhter assistance you can always contact our support department 24 hours a day 7 days a week. 

Did you find this article helpful?

We value your feedback!

Why was this article not helpful? (Check all that apply)
The article is too difficult or too technical to follow.
There is a step or detail missing from the instructions.
The information is incorrect or out-of-date.
It does not resolve the question/problem I have.
How did you find this article?
Please tell us how we can improve this article:
Email Address
Name

new! - Enter your name and email address above and we will post your feedback in the comments on this page!

Like this Article?

Related Questions

Here are a few questions related to this article that our customers have asked:
Ooops! It looks like there are no questions about this page.
Would you like to ask a question about this page? If so, click the button below!
Ask a Question

Post a Comment

Name:
Email Address:
Phone Number:
Comment:
Submit

Please note: Your name and comment will be displayed, but we will not show your email address.

0 Questions & Comments

Post a comment

Back to first comment | top

Need more Help?

Search

Ask the Community!

Get help with your questions from our community of like-minded hosting users and InMotion Hosting Staff.

Current Customers

Chat: Click to Chat Now E-mail: support@InMotionHosting.com
Call: 888-321-HOST (4678) Ticket: Submit a Support Ticket

Not a Customer?

Get web hosting from a company that is here to help. Sign up today!