Meta Robot Tag

You can use a special HTML  tag to tell robots not to index the content of a page, and/or not scan it for links to follow. This tag controls the behavior of the spider/robot.

<meta name=”robots” content=”selection”>

For more Detail on Robot