You may want to block a website, subdomain, or specific pages from Google and other search engines.
This is easy to do and has two methods.
Method 1 Robots.txt
- Enter the public_html or www folder in the main directory of your website via ftp.
- Create a text document named robots.txt.
- Write the codes below and save.
Block entire site from being indexed.
User-agent: * Disallow: /
Block certain pages from being indexed.
User-agent: * Disallow: /tags/* Disallow: /product/order/id/ Disallow: /put-your-admin-directory-here/
Method 2 Meta Tags
Add the following codes between the <head>code</head> tags of your website.
Use of noindex meta code:
<meta name="robots" content="noindex">
Use of nofollow meta code:
<meta name="robots" content="nofollow">
Using noindex and nofollow meta code:
<meta name="robots" content="noindex,nofollow">
To stop a snippet from displaying in search results:
<meta name="robots" content="max-snippet:0">
To allow up to 35 characters to be shown in the snippet:
<meta name="robots" content="max-snippet:35">
To specify that there’s no limit on the number of characters that can be shown in the snippet:
<meta name="robots" content="max-snippet:-1">
Blocking Rules
noindex – Do not show this page, media, or resource in search results. If you don’t specify this rule, the page, media, or resource may be indexed and shown in search results.
nofollow – Do not follow the links on this page. If you don’t specify this rule, Google may use the links on the page to discover those linked pages. Learn more about nofollow.
none – Equivalent to noindex, nofollow.
nosnippet – Do not show a text snippet or video preview in the search results for this page. A static image thumbnail (if available) may still be visible, when it results in a better user experience.