Robots.txt file

Robots.txt is a text file which contains parameters of the site indexing for web crawlers. It is usually used to ignore pages in the search engine results.

Robots.txt tab is found in the Settings → Robots.txt.

If you want to exclude the whole site from web indexing (from all search engines), copy the text below and paste to the File content field:

User-agent: *
Disallow: /

If you want to exclude only a separate page, copy:

User-agent: *
Disallow: /page/

Here /page/ is the URL of the page you need to exclude. URL is shown in the page settings.

To see the page URL, click the “…” button on the page thumbnail and select Settings.

Copy the link from the Page URL and paste.

For example: Disallow: /page_copy2/

Learn more about creating a robots.txt file in this Google article.

Search engines may consider as duplicates several pages with almost identical content, or one page with multiple URLs.

If this occurs, a search engine chooses only one page to show in the search results. This page is called canonical.