Block Documents

Not all web pages on a website will interest users as search results. Constraining search results can be useful when:

  • Websites contain landing pages, support pages, contact pages, archives, and so forth. When a user searches for a product, these pages can appear among the search results.

  • Websites have major sections, for example, Products, Used Products, Technology, etc.

Comparison of exclusion and blocking

Site Search provides two ways to omit unnecessary or undesired documents from search results:

Approach Description

Exclude documents
(Web Crawler)

Omit documents from the index when a web data source is crawled. Site Search omits documents by comparing a series of exclusion criteria (which are regular expressions) with parts of the uniform resource locators (URLs) for the web pages (the path component and the query component). A web page (or other document, such as a CSS file) is excluded if any exclusion criterion matches the compared part of the resource’s URL.

Exclusion criteria can contain characters that are valid in URLs as well as * (asterisk), which matches zero or more characters. Matching is case sensitive.

An excluded document is not in the index, so users won’t be able to find the document by searching in embedded Site Search modules or in search apps that use the Site Search APIs.

Including previously excluded documents or excluding previously included ones reindexes the data source.

Block documents
(All data sources)

Omit specific documents one-by-one from search results for all queries.

Blocked documents are in the index, so blocking a document or unblocking it doesn’t necessitate reindexing.

Important
Excluding and blocking documents are not intended to provide data security or privacy. Site Search is intended for use with the public Web. Excluded documents still exist on the indexed website. Users can find the documents by searching on the source websites or by having document URLs. Similarly, blocked documents still exist in the data sources from which they were indexed. Also, changes in document names and locations on a website can undo exclusion or blocking of documents.

For information about excluding documents, see Exclude documents.

Block documents

Block documents to omit specific documents from search results for all search queries.

To block documents
  1. In the Page Builder, perform a search or searches that reveal the documents you want block. To browse multiple pages of search results, use the page controls at the bottom of the pages.

  2. Hover over the search result for a document you want to block. The search result turns green and controls appear in the upper right corner of the result:

    Hover over search result

  3. Click Block Block.

Unblock documents

Unblock documents that you decide to make searchable again.

To unblock documents
  1. In the Site Search menu, click Blocked Documents.

    You should see the documents that you’ve blocked.

  2. Select the documents you want to unblock by selecting the checkboxes, and then click Restore Selected.

    Site Search unblocks the documents. Users can find them again in search results.