Index Configuration Guide: Managing Stop Words

Stop words are common words such as "the", "is", "at" and "which" that the search engine ignores when it indexes your content and processes queries. In SearchJet you decide which words go on the stop word list. This guide explains what stop words do, which words to add (and which to leave off), and how to change the list in your index settings.

What stop words do

Words like "a", "an", "the", "in", "on" and "is" appear on almost every page. Matching on them adds noise without making results more relevant. When the engine ignores them, it can focus on the words that carry meaning.

Term	Definition	Benefit
Stop word	A common word the search engine ignores when matching.	Improves relevance by focusing the query on meaningful keywords, and improves performance by shrinking the index and the work done for each query.
Examples	"the", "a", "an", "of", "in", "is"	A search for "The best laptop in London" is matched mainly on "best laptop London".

Filtering common words without thinking can cause problems, though. A product called "The North Face" loses part of its name, and a legal document about "not guilty" loses the word that decides its meaning. That is why the list is yours to control.

Choosing your stop words

There is no single right list. Start from how people search your site.

Words worth adding:

• Words that appear on every page of your site. On a store where every product page contains "shop" or "buy", those words don't help tell one result from another.
• Filler words in your visitors' language. On multilingual sites, add the common articles and prepositions for each language you index, such as Spanish "el" or French "les". Otherwise they add noise to results.

Words to keep off the list:

• Parts of product names. Stores should keep words like "mini", "pro", "plus" and "max" searchable. A customer searching for "iPhone Pro Max" should not lose "pro" and "max". The same goes for "the" if you sell titles such as "The Matrix" or "The Godfather".
• Words that change meaning. Legal and academic sites should keep "not", "no", "nor" and "vs". Ignoring "not" in "not liable" reverses the meaning.
• Technical terms. Documentation sites should keep "or", "and" and "not", which are operators in many programming languages. Filtering them breaks searches for terms like "string or array".

Adding a stop word

The Stop Words tab is in your index's Index Settings. To add a word:

1. Open Index Settings for the index you want to change and select the Stop Words tab.
2. Find the input field for adding new stop words.
3. Type the term, for example for. Add each stop word as a single lowercase word.
4. Click Add to add the word to the index's stop word list.

Repeat for each word you want to add. To make a word searchable again, remove it from the list.

Applying changes

Changes to the list do not take effect until you save them. When you're done editing, click Save Changes at the bottom of the page.

After saving, the index applies the new list to your content. On a large index this can take a short time before it shows up in results.

Checking the results

After each change, run a few real queries from your site and compare them with what you saw before:

• Search for product or brand names that contain common words, such as "The North Face", and check that they still rank first.
• Search for phrases where small words matter, such as negations, and check that the results still make sense.
• If results get worse, take the word back off the list and save again.

Stop words are a small setting with a big effect on relevance. Review the list when you add a new product line, language or content type. For more on setting up an index, see the quick start guide.