Remove Duplicate Words

Removing repeated words

Every repeat keeps the first time each word appears and deletes every later copy, wherever it is. The space before a removed word goes with it, and line breaks are preserved.

This mode is meant for lists of keywords, tags or names. On ordinary prose it will also delete words that grammar needs, such as the second “the” in a paragraph, so use Only doubled words for prose.

Consecutive duplicates vs all duplicates

Only doubled words (the the) removes a word only when the same word comes directly before it on the same line, with nothing but spaces in between. That catches the classic typing slip where a word is repeated across a line break in your editor, without touching legitimate repeats elsewhere.

A sentence boundary breaks a pair: in “It ended. Ended it.” nothing is removed, because the first “ended” is followed by a full stop.

Case and punctuation options

With Ignore case ticked (the default), “SEO” and “seo” are the same word and the first spelling is kept.

By default, punctuation at the start or end of a word is ignored when comparing, so tools, and tools match. When a repeated word is removed, punctuation attached to it is removed with it: seo, tools, SEO, free tools, free becomes seo, tools, free. Tick Punctuation is part of the word to remove only exact matches, so “end.” and “end” are kept as different words.

Cleaning keyword and tag lists

For comma- or space-separated tags, Every repeat with Ignore case gives a clean list in one step. If your list has one phrase per line (“running shoes”, “trail running shoes”), use remove duplicate lines instead: it compares whole lines, whereas this tool would split the phrases into single words. To see which words you repeat most in an article, use the word counter.