Remove Duplicate

It finds repeated items in your data and keeps one copy of each. An item can be a line, a word, or a piece of text split on a separator you choose. That covers the usual jobs: an email list with the same address in it three times, survey answers pasted twice, CSV rows that overlap after a merge, or a keyword list built from several sources.

Duplicates distort whatever you do next. A mailing list sends two messages to one person, a count of responses runs high, and a customer shows up as two records. Removing them first keeps the rest of the work honest.

1. Paste your items into the input box, one per line, or upload a .txt or .csv file.

2. Choose whether to work on lines, words or a custom separator.

3. Set your matching options: exact or fuzzy, case-sensitive or not, and whether to trim whitespace and ignore empty lines.

4. Click Remove Duplicates. The tool shows how many entries were kept and how many were removed.

5. Copy the cleaned list or download it.

Exact matching treats two items as duplicates only when they are identical. It is the right choice for IDs, email addresses and phone numbers, and it is fast on large lists.

Fuzzy matching also catches near-duplicates, such as "St." and "Street" or a name with a small typo. It uses a similarity threshold you can adjust: start high, around 0.9, for slight typos, and lower it toward 0.8 for abbreviations. Because it compares items against each other, it is slower, so use it on smaller batches.

Case-sensitive or case-insensitive. "Apple" and "apple" are different items in case-sensitive mode and the same item in case-insensitive mode. Email addresses are usually best compared case-insensitively.

Trim whitespace and ignore empty lines. A trailing space makes two identical entries look different. Trimming removes that difference before comparing, and ignoring empty lines stops blank rows from being treated as items.

Preserve order, sort and count. Preserve order keeps the first occurrence of each item where it was, which matters for logs and anything chronological. Sorting arranges the result alphabetically. Count duplicates shows how many times each item appeared.

Word has no one-click command for removing repeated paragraphs. The quickest route is to move the text through this tool and back. The tool works on plain text, so use a copy of your document.

1. Save a copy of your Word document so the original stays untouched.

2. Select the paragraphs you want to clean and copy them. Pasted as plain text, each paragraph lands on its own line.

3. Paste into the tool, choose lines, and turn on trim whitespace and ignore empty lines so blank spacing does not count as an item.

4. Keep exact matching to remove only paragraphs that are word-for-word identical. Switch to fuzzy matching to catch paragraphs that differ by a small edit, then review what it removed.

5. Run the tool, turn on preserve order so the first copy of each paragraph stays in place, then copy the result.

6. Paste it back into your document copy and reapply any formatting, since headings, bold text, tables and images do not carry through plain text.

Two things to watch. A paragraph that contains a manual line break (Shift+Enter) may be read as more than one line. And two paragraphs that differ by a single word count as duplicates only in fuzzy mode, so check the removed count against what you expected.

The right method depends on what is repeated.

A list of words, names or keywords in a Word file. Copy the list, paste it into the tool with one item per line, and run it as described above. This works well for keyword lists, name lists and anything that is really a column of items.

A repeated word inside a sentence, such as "the the". Do not run prose through a duplicate remover. Removing repeated words from sentences strips out words that belong there and changes the meaning. Fix those by hand.

• Email lists: one address, one entry, before you send a campaign.

• Survey and form responses: clear out accidental double submissions.

• CSV data: tidy rows that overlap after combining files.

• Customer records: spot the same customer entered twice with small variations, using fuzzy matching.

• Keyword and word lists: merge lists from several sources into one clean set.