In short: RowSlice's CSV duplicate remover deletes repeated rows from a CSV file inside your browser, so the file is never uploaded. It compares whole rows or only the columns you choose, such as email, keeps either the first or the last copy of each row, and leaves the row order unchanged. The tool sets no file-size limit of its own, but one run can compare up to about 33 million different rows (or different combinations of values in the chosen columns), or about 8.4 million when you keep the last copy. Past that, it stops with a message instead of saving a partial result.
How to Remove Duplicates from a CSV
- Choose or drag in your CSV file and check the preview.
- Decide what makes two rows the same: every column matches for exact copies, or these columns match to compare only key columns like
email. - Choose which copy of a repeated row to keep: the first, which is the default, or the last, for a file in which the newest version of a record is the one nearest the bottom.
- Tick Ignore upper and lower case and Ignore spaces if the data was typed by people.
- Press Remove Duplicates and download the cleaned file.
Remove Duplicates Based on One Column (Email, ID or SKU)
Exact duplicate rows usually come from the same export being run twice, or from merging files whose date ranges overlap. Comparing every column removes those and nothing else, so it is the safe default.
Duplicate records are different: the same customer signed up twice with a different phone number, or the same product appears with two prices. The rows are not identical, so a whole-row comparison keeps both. Compare only the column that should be unique — the email address, the order number, the SKU — and the tool keeps one row for each.
Which copy is kept matters as soon as the two rows differ. The default keeps the first copy, the one nearest the top of the file. Choose the last copy when the file is in the order the records were written — an export sorted oldest first, or a log appended to over time — because there the later row is the current one and the earlier row is out of date. That setting reads the file twice: which copy is the last one is not known until the end of the file, and by then the copies before it would already have been written, so the first pass finds the last copy of each row and the second writes the result. Expect it to take about twice as long. Row order is your file's own order either way, and no row is moved.
| name | signup | Every column | Email only (ignore case) | |
|---|---|---|---|---|
| ann@shop.com | Ann | 2026-01-04 | Kept | Kept |
| ann@shop.com | Ann | 2026-01-04 | Removed | Removed |
| Ann@Shop.com | Ann B. | 2026-03-18 | Kept | Removed |
| bob@shop.com | Bob | 2026-02-11 | Kept | Kept |
The third row is only caught because Ignore upper and lower case is ticked: Ann@Shop.com and ann@shop.com are different text otherwise. Both result columns show the default, which keeps the first copy of each group; keeping the last copy removes just as many rows, but the copy left standing is the lowest one — Ann@Shop.com in the email-only column. For mailing lists, including stray spaces and Gmail's dots and plus signs, see our guide to removing duplicate emails from a CSV list.
Find Duplicates Without Deleting Anything
Sometimes you only want to know whether a file has duplicates, and which rows they are, before anything is removed. Tick Also save the removed rows to a separate file. The result line tells you how many duplicate rows were found, and you get two files: yourfile-deduplicated.csv with the rows that were kept, and yourfile-duplicates.csv with the header and every row that was taken out. Your original file is not changed, so if you decide the duplicates should stay, simply keep using it.
Two things to know. The duplicates file holds the copies that were not kept — the later ones when you keep the first copy, the earlier ones when you keep the last — and the copy that was kept is in the cleaned file, not in this one. And the tool does not highlight or mark duplicates inside your original file; it only writes these two new files.
To learn whether a file has repeats at all without producing any files, check it with the CSV validator instead. The validator counts repeated whole rows and repeated values in a column you choose, such as an order number that is meant to be unique, and reports rather than rewrites.
Merge Several CSV Files, Then Remove Duplicates
The tool cleans one file at a time, so duplicates spread across several exports need one extra step. First merge the CSV files into one, then remove duplicates from the merged file. Rows that were in different files now sit in the same file, so the repeats are found, and the copy that is kept comes from whichever file was earliest in the merge — or the latest, if you choose to keep the last copy.
One catch: if you ticked Add a source_file column while merging, every row ends with the name of the file it came from, so two copies from different files no longer match on every column. Choose these columns match and tick every column except source_file, or only the key column such as email.
Merging is the wrong first step when the two files are the same report run twice and the question is what moved between them rather than which rows repeat. Comparing the two exports matches their rows on an ID column and counts the rows added, removed and changed, leaving both files as they are.
Why Not Use Excel's Remove Duplicates?
Excel's Data › Remove Duplicates works well for files that fit on a sheet. It cannot help once the file has more than 1,048,576 rows, because Excel never loads the rest. It also changes values as the CSV is opened — leading zeros disappear from codes, long numbers are rounded — so two IDs that were different in the file can become identical in Excel and one of them gets deleted.
RowSlice reads the raw text of your CSV, so 00123 and 123 stay different values. It reads the file a few megabytes at a time and keeps only a small fingerprint of each different row in memory, so the file can be far longer than Excel's limit. What counts is the number of different rows: up to about 33 million per run when you keep the first copy of each row, about 8.4 million when you keep the last, however many repeats the file contains. Your data never leaves your computer.
Last updated .