# Text Line Sorter and Deduplicator

> Sorts, deduplicates, trims and reverses the lines of a text file in your browser. The six steps run in a fixed order — trim, drop blanks, deduplicate, order, reverse — and the file is never sent anywhere.

Use it: https://tessalor.com/en/documents/text-line-tools

This tool runs entirely in the browser. Nothing entered into it is uploaded.

## How it is done

1. Read the chosen file in this tab, refuse it if it is UTF-16 or binary, and decode the rest as UTF-8, discarding any byte-order mark at the front.
2. Split the text into lines on a carriage return, a line feed, or the two together, so a Windows, Unix or classic Mac file all give the same lines. A final newline is treated as the terminator of the last line, not as an empty line after it.
3. Trim leading and trailing whitespace from each line, if that is switched on. This runs first, because two lines that differ only in trailing spaces are the same line to the person reading them.
4. Remove lines holding nothing but whitespace, if that is switched on. A line of three spaces counts as blank whether or not trimming is on.
5. Remove repeated lines, keeping the first appearance. Matching ignores case unless "Match case" is on, in which case the comparison is exact.
6. Order the lines A to Z or Z to A with the Unicode Collation Algorithm through Intl.Collator, pinned to English rules, or leave the file's own order alone.
7. Reverse the whole list last, if that is switched on, so it flips whatever order the previous step produced. Write the lines back out as UTF-8 with a newline after each one.

## Assumptions

- Trimming runs before deduplicating and deduplicating runs before sorting. Change that order and the answers change — "milk " and "milk" are one entry with trimming on and two with it off.
- Lines that compare exactly equal keep the order the file had them in, because every sort here is stable. With case ignored, "Milk" and "milk" are equal, so whichever appeared first stays first.
- Ordering uses English collation rules regardless of the language of the page, so the same file and the same settings always produce the same file. Under other locales a handful of letters would sort elsewhere — in Swedish, for example, Å comes after Z rather than beside A.
- Case-insensitive matching uses Unicode lower-casing, so ß and SS are still different lines. Accents are always a real difference — "résumé" and "resume" are never treated as duplicates.
- The whole file is held in memory at once, which is why it is capped at 20 MB. Anything larger belongs in sort, awk or a database rather than a browser tab.
- The file that comes back is UTF-8 with Unix line endings, with no byte-order mark, whatever the file that went in had.

## Inputs

| Name | Label | Type | Default | Range |
| --- | --- | --- | --- | --- |
| `source` | Text file | file | chosen on your device | text/plain, text/csv, up to 20 MB |
| `trim` | Trim spaces from each line | toggle | true |  |
| `removeBlank` | Remove blank lines | toggle | true |  |
| `dedupe` | Remove duplicate lines | toggle | true |  |
| `caseSensitive` | Match case | toggle | false |  |
| `sort` | Order | select | ascending |  |
| `reverse` | Reverse the order | toggle | false |  |
| `natural` | Read numbers as numbers | toggle | true |  |

## Outputs

- `linesOut` — Lines out (integer), primary
- `linesIn` — Lines in (integer)
- `duplicates` — Duplicates removed (integer)
- `blanks` — Blank lines removed (integer)

## Questions

### Is the list I paste in uploaded anywhere?

No. The file is read in this browser tab and processed by a worker running on your own machine. There is no server to send it to, and the automated test for this page fails if any request leaves this site while a file is loaded.

### Why did "Milk" disappear when "milk" was already in the list?

Because matching ignores case unless you turn "Match case" on. A list of names, addresses or products usually wants Milk and milk treated as one entry, so that is the default. Turn "Match case" on and both are kept, and they will also sort apart from each other.

### Why is Zebra after apple rather than before it?

Because the lines are ordered by English collation rules rather than by character codes. Every capital letter has a lower character code than every lower-case one, so a raw code order gives Zebra, apple, banana — which is the order a computer sees and nobody wants. Collation also puts Ångström beside Anchor instead of after Zulu.

### Why is item10 after item2 instead of between item1 and item2?

Because "Read numbers as numbers" is on by default, so a run of digits inside a line is compared as a number. Turn it off under "More options" for a strict character-by-character order, which gives item1, item10, item2.

### What counts as a blank line?

A line containing nothing, or nothing but spaces and tabs. That holds whether or not trimming is on, so the two toggles cannot disagree about what blank means. A line of three spaces is removed by "Remove blank lines" even with trimming switched off.

### My file came back with different line endings.

It did. Windows CRLF and classic Mac CR are both read correctly, and what is written back always uses a single newline after each line, with one at the end of the file. Every editor in current use opens that, including Notepad.

### Why was my file refused?

Two things are refused rather than mangled. A file saved as UTF-16 begins with a byte-order mark that says so, and decoding it as UTF-8 would produce a page of replacement characters that looks like the tool broke the file — re-save it as UTF-8 instead. A file with a zero byte near the start is binary rather than text, and running line tools over it would produce nonsense.

## Sources

- [ECMAScript Language Specification: Array.prototype.sort](https://tc39.es/ecma262/) — Ecma International (TC39), applies to ECMAScript 2027 draft. Retrieved 2026-07-30.
- [ECMAScript Internationalization API Specification: Intl.Collator](https://tc39.es/ecma402/) — Ecma International (TC39), applies to ECMAScript 2027 draft. Retrieved 2026-07-30.
- [Unicode Technical Standard #10: Unicode Collation Algorithm](https://www.unicode.org/reports/tr10/) — Unicode Consortium, applies to 2026. Retrieved 2026-07-30.
- [Encoding Standard, UTF-8 decode](https://encoding.spec.whatwg.org/) — WHATWG, applies to 2026. Retrieved 2026-07-30.

---

Estimate, not advice. See https://tessalor.com/en/disclaimer.
Machine-readable catalogue: https://tessalor.com/api/tools.json