Reverse one or two legacy readings of UTF-8 text
Admin ·
Text can become garbled when a reader uses a different encoding from the writer. If the visible characters retain the original byte values, a strict UTF-8 decode can suggest a repair that you can check against the surrounding words.
Read the three repair candidates in the example
Load example and select Analyze encoding. The preview starts Café — é. The three candidates occur at columns 4, 7 and 11. The repaired bytes C3 A9 produce é. The three bytes E2 80 94 produce an em dash.
The garbled dash contains the euro sign and a curly quotation mark, whose byte mappings identify the Windows-1252 interpretation. The é span can be reversed through either of the supported legacy decoders.
| Original span | Proposed text | Byte evidence |
|---|---|---|
| é | é | C3 A9 |
| — | — | E2 80 94 |
| é | é | C3 83 and C2 A9, then C3 A9 |
WHATWG Encoding Standard, single-byte decodersWHATWG Windows-1252 indexRFC 3629, UTF-8 syntax
Compare one pass and two passes on the last accent
Change Maximum repair passes to One pass and analyze again. The last garbled span becomes é. Change back to Two passes to recover é in the preview. The first pass brings two reversible sequences next to each other, making the second UTF-8 decode possible.
Clear the checkbox for column 11 to keep the source spelling for that span. The Japanese text and bicycle emoji stay in the output. The U+FFFD marker stays too. Recovering text at that position requires the original bytes or another copy.
RFC 3629, UTF-8 syntaxThe Unicode Standard, Unicode encoding forms
Save the selected repairs with deliberate cleanup
Review the Repaired text preview, then download the UTF-8 result. Keep source form for an encoding-only repair. Select NFC separately when a project requires combining sequences to use canonical composed forms where available.
The optional switches can remove one leading U+FEFF marker, delete U+200B zero-width spaces, or trim spaces and tabs at line ends. Leave a switch off when those characters carry meaning. Download JSON to retain the selected spans, bytes and cleanup choices.
Café — é.Unicode Standard Annex 15, Unicode Normalization FormsThe Unicode Standard, Unicode encoding forms
Common questions
Why does the tool call them repair candidates?
The same byte sequence can arise in genuine text. Compare the proposed spelling and context before using the result.
Which text files can I open?
Use UTF-8 or UTF-16 or UTF-32 with a byte-order mark. The tool can inspect a Unicode file containing already-garbled characters.
Sources
Put it into practice.
- Mojibake Detector and Fixer
- Text Encoding Detector
- Non-ASCII Character Finder
- Unicode Code Point Viewer