Technology · PRACTICAL GUIDE
Python JSONDecodeError: Extra data? Identify the record framing before parsing
The first JSON value may be valid while more text follows. Preserve the original bytes, identify one document, JSON Lines or adjacent values, and verify the whole input before converting it.
Extra data means Python finished decoding one JSON value and then encountered non-whitespace. It does not, by itself, prove the file is JSON Lines or that line 2 begins a safe record. Preserve the original bytes and establish the producer’s framing contract before choosing a parser or rewriting anything.
Preserve the evidence and classify the suffix
Copy or hash the original file before experimenting. Record the input encoding, whether a BOM is present, the exception message, pos, lineno and colno, and a short escaped preview around the failure. Do not delete whitespace, replace every }{, or wrap raw text in brackets: those edits can alter strings, hide corruption or invent separators.
Use JSONDecoder.raw_decode on decoded text to see where the first value ends, then inspect only what follows. If the remainder is whitespace, you had one document. If another value begins, determine whether the producer promises one complete value per line, emits adjacent values, or accidentally appended unrelated bytes. The parser choice comes from that contract, not from the error line alone.
Choose the parser from the record contract
For one JSON document, json.load or json.loads is the correct strict check: a top-level object, array, string, number, boolean or null is one value, and unexplained trailing material should remain an error. If you control a collection, a single top-level array is the most portable JSON representation.
For JSON Lines, open text as UTF-8 and decode each nonblank physical line only when every line is one complete JSON value. The format allows objects, arrays and scalar values; it does not allow a blank line. A pretty-printed object spanning several lines is not one JSON Lines record and will fail if parsed line by line.
Consume adjacent values without guessing delimiters
When the producer truly emits adjacent JSON values without a line guarantee, a whitespace-skipping raw_decode loop can return each value and the character index where it ended. Validate the expected type or schema for every record, retain the record number and start index on failure, and require the final index to reach EOF after whitespace.
raw_decode accepts a string beginning with a JSON document; it is not automatically a bounded streaming reader. A large file needs a deliberate buffering design or a reviewed streaming parser. If you can change the producer, fix its framing instead of institutionalizing a repair script. Never manufacture commas with a global }{ replacement, because the same characters can occur inside strings and malformed boundaries need explicit rejection.
Map character positions to bytes and validate before conversion
Python documents JSONDecodeError.pos as the start index in doc, with lineno and colno derived from that decoded string. In our Python 3.12.14 fixture, {"é":1}{"b":2} failed at pos 7, but eight UTF-8 bytes preceded the second value. To locate the raw byte boundary after decoding UTF-8, use len(text[:pos].encode("utf-8")); never seek a binary file directly to pos when earlier text may be non-ASCII.
The same fixture produced Extra data at line 2 column 1 for true JSON Lines, line 1 column 8 for compact adjacent values, and line 4 column 1 for two pretty-printed adjacent objects. raw_decode consumed all three sequences, while line parsing worked only for JSON Lines and failed on the pretty-printed sequence. Convert only after every value passes syntax and application checks, the complete suffix is consumed, and record counts match; then serialize parsed values to a new temporary array or JSON Lines file and keep the original for rollback.
