Skip to content

change: json_decode rejects malformed JSON instead of silent partial (#495)#517

Merged
InauguralPhysicist merged 1 commit into
mainfrom
fix/495-json-decode-strict
Jul 9, 2026
Merged

change: json_decode rejects malformed JSON instead of silent partial (#495)#517
InauguralPhysicist merged 1 commit into
mainfrom
fix/495-json-decode-strict

Conversation

@InauguralPhysicist

Copy link
Copy Markdown
Collaborator

The recursive-descent JSON parser was lenient: arrays/objects break on an unexpected token and returned the partial container, json_decode never checked for trailing input, and numbers stopped at the first non-numeric char. So malformed input succeeded silently — and a genuine JSON null was indistinguishable from a parse failure (both returned null).

Now raises value (was a silent partial)

Input Was Now
{ , {] {} raises
[1,2, , [1,2,3 [1, 2] / [1, 2, 3] raises
{"a": , {"a":1,} {"a": null} / {"a": 1} raises
[1,2,3] trailing [1, 2, 3] raises
42abc 42 raises
"" , notjson null raises
unterminated string, >200 deep partial / truncated raises

Valid JSON is unchanged, and a bare null still decodes to null — now distinguishable from a parse failure.

Design

A thread-local strict-parse error flag is set by the parse functions on any malformed condition (unexpected token, truncation, unterminated string, over-deep nesting). builtin_json_decode resets it on entry and, after parsing one value, requires only whitespace remains — raising value otherwise. The flag is checked only by json_decode: json_path and the ext_http header/body parsers call the same parser but ignore it, keeping their historical lenient behavior.

Load-bearing check

Nothing tested json_decode == null for validity, and every stdlib/test/example feeds valid JSON. Zero suite tests broke except test_json_depth — a >200-deep document now raises catchably rather than returning a truncated value; the security property ("no SIGSEGV") is preserved and re-asserted.

Tests

Suite section [116] extended — test_raise_on_bad_input.eigs, now 43 checks (truncated/partial/garbage raise; valid arrays/objects and a bare null still parse).

Remaining from the audit

Closes #495

🤖 Generated with Claude Code

…495)

The recursive-descent JSON parser was lenient: arrays/objects `break` on an
unexpected token and returned the *partial* container, numbers stopped at the
first non-numeric char, and json_decode never checked for trailing input. So
truncated documents ("[1,2,", "{\"a\":"), partial/garbage containers ("{]",
"[1,2,3,]"), unterminated strings, over-deep nesting, and trailing garbage
("[1,2,3] trailing", "42abc") all succeeded with rc=0 — and a genuine JSON
`null` was indistinguishable from a parse failure (both returned null).

Add a thread-local strict-parse error flag that the parse functions SET on
any malformed condition; builtin_json_decode resets it on entry, and after
parsing one value requires that only whitespace remains, raising `value`
otherwise. Valid JSON is unchanged. The flag is CHECKED only by json_decode —
json_path and the ext_http header/body parsers call the same parser but
ignore the flag, keeping their historical lenient behavior.

Load-bearing check: nothing tested `json_decode == null` for validity, and
every stdlib/test/example feeds valid JSON — zero suite tests broke except
test_json_depth (a >200-deep document now raises catchably rather than
returning a truncated value; the security property "no SIGSEGV" is preserved
and re-asserted).

Regression: suite section [116] extended (test_raise_on_bad_input.eigs, now
43 checks — truncated/partial/garbage raise; valid arrays/objects and a bare
`null` still parse).

Closes #495

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@InauguralPhysicist InauguralPhysicist merged commit 4e676ad into main Jul 9, 2026
17 checks passed
@InauguralPhysicist InauguralPhysicist deleted the fix/495-json-decode-strict branch July 9, 2026 07:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

json_decode: accepts truncated, partial, and trailing-garbage JSON as success

1 participant