change: json_decode rejects malformed JSON instead of silent partial (#495)#517
Merged
Merged
Conversation
…495) The recursive-descent JSON parser was lenient: arrays/objects `break` on an unexpected token and returned the *partial* container, numbers stopped at the first non-numeric char, and json_decode never checked for trailing input. So truncated documents ("[1,2,", "{\"a\":"), partial/garbage containers ("{]", "[1,2,3,]"), unterminated strings, over-deep nesting, and trailing garbage ("[1,2,3] trailing", "42abc") all succeeded with rc=0 — and a genuine JSON `null` was indistinguishable from a parse failure (both returned null). Add a thread-local strict-parse error flag that the parse functions SET on any malformed condition; builtin_json_decode resets it on entry, and after parsing one value requires that only whitespace remains, raising `value` otherwise. Valid JSON is unchanged. The flag is CHECKED only by json_decode — json_path and the ext_http header/body parsers call the same parser but ignore the flag, keeping their historical lenient behavior. Load-bearing check: nothing tested `json_decode == null` for validity, and every stdlib/test/example feeds valid JSON — zero suite tests broke except test_json_depth (a >200-deep document now raises catchably rather than returning a truncated value; the security property "no SIGSEGV" is preserved and re-asserted). Regression: suite section [116] extended (test_raise_on_bad_input.eigs, now 43 checks — truncated/partial/garbage raise; valid arrays/objects and a bare `null` still parse). Closes #495 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The recursive-descent JSON parser was lenient: arrays/objects
breakon an unexpected token and returned the partial container,json_decodenever checked for trailing input, and numbers stopped at the first non-numeric char. So malformed input succeeded silently — and a genuine JSONnullwas indistinguishable from a parse failure (both returnednull).Now raises
value(was a silent partial){,{]{}[1,2,,[1,2,3[1, 2]/[1, 2, 3]{"a":,{"a":1,}{"a": null}/{"a": 1}[1,2,3] trailing[1, 2, 3]42abc42"",notjsonnullValid JSON is unchanged, and a bare
nullstill decodes tonull— now distinguishable from a parse failure.Design
A thread-local strict-parse error flag is set by the parse functions on any malformed condition (unexpected token, truncation, unterminated string, over-deep nesting).
builtin_json_decoderesets it on entry and, after parsing one value, requires only whitespace remains — raisingvalueotherwise. The flag is checked only byjson_decode:json_pathand theext_httpheader/body parsers call the same parser but ignore it, keeping their historical lenient behavior.Load-bearing check
Nothing tested
json_decode == nullfor validity, and every stdlib/test/example feeds valid JSON. Zero suite tests broke excepttest_json_depth— a >200-deep document now raises catchably rather than returning a truncated value; the security property ("no SIGSEGV") is preserved and re-asserted.Tests
Suite section [116] extended —
test_raise_on_bad_input.eigs, now 43 checks (truncated/partial/garbage raise; valid arrays/objects and a barenullstill parse).main(with for-in uses live list length: append during iteration never terminates #491); ASan +detect_leaks=1: 2677/2677, leak tally 0 (partial-container error path is leak-clean).Remaining from the audit
is/of→ null (expression parser)Closes #495
🤖 Generated with Claude Code