Background
#222 adds $ref support to JSON configs — a value of the form {"$ref": "path/to/other.json"} loads the entire referenced file. This is enough for the common case (factor a dataset_mixture into its own file).
Sometimes you want to reference only part of another file — e.g. pull a single named dataset entry out of a shared all_datasets.json rather than maintain one file per dataset. Standard JSON Pointer fragment syntax would express this as:
{
"datasets": [
{"$ref": "all_datasets.json#/droid_100"},
{"$ref": "all_datasets.json#/aloha_cabinet"}
]
}Proposed scope
- Parse the fragment after
# as a JSON Pointer (RFC 6901): /key1/key2/0/key3.
- Resolve refs as today, then walk the pointer through the loaded structure.
- Raise
RefError with a helpful message if any pointer segment is missing.
- Sibling-key deep-merge semantics carry over unchanged: if the resolved fragment is a JSON object, sibling keys merge in; otherwise sibling keys must be absent.
- Cycle detection should account for
(file_path, pointer) tuples, not just file paths, so two refs into different sub-keys of the same file aren't treated as a cycle but a self-referential pointer is.
Out of scope (separate follow-up)
Acceptance
- New tests in
tests/configs/test_refs.py covering: pointer to nested dict, pointer to list element, pointer to scalar, missing intermediate key, missing trailing key, sibling override on a fragment, fragment vs file cycle distinction.
- Module docstring in
src/opentau/configs/refs.py updated to describe the syntax.
Background
#222 adds
$refsupport to JSON configs — a value of the form{"$ref": "path/to/other.json"}loads the entire referenced file. This is enough for the common case (factor adataset_mixtureinto its own file).Sometimes you want to reference only part of another file — e.g. pull a single named dataset entry out of a shared
all_datasets.jsonrather than maintain one file per dataset. Standard JSON Pointer fragment syntax would express this as:{ "datasets": [ {"$ref": "all_datasets.json#/droid_100"}, {"$ref": "all_datasets.json#/aloha_cabinet"} ] }Proposed scope
#as a JSON Pointer (RFC 6901):/key1/key2/0/key3.RefErrorwith a helpful message if any pointer segment is missing.(file_path, pointer)tuples, not just file paths, so two refs into different sub-keys of the same file aren't treated as a cycle but a self-referential pointer is.Out of scope (separate follow-up)
$ref(also not supported in feat(configs): support $ref include in JSON configs #222).Acceptance
tests/configs/test_refs.pycovering: pointer to nested dict, pointer to list element, pointer to scalar, missing intermediate key, missing trailing key, sibling override on a fragment, fragment vs file cycle distinction.src/opentau/configs/refs.pyupdated to describe the syntax.