docs: comprehensive documentation for developers and AI agents

.claude/commands/add-codec.md

@@ -0,0 +1,51 @@
+# Command: add-codec
+
+Add a codec (encoder/decoder pair) to an existing store or create a new codec for `ValueCodecs`/`KeyCodecs`.
+
+## Usage
+
+```
+/add-codec <description of what to encode/decode>
+```
+
+## Examples
+
+```
+/add-codec add msgpack serialization to my store
+/add-codec add base64 encoding to keys
+/add-codec create a Zod-validated value codec for UserSchema
+```
+
+## What this command does
+
+1. Reads `dol/kv_codecs.py` to understand existing codecs and patterns
+2. Reads `dol/trans.py` for `Codec`, `ValueCodec`, `KeyCodec` definitions
+3. Implements the codec as:
+   - A `ValueCodec(encoder=..., decoder=...)` or `KeyCodec(encoder=..., decoder=...)` if it's a reusable pair
+   - A `wrap_kvs(store, obj_of_data=..., data_of_obj=...)` if it's one-off
+4. Shows how to apply it to a store and how to compose it with existing codecs
+
+## Output pattern
+
+```python
+from dol.trans import ValueCodec
+import msgpack
+
+msgpack_codec = ValueCodec(
+    encoder=msgpack.dumps,
+    decoder=msgpack.loads,
+)
+
+# Apply to any store
+MsgpackStore = msgpack_codec(dict)
+
+# Compose: msgpack + gzip
+from dol import ValueCodecs
+compressed_msgpack = msgpack_codec + ValueCodecs.gzip()
+```
+
+## Notes
+
+- Use `Codec.compose_with` / `+` operator to chain codecs
+- See `dol/kv_codecs.py` for real examples of `ValueCodecs.*` factories
+- See `misc/docs/python_design.md` section "Codec Abstraction" for details

.claude/commands/explain-store.md

@@ -0,0 +1,46 @@
+# Command: explain-store
+
+Explain how a given dol store works, tracing the key/value transform pipeline.
+
+## Usage
+
+```
+/explain-store <store_expression_or_class_name>
+```
+
+## What this command does
+
+1. Finds the store definition in the codebase
+2. Traces the MRO (method resolution order) to identify all transform layers
+3. For each `wrap_kvs` layer, shows what transforms are applied
+4. Draws the data flow diagram for read and write operations
+5. Shows a concrete example: what happens when you do `store['some_key']` and `store['some_key'] = value`
+
+## Example output format
+
+```
+Store: JsonFileStore (wrap_kvs applied to Files)
+
+Read pipeline:
+  key='report'
+  → id_of_key: 'report' + '.json' = 'report.json'
+  → Files.__getitem__('report.json')
+  → raw_data: b'{"x": 1}'
+  → obj_of_data: json.loads(raw_data)
+  → returns: {'x': 1}
+
+Write pipeline:
+  key='report', obj={'x': 1}
+  → id_of_key: 'report.json'
+  → data_of_obj: json.dumps({'x': 1}) = '{"x": 1}'
+  → Files.__setitem__('report.json', '{"x": 1}')
+
+Iteration:
+  Files.__iter__() → ['report.json', 'data.json', ...]
+  → key_of_id: strip '.json' → ['report', 'data', ...]
+```
+
+## Notes
+
+- See `dol/base.py:Store.__getitem__` for the actual implementation
+- See `misc/docs/python_design.md` for the full class hierarchy

.claude/commands/new-store.md

@@ -0,0 +1,52 @@
+# Command: new-store
+
+Create a new dol store wrapping a given backend, with key/value transforms.
+
+## Usage
+
+```
+/new-store <description of what the store should do>
+```
+
+## What this command does
+
+1. Reads `dol/base.py` to understand `KvReader`/`KvPersister`/`Store`
+2. Reads `dol/trans.py` to understand `wrap_kvs` and `store_decorator`
+3. Asks clarifying questions if needed:
+   - What is the backend? (files, S3, DB, dict, custom)
+   - What are the keys? (strings, paths, tuples?)
+   - What are the values? (bytes, JSON, Python objects?)
+   - Read-only or read-write?
+   - Any key transformation needed? (prefix, suffix, format conversion)
+   - Any value serialization needed? (JSON, pickle, gzip, custom)
+4. Generates the store class/factory using `wrap_kvs` (preferred) or subclassing
+5. Adds a docstring and a doctest example
+
+## Output pattern
+
+```python
+from dol import wrap_kvs, KvReader  # or KvPersister
+# + relevant codec imports
+
+def make_<name>_store(root_path, ...):
+    """<docstring>
+
+    >>> s = make_<name>_store(...)
+    >>> s['key'] = value
+    >>> s['key']
+    value
+    """
+    return wrap_kvs(
+        <backend>,
+        id_of_key=...,
+        key_of_id=...,
+        obj_of_data=...,
+        data_of_obj=...,
+    )
+```
+
+## Notes
+
+- Always include a working doctest using `dict` as the backend
+- Follow the `X_of_Y` naming convention for transforms
+- See `CLAUDE.md` and `misc/docs/python_design.md` for full context

.claude/rules/dol-conventions.md

@@ -0,0 +1,54 @@
+# dol Coding Conventions
+
+## Interface Choices
+
+- **New read-only stores**: subclass `KvReader` (from `dol.base`), not `Mapping` directly.
+- **New read-write stores**: subclass `KvPersister` (from `dol.base`).
+- **Avoid subclassing `Store` directly** unless you need the `_id_of_key` / `_key_of_id` / `_data_of_obj` / `_obj_of_data` hook protocol. Prefer `wrap_kvs` instead — it's more composable and doesn't create a new class in the inheritance chain.
+
+## Naming Conventions
+
+- Transform functions use `X_of_Y` naming: `key_of_id` (given `_id`, return `key`), `id_of_key` (given `key`, return `_id`). Follow this in new code.
+- Use `_id` for inner/backend keys, `k` or `key` for outer/interface keys.
+- Use `data` for serialized/raw backend values, `obj` or `v` or `value` for outer Python objects.
+
+## `wrap_kvs` Usage
+
+- Prefer `wrap_kvs` over subclassing when adding key/value transforms.
+- Use `obj_of_data`/`data_of_obj` (no key context) when the transform is the same for all values.
+- Use `postget`/`preset` (with key context) only when the transform depends on the key (e.g., file extension).
+- **Do not use `bytes.decode` directly** as `obj_of_data` — use `lambda b: b.decode()` instead (signature-inference bug, Issue #9).
+
+## Store Construction
+
+- Always test new stores with `dict()` as backend before using real storage.
+- Make stores parametrizable: accept `store=None` (or `store_factory=dict`) as a parameter.
+- Return the store from a factory function rather than hardcoding the backend.
+
+## Purity and Dependencies
+
+- Core `dol` has no external dependencies. Keep new core code dependency-free.
+- If a new feature needs an external dependency, put it in a separate module with a clear optional import and a helpful error message if the dependency is missing.
+
+## Codecs
+
+- Use `ValueCodecs` and `KeyCodecs` from `dol.kv_codecs` for common encoders/decoders.
+- For custom codecs, use `dol.trans.ValueCodec` / `KeyCodec` dataclasses (not ad-hoc lambdas) when the codec needs to be reused or composed.
+- Compose codecs with `+` operator: `ValueCodecs.pickle() + ValueCodecs.gzip()`.
+
+## Caching
+
+- Use `@cache_this` for expensive properties/methods, specifying an explicit `cache=` store if persistence across sessions is needed.
+- Use `cache_vals(store)` to add an in-memory read cache to a slow store.
+- Use `store_cached(cache_store)(func)` to persist function results.
+
+## Testing
+
+- Tests live in `dol/tests/`.
+- Doctests in module docstrings are the primary documentation for functions — keep them runnable.
+- Use `utils_for_tests.py` for shared test fixtures.
+
+## No Silent Failures
+
+- If a transform function is passed that will fail silently, raise an informative error early (e.g., in `__init__` or at decoration time).
+- Prefer explicit errors over silent incorrect behavior.