diff --git a/architecture.md b/architecture.md index 8829bdd..714ee5d 100644 --- a/architecture.md +++ b/architecture.md @@ -66,6 +66,21 @@ Every Gall agent implements this interface: -- ``` +#### State fields resolve without `.state` + +Because `=| state-0` pins state at the head of the subject and `=* state -` aliases it, wing resolution finds the inner fields *directly*. Prefer the unprefixed form: + +```hoon +:: GOOD: wing search finds books inside the head-pinned state +=. books (~(put by books) flag entry) +?: (~(has by invites) flag) ... + +:: ALSO FINE but redundant — same lookup, more typing +=. books.state (~(put by books.state) flag entry) +``` + +Keep the explicit `.state` only when local shadowing forces it (e.g. an arm that takes a `=state` argument). Inner-struct accesses like `members.notebook-state` or `notebook.notebook-state` are unaffected — those reach through a value bound by `=/` or a sub-core door, not the agent state at subject head. + ### State Versioning and Migration For persistent production agents, default to tagging state with a version number in the head and migrating through each version sequentially in `on-load`: @@ -89,6 +104,24 @@ For persistent production agents, default to tagging state with a version number -- ``` +**Always write `state-N-to-N+1`, never `state-N-to-current`.** Each migration arm bumps exactly one version. The `=?` chain in `on-load` walks through every step. This is the single most important migration pattern — the alternative is wrong: + +```hoon +:: WRONG: nested ?: cascade where each old version migrates directly to current. +:: Each branch duplicates every backfill from every later version. Adding state-9 +:: forces editing every prior branch. +?: =(tag %5) + =/ s=state-5 !<(state-5 old) + :: ...300 lines of code that turns state-5 directly into state-8... + =/ s8=state-8 ... + cor(state s8) +?: =(tag %4) + =/ s=state-4 !<(state-4 old) + :: ...same 300 lines, slightly different starting fields... +``` + +With the linear chain, you write `state-5-to-6` once and every prior version flows through it for free. State types `state-0` through `state-N` all stay defined in `sur/` (they're load-bearing for migrations); only the migration arms in `on-load` reach them. + ### Cards (Effects / Messages) Cards are the agent's output — instructions to the runtime: @@ -280,6 +313,88 @@ Large agents separate message types into distinct roles. This is not just naming +$ r-seat u-seat ``` +The outer types route by identity (`flag`, `nest`); the **inner subtypes drop the subject** and carry only the verb. The outer envelope already pinned which group/channel/note we're talking about, so repeating it inside is noise: + +```hoon +:: a-group: actions on a specific group. Outer [%group =flag =a-group] +:: carries the flag; inner verbs do not repeat it. +:: ++$ a-group + $% [%rename title=@t] + [%delete ~] + [%channel =nest =a-channel] :: nested again: outer drops, inner verbs follow + [%invite who=ship] + == + +:: c-group: same shape minus client-only verbs (e.g. %restore, %publish), +:: plus peer-only verbs (e.g. %member-join). %rename, %delete, %channel, +:: %invite are shared 1:1 with a-group. +:: ++$ c-group + $% [%rename title=@t] + [%delete ~] + [%channel =nest =c-channel] + [%invite who=ship] + [%member-join ~] :: peer-only: not exposed to UI + [%member-leave ~] + == +``` + +### Four principles for getting ACUR right + +**1. The outer type carries identity; the inner type carries only the verb.** +Don't repeat `flag=` / `notebook-id=` / `channel-id=` inside the inner tags — the outer wrapper already routed it there. Compare: + +```hoon +:: GOOD: nested envelope, inner verbs are clean +[%notebook =flag [%create-folder parent=(unit @ud) name=@t]] +[%notebook =flag [%note id=@ud [%update body=@t expected-revision=@ud]]] + +:: BAD: flat shape with redundant routing fields on every verb +[%create-folder notebook-id=@ud parent=(unit @ud) name=@t] +[%update-note notebook-id=@ud note-id=@ud body=@t expected-revision=@ud] +``` + +**2. `src.bowl` is the actor — never thread `actor=ship` through actions, commands, or updates.** +The runtime already authenticates the sender. Adding `actor=` fields invites forgery (the sender can lie about who sent it) and bloats every type. The `?> =(our src):bowl` (a-*) or permission-check (c-*) gate is what enforces identity. + +```hoon +:: GOOD: src.bowl on every poke; permission checks read it directly +[%rename title=@t] :: actor = src.bowl + +:: BAD: actor field threaded through every variant +[%rename title=@t actor=ship] :: forgeable; redundant +``` + +**3. Updates are *fat* — `%updated` carries the whole post-change entity, not a delta.** +Subscribers should be able to apply an update by overwriting the whole entity, no field-by-field merging. Attribution (e.g. `updated-by=ship`) lives **on the entity**, not on the update arm. + +```hoon +:: GOOD: fat updates ++$ u-notebook + $% [%created =notebook =visibility] + [%updated =notebook] :: the whole post-change notebook + [%deleted ~] + [%note nid=@ud =u-note] + [%folder fid=@ud =u-folder] + == + +:: BAD: delta-shaped updates that prefix every tag with the subject ++$ u-notebook + $% [%notebook-created =notebook] + [%notebook-renamed notebook-id=@ud title=@t actor=ship] + [%notebook-deleted notebook-id=@ud actor=ship] + [%note-renamed note-id=@ud title=@t actor=ship] + == +``` + +**4. The a-/c- split is a trust boundary, not a vocabulary split.** +The test for whether a verb belongs in a-* or c-*: would `?> =(our src):bowl` crash on a legitimate caller? +- **a-*** verbs originate locally — the gate `?> =(our src):bowl` is enforceable on every arm. +- **c-*** verbs may arrive from any peer ship — they get `?> (can-edit src.bowl)` or similar permission checks instead. + +A common mistake: putting a *cross-ship command* (e.g. `%notify-invite`, where the host pokes the invitee) in a-* alongside local UI actions. The local-only assertion then crashes on the remote sender. If the verb requires a sender other than `our.bowl`, it belongs in c-*. Most a-* verbs do have a c-* counterpart (the local UI sends `a-foo`; the local core forwards a `c-foo` to the host); the few that don't are purely local concerns (UI-side state toggles, optimistic publishes). + **The data flow:** ``` @@ -295,6 +410,37 @@ Client UI --(a-action)--> go-core --(c-command)--> se-core (host) to members) subscribers) ``` +### Always poke the host + +A subtle but important operational pattern follows from the trust boundary in principle 4: the **action handler always pokes the host with a c-command, regardless of whether the host is us or a remote ship**. If `host == our.bowl`, Gall loops the poke back through `+poke %notes-command` and dispatches it through the host engine (`se-core`) — the action handler never branches on host vs. subscriber. + +```hoon +:: GOOD: a-action handler always emits a poke to the host +:: Gall loops it back to ourselves if we're the host. +++ no-action + |= act=action:n + ^+ no-core + ?> ?=(%notebook -.act) + =/ cmd=command:n [%notebook flag.act (a-notebook-to-c-notebook a-notebook.act)] + %- emit + :* %pass /notes/poke/(scot %p ship.flag.act)/[name.flag.act] + %agent [ship.flag.act %notes] + %poke notes-command+!>(cmd) + == + +:: BAD: branching on host/sub at the action layer +++ poke + :: ... + =/ entry (~(got by books) flag) + ?: ?=(%pub -.net.entry) + se-abet:(se-poke:(se-abed:se-core flag) ...) :: host path + no-abet:(no-action:(no-abed:no-core flag) act) :: subscriber path +``` + +The collapse this enables: c-command processing is the **single dispatch point** for all state-changing logic. The action handler converts shape (`a-` to `c-`) and routes; the host engine does all the work. There's no "if we're the host, run it locally" shortcut. This costs one Gall hop on self-pokes — negligible — and buys exactly one place where validation, permission checks, log appending, and update fanout live. + +The `++ no-action` arm above is in `no-core` (the subscriber-side engine) but the pattern is the same when we're the host: the poke goes to `[ship.flag %notes]`, which is `[our.bowl %notes]`, which Gall delivers as a self-poke back to `+poke %notes-command`. From there `se-poke:(se-abed:se-core flag)` does the real work. + The response path in the agent converts to multiple versions simultaneously: ```hoon @@ -377,6 +523,35 @@ Marks define how data is serialized and converted between formats. Every mark is - `grad` — revision control strategy (usually `%noun`) - JSON arms in grow/grab enable HTTP API integration +### Typed Marks for Peek Endpoints + +When a peek endpoint serves both noun-typed clients (in-ship scries, agent-to-agent reads) and JSON-over-HTTP clients (`.json` query, web UI), define a typed mark per endpoint and let `++ grow ++ json` do the encoding. The agent's `+on-peek` returns the raw typed value — Eyre handles JSON conversion on the wire. + +```hoon +:: /mar/notes/notebook.hoon — typed mark for one notebook +/- n=notes +/+ notes-json +|_ nd=notebook-detail:n +++ grad %noun +++ grab |% ++ noun notebook-detail:n -- +++ grow + |% + ++ noun nd :: raw value for noun callers + ++ json (notebook-detail:enjs:notes-json nd) :: JSON for HTTP callers + -- +-- +``` + +```hoon +:: in +on-peek: build the noun-typed value, return through the typed mark +[%x %v0 %notebook ship=@ name=@ ~] + =/ =flag:n [(slav %p ship.pole) `@tas`name.pole] + ?~ entry=(get-book flag) ~ + ``notes-notebook+!>(`notebook-detail:n`[flag notebook.notebook-state.u.entry]) +``` + +This is preferable to `json+!>(...)` for any peek that might be consumed by another agent — that path then has to decode JSON instead of getting a typed noun. Once you commit to the wire-format-agnostic shape, you also factor out a small surface type per endpoint (`notebook-summary`, `member-record`, `published-record`, etc.), which makes the noun side legible without parsing JSON. Reach for this any time a peek endpoint is HTTP-facing *and* might be consumed in-Urbit. + --- ## Versioning Strategy diff --git a/patterns.md b/patterns.md index 8b110e7..7cb7a7a 100644 --- a/patterns.md +++ b/patterns.md @@ -42,6 +42,143 @@ Use `=*` (tistar) when you just want a shorter name for an existing wing or expr Use `=/` in place of `=*` only when you will be modifying the value but need to track the original, or when you want to freeze the value at bind time for some other reason. +**Audit pass criteria.** When reviewing a freshly written arm, scan every `=/` and decide: + +- **Source is a pure wing access (`a.b.c`)? → `=*`** if you reference it 2+ times and the path is verbose enough to obscure the call sites; **inline directly** if used once or if the source is short (roughly 14 characters or less). +- **Source is a computed value** (function call, literal, constructed cell)? → keep `=/`. These aren't aliases; they freeze a result. +- **`=/ =face:type source`?** → keep. The `=face` form is a face-mold cast, not aliasing. +- **Used zero times?** → delete. + +The rule of thumb: `=/` for results, `=*` for aliases, inline for once-and-short. Mixing them up isn't wrong, just noisy — `=/ fid=@ud id.c-notebook.cmd` adds a redundant `@ud` annotation and a copy where `=* fid id.c-notebook.cmd` would do. + +### Faces Default to the Type Name + +When binding a single value of some type, the default is the `=face:type` form — Hoon auto-creates a face matching the type name, so you don't write the name twice: + +```hoon +:: GOOD: =face:type — face is the type's name +=/ =flag:n [(slav %p ship.pole) `@tas`name.pole] +=/ =notebook:n [nid title.act [our now now our]:bowl] +=/ =note:n (~(got by notes.notebook-state) nid) + +:: REDUNDANT: writing the face name explicitly when it matches the type +=/ flag=flag:n [(slav %p ship.pole) `@tas`name.pole] + +:: BAD: ad-hoc abbreviation with no semantic motivation +=/ nb=notebook:n [nid title.act [our now now our]:bowl] +``` + +The slight stutter in lines like `se-core(flag flag, ...)` (a wing-replace where the new value's face matches the wing being replaced) is acceptable. Wing resolution is right-to-left dot lookup, so a local face named `notebook` doesn't shadow `.notebook` of `notebook-state` in expressions like `notebook.notebook-state` — the dotted path still resolves correctly. + +#### When to use a semantic face instead + +Reach for a non-default face when the **role** the value plays is more informative than its type — typically when multiple values of the same type coexist in scope and you need to tell them apart. The face describes *which one*, the type tag still carries *what kind*: + +```hoon +:: GOOD: two notebooks in scope; face conveys role, type tag stays explicit +=/ old-nb=notebook:n notebook.notebook-state +=. notebook.notebook-state notebook(title 'renamed', updated-by src.bowl) +=/ new-nb=notebook:n notebook.notebook-state +(diff-notebooks old-nb new-nb) + +:: GOOD: source vs destination of a move +=/ src-folder=folder:n (~(got by folders.notebook-state) from-fid) +=/ dst-folder=folder:n (~(got by folders.notebook-state) to-fid) + +:: GOOD: a placeholder vs the real thing +=/ placeholder-net=net:n [%sub *@da |] +:: ...later when the snapshot arrives, real-net is the one we save +``` + +The rule of thumb: if the face would just restate the type, use `=type` and let the type name speak. If the face would carry information the type alone doesn't (`old`, `new`, `src`, `dst`, `placeholder`, `target`), use a semantic name and keep the type tag explicit. + +Same logic for shadowing collisions — if a local `flag` would clash with `flag.act` you're already destructuring, pick a face that disambiguates (`book-flag`, `target-flag`) rather than dropping back to a vowel-stripped abbreviation. + +### Cast Above Assertions + +In a gate arm, the cast (`^+ se-core`, `^- type`) goes immediately after the gate sample, *above* any `?>`/`?<` assertions: + +```hoon +:: GOOD: cast first, then narrow +++ se-rename-notebook + |= cmd=c-cmd:n + ^+ se-core :: cast: tells the type system what we produce + ?> ?=(%rename -.c-notebook.cmd) :: narrow inside that cast's scope + ?> (se-is-owner src.bowl) + ... + +:: BAD: assertions before the cast +++ se-rename-notebook + |= cmd=c-cmd:n + ?> ?=(%rename -.c-notebook.cmd) + ^+ se-core :: cast comes too late + ... +``` + +The cast is a contract about the arm's output. Assertions narrow the *input* types — they should run inside the cast's scope, not before it. This ordering also matches how the rest of the body reads: cast on top, then guards, then logic. + +### Tuple Types with `*` for Don't-Care Parts + +When binding a value whose type is a tuple but you only access part of it, replace the unused fields with `*`: + +```hoon +:: GOOD: only -.net.entry is checked, so the second half is * +=/ entry=[=net:n *] + (~(got by books) flag) +?: ?=(%pub -.net.entry) + ... + +:: GOOD: only the notebook-state half is read +=/ entry=[* =notebook-state:n] + (~(got by books) flag) +=* title title.notebook.notebook-state.entry + +:: BAD: full type when half is unused +=/ entry=[=net:n =notebook-state:n] + (~(got by books) flag) +?: ?=(%pub -.net.entry) :: notebook-state.entry never used + ... +``` + +`*` (the bunt of any noun) is the hoon idiom for "I don't care about this part". The compiler still type-checks the parts you *do* annotate. This applies inside gate samples too — `|= [f=flag-v9:n [* =notebook-state:n]]` for a turn body that only reads the notebook-state half. + +### Same-Subject Cell Collapse `[a b c]:subject` + +When constructing a cell whose elements are all wings of the same subject, **and** that cell is the *last* sub-expression of the surrounding form, collapse to `[a b c]:subject`: + +```hoon +:: GOOD: gate args are wings of bowl; the cell is the last (only) expression +(slugify [title id]:notebook.notebook-state) + +:: GOOD: bowl-tail of the notebook tuple. The cell is the LAST element +:: of the outer tuple, so the splice extends through. +=/ =notebook:n + [nid title.act [our now now our]:bowl] + +:: GOOD: list-item construction — last sub-expression is :notebook-state +`[flag [notebook visibility]:notebook-state] +``` + +The "last sub-expression" caveat matters because `[a b c]:subject` is structurally `=>(subject [a b c])` — the right-associative noun shape splices the subject into the trailing slot. If the cell isn't last, the splice doesn't apply and you have to write the wings out: + +```hoon +:: WRONG to apply collapse here: revision=@ud follows the bowl-cell, +:: so [src now now src]:bowl is NOT the last element of the outer tuple. +:: The note construction must spell every wing: +=/ =note:n + :* nid + id.notebook.notebook-state + fid + title.c-notebook.cmd + ~ + body-md.c-notebook.cmd + src.bowl now.bowl now.bowl src.bowl :: inline; collapse not available + revision=0 + == +``` + +Reach for this when an arm has multiple `[our now now our]:bowl` or `[title id]:foo`-shape constructions. It's micro-optimization — three keystrokes saved per use — but it reduces visual repetition in dense type-construction code. + ### Bare Computed Arms for Predicates When a predicate depends only on state and needs no arguments, write it as a bare arm — no gate: @@ -282,6 +419,56 @@ This is a good pattern when you already have nested helper doors or wrappers. It Note: abed-abet is a specific instance of a more generic pattern. Old base desk code (dojo, clay) applies this pattern for state/change accumulation on specific parts of state. That kind of factoring lets you capture logic relating to specific parts of state in dedicated "engines" which maintain all the invariants, and — because they all produce the modified engine core — can be chained very easily. +### Per-Entity Engines (abed/abet) + +When state contains a map of entities (groups, channels, notebooks, threads) and many operations are *scoped to one entity*, factor each entity's logic into a sub-core that loads-by-id, mutates, and writes-back. The pattern has two named arms, with n number of operation arms: + +- `++ se-abed` — load: take the entity id, look it up in state, return a sub-core with that entity's data hoisted into the immediate subject +- entity-mutating arms — `++ se-rename`, `++ se-update-note`, etc. — operate on the loaded subject, accumulating cards +- `++ se-abet` — write back: stash the (possibly mutated) entity into state, hand control back to the parent core + +This collapses the "look up by flag → mutate → write back" boilerplate that would otherwise repeat at every call site: + +```hoon +:: the sub-core: a door over [identifier, entity-data, accumulator] +++ se-core + |_ [=flag =net =notebook-state gone=_|] + ++ se-core . + ++ emit |=(=card se-core(cor cor(cards [card cards]))) :: accumulate via parent + :: + ++ se-abed :: load: flag -> populated se-core + |= f=flag + ^+ se-core + ?> =(ship.f our.bowl) :: host-side assertion + ?~ entry=(~(get by books) f) ~|(not-found+f !!) + se-core(flag f, net net.u.entry, notebook-state notebook-state.u.entry) + :: + ++ se-abet :: write back: persist + return parent core + ^+ cor + ?: gone :: marked deleted + cor(books (~(del by books) flag)) + cor(books (~(put by books) flag [net notebook-state])) + :: + ++ se-rename :: example mutating arm + |= title=@t + ^+ se-core + =. title.notebook.notebook-state title + (se-update [%updated notebook.notebook-state]) :: fat update + -- + +:: call site: the chain reads top-to-bottom +:: (load flag) -> (apply mutation) -> (write back) +=. cor se-abet:(se-rename:(se-abed:se-core flag) title) +``` + +Three things this pattern unlocks: + +1. **Top-level dispatch arms collapse.** Instead of `+peek` having seven arms — one per resource — that each parse the flag, look it up, check permissions, and encode JSON, `+peek` becomes a single delegate to `++ no-peek` inside the sub-core, and the sub-core handles all per-notebook reads with the entity already loaded. +2. **Permission and structural checks live in one place.** `se-abed` enforces "this is host-side" once; downstream arms don't re-check. A parallel `no-core` (subscriber-side) does the same for `?=(%sub -.net)`-only operations — though typically `no-abed` accepts any net and the `%sub` guard moves into the specific arms that need it, so peek/watch can use the same loaded core regardless of host/subscriber mode. +3. **`abet` makes deletion symmetric.** A `gone=_|` flag on the sub-core lets a delete arm signal "remove this entity" without the call site knowing the difference; `abet` reads the flag and calls `del` instead of `put`. + +Use this pattern when state contains `(map id entity)` and at least three or four operations route to that entity. + ### Cascading Guards (Early Return) Hoon doesn't have `return`. Instead, stack conditional checks that handle edge cases first, falling through to the main logic: @@ -528,6 +715,94 @@ When HTTP handler or admin logic should follow the same path as a typed poke, re == ``` +### Deduplicate Three-Repeat Idioms Behind a Helper Arm + +Once a small **stateless** transform — a URL parse, a permission check on an explicit argument, a JSON envelope shape — appears three or more times in the same file, name it. The classic case is a one-liner like `++ strip-query`, which drops the query string from a URL tape and is called from every URL-matching branch in an HTTP handler: + +```hoon +:: GOOD: a one-line helper, defined once +++ strip-query + |= url=tape + ^- tape + =/ qi=(unit @ud) (find "?" url) + ?~ qi url + (scag u.qi url) + +:: call sites — three branches in the same +serve-http arm: +=/ url-path=tape (strip-query (trip url.request.inbound-request)) +... +=/ pub-path=tape (strip-query (slag 11 url-tape)) :: /notes/pub/... +... +=/ share-path=tape (strip-query (slag 13 url-tape)) :: /notes/share/... + +:: BAD: open-coding the same find/scag dance at every site +=/ url-tape=tape (trip url.request.inbound-request) +=/ qi=(unit @ud) (find "?" url-tape) +=/ url-path=tape ?~(qi url-tape (scag u.qi url-tape)) +... +=/ rest=tape (slag 11 url-tape) +=/ qi=(unit @ud) (find "?" rest) +=/ pub-path=tape ?~(qi rest (scag u.qi rest)) +... +``` + +Good targets: URL/path parsing, permission predicates that take an explicit subject (`++ can-edit |= who=ship`), JSON envelope construction, small format conversions. A one-line helper that eliminates 8 open-coded copies is worth more than declaring the same arm inline 8 times. + +Pair this with the inline `?~ name=expr` form so a `(get …)`-style helper's call sites stay a single line: + +```hoon +?~ qi=(find "?" url) url :: bind + null-check inline + +:: vs the unrolled form +=/ qi=(unit @ud) (find "?" url) +?~ qi url +``` + +#### When restructuring beats a helper arm + +The previous example is the right move because `strip-query` is a pure function of its argument — there is no entity it operates on, no `?>` it would do for you, no "next call" it would route to. When the duplication is *shaped differently* — every call site loads the same entity by id and then operates on its fields — a helper arm is a half-measure. A `++ get-book` that looks up a notebook is one call site's worth of cleanup, but every call site still has to: + +```hoon +?~ entry=(get-book flag) ``json+!>(~) :: null-check +?> (can-view-flag flag src.bowl) :: permission check +=/ fld-list (turn ~(val by folders.notebook-state.u.entry) ...) :: reach through .u.entry +``` + +The repeated work isn't the lookup — it's the *load entity → check → reach into fields*. A helper arm only collapses the first line. The real fix is to push all of that into a per-entity engine (see "Per-Entity Engines (abed/abet)" above), where `abed` does the load + permission check once and the inner arms see the entity already in subject. + +Rule of thumb: if your candidate helper takes only an id (`flag`, `nest`, `note-id`) and every caller follows up by reading the looked-up value's fields, the right answer is probably an `abed`-shaped sub-core, not a helper arm. If the candidate is a pure transform with no entity in sight, a helper arm is the right call. + +### `|^` (kelt) for Arm-Scoped Helpers + +When a handler accumulates four or five small helpers that nobody else needs to call, wrap the arm in `|^` and nest them inside. The helpers become inaccessible from the rest of the core, which is what you want — they were never meant to be public: + +```hoon +++ poke + |= [=mark =vase] + ^+ cor + |^ :: scoped helpers nested below + ?+ mark ~|(bad-mark+mark !!) + %notes-action + :: ...dispatch into one of the local handlers below... + :: + %notes-command + :: ... + == + :: + ++ join-remote :: nested: only callable from inside +poke + |= =flag + ^+ cor + ... + :: + ++ handle-send-invite + |= [=flag who=ship] + ^+ cor + ... + -- +``` + +This is the same `|^` you use inside `on-load` to scope migration arms — same purpose, different host arm. Reach for it when arm-private helpers start cluttering the surrounding core's namespace. + ### Helper Arm Design: Args vs State Readers When a helper always operates on the current state values, don't pass them as arguments — just read from state. When a helper needs a value that may differ from current state, take it as an argument: