Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/ai-autopilot-bootstrap-orchestrator.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@gemstack/ai-autopilot": minor
---

Add bootstrap mode's orchestrator core: the spine that sequences autopilot's primitives into scope → architect → build → full-fledged loop, taking a user from nothing to a running, production-grade app. `Bootstrap` owns the control flow (the loop, the gate, the interrupt) over four injectable steps, narrating each phase over the generic surface stream and recording the architect's choices to the decisions ledger — no permission asked. The full-fledged loop repeats the production-grade checklist with fresh context, improving against its `{ blockers }` verdict until it is empty or a `maxPasses` budget stops it; prototype scope skips it. Default step builders wire the steps onto the real primitives — `agentArchitect` (an `ai-sdk` agent + the decisions briefing), `supervisorBuild` (the `Supervisor` over personas + runner), and `loopChecklist` / `loopImprove` (the Loop) — so the same orchestrator runs against real agents in production or stubs + `FakeRunner` in a test. Verified end-to-end offline. Closes #122.
5 changes: 5 additions & 0 deletions .changeset/ai-autopilot-generic-surface.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@gemstack/ai-autopilot": minor
---

Generalize the surface stream to be event-type generic. `EventStream<E>`, `AutopilotHandle<E, R>`, and `launchAutopilot<E, R>` now take the event (and result) type as parameters, defaulting to the supervisor's `SupervisorEvent` / `SupervisorRun` so every existing supervisor surface is unchanged. This lets bootstrap (and any future surface) stream its own narration events and return its own result over the same replayable, detached transport. Closes #120.
5 changes: 5 additions & 0 deletions .changeset/ai-autopilot-production-grade-checklist.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@gemstack/ai-autopilot": minor
---

Add the `production-grade` checklist prompt and a `{ blockers }` verdict convention. The new built-in prompt judges an app against a production-grade checklist (auth, data layer, error handling, instrumentation, emailing, validation, tests, build/config) and ends with a machine-readable `{ "blockers": [...] }` verdict — an empty list means production-grade. `parseVerdict()` / `isPassing()` read it back, and the loop now gates on that outcome: a `PromptOutcome` carries `verdict` and a `passing` flag (executed *and* no blockers), and `continueOnError: false` stops the chain on `!passing`. Backward compatible — with no verdict reported, `passing === ok`; pass `verdict: null` for an execution-only gate. This is what bootstrap's full-fledged loop repeats against until blockers is empty. Closes #121.
45 changes: 45 additions & 0 deletions packages/ai-autopilot/prompts/production-grade.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
name: production-grade
description: Checklist gate — is this app production-grade? Returns a { blockers } verdict.
appliesTo: ["**/*"]
metadata:
title: Production-grade checklist
loopId: production-grade
passes: 1
---

You are the production-grade **gate** for a Vike + universal-orm app. Judge the
app *as it stands now* against the checklist below. You are not reviewing a diff;
you are deciding whether this is something you would put in front of real users.

Go through the app and check each area. For every item, either it is genuinely
handled or it is a **blocker** — do not give credit for a stub, a TODO, or a
console.log standing in for the real thing.

1. **Auth** — real sign-up / sign-in / sign-out, sessions scoped per user, routes
and data access gated. No hardcoded or bypassable auth.
2. **Data layer** — schema + migrations exist and run; every query is scoped to
its owner (no IDOR); writes return the data callers rely on.
3. **Error handling** — no unhandled rejections or naked `throw` reaching the
user; failures surface as proper responses, not a white screen or a 500 stack.
4. **Instrumentation** — logging and error reporting are wired so a failure in
production is observable, not silent.
5. **Emailing** — transactional mail (verification, password reset, receipts) is
sent through a real transport, not a stub, where the app's flows need it.
6. **Validation** — untrusted input is validated at the edge before it reaches
the ORM or filesystem.
7. **Tests** — the core flows have automated tests that actually run and pass.
8. **Build & config** — the app builds clean, secrets come from env (none
committed), and it starts with a documented command.

Weigh each blocker by whether it would actually bite a real user or operator;
skip theoretical nits. When you are unsure an item is truly done, treat it as a
blocker and say why.

End your response with **exactly one** fenced JSON block giving the verdict — the
list of concrete, still-required work. An empty list means the app is
production-grade.

```json
{ "blockers": ["short, actionable description of each remaining gap"] }
```
173 changes: 173 additions & 0 deletions packages/ai-autopilot/src/bootstrap/bootstrap.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
import { describe, it } from 'node:test'
import assert from 'node:assert/strict'
import { Bootstrap, createBootstrap, BootstrapAborted } from './bootstrap.js'
import { DecisionLedger } from '../decisions/ledger.js'
import type { BootstrapEvent, BootstrapSteps, ScopeAnswer } from './types.js'
import type { SupervisorRun } from '../types.js'
import type { Verdict } from '../loop/verdict.js'

const zeroUsage = { promptTokens: 0, completionTokens: 0, totalTokens: 0 }
const fakeRun: SupervisorRun = { text: 'built', plan: [], results: [], usage: zeroUsage, stoppedEarly: false }

/** Build a set of stub steps; override any of them per test. */
function stubSteps(over: Partial<BootstrapSteps> = {}): BootstrapSteps {
return {
scope: () => ({ scope: 'full', intent: 'a shop' }) satisfies ScopeAnswer,
architect: () => ({
stack: 'Vike + universal-orm',
narration: 'Server-rendered shop with a Postgres data layer',
decisions: [{ choice: 'Use Vike for SSR', why: 'SEO + fast first paint' }],
}),
build: () => fakeRun,
checklist: () => ({ blockers: [] }) satisfies Verdict,
...over,
}
}

describe('Bootstrap — happy path (full scope, passes first checklist)', () => {
it('sequences scope → architect → build → loop and returns a production-grade result', async () => {
const events: BootstrapEvent[] = []
const boot = new Bootstrap({ steps: stubSteps(), onEvent: e => events.push(e) })
const result = await boot.run()

assert.equal(result.scope, 'full')
assert.equal(result.intent, 'a shop')
assert.equal(result.plan.stack, 'Vike + universal-orm')
assert.equal(result.run, fakeRun)
assert.equal(result.passes, 1)
assert.deepEqual(result.blockers, [])
assert.equal(result.productionGrade, true)
assert.equal(result.stoppedEarly, false)

// narration order: scope, architect, its narration, build narration, loop, checklist, done
const types = events.map(e => e.type)
assert.deepEqual(types, ['scope', 'architect', 'narrate', 'narrate', 'narrate', 'checklist', 'done'])
const scoped = events[0]
assert.ok(scoped?.type === 'scope' && scoped.scope === 'full')
})

it('records the architect decisions to the ledger', async () => {
const ledger = new DecisionLedger()
const boot = new Bootstrap({ steps: stubSteps(), ledger })
await boot.run()
assert.equal(boot.decisions, ledger)
assert.equal(ledger.size, 1)
assert.equal(ledger.all()[0]?.status, 'accepted')
assert.match(ledger.all()[0]!.title, /Vike for SSR/)
})
})

describe('Bootstrap — full-fledged loop', () => {
it('improves against blockers and repeats until the checklist is clean', async () => {
const verdicts: Verdict[] = [{ blockers: ['no auth', 'no tests'] }, { blockers: [] }]
const improveCalls: readonly string[][] = []
let call = 0
const boot = new Bootstrap({
steps: stubSteps({
checklist: () => verdicts[call++]!,
improve: ({ blockers }) => {
;(improveCalls as string[][]).push([...blockers])
},
}),
})
const result = await boot.run()

assert.equal(result.passes, 2)
assert.deepEqual(result.blockers, [])
assert.equal(result.productionGrade, true)
assert.equal(result.stoppedEarly, false)
// improve ran once, against pass 1's blockers
assert.deepEqual(improveCalls, [['no auth', 'no tests']])
})

it('stops early at maxPasses with blockers still open', async () => {
const events: BootstrapEvent[] = []
let improves = 0
const boot = new Bootstrap({
maxPasses: 2,
steps: stubSteps({
checklist: () => ({ blockers: ['still no auth'] }),
improve: () => { improves++ },
}),
onEvent: e => events.push(e),
})
const result = await boot.run()

assert.equal(result.passes, 2)
assert.deepEqual(result.blockers, ['still no auth'])
assert.equal(result.productionGrade, false)
assert.equal(result.stoppedEarly, true)
// improve runs only between passes, so once for 2 passes (no improve after the last)
assert.equal(improves, 1)
assert.equal(events.filter(e => e.type === 'checklist').length, 2)
})
})

describe('Bootstrap — prototype scope', () => {
it('skips the full-fledged loop entirely', async () => {
const events: BootstrapEvent[] = []
let checklistRuns = 0
const boot = new Bootstrap({
steps: stubSteps({
scope: () => ({ scope: 'prototype', intent: 'quick demo' }),
checklist: () => { checklistRuns++; return { blockers: [] } },
}),
onEvent: e => events.push(e),
})
const result = await boot.run()

assert.equal(result.scope, 'prototype')
assert.equal(result.passes, 0)
assert.equal(result.productionGrade, false) // not gated, so not claimed
assert.equal(checklistRuns, 0)
assert.equal(events.some(e => e.type === 'checklist'), false)
})
})

describe('Bootstrap — interrupt + isolation', () => {
it('aborts between phases when the signal fires', async () => {
const controller = new AbortController()
const boot = new Bootstrap({
signal: controller.signal,
steps: stubSteps({
architect: () => {
controller.abort() // user interrupts during the architect phase
return { stack: 'x', narration: '', decisions: [] }
},
}),
})
await assert.rejects(() => boot.run(), BootstrapAborted)
})

it('does not start when already aborted', async () => {
const controller = new AbortController()
controller.abort()
let scopeRan = false
const boot = new Bootstrap({
signal: controller.signal,
steps: stubSteps({ scope: () => { scopeRan = true; return { scope: 'full', intent: 'x' } } }),
})
await assert.rejects(() => boot.run(), BootstrapAborted)
assert.equal(scopeRan, false)
})

it('isolates a throwing onEvent callback', async () => {
const boot = createBootstrap({
steps: stubSteps(),
onEvent: () => { throw new Error('observer bug') },
})
// run completes despite the observer throwing on every event
const result = await boot.run()
assert.equal(result.productionGrade, true)
})
})

describe('Bootstrap — construction', () => {
it('rejects missing steps and a bad maxPasses', () => {
// @ts-expect-error missing steps
assert.throws(() => new Bootstrap({}), /requires `steps`/)
// @ts-expect-error missing build
assert.throws(() => new Bootstrap({ steps: { scope: () => ({ scope: 'full', intent: '' }), architect: () => ({ stack: '', narration: '', decisions: [] }) } }), /`build` step/)
assert.throws(() => new Bootstrap({ steps: stubSteps(), maxPasses: 0 }), /maxPasses/)
})
})
Loading
Loading