Skip to content

Lifecycle simulation test suite#259

Merged
CalebisGross merged 5 commits intomainfrom
feat/extract-stub-llm
Mar 20, 2026
Merged

Lifecycle simulation test suite#259
CalebisGross merged 5 commits intomainfrom
feat/extract-stub-llm

Conversation

@CalebisGross
Copy link
Copy Markdown
Collaborator

@CalebisGross CalebisGross commented Mar 20, 2026

Summary

  • Extract shared stub LLM from cmd/benchmark-quality/ to internal/testutil/stubllm/ for reuse
  • Add cmd/lifecycle-test/ — standalone Go binary simulating the full mnemonic user journey from fresh install through 3 months of usage

Lifecycle Test: 8 Phases

All 8 phases pass with 23/23 assertions green (~9s total runtime with stub LLM).

Phase Simulates Assertions
install Day 0: DB init 5
first-use Day 1: first memories 7
ingest Day 2: project ingestion 2
daily Days 3-14: mixed usage 3
consolidation Week 2: decay cycles 1
dreaming Week 3-4: dream + abstraction 0 (metrics)
growth Month 1-3: scale to 500+ 3
longterm Month 3: aggressive decay 2

Test plan

  • make check, make build, make test all pass
  • golangci-lint run — 0 issues
  • make lifecycle-test — 23/23 assertions pass

Closes #249, #250, #251, #252, #253, #254, #255, #256

CalebisGross and others added 3 commits March 20, 2026 10:46
Move the deterministic bag-of-words LLM provider from
cmd/benchmark-quality/stubllm.go to a shared package so it can be
reused by the upcoming lifecycle test suite. The benchmark-quality
binary now imports from the shared package via thin re-exports.

No behavior change — same vocabulary, embeddings, and response
templates.

Closes #249

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add cmd/lifecycle-test/, a standalone Go binary that simulates the full
mnemonic user journey from fresh install through 3 months of usage.

8 phases exercise all cognitive agents against a real SQLite database
with a deterministic stub LLM:

1. Install — DB init, schema verification
2. First Use — 10 seed memories, encoding, episoding, retrieval
3. Ingest — synthetic project ingestion with dedup verification
4. Daily Use — 250+ memories over 12 simulated days
5. Consolidation — 10 decay cycles, signal/noise separation
6. Dreaming — dream replay, abstraction, metacognition audit
7. Growth — scale to 500+ memories over 3 months
8. Long-term Health — aggressive decay, archival verification

Includes terminal + markdown report output and `make lifecycle-test`
Makefile target.

Closes #250, closes #251, closes #252, closes #253, closes #254,
closes #255, closes #256

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@CalebisGross CalebisGross changed the title Extract shared stub LLM to internal/testutil/stubllm Lifecycle simulation test suite Mar 20, 2026
CalebisGross and others added 2 commits March 20, 2026 11:48
- Grow from 30 to 80+ unique memory templates across 3 projects
  (mnemonic, felix-lm, sample-project)
- Split templates into mcpSignalTemplates and noiseTemplates for
  clearer source distribution control
- Append day+index suffix to each generated memory to avoid
  encoding dedup on repeated templates
- Add more filesystem, terminal, and clipboard noise variety

Closes #257

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Document the lifecycle simulation test suite, make lifecycle-test
target, and internal/testutil/ shared package in the project guide.

Closes #258

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@CalebisGross CalebisGross merged commit e0950e3 into main Mar 20, 2026
6 checks passed
@CalebisGross CalebisGross deleted the feat/extract-stub-llm branch March 20, 2026 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Extract shared stub LLM from benchmark-quality

1 participant