feat(architecture): add opt-in agent and integration adapters

[tomasanchez]

Summary

• add an opt-in MCP primary adapter using the official Python SDK
• add a broker-neutral IntegrationMessageBus service-layer port
• add a transactional outbox with SQLAlchemy persistence and Alembic migration
• add Kafka as one optional secondary adapter, plus a standalone relay worker
• add a local single-node KRaft Kafka broker and optional pinned Kafbat UI observability profile
• document the architecture in ADR 0015 and ADR 0016 and extend the agent guidance

Why

Agent clients benefit from MCP tools and resources, but MCP should not become a default dependency or automatic HTTP route mirror. External event publication also needs a Cosmic Python boundary: application orchestration depends on IntegrationMessageBus, while Kafka remains replaceable by RabbitMQ, SQS, or another adapter.

The transactional outbox avoids a dual-write gap between relational persistence and external broker publication. The relay provides at-least-once delivery, with eventId documented as the consumer deduplication key.

Showcase

User registration now demonstrates the full flow:

1. the aggregate raises internal UserRegistered
2. the SQLAlchemy unit of work persists the user and a versioned UserRegisteredV1 outbox row atomically
3. the relay publishes through the broker-neutral external message bus port
4. the optional Kafka adapter sends a camel-case envelope to users.events, keyed by userId
5. Kafbat UI can inspect topics, partitions, messages, keys, and consumer groups locally

The MCP addon demonstrates onboard_user as a tool and users://{user_id} as a resource backed by the existing command bus and CQRS reader.

Validation

• uv run pre-commit run --all-files
• make lint
• make cover (56 passed, 100% coverage)
• make adr-check
• uv lock --check
• git diff --check
• docker compose config --quiet
• docker compose --profile observability config --quiet
• live Kafka round trip through IntegrationMessageBus
• Kafbat UI startup and /actuator/health probe
• lean install verification: 52 passed, 2 skipped

[Copilot]

Pull request overview

Introduces two opt-in integration surfaces and the supporting outbox infrastructure to extend the Cosmic Python template without making MCP or Kafka required dependencies.

Changes:

• Add MCP primary adapter (tool + resource) and Kafka secondary adapter for an external IntegrationMessageBus port, both behind optional extras.
• Add a transactional outbox (outbox_events table, migration, staging in SqlAlchemyUnitOfWork, broker-neutral relay use case) with a versioned UserRegisteredV1 integration event.
• Document the design in ADR 0015 (MCP) and ADR 0016 (outbox), update AGENTS guidance, README, and Make/Compose targets.

Reviewed changes

Copilot reviewed 39 out of 40 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
uv.lock	Locks new transitive deps for the mcp and kafka extras.
pyproject.toml	Declares mcp and kafka optional-dependency groups and updates coverage omits.
Makefile	Adds mcp, kafka-up/relay/consume/ui-* targets; runs lint/coverage with both extras.
docker-compose.yaml	Adds local KRaft Kafka and optional Kafbat UI under observability profile.
README.md	Documents MCP/Kafka addons and KAFKA_* env vars.
AGENTS.md	Adds guidance for MCP adapter and outbox-based external publication.
docs/adr/0015-*.md	New ADR: MCP is an opt-in primary adapter.
docs/adr/0016-*.md	New ADR: external buses go through a transactional outbox.
docs/adr/README.md, docs/cosmic-python-coverage.md	Index and coverage matrix updates for the new ADRs.
migrations/env.py, migrations/versions/20260531_0002_*.py	Registers OutboxRecord for Alembic and creates outbox_events.
src/template/adapters/models/outbox.py	New SQLAlchemy mapping for outbox rows.
src/template/adapters/outbox.py	Stages integration events and provides relay UoW/repository.
src/template/adapters/unit_of_work.py	Stages integration events during commit with id-based deduplication.
src/template/integration_events/{__init__,user}.py	Versioned camel-case UserRegisteredV1 envelope.
src/template/service_layer/integration_messagebus.py	Broker-neutral publish port.
src/template/service_layer/outbox.py	Relay use case, abstract outbox UoW, OutboxMessage.
src/template/settings/kafka_settings.py	KAFKA_* settings for the relay/adapter.
src/template/addons/__init__.py, addons/mcp/*, addons/kafka/*	MCP application/server and Kafka adapter/worker behind optional imports.
tests/unit/..., tests/integration/...	New unit/integration tests for outbox, integration events, Kafka adapter, MCP application/server, and updates to persistence/migration tests.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.