feat: Confluence ingestion — read from Confluence spaces without manual export

[rajnavakoti]

Description

Enterprise teams store domain knowledge in Confluence. When they try to use AI agents, the agents can't read Confluence effectively — it was written for humans, not machines. Teams end up duplicating docs into markdown, creating a single-source-of-truth problem.

Context Blocks should ingest directly from Confluence spaces, run DDC extraction, and produce typed entities — no manual export or duplication needed. This positions CB as a layer above Confluence, not a replacement.

Nobody in the OSS ecosystem has solved Confluence → typed entity extraction. Existing tools only do Confluence → flat markdown chunks. They lose diagram semantics, entity typing, quality classification, and gap detection.

Phased approach

Phase 1 — Document the export workflow (no code)

• Document using confluence2md or confluence-markdown-exporter as a pre-step, then cb extract as normal
• Add to README as a supported ingestion workflow
• Immediate value, zero engineering effort

Phase 2 — MCP connector (recommended primary path)

• Atlassian's official remote MCP server went GA Feb 2026 — Claude can read/write Jira + Confluence via MCP
• Add cb ingest --source confluence-mcp --space <SPACE> that pulls pages via MCP protocol
• Runs pages through DDC extraction pipeline → typed entities
• No markdown export step, no doc duplication

Phase 3 — Diagram recovery (hard, novel, roadmap)

• draw.io diagrams embedded in Confluence are lost during export (become image references or are dropped)
• Extract draw.io XML via API → pass to LLM → get back Mermaid
• LLM-based XML → Mermaid conversion works well per community reports
• Genuinely novel capability — no existing tool does this

Existing tools to leverage (don't reinvent)

Tool	Language	Notes
confluence2md	Go CLI	Zero deps, embedded pandoc, "LLM/RAG-ready output"
confluence-markdown-exporter	Python	API-based, recursive child pages
confluence-rag-indexer	Python	pip install, pg_vector, skips "Meeting Notes"
Atlassian MCP server (official)	Remote	GA Feb 2026, read/write Jira + Confluence
cosmix/confluence-mcp	Community	Unofficial MCP server
Mermaid AI	SaaS	Screenshot → Mermaid via image recognition
FlowForge	Research	draw.io → Mermaid conversion

Acceptance Criteria

• Phase 1: README documents Confluence → CB workflow using existing export tools
• Phase 2: cb ingest --source confluence pulls pages from a Confluence space and extracts entities
• Phase 2: No markdown files are created on disk — pages are streamed through the extraction pipeline
• Phase 2: Page hierarchy (parent/child) is preserved as entity relationships
• Phase 3: draw.io diagrams embedded in pages are recovered as Mermaid in entity files

Out of Scope

• Writing back to Confluence
• Real-time sync / webhook-based updates
• Jira ingestion (separate issue, related)

[rajnavakoti]

Addendum — image extraction as default before diagram conversion:

For embedded images (draw.io, screenshots, architecture diagrams), the pragmatic default should be:

1. Extract image as-is → save as image file alongside the entity
2. Reference in entity markdown → ![description](./diagrams/filename.png)
3. Entity frontmatter carries the typed context — relationships, layer, type are all in the structured fields, not the diagram

This gives you:

• Humans see the diagram (image renders in any markdown viewer, Obsidian, the CB viewer)
• AI agents get the structured context from the entity's frontmatter + text, and multimodal models can also read the image directly
• Nothing is lost from the source

Mermaid conversion (Phase 3) becomes an upgrade path for when you want diagrams to be editable and text-searchable — not a prerequisite for useful extraction.

Order of operations:

• Phase 2.5: Extract images, save as files, reference in entities (default)
• Phase 3: Optionally convert to Mermaid for editability (upgrade)