From d61e61b7fc7f5a8e3b8a0c98297f81dda4107199 Mon Sep 17 00:00:00 2001 From: Kevin Hopper Date: Sun, 12 Apr 2026 16:23:23 -0500 Subject: [PATCH] =?UTF-8?q?F.12:=20Cross-app=20bridging=20=E2=80=94=20matr?= =?UTF-8?q?ix-bridges=20+=20crow-native=20crosspost?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Final PR in the Phase 2 rollout. Stacked on F.11 (identity attestation). Two mechanisms land together: **F.12.1 — matrix-bridges meta-bundle** (bundles/matrix-bridges/) Opt-in Matrix appservice bridges (mautrix-signal / telegram / whatsapp) as compose-profile-gated sidecars on the shared crow-federation network. Each bridge has its own legal profile called out in the consent text: Signal ToS prohibits bot relays (terminate risk), Telegram tolerates but throttles, WhatsApp may ban the linked phone (Meta actively detects and blocks multi-device relays). - manifest.json + docker-compose.yml — three profiles-gated services, scaled mem_limits (signal 512m, telegram 768m, whatsapp 2g), each generates /data/registration.yaml on first boot. - scripts/post-install.sh — orchestration pipeline: start enabled profiles → wait for registration YAMLs → docker cp into crow-dendrite:/etc/dendrite/appservices/ → patch dendrite.yaml's app_service_api.config_files (idempotent in-container grep+awk) → restart crow-dendrite (registrations read ONLY at startup; hot reload no-ops) → wait for health → print per-bridge pairing instructions. - skills/matrix-bridges.md — per-bridge legal + privacy caveats, hardware table, enable/disable workflow, pairing instructions, F.11 attestation integration, troubleshooting. - No MCP server; bridge state lives in Dendrite + the bridge bots. **F.12.2 — Crow-native cross-posting** (transforms + tools) - servers/gateway/crossposting/transforms.js — 6 pure-function pairs: writefreely→mastodon, gotosocial→mastodon, pixelfed→mastodon, funkwhale→mastodon, peertube→mastodon, blog→gotosocial. Each respects target char limits, emits 'via ' attribution footer, strips HTML → plaintext. - scripts/init-db.js — two tables: crosspost_rules (operator-visible opt-in) + crosspost_log (idempotency + audit, UNIQUE(idempotency_key, source_app, target_app), 7-day idempotency window). - servers/sharing/server.js — five new MCP tools: crow_list_crosspost_transforms — enumerate pairs crow_crosspost(source_app, source_post_id, source_post, target_app, idempotency_key, trigger?, delay_seconds?, confirm) — idempotency required. Queues with 60s delay on on_publish/on_tag; fires immediately on manual. Raises Crow notification with cancel link when delayed. Returns transformed_preview + log_id but does NOT publish directly — caller invokes target's _post tool. crow_crosspost_cancel(log_id) — idempotent. crow_crosspost_mark_published(log_id, target_post_id) — closes audit trail after target publish succeeds. crow_list_crossposts(status?, limit?) — recent entries. - skills/crow-crosspost.md — manual + rule-driven workflows, safety notes (no fake undo, DMs caveat, attribution), F.11 integration. Publish-time safety is the 60s delay + notification + operator cancel per the plan. Explicitly NOT a fake post-publish undo — delete- propagation across the fediverse is unreliable; every publish is permanent. Idempotency scope is per-Crow-instance by design. **Design notes** - crow_crosspost is the first MCP tool that doesn't execute its action directly — it produces the transformed payload + audit log, and publishing requires calling the target bundle's own post tool. This keeps transforms pure and the idempotency + delay + cancel layer above the publish layer. - Scheduler dispatcher that fires queued entries at scheduled_at is NOT shipped here — until F.12.3, tool callers check the log. Manual (trigger="manual", delay_seconds=0) cross-posts work today. **Integration with F.11** When source + target handles are attested via crow_identity_attest, cross-posts inherit the identity claim: a verifier fetching /.well-known/crow-identity.json sees both handles bound to the same crow_id. **Registry / discovery** - registry/add-ons.json — matrix-bridges entry before developer-kit. - skills/superpowers.md — two trigger rows (matrix bridges + crosspost), EN+ES. - CLAUDE.md — crosspost_rules + crosspost_log schema docs, Skills Reference entries for crow-crosspost.md + matrix-bridges.md. **Verified** - node --check on all new/modified JS files - bash -n on bundles/matrix-bridges/scripts/post-install.sh - node scripts/init-db.js runs cleanly; both new tables land - createSharingServer() boots with all 5 new tools registered - Transform round-trip exercised: 6 pairs + unknown-pair error - docker compose config parses with required env set - registry JSON validates - npm run check passes **Phase 2 complete — 11 PRs shipped:** F.0 Caddy helpers + hardware gate + rate limiter + storage-translators F.1 GoToSocial, F.2 WriteFreely, F.3 Matrix-Dendrite, F.4 Funkwhale, F.5 Pixelfed, F.6 Lemmy, F.7 Mastodon, F.8 PeerTube, F.11 identity attestation, F.12 cross-app bridging (this PR). Outstanding follow-ups (separate PRs): - Scheduler dispatcher for queued crossposts (fires at scheduled_at) - GC sweeper for crosspost_log (>30 days) - Nest panel for moderation_actions + crosspost queue - Mastodon-API helper hoist (trigger: 4th Mastodon-compat bundle) --- CLAUDE.md | 4 + bundles/matrix-bridges/docker-compose.yml | 90 ++++++ bundles/matrix-bridges/manifest.json | 39 +++ bundles/matrix-bridges/package.json | 7 + .../matrix-bridges/scripts/post-install.sh | 153 ++++++++++ .../matrix-bridges/skills/matrix-bridges.md | 118 ++++++++ registry/add-ons.json | 35 +++ scripts/init-db.js | 43 +++ servers/gateway/crossposting/transforms.js | 135 +++++++++ servers/sharing/server.js | 264 ++++++++++++++++++ skills/crow-crosspost.md | 126 +++++++++ skills/superpowers.md | 2 + 12 files changed, 1016 insertions(+) create mode 100644 bundles/matrix-bridges/docker-compose.yml create mode 100644 bundles/matrix-bridges/manifest.json create mode 100644 bundles/matrix-bridges/package.json create mode 100755 bundles/matrix-bridges/scripts/post-install.sh create mode 100644 bundles/matrix-bridges/skills/matrix-bridges.md create mode 100644 servers/gateway/crossposting/transforms.js create mode 100644 skills/crow-crosspost.md diff --git a/CLAUDE.md b/CLAUDE.md index 209bb88..0881dd8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -204,6 +204,8 @@ Uses `@libsql/client` for local SQLite files (default: `~/.crow/data/crow.db`, g - **moderation_actions** — F.11 queued destructive moderation actions from federated bundles (bundle_id, action_type, payload_json, requested_at, expires_at, status, idempotency_key UNIQUE). 72h default TTL; operator confirms via Nest panel - **identity_attestations** — F.11 signed bindings (crow_id, app, external_handle, app_pubkey?, sig, version, revoked_at). Published via gateway `/.well-known/crow-identity.json`. UNIQUE(crow_id, app, external_handle, version) — new version row per rotation - **identity_attestation_revocations** — F.11 signed revocations (attestation_id FK CASCADE, revoked_at, reason, sig). Published via `/.well-known/crow-identity-revocations.json` +- **crosspost_rules** — F.12.2 opt-in crosspost config (source_app, source_trigger, target_app, transform, active). Triggers: `on_publish`, `on_tag:`, `manual` +- **crosspost_log** — F.12.2 audit + idempotency log (idempotency_key, source_app, source_post_id, target_app, status, target_post_id, scheduled_at, published_at, cancelled_at). UNIQUE(idempotency_key, source_app, target_app). 7-day idempotency window; >30 days GC'd daily - **iptv_playlists** — IPTV M3U playlist sources (name, url, auto_refresh, channel_count) - **iptv_channels** — IPTV channels from playlists (playlist_id FK, name, stream_url, tvg_id, group_title, is_favorite) - **iptv_epg** — Electronic Program Guide entries (channel_tvg_id, title, start_time, end_time, indexed) @@ -440,6 +442,7 @@ Consult `skills/superpowers.md` first — it routes user intent to the right ski - `developer-kit.md` — Developer kit: scaffold, test, and submit Crow extensions to the registry - `network-setup.md` — Tailscale remote access guidance - `crow-identity.md` — F.11 identity attestations: sign per-app handles (Mastodon/Funkwhale/Matrix/etc.) with the Crow root Ed25519 key, publish via `/.well-known/crow-identity.json`, verify + revoke. Off by default; opt-in per handle — public linkage is effectively permanent +- `crow-crosspost.md` — F.12.2 cross-app publishing: mirror a post from one federated bundle to another via pure-function transforms (writefreely→mastodon, peertube→mastodon, pixelfed→mastodon, funkwhale→mastodon, gotosocial→mastodon, blog→gotosocial). Idempotency_key required; 60s publish-delay safety valve with operator cancel; no fake undo-after-publish - `add-ons.md` — Add-on browsing, installation, removal - `scheduling.md` — Scheduled and recurring task management - `tutoring.md` — Socratic tutoring with progress tracking @@ -469,6 +472,7 @@ Add-on skills (activated when corresponding add-on is installed): - `gotosocial.md` — GoToSocial ActivityPub microblog: post, follow, search, moderate (block_user/mute inline; defederate/block_domain/import_blocklist queued for operator confirmation), media prune, federation health - `writefreely.md` — WriteFreely federated blog: create/update/publish/unpublish posts, list collections, fetch public posts, export; minimalist publisher (no comments, no moderation queue — WF is publish-oriented only) - `matrix-dendrite.md` — Matrix homeserver on Dendrite: create/join/leave rooms, send messages, sync, invite users, federation health; appservice registration prep for F.12 bridges; :8448-vs-well-known either/or federation story +- `matrix-bridges.md` — F.12.1 Matrix appservice bridges meta-bundle (mautrix-signal/telegram/whatsapp). Opt-in per bridge; each has distinct legal/privacy risks (Signal ToS prohibits bots; Meta may ban bridged WhatsApp numbers). post-install.sh writes appservice YAMLs into Dendrite + restarts it. Requires matrix-dendrite bundle - `funkwhale.md` — Funkwhale federated music pod: library listing, search, upload, follow remote channels/libraries, playlists, listening history, moderation (block_user/mute inline; block_domain/defederate queued), media prune; on-disk or S3 audio storage via storage-translators.funkwhale() - `pixelfed.md` — Pixelfed federated photo-sharing: post photos (upload+status), feed, search, follow, moderation (block_user/mute inline; block_domain/defederate/import_blocklist queued), admin reports, remote reporting, media prune; Mastodon-compatible REST API; on-disk or S3 media via storage-translators.pixelfed() - `lemmy.md` — Lemmy federated link aggregator: status, list/follow/unfollow communities, post (link + body), comment, feed (Subscribed/Local/All), search, moderation (block_user/block_community inline; block_instance/defederate queued), admin reports, pict-rs media prune; Lemmy v3 REST API; community-scoped federation diff --git a/bundles/matrix-bridges/docker-compose.yml b/bundles/matrix-bridges/docker-compose.yml new file mode 100644 index 0000000..28ae99d --- /dev/null +++ b/bundles/matrix-bridges/docker-compose.yml @@ -0,0 +1,90 @@ +# Matrix Bridges — Signal + Telegram + WhatsApp appservice sidecars. +# +# Each bridge is a `profiles`-gated service; the post-install.sh script +# enables the relevant profile based on BRIDGE_*_ENABLED env flags. +# All bridges join the crow-federation network so Dendrite can reach them +# over the internal docker DNS (appservice URL = http://:). +# +# Data: +# ~/.crow/matrix-bridges/signal/ mautrix-signal sqlite + config +# ~/.crow/matrix-bridges/telegram/ mautrix-telegram sqlite + config +# ~/.crow/matrix-bridges/whatsapp/ mautrix-whatsapp sqlite + config +# ~/.crow/matrix-bridges/signald/ signald state (mautrix-signal sidecar) +# +# Images: dock.mau.dev official mautrix tags. Pinned in each service. + +networks: + crow-federation: + external: true + default: + +services: + mautrix-signal: + image: dock.mau.dev/mautrix/signal:latest + container_name: crow-mautrix-signal + profiles: ["signal"] + networks: + - default + - crow-federation + environment: + MAUTRIX_HOMESERVER_ADDRESS: ${MATRIX_BRIDGE_HOMESERVER_URL} + MAUTRIX_HOMESERVER_DOMAIN: ${MATRIX_BRIDGE_DOMAIN} + volumes: + - ${MATRIX_BRIDGES_DATA_DIR:-~/.crow/matrix-bridges}/signal:/data + init: true + mem_limit: 512m + restart: unless-stopped + # Bridge generates /data/registration.yaml on first boot; post-install + # copies it into crow-dendrite:/etc/dendrite/appservices/signal.yaml + healthcheck: + test: ["CMD-SHELL", "test -f /data/registration.yaml"] + interval: 30s + timeout: 5s + retries: 20 + start_period: 30s + + mautrix-telegram: + image: dock.mau.dev/mautrix/telegram:latest + container_name: crow-mautrix-telegram + profiles: ["telegram"] + networks: + - default + - crow-federation + environment: + MAUTRIX_HOMESERVER_ADDRESS: ${MATRIX_BRIDGE_HOMESERVER_URL} + MAUTRIX_HOMESERVER_DOMAIN: ${MATRIX_BRIDGE_DOMAIN} + MAUTRIX_TELEGRAM_API_ID: ${BRIDGE_TELEGRAM_API_ID:-} + MAUTRIX_TELEGRAM_API_HASH: ${BRIDGE_TELEGRAM_API_HASH:-} + volumes: + - ${MATRIX_BRIDGES_DATA_DIR:-~/.crow/matrix-bridges}/telegram:/data + init: true + mem_limit: 768m + restart: unless-stopped + healthcheck: + test: ["CMD-SHELL", "test -f /data/registration.yaml"] + interval: 30s + timeout: 5s + retries: 20 + start_period: 30s + + mautrix-whatsapp: + image: dock.mau.dev/mautrix/whatsapp:latest + container_name: crow-mautrix-whatsapp + profiles: ["whatsapp"] + networks: + - default + - crow-federation + environment: + MAUTRIX_HOMESERVER_ADDRESS: ${MATRIX_BRIDGE_HOMESERVER_URL} + MAUTRIX_HOMESERVER_DOMAIN: ${MATRIX_BRIDGE_DOMAIN} + volumes: + - ${MATRIX_BRIDGES_DATA_DIR:-~/.crow/matrix-bridges}/whatsapp:/data + init: true + mem_limit: 2g + restart: unless-stopped + healthcheck: + test: ["CMD-SHELL", "test -f /data/registration.yaml"] + interval: 30s + timeout: 5s + retries: 20 + start_period: 30s diff --git a/bundles/matrix-bridges/manifest.json b/bundles/matrix-bridges/manifest.json new file mode 100644 index 0000000..87bd7c2 --- /dev/null +++ b/bundles/matrix-bridges/manifest.json @@ -0,0 +1,39 @@ +{ + "id": "matrix-bridges", + "name": "Matrix Bridges", + "version": "1.0.0", + "description": "Meta-bundle for Matrix appservice bridges (Signal, Telegram, WhatsApp, custom crow-matrix-bridge). Opt-in per bridge; each has its own consent text and hardware requirements. Installs appservice registrations into Dendrite and restarts the homeserver.", + "type": "bundle", + "author": "Crow", + "category": "federated-comms", + "tags": ["matrix", "bridges", "signal", "telegram", "whatsapp", "mautrix", "meta-bundle"], + "icon": "message-circle", + "docker": { "composefile": "docker-compose.yml" }, + "server": null, + "skills": ["skills/matrix-bridges.md"], + "consent_required": true, + "install_consent_messages": { + "en": "This meta-bundle installs Matrix appservice bridges that relay messages between Matrix (your Dendrite homeserver from F.3) and closed platforms (Signal, Telegram, WhatsApp). EACH BRIDGE HAS ITS OWN LEGAL + PRIVACY PROFILE. Signal's ToS explicitly prohibits automated relays and bot accounts — running mautrix-signal against a personal Signal number risks Signal terminating that number with no appeal. Telegram tolerates bot bridges but throttles aggressively on new accounts. WhatsApp actively blocks accounts that register as a 'multi-device' client from a non-WhatsApp app; mautrix-whatsapp expects the target phone to be opted-in and the owner accepts that Meta may ban the number. All bridges require double-puppeting (bridge bot posts AS you, not as a proxy), which means your Matrix account can inadvertently leak every Signal/Telegram/WhatsApp message to Dendrite admins + your homeserver's federation peers. Each bridge enabled here adds 500 MB-2 GB of RAM + 5-20 GB of disk (mautrix-whatsapp alone can spike 2 GB during media-heavy chats). Only enable bridges you actively need; disable bridges you stop using (stale bridges keep consuming resources AND keep the bridge bot in every room it joined). Dendrite must be RESTARTED after bridges are enabled — appservice registrations are read only at startup; hot-reload silently no-ops.", + "es": "Este meta-paquete instala puentes appservice de Matrix que retransmiten mensajes entre Matrix (tu homeserver Dendrite de F.3) y plataformas cerradas (Signal, Telegram, WhatsApp). CADA PUENTE TIENE SU PROPIO PERFIL LEGAL + DE PRIVACIDAD. Los ToS de Signal prohíben explícitamente los relays automatizados y cuentas bot — ejecutar mautrix-signal contra un número Signal personal arriesga que Signal termine ese número sin apelación. Telegram tolera puentes bot pero limita agresivamente cuentas nuevas. WhatsApp bloquea activamente cuentas que se registran como cliente 'multi-dispositivo' desde una app que no es WhatsApp; mautrix-whatsapp espera que el teléfono destino esté opt-in y el dueño acepta que Meta puede banear el número. Todos los puentes requieren doble-puppeting (el bot del puente publica COMO tú, no como proxy), lo que significa que tu cuenta de Matrix puede filtrar inadvertidamente todos los mensajes de Signal/Telegram/WhatsApp a admins de Dendrite + pares federados de tu homeserver. Cada puente habilitado añade 500 MB-2 GB de RAM + 5-20 GB de disco (mautrix-whatsapp solo puede subir a 2 GB en chats con muchos medios). Solo habilita los puentes que activamente necesitas; deshabilita los que dejas de usar (puentes obsoletos siguen consumiendo recursos Y mantienen al bot del puente en cada sala a la que se unió). Dendrite debe REINICIARSE después de habilitar puentes — los registros appservice se leen solo al inicio; un hot-reload silenciosamente no hace nada." + }, + "requires": { + "env": ["MATRIX_BRIDGE_HOMESERVER_URL", "MATRIX_BRIDGE_DOMAIN"], + "bundles": ["caddy", "matrix-dendrite"], + "min_ram_mb": 500, + "recommended_ram_mb": 2000, + "min_disk_mb": 5000, + "recommended_disk_mb": 30000 + }, + "env_vars": [ + { "name": "MATRIX_BRIDGE_HOMESERVER_URL", "description": "Internal URL of the Dendrite client-server API (http://dendrite:8008 over crow-federation).", "default": "http://dendrite:8008", "required": true }, + { "name": "MATRIX_BRIDGE_DOMAIN", "description": "MATRIX_SERVER_NAME from the matrix-dendrite bundle (appears in bridge bot MXIDs as @signalbot:).", "required": true }, + { "name": "BRIDGE_SIGNAL_ENABLED", "description": "Enable mautrix-signal. RISK: Signal ToS prohibits automated clients — number may be terminated.", "default": "false", "required": false }, + { "name": "BRIDGE_TELEGRAM_ENABLED", "description": "Enable mautrix-telegram. Requires a Telegram API ID + hash.", "default": "false", "required": false }, + { "name": "BRIDGE_TELEGRAM_API_ID", "description": "Telegram API ID from my.telegram.org/apps.", "required": false, "secret": true }, + { "name": "BRIDGE_TELEGRAM_API_HASH", "description": "Telegram API hash.", "required": false, "secret": true }, + { "name": "BRIDGE_WHATSAPP_ENABLED", "description": "Enable mautrix-whatsapp. RISK: WhatsApp may ban the linked phone number.", "default": "false", "required": false } + ], + "ports": [], + "webUI": null, + "notes": "Enabled bridges become sidecar containers. post-install.sh generates appservice registration YAMLs, calls matrix_register_appservice via docker exec, copies the YAMLs into /etc/dendrite/appservices/, and restarts the crow-dendrite container (registrations are read only at startup). To add/remove bridges: toggle BRIDGE_*_ENABLED in .env, run `crow bundle restart matrix-bridges`, then restart Dendrite manually." +} diff --git a/bundles/matrix-bridges/package.json b/bundles/matrix-bridges/package.json new file mode 100644 index 0000000..c64d704 --- /dev/null +++ b/bundles/matrix-bridges/package.json @@ -0,0 +1,7 @@ +{ + "name": "crow-matrix-bridges", + "version": "1.0.0", + "description": "Matrix appservice bridges meta-bundle (Signal/Telegram/WhatsApp). No MCP server — bridge state is managed via Dendrite's native tools and the bridge bots themselves.", + "type": "module", + "main": null +} diff --git a/bundles/matrix-bridges/scripts/post-install.sh b/bundles/matrix-bridges/scripts/post-install.sh new file mode 100755 index 0000000..c0324df --- /dev/null +++ b/bundles/matrix-bridges/scripts/post-install.sh @@ -0,0 +1,153 @@ +#!/usr/bin/env bash +# Matrix Bridges post-install. +# +# 1. For each BRIDGE_*_ENABLED=true flag, start the corresponding compose +# profile so the bridge container boots and generates its +# /data/registration.yaml. +# 2. Wait for registration.yaml to appear. +# 3. Copy the YAML into the crow-dendrite container's appservices dir. +# 4. Patch dendrite.yaml's app_service_api.config_files list to include +# the new YAML (idempotent — skip if already present). +# 5. Restart crow-dendrite (appservice registrations are read ONLY at +# startup; hot reload silently no-ops). +# 6. Print bridge-bot MXIDs and next steps (DM the bot to start pairing). + +set -euo pipefail + +BUNDLE_DIR="$(cd "$(dirname "$0")/.." && pwd)" +ENV_FILE="${BUNDLE_DIR}/.env" +if [ -f "$ENV_FILE" ]; then + set -a; . "$ENV_FILE"; set +a +fi + +COMPOSE="docker compose -f ${BUNDLE_DIR}/docker-compose.yml" + +# Map BRIDGE_*_ENABLED → (profile, bridge_id, registration_filename) +BRIDGE_RECORDS=() +if [ "${BRIDGE_SIGNAL_ENABLED:-false}" = "true" ]; then BRIDGE_RECORDS+=("signal:mautrix-signal:signal.yaml"); fi +if [ "${BRIDGE_TELEGRAM_ENABLED:-false}" = "true" ]; then BRIDGE_RECORDS+=("telegram:mautrix-telegram:telegram.yaml"); fi +if [ "${BRIDGE_WHATSAPP_ENABLED:-false}" = "true" ]; then BRIDGE_RECORDS+=("whatsapp:mautrix-whatsapp:whatsapp.yaml"); fi + +if [ ${#BRIDGE_RECORDS[@]} -eq 0 ]; then + cat </dev/null; then + echo " → ${full_container} registration.yaml ready" + break + fi + sleep 5 + done +done + +# Copy each registration.yaml into crow-dendrite +if ! docker ps --format '{{.Names}}' | grep -qw crow-dendrite; then + echo "ERROR: crow-dendrite is not running. Start matrix-dendrite bundle first." >&2 + exit 1 +fi + +docker exec crow-dendrite mkdir -p /etc/dendrite/appservices +for rec in "${BRIDGE_RECORDS[@]}"; do + IFS=':' read -r profile container yaml <<< "$rec" + full_container="crow-${container}" + TMP="$(mktemp)" + docker cp "$full_container:/data/registration.yaml" "$TMP" + docker cp "$TMP" "crow-dendrite:/etc/dendrite/appservices/$yaml" + rm -f "$TMP" + echo " → copied $yaml into crow-dendrite:/etc/dendrite/appservices/" +done + +# Patch dendrite.yaml to include the new appservice config_files (idempotent) +CFG=/etc/dendrite/dendrite.yaml +for rec in "${BRIDGE_RECORDS[@]}"; do + IFS=':' read -r profile container yaml <<< "$rec" + entry=" - appservices/$yaml" + if docker exec crow-dendrite grep -qF "appservices/$yaml" "$CFG" 2>/dev/null; then + echo " → dendrite.yaml already references appservices/$yaml" + continue + fi + # Add / update app_service_api.config_files block. Whole-file awk for safety. + docker exec crow-dendrite sh -c " + if ! grep -q 'app_service_api:' $CFG; then + printf '\napp_service_api:\n config_files:\n - appservices/$yaml\n' >> $CFG + elif ! grep -q 'config_files:' $CFG; then + awk '/app_service_api:/{print;print \" config_files:\";print \" - appservices/$yaml\";next}1' $CFG > $CFG.tmp && mv $CFG.tmp $CFG + else + awk -v ENT=\" - appservices/$yaml\" ' + {print} + /^[[:space:]]*config_files:/{print ENT} + ' $CFG > $CFG.tmp && mv $CFG.tmp $CFG + fi + " + echo " → patched dendrite.yaml with appservices/$yaml" +done + +# Restart Dendrite (appservice registrations are only read at startup) +echo "Restarting crow-dendrite (appservice registrations read at startup only)…" +DENDRITE_COMPOSE="${BUNDLE_DIR}/../matrix-dendrite/docker-compose.yml" +if [ -f "$DENDRITE_COMPOSE" ]; then + docker compose -f "$DENDRITE_COMPOSE" restart dendrite +else + docker restart crow-dendrite >/dev/null +fi + +# Wait for Dendrite to come back healthy +echo "Waiting for Dendrite to report healthy after restart (up to 60s)…" +for i in $(seq 1 12); do + if docker inspect crow-dendrite --format '{{.State.Health.Status}}' 2>/dev/null | grep -qw healthy; then + echo " → dendrite healthy" + break + fi + sleep 5 +done + +cat < + docker exec crow-dendrite cat /etc/dendrite/appservices/ + +EOF diff --git a/bundles/matrix-bridges/skills/matrix-bridges.md b/bundles/matrix-bridges/skills/matrix-bridges.md new file mode 100644 index 0000000..c27427f --- /dev/null +++ b/bundles/matrix-bridges/skills/matrix-bridges.md @@ -0,0 +1,118 @@ +--- +name: matrix-bridges +description: Matrix appservice bridges — Signal, Telegram, WhatsApp, custom crow-matrix-bridge. Opt-in per bridge. +triggers: + - "matrix bridge" + - "bridge signal" + - "bridge telegram" + - "bridge whatsapp" + - "mautrix" + - "signal to matrix" + - "telegram to matrix" +tools: [] +--- + +# Matrix Bridges — relay closed-platform chat into Matrix + +This meta-bundle installs Matrix appservice bridges from the mautrix project: Signal, Telegram, WhatsApp. Each bridge runs as a sidecar container on the shared `crow-federation` network; Dendrite (from F.3) registers each bridge's appservice YAML and routes events to/from it. + +This bundle is **opt-in per bridge.** Enable only the bridges you need. + +## Legal + privacy caveats + +Each bridge has a distinct legal profile — they are not equivalent. + +- **Signal** explicitly prohibits automated clients and bot relays in its Terms of Service. Running mautrix-signal against a personal Signal number risks Signal terminating that number **with no appeal.** Use only if you accept that risk or you're using a burner. +- **Telegram** tolerates bot bridges via its official API, but throttles aggressively on new accounts. Requires a Telegram API ID + hash from [my.telegram.org/apps](https://my.telegram.org/apps). +- **WhatsApp** (mautrix-whatsapp) works by registering as a "multi-device companion" on your phone's WhatsApp account. **Meta may ban the linked phone number.** Assume the number is at risk; don't bridge a number you can't afford to lose. + +Beyond platform-ToS risk, bridges create a privacy surface: + +- **Double-puppeting** — the bridge bot impersonates YOU inside Matrix rooms so your messages appear authored by `@you:your-server`. This is usually what you want, but it means every federated Matrix server with a member in your room sees messages that originated on Signal/Telegram/WhatsApp. +- **Media relay** — bridges fetch media from the remote platform and re-upload it to Dendrite. Retention matches Dendrite's media cache. +- **History** — most bridges backfill room history by default. First-time bridging a large chat can produce hours of sync activity. + +If you don't want any of the above, don't install this bundle. + +## Hardware + +Each enabled bridge adds 500 MB - 2 GB of RAM and 5-20 GB of disk. + +| Bridge | RAM idle | RAM under load | Disk (1y of use) | +|----------|----------|----------------|-------------------| +| Signal | 256 MB | 512 MB | 2-5 GB | +| Telegram | 384 MB | 768 MB | 5-15 GB | +| WhatsApp | 512 MB | 1.5-2 GB | 10-20 GB | + +Media-heavy chats (WhatsApp group with family photos) push the upper bound. + +## Enable a bridge + +Edit the bundle's `.env`: + +``` +BRIDGE_SIGNAL_ENABLED=true +# or +BRIDGE_TELEGRAM_ENABLED=true +BRIDGE_TELEGRAM_API_ID= +BRIDGE_TELEGRAM_API_HASH= +# or +BRIDGE_WHATSAPP_ENABLED=true +``` + +Then `crow bundle restart matrix-bridges`. The post-install script: + +1. Starts the corresponding compose profile so the bridge container generates its `/data/registration.yaml`. +2. Copies each registration YAML into `crow-dendrite:/etc/dendrite/appservices/.yaml`. +3. Patches `dendrite.yaml` to include each registration (idempotent). +4. **Restarts Dendrite** — appservice registrations are read ONLY at startup. Hot reload silently no-ops. Mid-restart, the homeserver is unreachable for ~20 seconds. + +## Pair your accounts + +After the bridge bot is alive inside Dendrite, DM it from any Matrix client you're logged into as your personal user (Element is the reference client): + +- **Signal**: DM `@signalbot:` → send `login`. A QR code appears; scan it from Signal → Settings → Linked Devices. +- **Telegram**: DM `@telegrambot:` → send `login` → enter phone number → enter SMS code or 2FA password. +- **WhatsApp**: DM `@whatsappbot:` → send `login qr` → scan from WhatsApp → Settings → Linked Devices. + +Bridged rooms appear in your Matrix client automatically once pairing completes. + +## Disable / remove a bridge + +Bridges leave state behind even after disabling: + +1. Stop the bridge sidecar: toggle `BRIDGE_*_ENABLED=false` in `.env` and `crow bundle restart matrix-bridges`. +2. Log out the bridge bot from the remote platform (Signal → Linked Devices → Unlink; WhatsApp → Linked Devices → Log Out). +3. Optionally remove the appservice YAML from Dendrite: + ```bash + docker exec crow-dendrite rm /etc/dendrite/appservices/.yaml + # Remove the corresponding config_files entry from dendrite.yaml by hand + docker compose -f bundles/matrix-dendrite/docker-compose.yml restart dendrite + ``` +4. Bridged rooms in your Matrix client become inert (bridge bot appears offline, no new messages flow in either direction, but history is preserved). + +Leaving a bridge enabled but logged-out is NOT equivalent to removing it — the bridge process continues to run and consume resources. + +## F.11 identity attestation + +You can optionally attest that `@signalbot:your-domain` belongs to your Crow identity via `crow_identity_attest`. This is most useful if you have public bridged rooms where you want remote viewers to confirm the bridge is operated by you, not a spoofed actor on a homeserver with the same name. + +``` +crow_identity_attest { + "app": "matrix-dendrite", + "external_handle": "@signalbot:your-domain.example", + "confirm": "yes" +} +``` + +## Bridge updates + CVE awareness + +The mautrix project pushes fixes (including security) frequently. This bundle uses the floating `:latest` tag to stay current. `crow bundle restart matrix-bridges` pulls the newest image. Review mautrix changelogs before bridging against production accounts; breaking config changes sometimes require manual YAML edits under `~/.crow/matrix-bridges//config.yaml`. + +## Troubleshooting + +- **"Bridge bot not responding"** — check `docker logs crow-mautrix-`. Missing registration.yaml in Dendrite is the usual culprit; re-run `post-install.sh`. +- **"401 from homeserver"** — bridge's as_token doesn't match what Dendrite has. Copy the registration YAML again + restart Dendrite. +- **Paired but messages don't flow** — check the bridge's `.env`: `MAUTRIX_HOMESERVER_DOMAIN` must exactly match `MATRIX_SERVER_NAME` from the matrix-dendrite bundle. A mismatch silently drops every event. +- **Signal bot stuck "connecting…"** — signald (the Signal sidecar mautrix-signal talks to) sometimes wedges. `docker restart crow-mautrix-signal`. +- **WhatsApp pairing fails repeatedly** — Meta is almost certainly detecting and blocking. Try from a different account. This is expected and not a bundle bug. diff --git a/registry/add-ons.json b/registry/add-ons.json index 9d9a053..e0e9c78 100644 --- a/registry/add-ons.json +++ b/registry/add-ons.json @@ -3385,6 +3385,41 @@ "webUI": null, "notes": "Three containers (peertube + postgres + redis). chocobozzz/peertube:production-bookworm. Expose via caddy_add_federation_site { domain: PEERTUBE_WEBSERVER_HOSTNAME, upstream: 'peertube:9000', profile: 'activitypub-peertube' }. Admin password printed to logs on first boot — rotate IMMEDIATELY via `docker exec -it crow-peertube npm run reset-password -- -u root`." }, + { + "id": "matrix-bridges", + "name": "Matrix Bridges", + "description": "Meta-bundle for Matrix appservice bridges (Signal, Telegram, WhatsApp). Opt-in per bridge; each has its own legal + privacy profile. Requires matrix-dendrite.", + "type": "bundle", + "version": "1.0.0", + "author": "Crow", + "category": "federated-comms", + "tags": ["matrix", "bridges", "signal", "telegram", "whatsapp", "mautrix", "meta-bundle"], + "icon": "message-circle", + "docker": { "composefile": "docker-compose.yml" }, + "server": null, + "skills": ["skills/matrix-bridges.md"], + "consent_required": true, + "requires": { + "env": ["MATRIX_BRIDGE_HOMESERVER_URL", "MATRIX_BRIDGE_DOMAIN"], + "bundles": ["caddy", "matrix-dendrite"], + "min_ram_mb": 500, + "recommended_ram_mb": 2000, + "min_disk_mb": 5000, + "recommended_disk_mb": 30000 + }, + "env_vars": [ + { "name": "MATRIX_BRIDGE_HOMESERVER_URL", "description": "Internal URL of Dendrite (http://dendrite:8008).", "default": "http://dendrite:8008", "required": true }, + { "name": "MATRIX_BRIDGE_DOMAIN", "description": "MATRIX_SERVER_NAME from matrix-dendrite bundle.", "required": true }, + { "name": "BRIDGE_SIGNAL_ENABLED", "description": "Enable mautrix-signal. RISK: Signal ToS prohibits bots.", "default": "false", "required": false }, + { "name": "BRIDGE_TELEGRAM_ENABLED", "description": "Enable mautrix-telegram.", "default": "false", "required": false }, + { "name": "BRIDGE_TELEGRAM_API_ID", "description": "Telegram API ID from my.telegram.org/apps.", "required": false, "secret": true }, + { "name": "BRIDGE_TELEGRAM_API_HASH", "description": "Telegram API hash.", "required": false, "secret": true }, + { "name": "BRIDGE_WHATSAPP_ENABLED", "description": "Enable mautrix-whatsapp. RISK: Meta may ban the phone.", "default": "false", "required": false } + ], + "ports": [], + "webUI": null, + "notes": "Opt-in per bridge via BRIDGE_*_ENABLED env flags. post-install.sh copies each bridge's /data/registration.yaml into crow-dendrite:/etc/dendrite/appservices/ + patches dendrite.yaml + restarts crow-dendrite. Pair by DM'ing the bridge bot (@signalbot:yourdomain → 'login') from a Matrix client." + }, { "id": "developer-kit", "name": "Developer Kit", diff --git a/scripts/init-db.js b/scripts/init-db.js index 5a2c816..e1b405c 100644 --- a/scripts/init-db.js +++ b/scripts/init-db.js @@ -547,6 +547,49 @@ await initTable("identity_attestation_revocations table", ` await addColumnIfMissing("contacts", "external_handle", "TEXT"); await addColumnIfMissing("contacts", "external_source", "TEXT"); +// --- F.12: Crosspost rules + log --- +// crosspost_rules holds the operator's opt-in config: "when a new post appears +// in app X, publish a transformed copy to app Y". Triggers: on_publish (with +// 60s grace), on_tag:, manual. +// crosspost_log is the idempotency + audit log — duplicate idempotency keys +// within 7 days return the cached result; entries >30 days are GC'd daily. +await initTable("crosspost_rules table", ` + CREATE TABLE IF NOT EXISTS crosspost_rules ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + source_app TEXT NOT NULL, + source_trigger TEXT NOT NULL, + target_app TEXT NOT NULL, + transform TEXT, + active INTEGER DEFAULT 1, + created_at INTEGER NOT NULL, + updated_at INTEGER NOT NULL + ); + + CREATE INDEX IF NOT EXISTS idx_crosspost_rules_active ON crosspost_rules(active, source_app, source_trigger); +`); + +await initTable("crosspost_log table", ` + CREATE TABLE IF NOT EXISTS crosspost_log ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + idempotency_key TEXT NOT NULL, + source_app TEXT NOT NULL, + source_post_id TEXT NOT NULL, + target_app TEXT NOT NULL, + transform TEXT, + status TEXT NOT NULL DEFAULT 'queued', + target_post_id TEXT, + error TEXT, + scheduled_at INTEGER NOT NULL, + published_at INTEGER, + cancelled_at INTEGER, + created_at INTEGER NOT NULL + ); + + CREATE UNIQUE INDEX IF NOT EXISTS idx_crosspost_log_idem ON crosspost_log(idempotency_key, source_app, target_app); + CREATE INDEX IF NOT EXISTS idx_crosspost_log_scheduled ON crosspost_log(status, scheduled_at); + CREATE INDEX IF NOT EXISTS idx_crosspost_log_created ON crosspost_log(created_at DESC); +`); + // --- Per-Device Context Support --- // Existing installs have section_key UNIQUE constraint that blocks device overrides. // Migration: add device_id column, drop the old UNIQUE constraint, add partial indexes. diff --git a/servers/gateway/crossposting/transforms.js b/servers/gateway/crossposting/transforms.js new file mode 100644 index 0000000..d7f51c9 --- /dev/null +++ b/servers/gateway/crossposting/transforms.js @@ -0,0 +1,135 @@ +/** + * F.12.2: Cross-app transform library. + * + * One function per (source_app, target_app) pair. Each transform takes + * the source post's metadata + content and returns a target-app-shaped + * payload ready for publication. + * + * Transforms are PURE FUNCTIONS — no network I/O here. The crow_crosspost + * dispatcher calls the transform, then hands the result to the target + * bundle's own publish API. This lets transforms be unit-tested in + * isolation and keeps the retry/idempotency layer above them. + * + * Rules for adding a transform: + * 1. Pick a (source_app, target_app) pair where BOTH bundles speak the + * fediverse (don't transform into a closed platform). + * 2. Respect the target's limits: Mastodon's 500-char default, GoToSocial's + * 5000-char cap, etc. + * 3. Always include an attribution footer ("via ") so the + * source stays authoritative. Delete-propagation is unreliable; the + * footer lets viewers navigate to the canonical post. + * 4. Strip HTML → plaintext (or markdown) for the target unless the + * target explicitly supports the same HTML subset. + */ + +const TRANSFORMS = { + /** + * WriteFreely long-form → Mastodon toot. + * Summarize: title + excerpt + canonical URL. Assumes the target is + * Mastodon (500 chars default). + */ + "writefreely→mastodon": (post) => { + const body = String(post.content || "").replace(/<[^>]+>/g, " ").replace(/\s+/g, " ").trim(); + const title = (post.title || "").trim(); + const url = post.url || ""; + const head = title ? `📝 ${title}\n\n` : ""; + const budget = 480 - head.length - url.length - 8; + const excerpt = body.length > budget ? body.slice(0, Math.max(0, budget - 1)) + "…" : body; + const text = `${head}${excerpt}\n\n${url}`.trim(); + return { + status: text, + visibility: "public", + language: post.language || undefined, + }; + }, + + /** + * GoToSocial toot → Mastodon toot. Identity-friendly passthrough since + * both speak the same API; just adds "via " footer so the + * crosspost is visibly a mirror rather than a fresh status. + */ + "gotosocial→mastodon": (post) => { + const body = String(post.status || post.content || ""); + const url = post.url || ""; + const footer = url ? `\n\nvia ${url}` : ""; + const budget = 500 - footer.length; + const main = body.length > budget ? body.slice(0, budget - 1) + "…" : body; + return { + status: main + footer, + visibility: post.visibility || "public", + spoiler_text: post.spoiler_text || undefined, + }; + }, + + /** + * Pixelfed photo-post → Mastodon toot. + * Mastodon supports media_ids but crossposting images means re-uploading + * to the target — that's caller's responsibility. This transform emits + * the text status + leaves media_urls as a list for the caller to + * rehydrate as local uploads. + */ + "pixelfed→mastodon": (post) => { + const caption = (post.content_excerpt || post.content || "").replace(/<[^>]+>/g, " ").trim(); + const url = post.url || ""; + const footer = url ? `\n\n📷 ${url}` : ""; + const budget = 480 - footer.length; + const text = caption.length > budget ? caption.slice(0, budget - 1) + "…" : caption; + return { + status: text + footer, + visibility: post.visibility || "public", + sensitive: post.sensitive === true, + media_urls: (post.media_urls || []).slice(0, 4), + }; + }, + + /** + * Funkwhale track → Mastodon toot (link post with title + artist). + */ + "funkwhale→mastodon": (post) => { + const title = post.title || post.name || "Track"; + const artist = post.artist ? ` — ${post.artist}` : ""; + const album = post.album ? ` (${post.album})` : ""; + const url = post.url || ""; + const text = `🎵 ${title}${artist}${album}\n\n${url}`.slice(0, 500); + return { status: text, visibility: "public" }; + }, + + /** + * PeerTube video → Mastodon toot (link post with title + duration). + */ + "peertube→mastodon": (post) => { + const title = post.name || post.title || "Video"; + const duration = post.duration_seconds + ? ` (${Math.floor(post.duration_seconds / 60)}:${String(post.duration_seconds % 60).padStart(2, "0")})` + : ""; + const channel = post.channel ? ` — ${post.channel}` : ""; + const url = post.url || ""; + const text = `🎬 ${title}${duration}${channel}\n\n${url}`.slice(0, 500); + return { status: text, visibility: "public" }; + }, + + /** + * Blog post (crow-blog native) → GoToSocial. + */ + "blog→gotosocial": (post) => { + const title = (post.title || "").trim(); + const url = post.url || ""; + const excerpt = (post.excerpt || "").replace(/<[^>]+>/g, " ").trim(); + const head = title ? `📝 ${title}\n\n` : ""; + const footer = url ? `\n\n${url}` : ""; + const budget = 4900 - head.length - footer.length; + const body = excerpt.length > budget ? excerpt.slice(0, budget - 1) + "…" : excerpt; + return { status: head + body + footer, visibility: "public" }; + }, +}; + +export function transform(sourceApp, targetApp, post) { + const key = `${sourceApp}→${targetApp}`; + const fn = TRANSFORMS[key]; + if (!fn) { + throw new Error(`No transform registered for ${sourceApp} → ${targetApp}. Supported pairs: ${Object.keys(TRANSFORMS).join(", ")}`); + } + return fn(post); +} + +export const SUPPORTED_PAIRS = Object.freeze(Object.keys(TRANSFORMS)); diff --git a/servers/sharing/server.js b/servers/sharing/server.js index 762eb1d..5a969ee 100644 --- a/servers/sharing/server.js +++ b/servers/sharing/server.js @@ -45,6 +45,7 @@ import { verifyRevocation, SUPPORTED_APPS as ATTESTATION_APPS, } from "../shared/identity-attestation.js"; +import { transform as crosspostTransform, SUPPORTED_PAIRS as CROSSPOST_PAIRS } from "../gateway/crossposting/transforms.js"; import { PeerManager } from "./peer-manager.js"; import { SyncManager } from "./sync.js"; import { InstanceSyncManager } from "./instance-sync.js"; @@ -2426,5 +2427,268 @@ export function createSharingServer(dbPath, options = {}) { }, ); + // --- F.12.2: Crow-native cross-posting --- + + server.tool( + "crow_list_crosspost_transforms", + "List the available (source, target) transform pairs for crow_crosspost. Each pair is a pure function in servers/gateway/crossposting/transforms.js.", + {}, + async () => ({ + content: [{ + type: "text", + text: JSON.stringify({ pairs: CROSSPOST_PAIRS }, null, 2), + }], + }), + ); + + server.tool( + "crow_crosspost", + "Cross-post a status from one federated bundle to another via the shared transform library. Requires idempotency_key — duplicate keys within 7 days return the cached result. on_publish trigger queues with a 60-second delay + cancel notification (no fake undo-after-publish).", + { + source_app: z.string().min(1).max(50), + source_post_id: z.string().min(1).max(200).describe("The source app's native post id. Used for idempotency + audit."), + source_post: z.object({}).passthrough().describe("Source post shape — transforms pull fields from this object (title, content, url, media, etc.)."), + target_app: z.string().min(1).max(50), + idempotency_key: z.string().min(8).max(200).describe("Required. Typically sha256(source_app+source_post_id+target_app). Per-Crow-instance scope."), + trigger: z.enum(["manual", "on_publish", "on_tag"]).optional().describe("manual fires immediately; on_publish/on_tag enqueue with 60s delay."), + delay_seconds: z.number().int().min(0).max(86400).optional().describe("Override the default 60s delay. 0 = fire immediately (manual default)."), + confirm: z.literal("yes").describe("Cross-posts cannot be reliably retracted; confirm intent."), + }, + async ({ source_app, source_post_id, source_post, target_app, idempotency_key, trigger, delay_seconds }) => { + try { + // Validate the transform exists before creating a queue entry + let transformed; + try { + transformed = crosspostTransform(source_app, target_app, source_post); + } catch (err) { + return { content: [{ type: "text", text: `Error: ${err.message}` }] }; + } + + const db = createDbClient(); + try { + // Idempotency check (last 7 days) + const sevenDaysAgo = Math.floor(Date.now() / 1000) - 7 * 86400; + const existing = await db.execute({ + sql: `SELECT id, status, target_post_id, scheduled_at, published_at, cancelled_at + FROM crosspost_log + WHERE idempotency_key = ? AND source_app = ? AND target_app = ? + AND created_at >= ? + LIMIT 1`, + args: [idempotency_key, source_app, target_app, sevenDaysAgo], + }); + if (existing.rows.length > 0) { + const r = existing.rows[0]; + return { + content: [{ + type: "text", + text: JSON.stringify({ + status: "idempotent_hit", + log_id: Number(r.id), + prior_status: r.status, + target_post_id: r.target_post_id || null, + scheduled_at: Number(r.scheduled_at), + published_at: r.published_at ? Number(r.published_at) : null, + cancelled_at: r.cancelled_at ? Number(r.cancelled_at) : null, + note: "Duplicate idempotency_key within 7 days — returning cached entry without re-queuing.", + }, null, 2), + }], + }; + } + + const effectiveTrigger = trigger || "manual"; + const isImmediate = effectiveTrigger === "manual"; + const delay = delay_seconds != null ? delay_seconds : (isImmediate ? 0 : 60); + const now = Math.floor(Date.now() / 1000); + const scheduledAt = now + delay; + const status = delay > 0 ? "queued" : "ready"; + + const inserted = await db.execute({ + sql: `INSERT INTO crosspost_log + (idempotency_key, source_app, source_post_id, target_app, + transform, status, scheduled_at, created_at) + VALUES (?, ?, ?, ?, ?, ?, ?, ?) + RETURNING id`, + args: [ + idempotency_key, source_app, source_post_id, target_app, + `${source_app}→${target_app}`, status, scheduledAt, now, + ], + }); + const logId = Number(inserted.rows[0].id); + + if (delay > 0) { + try { + await createNotification(db, { + title: `About to cross-post to ${target_app}`, + body: `Source: ${source_app}#${source_post_id}. Firing in ${delay}s unless cancelled. Cancel via crow_crosspost_cancel({ log_id: ${logId} }).`, + type: "peer", + source: "crosspost", + priority: "medium", + action_url: `/dashboard/crosspost?log_id=${logId}`, + }); + } catch {} + } + + return { + content: [{ + type: "text", + text: JSON.stringify({ + log_id: logId, + status, + scheduled_at: scheduledAt, + delay_seconds: delay, + transform: `${source_app}→${target_app}`, + transformed_preview: transformed, + note: delay > 0 + ? `Queued with ${delay}s cancel window. Target bundle's publish tool must be invoked when scheduled_at arrives — this tool only produces the transformed payload + audit log entry, it does NOT publish directly.` + : "Ready to publish. Target bundle's publish tool must be invoked now — this tool only produces the transformed payload.", + }, null, 2), + }], + }; + } finally { + try { db.close(); } catch {} + } + } catch (err) { + return { content: [{ type: "text", text: `Error: ${err.message}` }] }; + } + }, + ); + + server.tool( + "crow_crosspost_cancel", + "Cancel a queued cross-post before its scheduled_at fires. Idempotent — cancelling an already-published entry returns the published target_post_id. Cancelling an already-cancelled entry is a no-op.", + { + log_id: z.number().int(), + }, + async ({ log_id }) => { + try { + const db = createDbClient(); + try { + const row = await db.execute({ + sql: "SELECT status, target_post_id, cancelled_at, published_at, scheduled_at FROM crosspost_log WHERE id = ?", + args: [log_id], + }); + if (row.rows.length === 0) { + return { content: [{ type: "text", text: "Error: crosspost not found." }] }; + } + const r = row.rows[0]; + if (r.status === "published") { + return { + content: [{ + type: "text", + text: JSON.stringify({ + status: "already_published", + target_post_id: r.target_post_id, + published_at: Number(r.published_at), + note: "Published cross-posts cannot be retracted via this tool — use the target bundle's delete verb + accept that delete-propagation is unreliable.", + }, null, 2), + }], + }; + } + if (r.cancelled_at) { + return { + content: [{ + type: "text", + text: JSON.stringify({ status: "already_cancelled", cancelled_at: Number(r.cancelled_at) }, null, 2), + }], + }; + } + const now = Math.floor(Date.now() / 1000); + await db.execute({ + sql: "UPDATE crosspost_log SET status = 'cancelled', cancelled_at = ? WHERE id = ?", + args: [now, log_id], + }); + return { content: [{ type: "text", text: JSON.stringify({ status: "cancelled", cancelled_at: now }, null, 2) }] }; + } finally { + try { db.close(); } catch {} + } + } catch (err) { + return { content: [{ type: "text", text: `Error: ${err.message}` }] }; + } + }, + ); + + server.tool( + "crow_crosspost_mark_published", + "Mark a queued cross-post as published (called by the target bundle's publish flow after the actual remote post is created). This tool ONLY updates the audit log — it does NOT perform the publication itself.", + { + log_id: z.number().int(), + target_post_id: z.string().min(1).max(200), + }, + async ({ log_id, target_post_id }) => { + try { + const db = createDbClient(); + try { + const now = Math.floor(Date.now() / 1000); + const res = await db.execute({ + sql: `UPDATE crosspost_log SET status = 'published', target_post_id = ?, published_at = ? + WHERE id = ? AND status != 'cancelled'`, + args: [target_post_id, now, log_id], + }); + if (res.rowsAffected === 0) { + return { content: [{ type: "text", text: "Error: log row not found or already cancelled." }] }; + } + return { content: [{ type: "text", text: JSON.stringify({ status: "published", target_post_id, published_at: now }, null, 2) }] }; + } finally { + try { db.close(); } catch {} + } + } catch (err) { + return { content: [{ type: "text", text: `Error: ${err.message}` }] }; + } + }, + ); + + server.tool( + "crow_list_crossposts", + "List recent cross-posts from the log with their status (queued/ready/published/cancelled/error).", + { + status: z.enum(["queued", "ready", "published", "cancelled", "error"]).optional(), + limit: z.number().int().min(1).max(200).optional(), + }, + async ({ status, limit }) => { + try { + const db = createDbClient(); + try { + const clauses = []; + const args = []; + if (status) { clauses.push("status = ?"); args.push(status); } + args.push(limit ?? 50); + const rows = await db.execute({ + sql: `SELECT id, source_app, source_post_id, target_app, transform, status, + target_post_id, scheduled_at, published_at, cancelled_at, error, created_at + FROM crosspost_log + ${clauses.length ? "WHERE " + clauses.join(" AND ") : ""} + ORDER BY created_at DESC LIMIT ?`, + args, + }); + return { + content: [{ + type: "text", + text: JSON.stringify({ + count: rows.rows.length, + crossposts: rows.rows.map(r => ({ + id: Number(r.id), + source_app: r.source_app, + source_post_id: r.source_post_id, + target_app: r.target_app, + transform: r.transform, + status: r.status, + target_post_id: r.target_post_id || null, + scheduled_at: Number(r.scheduled_at), + published_at: r.published_at ? Number(r.published_at) : null, + cancelled_at: r.cancelled_at ? Number(r.cancelled_at) : null, + error: r.error || null, + })), + }, null, 2), + }], + }; + } finally { + try { db.close(); } catch {} + } + } catch (err) { + return { content: [{ type: "text", text: `Error: ${err.message}` }] }; + } + }, + ); + return server; } diff --git a/skills/crow-crosspost.md b/skills/crow-crosspost.md new file mode 100644 index 0000000..eb5ad83 --- /dev/null +++ b/skills/crow-crosspost.md @@ -0,0 +1,126 @@ +--- +name: crow-crosspost +description: F.12 cross-app publishing — mirror a post from one federated bundle to another via pure-function transforms. +triggers: + - "cross-post" + - "crosspost" + - "mirror post to" + - "publish to both" + - "mastodon and" + - "share my post to" +tools: + - crow_crosspost + - crow_crosspost_cancel + - crow_crosspost_mark_published + - crow_list_crossposts + - crow_list_crosspost_transforms +--- + +# Crow Cross-Posting (F.12.2) + +Publish a post from one federated bundle (source) to another (target) via a pure-function transform from `servers/gateway/crossposting/transforms.js`. This skill is for **operator-initiated or rule-driven** cross-posts — NOT for automatic firehose mirrors. + +## Design invariants + +- **Idempotency is required.** Every call to `crow_crosspost` needs an `idempotency_key` (typically `sha256(source_app + source_post_id + target_app)`). Duplicate keys within 7 days return the cached result; they do not re-queue. +- **60-second publish delay by default.** When a rule fires on `on_publish` or `on_tag`, the cross-post is queued with a 60-second delay and a Crow notification surfaces ("About to cross-post to mastodon. Cancel?"). The operator can cancel before it fires. `trigger: "manual"` fires immediately (operator already chose). +- **No fake undo after publish.** Cross-posts cannot be reliably retracted. Delete activities propagate asynchronously and inconsistently across the fediverse. Treat every publish as permanent. +- **This tool only produces the transformed payload + audit log entry.** It does NOT invoke the target bundle's publish API directly — the caller (usually a rule dispatcher) must invoke the target's `_post` / `_post_photo` / `_upload_video` tool with the transformed payload, then call `crow_crosspost_mark_published` to close the log entry. + +## Available transforms + +``` +crow_list_crosspost_transforms {} +# → { pairs: ["writefreely→mastodon", "gotosocial→mastodon", "pixelfed→mastodon", +# "funkwhale→mastodon", "peertube→mastodon", "blog→gotosocial"] } +``` + +Transforms are pure functions — no network I/O. Each respects the target app's character limits, includes an attribution footer linking to the canonical source, and strips HTML → plaintext for targets that don't accept HTML. + +Adding a new transform: edit `servers/gateway/crossposting/transforms.js`, pick a `(source_app, target_app)` pair where BOTH speak the fediverse, follow the rules in the file's top comment. + +## Common workflows + +### Manual one-off cross-post + +``` +# 1. Compute the idempotency key +idem = sha256("writefreely:42:mastodon") + +# 2. Queue (fires immediately with trigger=manual) +crow_crosspost { + "source_app": "writefreely", + "source_post_id": "42", + "source_post": { + "title": "My latest post", + "content": "

Long-form body…

", + "url": "https://blog.example.com/my-latest-post" + }, + "target_app": "mastodon", + "idempotency_key": "", + "trigger": "manual", + "confirm": "yes" +} +# → { log_id, status: "ready", transformed_preview: { status: "📝 My latest post..." } } + +# 3. Publish the transformed payload via the target bundle's tool +mastodon_post { + "status": "", + "visibility": "public" +} +# → { id: "109xxx", url: "..." } + +# 4. Close the audit log entry +crow_crosspost_mark_published { + "log_id": , + "target_post_id": "109xxx" +} +``` + +### Rule-driven cross-post with delay + +``` +crow_crosspost { + "source_app": "pixelfed", + "source_post_id": "8b3f...", + "source_post": { "content_excerpt": "Sunset shot", "url": "https://photos.example.com/p/8b3f", "sensitive": false }, + "target_app": "mastodon", + "idempotency_key": "", + "trigger": "on_publish", + "confirm": "yes" +} +# → { log_id, status: "queued", scheduled_at: , delay_seconds: 60 } +# A Crow notification surfaces with the cancel link. +``` + +Operator can cancel before `scheduled_at`: +``` +crow_crosspost_cancel { "log_id": } +``` + +After `scheduled_at` passes, a future dispatcher (not yet shipped — lands in a follow-up) calls the target app's publish verb + `crow_crosspost_mark_published`. + +### List recent cross-posts + +``` +crow_list_crossposts { "status": "queued" } +# → Pending queue with countdown to scheduled_at + +crow_list_crossposts { "limit": 50 } +# → All recent cross-posts with status + target_post_id +``` + +## Integration with F.11 identity attestation + +If you've attested your handles on the source and target apps, cross-posts inherit the identity claim: a verifier fetching `.well-known/crow-identity.json` on your gateway sees both handles bound to the same crow_id. This makes cross-posts visibly yours rather than looking like spam from an unrelated account. + +## Safety notes + +- **Publish delay is the safety valve.** Don't set `delay_seconds: 0` on `on_publish` rules unless you're confident. The 60s default exists because LLMs making autonomous posts can make errors that are easier to catch pre-publish than retract post-publish. +- **Never cross-post DMs.** The `source_post.visibility` field is passed through but defaults to `public` — if the source post was `direct` or `private`, the operator should explicitly set the target's visibility to match. +- **Cross-posts are attribution-footered.** The transform always emits a `via ` line so viewers on the target can navigate back to the canonical post. If you strip the footer in a custom transform, viewers will see a bare mirror — and delete-propagation being unreliable, that mirror may outlive the source. +- **Idempotency scope is per-Crow-instance.** Two different Crows cross-posting the same source post will not dedupe each other — that's by design. + +## Log retention + +Entries >30 days are garbage-collected by the daily cleanup sweeper (not yet wired — manual `DELETE FROM crosspost_log WHERE created_at < strftime('%s', 'now', '-30 days')` until F.12.3 or similar). diff --git a/skills/superpowers.md b/skills/superpowers.md index 42f5d40..ff2000e 100644 --- a/skills/superpowers.md +++ b/skills/superpowers.md @@ -88,6 +88,8 @@ This is the master routing skill. Consult this **before every task** to determin | "mastodon", "toot", "fediverse", "activitypub", "@user@server", "federated timeline", "boost", "mastodon instance" | "mastodon", "tootear", "fediverso", "activitypub", "@usuario@servidor", "linea temporal federada", "reblog" | mastodon | crow-mastodon | | "peertube", "upload video", "federated video", "video channel", "fediverse video", "youtube alternative", "webtorrent" | "peertube", "subir video", "video federado", "canal video", "video fediverso", "alternativa youtube" | peertube | crow-peertube | | "attest identity", "prove I am", "link my mastodon", "verify handle", "keyoxide", "crow identity attestation", "revoke attestation" | "atestar identidad", "probar que soy", "vincular mi mastodon", "verificar handle", "keyoxide", "atestación identidad crow" | crow-identity | crow-sharing | +| "matrix bridge", "bridge signal", "bridge telegram", "bridge whatsapp", "mautrix", "signal to matrix", "telegram to matrix" | "puente matrix", "puente signal", "puente telegram", "puente whatsapp", "mautrix", "signal a matrix" | matrix-bridges | crow-matrix-dendrite | +| "cross-post", "crosspost", "mirror post to", "publish to both", "mastodon and", "share my post to" | "publicación cruzada", "crosspost", "replicar publicación", "publicar en ambos" | crow-crosspost | crow-sharing | | "tutor me", "teach me", "quiz me", "help me understand" | "enséñame", "explícame", "evalúame" | tutoring | crow-memory | | "wrap up", "summarize session", "what did we do" | "resumir sesión", "qué hicimos" | session-summary | crow-memory | | "change language", "speak in..." | "cambiar idioma", "háblame en..." | i18n | crow-memory |