Automate app-features.json sync from concurrentie-analyse

[rubenvdlinde]

Context

data/app-features.json was added in 0d8e8f4. It's generated by scripts/export_app_features.py in the private concurrentie-analyse (Specter) repo, which has direct access to the intelligence Postgres DB.

Today the sync is manual: someone with DB access runs

cd concurrentie-analyse
python scripts/export_app_features.py --output ../design-system/data/app-features.json
git add data/app-features.json && git commit && git push

That's fine for now — the data only changes when the team accepts new feature decisions in Specter, which isn't daily. But it should be automated before the website starts rendering features per app.

What's already automated for comparison

data/app-downloads.json runs on a daily cron via .github/workflows/app-downloads.yml. That works because the data source (GitHub releases + Nextcloud app-store API) is publicly readable from this repo's runners.

The features case is harder because the source is a private Postgres in concurrentie-analyse.

Proposed automation

Schedule the workflow inside concurrentie-analyse (which already has DB access secrets), have it generate the JSON, and push it to this repo via a deploy key or fine-grained PAT scoped to data/app-features.json. Sketch:

# in concurrentie-analyse/.github/workflows/export-features.yml
on:
  schedule: [cron: '0 7 * * 1']  # weekly, Monday 07:00 UTC
  workflow_dispatch:
jobs:
  export:
    runs-on: self-hosted  # needs DB network access
    steps:
      - run: python scripts/export_app_features.py --output /tmp/app-features.json
      - uses: actions/checkout@v4
        with:
          repository: ConductionNL/design-system
          token: \${{ secrets.DESIGN_SYSTEM_PAT }}
          path: design-system
      - run: cp /tmp/app-features.json design-system/data/app-features.json
      - run: |
          cd design-system
          git config user.name "specter-bot"
          git config user.email "ops@conduction.nl"
          git add data/app-features.json
          git diff --cached --quiet || (git commit -m "chore: refresh app-features.json" && git push)

Tasks

• Provision a fine-grained PAT scoped to Contents: write on ConductionNL/design-system, store as DESIGN_SYSTEM_PAT in concurrentie-analyse secrets
• Add the workflow above to concurrentie-analyse/.github/workflows/
• Decide on cadence (weekly Monday looks right; daily is overkill for accepted-feature changes)
• Document the contract in data/README.md (what the JSON contains, what's filtered out, who regenerates it)

Filtering policy (already enforced in the script)

For reviewers checking the data is safe to ship publicly:

• Dropped entirely: demand_score, tender_mentions, competitor_coverage, external_mentions, scientific_citations, standards_compliance, priority, description (auto-generated, leaks competitor names + cluster counts)
• Kept: app_slug, slug, name, category, capability (only when a CAPABILITY: section exists in capability_text, plus a defence-in-depth marker check)
• Source: app_feature_decisions JOIN canonical_features WHERE status='accepted' — the curated subset, not the raw 28k-row signal pool

Related

• Apps page now consumes app-downloads.json (a8d97d8)
• Features JSON shipped: 0d8e8f4 (396 features across 12 apps)