Skip to content

Add automated database schema diagram generation #187

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Copilot wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Add automated database schema diagram generation #187

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
6627ffe
Initial plan
Copilot Feb 7, 2026
503149d
Add GitHub Actions workflow for schema diagram generation
Copilot Feb 7, 2026
4afe8c2
Fix yamllint issues and update documentation
Copilot Feb 7, 2026
52b7dc3
Remove unused imports from database creation script
Copilot Feb 7, 2026
cd62a6e
Address PR feedback: remove auto-commit, reuse install script, use py…
Copilot Feb 7, 2026
cb03740
Fix SchemaCrawler-Action reference to use tag instead of commit SHA
Copilot Feb 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/scripts/create-sample-db.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
#!/bin/bash
# Create a sample SQLite database from the hadiscover models for schema diagram generation

set -e

echo "Creating sample database from models..."

# Get the script directory and project root
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
BACKEND_DIR="$PROJECT_ROOT/backend"

# Install backend dependencies using the shared script
echo "Installing backend dependencies..."
cd "$PROJECT_ROOT"
.github/scripts/install-backend-deps.sh

Comment on lines 12 to 17
Copy link

Copilot AI Feb 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This script creates/activates a backend/venv and installs dependencies on every run. In CI this adds time and duplicates the existing .github/scripts/install-backend-deps.sh pattern used elsewhere. Consider reusing the existing dependency-install script (and/or avoid creating a venv inside the repo) to keep workflows consistent and faster.

Suggested change
# Change to backend directory
cd "$BACKEND_DIR"
# Activate virtual environment if it exists, otherwise create it
if [ ! -d "venv" ]; then
echo "Creating virtual environment..."
python3 -m venv venv
fi
source venv/bin/activate
# Install dependencies
echo "Installing dependencies..."
pip install -q -r requirements.txt
INSTALL_SCRIPT="$PROJECT_ROOT/.github/scripts/install-backend-deps.sh"
# Install backend dependencies using the shared script when available
if [ -x "$INSTALL_SCRIPT" ]; then
echo "Using shared backend dependency install script..."
(cd "$PROJECT_ROOT" && "$INSTALL_SCRIPT")
# Activate virtual environment if it was created by the install script
cd "$BACKEND_DIR"
if [ -d "venv" ]; then
# shellcheck disable=SC1091
source venv/bin/activate
fi
else
echo "Shared install script not found; falling back to local venv setup..."
cd "$BACKEND_DIR"
# Activate virtual environment if it exists, otherwise create it
if [ ! -d "venv" ]; then
echo "Creating virtual environment..."
python3 -m venv venv
fi
# shellcheck disable=SC1091
source venv/bin/activate
# Install dependencies (only when using the local fallback)
echo "Installing dependencies..."
pip install -q -r requirements.txt
fi

Copilot uses AI. Check for mistakes.
Copy link
Contributor Author

Copilot AI Feb 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to reuse the existing .github/scripts/install-backend-deps.sh script instead of duplicating the dependency installation logic. The script now calls the shared install script which is consistent with other workflows. (commit cd62a6e)

# Create a temporary Python script to generate the database
cat > /tmp/create_db.py << 'EOF'
"""Generate sample SQLite database from SQLAlchemy models."""
from app.models.database import Base
from sqlalchemy import create_engine

# Create engine and database
Comment on lines +18 to +24
Copy link

Copilot AI Feb 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This script writes to a predictable path /tmp/create_db.py. Using a fixed temp filename can fail if the file already exists and is generally unsafe; prefer mktemp (and a trap cleanup) to create a unique temp file and ensure it’s removed even on errors.

Copilot uses AI. Check for mistakes.
engine = create_engine('sqlite:///sample_schema.db')
Base.metadata.create_all(engine)

print("Sample database created successfully at backend/sample_schema.db")
Comment on lines +21 to +28
Copy link

Copilot AI Feb 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If backend/sample_schema.db already exists, Base.metadata.create_all() won’t remove/modify existing tables/columns, which can produce a stale schema diagram on reruns. Remove any existing DB file before creating it (or explicitly create it at a fresh path) so the diagram always reflects the current models.

Suggested change
from app.models.database import Base
from sqlalchemy import create_engine
# Create engine and database
engine = create_engine('sqlite:///sample_schema.db')
Base.metadata.create_all(engine)
print("Sample database created successfully at backend/sample_schema.db")
from pathlib import Path
from sqlalchemy import create_engine
from app.models.database import Base
# Determine database path in current working directory (backend/)
db_path = Path("sample_schema.db")
# Remove existing database file to avoid stale schemas
if db_path.exists():
db_path.unlink()
# Create engine and database
engine = create_engine(f"sqlite:///{db_path}")
Base.metadata.create_all(engine)
print(f"Sample database created successfully at {db_path.resolve()}")

Copilot uses AI. Check for mistakes.
EOF

# Set PYTHONPATH and run the script
cd "$BACKEND_DIR"
export PYTHONPATH="$BACKEND_DIR:$PYTHONPATH"
python /tmp/create_db.py

# Clean up
rm /tmp/create_db.py

echo "Database creation complete."
61 changes: 61 additions & 0 deletions .github/workflows/schema-diagram.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
name: Generate Database Schema Diagram

# yamllint disable-line rule:truthy
on:
workflow_dispatch: null
Comment on lines +4 to +6
Copy link

Copilot AI Feb 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The yamllint suppression comment is on its own line, but disable-line only applies to the line it’s on. In this repo other workflows use on: # yamllint disable-line rule:truthy, so this file may fail YAML linting. Move the comment onto the on: line (or use an appropriate disable-next-line directive) to match the existing pattern.

Copilot uses AI. Check for mistakes.
push:
branches:
- main
paths:
- "backend/app/models/**"
- ".github/workflows/schema-diagram.yml"
- ".github/scripts/create-sample-db.sh"

permissions:
contents: read

jobs:
generate-schema-diagram:
name: Generate Schema Diagram
runs-on: ubuntu-latest

steps:
- name: Checkout code
# v6
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
with:
persist-credentials: false

- name: Setup Python
# v6
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405
with:
python-version: "3.14"
cache: "pip"
cache-dependency-path: backend/requirements.txt

- name: Create sample database
run: .github/scripts/create-sample-db.sh

- name: Generate schema diagram
# v17.6.0
uses: schemacrawler/SchemaCrawler-Action@v17.6.0
with:
entrypoint: /schemacrawler.sh
args: >-
--server=sqlite
--database=backend/sample_schema.db
--info-level=maximum
--command=schema
--output-format=png
--output-file=schema-diagram.png

- name: Upload schema diagram
# v4
uses: >-
actions/upload-artifact@c0d66db0b43eb92e5f6e4fc5f8f67c6d8c1e2d10
with:
name: database-schema-diagram
path: schema-diagram.png
retention-days: 90
2 changes: 2 additions & 0 deletions ARCHITECTURE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,8 @@ SQLite Database ←→ GitHub API

## Database Schema

A visual schema diagram is automatically generated on each push to main when models change. Download it from the workflow artifacts at [Actions > Generate Database Schema Diagram](../../actions/workflows/schema-diagram.yml).

### repositories

```sql
Expand Down