Schemas are defined in code, not SQL. ORM models are generated dynamically. MySQL tables are created and seeded on demand. Docker/CI builds can bootstrap databases automatically on startup. Queries automatically adapt to MySQL vs. SQLite and dataset-specific differences.

.github/workflows/bar-api.yml

@@ -12,6 +12,7 @@ jobs:
 
     runs-on: ubuntu-24.04
     strategy:
+      fail-fast: false
       matrix:
         python-version: [3.10.18, 3.11, 3.12, 3.13]
 

.github/workflows/codeql.yml

@@ -29,16 +29,16 @@ jobs:
 
     # Initializes the CodeQL tools for scanning.
     - name: Initialize CodeQL
-      uses: github/codeql-action/init@v2
+      uses: github/codeql-action/init@v3
       with:
         languages: ${{ matrix.language }}
 
     # Autobuild attempts to build any compiled languages  (C/C++, C#, or Java).
     # If this step fails, then you should remove it and run the build manually (see below)
     - name: Autobuild
-      uses: github/codeql-action/autobuild@v2
+      uses: github/codeql-action/autobuild@v3
 
     - name: Perform CodeQL Analysis
-      uses: github/codeql-action/analyze@v2
+      uses: github/codeql-action/analyze@v3
       with:
         category: "/language:${{matrix.language}}"

.gitignore

@@ -141,3 +141,6 @@ dmypy.json
 output/*
 !output
 !output/placeholder.txt
+
+# Local sqlite mirrors generated from MySQL dumps
+config/databases/*.db

README.md

@@ -5,3 +5,20 @@
 [![Website Status](https://img.shields.io/website?url=http%3A%2F%2Fbar.utoronto.ca%2Fapi%2F)](http://bar.utoronto.ca/api/) ![GitHub repo size](https://img.shields.io/github/repo-size/BioAnalyticResource/BAR_API) [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) [![Documentation Status](https://readthedocs.org/projects/bar-api/badge/?version=latest)](https://bar-api.readthedocs.io/en/latest/?badge=latest)
 
 This is the official repository for the Bio-Analytic Resource API. The API documentation can be found [here](https://bar-api.readthedocs.io/en/latest/).
+
+## Local eFP sqlite mirrors
+
+The `/efp_proxy/expression/{database}/{gene_id}` endpoint first tries to query a live SQLAlchemy bind (as configured in `config/BAR_API.cfg`). When you do not have access to those remote MySQL instances, you can still exercise the API by generating local SQLite mirrors from the shipped MySQL dumps:
+
+```bash
+# Build every mirror (takes a few minutes)
+python scripts/build_sqlite_mirrors.py
+
+# Or only the datasets you care about
+python scripts/build_sqlite_mirrors.py klepikova sample_data
+
+# Use --force to overwrite existing *.db files
+python scripts/build_sqlite_mirrors.py klepikova --force
+```
+
+The script transforms the dumps in `config/databases/*.sql` into companion `*.db` files in the same directory. The eFP proxy will automatically fall back to those mirrors whenever the corresponding MySQL bind is unavailable, while production deployments continue to use the remote databases as soon as they are reachable.

api/__init__.py

@@ -64,6 +64,7 @@ def create_app():
     from api.resources.efp_image import efp_image
     from api.resources.fastpheno import fastpheno
     from api.resources.llama3 import llama3
+    from api.resources.efp_proxy import efp_proxy_ns
 
     bar_api.add_namespace(gene_information)
     bar_api.add_namespace(rnaseq_gene_expression)
@@ -78,6 +79,7 @@ def create_app():
     bar_api.add_namespace(efp_image)
     bar_api.add_namespace(fastpheno)
     bar_api.add_namespace(llama3)
+    bar_api.add_namespace(efp_proxy_ns)
     bar_api.init_app(bar_app)
     return bar_app
 

api/models/bar_utils.py

@@ -0,0 +1,4 @@
+# Bridge file to maintain backward compatibility with imports
+from api.utils.bar_utils import BARUtils
+
+__all__ = ['BARUtils']

api/models/efp_dynamic.py

@@ -0,0 +1,50 @@
+"""
+dynamic sqlalchemy model generation for simple efp databases
+"""
+
+from __future__ import annotations
+
+from typing import Dict
+
+from sqlalchemy import Float, Integer, String, Text
+from sqlalchemy.dialects.mysql import INTEGER
+
+from api import db
+from api.models.efp_schemas import SIMPLE_EFP_DATABASE_SCHEMAS
+
+
+def _to_sqla_type(column_spec):
+    """map a simple column spec to a sqlalchemy column type"""
+    col_type = column_spec.get("type")
+    if col_type == "string":
+        return String(column_spec["length"])
+    if col_type == "integer":
+        if column_spec.get("unsigned"):
+            return INTEGER(unsigned=True)
+        return Integer
+    if col_type == "float":
+        return Float
+    if col_type == "text":
+        return Text
+    raise ValueError(f"Unsupported column type: {col_type}")
+
+
+def _generate_model(bind_key: str, spec) -> db.Model:
+    """build a concrete sqlalchemy model for the given schema"""
+    attrs = {"__bind_key__": bind_key, "__tablename__": spec["table_name"]}
+
+    for column in spec["columns"]:
+        kwargs = {"nullable": column.get("nullable", True)}
+        if column.get("primary_key"):
+            kwargs["primary_key"] = True
+        attrs[column["name"]] = db.mapped_column(_to_sqla_type(column), **kwargs)
+
+    class_name = "".join([part.capitalize() for part in bind_key.split("_")]) + "SampleData"
+    return type(class_name, (db.Model,), attrs)
+
+
+SIMPLE_EFP_SAMPLE_MODELS: Dict[str, db.Model] = {
+    db_name: _generate_model(db_name, spec) for db_name, spec in SIMPLE_EFP_DATABASE_SCHEMAS.items()
+}
+
+__all__ = ["SIMPLE_EFP_SAMPLE_MODELS"]