WSGI: Gunicorn + Redis

[davidborland]

Current architecture in hosted mode keeps per-user app state in a process-local dictionary named _session_cache, keyed by a session_id stored in Flask session cookies. This can break using Gunicorn with workers > 1. Should be fine for modest number of users (10-25 concurrent).

Currently:

1. Upload flow writes temp files and stores lightweight session info in Flask session.
2. A session_id is generated and used as key into _session_cache.
3. Each key points to a TaterApp instance with loaded docs/widgets.
4. Callbacks resolve current app at request time via session_id lookup.

Benefits:

1. Very low complexity.
2. Fast lookup and no network round trips.
3. Easy callback model for Dash because state is in memory.

Limitations:

1. Multi-worker mismatch: with Gunicorn workers > 1, each worker has its own _session_cache, so a request may land on a worker that does not have that user’s TaterApp.
2. Horizontal scaling mismatch: multiple containers/pods do not share this cache.
3. Volatile state: process restart clears cache; users lose in-memory session state.
4. Memory growth: no eviction/TTL means long-lived server can accumulate stale sessions.
5. Session affinity dependency: behavior appears fine only when traffic consistently returns to same process.
6. Concurrency risk: mutable in-memory objects can be touched by concurrent requests/threads unless carefully controlled.

Implications:

1. Safe default is single worker/process for hosted mode.
2. Rolling deploys or restarts will disrupt active annotation sessions.
3. Autoscaling without shared session storage will produce intermittent “works/fails” behavior.

For now:

1. Run one Gunicorn worker.
2. Add cache cleanup policy (TTL + max entries).
3. Add explicit logging/metrics for cache hits/misses and session rebuilds.
4. Document that hosted mode is single-process unless shared state is added.

Future:

1. Move session app state out of process memory:
2. Redis or database-backed session/app state.
3. Shared file/object store for temp artifacts.
4. Keep only small identifiers in cookie session.
5. Rebuild or fetch TaterApp state deterministically per request/session.
6. Then scale workers/replicas safely.