Skip to content

fix(history): implement thread-safe connection pool for SQLite to fix resource leaks#35

Open
mhajder wants to merge 1 commit into
open-webui:mainfrom
mhajder:fix/history
Open

fix(history): implement thread-safe connection pool for SQLite to fix resource leaks#35
mhajder wants to merge 1 commit into
open-webui:mainfrom
mhajder:fix/history

Conversation

@mhajder

@mhajder mhajder commented Jun 6, 2026

Copy link
Copy Markdown

Refactors SyncHistory to use a queue.Queue connection pool instead of threading.local().

Previously, connections tied to threading.local() in a background worker pool would remain open until the thread died, leading to a steady leak of open file descriptors and locked connections.

Changes:

  • Replaced threading.local() with a bounded queue.Queue pool (size=5) of sqlite3.Connection objects configured with check_same_thread=False.
  • Safely wrapped the initial schema connection in a try...finally block to ensure close() is called, fixing a silent file lock leak.
  • Added PRAGMA synchronous=NORMAL alongside journal_mode=WAL for drastically improved concurrent write performance without sacrificing durability.
  • Updated all database operations to borrow from the pool using a context manager, allowing true multi-threaded concurrency (concurrent readers/writers via WAL) while strictly bounding the total number of open database connections.
  • Updated close() to gracefully drain the queue and explicitly terminate all pooled connections.

… resource leaks

Refactors `SyncHistory` to use a `queue.Queue` connection pool instead of `threading.local()`.

Previously, connections tied to `threading.local()` in a background worker pool would remain open until the thread died, leading to a steady leak of open file descriptors and locked connections.

Changes:
- Replaced `threading.local()` with a bounded `queue.Queue` pool (size=5) of `sqlite3.Connection` objects configured with `check_same_thread=False`.
- Safely wrapped the initial schema connection in a `try...finally` block to ensure `close()` is called, fixing a silent file lock leak.
- Added `PRAGMA synchronous=NORMAL` alongside `journal_mode=WAL` for drastically improved concurrent write performance without sacrificing durability.
- Updated all database operations to borrow from the pool using a context manager, allowing true multi-threaded concurrency (concurrent readers/writers via WAL) while strictly bounding the total number of open database connections.
- Updated `close()` to gracefully drain the queue and explicitly terminate all pooled connections.
@mhajder mhajder marked this pull request as ready for review June 15, 2026 19:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant