Skip to content

increase sql retries #413

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dphuang2 merged 3 commits into from
Jan 16, 2026
Merged

increase sql retries #413

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1445d75
increase sql retries
Jan 14, 2026
d4a445b
make connection more robust
Jan 16, 2026
0843129
Implement robust database connection with retry logic to handle pragm…
Jan 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions eval_protocol/dataset_logger/sqlite_evaluation_row_store.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,10 @@ class EvaluationRow(BaseModel): # type: ignore

self._EvaluationRow = EvaluationRow

self._db.connect()
# Wrap connect() in retry logic since setting pragmas can fail with "database is locked"
execute_with_sqlite_retry(lambda: self._db.connect(reuse_if_open=True))
# Use safe=True to avoid errors when tables/indexes already exist
self._db.create_tables([EvaluationRow], safe=True)
execute_with_sqlite_retry(lambda: self._db.create_tables([EvaluationRow], safe=True))

@property
def db_path(self) -> str:
Expand Down
9 changes: 5 additions & 4 deletions eval_protocol/event_bus/sqlite_event_bus_database.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@


# Retry configuration for database operations
SQLITE_RETRY_MAX_TRIES = 5
SQLITE_RETRY_MAX_TIME = 30 # seconds
SQLITE_RETRY_MAX_TRIES = 10
SQLITE_RETRY_MAX_TIME = 60 # seconds


def _is_database_locked_error(e: Exception) -> bool:
Expand Down Expand Up @@ -181,9 +181,10 @@ class Event(BaseModel): # type: ignore
processed = BooleanField(default=False) # Track if event has been processed

self._Event = Event
self._db.connect()
# Wrap connect() in retry logic since setting pragmas can fail with "database is locked"
execute_with_sqlite_retry(lambda: self._db.connect(reuse_if_open=True))
Comment thread
dphuang2 marked this conversation as resolved.
Outdated
# Use safe=True to avoid errors when tables already exist
self._db.create_tables([Event], safe=True)
execute_with_sqlite_retry(lambda: self._db.create_tables([Event], safe=True))

def publish_event(self, event_type: str, data: Any, process_id: str) -> None:
"""Publish an event to the database."""
Expand Down
Loading