Skip to content

tswriter process does not crash when database connection is lost #103

@dajtxx

Description

@dajtxx

The tswriter process kept reporting it's database connection was closed, but this did not crash the process which would have triggered docker to restart it and solve the problem.

It looks like a try/except catches the error and doesn't look closely enough to realise it should be fatal, or a new connection is required.

prod-tswriter-1  | 2025-09-10T09:10:27+1000|INFO   |fceecfec-6084-4263-b13d-883f22282351|BaseWriter|Adding message to delivery table
prod-tswriter-1  | 2025-09-10T09:10:27+1000|INFO   |BaseWriter|Processing 1 messages
prod-tswriter-1  | 2025-09-10T09:10:27+1000|INFO   |fceecfec-6084-4263-b13d-883f22282351|BaseWriter|msg from table 15413, 0
prod-tswriter-1  | 2025-09-10T09:10:27+1000|INFO   |fceecfec-6084-4263-b13d-883f22282351|BaseWriter|Accepted message from physical / logical device ids 426 / 381
prod-tswriter-1  | 2025-09-10T09:10:27+1000|INFO   |fceecfec-6084-4263-b13d-883f22282351|BaseWriter|Wombat-B8D61A0169A0 / SCMN-lite-Dareton
prod-tswriter-1  | 2025-09-10T09:10:27+1000|ERROR  |fceecfec-6084-4263-b13d-883f22282351|TimescaleDB|Error while processing message.
prod-tswriter-1  | Traceback (most recent call last):
prod-tswriter-1  |   File "/home/broker/python/delivery/TimescaleDB.py", line 60, in on_message
prod-tswriter-1  |     with _conn.cursor() as curs:
prod-tswriter-1  |          ^^^^^^^^^^^^^^
prod-tswriter-1  | psycopg2.InterfaceError: connection already closed
prod-tswriter-1  |
prod-tswriter-1  | During handling of the above exception, another exception occurred:
prod-tswriter-1  |
prod-tswriter-1  | Traceback (most recent call last):
prod-tswriter-1  |   File "/home/broker/python/delivery/TimescaleDB.py", line 65, in on_message
prod-tswriter-1  |     _conn.rollback()
prod-tswriter-1  | psycopg2.InterfaceError: connection already closed
prod-tswriter-1  | 2025-09-10T09:10:27+1000|ERROR  |fceecfec-6084-4263-b13d-883f22282351|BaseWriter|Message processing failed, dropping message.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions