Describe the bug
I am using pgbackweb to back up a PostgreSQL 15 database to Hetzner S3 Object Storage. The backup process fails during the upload phase with 504 Gateway Timeout and 400 Bad Request errors.
The db connection is not closed after these errors. It remains active, holding an open transaction slot on the PostgreSQL database. This prevented our Postgres Autovacuum from cleaning up dead tuples until we restarted pgbackweb manually.
To Reproduce
- Configure a backup destination using Hetzner S3 (Region:
nbg1, Endpoint: https://nbg1.your-objectstorage.com).
- Run a backup for a larger database (our backup size is > 5GB)
- The upload fails with S3 timeout errors (likely due to chunks being too large for the connection/gateway).
- Check
pg_stat_activity on the database; the COPY ... TO stdout query remains active indefinitely (we observed one running for 17 days).
Expected behavior
If the S3 upload fails, pgbackweb should:
- Retry a reasonable number of times.
- If retries fail, close the database connection.
This bugfix/issue-91-idle-connections seems like a fix for the connection issue. However, recently still all our backups fail with the logs below.
Logs
failed to upload file to S3: upload multipart failed, upload id: <ID>, cause: operation error S3: UploadPart, exceeded maximum number of attempts, 3, https response error StatusCode: 504, RequestID: N/A, HostID: N/A, api error GatewayTimeout: The server did not respond in time.
failed to upload file to S3: upload multipart failed, upload id: <ID>, cause: operation error S3: UploadPart, https response error StatusCode: 400, RequestID: , HostID: , api error BadRequest: Bad Request
Describe the bug
I am using
pgbackwebto back up a PostgreSQL 15 database to Hetzner S3 Object Storage. The backup process fails during the upload phase with504 Gateway Timeoutand400 Bad Requesterrors.The db connection is not closed after these errors. It remains active, holding an open transaction slot on the PostgreSQL database. This prevented our Postgres Autovacuum from cleaning up dead tuples until we restarted pgbackweb manually.
To Reproduce
nbg1, Endpoint:https://nbg1.your-objectstorage.com).pg_stat_activityon the database; theCOPY ... TO stdoutquery remains active indefinitely (we observed one running for 17 days).Expected behavior
If the S3 upload fails,
pgbackwebshould:This bugfix/issue-91-idle-connections seems like a fix for the connection issue. However, recently still all our backups fail with the logs below.
Logs