Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
d7a3c4a
feat: add sqla broker
Arseniy-Popov Dec 3, 2025
bbac874
feat: add sqla broker
Arseniy-Popov Dec 3, 2025
fb3a9b9
feat: add sqla broker
Arseniy-Popov Dec 4, 2025
9a6b866
feat: add sqla broker
Arseniy-Popov Dec 4, 2025
29b80f8
feat: add sqla broker
Arseniy-Popov Dec 5, 2025
ba5daad
feat: add sqla broker
Arseniy-Popov Dec 5, 2025
4dc0532
feat: add sqla broker
Arseniy-Popov Dec 6, 2025
c9ec348
feat: add sqla broker
Arseniy-Popov Dec 7, 2025
35a3b48
feat: add sqla broker
Arseniy-Popov Dec 8, 2025
96de0cd
feat: add sqla broker
Arseniy-Popov Dec 10, 2025
f0775a1
feat: add sqla broker
Arseniy-Popov Dec 15, 2025
0464c49
feat: add sqla broker
Arseniy-Popov Dec 19, 2025
33661b1
feat: add sqla broker
Arseniy-Popov Dec 23, 2025
27707b5
feat: add sqla broker
Arseniy-Popov Dec 25, 2025
d201ee4
feat: add sqla broker
Arseniy-Popov Jan 4, 2026
9b8c3d3
feat: add sqla broker
Arseniy-Popov Jan 5, 2026
fed9855
feat: add sqla broker
Arseniy-Popov Jan 9, 2026
8ecbc91
feat: add sqla broker
Arseniy-Popov Jan 13, 2026
37314b7
feat: add sqla broker
Arseniy-Popov Jan 13, 2026
016686f
feat: add sqla broker
Arseniy-Popov Jan 13, 2026
8c468af
feat: add sqla broker
Arseniy-Popov Jan 17, 2026
b4da4f6
feat: add sqla broker
Arseniy-Popov Jan 18, 2026
315a2b1
feat: add sqla broker
Arseniy-Popov Jan 23, 2026
3634a6f
feat: add sqla broker
Arseniy-Popov Feb 6, 2026
9722c13
feat: add sqla broker
Arseniy-Popov Feb 6, 2026
068c024
feat: add sqla broker
Arseniy-Popov Feb 6, 2026
d18bd25
feat: add sqla broker
Arseniy-Popov Feb 6, 2026
3c40534
feat: add sqla broker
Arseniy-Popov Feb 7, 2026
9737503
feat: add sqla broker
Arseniy-Popov Feb 7, 2026
d2c4767
feat: add sqla broker
Arseniy-Popov Feb 8, 2026
88f33d5
Merge branch 'main' into feat/sqla-broker
Arseniy-Popov Feb 8, 2026
b5f7223
feat: add sqla broker
Arseniy-Popov Feb 8, 2026
0c8f8c5
feat: add sqla broker
Arseniy-Popov Feb 9, 2026
6cd7f74
feat: add sqla broker
Arseniy-Popov Feb 9, 2026
5dc1b4e
feat: add sqla broker
Arseniy-Popov Feb 9, 2026
6884927
feat: add sqla broker
Arseniy-Popov Feb 9, 2026
128c4d1
Merge branch 'main' into feat/sqla-broker
Lancetnik Feb 10, 2026
2628408
feat: add sqla broker
Arseniy-Popov Feb 10, 2026
e760f36
feat: add sqla broker
Arseniy-Popov Feb 10, 2026
b9ce283
feat: add sqla broker
Arseniy-Popov Feb 10, 2026
e78d972
feat: add sqla broker
Arseniy-Popov Feb 10, 2026
8ce7e7c
feat: add sqla broker
Arseniy-Popov Feb 10, 2026
69375ee
feat: add sqla broker
Arseniy-Popov Feb 17, 2026
a81704f
feat: add sqla broker
Arseniy-Popov Feb 17, 2026
310029f
feat: add sqla broker
Arseniy-Popov Feb 17, 2026
a494ec7
feat: add sqla broker
Arseniy-Popov Feb 17, 2026
19dac15
feat: add sqla broker
Arseniy-Popov Mar 1, 2026
6f3cb1d
feat: add sqla broker
Arseniy-Popov Mar 1, 2026
ca90a8b
Merge branch 'main' into feat/sqla-broker
Arseniy-Popov Mar 1, 2026
5cdd546
feat: add sqla broker
Arseniy-Popov Mar 1, 2026
1cb02c9
feat: add sqla broker
Arseniy-Popov Mar 1, 2026
6c2de70
feat: add sqla broker
Arseniy-Popov Mar 8, 2026
870b90e
feat: add sqla broker
Arseniy-Popov Mar 8, 2026
a3c0e4f
feat: add sqla broker
Arseniy-Popov Mar 17, 2026
27d871a
Merge branch 'main' into feat/sqla-broker
Lancetnik Apr 2, 2026
b1f392e
feat: add sqla broker
Arseniy-Popov Apr 3, 2026
3683ccb
feat: add sqla broker
Arseniy-Popov Apr 3, 2026
77cec00
feat: add sqla broker
Arseniy-Popov Apr 3, 2026
7e76c4a
feat: add sqla broker
Arseniy-Popov Apr 3, 2026
15d8f53
feat: add sqla broker
Arseniy-Popov Apr 3, 2026
312a0b2
feat: add sqla broker
Arseniy-Popov Apr 4, 2026
2c06080
feat: add sqla broker
Arseniy-Popov Apr 4, 2026
be78542
feat: add sqla broker
Arseniy-Popov Apr 4, 2026
58417ee
feat: add sqla broker
Arseniy-Popov Apr 4, 2026
0f87113
feat: add sqla broker
Arseniy-Popov Apr 4, 2026
53632a5
Merge branch 'main' into feat/sqla-broker
Lancetnik Apr 7, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions .github/workflows/pr_tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -456,6 +456,61 @@ jobs:
if-no-files-found: error
include-hidden-files: true

test-sqla-real:
if: github.event.pull_request.draft == false
runs-on: ubuntu-latest
needs:
- test-basic
services:
postgres:
image: postgres:latest # zizmor: ignore[unpinned-images]
env:
POSTGRES_DB: broker
POSTGRES_USER: broker
POSTGRES_PASSWORD: brokerpass # pragma: allowlist secret
ports:
- 5432:5432
mysql:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need 2 databases to test implementation?

image: mysql:latest # zizmor: ignore[unpinned-images]
env:
MYSQL_DATABASE: broker
MYSQL_USER: broker
MYSQL_PASSWORD: brokerpass # pragma: allowlist secret
MYSQL_ROOT_PASSWORD: brokerpass # pragma: allowlist secret
ports:
- 3306:3306
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Grant MySQL privileges
run: |
until mysqladmin ping -h 127.0.0.1 --silent; do sleep 1; done
mysql -h 127.0.0.1 -u root -pbrokerpass < tests/brokers/sqla/infra/init_mysql.sql
- uses: astral-sh/setup-uv@61cb8a9741eeb8a550a1b8544337180c0fc8476b # v7.2.0
with:
version: "latest"
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.13"
- name: Install Dependencies
run: |
uv pip install --system --group optionals --group testing .
- name: Test
run: >
pytest --cov --cov-report=
-vv -m "(slow and sqla and connected) or (sqla and connected)"
- name: Rename coverage file
run: mkdir coverage && mv .coverage coverage/.coverage.sqla-py
- name: Store coverage files
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
with:
name: .coverage.sqla-py
path: coverage
if-no-files-found: error
include-hidden-files: true

coverage-combine:
if: github.event.pull_request.draft == false
needs:
Expand Down
25 changes: 25 additions & 0 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,31 @@ services:
security_opt:
- no-new-privileges:true

postgres:
image: postgres:latest
environment:
POSTGRES_DB: broker
POSTGRES_USER: broker
POSTGRES_PASSWORD: brokerpass # pragma: allowlist secret
ports:
- "5432:5432"
security_opt:
- no-new-privileges:true

mysql:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need 2 databases to run tests?

image: mysql:latest
environment:
MYSQL_DATABASE: broker
MYSQL_USER: broker
MYSQL_PASSWORD: brokerpass # pragma: allowlist secret
MYSQL_ROOT_PASSWORD: brokerpass # pragma: allowlist secret
ports:
- "3306:3306"
volumes:
- ./tests/brokers/sqla/infra/init_mysql.sql:/docker-entrypoint-initdb.d/99-grant-all.sql:ro
security_opt:
- no-new-privileges:true

faststream:
build: .
volumes:
Expand Down
40 changes: 18 additions & 22 deletions docs/docs/en/getting-started/acknowledgement.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Due to the possibility of unexpected errors during message processing, FastStrea
- [**RabbitMQ**](../rabbit/index.md){.internal-link}
- [**NATS JetStream**](../nats/jetstream/index.md){.internal-link}
- [**Redis Streams**](../redis/streams/index.md){.internal-link}
- [**SQLA**](../sqla/index.md){.internal-link}

### Usage

Expand All @@ -36,17 +37,25 @@ async def handler(msg: str, logger: Logger) -> None:

### Available Options

Each `AckPolicy` variant includes behavior examples for both successful processing and error scenarios. Note that broker-specific behaviors are also included.
| Policy | Description | On Success | On Error |
| --- | --- | --- | --- |
| `ACK_FIRST` | ACK upon receipt, prior to processing | ACK | ACK |
| `ACK` | ACK after processing, regardless of exceptions | ACK | ACK |
| `REJECT_ON_ERROR` | ACK on successful processing, REJECT on exception (no redelivery) | ACK | REJECT |
| `NACK_ON_ERROR` | ACK on successful processing, NACK on exception (with redelivery) | ACK | NACK |
| `MANUAL` | No automatic acknowledgement. User must manually handle the completion via message methods<ul><li> `#!python msg.ack()`</li><li>`#!python msg.nack()`<li>`#!python msg.reject()`</li></ul> | | |

| Policy | Description | On Success | On Error | Broker Notes |
| --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | ----------------------------------------------------- | ------------------------------------------------------------------------ |
| `ACK_FIRST` | Acknowledge immediately upon receipt, before processing begins. | Message is acknowledged early;<br/>may be lost if processing fails. | Acknowledged despite error;<br/>message not re-delivered. | Kafka commits offset;<br/>NATS, Redis, and RabbitMQ acknowledge immediately. |
| `ACK` | Acknowledge only after processing completes, regardless of success. | Ack after success. | Ack sent anyway;<br/>message not redelivered. | Kafka: offset commit; others: explicit ack. |
| `REJECT_ON_ERROR` | Reject message if an unhandled exception occurs, permanently discarding it;<br/>otherwise, ack. | Ack after success. | Message discarded; no retry. | RabbitMQ/NATS drops message. Kafka commits offset. |
| `NACK_ON_ERROR` | Nack on error to allow message redelivery, ack after success otherwise. | Ack after success. | Redeliver; attempt to resend message. | Redis Streams and RabbitMQ redelivers; Kafka commits as fallback. |
| `MANUAL` | No automatic acknowledgement. User must manually handle the completion via message methods<ul><li> `#!python msg.ack()`</li><li>`#!python msg.nack()`<li>`#!python msg.reject()`</li></ul> | | | |
### Broker Behaviors

---
Here is how **FastStream's** ACK / NACK / REJECT commands map to brokers' behaviors:

| Broker | `ACK` | `NACK` | `REJECT` |
| ------ | ----- | ------ | -------- |
| [RabbitMQ](https://www.rabbitmq.com/docs/confirms#acknowledgement-modes){.external-link target="_blank"} | Protocol ack | Protocol nack | Protocol reject |
| [NATS JetStream](https://docs.nats.io/using-nats/developer/develop_jetstream#acknowledging-messages){.external-link target="_blank"} | Protocol ack | Protocol nak | Protocol term |
| [Redis Streams](https://redis.io/docs/latest/commands/xack/){.external-link target="_blank"} | Xack call | Do nothing | Do nothing |
| Kafka | Commits offset | Seek offset and read message again | Commits offset (same as `ACK`) |
| [SQLA](../sqla/index.md){.internal-link} | Mark msg as COMPLETED | Mark msg as RETRYABLE if retry strategy allows and FAILED otherwise | Mark msg as FAILED |

### When to Use

Expand All @@ -56,8 +65,6 @@ Each `AckPolicy` variant includes behavior examples for both successful processi
- Use `NACK_ON_ERROR` to retry messages in case of failure.
- Use `MANUAL` to fully manually control message acknowledgment (for example, calling `#!python message.ack()` yourself).

---

### Extended Examples

#### Automatic Retry on Failure
Expand Down Expand Up @@ -95,14 +102,3 @@ async def handle_event(msg: str) -> None:
```

You can also manage manual acknowledgement using middleware. For more information, [error handling middleware documentation](./middlewares/exception.md){.internal-link}.

### Broker Behavior Summary

However, not all brokers support our semantics. Here is a brief overview of **FastStream's** ACK / NACK / REJECT command mapping to brokers' acknowledgment policies:

| Broker | `ACK` | `NACK` | `REJECT` |
| ------ | ----- | ------ | -------- |
| [RabbitMQ](https://www.rabbitmq.com/docs/confirms#acknowledgement-modes){.external-link target="_blank"} | Protocol ack | Protocol nack | Protocol reject |
| [NATS JetStream](https://docs.nats.io/using-nats/developer/develop_jetstream#acknowledging-messages){.external-link target="_blank"} | Protocol ack | Protocol nak | Protocol term |
| [Redis Streams](https://redis.io/docs/latest/commands/xack/){.external-link target="_blank"} | Xack call | Do nothing | Do nothing |
| Kafka | Commits offset | Seek offset and read message again | Commits offset (same as `ACK`) |
Loading
Loading