Skip to content

Commit 34e963b

Browse files
peachdawnleachpeachdawnleach
committed
Added nested steps and ported to 26.1
Added nested steps and ported to 26.1
1 parent 4b5aeb4 commit 34e963b

File tree

2 files changed

+102
-35
lines changed

2 files changed

+102
-35
lines changed

‎src/current/v25.4/manage-logical-data-replication.md‎

Lines changed: 34 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -96,44 +96,50 @@ To resolve a row in the DLQ:
9696
106677386757203 | 2025-04-25 25:32:28.435439+00 | {"created_at": "2025-04-25:35:00.499499", "payload": "blahblahblah=", "my_id": 207}
9797
~~~
9898

99-
1. Check the value of the row and the replicated time:
99+
1. Determine whether the value of the row matches on the source and the destination:
100100

101-
{% include_cached copy-clipboard.html %}
102-
~~~ sql
103-
SELECT * FROM foo WHERE my_id = 207;
104-
SELECT replicated_time FROM show logical replication jobs;
105-
~~~
101+
1. Check the value of the row and the replicated time:
106102

107-
1. On the source, check the value of the row as of the replicated time:
103+
{% include_cached copy-clipboard.html %}
104+
~~~ sql
105+
SELECT * FROM foo WHERE my_id = 207;
106+
SELECT replicated_time FROM show logical replication jobs;
107+
~~~
108108

109-
{% include_cached copy-clipboard.html %}
110-
~~~ sql
111-
SELECT * FROM foo WHERE my_id = 207 AS OF SYSTEM TIME {replicated time};
112-
~~~
109+
1. On the source, check the value of the row as of the replicated time:
113110

114-
1. If the value of the row is the same on both the source and the destination, delete the row from the DLQ on the destination:
111+
{% include_cached copy-clipboard.html %}
112+
~~~ sql
113+
SELECT * FROM foo WHERE my_id = 207 AS OF SYSTEM TIME {replicated time};
114+
~~~
115115

116-
{% include_cached copy-clipboard.html %}
117-
~~~ sql
118-
DELETE FROM crdb_replication.dlq_271_foo WHERE id = 106677386757203;
119-
~~~
116+
1. Determine a course of action based on the results of the previous steps:
120117

121-
1. If the row's value on the destination is different from its value on the source, but the row's value on the source equals its value in the DLQ, update the row on the destination to have the same value as on the source:
118+
1. If the value of the row is the same on both the source and the destination, delete the row from the DLQ on the destination:
122119

123-
{% include_cached copy-clipboard.html %}
124-
~~~ sql
125-
UPSERT into foo VALUES (207, '2025-04-25:35:00.499499', 'blahblahblah=')
126-
~~~
120+
{% include_cached copy-clipboard.html %}
121+
~~~ sql
122+
DELETE FROM crdb_replication.dlq_271_foo WHERE id = 106677386757203;
123+
~~~
127124

128-
1. If the row's value on the destination is different from its value on the source, and the row's value on the source equals its value in the DLQ, refresh the replicated time and retry the equality queries above. If the same results hold after a few retries with refreshed replicated times, there is likely a more recent entry for the row in the DLQ. To find the the more recent entry, find all rows in the DLQ with the matching primary key:
125+
1. If the row's value on the destination is different from its value on the source, but the row's value on the source equals its value in the DLQ, update the row on the destination to have the same value as on the source:
129126

130-
{% include_cached copy-clipboard.html %}
131-
~~~ sql
132-
# On the destination:
133-
SELECT id, dlq_timestamp, incoming_row FROM crdb_replication.dlq_271_foo WHERE incoming_row->>'my_id' = 207;
134-
~~~
127+
{% include_cached copy-clipboard.html %}
128+
~~~ sql
129+
UPSERT into foo VALUES (207, '2025-04-25:35:00.499499', 'blahblahblah=')
130+
~~~
131+
132+
1. If the row's value on the destination is different from its value on the source, and the row's value on the source equals its value in the DLQ, refresh the replicated time and retry the equality queries above. If the same results hold after a few retries with refreshed replicated times, there is likely a more recent entry for the row in the DLQ.
133+
134+
1. To find the more recent entry, find all rows in the DLQ with the matching primary key:
135+
136+
{% include_cached copy-clipboard.html %}
137+
~~~ sql
138+
# On the destination:
139+
SELECT id, dlq_timestamp, incoming_row FROM crdb_replication.dlq_271_foo WHERE incoming_row->>'my_id' = 207;
140+
~~~
135141

136-
1. If there are more recent entries for the row, delete the less recent entries and resolve the row using the most recent entry.
142+
1. If there are more recent entries for the row, delete the less recent entries and resolve the row using the most recent entry.
137143

138144
## Schema changes
139145

‎src/current/v26.1/manage-logical-data-replication.md‎

Lines changed: 68 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -33,13 +33,7 @@ When a conflict cannot apply due to violating [constraints]({% link {{ page.vers
3333

3434
### Dead letter queue (DLQ)
3535

36-
When the LDR job starts, it will create a DLQ table with each replicating table so that unresolved conflicts can be tracked. The DLQ will contain the writes that LDR cannot apply after the retry period of a minute, which could occur if there is a unique index on the destination table (for more details, refer to [Unique seconday indexes]({% link {{ page.version.version }}/set-up-logical-data-replication.md %}#unique-secondary-indexes)).
37-
38-
{{site.data.alerts.callout_info}}
39-
LDR will not pause when the writes are sent to the DLQ, you must manage the DLQ manually.
40-
{{site.data.alerts.end}}
41-
42-
To manage the DLQ, you can evaluate entries in the `incoming_row` column and apply the row manually to another table with SQL statements.
36+
When the LDR job starts, it creates a DLQ table with each replicating table so that unresolved conflicts can be tracked. The DLQ contains the writes that LDR cannot apply after the retry period of a minute, which could occur if there is a unique index on the destination table (for more details, refer to [Unique secondary indexes]({% link {{ page.version.version }}/set-up-logical-data-replication.md %}#unique-secondary-indexes)).
4337

4438
As an example, for an LDR stream created on the `movr.public.promo_codes` table:
4539

@@ -80,6 +74,73 @@ CONSTRAINT dlq_113_public_promo_codes_pkey PRIMARY KEY (ingestion_job_id ASC, dl
8074
)
8175
~~~
8276

77+
#### Resolve rows in the DLQ
78+
79+
LDR does not pause when writes are sent to the DLQ. You must manage the DLQ manually by examining each entry in the DLQ and either manually reinserting the row or deleting the entry from the DLQ. If you have multiple DLQ entries, resolve them in order from most recent to least recent.
80+
81+
To resolve a row in the DLQ:
82+
83+
1. On the destination, find the primary key value in the `incoming_row` column.
84+
85+
{% include_cached copy-clipboard.html %}
86+
~~~ sql
87+
SELECT id, dlq_timestamp, incoming_row FROM crdb_replication.dlq_271_foo;
88+
~~~
89+
90+
In this example result, `incoming_row` contains a primary key value of `207` identified by the column `my_id`:
91+
92+
{% include_cached copy-clipboard.html %}
93+
~~~ sql
94+
id | dlq_timestamp | incoming_row
95+
----------------------+---------------------+----------+-------------------------------+-----------------------------------------------------------------
96+
106677386757203 | 2025-04-25 25:32:28.435439+00 | {"created_at": "2025-04-25:35:00.499499", "payload": "blahblahblah=", "my_id": 207}
97+
~~~
98+
99+
1. Determine whether the value of the row matches on the source and the destination:
100+
101+
1. Check the value of the row and the replicated time:
102+
103+
{% include_cached copy-clipboard.html %}
104+
~~~ sql
105+
SELECT * FROM foo WHERE my_id = 207;
106+
SELECT replicated_time FROM show logical replication jobs;
107+
~~~
108+
109+
1. On the source, check the value of the row as of the replicated time:
110+
111+
{% include_cached copy-clipboard.html %}
112+
~~~ sql
113+
SELECT * FROM foo WHERE my_id = 207 AS OF SYSTEM TIME {replicated time};
114+
~~~
115+
116+
1. Determine a course of action based on the results of the previous steps:
117+
118+
1. If the value of the row is the same on both the source and the destination, delete the row from the DLQ on the destination:
119+
120+
{% include_cached copy-clipboard.html %}
121+
~~~ sql
122+
DELETE FROM crdb_replication.dlq_271_foo WHERE id = 106677386757203;
123+
~~~
124+
125+
1. If the row's value on the destination is different from its value on the source, but the row's value on the source equals its value in the DLQ, update the row on the destination to have the same value as on the source:
126+
127+
{% include_cached copy-clipboard.html %}
128+
~~~ sql
129+
UPSERT into foo VALUES (207, '2025-04-25:35:00.499499', 'blahblahblah=')
130+
~~~
131+
132+
1. If the row's value on the destination is different from its value on the source, and the row's value on the source equals its value in the DLQ, refresh the replicated time and retry the equality queries above. If the same results hold after a few retries with refreshed replicated times, there is likely a more recent entry for the row in the DLQ.
133+
134+
1. To find the more recent entry, find all rows in the DLQ with the matching primary key:
135+
136+
{% include_cached copy-clipboard.html %}
137+
~~~ sql
138+
# On the destination:
139+
SELECT id, dlq_timestamp, incoming_row FROM crdb_replication.dlq_271_foo WHERE incoming_row->>'my_id' = 207;
140+
~~~
141+
142+
1. If there are more recent entries for the row, delete the less recent entries and resolve the row using the most recent entry.
143+
83144
## Schema changes
84145

85146
When you start LDR on a table, the job will lock the schema, which will prevent any accidental [schema changes]({% link {{ page.version.version }}/online-schema-changes.md %}) that would cause issues for LDR. There are some [supported schema changes](#supported-schema-changes) that you can perform on a replicating table, otherwise it is necessary to stop LDR in order to [coordinate the schema change](#coordinate-other-schema-changes).

0 commit comments

Comments
 (0)