CASSSIDECAR-243: Implementation of CDCPublisher #294

bbotella · 2025-12-01T22:28:55Z

Patch by James Berragan, Jyothsna Konisa, Bernardo Botella; reviewed by for CASSSIDECAR-243

jyothsnakonisa

Looks good overall, leaving some comments.

server/src/main/java/org/apache/cassandra/sidecar/cdc/CachingSchemaStore.java

server/src/main/java/org/apache/cassandra/sidecar/modules/CdcModule.java

server/src/main/java/org/apache/cassandra/sidecar/tasks/CdcConfigRefresherNotifierTask.java

server/src/main/java/org/apache/cassandra/sidecar/tasks/ClusterTopologyMonitor.java

jyothsnakonisa · 2025-12-10T00:02:19Z

server/src/test/java/org/apache/cassandra/sidecar/cdc/CdcConfigImplTest.java

    }

-    @Test
-    void testConfigChanged() throws Exception


Why did you remove this test? I think this is valuable to have a test for checking callback and config updating when config is changed.

This became a periodic task, and the callback was moved to a vertx signal instead.

But I think the test is sill valid right? may be we should have the test to check if the vertx signal is made when config is changed

jyothsnakonisa · 2025-12-10T23:32:17Z

Thanks @bbotella for addressing my comments. There is this comment #294 (comment) which is not resolved. We need to look into that codepath as it is critical for leader election and mutation publishing.

Also, can you please check if there is a way to add an end to end test that checks if the mutations are published(in-memory) might not need to have kafka for the test.

Other than those two everything else looks good to me.

jyothsnakonisa

Thanks Bernardo for the patch!

CASSSIDECAR-242: Implementation of CachingSchemaStore House cleanup WIP commit Working CDC on Sidecar Latest changes Working CDC Working Working

jberragan · 2025-12-20T07:35:10Z

server/src/main/java/org/apache/cassandra/sidecar/codecs/BigIntegerCodec.java

+     */
+    public BigInteger decodeFromWire(int pos, Buffer buf)
+    {
+        return decodeFromWire(new MutableInt(pos), buf);


I think the MutableInt is only required if we need to track the change in position in a wrapper serializer, this would be marginally faster as:

Suggested change

return decodeFromWire(new MutableInt(pos), buf);

return new BigInteger(CommonCodecs.BYTE_ARRAY.decodeFromWire(pos, buf));

jberragan · 2025-12-20T09:37:42Z

server/src/main/java/org/apache/cassandra/sidecar/cdc/CdcConfigImpl.java

-    private volatile Map<String, String> kafkaConfigMappings = Map.of();
-    private volatile Map<String, String> cdcConfigMappings = Map.of();
+
+    private Map<String, String> kafkaConfigMappings = Map.of();


These look like they should be volatile

There not accessed with synchronized so better to make them volatile

jberragan · 2025-12-20T09:49:55Z

server/src/main/java/org/apache/cassandra/sidecar/cdc/CdcManager.java

+                                    }
+                                });
+                            }))
+                   .collect(Collectors.toList());


Result of .collect(Collectors.toList()); is ignored/not used.

jberragan · 2025-12-20T09:56:03Z

server/src/main/java/org/apache/cassandra/sidecar/cdc/CdcManager.java

+        }
+
+        // NEW: Deduplicate by (instanceId, tokenRange) to prevent duplicate consumers
+        Map<String, SidecarCdc> uniqueConsumers = new HashMap<>();


Suggested change

Map<String, SidecarCdc> uniqueConsumers = new HashMap<>();

Map<String, SidecarCdc> uniqueConsumers = new HashMap<>(ownedRanges.values().stream().mapToInt(Set::size).sum());

jberragan · 2025-12-20T10:01:55Z

server/src/main/java/org/apache/cassandra/sidecar/cdc/CdcManager.java

+        return -1;
+    }
+
+    private boolean resolveToSameAddress(String host1, String host2)


It seems like this only expects ip addresses to be input, and so will not perform DNS resolution. Can we make that explicit? We can also make the method static and add some unit tests here.

jberragan · 2025-12-20T10:02:27Z

server/src/main/java/org/apache/cassandra/sidecar/cdc/CdcManager.java

+    {
+        for (InstanceMetadata instance : instanceFetcher.allLocalInstances())
+        {
+            String configuredHost = instance.ipAddress();


This is an ip address, no?

Suggested change

String configuredHost = instance.ipAddress();

String configuredIpAddress = instance.ipAddress();

jberragan · 2025-12-21T08:08:51Z

server/src/main/java/org/apache/cassandra/sidecar/cdc/CdcPublisher.java

+    private final ICdcStats cdcStats;
+    private final SidecarConfiguration sidecarConfiguration;
+    private CdcManager cdcManager;
+    private Serializer<CdcEvent> avroSerializer;


Can be final:

Suggested change

private Serializer<CdcEvent> avroSerializer;

private final Serializer<CdcEvent> avroSerializer;

private final Provider<RangeManager> rangeManagerProvider;

jberragan

Mostly nits, +1 (nb)

bbotella requested a review from jyothsnakonisa December 1, 2025 22:28

jyothsnakonisa reviewed Dec 10, 2025

View reviewed changes

jyothsnakonisa approved these changes Dec 19, 2025

View reviewed changes

bbotella added 13 commits December 19, 2025 19:21

CASSSIDECAR-242: Implementation of CachingSchemaStore

7b884c3

CASSSIDECAR-242: Implementation of CachingSchemaStore House cleanup WIP commit Working CDC on Sidecar Latest changes Working CDC Working Working

Revert not needed changes

c77bf28

Fix RAT

0b1bb8c

Fix checkstyle

d8b182e

Fix tests

ac828e1

Add readme info

5b2b462

Applies feedback

971bd9b

Add missing license

fbef67d

Fix deferred injection

f3195d9

Re add config changed test

f694574

Fix checkstyle

ad67aa1

Fix checkstyle

56299c4

Add Changes.txt

b594d40

bbotella force-pushed the CASSSIDECAR-243-CDCPublisher branch from 73efd40 to b594d40 Compare December 19, 2025 18:22

jberragan reviewed Dec 20, 2025

View reviewed changes

jberragan reviewed Dec 21, 2025

View reviewed changes

jberragan approved these changes Dec 21, 2025

View reviewed changes

	return decodeFromWire(new MutableInt(pos), buf);
	return new BigInteger(CommonCodecs.BYTE_ARRAY.decodeFromWire(pos, buf));

	Map<String, SidecarCdc> uniqueConsumers = new HashMap<>();
	Map<String, SidecarCdc> uniqueConsumers = new HashMap<>(ownedRanges.values().stream().mapToInt(Set::size).sum());

	String configuredHost = instance.ipAddress();
	String configuredIpAddress = instance.ipAddress();

	private Serializer<CdcEvent> avroSerializer;
	private final Serializer<CdcEvent> avroSerializer;
	private final Provider<RangeManager> rangeManagerProvider;

CASSSIDECAR-243: Implementation of CDCPublisher #294

Are you sure you want to change the base?

CASSSIDECAR-243: Implementation of CDCPublisher #294

Uh oh!

Conversation

bbotella commented Dec 1, 2025

Uh oh!

jyothsnakonisa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jyothsnakonisa commented Dec 10, 2025

Uh oh!

jyothsnakonisa left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jberragan left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants