part: Fix MapTxn reusability by joamaki · Pull Request #140 · cilium/statedb

joamaki · 2026-01-26T15:54:29Z

The contract for MapTxn.Commit was that the MapTxn could be reused for further modifications, e.g.
one could keep a single long-living MapTxn to produce new Map instances. This was broken by #132 as the txnID was not bumped on commit. The existing test case
did not catch the issue as it was only checking the singleton case where Iterator() was called that did
bump txnID.

Fix the problem by unconditionally bumping txnID on MapTxn.Commit.

The MapTxn reuse test only tested for the singleton case. Extend to also test when both previous and new maps are not singletons. Signed-off-by: Jussi Maki <jussi@isovalent.com>

github-actions · 2026-01-26T16:05:03Z

$ make
go build ./...
go: downloading go.yaml.in/yaml/v3 v3.0.3
go: downloading github.com/cilium/hive v0.0.0-20250731144630-28e7a35ed227
go: downloading golang.org/x/time v0.5.0
go: downloading github.com/spf13/cobra v1.8.0
go: downloading github.com/spf13/pflag v1.0.5
go: downloading github.com/cilium/stream v0.0.0-20240209152734-a0792b51812d
go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de
go: downloading github.com/spf13/viper v1.18.2
go: downloading go.uber.org/dig v1.17.1
go: downloading golang.org/x/term v0.16.0
go: downloading github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc
go: downloading github.com/mitchellh/mapstructure v1.5.0
go: downloading golang.org/x/sys v0.17.0
go: downloading golang.org/x/tools v0.17.0
go: downloading github.com/fsnotify/fsnotify v1.7.0
go: downloading github.com/sagikazarmark/slog-shim v0.1.0
go: downloading github.com/spf13/afero v1.11.0
go: downloading github.com/spf13/cast v1.6.0
go: downloading github.com/subosito/gotenv v1.6.0
go: downloading github.com/hashicorp/hcl v1.0.0
go: downloading gopkg.in/ini.v1 v1.67.0
go: downloading github.com/magiconair/properties v1.8.7
go: downloading github.com/pelletier/go-toml/v2 v2.1.0
go: downloading gopkg.in/yaml.v3 v3.0.1
go: downloading golang.org/x/text v0.14.0
STATEDB_VALIDATE=1 go test ./... -cover -vet=all -test.count 1
go: downloading github.com/stretchr/testify v1.8.4
go: downloading go.uber.org/goleak v1.3.0
go: downloading golang.org/x/exp v0.0.0-20240119083558-1b970713d09a
go: downloading github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2
ok  	github.com/cilium/statedb	422.040s	coverage: 78.6% of statements
ok  	github.com/cilium/statedb/index	0.006s	coverage: 28.7% of statements
ok  	github.com/cilium/statedb/internal	0.031s	coverage: 42.9% of statements
ok  	github.com/cilium/statedb/lpm	4.413s	coverage: 77.6% of statements
ok  	github.com/cilium/statedb/part	61.937s	coverage: 87.2% of statements
ok  	github.com/cilium/statedb/reconciler	0.250s	coverage: 92.3% of statements
	github.com/cilium/statedb/reconciler/benchmark		coverage: 0.0% of statements
	github.com/cilium/statedb/reconciler/example		coverage: 0.0% of statements
go test -race ./... -test.count 1
ok  	github.com/cilium/statedb	39.256s
ok  	github.com/cilium/statedb/index	1.013s
ok  	github.com/cilium/statedb/internal	1.039s
ok  	github.com/cilium/statedb/lpm	2.883s
ok  	github.com/cilium/statedb/part	36.236s
ok  	github.com/cilium/statedb/reconciler	1.348s
?   	github.com/cilium/statedb/reconciler/benchmark	[no test files]
?   	github.com/cilium/statedb/reconciler/example	[no test files]
go test ./... -bench . -benchmem -test.run xxx
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkDB_WriteTxn_1-4                      	  659770	      1676 ns/op	    596650 objects/sec	    1000 B/op	      16 allocs/op
BenchmarkDB_WriteTxn_10-4                     	 1681057	       760.7 ns/op	   1314594 objects/sec	     520 B/op	       8 allocs/op
BenchmarkDB_WriteTxn_100-4                    	 2095190	       549.6 ns/op	   1819416 objects/sec	     490 B/op	       7 allocs/op
BenchmarkDB_WriteTxn_1000-4                   	 1884648	       618.1 ns/op	   1617829 objects/sec	     447 B/op	       7 allocs/op
BenchmarkDB_WriteTxn_100_SecondaryIndex-4     	  791281	      1321 ns/op	    756762 objects/sec	    1007 B/op	      20 allocs/op
BenchmarkDB_WriteTxn_CommitOnly_100Tables-4   	  982501	      1195 ns/op	    1112 B/op	       5 allocs/op
BenchmarkDB_WriteTxn_CommitOnly_1Table-4      	 1539066	       777.8 ns/op	     224 B/op	       5 allocs/op
BenchmarkDB_NewWriteTxn-4                     	 1685385	       712.4 ns/op	     200 B/op	       4 allocs/op
BenchmarkDB_WriteTxnCommit100-4               	  988246	      1204 ns/op	    1096 B/op	       5 allocs/op
BenchmarkDB_NewReadTxn-4                      	640815397	         1.871 ns/op	       0 B/op	       0 allocs/op
BenchmarkDB_Modify-4                          	    1710	    684041 ns/op	   1461901 objects/sec	  479651 B/op	    8072 allocs/op
BenchmarkDB_GetInsert-4                       	    1585	    759005 ns/op	   1317515 objects/sec	  455644 B/op	    8072 allocs/op
BenchmarkDB_RandomInsert-4                    	    1929	    639509 ns/op	   1563699 objects/sec	  447630 B/op	    7072 allocs/op
BenchmarkDB_RandomReplace-4                   	     468	   2537485 ns/op	    394091 objects/sec	 1924794 B/op	   29102 allocs/op
BenchmarkDB_SequentialInsert-4                	    1944	    618243 ns/op	   1617486 objects/sec	  447628 B/op	    7072 allocs/op
BenchmarkDB_SequentialInsert_Prefix-4         	     478	   2518147 ns/op	    397117 objects/sec	 3563248 B/op	   45542 allocs/op
BenchmarkDB_Changes_Baseline-4                	    1540	    747674 ns/op	   1337482 objects/sec	  507780 B/op	    9163 allocs/op
BenchmarkDB_Changes-4                         	     915	   1289747 ns/op	    775346 objects/sec	  709122 B/op	   12314 allocs/op
BenchmarkDB_RandomLookup-4                    	   22234	     54046 ns/op	  18502596 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_SequentialLookup-4                	   26786	     44831 ns/op	  22306111 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_Prefix_SecondaryIndex-4           	    6793	    164810 ns/op	   6067590 objects/sec	  124920 B/op	    1025 allocs/op
BenchmarkDB_FullIteration_All-4               	     794	   1474816 ns/op	  67805062 objects/sec	     104 B/op	       4 allocs/op
BenchmarkDB_FullIteration_Prefix-4            	     757	   1503790 ns/op	  66498627 objects/sec	     136 B/op	       5 allocs/op
BenchmarkDB_FullIteration_Get-4               	     192	   6144658 ns/op	  16274298 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_FullIteration_Get_Secondary-4     	      87	  12694049 ns/op	   7877707 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_FullIteration_ReadTxnGet-4        	     195	   6079033 ns/op	  16449985 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_PropagationDelay-4                	  578814	      1784 ns/op	        15.00 50th_µs	        19.00 90th_µs	        49.00 99th_µs	    1121 B/op	      19 allocs/op
BenchmarkDB_WriteTxn_100_LPMIndex-4           	  507012	      2401 ns/op	    416495 objects/sec	    1778 B/op	      37 allocs/op
BenchmarkDB_WriteTxn_1_LPMIndex-4             	  128710	     15138 ns/op	     66057 objects/sec	   15728 B/op	      81 allocs/op
BenchmarkDB_LPMIndex_Get-4                    	     349	   3508842 ns/op	   2849943 objects/sec	       0 B/op	       0 allocs/op
BenchmarkWatchSet_4-4                         	 2277927	       520.2 ns/op	     296 B/op	       4 allocs/op
BenchmarkWatchSet_16-4                        	  733548	      1574 ns/op	    1096 B/op	       5 allocs/op
BenchmarkWatchSet_128-4                       	   89757	     13505 ns/op	    8904 B/op	       5 allocs/op
BenchmarkWatchSet_1024-4                      	    9093	    135322 ns/op	   73743 B/op	       5 allocs/op
PASS
ok  	github.com/cilium/statedb	43.668s
PASS
ok  	github.com/cilium/statedb/index	0.004s
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb/internal
cpu: AMD EPYC 7763 64-Core Processor                
Benchmark_SortableMutex-4   	 6227950	       192.9 ns/op	       0 B/op	       0 allocs/op
PASS
ok  	github.com/cilium/statedb/internal	1.206s
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb/lpm
cpu: AMD EPYC 7763 64-Core Processor                
Benchmark_txn_insert/batchSize=1-4         	    1888	    639987 ns/op	   1562533 objects/sec	  838409 B/op	   13975 allocs/op
Benchmark_txn_insert/batchSize=10-4        	    3046	    393391 ns/op	   2541998 objects/sec	  385195 B/op	    6668 allocs/op
Benchmark_txn_insert/batchSize=100-4       	    3273	    368365 ns/op	   2714698 objects/sec	  345613 B/op	    6027 allocs/op
Benchmark_txn_delete/batchSize=1-4         	    1550	    763457 ns/op	   1309831 objects/sec	 1286469 B/op	   13976 allocs/op
Benchmark_txn_delete/batchSize=10-4        	    3129	    379933 ns/op	   2632040 objects/sec	  372417 B/op	    5769 allocs/op
Benchmark_txn_delete/batchSize=100-4       	    3427	    341554 ns/op	   2927795 objects/sec	  286754 B/op	    5038 allocs/op
Benchmark_LPM_Lookup-4                     	    7890	    151964 ns/op	   6580527 objects/sec	       0 B/op	       0 allocs/op
Benchmark_LPM_All-4                        	  135069	      9004 ns/op	 111066113 objects/sec	      32 B/op	       1 allocs/op
Benchmark_LPM_Prefix-4                     	  131523	      9143 ns/op	 109377665 objects/sec	      32 B/op	       1 allocs/op
Benchmark_LPM_LowerBound-4                 	  242156	      4887 ns/op	 102318568 objects/sec	     288 B/op	       2 allocs/op
PASS
ok  	github.com/cilium/statedb/lpm	11.968s
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb/part
cpu: AMD EPYC 7763 64-Core Processor                
Benchmark_Uint64Map_Random-4                  	    1533	    749447 ns/op	   1334316 items/sec	 2525400 B/op	    6020 allocs/op
Benchmark_Uint64Map_Sequential-4              	    1845	    631162 ns/op	   1584380 items/sec	 2216723 B/op	    5754 allocs/op
Benchmark_Uint64Map_Sequential_Insert-4       	    1957	    583257 ns/op	   1714510 items/sec	 2208720 B/op	    4753 allocs/op
Benchmark_Uint64Map_Sequential_Txn_Insert-4   	   10000	    116271 ns/op	   8600595 items/sec	   86352 B/op	    2028 allocs/op
Benchmark_Uint64Map_Random_Insert-4           	    1795	    682914 ns/op	   1464314 items/sec	 2520552 B/op	    5051 allocs/op
Benchmark_Uint64Map_Random_Txn_Insert-4       	    6537	    191837 ns/op	   5212769 items/sec	  117663 B/op	    2398 allocs/op
Benchmark_Insert_RootOnlyWatch-4              	    9721	    116066 ns/op	   8615756 objects/sec	   71504 B/op	    2033 allocs/op
Benchmark_Insert-4                            	    7503	    161977 ns/op	   6173715 objects/sec	  186937 B/op	    3060 allocs/op
Benchmark_Modify-4                            	   12201	     98244 ns/op	  10178717 objects/sec	   58224 B/op	    1007 allocs/op
Benchmark_GetInsert-4                         	    8764	    133653 ns/op	   7482071 objects/sec	   58224 B/op	    1007 allocs/op
Benchmark_Replace-4                           	31434549	        37.27 ns/op	  26830211 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Replace_RootOnlyWatch-4             	31358463	        38.14 ns/op	  26222330 objects/sec	       0 B/op	       0 allocs/op
Benchmark_txn_1-4                             	 5863519	       204.4 ns/op	   4892219 objects/sec	     168 B/op	       3 allocs/op
Benchmark_txn_10-4                            	 9928540	       119.4 ns/op	   8376001 objects/sec	      86 B/op	       2 allocs/op
Benchmark_txn_100-4                           	11951226	        99.94 ns/op	  10006511 objects/sec	      80 B/op	       2 allocs/op
Benchmark_txn_1000-4                          	10527907	       112.4 ns/op	   8899500 objects/sec	      65 B/op	       2 allocs/op
Benchmark_txn_delete_1-4                      	 4913480	       242.9 ns/op	   4117190 objects/sec	     664 B/op	       4 allocs/op
Benchmark_txn_delete_10-4                     	10668631	       111.5 ns/op	   8965056 objects/sec	     106 B/op	       1 allocs/op
Benchmark_txn_delete_100-4                    	11176939	       104.5 ns/op	   9566851 objects/sec	      47 B/op	       1 allocs/op
Benchmark_txn_delete_1000-4                   	13864611	        86.42 ns/op	  11571848 objects/sec	      24 B/op	       1 allocs/op
Benchmark_Get-4                               	   45021	     26745 ns/op	  37389750 objects/sec	       0 B/op	       0 allocs/op
Benchmark_All-4                               	  148039	      8449 ns/op	 118355535 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Iterator_All-4                      	  161503	      7577 ns/op	 131979365 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Iterator_Next-4                     	  153061	      7811 ns/op	 128024466 objects/sec	     896 B/op	       1 allocs/op
Benchmark_Hashmap_Insert-4                    	   14857	     80749 ns/op	  12384080 objects/sec	   74264 B/op	      20 allocs/op
Benchmark_Hashmap_Get_Uint64-4                	  136005	      8811 ns/op	 113500124 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Hashmap_Get_Bytes-4                 	  111574	     10738 ns/op	  93127593 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Delete_Random-4                     	      74	  15810182 ns/op	   6325038 objects/sec	 2111869 B/op	  102364 allocs/op
Benchmark_find16-4                            	202460341	         5.923 ns/op	       0 B/op	       0 allocs/op
Benchmark_findIndex16-4                       	78868462	        13.41 ns/op	       0 B/op	       0 allocs/op
Benchmark_find4-4                             	423497674	         2.831 ns/op	       0 B/op	       0 allocs/op
Benchmark_findIndex4-4                        	320840932	         3.737 ns/op	       0 B/op	       0 allocs/op
PASS
ok  	github.com/cilium/statedb/part	39.364s
PASS
ok  	github.com/cilium/statedb/reconciler	0.004s
?   	github.com/cilium/statedb/reconciler/benchmark	[no test files]
?   	github.com/cilium/statedb/reconciler/example	[no test files]
go run ./reconciler/benchmark -quiet
1000000 objects reconciled in 2.10 seconds (batch size 1000)
Throughput 476394.94 objects per second
817MB total allocated, 6015205 in-use objects, 338MB bytes in use

bimmlerd

yeah, nice catch

Now that txnIDs are the mechanism to detect if a node can be mutated or not we need to bump txnID in when committing to allow MapTxn to keep reusing part.Txn. Rename 'prevTxnID' to 'nextTxnID' and move incrementing the txnID from 'Tree.Txn()' into 'Txn.Commit()'. Signed-off-by: Jussi Maki <jussi@isovalent.com>

part: Test MapTxn reusability with non-singleton

6e060d9

The MapTxn reuse test only tested for the singleton case. Extend to also test when both previous and new maps are not singletons. Signed-off-by: Jussi Maki <jussi@isovalent.com>

joamaki requested a review from bimmlerd January 26, 2026 15:54

joamaki requested a review from a team as a code owner January 26, 2026 15:54

bimmlerd approved these changes Jan 27, 2026

View reviewed changes

joamaki force-pushed the pr/joamaki/fix-maptxn-reusability branch from eff50b0 to 73f0877 Compare January 27, 2026 09:17

joamaki merged commit 037cd18 into main Jan 27, 2026
1 check passed

joamaki deleted the pr/joamaki/fix-maptxn-reusability branch January 27, 2026 09:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

part: Fix MapTxn reusability#140

part: Fix MapTxn reusability#140
joamaki merged 2 commits intomainfrom
pr/joamaki/fix-maptxn-reusability

joamaki commented Jan 26, 2026

Uh oh!

github-actions bot commented Jan 26, 2026 •

edited

Loading

Uh oh!

bimmlerd left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

joamaki commented Jan 26, 2026

Uh oh!

github-actions bot commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bimmlerd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions bot commented Jan 26, 2026 •

edited

Loading