Skip to content

Sanitize null bytes#779

Draft
tsg wants to merge 2 commits intov0.9.xfrom
sanitize_null_bytes
Draft

Sanitize null bytes#779
tsg wants to merge 2 commits intov0.9.xfrom
sanitize_null_bytes

Conversation

@tsg
Copy link
Member

@tsg tsg commented Mar 19, 2026

Description

This adds a new processor that can remove NULLs before writing to the target db. See #778

Related Issue(s)

Type of Change

Please select the relevant option(s):

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📚 Documentation update
  • 🔧 Refactoring (no functional changes)
  • ⚡ Performance improvement
  • 🧪 Test coverage improvement
  • 🔨 Build/CI changes
  • 🧹 Code cleanup

Changes Made

  • Add ne wprocessor
  • Add new config var

Checklist

  • Code follows project style guidelines
  • Self-review completed
  • Code is well-commented
  • Documentation updated where necessary

Additional Notes

@github-actions
Copy link

Merging this branch changes the coverage (1 decrease, 2 increase)

Impacted Packages Coverage Δ 🤖
github.com/xataio/pgstream/cmd/config 84.17% (+0.10%) 👍
github.com/xataio/pgstream/pkg/stream 35.78% (-0.18%) 👎
github.com/xataio/pgstream/pkg/wal/processor/sanitizer 88.89% (+88.89%) 🌟

Coverage by file

Changed files (no unit tests)

Changed File Coverage Δ Total Covered Missed 🤖
github.com/xataio/pgstream/cmd/config/config_env.go 90.39% (+0.08%) 229 (+2) 207 (+2) 22 👍
github.com/xataio/pgstream/cmd/config/config_yaml.go 89.06% (+0.06%) 192 (+1) 171 (+1) 21 👍
github.com/xataio/pgstream/pkg/stream/config.go 69.44% (ø) 36 25 11
github.com/xataio/pgstream/pkg/stream/stream.go 0.00% (ø) 129 (+3) 0 129 (+3)
github.com/xataio/pgstream/pkg/wal/processor/sanitizer/wal_sanitizer.go 88.89% (+88.89%) 27 (+27) 24 (+24) 3 (+3) 🌟

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.

Changed unit test files

  • github.com/xataio/pgstream/pkg/wal/processor/sanitizer/wal_sanitizer_test.go

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant