[clickhouse-secondary-0] 2026.06.18 15:42:40.542970 [ 88 ] <Fatal> BaseDaemon: ########## Short fault info ############
[clickhouse-secondary-0] 2026.06.18 15:42:40.543029 [ 88 ] <Fatal> BaseDaemon: (version 25.8.16.10002.altinitystable (altinity build), build id: EEA8F02F2816016AC37620F31126A40D677A648A, git hash: c63afbef60f93b08179bb2f2727c8cc77391d3cc, architecture: x86_64) (from thread 1480) Received signal 11
[clickhouse-secondary-0] 2026.06.18 15:42:40.543041 [ 88 ] <Fatal> BaseDaemon: Signal description: Segmentation fault
[clickhouse-secondary-0] 2026.06.18 15:42:40.543073 [ 88 ] <Fatal> BaseDaemon: Address: 0x7fad35600038. Access: read. Address not mapped to object.
[clickhouse-secondary-0] 2026.06.18 15:42:40.543109 [ 88 ] <Fatal> BaseDaemon: Stack trace: 0x000000000ca3309f 0x0000000018af713a 0x00000000188330df 0x0000000018831b6a 0x0000000018833b1d 0x0000000018832233 0x0000000018b00458 0x000000001a00fbbd 0x000000001a00ed54 0x0000000015c091d3 0x0000000019d0dc5a 0x0000000019d2f742 0x0000000019d21290 0x0000000019d25143 0x00000000136f4feb 0x00000000136fc366 0x00000000136f1fd2 0x00000000136f9a9a 0x00007faf64127ac3 0x00007faf641b98d0
[clickhouse-secondary-0] 2026.06.18 15:42:40.543127 [ 88 ] <Fatal> BaseDaemon: ########################################
[clickhouse-secondary-0] 2026.06.18 15:42:40.544041 [ 88 ] <Fatal> BaseDaemon: (version 25.8.16.10002.altinitystable (altinity build), build id: EEA8F02F2816016AC37620F31126A40D677A648A, git hash: c63afbef60f93b08179bb2f2727c8cc77391d3cc) (from thread 1480) (query_id: <query_id>) (query: WITH
toDateTime64('2026-02-04 00:00:00', 3) AS start_ts,
toDateTime64('2026-02-05 00:00:00', 3) AS end_ts,
4294967295 AS uint32_max
SELECT
count() AS rows_checked,
countIf(
arrayExists(x -> ifNull((x < 0) OR (x > uint32_max), 0), `nested_column.counter_1`)
OR arrayExists(x -> ifNull((x < 0) OR (x > uint32_max), 0), `nested_column.counter_2`)
OR arrayExists(x -> ifNull((x < 0) OR (x > uint32_max), 0), `nested_column.counter_3`)
OR arrayExists(x -> ifNull((x < 0) OR (x > uint32_max), 0), `nested_column.counter_4`)
OR arrayExists(x -> ifNull((x < 0) OR (x > uint32_max), 0), `nested_column.counter_5`)
) AS invalid_rows
FROM large_replicated_table
PREWHERE (event_timestamp >= start_ts) AND (event_timestamp < end_ts)
WHERE notEmpty(`nested_column.name`)) Received signal Segmentation fault (11)
[clickhouse-secondary-0] 2026.06.18 15:42:40.544061 [ 88 ] <Fatal> BaseDaemon: Address: 0x7fad35600038. Access: read. Address not mapped to object.
[clickhouse-secondary-0] 2026.06.18 15:42:40.544073 [ 88 ] <Fatal> BaseDaemon: Stack trace: 0x000000000ca3309f 0x0000000018af713a 0x00000000188330df 0x0000000018831b6a 0x0000000018833b1d 0x0000000018832233 0x0000000018b00458 0x000000001a00fbbd 0x000000001a00ed54 0x0000000015c091d3 0x0000000019d0dc5a 0x0000000019d2f742 0x0000000019d21290 0x0000000019d25143 0x00000000136f4feb 0x00000000136fc366 0x00000000136f1fd2 0x00000000136f9a9a 0x00007faf64127ac3 0x00007faf641b98d0
[clickhouse-secondary-0] 2026.06.18 15:42:40.544130 [ 88 ] <Fatal> BaseDaemon: 2. memcpy @ 0x000000000ca3309f
[clickhouse-secondary-0] 2026.06.18 15:42:40.544183 [ 88 ] <Fatal> BaseDaemon: 3. void DB::filterArraysImpl<long>(DB::PODArray<long, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, DB::PODArray<long, 4096ul, Allocator<false, false>, 63ul, 64ul>&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 63ul, 64ul>&, DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, long) @ 0x0000000018af713a
[clickhouse-secondary-0] 2026.06.18 15:42:40.544210 [ 88 ] <Fatal> BaseDaemon: 4. COW<DB::IColumn>::immutable_ptr<DB::IColumn> DB::ColumnArray::filterNumber<long>(DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, long) const @ 0x00000000188330df
[clickhouse-secondary-0] 2026.06.18 15:42:40.544254 [ 88 ] <Fatal> BaseDaemon: 5. DB::ColumnArray::filter(DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, long) const @ 0x0000000018831b6a
[clickhouse-secondary-0] 2026.06.18 15:42:40.544280 [ 88 ] <Fatal> BaseDaemon: 6. DB::ColumnArray::filterNullable(DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, long) const @ 0x0000000018833b1d
[clickhouse-secondary-0] 2026.06.18 15:42:40.544303 [ 88 ] <Fatal> BaseDaemon: 7. DB::ColumnArray::filter(DB::PODArray<char8_t, 4096ul, Allocator<false, false>, 63ul, 64ul> const&, long) const @ 0x0000000018832233
[clickhouse-secondary-0] 2026.06.18 15:42:40.544320 [ 88 ] <Fatal> BaseDaemon: 8. DB::FilterDescription::filter(DB::IColumn const&, long) const @ 0x0000000018b00458
[clickhouse-secondary-0] 2026.06.18 15:42:40.544346 [ 88 ] <Fatal> BaseDaemon: 9. DB::FilterTransform::doTransform(DB::Chunk&) @ 0x000000001a00fbbd
[clickhouse-secondary-0] 2026.06.18 15:42:40.544367 [ 88 ] <Fatal> BaseDaemon: 10. DB::FilterTransform::transform(DB::Chunk&) @ 0x000000001a00ed54
[clickhouse-secondary-0] 2026.06.18 15:42:40.544387 [ 88 ] <Fatal> BaseDaemon: 11. DB::ISimpleTransform::transform(DB::Chunk&, DB::Chunk&) @ 0x0000000015c091d3
[clickhouse-secondary-0] 2026.06.18 15:42:40.544405 [ 88 ] <Fatal> BaseDaemon: 12. DB::ISimpleTransform::work() @ 0x0000000019d0dc5a
[clickhouse-secondary-0] 2026.06.18 15:42:42.012518 [ 88 ] <Fatal> BaseDaemon: Report this error to https://github.com/Altinity/ClickHouse/issues
Type of problem
Bug report: ClickHouse process crashed with SIGSEGV during a SELECT query.
Summary
We had a production incident on ClickHouse
25.8.16.10002.altinitystablewhere SELECT queries reading and filteringArray(Nullable(...))columns caused SIGSEGV on both primary and secondary replicas.The stack traces point to
ColumnArray::filterNullablewhile applying row filters to nested array data.The SIGSEGV sequence we observed was:
Array(Nullable(Int64))columns, filtered bynotEmpty(Array(Nullable(String))), caused SIGSEGV on the primary replica.We have not tested this against a ClickHouse Official build of the same version. The incident happened on an Altinity Stable build in a production Kubernetes cluster.
Environment
We observed crashes on both a primary replica and a secondary replica, from different query executions.
Table Shape
The full table schema is large. These are the relevant properties and column types:
Queries
This broad validation query caused SIGSEGV on the primary replica:
A similar broad validation query with the same column expressions caused SIGSEGV on the secondary replica, using a narrower date range:
The same date range with only `count()` succeeded:
Result:
Related `notEmpty(Array(Nullable(String)))` query that failed with a `4.00 EiB` allocation attempt:
This narrower query caused SIGSEGV on the secondary replica after the `4.00 EiB` allocation attempt:
We do not currently have a minimal reproducer or obfuscated sample data. The crashes happened while reading old production parts.
Expected Behavior
A
SELECTquery overArray(Nullable(...))columns should not crash the server.Observed Behavior
A related query on the secondary replica failed with an impossible memory allocation request before the narrower query crashed again. The same query log shows `MemoryTracker` reporting both current and peak query memory usage as `4.00 EiB`:
The broad primary-replica query crashed with SIGSEGV while filtering `Array(Nullable(Int64))` columns:
The similar broad secondary-replica query crashed with SIGSEGV while filtering `Array(Nullable(Int64))` columns:
The narrower secondary-replica query crashed with SIGSEGV while filtering `Array(Nullable(String))`:
Possible Related Code Path
Based on the stack traces and query shapes, the crashes happen while applying row filters to
Array(Nullable(...))columns. The relevant paths shown in the logs are:The
4.00 EiBallocation attempt may be related because it happened in the same investigation path while reading and filtering theArray(Nullable(String))column. We do not know whether it shares the same root cause as the SIGSEGVs.Questions
25.8.16.10002.altinitystable?ColumnArray::filterNullable, lazy materialization, or filtering ofArray(Nullable(...))columns?