KAFKA-18309: Change log.segment.bytes config type from INT to LONG

[acgtun]

Motivation

The log.segment.bytes config (and its topic-level synonym segment.bytes) is currently defined as INT type, which limits the maximum configurable segment size to Integer.MAX_VALUE (~1.99 GB). This is an artificial limitation for users who want larger log segments.

Changes

Change the config type from INT to LONG across the config definition, storage, and propagation layers:

File	Change
LogConfig.java	DEFAULT_SEGMENT_BYTES → long; config defs INT → LONG; segmentSize() returns long; initFileSize() safely caps at Integer.MAX_VALUE
AbstractKafkaConfig.java	logSegmentBytes() → Long via getLong()
RollParams.java	maxSegmentBytes → long
Cleaner.java	groupSegmentsBySize() maxSize param → long
Test utils (Scala + Java)	Updated parameter types to Long/long

Compatibility

• Backward compatible: Existing INT values (stored/transmitted as strings like "1073741824") parse correctly as LONG.
• Forward compatible: Values ≤ Integer.MAX_VALUE work on older brokers. Values > Integer.MAX_VALUE require the new version.
• No wire protocol impact — this is a broker/topic-level config only, not part of any Kafka RPC schema.

Testing

All existing unit and integration tests pass, including LogConfigTest, KafkaConfigTest, UnifiedLogTest, LocalLogTest, LogCleanerManagerTest, KafkaRaftLogTest, DefaultSupportedConfigCheckerTest, and StaticBrokerConfigTest.

[nileshkumar3]

The config type change and initFileSize() cap look good. One concern: LogSegment.size() and FileRecords.sizeInBytes() both return int (backed by AtomicInteger), so values above Integer.MAX_VALUE are accepted by the config but can't actually be enforced by the storage layer.

Specifically, shouldRoll() compares int size against long maxSegmentBytes — when maxSegmentBytes > ~2 GB, the int is promoted to long for comparison but can never exceed it, so size-based rolling silently becomes a no-op. Similarly, the batch-size guard in UnifiedLog (validRecords.sizeInBytes() > config().segmentSize()) becomes unreachable since sizeInBytes() returns int.

Consider adding between(1024 * 1024, Integer.MAX_VALUE) as the validator so the type is LONG (future-proof), but the range is honest about what the storage layer supports today. The upper bound can be lifted later when FileRecords and LogSegment are widened to long.

[chia7712]

@acgtun Please check and fix the JIRA number in the PR title. Also, this change requires a KIP (Kafka Improvement Proposal). Since the current .index file format uses a 4-byte integer for the physical position, it cannot support segments larger than 2GB. A KIP is needed to discuss how to upgrade the index format.

[acgtun]

@acgtun Please check and fix the JIRA number in the PR title. Also, this change requires a KIP (Kafka Improvement Proposal). Since the current .index file format uses a 4-byte integer for the physical position, it cannot support segments larger than 2GB. A KIP is needed to discuss how to upgrade the index format.

Thanks @chia7712. If the .index file format only uses 4-bytes and update the index format, this could be much more complicated.