HBASE-30242 Add Region server level HFile Compression Metrics#8389
HBASE-30242 Add Region server level HFile Compression Metrics#8389Umeshkumar9414 wants to merge 1 commit into
Conversation
Co-authored-by: Opus 4.6 <noreply@anthropic.com>
ee2c6d6 to
40b6aa3
Compare
| } | ||
|
|
||
| @Override | ||
| public double getStoreFileCompressionRatio() { |
There was a problem hiding this comment.
Generally for compression factor, higher is better. This is what command line compression tools like xz or zstd or etc. typically report. And for compression savings, higher is also better, e.g 1 − compressed / uncompressed, which is easy to read as a percentage. This method provides the inverse of that so it makes the desired direction "down", which is awkward for Prometheus or Grafana dashboards. Operators reading a dashboard tile labeled storeFileCompressionRatio with a value e.g. of 0.33 will routinely interpret it the opposite of what is intended.
Recommendation: either rename to storeFileCompressedToUncompressedRatio so the direction is clearly established by the name of the metric, or reimplement as uncompressed / compressed, or storeFileCompressionSavingsRatio as (1 − compressed/uncompressed).
| @Override | ||
| public double getStoreFileCompressionRatio() { | ||
| long uncompressed = aggregate.storeFileUncompressedSize; | ||
| if (uncompressed == 0) { |
There was a problem hiding this comment.
A value of "1.0" will generally look identical to an operator like somehow compression has stopped working, and will generate false positive alerts.
(Claude calls this a "footgun for dashboards", lol. He means well...)
Recommendation: Return 0.0 instead, and update the description to something like "Returns 0.0 when there is no data."
There was a problem hiding this comment.
Pull request overview
Adds RegionServer-level metrics for total uncompressed HFile bytes and an overall store-file compression ratio, surfaced through the existing MetricsRegionServer wrapper/source plumbing and validated via unit tests.
Changes:
- Extend
MetricsRegionServerWrapperwithgetStoreFileUncompressedSize()andgetStoreFileCompressionRatio(). - Compute/aggregate uncompressed store bytes in
MetricsRegionServerWrapperImpland export both gauges viaMetricsRegionServerSourceImpl. - Update RegionServer metrics unit tests and wrapper stubs to cover the new gauges.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java | Asserts the two new gauges are emitted by the metrics source. |
| hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperStub.java | Adds stub implementations returning deterministic values for the new metrics. |
| hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java | Aggregates uncompressed bytes from stores and computes RS-level compression ratio. |
| hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapper.java | Extends wrapper interface with uncompressed size and compression ratio accessors. |
| hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java | Exports the new wrapper values as gauges in the metrics record. |
| hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java | Defines metric names and descriptions for the two new gauges. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| /** | ||
| * Get the compression ratio of store files on this region server. This is the ratio of compressed | ||
| * on-disk size to uncompressed data size. Returns 1.0 when there is no data. | ||
| */ | ||
| double getStoreFileCompressionRatio(); |
| String STOREFILE_UNCOMPRESSED_SIZE_DESC = "Total uncompressed size of storefiles being served."; | ||
| String STOREFILE_COMPRESSION_RATIO_DESC = | ||
| "Compression ratio of storefiles (compressed/uncompressed). Lower values indicate better" | ||
| + " compression. Returns 1.0 when there is no data."; |
Local testing

Note : In case of compression NONE, ratio goes above 1 becuase storeFileSize includes HFile trailer/index overhead that isn't counted in the uncompressed data blocks.