Skip to content

Add mimir_utils TSDB metric analyzer CLI#227

Open
kdeal wants to merge 2 commits intomainfrom
codex/create-new-cli-tool-mimir_utils
Open

Add mimir_utils TSDB metric analyzer CLI#227
kdeal wants to merge 2 commits intomainfrom
codex/create-new-cli-tool-mimir_utils

Conversation

@kdeal
Copy link
Owner

@kdeal kdeal commented Jan 4, 2026

Summary

  • add a new mimir_utils Go CLI project to house TSDB analysis utilities
  • implement a top-metrics subcommand that scans TSDB blocks and reports the largest metrics by stored bytes
  • display byte usage in a human-readable format in the CLI output
  • document the new project alongside the other utilities in the repository README

Codex Task

Summary by CodeRabbit

  • New Features

    • Added mimir_utils command-line tool for analyzing Mimir and Prometheus time-series databases.
    • New top-metrics subcommand lists largest metrics by storage usage with bytes, series, and chunks.
  • Documentation

    • Added README section describing the new utilities.
  • Tests

    • Added unit test validating human-readable byte formatting.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jan 4, 2026

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

This pull request adds a new Go module mimir_utils implementing command-line utilities to inspect Mimir/Prometheus TSDB blocks. It introduces module config (go.mod), a CLI entrypoint and dispatcher, a top-metrics subcommand that scans block directories, reads index and chunk data to aggregate per-metric bytes/series/chunks, a human-readable byte formatter, and a unit test for that formatter.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Add mimir_utils TSDB metric analyzer CLI' clearly and concisely summarizes the main change: introducing a new Go CLI tool for analyzing TSDB metrics.
✨ Finishing touches
  • 📝 Generate docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
mimir_utils/internal/cli/top_metrics.go (1)

53-71: Minor: exp == 0 condition is unreachable dead code.

Since values < 1024 return early at line 54-56, exp will always be ≥ 1 when reaching line 67. The exp == 0 check and the "B" entry in the units slice (index 0) are effectively dead code.

Consider simplifying by removing the unreachable branch and starting units at "KiB":

🔎 Proposed simplification
 func humanReadableBytes(bytes int64) string {
 	if bytes < 1024 {
 		return fmt.Sprintf("%d B", bytes)
 	}

 	const unit = 1024.0
-	units := []string{"B", "KiB", "MiB", "GiB", "TiB", "PiB"}
+	units := []string{"KiB", "MiB", "GiB", "TiB", "PiB"}
 	val := float64(bytes)
 	exp := math.Floor(math.Log(val) / math.Log(unit))
-	if int(exp) >= len(units) {
-		exp = float64(len(units) - 1)
+	idx := int(exp) - 1 // Adjust since units starts at KiB
+	if idx >= len(units) {
+		idx = len(units) - 1
 	}

-	value := val / math.Pow(unit, exp)
-	if value >= 10 || exp == 0 {
+	value := val / math.Pow(unit, float64(idx+1))
+	if value >= 10 {
 		return fmt.Sprintf("%.0f %s", value, units[int(exp)])
 	}
-	return fmt.Sprintf("%.1f %s", value, units[int(exp)])
+	return fmt.Sprintf("%.1f %s", value, units[idx])
 }

Alternatively, keeping the current code works correctly—just noting the redundancy.

mimir_utils/internal/analyzer/top_metrics.go (2)

58-80: Consider returning empty slice instead of error when no blocks found.

Returning an error when no blocks exist prevents distinguishing between "directory doesn't exist" (actual error) and "directory exists but is empty" (valid state). The CLI already handles empty results gracefully with "No metrics found."

🔎 Proposed change
 	if len(blocks) == 0 {
-		return nil, fmt.Errorf("no TSDB blocks found in %s", root)
+		return blocks, nil
 	}

This makes the API more composable—callers can decide how to handle empty directories.


121-127: Fragile error handling using string matching.

Checking strings.Contains(err.Error(), "reference") is brittle—error messages can change between library versions. Both branches also produce nearly identical error messages ("chunk %d" vs "read chunk %d").

Consider simplifying to a single error format or using error type assertions if the library exports specific error types.

🔎 Proposed simplification
 		chk, err := chunkReader.Chunk(meta.Ref)
 		if err != nil {
-			if strings.Contains(err.Error(), "reference") {
-				return fmt.Errorf("chunk %d: %w", meta.Ref, err)
-			}
-			return fmt.Errorf("read chunk %d: %w", meta.Ref, err)
+			return fmt.Errorf("read chunk ref %d: %w", meta.Ref, err)
 		}

This also allows removing the "strings" import if not used elsewhere.

📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f388878 and ad156e3.

📒 Files selected for processing (7)
  • README.md
  • mimir_utils/cmd/mimir_utils/main.go
  • mimir_utils/go.mod
  • mimir_utils/internal/analyzer/top_metrics.go
  • mimir_utils/internal/cli/main.go
  • mimir_utils/internal/cli/top_metrics.go
  • mimir_utils/internal/cli/top_metrics_test.go
🧰 Additional context used
🧬 Code graph analysis (2)
mimir_utils/internal/cli/top_metrics.go (1)
mimir_utils/internal/analyzer/top_metrics.go (1)
  • TopNMetrics (25-56)
mimir_utils/cmd/mimir_utils/main.go (1)
mimir_utils/internal/cli/main.go (2)
  • RootUsage (10-20)
  • Execute (23-39)
🔇 Additional comments (11)
README.md (1)

19-21: LGTM!

The documentation clearly describes the new utility and follows the existing pattern for documenting other tools in the repository.

mimir_utils/internal/cli/main.go (2)

9-20: LGTM!

The usage text is clear and follows standard CLI conventions. The function correctly outputs to flag.CommandLine.Output() which respects -h flag handling.


22-39: LGTM!

The dispatcher correctly handles subcommands, help flags, and provides appropriate error messages for missing or unknown subcommands. The implementation is clean and idiomatic.

mimir_utils/cmd/mimir_utils/main.go (1)

11-18: LGTM!

The entry point correctly wires the CLI usage handler and delegates execution. Error handling follows Go best practices by writing to stderr and exiting with a non-zero status.

mimir_utils/internal/cli/top_metrics_test.go (1)

5-29: LGTM!

The test is well-structured with comprehensive edge case coverage including boundaries (0, 1, 1023, 1024), fractional values, multiple unit scales, and overflow scenarios. The table-driven approach is idiomatic Go.

mimir_utils/go.mod (1)

3-3: Go 1.25.1 is a valid, released version.

Go 1.25.1 exists as a legitimate patch release (released September 3, 2025) that includes security fixes and bug fixes. The version specified in go.mod is appropriate.

mimir_utils/internal/cli/top_metrics.go (2)

13-33: LGTM on flag parsing and validation.

Clean implementation using flag.NewFlagSet with ContinueOnError, proper usage message, and required flag validation.


35-51: LGTM on analyzer integration and output.

Good handling of empty results, proper use of tabwriter for aligned output, and correct error propagation from w.Flush().

mimir_utils/internal/analyzer/top_metrics.go (3)

16-22: LGTM on MetricStat struct.

Clean struct definition with appropriate field types and good documentation.


24-56: LGTM on TopNMetrics aggregation and sorting logic.

Deterministic sorting (bytes descending, name ascending for ties) and proper limit handling.


82-146: LGTM on overall accumulateBlock structure.

Good resource management with defer Close(), proper error propagation with context wrapping, and correct postings iteration pattern with postings.Err() check.


go 1.25.1

require github.com/prometheus/prometheus v0.56.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🌐 Web query:

prometheus v0.56.0 release

💡 Result:

Do you mean Prometheus (the server), prometheus-operator, prometheus/common, or another Prometheus-related project? Which repository/version exactly are you asking about (e.g., prometheus v2.x, prometheus-operator v0.56.0, or prometheus/common v0.56.0)?


🌐 Web query:

github.com/prometheus/prometheus releases v0.56.0

💡 Result:

There is no release named v0.56.0 on the prometheus/prometheus Releases page. See the Prometheus releases list. [1]

(For context: several related Prometheus repositories do use 0.x versioning — e.g. prometheus/common and prometheus-operator have v0.56.0 tags/releases — but those are different repos, not prometheus/prometheus. [2][3])

Sources:

  • Prometheus releases (prometheus/prometheus). [1]
  • prometheus/common releases. [2]
  • prometheus-operator releases (example v0.56.0). [3]

Remove or correct the Prometheus dependency version—v0.56.0 does not exist for github.com/prometheus/prometheus.

The specified version v0.56.0 is not a valid release for github.com/prometheus/prometheus. The Prometheus server repository uses v2.x versioning. While v0.56.0 exists for other Prometheus-related packages (such as prometheus/common or prometheus-operator), it does not exist for the main prometheus/prometheus repository. This will cause module resolution failures. Verify and update the version to a valid release (e.g., a v2.x.x version) or correct the dependency package name if a different Prometheus library was intended.

🤖 Prompt for AI Agents
In mimir_utils/go.mod around line 5, the require entry
"github.com/prometheus/prometheus v0.56.0" references a non-existent release for
that module; replace it with a valid prometheus/prometheus v2.x.y tag (or remove
the dependency if the intent was to use a different prom-related package), run
"go get github.com/prometheus/prometheus@v2.x.y" to update go.mod/go.sum, and
verify builds to ensure module resolution succeeds.

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ad156e3544

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines 1 to 5
module mimir_utils

go 1.25.1

require github.com/prometheus/prometheus v0.56.0

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Add go.sum so prometheus dependency can build

go.mod adds github.com/prometheus/prometheus, but no go.sum was committed. Running go test ./... fails with “missing go.sum entry for module providing package github.com/prometheus/prometheus/tsdb/chunkenc” because the default readonly module mode refuses to generate sums. Please add the go.sum entries (or vendor the modules) so the new CLI can compile and test.

Useful? React with 👍 / 👎.

Comment on lines +18 to +19
{"many gibibytes", 25 * 1024 * 1024 * 1024, "25 GiB"},
{"overflow past units", 1 << 62, "4 PiB"},

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Correct overflow expectation in humanReadableBytes test

The overflow case feeds 1<<62 bytes (≈4 EiB) but expects the formatter to return "4 PiB". humanReadableBytes actually returns 4096 PiB for that input, so this test will fail and the expectation underreports the size by 1024×. Either extend the formatter to emit EiB (e.g., "4 EiB") or adjust the test input to match the PiB expectation.

Useful? React with 👍 / 👎.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between ad156e3 and b2fa414.

📒 Files selected for processing (3)
  • mimir_utils/go.mod
  • mimir_utils/internal/analyzer/top_metrics.go
  • mimir_utils/internal/cli/top_metrics.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • mimir_utils/internal/cli/top_metrics.go
  • mimir_utils/internal/analyzer/top_metrics.go
🔇 Additional comments (1)
mimir_utils/go.mod (1)

3-3: No action needed. Go version 1.25.1 is a stable, officially released version. As of January 2026, the latest stable release is Go 1.25.5 (December 2, 2025), confirming that 1.25.1 is a valid and established release within the standard Go versioning timeline.

Likely an incorrect or invalid review comment.


go 1.25.1

require github.com/prometheus/prometheus/v3 v3.8.1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

find . -name "go.sum" | head -20

Repository: kdeal/misc

Length of output: 36


🌐 Web query:

Prometheus prometheus/prometheus v3.8.1 release

💡 Result:

Prometheus v3.8.1 was released on December 16, 2025. It contains a bugfix for the Remote Write receiver (fixes wrong response headers for v1 flow that could cause false partial error logs/metrics) (PR/issue #17683). Binaries and checksums are available on the official download and GitHub release pages. (github.com)

Citations:


🌐 Web query:

Go 1.25.1 version release

💡 Result:

Go 1.25.1 was released on September 3, 2025. It’s a minor/patch release that includes security fixes (notably a net/http issue) and several bug fixes to the go command and packages such as net, os, os/exec, and testing/synctest. [1][2]

Sources:

  • Go release history (lists go1.25.1 — released 2025-09-03). [1]
  • Go 1.25 release notes / 1.25.1 summary. [2]

Add go.sum file to the mimir_utils module.

The Prometheus v3.8.1 dependency is valid and corrects the prior issue with v0.56.0. However, the go.sum file is still missing from the repository. Go modules require both go.mod and go.sum to ensure reproducible builds and verify dependency checksums. Generate and commit the go.sum file using go mod tidy before merging.

🤖 Prompt for AI Agents
In mimir_utils/go.mod around line 5, the module now requires
github.com/prometheus/prometheus/v3 v3.8.1 but the repository is missing the
corresponding go.sum file; run go mod tidy in the mimir_utils module root to
generate/update go.sum, verify modules resolve successfully, and commit the
generated go.sum to the repo before merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant