Skip to content

feat(config): support a cached logs location#875

Open
mcncl wants to merge 3 commits into
mainfrom
feat/buildkite_logs_factory
Open

feat(config): support a cached logs location#875
mcncl wants to merge 3 commits into
mainfrom
feat/buildkite_logs_factory

Conversation

@mcncl

@mcncl mcncl commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Description

This allows for configuration options for job log caching to be set, this will eventually allow for speedier resolution of job logs and allow for the CLI and MCP to work in harmony 💞

This is infrastructure only — no commands use Parquet yet. Follow-up PRs can wire bk job log --search / --tail via f.LogsClient.NewReaderByJobID.

Changes

  • add github.com/buildkite/buildkite-logs v0.10.0 and Factory.LogsClient
    • initialized alongside RestAPIClient
  • add internal/logs wrapper that shares the CLI's go-buildkite client (auth + HTTP transport)
  • add config for Parquet log cache, compatible with the MCP server:
    • log_cache_url / BKLOG_CACHE_URL (default: ~/.bklog via buildkite-logs)
    • log_max_bytes / BKLOG_MAX_LOG_BYTES (optional; library default when unset)
  • expose new keys via bk config get|set|list

Testing

  • Tests have run locally (with go test ./...)
  • Code is formatted (with go fmt ./...)
  • bk config set log_cache_url file://~/.bklog persists and bk config get log_cache_url returns it

@mcncl mcncl requested review from a team as code owners June 17, 2026 00:29
@mcncl mcncl requested a review from buildsworth-bk June 17, 2026 00:30
@socket-security

socket-security Bot commented Jun 17, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedgithub.com/​buildkite/​buildkite-logs@​v0.10.096100100100100

View full report

@socket-security

socket-security Bot commented Jun 17, 2026

Copy link
Copy Markdown

Warning

Review the following alerts detected in dependencies.

According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.

Action Severity Alert  (click "▶" to expand/collapse)
Warn High
Obfuscated code: golang github.com/google/flatbuffers is 90.0% likely obfuscated

Confidence: 0.90

Location: Package overview

From: ?golang/github.com/buildkite/buildkite-logs@v0.10.0golang/github.com/google/flatbuffers@v25.12.19+incompatible

ℹ Read more on: This package | This alert | What is obfuscated code?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore golang/github.com/google/flatbuffers@v25.12.19+incompatible. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

View full report

@buildsworth-bk-app

Copy link
Copy Markdown

On it! Follow along with my review here: https://buildkite.com/buildkite/buildsworth-review-pull-request/builds/789

@buildsworth-bk-app buildsworth-bk-app Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read through the new cache config and buildkite-logs wrapper. I found a couple of non-blocking edge cases in the config/URL plumbing; this is L2 for me because it adds a dependency-backed client path rather than just config metadata.

Want to dig deeper? The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:

Download the buildsworth logs from build 789, then answer my questions about the findings.

Comment thread internal/config/config.go Outdated
Comment thread internal/logs/cache_url.go Outdated
mcncl added 2 commits June 17, 2026 12:42
Reject negative BKLOG_MAX_LOG_BYTES env values so they fall through to
user config instead of disabling the size limit. Preserve file:// query
params such as no_tmp_dir=true when normalizing cache URLs.
@mcncl

mcncl commented Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

@buildsworth-bk throw ya boy another review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant