Skip to content

Add autosolve actions for automated issue resolution#14

Open
fantapop wants to merge 1 commit intomainfrom
pr/autosolve-go
Open

Add autosolve actions for automated issue resolution#14
fantapop wants to merge 1 commit intomainfrom
pr/autosolve-go

Conversation

@fantapop
Copy link
Copy Markdown
Contributor

@fantapop fantapop commented Mar 25, 2026

Summary

Go implementation of composite actions for Claude-powered automated issue resolution:

  • autosolve/assess — Runs Claude in read-only mode to evaluate whether a task is suitable for automated resolution
  • autosolve/implement — Runs Claude to implement a solution, validates changes, runs AI security review, pushes to a fork, and creates a draft PR

Key features

  • Precompiled Go binary (no Go toolchain needed at runtime)
  • Per-file batched AI security review with generated-file detection
  • Token usage tracking across phases with combined markdown summary
  • Retry logic with Claude session resumption
  • Skill file support for custom prompts

Testing

Tested end-to-end against cockroachlabs/ccloud-private-automation-testing.

Test plan

  • go test ./... passes
  • Precompiled binary check passes in CI

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a new autosolve Go-based automation tool and two composite GitHub Actions (autosolve/assess and autosolve/implement) to assess issue suitability and implement fixes with Claude, including PR creation, security checks, and usage tracking.

Changes:

  • Add Go implementation for assessment/implementation orchestration, prompt assembly, git/gh wrappers, and security checks.
  • Add composite actions (autosolve/assess, autosolve/implement) plus CI updates to run Go tests and validate the precompiled binary.
  • Add prompt templates and unit tests for core functionality.

Reviewed changes

Copilot reviewed 28 out of 30 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
autosolve/internal/security/security_test.go Adds unit tests for blocked-path and sensitive-file enforcement and .gitignore warnings.
autosolve/internal/security/security.go Implements blocked path checks, sensitive filename/extension detection, and symlink-to-blocked-path detection.
autosolve/internal/prompt/templates/security-preamble.md Adds system preamble intended to constrain the model’s behavior for safety.
autosolve/internal/prompt/templates/implementation-footer.md Adds implementation-phase instruction footer and required success/fail marker.
autosolve/internal/prompt/templates/assessment-footer.md Adds assessment-phase instruction footer and required proceed/skip marker.
autosolve/internal/prompt/prompt_test.go Adds tests for prompt construction, skill file inclusion, and custom criteria.
autosolve/internal/prompt/prompt.go Implements prompt assembly from templates + task inputs.
autosolve/internal/implement/implement_test.go Adds tests for retry logic, output writing, and summary extraction.
autosolve/internal/implement/implement.go Implements the implementation phase: retries, security checks, staging/commit/push, PR creation, and AI security review.
autosolve/internal/github/github.go Adds a gh-CLI-backed GitHub client for comments/labels/PR creation.
autosolve/internal/git/git.go Adds a git CLI abstraction and helper to list changed files.
autosolve/internal/config/config_test.go Adds tests for config parsing/validation and blocked path parsing.
autosolve/internal/config/config.go Adds config loading/validation from action inputs and auth validation.
autosolve/internal/claude/claude_test.go Adds tests for extracting markers/session IDs and usage tracking.
autosolve/internal/claude/claude.go Adds Claude CLI runner + result parsing + usage tracking persistence.
autosolve/internal/assess/assess_test.go Adds tests for assessment flow and summary extraction.
autosolve/internal/assess/assess.go Implements assessment phase invocation and outputs/summary writing.
autosolve/internal/action/action_test.go Adds tests for GitHub Actions output and step summary helpers.
autosolve/internal/action/action.go Adds helpers for outputs, summaries, and workflow annotations.
autosolve/implement/action.yml Defines the composite action to run autosolve implement and expose outputs.
autosolve/go.mod Introduces the autosolve Go module definition.
autosolve/cmd/autosolve/main.go Adds CLI entrypoint for assess and implement commands.
autosolve/build.sh Adds cross-compile script producing the committed Linux binary.
autosolve/assess/action.yml Defines the composite action to run autosolve assess and expose outputs.
autosolve/Makefile Adds build/test/clean targets for local development and CI.
autosolve/.gitignore Ignores the local dev binary output.
CHANGELOG.md Documents the addition of the autosolve actions.
.github/workflows/test.yml Updates CI to run Go tests and ensure the precompiled binary is up to date.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@fantapop fantapop requested a review from linhcrl March 25, 2026 01:35
@fantapop fantapop force-pushed the pr/autosolve-go branch 2 times, most recently from 1abbbb0 to 6fd24ba Compare March 25, 2026 06:52
@fantapop fantapop force-pushed the pr/autosolve-go branch 6 times, most recently from f818651 to 6bc6bc5 Compare March 27, 2026 22:21
@fantapop
Copy link
Copy Markdown
Contributor Author

One thing I'm running into here is that build the action and committing it each time easily gets out of date and is annoying. I'm going to look into alternatives.

- autosolve/assess: evaluate tasks for automated resolution suitability
  using Claude in read-only mode.
- autosolve/implement: autonomously implement solutions, validate
  security, push to fork, and create PRs using Claude. Includes AI
  security review, token usage tracking, and per-file batched diff
  analysis.
- Prefers roachdev wrapper for Claude CLI when available, falls back
  to native installer.
- Go binary is built from source at action runtime via setup-go.

Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants