Fix smoke-copilot-safe-inputs workflow testing disabled tools by Copilot · Pull Request #7022 · github/gh-aw

Copilot · 2025-12-20T00:03:21Z

Workflow run 20385697614 reported 3/5 tools unavailable because the test prompt requested operations from tools that were intentionally disabled or misconfigured.

Changes

Removed GitHub MCP testing - Test asked agent to "Review merged pull requests" via GitHub MCP, but github: false disables it
Removed Serena from tools config - Was configured but not connecting reliably
Updated test prompt - Now correctly focuses on validating safe-inputs as GitHub MCP alternative

Before/After

# Before: Testing tools that are disabled
tools:
  github: false      # Disabled...
  serena: ["go"]     # Unreliable...
# But prompt asks: "GitHub MCP Testing: Review the last 2 merged pull requests"

# After: Test only what's enabled
tools:
  github: false
# Prompt tests: safeinputs-gh tool for GitHub CLI access

The workflow now tests its actual purpose: validating safeinputs-gh as an alternative to GitHub MCP.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

https://api.github.com/user
- Triggering command: /usr/bin/gh gh api user --jq .login ithub/workflows GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE %H %ct %D GOMODCACHE go env -java.git -java.git /bin/sh GOINSECURE GOMOD GOMODCACHE /bin/sh (http block)
- Triggering command: /usr/bin/gh gh api user --jq .login om/different-ownremote.origin.url GO111MODULE ps GOINSECURE GOMOD GOMODCACHE ps oad-�� oad-artifact.git GO111MODULE /usr/bin/gh GOINSECURE GOMOD GOMODCACHE gh (http block)
- Triggering command: /usr/bin/gh gh api user --jq .login by.git GO111MODULE 3a80683510af07837ca565c9e8f19209c04/log.json GOINSECURE GOMOD GOMODCACHE /opt/hostedtoolcache/go/1.25.0/x--no-legend -V=f�� ithub/workflows GOPROXY (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

investigate why this workflow failed: https://github.com/githubnext/gh-aw/actions/runs/20385697614 3 missing tools

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>

…abled tools Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>

Mossaka · 2025-12-20T00:31:25Z

@copilot merge main

Copilot · 2025-12-20T00:35:22Z

@copilot merge main

Branch is already up to date with main. No merge needed.

github-actions · 2025-12-20T00:39:43Z

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

github-actions · 2025-12-20T00:39:43Z

🤖 DIAGNOSTIC COMPLETE: Smoke Copilot No Firewall STATUS: ALL_UNITS_OPERATIONAL. MISSION_SUCCESS.

github-actions · 2025-12-20T00:39:44Z

🎬 THE END — Smoke Claude MISSION: ACCOMPLISHED! The hero saves the day! ✨

github-actions · 2025-12-20T00:39:44Z

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

github-actions · 2025-12-20T00:39:44Z

✅ Firewall validation complete... Smoke Codex Firewall confirmed network sandboxing is operational. 🛡️

github-actions · 2025-12-20T00:39:45Z

🎉 Yo ho ho! Changeset Generator found the treasure and completed successfully! ⚓💰

github-actions · 2025-12-20T00:39:45Z

🎉 Yo ho ho! Smoke Copilot Safe Inputs found the treasure and completed successfully! ⚓💰

github-actions · 2025-12-20T00:42:00Z

Smoke Test Results

✅ File Writing: Created /tmp/gh-aw/agent/smoke-test-copilot-20386361727.txt
✅ Bash Tool: Verified file content successfully
❌ Safe Input gh Tool: Tool not available (expected safeinputs-gh tool not found in available tools)

Overall Status: FAIL - safeinputs-gh tool is not available in the current environment

Ahoy! This treasure was crafted by 🏴‍☠️ Smoke Copilot Safe Inputs fer issue #7022 🗺️

github-actions · 2025-12-20T00:42:09Z

Smoke Test Results - Run 20386361725

Last 2 Merged PRs:

Add toolchains support for AWF sandbox containers #7008: Add toolchains support for AWF sandbox containers
Remove redundant structure explanation from quick start #7017: Remove redundant structure explanation from quick start

Test Results:

✅ GitHub MCP Testing (fetched merged PRs)
✅ File Writing Testing (agent temp file)
✅ Bash Tool Testing (verified file content)
✅ GitHub MCP Default Toolset Testing (get_me unavailable as expected)
✅ Cache Memory Testing (cache file created/verified)
✅ Available Tools Display: add_comment, add_labels, create_issue, missing_tool, noop

Overall Status: PASS

cc: @Mossaka (assignee)

📰 BREAKING: Report filed by Smoke Copilot fer issue #7022 🗺️

github-actions · 2025-12-20T00:42:11Z

Smoke Test Results: Copilot Engine (No Firewall)

Last 2 Merged PRs:

Add toolchains support for AWF sandbox containers #7008: Add toolchains support for AWF sandbox containers
Remove redundant structure explanation from quick start #7017: Remove redundant structure explanation from quick start

Test Results:

✅ GitHub MCP: Retrieved PR data successfully
✅ File Writing: Created /tmp/gh-aw/agent/smoke-test-copilot-20386361756.txt
✅ Bash Tool: Verified file contents
✅ Playwright MCP: Navigated to github.com, title contains "GitHub"
⚠️ Safe Input gh Tool: Skipped (no safeinputs-gh tool available in environment)

Overall Status: PASS (4/4 core tests passed)

🤖 DIAGNOSTIC REPORT GENERATED BY Smoke Copilot No Firewall fer issue #7022 🗺️

github-actions · 2025-12-20T00:42:37Z

OpenAI curl blocked: ✅
GitHub MCP merged PRs: #7017 Remove redundant structure explanation from quick start; #7016 Add info message for GitHub MCP projects toolset token requirement ✅
File write/read /tmp/gh-aw/agent/smoke-test-codex-firewall-20386361743.txt: ✅
example.com blocked: ✅
Network: SANDBOXED
Overall: PASS

🔥 Firewall tested by Smoke Codex Firewall fer issue #7022 🗺️

github-actions · 2025-12-20T00:43:38Z

PRs reviewed: Remove redundant structure explanation from quick start; Add info message for GitHub MCP projects toolset token requirement
GitHub MCP review ✅
Agent file write/read ✅
Playwright title check ✅
Cache memory write/read ✅
safeinputs-gh (gh issues list --limit 3) ❌ (command not found in env)
Overall: FAIL

🔮 The oracle has spoken through Smoke Codex fer issue #7022 🗺️

Copilot AI self-assigned this Dec 20, 2025

Initial plan

79773ad

Copilot AI assigned Mossaka Dec 20, 2025

Copilot started work on behalf of Mossaka December 20, 2025 00:03 View session

Copilot AI and others added 2 commits December 20, 2025 00:12

Investigation: smoke-copilot-safe-inputs workflow 3 missing tools

ed9d9c8

Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>

Fix smoke-copilot-safe-inputs workflow - remove invalid tests for dis…

2054007

…abled tools Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>

Mossaka marked this pull request as ready for review December 20, 2025 00:31

Merge branch 'main' into copilot/investigate-workflow-failure

449dc51

Copilot AI changed the title ~~[WIP] Investigate missing tools in failed workflow~~ Fix smoke-copilot-safe-inputs workflow testing disabled tools Dec 20, 2025

Copilot AI requested a review from Mossaka December 20, 2025 00:32

Copilot finished work on behalf of Mossaka December 20, 2025 00:32

Copilot started work on behalf of Mossaka December 20, 2025 00:32 View session

Copilot finished work on behalf of Mossaka December 20, 2025 00:36

Mossaka added the smoke label Dec 20, 2025

github-actions bot added the smoke-copilot label Dec 20, 2025

github-actions bot added the smoke-copilot-no-firewall label Dec 20, 2025

github-actions bot added the smoke-codex-firewall label Dec 20, 2025

pelikhan merged commit e05f772 into main Dec 20, 2025
137 checks passed

pelikhan deleted the copilot/investigate-workflow-failure branch December 20, 2025 01:35

github-actions bot mentioned this pull request Dec 20, 2025

Fix GH_AW_WORKFLOW_ID propagation to safe_outputs job for create_pull_request #7029

Merged

Conversation

Copilot AI commented Dec 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Before/After

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Mossaka commented Dec 20, 2025

Uh oh!

Copilot AI commented Dec 20, 2025

Uh oh!

github-actions bot commented Dec 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 20, 2025 • edited by agentic-workflows-dev bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 20, 2025

Smoke Test Results

Uh oh!

github-actions bot commented Dec 20, 2025

Smoke Test Results - Run 20386361725

Uh oh!

github-actions bot commented Dec 20, 2025

Smoke Test Results: Copilot Engine (No Firewall)

Uh oh!

github-actions bot commented Dec 20, 2025

Uh oh!

github-actions bot commented Dec 20, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Dec 20, 2025 •

edited

Loading

github-actions bot commented Dec 20, 2025 •

edited

Loading

github-actions bot commented Dec 20, 2025 •

edited

Loading

github-actions bot commented Dec 20, 2025 •

edited

Loading

github-actions bot commented Dec 20, 2025 •

edited

Loading

github-actions bot commented Dec 20, 2025 •

edited

Loading

github-actions bot commented Dec 20, 2025 •

edited by agentic-workflows-dev bot

Loading

github-actions bot commented Dec 20, 2025 •

edited

Loading