Skip to content

Expose missing GUI scoring tools#7

Open
panDing19 wants to merge 1 commit into
LiberCoders:mainfrom
panDing19:fix/gui-missing-tool-metadata
Open

Expose missing GUI scoring tools#7
panDing19 wants to merge 1 commit into
LiberCoders:mainfrom
panDing19:fix/gui-missing-tool-metadata

Conversation

@panDing19

@panDing19 panDing19 commented Jun 25, 2026

Copy link
Copy Markdown

Problem

Several GUI tasks rely on app-direct GUI tools for scoring or expected actions, but the task metadata does not expose those tools. Agents running against the declared tool set cannot call tools that the benchmark expects them to use.

Affected tasks

  • TGUI01 through TGUI06
  • TGUI23
  • TGUI44
  • TGUI49
  • TGUI50

Fix

  • Add the missing GUI tool specs and endpoints to the affected task metadata.
  • Cover My Expenses, Loop Habit, Clock, Gmail Clone, Contacts, Fossify Calendar, and TestMall app-direct tools where required.
  • Add a regression test that checks the affected tasks expose both tools and tool_endpoints entries for the required scoring tools.

Tests

  • uv run pytest tests/test_gui_missing_tool_exposure.py -q

@panDing19 panDing19 marked this pull request as ready for review June 25, 2026 17:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant