Skip to content

MST-9509: relax connector fallback criteria#636

Closed
tmatup wants to merge 1 commit intomainfrom
fix-MST-9509
Closed

MST-9509: relax connector fallback criteria#636
tmatup wants to merge 1 commit intomainfrom
fix-MST-9509

Conversation

@tmatup
Copy link
Copy Markdown
Member

@tmatup tmatup commented May 7, 2026

Summary

Connector-feature tasks now accept the managed HTTP fallback shape that the prompt tells agents to use when a connector operation is absent from the registry.

This relaxes non-load-bearing fallback evidence for generate_schema and path_params, adds method_override fallback coverage for Azure AVD retrieveRegistrationToken, and updates method_override.yaml to use the shared fallback checker instead of a connector-key-only assertion.

Validation

  • Ran the updated fallback checker against captured run artifacts for skill-flow-ipe-generate-schema.
  • Ran the updated fallback checker against captured run artifacts for skill-flow-ipe-path-params.
  • Ran the updated fallback checker against captured run artifacts for skill-flow-ipe-method-override.
  • Validated changed connector-feature YAMLs against TaskDefinition.
  • git diff --check

Jira: https://uipath.atlassian.net/browse/MST-9509

Root-cause from run 2026-05-07_06-02-52:\n\nMST-9509: generate_schema and path_params produced valid managed HTTP fallback flows but failed on non-load-bearing literal evidence (timespan, tasklistid, taskid). method_override produced a valid Azure AVD retrieveRegistrationToken HTTP fallback but still used the old connector-key-only assertion. Relaxed the fallback checker to endpoint/method evidence and added method_override fallback coverage.\n\nVerified: updated checker passes against captured artifacts for skill-flow-ipe-generate-schema, skill-flow-ipe-path-params, and skill-flow-ipe-method-override; YAML validates with validate_yaml.py; git diff --check.
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 7, 2026

Claude finished @tmatup's task in 2m 1s —— View job


Coder-eval task lint (advisory)

1 task YAML changed; 0 Critical, 0 High, 1 Medium, 0 Low, 0 OK.

Rubric: .claude/commands/lint-task.md. This check is advisory and never blocks merge.

Evidence of passing run

  • ✅ Author confirms passing run: "Ran the updated fallback checker against captured run artifacts for skill-flow-ipe-method-override."

Per-task lint

tests/tasks/uipath-maestro-flow/connector_features/method_override.yaml — verdict: Medium

Issues:

  • [Medium] Validate-only flow tests miss correctness: integration-tier task tagged uipath-maestro-flow with no command_executed matching flow\s+debug (lines 62–68). Sandbox is tempdir-only so live execution is unavailable, but this trade-off isn't documented in the description field (lines 2–4).

Suggested fixes:

  • Append rationale to the description field, e.g.: "…on the Azure connector. Validate-only: no live tenant in tempdir sandbox." — this earns the description-rationale carve-out and downgrades the finding to Low.

Within-PR duplicates

  • No duplicate clusters detected (only one task YAML changed).

Conclusion

  • ⚠ 1 task has issues, max severity Medium. Advisory only — not blocking merge. The validate-only finding is systemic across the connector_features/ suite and can be addressed in bulk by adding rationale to each task's description.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 7, 2026

Claude finished @tmatup's task in 1m 46s —— View job


Review: MST-9509 — relax connector fallback criteria

  • Read review criteria and project conventions
  • Read full diff and changed files
  • Read existing PR conversation
  • Analyze changes and provide review feedback

Summary

Relaxes non-load-bearing evidence in the shared check_managed_http_fallback.py checker for generate_schema and path_params, adds a new _check_method_override function for Azure AVD retrieveRegistrationToken, and migrates method_override.yaml from an inline connector-key assertion to the shared checker — matching the pattern already used by the other three connector feature tasks.

Change-by-Change Review

1. tests/tasks/uipath-maestro-flow/connector_features/check_managed_http_fallback.py

Severity: OK

Three logical changes, all clean:

  • _check_generate_schema (line 47): Drops "timespan" from required needles. The remaining evidence (api.applicationinsights.io, /query, method, post, query) uniquely identifies the Application Insights query endpoint and HTTP method. timespan is an optional query parameter agents may omit or place differently — removing it avoids false negatives without weakening the signal.

  • _check_path_params (lines 60-65): Drops "tasklistid" and "taskid". In an HTTP fallback, path parameters are embedded directly in the URL (/lists/{id}/tasks/{id}) rather than appearing as separate named keys. The URL pattern evidence (tasks.googleapis.com/tasks/v1/lists, /tasks/, delete) already proves the agent targeted the correct Google Tasks endpoint with path params — the explicit key names were over-constraining.

  • _check_method_override (lines 85-98): New function with solid needle set: management.azure.com, desktopvirtualization, hostpools, retrieveregistrationtoken, method, post, api-version. These uniquely fingerprint the Azure AVD host-pool registration token endpoint and its POST override. The api-version needle is a good addition — it catches a common omission agents make with ARM endpoints.

Usage string (line 104) and checks dict (line 119) both updated. Alphabetical order in the dict is maintained.

2. tests/tasks/uipath-maestro-flow/connector_features/method_override.yaml

Severity: OK

  • Prompt addition (lines 32-35): Adds explicit HTTP fallback instructions, matching the pattern in generate_schema.yaml, path_params.yaml, and query_params.yaml. The phrasing ("Do not keep searching for a connector node that the registry does not expose") is identical to the other tasks — good consistency.

  • Success criterion change (lines 47-48): Replaces the inline python3 -c assertion (which only checked for the uipath-microsoft-azure connector key in the flow JSON) with check_managed_http_fallback.py ... method_override. The shared checker already handles the connector-key-present fast-path (line 113-115 of the Python file), so the native connector case is still covered. This is strictly better: it accepts either the native connector or a well-formed HTTP fallback, whereas the old assertion would fail on any fallback flow.

What's Missing

Nothing identified. All four connector feature tasks (generate_schema, path_params, query_params, method_override) now use the shared checker consistently. The relaxations are scoped to evidence that was non-load-bearing for the fallback path.

Area Ratings

Area Status Notes
Frontmatter N/A No skill SKILL.md changed
E2E Tests OK Task YAML is valid, success criteria correctly wired
Skill Body N/A No skill body changed
References & Assets N/A No references changed
Repo Hygiene OK No secrets, no cross-skill refs, changes scoped to connector_features

Issues for Manual Review

  • Verify the _check_method_override needles match the actual Azure AVD retrieveRegistrationToken REST endpoint shape. The review confirms the URL path components and POST method are correct per Azure ARM conventions, but domain-specific accuracy of field names (desktopvirtualization, hostpools) should be confirmed against the Azure API docs.

Conclusion

Clean PR. The relaxations are well-motivated (removing non-load-bearing evidence that caused false negatives), the new method_override check is consistent with the existing pattern, and the YAML migration to the shared checker eliminates a one-off inline assertion. No changes requested.

@tmatup tmatup closed this May 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant