changes against list of canonical skills: qa-knowledge, qa-structure, data-collection#128
Merged
Conversation
Contributor
Rosetta Triage ReviewSummary: This PR adds three new shared foundation skills for QA automation flows ( Findings: 🔴 CRITICAL1. New skills reference workflow phase files that do not exist in the repo — integration is broken as-is
🟠 MAJOR2. Required frontmatter fields missing on all three new SKILL.md files
3. Ordinal cross-skill reference creates fragile coupling
4. Dangling reference to non-existent
5.
🟡 MINOR6.
7. Mandatory prep-steps line missing in
8. Phone reservation convention mislabeled as "IETF reserved" in
9. Redaction-step sequencing is inconsistent across the three vendor bindings
Positive observations:
Suggestions:
Automated triage by Rosetta agent |
Contributor
Rosetta Triage ReviewSummary: This PR adds three new shared workflow skills to Because this PR modifies Overall quality: High — 0 critical issues. Skills are dense, safety-conscious, and follow established Rosetta idioms (fail-closed redaction gate, gap-not-fabrication discipline, progressive-disclosure routers). Findings:
Suggestions:
Automated triage by Rosetta agent |
isolomatov-gd
requested review from
isolomatov-gd,
kkhristenko51 and
omaiesh
as code owners
isolomatov-gd
requested changes
Jul 9, 2026
isolomatov-gd
requested changes
Jul 9, 2026
isolomatov-gd
left a comment
isolomatov-gd
left a comment
Contributor
There was a problem hiding this comment.
Apply changes requested
Co-authored-by: Igor Solomatov <isolomatov@griddynamics.com>
isolomatov-gd
previously approved these changes
Jul 9, 2026
Contributor
📋 Prompt Quality Validation Report❌ Validation FailedThe full markdown report and raw JSON output are available in the workflow artifacts for 5 days. Files With Issues
📄
|
| Severity | Gate | Details |
|---|---|---|
| Medium | Rosetta | Problem: Step 2 has malformed ACQUIRE calls: 'ACQUIRE references/tms-vendor-binding.md` FROM KB' and 'ACQUIRE references/documentation-vendor-binding.mdFROM KB' both use a double backtick before the path but a single backtick after it. The issue-vendor call in the same sentence, 'ACQUIREreferences/issue-vendor-binding.md FROM KB', is correctly single-backtick-paired. This breaks the canonical ACQUIRE alias syntax (ACQUIRE FROM KB) for 2 of 3 calls in the same line.<br>**Reason:**<br> ACQUIRE alias correctness is an explicit Rosetta conformance criterion (pa-rosetta.md). Inconsistent backtick pairing is sloppy authoring in a machine-parsed instruction line; while a capable LLM reading raw text will likely still resolve the intended filename, any stricter/regex-based ACQUIRE tooling risks mis-parsing or failing to extract the path, and the inconsistency (only 2 of 3 identical constructs affected) signals the line was not proofread after edits.<br>**Solution:**<br> Remove the extra leading backtick on both calls so all three read 'ACQUIRE references/.md` FROM KB' with matching single-backtick pairs, consistent with the issue-vendor call. |
📄 instructions/r3/core/skills/qa-structure/SKILL.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| Very High | Reference Integrity | Problem: The anti-pattern bullet says per-session artifacts must go to 'plans/testgen-{TICKET-KEY}/' (among other folders), but the only registered testgen-flow phase that actually creates this session (testgen-flow-project-config-loading.md, unmodified in this PR) creates 'agents/testgen/{TICKET-KEY}/' instead (with files testgen-state.md, testgen-project-config.md, initial-data.md there). Reason: An agent using both this skill and testgen-flow, as the skill claims to support, gets two different folders for the same session and may split or lose state. Solution: Change the stated path to match the real testgen-flow location (agents/testgen/{TICKET-KEY}/), or update testgen-flow-project-config-loading.md in the same change so both agree. |
| High | Rosetta | Problem: The testgen path in the anti-pattern list ('plans/testgen-{TICKET-KEY}/') is not the path used by the actual registered testgen-flow implementation, and no such convention is listed in docs/definitions/*. Reason: pa-rosetta.md requires referencing existing prompts by their real behavior, not inventing a conflicting one. Solution: Only state paths that match the real, registered workflow behavior, or explicitly mark this as a planned/future convention. |
| Medium | Self-Validation | Problem: There is no explicit instruction to verify a resolved path against the layout reference before writing; this is only implied by the anti-pattern wording ('binding to the layout reference verbatim'). Reason: An explicit self-check catches silent path drift before an artifact is written to the wrong place. Solution: Add an explicit checkpoint, e.g. 'before writing, confirm the resolved path matches the layout reference exactly'. |
| Medium | Example Grounding | Problem: No single worked example ties the abstract placeholders together (e.g. Jira key PROJ-123 resolving to plans/api-qa-PROJ-123/api-qa-project-config.md). Reason: A worked example reduces the chance of misapplying the placeholder syntax. Solution: Add one short concrete example line showing a resolved identifier and path. |
| Medium | Workflow Completeness | Problem: The skill gives only a lookup table (), not an ordered sequence of steps for using it (determine flow, ACQUIRE layout, resolve slug, read/write state). Reason: Without an explicit order, a caller must infer the sequence instead of following it. Solution: Add a short numbered 'how to use this skill' list. |
| Medium | Success Criteria | Problem: There is no explicit 'done-when' statement for correct use of this skill (e.g. resolved path matches the layout reference verbatim, required config keys are filled or N/A). Reason: A testable completion statement lets the calling phase self-check before it proceeds. Solution: Add one line defining completion, such as 'success = resolved path matches the layout reference exactly'. |
| Medium | Input Contract | Problem: The skill never states how the caller knows whether it is in the API or UI flow before choosing which layout reference to ACQUIRE ('XOR' assumes this is already known). Reason: Without a stated input, a caller could ACQUIRE the wrong layout or both, wasting a round trip. Solution: Add one line stating the expected input, e.g. 'caller already knows/states flow type (API or UI) before invoking this router'. |
📄 instructions/r3/core/skills/qa-structure/assets/api-qa-config-interview.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| High | Reference Integrity | Problem: Same 'RefSrc/' casing as above is presented as a literal folder path the agent should look in or ask the user about. Reason: A wrongly-cased path reference is effectively a broken reference on case-sensitive systems. Solution: Use the canonical lowercase 'refsrc/' spelling so the reference actually resolves. |
| High | Precision & Explicitness | Problem: Item 6 says 'In RefSrc/ folder (provide project name, e.g., RefSrc/my-backend/)' and 'discover from ARCHITECTURE.md RefSrc references', but the real target-project folder (per ARCHITECTURE.md and bootstrap-rosetta-files.md) is lowercase 'refsrc/'. Reason: On case-sensitive filesystems, 'RefSrc/my-backend/' will not match the real 'refsrc/my-backend/' directory, so the agent's own discovery step silently fails. Solution: Change both 'RefSrc/' occurrences to lowercase 'refsrc/'. |
| Medium | Rosetta | Problem: File opens directly with a '#' heading and carries no YAML frontmatter (name/description/tags/baseSchema), while docs/schemas/template.md requires that block and sibling template assets in r3 (coding-agents-prompt-authoring/assets/pa-prompt-brief.md, requirements-authoring/assets/ra-intent-capture.md) all carry it. Reason: Without frontmatter the publisher cannot extract name/description metadata, so the asset is invisible in KB listing attributes and fails schema conformance to docs/schemas/template.md. Applied uniformly to every qa-knowledge/qa-structure asset. Solution: Add the standard template frontmatter (name: api-qa-config-interview, description, tags: [], baseSchema: docs/schemas/template.md) at the top of the file. |
📄 instructions/r3/core/skills/qa-structure/assets/api-qa-project-config-template.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| Medium | Rosetta | Problem: File opens directly with a '#' heading and carries no YAML frontmatter (name/description/tags/baseSchema), while docs/schemas/template.md requires that block and sibling template assets in r3 (coding-agents-prompt-authoring/assets/pa-prompt-brief.md, requirements-authoring/assets/ra-intent-capture.md) all carry it. Reason: Without frontmatter the publisher cannot extract name/description metadata, so the asset is invisible in KB listing attributes and fails schema conformance to docs/schemas/template.md. Applied uniformly to every qa-knowledge/qa-structure asset. Solution: Add the standard template frontmatter (name: api-qa-project-config-template, description, tags: [], baseSchema: docs/schemas/template.md) at the top of the file. |
📄 instructions/r3/core/skills/qa-structure/assets/ui-qa-state-template.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| Medium | Rosetta | Problem: File opens directly with a '#' heading and carries no YAML frontmatter (name/description/tags/baseSchema), while docs/schemas/template.md requires that block and sibling template assets in r3 (coding-agents-prompt-authoring/assets/pa-prompt-brief.md, requirements-authoring/assets/ra-intent-capture.md) all carry it. Reason: Without frontmatter the publisher cannot extract name/description metadata, so the asset is invisible in KB listing attributes and fails schema conformance to docs/schemas/template.md. Applied uniformly to every qa-knowledge/qa-structure asset. Solution: Add the standard template frontmatter (name: ui-qa-state-template, description, tags: [], baseSchema: docs/schemas/template.md) at the top of the file. |
📄 instructions/r3/core/skills/qa-structure/references/config-schema.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| High | Reference Integrity | Problem: Row 'Backend Source Code -- backend_source_path' gives the example path 'RefSrc/my-backend/', but the real target-project folder (per ARCHITECTURE.md) is lowercase 'refsrc/'. Reason: Wrong case will not resolve on case-sensitive filesystems. Solution: Change the example to lowercase 'refsrc/my-backend/'. |
📄 instructions/r3/core/skills/qa-knowledge/SKILL.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| Medium | Goal Specification | Problem: when_to_use_skill promises 'assertion & coverage discipline, selector & page-object rules' as content this skill supplies, but the router -- which core_concepts itself names as the index of all conventions ('conventions are references -- see ') -- has no row for selector or page-object authoring rules, only failure taxonomies and artifact skeletons. Reason: A consuming agent reading when_to_use_skill expects this skill to supply selector/page-object rules, ACQUIREs it, and finds nothing for that need -- a wasted round trip and an unmet promise of scope. Solution: Add a resources row pointing to where selector/page-object conventions actually live, or narrow when_to_use_skill to only the conventions this skill's own router indexes. |
📄 instructions/r3/core/skills/qa-knowledge/references/api-qa-failure-taxonomy.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| Medium | Example Grounding | Problem: The 8 categories are abstract and some overlap closely (e.g. 'Response Assertion' vs 'Application Bug', 'Test Data' vs 'Timing / Race Condition'), but no worked example shows evidence mapped to a category, unlike the sibling ui-qa-failure-taxonomy.md, which adds a concrete rule (the page-source-directory check) for its trickiest category (Selector / Locator). Reason: Without a worked example, triage of ambiguous evidence is left entirely to the agent's own judgment, risking inconsistent category assignment run to run. Solution: Add one short worked example (2-3 lines) showing a piece of evidence and the category it resolves to, for the closest-overlapping pair. |
📄 instructions/r3/core/skills/qa-knowledge/assets/api-analysis-template.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| Medium | Rosetta | Problem: File opens directly with a '#' heading and carries no YAML frontmatter (name/description/tags/baseSchema), while docs/schemas/template.md requires that block and sibling template assets in r3 (coding-agents-prompt-authoring/assets/pa-prompt-brief.md, requirements-authoring/assets/ra-intent-capture.md) all carry it. Reason: Without frontmatter the publisher cannot extract name/description metadata, so the asset is invisible in KB listing attributes and fails schema conformance to docs/schemas/template.md. Applied uniformly to every qa-knowledge/qa-structure asset. Solution: Add the standard template frontmatter (name: api-analysis-template, description, tags: [], baseSchema: docs/schemas/template.md) at the top of the file. |
📄 instructions/r3/core/skills/qa-knowledge/assets/api-qa-test-impl-record.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| Medium | Rosetta | Problem: File opens directly with a '#' heading and carries no YAML frontmatter (name/description/tags/baseSchema), while docs/schemas/template.md requires that block and sibling template assets in r3 (coding-agents-prompt-authoring/assets/pa-prompt-brief.md, requirements-authoring/assets/ra-intent-capture.md) all carry it. Reason: Without frontmatter the publisher cannot extract name/description metadata, so the asset is invisible in KB listing attributes and fails schema conformance to docs/schemas/template.md. Applied uniformly to every qa-knowledge/qa-structure asset. Solution: Add the standard template frontmatter (name: api-qa-test-impl-record, description, tags: [], baseSchema: docs/schemas/template.md) at the top of the file. |
📄 instructions/r3/core/skills/qa-knowledge/assets/ui-qa-test-impl-record.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| Medium | Rosetta | Problem: File opens directly with a '#' heading and carries no YAML frontmatter (name/description/tags/baseSchema), while docs/schemas/template.md requires that block and sibling template assets in r3 (coding-agents-prompt-authoring/assets/pa-prompt-brief.md, requirements-authoring/assets/ra-intent-capture.md) all carry it. Reason: Without frontmatter the publisher cannot extract name/description metadata, so the asset is invisible in KB listing attributes and fails schema conformance to docs/schemas/template.md. Applied uniformly to every qa-knowledge/qa-structure asset. Solution: Add the standard template frontmatter (name: ui-qa-test-impl-record, description, tags: [], baseSchema: docs/schemas/template.md) at the top of the file. |
📄 instructions/r3/core/skills/qa-knowledge/assets/test-spec-template.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| Medium | Rosetta | Problem: File opens directly with a '#' heading and carries no YAML frontmatter (name/description/tags/baseSchema), while docs/schemas/template.md requires that block and sibling template assets in r3 (coding-agents-prompt-authoring/assets/pa-prompt-brief.md, requirements-authoring/assets/ra-intent-capture.md) all carry it. Reason: Without frontmatter the publisher cannot extract name/description metadata, so the asset is invisible in KB listing attributes and fails schema conformance to docs/schemas/template.md. Applied uniformly to every qa-knowledge/qa-structure asset. Solution: Add the standard template frontmatter (name: test-spec-template, description, tags: [], baseSchema: docs/schemas/template.md) at the top of the file. |
| Medium | Failure Handling | Problem: Test File Mapping, Shared Utilities, and Execution Order give no fallback for the empty/not-applicable case (e.g. 'List each auth helper, request builder, response validator, data factory, teardown utility to create/reuse' has no 'or None' instruction), unlike sibling assets in this same skill that consistently define an explicit N/A/None marker for every field (api-analysis-template.md: 'or None'; api-qa-test-impl-record.md: 'None -- ...').Reason: Without a stated fallback, an agent facing an endpoint with no shared utilities or dependencies may leave the section blank or invent content instead of writing an explicit None, breaking the 'never blank' discipline used elsewhere in this same skill. Solution: Add an explicit fallback marker for each of these three sections, e.g. 'or None -- no shared utilities needed'. |
| Medium | Success Criteria | Problem: The template lists Summary, Test Scenarios, Test File Mapping, Shared Utilities, Execution Order, and Assumptions sections but never states a done-when for the whole test-specs.md artifact, unlike the two sibling record assets in this same skill (api-qa-test-impl-record.md: 'Done when all fields are filled...'; ui-qa-test-impl-record.md: 'Done when all five carry a real value...'). Reason: Without an explicit completion marker, the authoring skill has no testable way to confirm the test-specs.md is finished versus partially drafted. Solution: Add one line stating completion criteria, e.g. 'Done when every ATC-NNN is mapped to a test file and every endpoint from api-analysis.md has at least one scenario.' |
📄 instructions/r3/core/skills/qa-knowledge/assets/approval-gate.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| High | Success Criteria | Problem: The gate never states when it is considered finished. Steps 1-6 describe actions (present, wait, match token, escalate, batch, reject) but there is no 'done when' line, unlike sibling assets (failure-report-template.md, gap-finding-templates.md, ui-qa-clarification-templates.md) which each state one. Reason: Without a stated completion condition, a calling phase has no deterministic test for 'still pending' vs 'resolved', risking indefinite re-prompting or premature proceeding. Solution: Add one line such as 'Done when: an exact token is recorded as approved/partial/rejected, or a full reject is logged with rationale.' |
| Medium | Rosetta | Problem: File opens directly with a '#' heading and carries no YAML frontmatter (name/description/tags/baseSchema), while docs/schemas/template.md requires that block and sibling template assets in r3 (coding-agents-prompt-authoring/assets/pa-prompt-brief.md, requirements-authoring/assets/ra-intent-capture.md) all carry it. Reason: Without frontmatter the publisher cannot extract name/description metadata, so the asset is invisible in KB listing attributes and fails schema conformance to docs/schemas/template.md. Applied uniformly to every qa-knowledge/qa-structure asset. Solution: Add the standard template frontmatter (name: approval-gate, description, tags: [], baseSchema: docs/schemas/template.md) at the top of the file. |
| Medium | Example Grounding | Problem: No worked example of a full gate cycle appears anywhere in the file -- no sample closed-token list, no example partial-approval reply, no example of the max-retry escalation actually firing. Only the literal escalation question text is given. Reason: The gate's hardest-to-get-right behaviors (exact-token matching, partial-approval scoping) are described abstractly only; a concrete example removes ambiguity for the calling phase's author. Solution: Add one short worked example: a 2-item closed token list, a user reply outside it, and the resulting re-prompt. |
| Medium | Conflict Resolution | Problem: The gate defines its own retry cap ('after ≥3 re-prompts without an exact token, stop') but never reconciles it with hitl's own escalation rule ('after two mismatches: STOP all changes immediately'). Both are stop conditions on low-quality user replies, with different thresholds (3 vs 2) and no stated precedence.Reason: Two overlapping numeric escalation thresholds from related rules, with no explicit precedence, can make an executing agent apply the wrong one or oscillate between them. Solution: Add a line clarifying that the 3-re-prompt cap governs token-format non-conformance specifically, and how/if it relates to hitl's 2-mismatch stop rule. |
📄 instructions/r3/core/skills/qa-knowledge/assets/code-analysis-report-template.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| Medium | Rosetta | Problem: File opens directly with a '#' heading and carries no YAML frontmatter (name/description/tags/baseSchema), while docs/schemas/template.md requires that block and sibling template assets in r3 (coding-agents-prompt-authoring/assets/pa-prompt-brief.md, requirements-authoring/assets/ra-intent-capture.md) all carry it. Reason: Without frontmatter the publisher cannot extract name/description metadata, so the asset is invisible in KB listing attributes and fails schema conformance to docs/schemas/template.md. Applied uniformly to every qa-knowledge/qa-structure asset. Solution: Add the standard template frontmatter (name: code-analysis-report-template, description, tags: [], baseSchema: docs/schemas/template.md) at the top of the file. |
📄 instructions/r3/core/skills/qa-knowledge/assets/failure-report-template.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| Medium | Rosetta | Problem: File opens directly with a '#' heading and carries no YAML frontmatter (name/description/tags/baseSchema), while docs/schemas/template.md requires that block and sibling template assets in r3 (coding-agents-prompt-authoring/assets/pa-prompt-brief.md, requirements-authoring/assets/ra-intent-capture.md) all carry it. Reason: Without frontmatter the publisher cannot extract name/description metadata, so the asset is invisible in KB listing attributes and fails schema conformance to docs/schemas/template.md. Applied uniformly to every qa-knowledge/qa-structure asset. Solution: Add the standard template frontmatter (name: failure-report-template, description, tags: [], baseSchema: docs/schemas/template.md) at the top of the file. |
| Medium | Example Grounding | Problem: The Evidence label Unknown is defined with a strict precondition ('assign ONLY after page-source capture was attempted and the cause is still unresolvable') but neither worked example (API or UI) shows an entry using Unknown -- both worked examples use Confirmed.Reason: Unknown is the most misuse-prone label (risk of being applied without first attempting page-source capture); leaving it unillustrated makes it more likely to be misapplied than the two fully-demonstrated labels. Solution: Add a third short worked-example row (UI) showing Evidence label = Unknown with its required escalation citation. |
📄 instructions/r3/core/skills/qa-knowledge/assets/gap-finding-templates.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| High | Reference Integrity | Problem: Text states PII redaction happens 'via the always-on sensitive-data skill, which the gap_analysis mode applies' -- but no skill or workflow anywhere in the instructions tree defines a 'gap_analysis mode'. The term appears only in this file and in ui-qa-clarification-templates.md; a full-tree search finds no owning definition (e.g. not in requirements-use/SKILL.md).Reason: A cross-reference to a concept that does not exist anywhere is a broken pointer; an agent trying to resolve 'which mode applies this' has nothing to ACQUIRE. Solution: Remove the 'gap_analysis mode' attribution, or replace it with the actual owning skill/workflow phase name, or define the mode at the place it is claimed to live. |
| Medium | Rosetta | Problem: File opens directly with a '#' heading and carries no YAML frontmatter (name/description/tags/baseSchema), while docs/schemas/template.md requires that block and sibling template assets in r3 (coding-agents-prompt-authoring/assets/pa-prompt-brief.md, requirements-authoring/assets/ra-intent-capture.md) all carry it. Reason: Without frontmatter the publisher cannot extract name/description metadata, so the asset is invisible in KB listing attributes and fails schema conformance to docs/schemas/template.md. Applied uniformly to every qa-knowledge/qa-structure asset. Solution: Add the standard template frontmatter (name: gap-finding-templates, description, tags: [], baseSchema: docs/schemas/template.md) at the top of the file. |
📄 instructions/r3/core/skills/qa-knowledge/assets/page-source-capture-instructions.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| Medium | Rosetta | Problem: File opens directly with a '#' heading and carries no YAML frontmatter (name/description/tags/baseSchema), while docs/schemas/template.md requires that block and sibling template assets in r3 (coding-agents-prompt-authoring/assets/pa-prompt-brief.md, requirements-authoring/assets/ra-intent-capture.md) all carry it. Reason: Without frontmatter the publisher cannot extract name/description metadata, so the asset is invisible in KB listing attributes and fails schema conformance to docs/schemas/template.md. Applied uniformly to every qa-knowledge/qa-structure asset. Solution: Add the standard template frontmatter (name: page-source-capture-instructions, description, tags: [], baseSchema: docs/schemas/template.md) at the top of the file. |
| Medium | Example Grounding | Problem: The 'Not-captured branch' (no app access, login wall, incomplete SPA outerHTML, or missing/empty/garbled files) is described only abstractly ('do NOT guess selectors. Stop... or escalate'). Unlike the happy-path message, which is a full verbatim script, there is no example wording for what the agent should actually say to the user in this failure branch. Reason: This file's own rule is that user-facing wording must be literal, not paraphrased, because non-technical users rely on exact steps; leaving the failure branch to free-form paraphrase breaks that same bar on the path most likely to confuse a non-technical user. Solution: Add one short example escalation line the agent can send verbatim when capture fails, matching the same 'verbatim, do not paraphrase' bar used for the happy path. |
📄 instructions/r3/core/skills/qa-knowledge/assets/proposed-change-template.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| Medium | Rosetta | Problem: File opens directly with a '#' heading and carries no YAML frontmatter (name/description/tags/baseSchema), while docs/schemas/template.md requires that block and sibling template assets in r3 (coding-agents-prompt-authoring/assets/pa-prompt-brief.md, requirements-authoring/assets/ra-intent-capture.md) all carry it. Reason: Without frontmatter the publisher cannot extract name/description metadata, so the asset is invisible in KB listing attributes and fails schema conformance to docs/schemas/template.md. Applied uniformly to every qa-knowledge/qa-structure asset. Solution: Add the standard template frontmatter (name: proposed-change-template, description, tags: [], baseSchema: docs/schemas/template.md) at the top of the file. |
| Medium | Example Grounding | Problem: Both worked examples (QA/API and UI-QA) show only the 'approved' end-state. No worked example exists for 'rejected', 'partial', or the 'In-scope: no -- STOP, escalate' branch, even though all are explicit enum/branch values defined in the same template. Reason: The unillustrated branches (partial-approval scoping, rejection, out-of-scope escalation) are exactly the error-prone paths where a concrete example most reduces ambiguity; only the simplest path is demonstrated. Solution: Add one short worked example showing a partial or rejected Approval status, or an in-scope:no escalation. |
| Medium | Success Criteria | Problem: No explicit done-when statement covers a batch of proposed changes -- e.g. that every change in a set must reach a resolved Approval status (approved/rejected/partial) before corrections are applied. Only the iteration cap defines a stop condition, and only for a single repeatedly-failing change. Reason: Without this, nothing states that a pending item blocks proceeding, so an agent could apply a batch while some entries are still unresolved.Solution: Add a line such as 'Done when every Proposed Change in the batch has a non-pending Approval status.' |
📄 instructions/r3/core/skills/qa-knowledge/assets/ui-qa-clarification-templates.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| High | Reference Integrity | Problem: Line: 'Full catalog owned by the requirements-use gap_analysis mode.' The full requirements-use/SKILL.md defines no 'modes' concept and no 'gap_analysis' anything; the term does not appear anywhere else in the instructions tree except this file and gap-finding-templates.md.Reason: A named 'owner' that does not exist is a dead-end reference; an agent that goes looking for the fuller catalog to resolve an edge case will find nothing at the claimed location. Solution: Point to the actual location of the full D1-D5 catalog (name the real owning skill/asset), or state that the D1-D5 list given here already is the full catalog and drop the cross-reference. |
| Medium | Rosetta | Problem: File opens directly with a '#' heading and carries no YAML frontmatter (name/description/tags/baseSchema), while docs/schemas/template.md requires that block and sibling template assets in r3 (coding-agents-prompt-authoring/assets/pa-prompt-brief.md, requirements-authoring/assets/ra-intent-capture.md) all carry it. Reason: Without frontmatter the publisher cannot extract name/description metadata, so the asset is invisible in KB listing attributes and fails schema conformance to docs/schemas/template.md. Applied uniformly to every qa-knowledge/qa-structure asset. Solution: Add the standard template frontmatter (name: ui-qa-clarification-templates, description, tags: [], baseSchema: docs/schemas/template.md) at the top of the file. |
📄 instructions/r3/core/skills/qa-knowledge/assets/ui-qa-plan-template.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| Medium | Rosetta | Problem: File opens directly with a '#' heading and carries no YAML frontmatter (name/description/tags/baseSchema), while docs/schemas/template.md requires that block and sibling template assets in r3 (coding-agents-prompt-authoring/assets/pa-prompt-brief.md, requirements-authoring/assets/ra-intent-capture.md) all carry it. Reason: Without frontmatter the publisher cannot extract name/description metadata, so the asset is invisible in KB listing attributes and fails schema conformance to docs/schemas/template.md. Applied uniformly to every qa-knowledge/qa-structure asset. Solution: Add the standard template frontmatter (name: ui-qa-plan-template, description, tags: [], baseSchema: docs/schemas/template.md) at the top of the file. |
| Medium | Precision & Explicitness | Problem: Line 7 says 'TestRail (TMS), Confluence (Wiki), Jira (issue tracker) used here as an example, replace with what currently used' but the template body below never uses Jira anywhere (only a TestRail Case field and Confluence field exist in 'Source'). Reason: Naming a tool as 'used here' when no field for it exists can make an author hunt for a Jira field that does not exist in the skeleton. Solution: Drop 'Jira (issue tracker)' from this line, or add a Jira field to the Source section if it is actually meant to be captured. |
📄 instructions/r3/core/skills/sensitive-data/SKILL.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| High | Cognitive Budget | Problem: Item 7 alone does the work of 3-4 separate checks (scan for secret literals, scan for PII shapes, scan for customer names, fail-closed if the scan didn't run) collapsed into a single numbered step. Reason: Rosetta hardening calls for directives to be decomposed for ~5-step reliability; one overloaded step raises the chance a sub-clause is silently skipped. Solution: Decompose item 7 into discrete sub-steps (e.g. 7a scan secret literals, 7b scan PII shapes, 7c fail-closed if scan did not run) so each is independently executable and verifiable. |
| High | Structural Coherence | Problem: New item 7 ('Scan information of your output or artifacts for: Bearer , Authorization:, ... Fail-closed -- if the scan cannot run, do not emit') packs 11 literal secret patterns, 3 PII-shape exceptions, a customer-name clause, a non-exhaustive-floor clause, and a fail-closed clause into one unbroken numbered line with no sub-list or grouping.Reason: One run-on directive mixing many unrelated sub-facts breaks at-a-glance readability and risks the agent skipping or mis-parsing part of the list. Solution: Break item 7 into a short lead-in plus a sub-list of pattern groups, or move the literal pattern catalogue into a separate references asset ACQUIRE'd at pre-emit-scan time, leaving step 7 as 'run the pre-emit scan; fail-closed if it cannot run.' |
| Medium | Failure Handling | Problem: New item 7 ends with 'Fail-closed -- if the scan cannot run, do not emit', but for a self-performed text scan the prompt never defines what 'cannot run' means, so the agent has no concrete trigger to stop on. Reason: Without a defined trigger the fail-closed rule may never fire, weakening the new safety guarantee it is meant to provide. Solution: Add one concrete condition that counts as 'scan cannot run' for a self-scan (for example: the agent cannot enumerate its own output, or a required tool is unavailable), so the fail-closed stop is actionable. |
| Medium | Precision & Explicitness | Problem: Item 8 says 'Use IETF reserved ranges for PII placeholders: emails test.user-1@example.com; phones +1-555-0100-+1-555-0199'. The example.com/.org domain reservation is IETF (RFC 2606), but the 555-0100-0199 fictional phone block is reserved by NANPA (North American Numbering Plan), not IETF.Reason: Mislabeling the standards body in a compliance-facing instruction undermines the guidance's credibility if a maintainer checks the citation. Solution: Split the attribution, e.g. 'Use IETF-reserved domains (RFC 2606) for email placeholders and NANPA-reserved 555-01XX numbers for phone placeholders.' |
isolomatov-gd
approved these changes
Jul 9, 2026
isolomatov-gd
approved these changes
Jul 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add three shared workflow skills
This PR presents 3 new skills which are currently not in the Canonical List of Skills on
main.This is an extraction from the larger PR #110 and serves as a preparation for merging the workflows themselves.
data-collection
What: Read-only retrieval from systems-of-record (Jira or another issue tracker; Confluence or another documentation system; TestRail or another TMS) via MCP -- plus a codebase scan. It normalizes the collected material into a provided raw-context artifact. It retrieves and records; it never acts on what it reads.
Why created: Every flow (API-QA, UI-QA, TG) needs the same disciplined "pull the source material, flag gaps, redact before writing" step.
Why separate: It's the only input concern and the only one that touches external MCPs. Folding it in would drag MCP plumbing into skills that have nothing to do with retrieval.
Used by TG, API-QA, UI-QA. Potentially,
data-collectioncan also be used by a coding flow, or a debugging flow, or a PR-review flow, or by any system/agent/step that needs to pull source-of-record material into an artifact.qa-structure
What: Defines where QA artifacts live: canonical paths, the {IDENTIFIER} / slug rules, the config-key schema, and the state-file shape. Phases bind to these paths verbatim and never invent their own.
Why created: Without one authority for paths/identifiers/state, phases drift and collide.
Why separate: Paths/state evolve independently of domain knowledge.
qa-knowledge
What: QA-domain knowledge layer, i.e. conventions general skills don't own: failure taxonomies, redaction scope, assertion/coverage discipline, selector & page-object rules, and the artifact skeletons (templates) each phase emits. A router that ACQUIREs the one asset/reference the current step needs.
Why created: These conventions and templates are shared across API-QA and UI-QA; centralizing them keeps them DRY.
Why separate: It's the domain-knowledge/templates concern, orthogonal to where things go (qa-structure) and how you fetch inputs (data-collection).
Notes for Github Triage Bot
qa-knowledge/assets/ui-qa-clarification-templates.mdandqa-knowledge/assets/gap-finding-templates.mdreference to the requirements-use gap_analysis mode which will also be added in a follow-up PR.