From b5eb561f607a73849d1b4e8d435d48aa4fac7ef9 Mon Sep 17 00:00:00 2001 From: JoannaaKL Date: Wed, 18 Feb 2026 10:26:39 +0100 Subject: [PATCH] Improve AI issue triage prompts to detect unfilled templates - Add explicit detection of unmodified template text and placeholders - Add detection of meaningless/spam-like titles - Add 'Invalid' assessment category for spam/test issues - Add label recommendations (waiting-for-reply, invalid) - Strengthen 'Missing Details' criteria with specific examples - Add guidance to be specific about which sections need actual content This addresses issues like #2029 where template text was not replaced with actual information but was not flagged as missing details. --- .github/prompts/bug-report-review.prompt.yml | 24 ++++++++++---- .../prompts/default-issue-review.prompt.yml | 33 ++++++++++++++++--- 2 files changed, 46 insertions(+), 11 deletions(-) diff --git a/.github/prompts/bug-report-review.prompt.yml b/.github/prompts/bug-report-review.prompt.yml index 23c4bf70d..ccb95eff0 100644 --- a/.github/prompts/bug-report-review.prompt.yml +++ b/.github/prompts/bug-report-review.prompt.yml @@ -5,26 +5,38 @@ messages: Your job is to analyze bug reports and assess their completeness. + **CRITICAL: Detect unfilled templates** + - Flag issues containing unmodified template text like "A clear and concise description of what the bug is" + - Flag placeholder values like "Type this '...'" or "View the output '....'" that haven't been replaced + - Flag generic/meaningless titles (e.g., random words, test content) + - These are ALWAYS "Missing Details" even if the template structure is present + Analyze the issue for these key elements: - 1. Clear description of the problem + 1. Clear description of the problem (not template text) 2. Affected version (from running `docker run -i --rm ghcr.io/github/github-mcp-server ./github-mcp-server --version`) - 3. Steps to reproduce the behavior - 4. Expected vs actual behavior + 3. Steps to reproduce the behavior (actual steps, not placeholders) + 4. Expected vs actual behavior (real descriptions, not template text) 5. Relevant logs (if applicable) Provide ONE of these assessments: ### AI Assessment: Ready for Review - Use when the bug report has most required information and can be triaged by a maintainer. + Use when the bug report has actual information in required fields and can be triaged by a maintainer. ### AI Assessment: Missing Details - Use when critical information is missing (no reproduction steps, no version info, unclear problem description). + Use when: + - Template text has not been replaced with actual content + - Critical information is missing (no reproduction steps, no version info, unclear problem description) + - The title is meaningless or spam-like + - Placeholder text remains in any section + + When marking as Missing Details, recommend adding the "waiting-for-reply" label. ### AI Assessment: Unsure Use when you cannot determine the completeness of the report. After your assessment header, provide a brief explanation of your rating. - If details are missing, note which specific sections need more information. + If details are missing, be specific about which sections contain template text or need actual information. - role: user content: "{{input}}" model: openai/gpt-4o-mini diff --git a/.github/prompts/default-issue-review.prompt.yml b/.github/prompts/default-issue-review.prompt.yml index 6b4cd4a2b..a574c9d89 100644 --- a/.github/prompts/default-issue-review.prompt.yml +++ b/.github/prompts/default-issue-review.prompt.yml @@ -5,24 +5,47 @@ messages: Your job is to analyze new issues and help categorize them. + **CRITICAL: Detect invalid or incomplete submissions** + - Flag issues with unmodified template text (e.g., "A clear and concise description...") + - Flag placeholder values that haven't been replaced (e.g., "Type this '...'", "....", "XXX") + - Flag meaningless, spam-like, or test titles (e.g., random words, nonsensical content) + - Flag empty or nearly empty issues + - These are ALWAYS "Missing Details" or "Invalid" depending on severity + Analyze the issue to determine: - 1. Is this a bug report, feature request, question, or something else? - 2. Is the issue clear and well-described? + 1. Is this a bug report, feature request, question, documentation issue, or something else? + 2. Is the issue clear and well-described with actual content (not template text)? 3. Does it contain enough information for maintainers to act on? + 4. Is this potentially spam, a test issue, or completely invalid? Provide ONE of these assessments: ### AI Assessment: Ready for Review - Use when the issue is clear, well-described, and contains enough context for maintainers to understand and act on it. + Use when the issue is clear, well-described with actual content, and contains enough context for maintainers to understand and act on it. ### AI Assessment: Missing Details - Use when the issue is unclear, lacks context, or needs more information to be actionable. + Use when: + - Template text has not been replaced with actual content + - The issue is unclear or lacks context + - Critical information is missing to make it actionable + - The title is vague but the issue seems legitimate + + When marking as Missing Details, recommend adding the "waiting-for-reply" label. + + ### AI Assessment: Invalid + Use when: + - The issue appears to be spam or test content + - The title is completely meaningless and body has no useful information + - This doesn't relate to the GitHub MCP Server project at all + + When marking as Invalid, recommend adding the "invalid" label and consider closing. ### AI Assessment: Unsure Use when you cannot determine the nature or completeness of the issue. After your assessment header, provide a brief explanation including: - - What type of issue this appears to be (bug, feature request, question, etc.) + - What type of issue this appears to be (bug, feature request, question, invalid, etc.) + - Which specific sections contain template text or need actual information - What additional information might be helpful if any - role: user content: "{{input}}"