Skip to content

feat: Add What Happens Next, Winners & Losers, FAQ sections with Schema.org FAQPage and content depth validation#1661

Merged
pethers merged 15 commits intomainfrom
copilot/improve-article-template-quality
Apr 10, 2026
Merged

feat: Add What Happens Next, Winners & Losers, FAQ sections with Schema.org FAQPage and content depth validation#1661
pethers merged 15 commits intomainfrom
copilot/improve-article-template-quality

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 10, 2026

  • Add WhatHappensNextItem, WinnersLosersEntry, FAQItem types to scripts/types/editorial.ts
  • Add faqItems?: FAQItem[] field to ArticleData in scripts/types/article.ts
  • Add section title translations (14 languages) to scripts/article-template/constants.ts
  • Add generateWhatHappensNextSection(), generateWinnersLosersSection(), generateFaqSection() helpers
  • Export new helpers from scripts/article-template/index.ts
  • Add makeWhatHappensNextDirective(), makeWinnersLosersDirective(), and makeFaqDirective() AI directive builders
  • Wire new directives into committee-reports, propositions, motions, interpellations
  • Add FAQPage Schema.org structured data via JSON.stringify() (safe against newlines/backslashes)
  • Extended speakable CSS selectors (including .faq-section)
  • Content depth validation: hasWhatHappensNext(), hasWinnersLosers(), countSpecificClaims(), hasSubstantialLede()
  • Updated QualityMetrics and QualityThresholds in scripts/types/validation.ts
  • generateSectionTransition() in scripts/editorial-pillars.ts for context-aware inter-section transitions
  • EditorialSection union extended with what-happens-next, winners-losers, faq
  • Added faq to requiredSections for committee-reports, propositions, motions, interpellations
  • Fixed Winners & Losers regex: (?:&|and) instead of overly permissive [&and]+
  • Deduped document IDs in countSpecificClaims() via Set, capped at 5
  • Fixed countSpecificClaims() docstring to match actual behavior (no currency/attribution claims)
  • Added all 14 language heading patterns to hasWhatHappensNext()/hasWinnersLosers() (da, no, fi, nl, ar, he)
  • Per-language default context in generateSectionTransition() via DEFAULT_TRANSITION_CONTEXT (prevents mixed-language output)
  • Updated generateSectionTransition() JSDoc: context is optional with localized defaults, not required
  • Removed redundant (faqItems as FAQItem[]) casts and unused FAQItem import in template.ts
  • Replaced countSectionWords() simple regex with stack-based HTML parser to correctly handle nested <div>/<section> elements
  • Added makeFaqDirective() and wired into all four article types that require FAQ
  • Validate item.significance in generateWhatHappensNextSection() — defaults to medium for unknown values
  • Validate e.outcome in generateWinnersLosersSection() — defaults to mixed for unknown values
  • Fixed percentage comment in countSpecificClaims() to match actual behavior (no context check)
  • Use replacer functions in generateSectionTransition() to prevent $ mangling in topic/actor values
  • Fixed pre-existing CI failures in news-evening-analysis.md: added STEP 1 emphasis, date support annotations, numbered cross-referencing examples
  • Added 39 unit tests for content depth validators
  • All targeted tests pass

@github-actions github-actions bot added the size-xs Extra small change (< 10 lines) label Apr 10, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🏷️ Automatic Labeling Summary

This PR has been automatically labeled based on the files changed and PR metadata.

Applied Labels: size-xs

Label Categories

  • 🗳️ Content: news, dashboard, visualization, intelligence
  • 💻 Technology: html-css, javascript, workflow, security
  • 📊 Data: cia-data, riksdag-data, data-pipeline, schema
  • 🌍 I18n: i18n, translation, rtl
  • 🔒 ISMS: isms, iso-27001, nist-csf, cis-controls
  • 🏗️ Infrastructure: ci-cd, deployment, performance, monitoring
  • 🔄 Quality: testing, accessibility, documentation, refactor
  • 🤖 AI: agent, skill, agentic-workflow

For more information, see .github/labeler.yml.

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

…chema, and content depth validation

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/0ddc5fe4-4563-49ee-963e-b7e378213172

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
@github-actions github-actions bot added refactor Code refactoring size-l Large change (250-1000 lines) labels Apr 10, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

…s and context requirements

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/0ddc5fe4-4563-49ee-963e-b7e378213172

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copilot AI changed the title [WIP] Enhance article template and structure for better engagement feat: Add What Happens Next, Winners & Losers, FAQ sections with Schema.org FAQPage and content depth validation Apr 10, 2026
Copilot AI requested a review from pethers April 10, 2026 09:35
@pethers pethers marked this pull request as ready for review April 10, 2026 09:57
Copilot AI review requested due to automatic review settings April 10, 2026 09:57
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds richer, more structured article sections (timeline, actor outcomes, FAQ) to improve reader usefulness and SEO, and extends the quality enhancer to detect/validate presence and content depth for key sections.

Changes:

  • Introduces new editorial data types and quality metrics/thresholds (FAQ items, timeline entries, winners/losers entries, specific-claims + lede depth).
  • Extends article HTML templating with new section renderers and conditional Schema.org FAQPage JSON-LD + expanded speakable selectors.
  • Adds new editorial requirements/directives and new section transitions + section presence detection in the quality enhancer.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
scripts/types/validation.ts Adds new quality thresholds and metrics fields for section presence and content depth.
scripts/types/editorial.ts Defines structured types for What Happens Next, Winners/Losers, and FAQ items.
scripts/types/article.ts Adds faqItems to ArticleData for rendering + structured data.
scripts/editorial-pillars.ts Adds context-aware inter-section transition templates and generator.
scripts/editorial-framework.ts Extends EditorialSection and updates required sections for specific article profiles.
scripts/article-template/template.ts Emits conditional FAQPage JSON-LD and renders FAQ section; expands speakable selectors.
scripts/article-template/registry.ts Adds AI directive factories and wires them into four article types.
scripts/article-template/index.ts Re-exports and exposes new section generators from the template module.
scripts/article-template/helpers.ts Implements HTML generators for timeline, winners/losers, and FAQ sections.
scripts/article-template/constants.ts Adds localized titles/labels for the new sections across 14 languages.
scripts/article-quality-enhancer.ts Adds detection for new sections and introduces specific-claims + lede-depth validation.

function hasWinnersLosers(content: string): boolean {
const patterns: readonly RegExp[] = [
/class=["'][^"']*\bwinners-losers\b/,
/winners\s*[&and]+\s*losers/i,
Copy link

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Winners & Losers detector regex /winners\s*[&and]+\s*losers/i is overly permissive because [&and]+ is a character class (matches any mix of '&', 'a', 'n', 'd'), which can trigger false positives (e.g. "winners n losers"). Use an alternation like (?:&|and) (and keep surrounding whitespace handling) to match only the intended separator.

Suggested change
/winners\s*[&and]+\s*losers/i,
/winners\s*(?:&|and)\s*losers/i,

Copilot uses AI. Check for mistakes.
Comment on lines +425 to +440
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "FAQPage",
"mainEntity": [${(faqItems as FAQItem[]).map(item => `
{
"@type": "Question",
"name": "${escapeHtml(item.question)}",
"acceptedAnswer": {
"@type": "Answer",
"text": "${escapeHtml(item.answer)}"
}
}`).join(',')}
]
}
</script>` : ''}
Copy link

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The FAQPage JSON-LD is built via string interpolation with escapeHtml(), but escapeHtml() does not JSON-escape newlines/backslashes. If an FAQ question/answer contains a newline (common in AI output), the resulting JSON-LD becomes invalid. Consider constructing the object and embedding it with JSON.stringify(...), or at least normalizing whitespace (similar to sanitizeArticleBody()) for question/answer fields before interpolating.

Suggested change
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "FAQPage",
"mainEntity": [${(faqItems as FAQItem[]).map(item => `
{
"@type": "Question",
"name": "${escapeHtml(item.question)}",
"acceptedAnswer": {
"@type": "Answer",
"text": "${escapeHtml(item.answer)}"
}
}`).join(',')}
]
}
</script>` : ''}
<script type="application/ld+json">${JSON.stringify({
'@context': 'https://schema.org',
'@type': 'FAQPage',
mainEntity: (faqItems as FAQItem[]).map(item => ({
'@type': 'Question',
name: item.question,
acceptedAnswer: {
'@type': 'Answer',
text: item.answer,
},
})),
}).replace(/</g, '\\u003c')}</script>` : ''}

Copilot uses AI. Check for mistakes.
Comment on lines 172 to 177
'policy-mindmap',
'deep-analysis',
'watch-points',
'what-happens-next',
'winners-losers',
'sources-methodology',
Copy link

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description/title mention adding an FAQ section and adding it to requiredSections for these article types, but the updated requiredSections arrays here only add what-happens-next and winners-losers (no faq). Either add faq to the required sections for the intended profiles, or adjust the PR description/requirements so the gate matches the actual intent.

Copilot uses AI. Check for mistakes.
@pethers
Copy link
Copy Markdown
Member

pethers commented Apr 10, 2026

@copilot apply changes based on the comments in this thread

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

…age JSON-LD, add faq to requiredSections

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/a8a68e49-c93b-4008-aaae-b4b1fb9d9150

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 10, 2026

@copilot apply changes based on the comments in this thread

Applied all three review comments in bf2575f:

  1. Regex fix (article-quality-enhancer.ts:217): Changed [&and]+(?:&|and) to prevent false positives like "winners n losers".
  2. FAQPage JSON-LD (template.ts): Replaced escapeHtml() string interpolation with JSON.stringify() + \u003c escaping, so newlines/backslashes in AI-generated FAQ answers produce valid JSON-LD.
  3. Missing faq in requiredSections (editorial-framework.ts): Added 'faq' to all four article types (committee-reports, propositions, motions, interpellations).

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

@pethers
Copy link
Copy Markdown
Member

pethers commented Apr 10, 2026

@copilot apply changes based on the comments in this thread

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 10, 2026

@copilot apply changes based on the comments in this thread

Applied in b0d08af:

  1. Stack-based parser (countSectionWords): Replaced the simple regex (which terminated early on nested </div>/</section> tags) with a stack-based parser that tracks opening/closing tags and only stops at the true matching close tag.
  2. Nested elements test: Added a focused test with nested <div> + <section> inside the target section — confirms all 8 words are counted correctly instead of stopping at the first inner </div>.

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 4 comments.

Comment on lines +219 to +228
const rows = items
.filter(item => item.date && item.event)
.map(item => {
const sigClass = `significance-${item.significance}`;
const sigLabel = sigLabels[item.significance];
return ` <li class="timeline-item ${sigClass}">
<time class="timeline-date" datetime="${escapeHtml(item.date)}">${escapeHtml(item.date)}</time>
<span class="timeline-event">${escapeHtml(item.event)}</span>
<span class="timeline-significance" aria-label="${escapeHtml(sigLabel)}">${escapeHtml(sigLabel)}</span>
</li>`;
Copy link

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generateWhatHappensNextSection() builds CSS classes and looks up SIGNIFICANCE_LABELS using item.significance, but it never validates that the incoming value is one of high|medium|low. If the timeline items come from untyped/JSON/AI input, an unexpected value will produce a class like significance-undefined and an empty label. Consider filtering invalid significance values or defaulting to a safe value (e.g. medium) before rendering.

Copilot uses AI. Check for mistakes.
Comment on lines +262 to +271
const rows = entries
.filter(e => e.actor && e.evidence)
.map(e => {
const outcomeClass = `outcome-${e.outcome}`;
const outcomeLabel = outcomeLabels[e.outcome];
return ` <li class="wl-entry ${outcomeClass}">
<span class="wl-actor">${escapeHtml(e.actor)}</span>
<span class="wl-outcome">${escapeHtml(outcomeLabel)}</span>
<span class="wl-evidence">${escapeHtml(e.evidence)}</span>
</li>`;
Copy link

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generateWinnersLosersSection() uses e.outcome to build outcome-${e.outcome} and to index OUTCOME_LABELS, but it doesn't guard against unexpected values at runtime (e.g. from parsed AI output). That can yield outcome-undefined and empty labels. Consider validating/normalizing outcome (wins/loses/mixed) or defaulting to mixed when unknown.

Copilot uses AI. Check for mistakes.
// Cap at 5 to prevent a heavily statistics-driven article from
// single-handedly satisfying the minSpecificClaims threshold via
// repetitive figures alone (e.g. budget tables with 20+ percentages).
const percentMatches = text.match(/\b\d+(?:\.\d+)?%/g);
Copy link

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment says “Percentage figures with surrounding context (e.g. "increased by 12%")”, but the implementation counts any percentage token via /\b\d+(?:\.\d+)?%/g (no context check). Either adjust the comment to match the current behavior, or implement the intended context requirement to avoid over-counting unrelated percentages (e.g. in tables).

Suggested change
const percentMatches = text.match(/\b\d+(?:\.\d+)?%/g);
const percentMatches = text.match(
/\b(?:increased?|decreased?|rose|risen|fall(?:s|en)?|fell|dropped?|declined?|grew|growth|shrank|reduced?|up|down|change(?:d)?|gain(?:ed)?|loss(?:es)?|surge(?:d)?|jump(?:ed)?|improv(?:ed|ement)|worsen(?:ed|ing)?|inflation|unemployment|approval|support)\s+(?:by|of|to|at)?\s*\d+(?:\.\d+)?%/gi,
);

Copilot uses AI. Check for mistakes.
Comment on lines +376 to +377
.replace(/\{topic\}/g, topic)
.replace(/\{actor\}/g, actor);
Copy link

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generateSectionTransition() interpolates {topic}/{actor} using String.prototype.replace with a string replacement. In JS, replacement strings treat $ sequences specially (e.g. $&, $1), so a topicKeyword/actorName containing $ can be mangled. Use a replacer function (e.g. replace(/\{topic\}/g, () => topic)) or escape $ in the replacement values.

Suggested change
.replace(/\{topic\}/g, topic)
.replace(/\{actor\}/g, actor);
.replace(/\{topic\}/g, () => topic)
.replace(/\{actor\}/g, () => actor);

Copilot uses AI. Check for mistakes.
@pethers
Copy link
Copy Markdown
Member

pethers commented Apr 10, 2026

@copilot apply changes based on the comments in this thread

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 10, 2026

@copilot apply changes based on the comments in this thread

Applied all four review comments in 22c249b:

  1. Significance validation (helpers.ts): generateWhatHappensNextSection() now validates item.significance against high|medium|low and defaults to medium for unknown values.
  2. Outcome validation (helpers.ts): generateWinnersLosersSection() now validates e.outcome against wins|loses|mixed and defaults to mixed for unknown values.
  3. Percentage comment fix (article-quality-enhancer.ts:337): Updated comment from "Percentage figures with surrounding context" to "Percentage figures" to match the actual regex behavior.
  4. $ safety in replace (editorial-pillars.ts): Switched to replacer functions (() => topic / () => actor) so $ sequences in topic/actor values aren't mangled by JS replacement string semantics.

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

@pethers
Copy link
Copy Markdown
Member

pethers commented Apr 10, 2026

@copilot analyse and fix root cause of TypeScript & JavaScript Testing / Unit Tests (Vitest) (pull_request)
TypeScript & JavaScript Testing / Unit Tests (Vitest) (pull_request)Failing after 1m

…s, and STEP 1 emphasis in news-evening-analysis.md

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/d35a12d5-b308-46ee-a102-7bc86c200799

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 10, 2026

@copilot apply changes based on the comments in this thread

Applied all four review comments in 22c249b:

  1. Significance validation (helpers.ts): generateWhatHappensNextSection() now validates item.significance against high|medium|low and defaults to medium for unknown values.
  2. Outcome validation (helpers.ts): generateWinnersLosersSection() now validates e.outcome against wins|loses|mixed and defaults to mixed for unknown values.
  3. Percentage comment fix (article-quality-enhancer.ts:337): Updated comment from "Percentage figures with surrounding context" to "Percentage figures" to match the actual regex behavior.
  4. $ safety in replace (editorial-pillars.ts): Switched to replacer functions (() => topic / () => actor) so $ sequences in topic/actor values aren't mangled by JS replacement string semantics.

@copilot analyse and fix root cause of [TypeScript & JavaScript Testing / Unit Tests (Vitest) (pull_request)](https://github.com/Hack23/riksdagsmonitor/actions/runs/24251762834/job/708...

Root cause: 3 pre-existing test failures in tests/agentic-workflow-mcp-queries.test.ts testing news-evening-analysis.md content — unrelated to this PR's changes (test file is identical to main). The workflow was missing:

  1. STEP 1: ALWAYS check emphasis in MCP Health Gate section
  2. supports from/tom / supports dateFrom/dateTo date annotation patterns
  3. Numbered Example 1: / Example 2: cross-referencing examples and Cross-reference related data sources text

Fixed in 77027a3. All 55 MCP query tests + 88 workflow architecture tests + 39 content depth tests now pass.

@github-actions github-actions bot added documentation Documentation updates workflow GitHub Actions workflows ci-cd CI/CD pipeline changes news News articles and content generation agentic-workflow Agentic workflow changes labels Apr 10, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

@pethers pethers merged commit 3eada64 into main Apr 10, 2026
16 checks passed
@pethers pethers deleted the copilot/improve-article-template-quality branch April 10, 2026 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agentic-workflow Agentic workflow changes ci-cd CI/CD pipeline changes documentation Documentation updates news News articles and content generation refactor Code refactoring size-l Large change (250-1000 lines) size-xl Extra large change (> 1000 lines) size-xs Extra small change (< 10 lines) testing Test coverage workflow GitHub Actions workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve News Article Content Quality — Editorial Templates & Article Structure (scripts/article-template/ + editorial)

3 participants