🔍 Quality: Analyzer Code Fix Coverage Gap — 19 of 65 MSTEST rules lack automated fixes

[Evangelink]

🎯 Repository Quality Improvement Report — Analyzer Code Fix Coverage Gap

Analysis Date: 2026-06-10
Focus Area: Analyzer Code Fix Coverage Gap
Strategy Type: Custom (repository-specific)

Executive Summary

MSTest ships 65 active Roslyn diagnostic rules (excluding suppressors and retired rule IDs), yet only 41 of those rules have a corresponding code fix — a coverage gap of roughly 37%. When a developer sees a squiggly and clicks the lightbulb, nothing happens for nearly one-in-three diagnostics. This creates friction exactly at the moment when automated remediation would be most helpful.

The 19 rules without code fixes split into two groups: about 7 are genuinely complex refactorings (extracting statements, replacing blocking calls with async equivalents, resolving multi-rule attribute conflicts) that require careful design; the remaining 12 are straightforward mechanical transformations where the fix involves adding, removing, or renaming a single attribute or modifier. The mechanical group should be tackled first for the best effort-to-impact ratio.

Given that the MSTest.Analyzers.CodeFixes project already has a mature pattern (each fixer implements CodeFixProvider, uses DiagnosticIds.*RuleId constants, and is tested inline via VerifyCS.VerifyCodeFixAsync), implementing new fixers is a well-understood task with no architectural unknowns.

Full Analysis Report

Current State Assessment

Metrics Collected:

Metric	Value	Status
Active diagnostic rule IDs	65	—
Diagnostic IDs with code fixes	41	⚠️
Coverage rate	63%	⚠️
Analyzer test files with VerifyCodeFixAsync	50 / 58	⚠️
Suppressors (no fix needed by design)	4	✅
Retired / reserved rule IDs	6	✅

Rules confirmed to lack code fixes (test files missing VerifyCodeFixAsync):

Rule ID	Rule Name	Severity	Fixability
MSTEST0001	UseParallelizeAttribute	Info	Easy — add [assembly: Parallelize()]
MSTEST0024	DoNotStoreStaticTestContext	Warning	Easy — remove static modifier
MSTEST0059	DoNotUseParallelizeAndDoNotParallelize	Warning	Easy — remove one conflicting attribute
MSTEST0035	UseDeploymentItemWithTestMethodOrTestClass	Warning	Easy — add [TestMethod] or [TestClass]
MSTEST0015	TestMethodShouldNotBeIgnored	Info	Medium — remove [Ignore] attribute
MSTEST0016	TestClassShouldHaveTestMethod	Warning	Medium — remove [TestClass] or add stub
MSTEST0036	DoNotUseShadowing	Warning	Medium — rename the shadowing member
MSTEST0043	UseRetryWithTestMethod	Warning	Medium — add [Retry(n)] attribute
MSTEST0051	AssertThrowsShouldContainSingleStatement	Info	Complex — extract to single-statement lambda
MSTEST0067	AvoidThreadSleepAndTaskWaitInTests	Info/Disabled	Complex — replace with await Task.Delay()
MSTEST0018	DynamicDataShouldBeValid	Warning	Complex — validation, no obvious single fix
MSTEST0014	DataRowShouldBeValid	Warning	Complex — validation, context-dependent
MSTEST0055	IgnoreStringMethodReturnValue	Warning	Complex — intent-dependent
MSTEST0048	TestContextPropertyUsage	Warning	Complex — multi-case property pattern
MSTEST0058	AvoidAssertsInCatchBlocks	Info	Complex — refactoring required
MSTEST0041	UseConditionBaseWithTestClass	Warning	Complex — base class wiring
MSTEST0050	GlobalTestFixtureShouldBeValid	Warning	Complex — signature validation
MSTEST0063	TestClassConstructorShouldBeValid	Warning	Complex — constructor signature
MSTEST0032	ReviewAlwaysTrueAssertCondition	Warning	Complex — no single correct transformation

Findings

Strengths

• 41 of 65 rules (63%) already have working code fixes — a solid foundation.
• The MSTest.Analyzers.CodeFixes project has a clear, mature pattern (CodeFixProvider + FixableDiagnosticIds + EquivalenceKey + RegisterCodeFixesAsync).
• Helper infrastructure (FixtureMethodFixer, AssertToAssertFixerHelpers) reduces duplication for common fix patterns.
• Fixer tests are co-located in the same analyzer test file via VerifyCS.VerifyCodeFixAsync, making the coverage gap easy to measure.

Areas for Improvement

• 37% of rules lack fixes (High): Users who encounter these diagnostics get no automated remediation, reducing the value of the linting experience.
• Mechanical fixes are low-hanging fruit (High): At least 7 of the missing fixes are single-line or two-line attribute mutations that should take <1 hour each to implement.
• MSTEST0001 is a common onboarding rule (Medium): New users frequently need to add [Parallelized] to their assembly; the absence of a one-click fix is a notable UX gap.
• MSTEST0059 conflicts need dual-option fix (Medium): When [Parallelized] and [DoNotParallelize] coexist, users should be offered "Remove Parallelized" or "Remove DoNotParallelize" as separate fix actions.
• MSTEST0067 (Thread.Sleep replacement) (Low): Disabled by default but the most educational fix — replacing Thread.Sleep(ms) with await Task.Delay(ms) and adjusting the method signature would be extremely useful when the rule is enabled.

---

🤖 Suggested Improvement Tasks

Task 1: Add code fix for MSTEST0001 — UseParallelizeAttribute

Priority: Medium
Estimated Effort: Small (1–2 hours)

Add a UseParallelizeAttributeFixer that registers against DiagnosticIds.UseParallelizedAttributeRuleId. When the assembly has no [assembly: Parallelize()] attribute, the fixer should insert one (defaulting to Workers = 0, Scope = ExecutionScope.MethodLevel). The fix location is the assembly-level symbol reported by the analyzer.

• File to create: src/Analyzers/MSTest.Analyzers.CodeFixes/UseParallelizeAttributeFixer.cs
• Test coverage: Add VerifyCS.VerifyCodeFixAsync cases to test/UnitTests/MSTest.Analyzers.UnitTests/UseParallelizeAttributeAnalyzerTests.cs

---

Task 2: Add code fix for MSTEST0024 — DoNotStoreStaticTestContext

Priority: High
Estimated Effort: Small (1–2 hours)

The analyzer flags assignments of TestContext to a static field/property. The fix is to remove the static modifier from the field or auto-property declaration. The reported diagnostic location is the SimpleAssignment operation, so the fixer needs to walk up to the field/property declaration containing the static keyword and remove it using SyntaxEditor or DocumentEditor.

• File to create: src/Analyzers/MSTest.Analyzers.CodeFixes/DoNotStoreStaticTestContextFixer.cs
• Test coverage: Add VerifyCS.VerifyCodeFixAsync cases to test/UnitTests/MSTest.Analyzers.UnitTests/DoNotStoreStaticTestContextAnalyzerTests.cs

---

Task 3: Add code fix for MSTEST0059 — DoNotUseParallelizeAndDoNotParallelize

Priority: High
Estimated Effort: Small (1–2 hours)

When both [assembly: Parallelize(...)] and [assembly: DoNotParallelize] are present, the diagnostic fires on one of the conflicting attributes. The fixer should offer two separate code actions — one to remove [Parallelize] and one to remove [DoNotParallelize] — so users can choose their intent. Both are AttributeSyntax removals, which is a straightforward DocumentEditor operation.

• File to create: src/Analyzers/MSTest.Analyzers.CodeFixes/DoNotUseParallelizeAndDoNotParallelizeFixer.cs
• Test coverage: Add VerifyCS.VerifyCodeFixAsync cases to test/UnitTests/MSTest.Analyzers.UnitTests/UseParallelizeAttributeAnalyzerTests.cs

---

Task 4: Add code fix for MSTEST0035 — UseDeploymentItemWithTestMethodOrTestClass

Priority: Medium
Estimated Effort: Small (1–2 hours)

When a method or class uses [DeploymentItem] but lacks [TestMethod] or [TestClass], the fixer should add the missing attribute. The reported diagnostic is on the [DeploymentItem] attribute; the fixer adds [TestMethod] to a method or [TestClass] to a class as applicable. This is an attribute-list insertion, similar to the existing AddTestClassFixer.

• File to create: src/Analyzers/MSTest.Analyzers.CodeFixes/UseDeploymentItemWithTestMethodOrTestClassFixer.cs
• Test coverage: Add VerifyCS.VerifyCodeFixAsync cases to test/UnitTests/MSTest.Analyzers.UnitTests/UseDeploymentItemWithTestMethodOrTestClassAnalyzerTests.cs

---

Task 5: Add code fix for MSTEST0067 — AvoidThreadSleepAndTaskWaitInTests

Priority: Low (rule is disabled by default)
Estimated Effort: Medium (3–5 hours)

Thread.Sleep(n) → await Task.Delay(n) and task.Wait() / task.Result → await task are the two most common patterns flagged by this analyzer. For Thread.Sleep, the fixer must:

1. Replace the Thread.Sleep(n) invocation with await Task.Delay(n).
2. Add async to the containing method if not already present.
3. Change the containing method return type from void → Task (or Task<T> if applicable).

This is more involved than a single-node edit but follows the same pattern as AvoidUsingAssertsInAsyncVoidContextFixer and PreferAsyncAssertionFixer.

• File to create: src/Analyzers/MSTest.Analyzers.CodeFixes/AvoidThreadSleepAndTaskWaitInTestsFixer.cs
• Test coverage: Add VerifyCS.VerifyCodeFixAsync cases to test/UnitTests/MSTest.Analyzers.UnitTests/AvoidThreadSleepAndTaskWaitInTestsAnalyzerTests.cs

---

📊 Historical Context

Previous Focus Areas

Date	Focus Area	Type
2026-05-22	Test Framework API Ergonomics	Custom
2026-05-25	Agentic Workflow Maintainability	Custom
2026-05-26	Workflow Ecosystem Health	Custom
2026-05-27	Test Diagnostic Experience	Custom
2026-06-08	TODO Comment Policy Compliance	Custom
2026-06-09	MSTest Diagnostic ID Governance	Custom

---

🎯 Recommendations

Immediate Actions (This Week)

1. Add code fix for MSTEST0024 (DoNotStoreStaticTestContext) — Priority: High, pure mechanical fix
2. Add code fix for MSTEST0059 (DoNotUseParallelizeAndDoNotParallelize) — Priority: High, dual-option remove

Short-term Actions (This Month)

1. Add code fix for MSTEST0001 (UseParallelizeAttribute) — Priority: Medium, common onboarding pattern
2. Add code fix for MSTEST0035 (UseDeploymentItemWithTestMethodOrTestClass) — Priority: Medium
3. Add code fix for MSTEST0067 (AvoidThreadSleepAndTaskWaitInTests) — Priority: Low, high-value for async modernization

Next analysis: 2026-06-11 — Focus area selected based on diversity algorithm

🤖 Automated content by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Repository Quality Improver workflow.{ai_credits_suffix} · [◷]( · ◷)

Add this agentic workflows to your repo

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/repository-quality-improver.md@main

• expires on Jun 12, 2026, 11:00 PM UTC