Skip to content

Improve pr-docs-check reliability guidance#17552

Open
IEvangelist wants to merge 1 commit into
mainfrom
ievangelist/dapine-pr-docs-failures
Open

Improve pr-docs-check reliability guidance#17552
IEvangelist wants to merge 1 commit into
mainfrom
ievangelist/dapine-pr-docs-failures

Conversation

@IEvangelist
Copy link
Copy Markdown
Member

Description

This improves reliability of the pr-docs-check agentic workflow by tightening prompt guidance around known safe-output failure modes, while keeping core workflow mechanics (auth, checkout, and branch-resolution flow) unchanged.

Key improvements:

  • Add hard reliability guardrails for generated docs edits so the agent avoids broad/high-risk doc patches and handles oversized change sets through report_incomplete.
  • Add explicit locked-source-PR handling so notification output does not attempt comment posting when the source PR is locked.
  • Recompile workflow artifacts to keep the lock files in sync with source guidance updates.

Validation:

  • gh aw compile .github/workflows/pr-docs-check.md completed successfully with no errors/warnings.

Fixes # (issue)

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • Yes
      • If yes, did you have an API Review for it?
        • Yes
        • No
      • Did you add <remarks /> and <code /> elements on your triple slash comments?
        • Yes
        • No
    • No
  • Does the change make any security assumptions or guarantees?
    • Yes
      • If yes, have you done a threat model and had a security review?
        • Yes
        • No
    • No

Add explicit guardrails in the pr-docs-check workflow prompt to reduce deterministic safe-output failures, then recompile workflow artifacts.

- Add constraints to keep generated docs patches targeted and under safe-output limits.
- Add explicit handling for locked source PRs to avoid hard-failing notify flow.
- Regenerate lock artifacts via gh aw compile.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@IEvangelist IEvangelist marked this pull request as ready for review May 27, 2026 18:08
Copilot AI review requested due to automatic review settings May 27, 2026 18:08
@github-actions
Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 17552

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 17552"

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR tightens the pr-docs-check agentic workflow prompt with reliability guardrails (limit doc patch size, fall back to report_incomplete on risky changes, and skip notify_source_pr when the source PR is locked) and recompiles the lock files. The compiled .lock.yml was regenerated with gh-aw v0.74.8, bringing along upstream toolchain bumps (firewall 0.25.49, mcpg v0.3.9, github-mcp-server v1.0.4, copilot CLI 1.0.48, model default claude-sonnet-4.5).

Changes:

  • Add "Reliability guardrails for generated doc edits" and "Locked source PR handling" sections to pr-docs-check.md.
  • Recompile pr-docs-check.lock.yml with gh-aw v0.74.8 (action/container pins, span-id propagation, DOCKER_HOST handling, token-steering AWF config).
  • Update actions-lock.json to pin github/gh-aw-actions/setup@v0.74.8 and drop the unused setup-cli entry.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
.github/workflows/pr-docs-check.md Adds hard reliability guardrails and locked-PR handling guidance to the workflow prompt.
.github/workflows/pr-docs-check.lock.yml Regenerated by gh aw compile with v0.74.8 toolchain bumps.
.github/aw/actions-lock.json Bumps gh-aw-actions/setup pin to v0.74.8; removes stale setup-cli entry.

@github-actions
Copy link
Copy Markdown
Contributor

CLI E2E Tests failed — 106 passed, 1 failed, 2 unknown (commit 40387de)

Failed Tests

View all recordings
Status Test Recording
AddPackageInteractiveWhileAppHostRunningDetached ▶️ View recording
AddPackageWhileAppHostRunningDetached ▶️ View recording
AgentCommands_AllHelpOutputs_AreCorrect ▶️ View recording
AgentInitCommand_DefaultSelection_InstallsDefaultSkills ▶️ View recording
AgentInitCommand_MigratesDeprecatedConfig ▶️ View recording
AgentMcpListStructuredLogsReturnsLogsFromStarterApp ▶️ View recording
AgentMcpListStructuredLogsReturnsLogsFromStarterApp_DevLocalhost ▶️ View recording
AgentMcpListStructuredLogsReturnsLogsFromStarterApp_Isolated ▶️ View recording
AllPublishMethodsBuildDockerImages ▶️ View recording
AspireAddAndStartWorkAgainstLegacyAppHostTs ▶️ View recording
AspireAddPackageVersionToDirectoryPackagesProps ▶️ View recording
AspireInitSingleFileAppHostRunsViaDotnetRunAppHost ▶️ View recording
AspireInitWithExistingAppHostDirRecreatesMissingNuGetConfigAndPreservesFiles ▶️ View recording
AspireInitWithSolutionFileGeneratesAppHostThatBuildsAgainstChannelHive ▶️ View recording
AspireStartUpdatesStaleTypeScriptAppHostPath ▶️ View recording
AspireUpdateRemovesAppHostPackageVersionFromDirectoryPackagesProps ▶️ View recording
AspireUpdateRemovesOrphanAppHostPackageVersionWhenSdkAlreadyCurrent ▶️ View recording
Banner_DisplayedOnFirstRun ▶️ View recording
Banner_DisplayedWithExplicitFlag ▶️ View recording
Banner_NotDisplayedWithNoLogoFlag ▶️ View recording
CertificatesClean_RemovesCertificates ▶️ View recording
CertificatesTrust_WithNoCert_CreatesAndTrustsCertificate ▶️ View recording
CertificatesTrust_WithUntrustedCert_TrustsCertificate ▶️ View recording
ConfigSetGet_CreatesNestedJsonFormat ▶️ View recording
CreateAndRunAspireStarterProject ▶️ View recording
CreateAndRunAspireStarterProjectWithBundle ▶️ View recording
CreateAndRunEmptyAppHostProject ▶️ View recording
CreateAndRunJavaEmptyAppHostProject ▶️ View recording
CreateAndRunJsReactProject ▶️ View recording
CreateAndRunPythonReactProject ▶️ View recording
CreateAndRunTypeScriptEmptyAppHostProject ▶️ View recording
CreateAndRunTypeScriptStarterProject ▶️ View recording
CreateJavaAppHostWithViteApp ▶️ View recording
CreateTypeScriptAppHostWithViteApp_AllowsGuestAppPackageManagerToDiffer ▶️ View recording
CreateTypeScriptAppHostWithViteApp_UsesConfiguredToolchain ▶️ View recording
DashboardRunWithAgentMcpListTracesReturnsNoTraces ▶️ View recording
DashboardRunWithAgentMcpListTracesReturnsNoTraces_DevLocalhost ▶️ View recording
DashboardRunWithOtelTracesReturnsNoTraces ▶️ View recording
DashboardRunWithOtelTracesReturnsNoTraces_DevLocalhost ▶️ View recording
DeployK8sBasicApiService ▶️ View recording
DeployK8sWithExternalHelmChart ▶️ View recording
DeployK8sWithGarnet ▶️ View recording
DeployK8sWithMongoDB ▶️ View recording
DeployK8sWithMySql ▶️ View recording
DeployK8sWithPostgres ▶️ View recording
DeployK8sWithRabbitMQ ▶️ View recording
DeployK8sWithRedis ▶️ View recording
DeployK8sWithSqlServer ▶️ View recording
DeployK8sWithValkey ▶️ View recording
DeployTypeScriptAppToKubernetes ▶️ View recording
DescribeCommandResolvesReplicaNames ▶️ View recording
DescribeCommandShowsRunningResources ▶️ View recording
DetachFormatJsonProducesValidJson ▶️ View recording
DetachFormatJsonProducesValidJsonWhenRestartingExistingInstance ▶️ View recording
DoPublishAndDeployListStepsWork ▶️ View recording
DocsCommand_RendersInteractiveMarkdownFromLocalSource ▶️ View recording
DoctorCommand_DetectsDeprecatedAgentConfig ▶️ View recording
DoctorCommand_TypeScriptAppHostReportsMissingConfiguredToolchain ▶️ View recording
DoctorCommand_WithSslCertDir_ShowsTrusted ▶️ View recording
DoctorCommand_WithoutSslCertDir_ShowsPartiallyTrusted ▶️ View recording
GatewayWithoutExternalEndpoint_FailsPublishWithGuidance ▶️ View recording
GeneratedAspireDevScript_StartsWatchMode_WithConfiguredToolchain ▶️ View recording
GlobalMigration_HandlesCommentsAndTrailingCommas ▶️ View recording
GlobalMigration_HandlesMalformedLegacyJson ▶️ View recording
GlobalMigration_PreservesAllValueTypes ▶️ View recording
GlobalMigration_SkipsWhenNewConfigExists ▶️ View recording
GlobalSettings_MigratedFromLegacyFormat ▶️ View recording
IngressWithoutExternalEndpoint_FailsPublishWithGuidance ▶️ View recording
InitTypeScriptAppHost_AugmentsExistingViteRepoInWorkspaceSubdirectory ▶️ View recording
InteractiveCSharpInitCreatesExpectedFiles ▶️ View recording
InvalidAppHostPathWithComments_IsHealedOnRun ▶️ View recording
JavaScriptHostingApisRunFromTypeScriptAppHost ▶️ View recording
LatestCliCanStartStableChannelAppHost ▶️ View recording
LatestCliCanStartStableChannelTypeScriptAppHost ▶️ View recording
LegacySettingsMigration_AdjustsRelativeAppHostPath ▶️ View recording
LogsCommandShowsResourceLogs ▶️ View recording
OtelLogsReturnsStructuredLogsFromStarterApp ▶️ View recording
OtelLogsReturnsStructuredLogsFromStarterAppIsolated ▶️ View recording
PsCommandListsRunningAppHost ▶️ View recording
PsFormatJsonOutputsOnlyJsonToStdout ▶️ View recording
PublishJavaScriptPatternsGeneratesExpectedDockerComposeArtifacts ▶️ View recording
PublishWithConfigureEnvFileUpdatesEnvOutput ▶️ View recording
PublishWithDockerComposeServiceCallbackSucceeds ▶️ View recording
PublishWithoutOutputPathUsesAppHostDirectoryDefault ▶️ View recording
ResourceCommand_FailedExecution_DisplaysAppHostLogPathAndLogContainsEntries ▶️ View recording
ResourceCommand_SetAndDeleteParameterUpdatesDescribeOutput ▶️ View recording
RestoreGeneratesSdkFiles ▶️ View recording
RestoreGeneratesSdkFiles_WithConfiguredToolchain ▶️ View recording
RestoreRefreshesGeneratedSdkAfterAddingIntegration ▶️ View recording
RestoreSupportsConfigOnlyHelperPackageAndCrossPackageTypes ▶️ View recording
RunFromParentDirectory_UsesExistingConfigNearAppHost ▶️ View recording
RunReportsSyntaxErrorsForDotNetAppHost ▶️ View recording
RunReportsSyntaxErrorsForTypeScriptAppHost ▶️ View recording
SecretCrudOnDotNetAppHost ▶️ View recording
SecretCrudOnTypeScriptAppHost ▶️ View recording
StagingChannel_ConfigureAndVerifySettings_ThenSwitchChannels ▶️ View recording
StartAndWaitForTypeScriptSqlServerAppHostWithNativeAssets ▶️ View recording
StartReportsSyntaxErrorsForDotNetAppHost ▶️ View recording
StartReportsSyntaxErrorsForTypeScriptAppHost ▶️ View recording
StopAllAppHostsFromAppHostDirectory ▶️ View recording
StopJavaPolyglotAppHostUsingApphostDirectory ▶️ View recording
StopNonInteractiveSingleAppHost ▶️ View recording
StopTypeScriptPolyglotAppHostUsingApphostDirectory ▶️ View recording
StopWithNoRunningAppHostExitsSuccessfully ▶️ View recording
UnAwaitedChainsCompileWithAutoResolvePromises ▶️ View recording
UpdateProjectChannelToStable_CSharpEmptyAppHost_PreservesAspireConfigChannel ▶️ View recording
UpdateProjectChannelToStable_CSharpSingleFileInit_PreservesAspireConfigChannel ▶️ View recording
UpdateProjectChannelToStable_TypeScriptSingleFileInit_PreservesAspireConfigChannel ▶️ View recording
UpdateProjectChannelToStable_TypeScript_PreviewsStablePackagesAndPreservesChannel ▶️ View failure recording

📹 Recordings uploaded automatically from CI run #26529541957

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants