Add DSPM for AI tests (MT.1172-MT.1176)#1703
Conversation
Up to standards ✅🟢 Issues
|
There was a problem hiding this comment.
Pull request overview
Adds Microsoft Purview DSPM for AI governance coverage to Maester by introducing five new MT.1152–MT.1156 tests (Copilot audit ingestion, sensitivity labels, insider risk, DLP, retention) along with their remediation and website documentation, config registration, test harness, and module exports.
Changes:
- Add 5 new Purview-focused Maester PowerShell tests (MT.1152–MT.1156) with detailed markdown result output.
- Add remediation markdown pages and Docusaurus website docs for each new test.
- Register the tests in
maester-config.json, add a consolidated Pester suite, and export the functions inMaester.psd1.
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| website/docs/tests/maester/MT.1152.md | Website doc for unified audit log ingestion prerequisite test. |
| website/docs/tests/maester/MT.1153.md | Website doc for published file-scoped sensitivity labels prerequisite test. |
| website/docs/tests/maester/MT.1154.md | Website doc for Insider Risk “Risky AI usage” policy test. |
| website/docs/tests/maester/MT.1155.md | Website doc for Copilot-scoped DLP policy test. |
| website/docs/tests/maester/MT.1156.md | Website doc for Copilot-scoped retention policy test. |
| tests/maester-config.json | Adds MT.1152–MT.1156 entries to the test catalog (ID/severity/title). |
| tests/Maester/Purview/Test-MtPurviewAi.Tests.ps1 | New Pester suite invoking the 5 Purview AI tests. |
| powershell/public/maester/purview/Test-MtPurviewAiAuditLogIngestion.ps1 | Implements MT.1152 (unified audit log ingestion enabled). |
| powershell/public/maester/purview/Test-MtPurviewAiAuditLogIngestion.md | Remediation/details page for MT.1152. |
| powershell/public/maester/purview/Test-MtPurviewAiSensitivityLabelsForFiles.ps1 | Implements MT.1153 (published file-scoped sensitivity labels). |
| powershell/public/maester/purview/Test-MtPurviewAiSensitivityLabelsForFiles.md | Remediation/details page for MT.1153. |
| powershell/public/maester/purview/Test-MtPurviewAiInsiderRiskPolicy.ps1 | Implements MT.1154 (Risky AI usage Insider Risk policy enabled). |
| powershell/public/maester/purview/Test-MtPurviewAiInsiderRiskPolicy.md | Remediation/details page for MT.1154. |
| powershell/public/maester/purview/Test-MtPurviewAiDlpPolicy.ps1 | Implements MT.1155 (Copilot-targeted DLP policy enabled, non-simulation). |
| powershell/public/maester/purview/Test-MtPurviewAiDlpPolicy.md | Remediation/details page for MT.1155. |
| powershell/public/maester/purview/Test-MtPurviewAiRetentionPolicy.ps1 | Implements MT.1156 (Copilot-targeted retention policy enabled). |
| powershell/public/maester/purview/Test-MtPurviewAiRetentionPolicy.md | Remediation/details page for MT.1156. |
| powershell/Maester.psd1 | Exports the 5 new test functions from the module. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
MT.1153 (Test-MtPurviewAiSensitivityLabelsForFiles): Cross-reference Get-LabelPolicy output to verify a file-scoped label is actually included in a published policy, not merely that *some* file-scoped label exists alongside *some* published policy. Adds a third row to the result table showing the published file-scoped label count. MT.1154 (Test-MtPurviewAiInsiderRiskPolicy): Narrow the 'cmdlet unavailable / not licensed' skip to CommandNotFoundException and matching 'is not recognized' / 'is unavailable' messages. All other exceptions now fall back to -SkippedBecause Error -SkippedError so transient service errors, throttling, and module/session issues are no longer masked as licensing skips.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Address Copilot follow-up review on PR maester365#1703: A label policy is now treated as 'published' only when Mode is 'Enforce' AND (when the Enabled property is exposed by the SCC schema) Enabled is true. The previous '-or' logic could classify a disabled Enforce-mode policy or an enabled non-Enforce-mode policy as published, which is inconsistent with the in-code comment and remediation docs.
|
@copilot resolve the merge conflicts in this pull request |
|
Thanks for putting this together — these DSPM for AI checks are a really useful addition, and the overall PR shape looks good: helper functions are exported, companion docs and website docs are present, the test IDs are wired through config, and the targeted PowerShell/Pester validation I ran did not show syntax or analyzer issues. I found two areas that I think should be tightened before merge so the checks do not false-fail tenants that are configured through the current Microsoft Purview surfaces:
Also, GitHub currently reports the PR as conflicting with Thanks again for the contribution — once the policy-surface detection is broadened, these look much closer to being merge-ready. |
SamErde
left a comment
There was a problem hiding this comment.
See previous comment. Thanks! 🙂
Adds five Maester tests covering Microsoft Purview Data Security Posture Management for AI: - MT.1172 Unified audit log ingestion is enabled for AI activity - MT.1173 Sensitivity labels are published for files used by Microsoft 365 Copilot - MT.1174 Insider Risk Management policy for Risky AI usage is enabled - MT.1175 DLP policy is configured for the Microsoft 365 Copilot location - MT.1176 Retention policy is configured for the Microsoft Copilot location MT.1175 detects Copilot DLP coverage across the multiple shapes exposed by Get-DlpCompliancePolicy: MicrosoftCopilotLocation, Workload (MicrosoftCopilot), Locations, and EnforcementPlanes (CopilotExperiences). MT.1176 queries both retention surfaces -- legacy Get-RetentionCompliancePolicy and the newer Get-AppRetentionCompliancePolicy (User:M365Copilot) -- via the Get-MtExo cache wrapper. Passes if either surface protects Copilot. Test IDs originally proposed as MT.1152-MT.1156 were renumbered to MT.1172-MT.1176 to avoid collision with the Defender antivirus metadata batch added upstream in 7ebade5.
|
Thanks for the careful review @SamErde — addressed both points and pushed Renumbered to MT.1172-MT.1176 MT.1175 — broader Copilot DLP detection
MT.1176 — dual retention surface
Companion |
The build-validation Pester test "Should contain Write-Verbose logging" in powershell/tests/functions/Common.Tests.ps1 requires every exported public function to contain at least one Write-Verbose call. Four of the five new DSPM for AI tests were missing this, causing 4 failures across ubuntu-latest, macOS-latest and windows-latest runners. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Thanks for the PR @OfirGavish — really nice set of DSPM for AI tests! 🎉 The
That test enforces a project-wide convention that every exported public function must contain at least one Since
That should turn the build-validation checks green. Feel free to amend / squash the commit into your branch however you prefer — and thanks again for the contribution! |
Summary
Adds 5 Microsoft Purview Data Security Posture Management (DSPM) for AI governance tests covering Microsoft 365 Copilot oversharing, audit, and retention controls. Reserved in #697.
Test-MtPurviewAiAuditLogIngestionTest-MtPurviewAiSensitivityLabelsForFilesTest-MtPurviewAiInsiderRiskPolicyTest-MtPurviewAiDlpPolicyTest-MtPurviewAiRetentionPolicyWhy
Maester currently has solid coverage for Copilot Studio agent governance (MT.1113-1122) and Intune controls (MT.1123, MT.1147-1151), but has no tests covering the Microsoft Purview DSPM for AI controls that govern Microsoft 365 Copilot itself. These 5 tests close that gap by checking the most impactful DSPM-for-AI prerequisites and protective controls a tenant should have configured before broadly enabling Copilot.
What each test does
InsiderRiskScenario/Name/PolicyTemplatefor forward compatibility.Get-DlpCompliancePolicy:MicrosoftCopilotLocation,Workload(MicrosoftCopilot),Locations, andEnforcementPlanes(e.g.CopilotExperiences).Get-RetentionCompliancePolicyand the newerGet-AppRetentionCompliancePolicy(User:M365Copilot) — via theGet-MtExocache wrapper. Passes if either surface protects Copilot.Files
powershell/public/maester/purview/Test-MtPurviewAi*.ps1(5 functions)powershell/public/maester/purview/Test-MtPurviewAi*.md(5 companion remediation pages)website/docs/tests/maester/MT.117[2-6].md(5 website docs)tests/Maester/Purview/Test-MtPurviewAi.Tests.ps1(consolidated Pester suite)tests/maester-config.json(5 new entries)powershell/Maester.psd1(5 new functions added toFunctionsToExport)powershell/public/cisa/exchange/Get-MtExo.ps1(addedRetentionCompliancePolicy+AppRetentionCompliancePolicyto the cached commands hashtable so MT.1176 can route through the standardGet-MtExowrapper)Conventions
All tests follow the standard Maester pattern:
Test-MtConnectionguard,Get-MtLicenseInformationwhere applicable (MT.1175 only - DLP requires Exo DLP licence),try/catchwithNotAuthorized/Errorskips,Add-MtTestResultDetailwith Well done. prefix on pass,%TestResult%markdown table with risk callout on detail tests. EXO/SCC cmdlets are routed throughGet-MtExo -Request <CmdletSuffix>so they participate in$__MtSession.ExoCache.Validation
Invoke-ScriptAnalyzer -Severity Warning,Errorclean on all 6 new PS1 files.maester-config.jsonparses (5 new entries verified).Test-ModuleManifestsucceeds.Test-MtPurviewAiSensitivityLabelsForFileshas[SuppressMessage('PSUseSingularNouns', ...)]with justification (matches existing pattern fromTest-MtAIAgentMcpTools).True(PASS) on the test tenant; MT.1174 + MT.1175 + MT.1176 returnedFalsewith correctly-rendered detail markdown showing the empty-state row.Notes for reviewers
Get-SPOTenant, but Maester does not connect to the SharePoint Management Shell. Pivoted to checking the actual prerequisite (published file-scoped sensitivity labels) viaGet-Label+Get-LabelPolicy.SkippedBecause Custom(with reason) since there is no specificNotLicensedIRMenum value inAdd-MtTestResultDetail. Happy to add one in a follow-up if preferred.MicrosoftCopilotLocation,Workload,Locations,EnforcementPlanes).Get-RetentionCompliancePolicyandGet-AppRetentionCompliancePolicyper @SamErde's review, with graceful fallback if the app-retention cmdlet isn't available in the connected SCC session. Each policy is tagged with aSurfacecolumn (Legacy/App) in the result detail.