Skip to content

Sanitize scenario exception reports #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
RMANOV merged 1 commit into from
Apr 25, 2026
Merged

Sanitize scenario exception reports #10

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions python/tests/test_strix_scenario_contract.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,29 @@ def test_validate_scenario_rejects_missing_seed_and_bad_bounds(tmp_path):
assert "coverage: min must be <= max" in result["errors"]


def test_validate_scenario_parse_errors_are_public_safe(tmp_path):
module = _load_module()
scenario = tmp_path / "broken.yaml"
scenario.write_text("scenario_id: [unterminated\n", encoding="utf-8")

result = module.validate_scenario(scenario)

assert result["status"] == "failed"
assert str(tmp_path) not in result["path"]
assert str(tmp_path) not in result["errors"][0]


def test_validate_scenario_read_errors_are_public_safe(tmp_path):
module = _load_module()
scenario = tmp_path / "missing.yaml"

result = module.validate_scenario(scenario)

assert result["status"] == "failed"
assert str(tmp_path) not in result["path"]
assert str(tmp_path) not in result["errors"][0]


Comment on lines +103 to +104
Copy link

Copilot AI Apr 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new public-safe tests cover POSIX absolute paths, but they won’t catch the common OSError.__str__ behavior of rendering filenames via repr(filename) (escaping backslashes). Adding a regression that constructs a FileNotFoundError(..., filename="C:\\tmp\\missing.yaml") and asserts the redaction removes both the raw and repr-escaped forms would help ensure Windows-style paths are also public-safe.

Suggested change
windows_path = "C:\\tmp\\missing.yaml"
windows_error = FileNotFoundError(2, "No such file or directory", windows_path)
sanitized_error = module._public_message(str(windows_error))
assert windows_path not in sanitized_error
assert repr(windows_path)[1:-1] not in sanitized_error

Copilot uses AI. Check for mistakes.
def test_public_scenarios_satisfy_contract():
module = _load_module()
scenario_dir = Path(__file__).resolve().parents[2] / "sim" / "scenarios"
Expand Down
25 changes: 24 additions & 1 deletion scripts/strix_scenario_contract.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,24 @@ def public_path(path: Path) -> str:
return str(path)


def public_exception_message(exc: Exception, path: Path) -> str:
"""Return an exception message with local filesystem paths redacted."""

message = str(exc) or type(exc).__name__
replacements = {str(path): public_path(path)}
filename = getattr(exc, "filename", None)
filename2 = getattr(exc, "filename2", None)
for candidate in (filename, filename2):
if candidate:
candidate_path = Path(candidate)
replacements[str(candidate_path)] = public_path(candidate_path)

for raw, safe in replacements.items():
if raw:
message = message.replace(raw, safe)
return message
Comment on lines +40 to +55
Copy link

Copilot AI Apr 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

public_exception_message() replaces only the literal str(path) / str(Path(exc.filename)) substrings. For OSError/FileNotFoundError, Python’s default __str__ often embeds the filename using repr(filename), which can escape backslashes (e.g. C:\\tmp\\x.yaml) and won’t match str(Path(...)), leaving absolute paths unredacted on Windows-style inputs. Consider also redacting the repr(...) form (and/or normalizing both slash styles) when applying replacements so the sanitization works across platforms.

Copilot uses AI. Check for mistakes.


def load_yaml(path: Path) -> dict[str, Any]:
data = yaml.safe_load(path.read_text(encoding="utf-8"))
if not isinstance(data, dict):
Expand All @@ -55,7 +73,12 @@ def validate_scenario(path: Path) -> dict[str, Any]:
try:
data = load_yaml(path)
except Exception as exc: # noqa: BLE001 - report parse errors as validation failures
return {"path": str(path), "status": "failed", "errors": [str(exc)], "warnings": []}
return {
"path": public_path(path),
"status": "failed",
"errors": [public_exception_message(exc, path)],
"warnings": [],
}

required = [
"scenario_id",
Expand Down
Loading