Conversation
There was a problem hiding this comment.
Pull request overview
Adds a new functional + security regression test suite for the training plugin to catch syntax issues, validate ability YAML content, and detect insecure patterns in Python sources.
Changes:
- Add AST-based syntax validation for
hook.pyand all plugin.pyfiles - Add abilities YAML discovery/parsing + basic schema checks
- Add security pattern scans for
verify=False,shell=True, and missingrequeststimeouts
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+68
to
+73
| """data/abilities/ directory should exist.""" | ||
| assert os.path.isdir(os.path.join(PLUGIN_DIR, "data", "abilities")) | ||
|
|
||
| def test_abilities_yaml_files_exist(self): | ||
| """There should be at least one YAML ability file.""" | ||
| assert len(self._yaml_files()) > 0, "No .yml files found in data/abilities/" |
Comment on lines
+75
to
+84
| def test_abilities_yaml_parseable(self): | ||
| """Each abilities YAML file should be parseable.""" | ||
| import yaml | ||
| for yf in self._yaml_files(): | ||
| with open(yf, "r") as fh: | ||
| try: | ||
| docs = list(yaml.safe_load_all(fh)) | ||
| except yaml.YAMLError as exc: | ||
| rel = os.path.relpath(yf, PLUGIN_DIR) | ||
| pytest.fail(f"YAML parse error in {rel}: {exc}") |
| Tests cover: | ||
| - hook.py syntax validation via ast.parse | ||
| - Abilities YAML validation (if present) | ||
| - Requirements.txt dependency checks (if present) |
Comment on lines
+169
to
+191
| pattern = re.compile(r"requests\.(get|post|put|delete|patch|head)\(") | ||
| for fpath in self._py_files(): | ||
| with open(fpath, "r") as fh: | ||
| source = fh.read() | ||
| for match in pattern.finditer(source): | ||
| start = match.start() | ||
| depth = 0 | ||
| end = start | ||
| for i in range(start, min(start + 500, len(source))): | ||
| if source[i] == "(": | ||
| depth += 1 | ||
| elif source[i] == ")": | ||
| depth -= 1 | ||
| if depth == 0: | ||
| end = i | ||
| break | ||
| call_text = source[start:end] | ||
| if "timeout" not in call_text: | ||
| line_num = source[:start].count("\n") + 1 | ||
| rel = os.path.relpath(fpath, PLUGIN_DIR) | ||
| pytest.fail( | ||
| f"requests call without timeout at {rel}:{line_num}" | ||
| ) |
Comment on lines
+25
to
+26
| with open(hook_path, "r") as fh: | ||
| source = fh.read() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add functional and security regression tests