Fix(eval): block build-script internet for submissions#41
Merged
Conversation
A submission's compile.sh runs as root in the build container and could otherwise smuggle install/download steps into the build. Block internet unconditionally during compile.sh via an in-container DNS blackhole (overwrite /etc/resolv.conf with nameserver 0.0.0.0, restore after) — no host privileges, works under docker-in-docker. Test-execution containers are left untouched. Internal-reference: b993951da7a644d49af19073b745c16c513cc316 Internal-reference: fad6005e0b633e103308ef4bc339848ad4bbc569
meta-cla
Bot
added
the
CLA Signed
klieret
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
Adds build-time internet isolation for submission compilation by DNS-blackholing inside the build container, reducing the ability for compile.sh to download/install dependencies during the build phase.
Changes:
- Add
utils/internet_control.pywith helpers to backup/overwrite/restore/etc/resolv.confto a DNS blackhole. - Wire the block/restore around the
compile.shstep inEvaluator._compile_executable. - Add unit tests covering both the DNS rewrite helpers and the evaluator wiring; document the behavior in
CLAUDE.md.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| tests/test_internet_control.py | Adds tests for DNS block/restore helpers and verifies evaluator wraps compile with block/restore (including failure path). |
| src/programbench/utils/internet_control.py | Implements in-container DNS blackhole by overwriting /etc/resolv.conf with nameserver 0.0.0.0 and restoring from a backup. |
| src/programbench/eval/eval.py | Calls DNS block before compile.sh and restores DNS afterward via try/finally. |
| CLAUDE.md | Documents the new build-time internet isolation behavior and threat model/trade-offs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A submission's compile.sh runs as root in the build container and could otherwise smuggle install/download steps into the build. Block internet unconditionally during compile.sh via an in-container DNS blackhole (overwrite /etc/resolv.conf with nameserver 0.0.0.0, restore after) — no host privileges, works under docker-in-docker. Test-execution containers are left untouched.
Internal-reference: b993951da7a644d49af19073b745c16c513cc316
Internal-reference: fad6005e0b633e103308ef4bc339848ad4bbc569
Closes #43