fix: correct grader path handling in ContainerGrader and entrypoint

ContainerGrader had two bugs affecting how grader files were located
inside the container at runtime:

1. Docker backend bind-mounted the grader problem directory at /grader,
   overwriting the grader_support package that the base image copies
   there.  Fixed by binding at /graders instead and passing the
   resulting absolute in-container path (/graders/<file>) to the
   entrypoint.

2. Kubernetes backend set working_dir to the grader problem directory
   (e.g. /graders/ps07/Robot/), preventing Python from finding the
   grader_support package which lives at /grader/grader_support/.
   Fixed by keeping working_dir=/grader (the base image WORKDIR) and
   passing the absolute grader path in args instead of just the
   basename.

entrypoint.py previously passed the full absolute path verbatim to
__import__(), which fails for paths containing slashes.  It now detects
absolute paths, inserts the parent directory into sys.path, and uses
only the basename as the importable module name.

Also updates grader_support/README.md to document the correct layout
(/graders/ for course grader scripts, /grader/ for grader_support) and
the gradelib compatibility note for course teams migrating from
Python 2 graders.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: blarghmatey

grader_support/README.md

@@ -5,7 +5,8 @@ This Dockerfile builds the base image that all course-specific grader images ext
 ### What it contains
 
 - Python 3.11 (slim)
-- The `grader_support` package (test framework and runner used by all graders)
+- The `grader_support` package (test framework and runner used by all graders) at `/grader/grader_support/`
+- A non-root `grader` user (UID 1000)
 - An entrypoint that reads student submissions from the `SUBMISSION_CODE` environment variable
 
 ### Building
@@ -22,19 +23,45 @@ make docker-build
 
 ### Course team usage
 
-Course teams create their own image `FROM grader-base` and add their grader scripts and any Python dependencies their graders require:
+Course teams create their own image `FROM grader-base` and add their grader scripts plus any Python dependencies required by the graders.
+
+#### Directory layout inside the container
+
+```
+/grader/                        ← WORKDIR (base image); grader_support lives here
+└── grader_support/             ← test framework (gradelib, run, entrypoint)
+
+/graders/                       ← course grader scripts (course team copies these)
+└── ps01/
+│   └── Problem1/
+│       ├── grade_Problem1.py   ← grader script (defines `grader = Grader()`)
+│       └── answer.py           ← reference solution
+└── ml/
+    └── cluster/
+        └── grade_cluster.py
+```
+
+`grader_root` in the handler config should point to `/graders/` (or a subdirectory of it).  The `SUBMISSION_CODE` env var carries student code; the entrypoint writes it to `/tmp` (a writable tmpfs even when the root filesystem is read-only).
+
+#### Example course Dockerfile
 
 ```dockerfile
-FROM registry.example.com/grader-base:latest
+# syntax=docker/dockerfile:1
+ARG GRADER_BASE_IMAGE=ghcr.io/mitodl/xqueue-watcher-grader-base:latest
+FROM ${GRADER_BASE_IMAGE}
+
+# pip must run as root; the base image ends with USER grader.
+USER root
+RUN pip install --no-cache-dir numpy==1.26.4 scipy==1.13.0
 
-# Install course-specific Python dependencies
-RUN pip install --no-cache-dir numpy==1.26.4 scipy==1.12.0
+# Copy grader scripts to /graders/.  Do NOT copy them to /grader/ — that
+# would overwrite the grader_support package from the base image.
+COPY --chown=grader:grader graders/ /graders/
 
-# Copy grader scripts into the image
-COPY graders/ /graders/
+USER grader
 ```
 
-Then reference the image in the xqueue-watcher handler config (`conf.d/my-course.json`):
+#### Example handler config (`conf.d/my-course.json`)
 
 ```json
 {
@@ -46,7 +73,7 @@ Then reference the image in the xqueue-watcher handler config (`conf.d/my-course
       {
         "HANDLER": "xqueue_watcher.containergrader.ContainerGrader",
         "KWARGS": {
-          "grader_root": "/graders/my-course/",
+          "grader_root": "/graders/",
           "image": "registry.example.com/my-course-grader:latest",
           "backend": "kubernetes",
           "cpu_limit": "500m",
@@ -59,6 +86,8 @@ Then reference the image in the xqueue-watcher handler config (`conf.d/my-course
 }
 ```
 
+The `grader` field inside each xqueue submission payload should be a path **relative to `grader_root`**, e.g. `"ps01/Problem1/grade_Problem1.py"`.
+
 ### Security properties
 
 Grader containers run with:
@@ -67,3 +96,11 @@ Grader containers run with:
 - No network access (`network_disabled: true` / Kubernetes NetworkPolicy)
 - CPU and memory limits enforced by the container runtime
 - Hard wall-clock timeout via `activeDeadlineSeconds` (Kubernetes) or `timeout` (Docker)
+
+### Important: `gradelib` compatibility
+
+The `grader_support/__init__.py` injects the framework's Python 3 `gradelib` and
+`graderutil` modules into `sys.modules` before any grader file is imported.  This
+means grader scripts that do `from gradelib import *` receive the framework version
+automatically, even if a legacy `gradelib.py` exists elsewhere on disk.  Course teams
+do not need to ship their own copy of `gradelib.py`.

grader_support/entrypoint.py

@@ -47,6 +47,18 @@ def main():
     sys.path.insert(0, "/tmp")
     sys.path.insert(0, cwd)
 
+    # grader_name may be an absolute path (e.g. /graders/ps07/Robot/grade_Robot.py)
+    # when the containergrader passes the full in-container path.  In that case, add
+    # the grader's directory to sys.path and use only the basename as the module name
+    # so that __import__() can find it.
+    if os.path.isabs(grader_name):
+        grader_dir = os.path.dirname(grader_name)
+        grader_module = os.path.basename(grader_name)
+        if grader_dir not in sys.path:
+            sys.path.insert(0, grader_dir)
+    else:
+        grader_module = grader_name
+
     # Rename so the module name matches what run.py expects (submission_name
     # without the .py extension becomes the importable module name).
     import shutil
@@ -56,7 +68,7 @@ def main():
 
     seed_int = int(seed)
     output = _run_module.run(
-        grader_name[:-3] if grader_name.endswith(".py") else grader_name,
+        grader_module[:-3] if grader_module.endswith(".py") else grader_module,
         submission_name[:-3] if submission_name.endswith(".py") else submission_name,
         seed_int,
     )

xqueue_watcher/containergrader.py

@@ -142,8 +142,13 @@ def _build_k8s_job(self, job_name, grader_path, code, seed):
         """Return a kubernetes Job manifest dict for the given grading run."""
         from kubernetes import client as k8s_client
 
-        grader_rel = str(Path(grader_path).basename())
-        grader_dir = str(Path(grader_path).dirname())
+        # Pass the absolute in-container path to the grader file.  The
+        # entrypoint handles absolute paths by adding the parent directory to
+        # sys.path before importing the module, so Python can find the file.
+        # working_dir must stay at /grader (the WORKDIR of the base image)
+        # so that `python -m grader_support.entrypoint` can locate the
+        # grader_support package.
+        grader_abs = str(grader_path)
 
         return k8s_client.V1Job(
             api_version="batch/v1",
@@ -170,8 +175,8 @@ def _build_k8s_job(self, job_name, grader_path, code, seed):
                             k8s_client.V1Container(
                                 name="grader",
                                 image=self.image,
-                                args=[grader_rel, "submission.py", str(seed)],
-                                working_dir=grader_dir,
+                                args=[grader_abs, "submission.py", str(seed)],
+                                working_dir="/grader",
                                 env=[
                                     k8s_client.V1EnvVar(
                                         name="SUBMISSION_CODE",
@@ -238,15 +243,20 @@ def _run_docker(self, grader_path, code, seed):
 
         grader_dir = str(Path(grader_path).dirname().absolute())
         grader_rel = str(Path(grader_path).basename())
+        # Mount the problem directory at /graders/ (not /grader/ which would
+        # overwrite the base image's grader_support package).  Pass the grader
+        # as an absolute in-container path so the entrypoint can add its parent
+        # directory to sys.path before importing.
+        container_grader_path = f"/graders/{grader_rel}"
 
         client = docker_sdk.from_env()
         try:
             result = client.containers.run(
                 image=self.image,
-                command=[grader_rel, "submission.py", str(seed)],
+                command=[container_grader_path, "submission.py", str(seed)],
                 working_dir="/grader",
                 environment={"SUBMISSION_CODE": code},
-                volumes={grader_dir: {"bind": "/grader", "mode": "ro"}},
+                volumes={grader_dir: {"bind": "/graders", "mode": "ro"}},
                 mem_limit=self.memory_limit,
                 nano_cpus=int(_parse_cpu_millis(self.cpu_limit) * 1_000_000),
                 network_disabled=True,