Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/RUNNER.md
Original file line number Diff line number Diff line change
Expand Up @@ -311,10 +311,10 @@ const result: FullFlowResult = await runner.callFullFlow(

**Built-in responder behavior** — controlled by request headers set before the origin phase:

| Header | Effect |
| -------------------- | ------------------------------------------------------------------------------- |
| `x-debugger-status` | HTTP status code for the generated response (default: `200`) |
| `x-debugger-content` | Response body mode: `"body-only"`, `"status-only"`, or full JSON echo (default) |
| Header | Effect |
| -------------------- | -------------------------------------------------------------------------------- |
| `x-debugger-status` | HTTP status code for the generated response (default: `200`) |
| `x-debugger-content` | Response body mode: `"body-only"`, `"status-only"`, or full JSON echo (default) |

When `x-debugger-content` is omitted, the built-in responder returns a JSON echo of the request method, headers, body, and URL. Both control headers are stripped before response hooks execute so they do not appear in hook input state.

Expand Down
38 changes: 19 additions & 19 deletions docs/TEST_CONFIG.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,25 +22,25 @@ The config schema is a union of two variants selected by `appType`:

### Top-Level Fields

| JSON Path | Type | Required (Schema) | Default | Description |
| ------------------ | --------- | -------------------------------------- | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `$schema` | `string` | No | — | URI pointing to the JSON Schema file for IDE autocompletion and validation. |
| `description` | `string` | No | — | Human-readable label for this test scenario. |
| `wasm` | `object` | No | — | WASM binary configuration. Required when running without a programmatic `wasmBuffer`. |
| `wasm.path` | `string` | Yes (if `wasm` present) | — | Path to the compiled `.wasm` binary, relative to the config file or absolute. |
| `wasm.description` | `string` | No | — | Human-readable label for the WASM binary. |
| `appType` | `string` | Yes (schema) / CDN has runtime default | `"proxy-wasm"` | App variant. `"proxy-wasm"` for CDN mode; `"http-wasm"` for HTTP mode. HTTP-WASM has no default. |
| `request` | `object` | **Yes** | — | Incoming HTTP request to simulate. |
| `request.method` | `string` | Yes (schema) / runtime default | `"GET"` | HTTP method (e.g. `"GET"`, `"POST"`). |
| `request.url` | `string` | **Yes** (CDN only) | — | Full URL for the simulated upstream request (e.g. `"https://example.com/api"`). CDN mode only. |
| `request.path` | `string` | **Yes** (HTTP-WASM only) | — | Request path (e.g. `"/api/submit"`). HTTP-WASM mode only. The WASM module acts as the origin server and receives only the path portion of the request. |
| `request.headers` | `object` | Yes (schema) / runtime default | `{}` | Key/value map of request headers. All keys and values must be strings. |
| `request.body` | `string` | Yes (schema) / runtime default | `""` | Request body as a plain string. Use an empty string for requests with no body. |
| `properties` | `object` | **Yes** (schema) / runtime default | `{}` | CDN property key/value pairs passed to the WASM execution context. Values may be any JSON type. |
| `dotenv` | `object` | No | — | Dotenv file loading configuration. |
| `dotenv.enabled` | `boolean` | No | — | Whether to load a `.env` file before execution. |
| `dotenv.path` | `string` | No | — | Path to the `.env` file. If omitted, resolves `.env` relative to the config file directory. |
| `httpPort` | `integer` | No | — | HTTP-WASM only. Pin the subprocess to a specific port (1024–65535) instead of dynamic allocation from the 8100–8199 pool. Throws if the port is busy. |
| JSON Path | Type | Required (Schema) | Default | Description |
| ------------------ | --------- | -------------------------------------- | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `$schema` | `string` | No | — | URI pointing to the JSON Schema file for IDE autocompletion and validation. |
| `description` | `string` | No | — | Human-readable label for this test scenario. |
| `wasm` | `object` | No | — | WASM binary configuration. Required when running without a programmatic `wasmBuffer`. |
| `wasm.path` | `string` | Yes (if `wasm` present) | — | Path to the compiled `.wasm` binary, relative to the config file or absolute. |
| `wasm.description` | `string` | No | — | Human-readable label for the WASM binary. |
| `appType` | `string` | Yes (schema) / CDN has runtime default | `"proxy-wasm"` | App variant. `"proxy-wasm"` for CDN mode; `"http-wasm"` for HTTP mode. HTTP-WASM has no default. |
| `request` | `object` | **Yes** | — | Incoming HTTP request to simulate. |
| `request.method` | `string` | Yes (schema) / runtime default | `"GET"` | HTTP method (e.g. `"GET"`, `"POST"`). |
| `request.url` | `string` | **Yes** (CDN only) | — | Full URL for the simulated upstream request (e.g. `"https://example.com/api"`). CDN mode only. Use `"built-in"` for the local responder. |
| `request.path` | `string` | **Yes** (HTTP-WASM only) | — | Request path (e.g. `"/api/submit"`). HTTP-WASM mode only. The WASM module acts as the origin server and receives only the path portion of the request. |
| `request.headers` | `object` | Yes (schema) / runtime default | `{}` | Key/value map of request headers. All keys and values must be strings. |
| `request.body` | `string` | Yes (schema) / runtime default | `""` | Request body as a plain string. Use an empty string for requests with no body. |
| `properties` | `object` | **Yes** (schema) / runtime default | `{}` | CDN property key/value pairs passed to the WASM execution context. Values may be any JSON type. |
| `dotenv` | `object` | No | — | Dotenv file loading configuration. |
| `dotenv.enabled` | `boolean` | No | — | Whether to load a `.env` file before execution. |
| `dotenv.path` | `string` | No | — | Path to the `.env` file. If omitted, resolves `.env` relative to the config file directory. |
| `httpPort` | `integer` | No | — | HTTP-WASM only. Pin the subprocess to a specific port (1024–65535) instead of dynamic allocation from the 8100–8199 pool. Throws if the port is busy. |

### Required vs. Default Distinction

Expand Down
14 changes: 7 additions & 7 deletions docs/TEST_FRAMEWORK.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,13 +195,13 @@ interface RunnerConfig {
}
```

| Field | Type | Description |
| -------------------------------- | ------------------------------- | ------------------------------------------------------------------------- |
| `dotenv.enabled` | `boolean` | Enable dotenv loading |
| `dotenv.path` | `string` | Directory to load dotenv files from; defaults to process CWD when omitted |
| `enforceProductionPropertyRules` | `boolean` | Override production property enforcement for the runner; default `true` |
| `runnerType` | `"http-wasm" \| "proxy-wasm"` | Override automatic WASM type detection |
| `httpPort` | `number` | Pin the HTTP server to a specific port (HTTP WASM only; throws if in use) |
| Field | Type | Description |
| -------------------------------- | ------------------------------ | ------------------------------------------------------------------------- |
| `dotenv.enabled` | `boolean` | Enable dotenv loading |
| `dotenv.path` | `string` | Directory to load dotenv files from; defaults to process CWD when omitted |
| `enforceProductionPropertyRules` | `boolean` | Override production property enforcement for the runner; default `true` |
| `runnerType` | `"http-wasm" \| "proxy-wasm"` | Override automatic WASM type detection |
| `httpPort` | `number` | Pin the HTTP server to a specific port (HTTP WASM only; throws if in use) |

## Functions

Expand Down
49 changes: 36 additions & 13 deletions fastedge-plugin-source/generate-docs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,11 @@ $existing_doc
"
fi

# Build prompt with sandwich output constraint:
# The OUTPUT CONSTRAINT appears at both the start and end of the prompt.
# This is critical for large prompts where the model may lose track of
# the instruction to output only raw markdown. Without it, the model
# sometimes produces conversational preamble or asks for permission.
local prompt
prompt="$(cat <<PROMPT
OUTPUT CONSTRAINT: Your output is piped directly to a file. Output ONLY raw markdown. No conversational text. No preamble. No "here is" or "I'll generate". No "the existing content is accurate", "no changes needed", "outputting verbatim", or any similar acknowledgement — this applies equally when the existing content needs no changes. No questions. No explanation. No permission requests. Start your very first character with # (the level-1 heading). End with the last line of markdown.
Expand Down Expand Up @@ -221,7 +226,8 @@ PROMPT
tmpfile=$(mktemp "$DOCS_DIR/.${target}.XXXXXX")
# Note: we do NOT trap-rm the tmpfile on RETURN. Failed-validation output is
# preserved under $DOCS_DIR/.failures/ for prompt-debugging (see below) —
# the success path mv's the tmpfile out, the interrupt path rm's it explicitly,
# the success path writes stripped content directly to docs/$target and deletes
# the tmpfile, the interrupt path rm's it explicitly,
# and after max_attempts we explicitly rm the working tmpfile.

# Failure-preservation directory. Accumulates across runs so you can compare
Expand All @@ -231,6 +237,9 @@ PROMPT
local failure_dir="$DOCS_DIR/.failures"
mkdir -p "$failure_dir"

# Retry loop: validate that the model produced raw markdown (starts with #)
# On large prompts, the model occasionally produces conversational output
# despite the sandwich constraint. Retrying usually succeeds.
local max_attempts=3
local attempt=1
while [ $attempt -le $max_attempts ]; do
Expand All @@ -253,14 +262,22 @@ PROMPT
# the doc. The original is still saved to .failures/ so the prompt can
# be tuned later.
local stripped
stripped=$(awk '/^#/ { found=1 } found' "$tmpfile")

if [ -n "$stripped" ]; then
# Detect preamble: anything before the first line starting with '#' is preamble.
# Matches the prompt's constraint ("first character is #") rather than requiring
# '# ' (hash + space), so a model that emits '#Title' still salvages.
# Fence-aware level-1-only salvage: track ``` fences so that #-prefixed
# lines inside a fence (shell comments, #include, etc.) don't trigger the
# heading-detection scan. Fence delimiter lines are NOT skipped — they fall
# through to `found { print }` so fenced code blocks are preserved intact
# in the output. Only a bare `# ` heading outside a fence sets found=1.
stripped=$(awk '/^```/ { in_fence = !in_fence } !in_fence && /^# / { found=1 } found' "$tmpfile")

# Post-strip validation: confirm the salvaged output actually starts with a
# level-1 heading (# followed by a space). Belt-and-suspenders against any
# remaining edge case; reject and retry rather than silently writing junk.
local first_nonempty
first_nonempty=$(printf '%s\n' "$stripped" | grep -m1 '.')
if [ -n "$stripped" ] && [[ "$first_nonempty" =~ ^"# " ]]; then
Comment thread
godronus marked this conversation as resolved.
# Detect preamble: anything before the first level-1 heading is preamble.
local first_heading_line
first_heading_line=$(grep -n -m1 '^#' "$tmpfile" | cut -d: -f1)
first_heading_line=$(grep -n -m1 '^# ' "$tmpfile" | cut -d: -f1)
if [ "${first_heading_line:-1}" -gt 1 ]; then
local preamble_copy="$failure_dir/${target}.preamble.attempt-${attempt}.$(date +%s).md"
cp "$tmpfile" "$preamble_copy"
Expand All @@ -272,16 +289,15 @@ PROMPT
return 0
fi

# No level-1 heading anywhere — genuine failure. Save and retry.
# No valid level-1 heading found — genuine failure. Save and retry.
local failed_copy="$failure_dir/${target}.attempt-${attempt}.$(date +%s).md"
cp "$tmpfile" "$failed_copy"
echo " Attempt $attempt/$max_attempts failed for $target (no '# ' heading found in output) — saved to $failed_copy, retrying..."
echo " Attempt $attempt/$max_attempts failed for $target (no level-1 heading found in output) — saved to $failed_copy, retrying..."
attempt=$((attempt + 1))
done

rm -f "$tmpfile"
echo " FAILED after $max_attempts attempts: docs/$target"
echo " Inspect saved attempts: $failure_dir/${target}.attempt-*.md"
return 1
}

Expand All @@ -293,6 +309,11 @@ run_tier() {
local pids=()
local failed=()

# Skip empty tiers
if [ ${#files[@]} -eq 0 ]; then
return 0
fi

echo "--- $tier_name (${#files[@]} files in parallel) ---"

for target in "${files[@]}"; do
Expand Down Expand Up @@ -343,8 +364,10 @@ if [ "$run_all" = true ]; then
exit 1
fi

# Regenerate llms.txt from docs/ contents
"$SCRIPT_DIR/generate-llms-txt.sh"
# Regenerate llms.txt from docs/ contents (if the script is installed)
if [ -x "$SCRIPT_DIR/generate-llms-txt.sh" ]; then
"$SCRIPT_DIR/generate-llms-txt.sh"
fi
else
# Specific files: run sequentially (user chose explicit order)
failed=()
Expand Down
Loading