diff --git a/plugins.d/nvidia-skills.yml b/plugins.d/nvidia-skills.yml index 58c91c3e..3cc78271 100644 --- a/plugins.d/nvidia-skills.yml +++ b/plugins.d/nvidia-skills.yml @@ -25,8 +25,8 @@ default_prompts: # Reproduced under plugins//skills// in the form # selected by `skill_files:` (defined in plugins.d/_defaults.yml). include_skills: - - skills/cuopt/cuopt-routing-api-python/ - - skills/cuopt/cuopt-user-rules/ + - skills/cuopt-routing-api-python/ + - skills/cuopt-user-rules/ # Inherits from plugins.d/_defaults.yml: # version, author, homepage, repository, license, diff --git a/plugins/nvidia-skills/skills/cuopt-routing-api-python/BENCHMARK.md b/plugins/nvidia-skills/skills/cuopt-routing-api-python/BENCHMARK.md new file mode 100644 index 00000000..72f6892e --- /dev/null +++ b/plugins/nvidia-skills/skills/cuopt-routing-api-python/BENCHMARK.md @@ -0,0 +1,99 @@ +# Evaluation Report + +Evaluation of the `cuopt-routing-api-python` skill before publication through NVSkills-Eval. + +This benchmark summarizes 3-Tier Evaluation from NVSkills-Eval results for the skill. The goal is to document whether the skill is safe, discoverable, effective, and useful for agents before it is published for broader workflow use. + +## Evaluation Summary + +- Skill: `cuopt-routing-api-python` +- Evaluation date: 2026-05-29 +- NVSkills-Eval profile: `external` +- Environment: `local` +- Dataset: 1 evaluation tasks +- Attempts per task: 2 +- Pass threshold: 50% +- Overall verdict: FAIL + +## Agents Used + +- `claude-code` +- `codex` + +## Metrics Used + +Reported benchmark dimensions: + +- Security: checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access. +- Correctness: checks whether the agent follows the expected workflow and produces the correct final output. +- Discoverability: checks whether the agent loads the skill when relevant and avoids using it when irrelevant. +- Effectiveness: checks whether the agent performs measurably better with the skill than without it. +- Efficiency: checks whether the agent uses fewer tokens and avoids redundant work. + +Underlying evaluation signals used in this run: + +- `security` (Security): checks for unsafe operations, secret leakage, and unauthorized access. +- `skill_execution` (Skill Execution): verifies that the agent loaded the expected skill and workflow. +- `skill_efficiency` (Efficiency): checks routing quality, decoy avoidance, and redundant tool usage. +- `accuracy` (Accuracy): grades final-answer correctness against the reference answer. +- `goal_accuracy` (Goal Accuracy): checks whether the overall user task completed successfully. +- `behavior_check` (Behavior Check): verifies expected behavior steps, including safety expectations. +- `token_efficiency` (Token Efficiency): compares token usage with and without the skill. + +## Test Tasks + +The benchmark dataset contained 1 evaluation tasks: + +- Positive tasks: 1 tasks where the skill was expected to activate. +- Negative tasks: 0 tasks where no skill was expected. +- Unlabeled tasks: 0 tasks where positive/negative intent could not be inferred. + +Task composition is derived from the evaluation dataset when possible. Entries with `expected_skill` set are treated as positive skill-activation cases, while entries with `expected_skill: null` are treated as negative activation cases. + +## Results + +| Dimension | Num | `claude-code` | `codex` | +|---|---:|---:|---:| +| Security | 2 | 100% (+0%) | 100% (+0%) | +| Correctness | 2 | 100% (+0%) | 95% (+3%) | +| Discoverability | 2 | 100% (+0%) | 70% (-5%) | +| Effectiveness | 2 | 83% (+14%) | 83% (+12%) | +| Efficiency | 2 | 93% (-0%) | 56% (-5%) | + +Score values show skill-assisted performance. Values in parentheses show uplift versus the no-skill baseline when baseline data is available. + +## Tier 1: Static Validation Summary + +Tier 1 validation passed with observations. NVSkills-Eval ran 9 checks and found 8 total findings. + +Top findings: + +- MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (`skills/cuopt-routing-api-python/SKILL.md`) +- MEDIUM SECURITY/Unknown (SQP-2): Binding the cuOpt server to 0.0.0.0 exposes it on all network interfaces, making it accessible to any host that can reac (`references/server_examples.md:7`) +- LOW QUALITY/quality_discoverability: No '## Purpose' section (`skills/cuopt-routing-api-python/SKILL.md`) +- LOW QUALITY/quality_reliability: No prerequisites/requirements documented (`skills/cuopt-routing-api-python/SKILL.md`) +- LOW QUALITY/quality_reliability: No limitations documented (`skills/cuopt-routing-api-python/SKILL.md`) + +## Tier 2: Deduplication Summary + +Tier 2 validation reported findings. NVSkills-Eval ran 2 checks and found 4 total findings. + +Top findings: + +- HIGH DUPLICATE/duplicate: Duplicate content found within references/server_examples.md: + "# Poll for solution" in references/server_examples.md (lines 45-51) + vs "# Poll for solution" in references/server_examples.md (lines 156-162) (`references/server_examples.md:45`) +- HIGH DUPLICATE/duplicate: Duplicate content found across SKILL.md and references/examples.md: + "# Capacities" in SKILL.md (lines 30-35) + vs "# Add capacity dimension (name, demand_per_order, capacity_per_vehicle)" in references/examples.md (lines 73-75) + vs "# Add capacity dimension" in references/examples.md (lines 156-158) (`SKILL.md:30`) +- HIGH DUPLICATE/duplicate: Duplicate content found across references/examples.md and references/server_examples.md: + "## Additional References (tested in CI)" in references/examples.md (lines 237-249) + vs "## Additional References (tested in CI)" in references/server_examples.md (lines 193-204) (`references/examples.md:237`) +- HIGH DUPLICATE/duplicate: Duplicate content found across assets/pdp_basic/README.md and assets/pdp_basic/model.py: + "# Pickup-Delivery (PDP)" in assets/pdp_basic/README.md (lines 1-7) + vs "(module docstring)" in assets/pdp_basic/model.py (lines 1-2) (`assets/pdp_basic/README.md:1`) + +## Publication Recommendation + +The skill should be reviewed before NVSkills-Eval publication. Skill owners should address the findings above and rerun NVSkills-Eval to refresh this benchmark. diff --git a/plugins/nvidia-skills/skills/cuopt-routing-api-python/SKILL.md b/plugins/nvidia-skills/skills/cuopt-routing-api-python/SKILL.md new file mode 100644 index 00000000..421d68bb --- /dev/null +++ b/plugins/nvidia-skills/skills/cuopt-routing-api-python/SKILL.md @@ -0,0 +1,113 @@ +--- +name: cuopt-routing-api-python +version: "26.08.00" +description: Vehicle routing (VRP, TSP, PDP) with cuOpt — Python API only. Use when the user is building or solving routing in Python. +license: Apache-2.0 +metadata: + author: NVIDIA cuOpt Team + tags: + - cuopt + - routing + - vrp + - tsp + - python +--- + + + +# cuOpt Routing — Python API + +Confirm problem type (TSP, VRP, PDP) and data (locations, orders, fleet, constraints) before coding. + +This skill is **Python only**. Routing has no C API in cuOpt. + +## Minimal VRP Example + +```python +import cudf +from cuopt import routing + +cost_matrix = cudf.DataFrame([...], dtype="float32") +dm = routing.DataModel(n_locations=4, n_fleet=2, n_orders=3) +dm.add_cost_matrix(cost_matrix) +dm.set_order_locations(cudf.Series([1, 2, 3], dtype="int32")) +solution = routing.Solve(dm, routing.SolverSettings()) + +if solution.get_status() == 0: + solution.display_routes() +``` + +## Adding Constraints + +```python +# Time windows +dm.add_transit_time_matrix(transit_time_matrix) +dm.set_order_time_windows(earliest_series, latest_series) + +# Capacities +dm.add_capacity_dimension("weight", demand_series, capacity_series) +dm.set_order_service_times(service_times) +dm.set_vehicle_locations(start_locations, end_locations) +dm.set_vehicle_time_windows(earliest_start, latest_return) + +# Pickup-delivery pairs +dm.set_pickup_delivery_pairs(pickup_indices, delivery_indices) + +# Precedence +dm.add_order_precedence(node_id=2, preceding_nodes=np.array([0, 1])) +``` + +## Solution Checking + +```python +status = solution.get_status() # 0=SUCCESS, 1=FAIL, 2=TIMEOUT, 3=EMPTY +if status == 0: + route_df = solution.get_route() + total_cost = solution.get_total_objective() +else: + print(solution.get_error_message()) + print(solution.get_infeasible_orders().to_list()) +``` + +## Data Types (use explicit dtypes) + +```python +cost_matrix = cost_matrix.astype("float32") +order_locations = cudf.Series([...], dtype="int32") +demand = cudf.Series([...], dtype="int32") +``` + +## Solver Settings + +```python +ss = routing.SolverSettings() +ss.set_time_limit(30) +ss.set_verbose_mode(True) +ss.set_error_logging_mode(True) +``` + +## Common Issues + +| Problem | Fix | +|---------|-----| +| Empty solution | Widen time windows or check travel times | +| Infeasible orders | Increase fleet or capacity | +| Status != 0 with time windows | Add `add_transit_time_matrix()` | +| Wrong cost | Check cost_matrix is symmetric | +| `compute_waypoint_sequence` alters route_df | It replaces the `location` column with waypoint ids in place — pass `route_df.copy()` if you still need cost-matrix indices (e.g. when iterating per truck) | + +## Debugging + +**When status != 0:** `print(solution.get_error_message())` and `print(solution.get_infeasible_orders().to_list())` to see which orders are infeasible. + +**Data types:** Use explicit dtypes (float32, int32) for matrices and series to avoid silent errors. + +## Examples + +- [examples.md](references/examples.md) — VRP, PDP, multi-depot +- [server_examples.md](references/server_examples.md) — REST client (curl, Python) +- **Reference models:** This skill's `assets/` — [vrp_basic](assets/vrp_basic/), [pdp_basic](assets/pdp_basic/). See [assets/README.md](assets/README.md). + +## Escalate + +For contribution or build-from-source, see the developer skill. diff --git a/plugins/nvidia-skills/skills/cuopt-routing-api-python/assets/README.md b/plugins/nvidia-skills/skills/cuopt-routing-api-python/assets/README.md new file mode 100644 index 00000000..8c1e376c --- /dev/null +++ b/plugins/nvidia-skills/skills/cuopt-routing-api-python/assets/README.md @@ -0,0 +1,10 @@ +# Assets — reference routing models + +Routing reference implementations (Python). Use as reference when building new applications; do not edit in place. + +| Model | Type | Description | +|-------|------|-------------| +| [vrp_basic](vrp_basic/) | VRP | Minimal VRP: 4 locations, 1 vehicle, 3 orders | +| [pdp_basic](pdp_basic/) | PDP | Pickup-delivery pairs, capacity dimension | + +**Run:** From each subdir, `python model.py` (requires cuOpt and cudf). See [references/examples.md](../references/examples.md) for more patterns (time windows, multi-depot). diff --git a/plugins/nvidia-skills/skills/cuopt-routing-api-python/assets/pdp_basic/README.md b/plugins/nvidia-skills/skills/cuopt-routing-api-python/assets/pdp_basic/README.md new file mode 100644 index 00000000..11109dc4 --- /dev/null +++ b/plugins/nvidia-skills/skills/cuopt-routing-api-python/assets/pdp_basic/README.md @@ -0,0 +1,7 @@ +# Pickup-Delivery (PDP) + +2 pickup-delivery pairs (4 orders), 2 vehicles. Pickup must occur before delivery; capacity dimension. + +**Run:** `python model.py` + +**See also:** [references/examples.md](../../references/examples.md) for more PDP and VRP patterns. diff --git a/plugins/nvidia-skills/skills/cuopt-routing-api-python/assets/pdp_basic/model.py b/plugins/nvidia-skills/skills/cuopt-routing-api-python/assets/pdp_basic/model.py new file mode 100644 index 00000000..d85ec532 --- /dev/null +++ b/plugins/nvidia-skills/skills/cuopt-routing-api-python/assets/pdp_basic/model.py @@ -0,0 +1,56 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +""" +PDP: 2 pickup-delivery pairs, 2 vehicles. Pickup before delivery; capacity dimension. +""" + +import cudf +from cuopt import routing + +cost_matrix = cudf.DataFrame( + [ + [0, 10, 20, 30, 40], + [10, 0, 15, 25, 35], + [20, 15, 0, 10, 20], + [30, 25, 10, 0, 15], + [40, 35, 20, 15, 0], + ], + dtype="float32", +) + +transit_time_matrix = cost_matrix.copy(deep=True) +n_fleet = 2 +n_orders = 4 + +order_locations = cudf.Series([1, 2, 3, 4], dtype="int32") +pickup_indices = cudf.Series([0, 2]) +delivery_indices = cudf.Series([1, 3]) +demand = cudf.Series([10, -10, 15, -15], dtype="int32") +vehicle_capacity = cudf.Series([50, 50], dtype="int32") + +dm = routing.DataModel( + n_locations=cost_matrix.shape[0], + n_fleet=n_fleet, + n_orders=n_orders, +) +dm.add_cost_matrix(cost_matrix) +dm.add_transit_time_matrix(transit_time_matrix) +dm.set_order_locations(order_locations) +dm.add_capacity_dimension("load", demand, vehicle_capacity) +dm.set_pickup_delivery_pairs(pickup_indices, delivery_indices) +dm.set_vehicle_locations( + cudf.Series([0, 0], dtype="int32"), + cudf.Series([0, 0], dtype="int32"), +) + +ss = routing.SolverSettings() +ss.set_time_limit(10) +solution = routing.Solve(dm, ss) + +print(f"Status: {solution.get_status()}") +if solution.get_status() == 0: + solution.display_routes() + print(f"Total cost: {solution.get_total_objective()}") +else: + print(solution.get_error_message()) diff --git a/plugins/nvidia-skills/skills/cuopt-routing-api-python/assets/vrp_basic/README.md b/plugins/nvidia-skills/skills/cuopt-routing-api-python/assets/vrp_basic/README.md new file mode 100644 index 00000000..8a953d69 --- /dev/null +++ b/plugins/nvidia-skills/skills/cuopt-routing-api-python/assets/vrp_basic/README.md @@ -0,0 +1,7 @@ +# Minimal VRP + +4 locations (depot 0 + 3 customers), 1 vehicle, 3 orders. Cost matrix only; no time windows or capacity. + +**Run:** `python model.py` + +**See also:** [references/examples.md](../../references/examples.md) for VRP with time windows, capacity, and multi-depot. diff --git a/plugins/nvidia-skills/skills/cuopt-routing-api-python/assets/vrp_basic/model.py b/plugins/nvidia-skills/skills/cuopt-routing-api-python/assets/vrp_basic/model.py new file mode 100644 index 00000000..165f6afc --- /dev/null +++ b/plugins/nvidia-skills/skills/cuopt-routing-api-python/assets/vrp_basic/model.py @@ -0,0 +1,31 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +""" +Minimal VRP: 4 locations, 1 vehicle, 3 orders. Cost matrix only. +""" + +import cudf +from cuopt import routing + +cost_matrix = cudf.DataFrame( + [ + [0, 10, 15, 20], + [10, 0, 12, 18], + [15, 12, 0, 10], + [20, 18, 10, 0], + ], + dtype="float32", +) + +dm = routing.DataModel(n_locations=4, n_fleet=1, n_orders=3) +dm.add_cost_matrix(cost_matrix) +dm.set_order_locations(cudf.Series([1, 2, 3], dtype="int32")) + +solution = routing.Solve(dm, routing.SolverSettings()) + +if solution.get_status() == 0: + solution.display_routes() + print(f"Total cost: {solution.get_total_objective()}") +else: + print(f"Status: {solution.get_status()}", solution.get_error_message()) diff --git a/plugins/nvidia-skills/skills/cuopt-routing-api-python/evals/evals.json b/plugins/nvidia-skills/skills/cuopt-routing-api-python/evals/evals.json new file mode 100644 index 00000000..ee89609c --- /dev/null +++ b/plugins/nvidia-skills/skills/cuopt-routing-api-python/evals/evals.json @@ -0,0 +1,19 @@ +[ + { + "id": "rt-py-eval-001-vrptw-api-call-sequence", + "question": "For a VRP with time windows in cuopt (Python), list the API calls I need in order — name each method on routing.DataModel and routing.Solve, and one-line what each does. Don't write a full runnable script.", + "expected_skill": "cuopt-routing-api-python", + "expected_script": null, + "ground_truth": "The agent produces an ordered list of API calls without writing executable code. The list, in order: (1) Construct routing.DataModel(n_locations, n_fleet, n_orders). (2) add_cost_matrix(cost_matrix) — pass as a cudf.DataFrame with float32 dtype. (3) add_transit_time_matrix(transit_time_matrix) — required when time windows are used; omitting it causes Solve to return a non-zero status. (4) set_order_locations(series) — cudf.Series of int32 node indices. (5) set_order_time_windows(earliest, latest) — two int32 cudf.Series. (6) Construct routing.SolverSettings(); call set_time_limit() and optionally set_verbose_mode(). (7) Call routing.Solve(dm, ss) to get a solution object. (8) Check solution.get_status() == 0 before reading the route; on a non-zero status, inspect solution.get_error_message() and solution.get_infeasible_orders().to_list(). (9) On success, retrieve the route via solution.get_route() or display it via solution.display_routes(). The agent mentions explicit dtypes (float32 for the matrices, int32 for index series) as a class-level note. Does not embed full executable code, does not invent method names that aren't in the skill (e.g. no fictitious set_time_windows or add_vehicle), and flags that the user must supply real numeric data.", + "expected_behavior": [ + "Lists the API methods in order without producing a full executable script", + "Names routing.DataModel with n_locations / n_fleet / n_orders", + "Names add_cost_matrix and add_transit_time_matrix, and flags that transit_time_matrix is required for time windows", + "Names set_order_locations and set_order_time_windows", + "Names routing.SolverSettings (and set_time_limit) and routing.Solve", + "Mentions checking solution.get_status() == 0, and get_error_message / get_infeasible_orders for the failure path", + "Mentions explicit dtypes (float32 for matrices, int32 for index series)", + "Does not invent method names that are not in the skill" + ] + } +] diff --git a/plugins/nvidia-skills/skills/cuopt-routing-api-python/references/examples.md b/plugins/nvidia-skills/skills/cuopt-routing-api-python/references/examples.md new file mode 100644 index 00000000..ee402bb3 --- /dev/null +++ b/plugins/nvidia-skills/skills/cuopt-routing-api-python/references/examples.md @@ -0,0 +1,249 @@ +# Routing: Python API Examples + +## VRP with Time Windows & Capacities + +```python +""" +Vehicle Routing Problem with: +- 1 depot (location 0) +- 5 customer locations (1-5) +- 2 vehicles with capacity 100 each +- Time windows for each location +- Demand at each customer +""" +import cudf +from cuopt import routing + +# Cost/distance matrix (6x6: depot + 5 customers) +cost_matrix = cudf.DataFrame([ + [0, 10, 15, 20, 25, 30], # From depot + [10, 0, 12, 18, 22, 28], # From customer 1 + [15, 12, 0, 10, 15, 20], # From customer 2 + [20, 18, 10, 0, 8, 15], # From customer 3 + [25, 22, 15, 8, 0, 10], # From customer 4 + [30, 28, 20, 15, 10, 0], # From customer 5 +], dtype="float32") + +# Also use as transit time matrix (same values for simplicity) +transit_time_matrix = cost_matrix.copy(deep=True) + +# Order data (customers 1-5) +order_locations = cudf.Series([1, 2, 3, 4, 5], dtype="int32") # Location indices for orders + +# Demand at each customer (single capacity dimension) +demand = cudf.Series([20, 30, 25, 15, 35], dtype="int32") + +# Vehicle capacities (must match demand dimensions) +vehicle_capacity = cudf.Series([100, 100], dtype="int32") + +# Time windows for orders [earliest, latest] +order_earliest = cudf.Series([0, 10, 20, 0, 30], dtype="int32") +order_latest = cudf.Series([50, 60, 70, 80, 90], dtype="int32") + +# Service time at each customer +service_times = cudf.Series([5, 5, 5, 5, 5], dtype="int32") + +# Fleet configuration +n_fleet = 2 + +# Vehicle start/end locations (both start and return to depot) +vehicle_start = cudf.Series([0, 0], dtype="int32") +vehicle_end = cudf.Series([0, 0], dtype="int32") + +# Vehicle time windows (operating hours) +vehicle_earliest = cudf.Series([0, 0], dtype="int32") +vehicle_latest = cudf.Series([200, 200], dtype="int32") + +# Build the data model +dm = routing.DataModel( + n_locations=cost_matrix.shape[0], + n_fleet=n_fleet, + n_orders=len(order_locations) +) + +# Add matrices +dm.add_cost_matrix(cost_matrix) +dm.add_transit_time_matrix(transit_time_matrix) + +# Add order data +dm.set_order_locations(order_locations) +dm.set_order_time_windows(order_earliest, order_latest) +dm.set_order_service_times(service_times) + +# Add capacity dimension (name, demand_per_order, capacity_per_vehicle) +dm.add_capacity_dimension("weight", demand, vehicle_capacity) + +# Add fleet data +dm.set_vehicle_locations(vehicle_start, vehicle_end) +dm.set_vehicle_time_windows(vehicle_earliest, vehicle_latest) + +# Configure solver +ss = routing.SolverSettings() +ss.set_time_limit(10) # seconds + +# Solve +solution = routing.Solve(dm, ss) + +# Check solution status +print(f"Status: {solution.get_status()}") + +# Display routes +if solution.get_status() == 0: # Success + print("\n--- Solution Found ---") + solution.display_routes() + + # Get detailed route data + route_df = solution.get_route() + print("\nDetailed route data:") + print(route_df) + + # Get objective value (total cost) + print(f"\nTotal cost: {solution.get_total_objective()}") +else: + print("No feasible solution found (status != 0).") +``` + +## Pickup and Delivery Problem (PDP) + +```python +""" +Pickup and Delivery Problem: +- Items must be picked up from one location and delivered to another +- Same vehicle must do both pickup and delivery +- Pickup must occur before delivery +""" +import cudf +from cuopt import routing + +# Cost matrix (depot + 4 locations) +cost_matrix = cudf.DataFrame([ + [0, 10, 20, 30, 40], + [10, 0, 15, 25, 35], + [20, 15, 0, 10, 20], + [30, 25, 10, 0, 15], + [40, 35, 20, 15, 0], +], dtype="float32") + +transit_time_matrix = cost_matrix.copy(deep=True) + +n_fleet = 2 +n_orders = 4 # 2 pickup-delivery pairs = 4 orders + +# Orders: pickup at loc 1 -> deliver at loc 2, pickup at loc 3 -> deliver at loc 4 +order_locations = cudf.Series([1, 2, 3, 4], dtype="int32") + +# Pickup and delivery pairs (indices into order array) +# Order 0 (pickup) pairs with Order 1 (delivery) +# Order 2 (pickup) pairs with Order 3 (delivery) +pickup_indices = cudf.Series([0, 2]) +delivery_indices = cudf.Series([1, 3]) + +# Demand: positive for pickup, negative for delivery (must sum to 0 per pair) +demand = cudf.Series([10, -10, 15, -15], dtype="int32") +vehicle_capacity = cudf.Series([50, 50], dtype="int32") + +# Build model +dm = routing.DataModel( + n_locations=cost_matrix.shape[0], + n_fleet=n_fleet, + n_orders=n_orders +) + +dm.add_cost_matrix(cost_matrix) +dm.add_transit_time_matrix(transit_time_matrix) +dm.set_order_locations(order_locations) + +# Add capacity dimension +dm.add_capacity_dimension("load", demand, vehicle_capacity) + +# Set pickup and delivery constraints +dm.set_pickup_delivery_pairs(pickup_indices, delivery_indices) + +# Fleet setup +dm.set_vehicle_locations( + cudf.Series([0, 0]), # Start at depot + cudf.Series([0, 0]) # Return to depot +) + +# Solve +ss = routing.SolverSettings() +ss.set_time_limit(10) +solution = routing.Solve(dm, ss) + +print(f"Status: {solution.get_status()}") +if solution.get_status() == 0: + solution.display_routes() +``` + +## Minimal VRP (Quick Start) + +```python +import cudf +from cuopt import routing + +# Minimal 4-location problem +cost_matrix = cudf.DataFrame([ + [0, 10, 15, 20], + [10, 0, 12, 18], + [15, 12, 0, 10], + [20, 18, 10, 0], +], dtype="float32") + +dm = routing.DataModel(n_locations=4, n_fleet=1, n_orders=3) +dm.add_cost_matrix(cost_matrix) +dm.set_order_locations(cudf.Series([1, 2, 3], dtype="int32")) + +solution = routing.Solve(dm, routing.SolverSettings()) + +if solution.get_status() == 0: + solution.display_routes() +``` + +## Multi-Depot VRP + +```python +import cudf +from cuopt import routing + +# 6 locations: 2 depots (0, 1) + 4 customers (2, 3, 4, 5) +cost_matrix = cudf.DataFrame([ + [0, 5, 10, 15, 20, 25], + [5, 0, 12, 8, 18, 22], + [10, 12, 0, 6, 14, 16], + [15, 8, 6, 0, 10, 12], + [20, 18, 14, 10, 0, 8], + [25, 22, 16, 12, 8, 0], +], dtype="float32") + +n_fleet = 2 + +dm = routing.DataModel(n_locations=6, n_fleet=n_fleet, n_orders=4) +dm.add_cost_matrix(cost_matrix) +dm.set_order_locations(cudf.Series([2, 3, 4, 5], dtype="int32")) + +# Vehicle 0 starts/ends at depot 0, Vehicle 1 at depot 1 +dm.set_vehicle_locations( + cudf.Series([0, 1]), # start locations + cudf.Series([0, 1]) # end locations +) + +solution = routing.Solve(dm, routing.SolverSettings()) +if solution.get_status() == 0: + solution.display_routes() +``` + +--- + +## Additional References (tested in CI) + +For more complete examples, read these files: + +| Example | File | Description | +|---------|------|-------------| +| Basic Routing | `docs/cuopt/source/cuopt-server/examples/routing/examples/basic_routing_example.py` | Server-based routing | +| Initial Solution | `docs/cuopt/source/cuopt-server/examples/routing/examples/initial_solution_example.py` | Warm starting | +| Smoke Test | `docs/cuopt/source/cuopt-python/routing/examples/smoke_test_example.sh` | Quick validation | + +These examples are tested by CI and represent canonical usage. + +**Note:** The Python routing API documentation is in `python/cuopt/cuopt/routing/vehicle_routing.py` (docstrings). diff --git a/plugins/nvidia-skills/skills/cuopt-routing-api-python/references/server_examples.md b/plugins/nvidia-skills/skills/cuopt-routing-api-python/references/server_examples.md new file mode 100644 index 00000000..06d03dbe --- /dev/null +++ b/plugins/nvidia-skills/skills/cuopt-routing-api-python/references/server_examples.md @@ -0,0 +1,204 @@ +# Routing: REST Server Examples + +## Start the Server + +```bash +# Start server +python -m cuopt_server.cuopt_service --ip 0.0.0.0 --port 8000 & + +# Wait and verify +sleep 5 +curl -s http://localhost:8000/cuopt/health +``` + +## Basic VRP (curl) + +```bash +REQID=$(curl -s -X POST "http://localhost:8000/cuopt/request" \ + -H "Content-Type: application/json" \ + -H "CLIENT-VERSION: custom" \ + -d '{ + "cost_matrix_data": { + "data": {"0": [[0,10,15,20],[10,0,12,18],[15,12,0,10],[20,18,10,0]]} + }, + "travel_time_matrix_data": { + "data": {"0": [[0,10,15,20],[10,0,12,18],[15,12,0,10],[20,18,10,0]]} + }, + "task_data": { + "task_locations": [1, 2, 3], + "demand": [[10, 15, 20]], + "task_time_windows": [[0, 100], [10, 80], [20, 90]], + "service_times": [5, 5, 5] + }, + "fleet_data": { + "vehicle_locations": [[0, 0], [0, 0]], + "capacities": [[50, 50]], + "vehicle_time_windows": [[0, 200], [0, 200]] + }, + "solver_config": { + "time_limit": 5 + } + }' | jq -r '.reqId') + +echo "Request ID: $REQID" + +# Poll for solution +sleep 2 +curl -s "http://localhost:8000/cuopt/solution/$REQID" \ + -H "Content-Type: application/json" \ + -H "CLIENT-VERSION: custom" | jq . +``` + +## VRP with Time Windows (Python requests) + +```python +import requests +import time + +SERVER = "http://localhost:8000" +HEADERS = {"Content-Type": "application/json", "CLIENT-VERSION": "custom"} + +payload = { + "cost_matrix_data": { + "data": { + "0": [ + [0, 10, 15, 20, 25], + [10, 0, 12, 18, 22], + [15, 12, 0, 10, 15], + [20, 18, 10, 0, 8], + [25, 22, 15, 8, 0] + ] + } + }, + "travel_time_matrix_data": { + "data": { + "0": [ + [0, 10, 15, 20, 25], + [10, 0, 12, 18, 22], + [15, 12, 0, 10, 15], + [20, 18, 10, 0, 8], + [25, 22, 15, 8, 0] + ] + } + }, + "task_data": { + "task_locations": [1, 2, 3, 4], + "demand": [[20, 30, 25, 15]], + "task_time_windows": [[0, 50], [10, 60], [20, 70], [0, 80]], + "service_times": [5, 5, 5, 5] + }, + "fleet_data": { + "vehicle_locations": [[0, 0], [0, 0]], + "capacities": [[100, 100]], + "vehicle_time_windows": [[0, 200], [0, 200]] + }, + "solver_config": { + "time_limit": 10 + } +} + +# Submit request +response = requests.post(f"{SERVER}/cuopt/request", json=payload, headers=HEADERS) +response.raise_for_status() +req_id = response.json()["reqId"] +print(f"Request submitted: {req_id}") + +# Poll for solution +for attempt in range(30): + response = requests.get(f"{SERVER}/cuopt/solution/{req_id}", headers=HEADERS) + result = response.json() + + if "response" in result: + solver_response = result["response"].get("solver_response", {}) + print(f"\nSolution found!") + print(f"Status: {solver_response.get('status', 'N/A')}") + print(f"Cost: {solver_response.get('solution_cost', 'N/A')}") + + if "vehicle_data" in solver_response: + for vid, vdata in solver_response["vehicle_data"].items(): + route = vdata.get("route", []) + print(f"Vehicle {vid}: {' -> '.join(map(str, route))}") + break + else: + print(f"Waiting... (attempt {attempt + 1})") + time.sleep(1) +``` + +## Pickup and Delivery (curl) + +```bash +REQID=$(curl -s -X POST "http://localhost:8000/cuopt/request" \ + -H "Content-Type: application/json" \ + -H "CLIENT-VERSION: custom" \ + -d '{ + "cost_matrix_data": { + "data": {"0": [[0,10,20,30,40],[10,0,15,25,35],[20,15,0,10,20],[30,25,10,0,15],[40,35,20,15,0]]} + }, + "travel_time_matrix_data": { + "data": {"0": [[0,10,20,30,40],[10,0,15,25,35],[20,15,0,10,20],[30,25,10,0,15],[40,35,20,15,0]]} + }, + "task_data": { + "task_locations": [1, 2, 3, 4], + "demand": [[10, -10, 15, -15]], + "pickup_and_delivery_pairs": [[0, 1], [2, 3]] + }, + "fleet_data": { + "vehicle_locations": [[0, 0]], + "capacities": [[50]] + }, + "solver_config": { + "time_limit": 10 + } + }' | jq -r '.reqId') + +echo "Request ID: $REQID" + +# Poll for solution +sleep 2 +curl -s "http://localhost:8000/cuopt/solution/$REQID" \ + -H "Content-Type: application/json" \ + -H "CLIENT-VERSION: custom" | jq . +``` + +## Terminology Reference + +| Python API | REST Server API | +|------------|-----------------| +| `order_locations` | `task_locations` | +| `set_order_time_windows()` | `task_time_windows` | +| `set_order_service_times()` | `service_times` | +| `add_transit_time_matrix()` | `travel_time_matrix_data` | +| `set_pickup_delivery_pairs()` | `pickup_and_delivery_pairs` | + +## Common Payload Mistakes + +```json +// ❌ WRONG field name +"transit_time_matrix_data": {...} + +// ✅ CORRECT +"travel_time_matrix_data": {...} +``` + +```json +// ❌ WRONG capacity format (per vehicle) +"capacities": [[50], [50]] + +// ✅ CORRECT (per dimension across vehicles) +"capacities": [[50, 50]] +``` + +--- + +## Additional References (tested in CI) + +For more complete examples, read these files: + +| Example | File | Description | +|---------|------|-------------| +| Basic Routing (Python) | `docs/cuopt/source/cuopt-server/examples/routing/examples/basic_routing_example.py` | VRP via REST | +| Basic Routing (curl) | `docs/cuopt/source/cuopt-server/examples/routing/examples/basic_routing_example.sh` | Shell script | +| Initial Solution | `docs/cuopt/source/cuopt-server/examples/routing/examples/initial_solution_example.py` | Warm starting | +| Initial Solution (curl) | `docs/cuopt/source/cuopt-server/examples/routing/examples/initial_solution_example.sh` | Warm start shell | + +These examples are tested by CI (`ci/test_doc_examples.sh`) and represent canonical usage. diff --git a/plugins/nvidia-skills/skills/cuopt-routing-api-python/skill-card.md b/plugins/nvidia-skills/skills/cuopt-routing-api-python/skill-card.md new file mode 100644 index 00000000..2e5e3a98 --- /dev/null +++ b/plugins/nvidia-skills/skills/cuopt-routing-api-python/skill-card.md @@ -0,0 +1,78 @@ +## Description:
+Vehicle routing (VRP, TSP, PDP) with cuOpt — Python API only. Use when the user is building or solving routing in Python.
+ +This skill is ready for commercial/non-commercial use.
+ +## Owner +NVIDIA
+ +### License/Terms of Use:
+Apache-2.0
+## Use Case:
+Developers and engineers building or solving vehicle routing problems (VRP, TSP, PDP) using the NVIDIA cuOpt Python API.
+ +### Deployment Geography for Use:
+Global
+ +## Known Risks and Mitigations:
+Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills.
+Mitigation: Review and scan skill before deployment.
+ +## Reference(s):
+- [Python API Examples (VRP, PDP, multi-depot)](references/examples.md)
+- [REST Server Examples](references/server_examples.md)
+- [cuOpt User Guide](https://docs.nvidia.com/cuopt/user-guide/latest/introduction.html)
+- [cuOpt Examples Repository](https://github.com/NVIDIA/cuopt-examples)
+ + +## Skill Output:
+**Output Type(s):** [Code, API Calls]
+**Output Format:** [Python code with cudf/cuOpt API calls]
+**Output Parameters:** [1D]
+**Other Properties Related to Output:** [None]
+ +## Evaluation Agents Used:
+- `claude-code`
+- `codex`
+ + + +## Evaluation Tasks:
+1 evaluation task (positive skill-activation), 2 attempts per task, pass threshold 50%.
+ +## Evaluation Metrics Used:
+Reported benchmark dimensions:
+- Security: Checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access.
+- Correctness: Checks whether the agent follows the expected workflow and produces the correct final output.
+- Discoverability: Checks whether the agent loads the skill when relevant and avoids using it when irrelevant.
+- Effectiveness: Checks whether the agent performs measurably better with the skill than without it.
+- Efficiency: Checks whether the agent uses fewer tokens and avoids redundant work.
+ +Underlying evaluation signals used in this run:
+- `security`: Checks for unsafe operations, secret leakage, and unauthorized access.
+- `skill_execution`: Verifies that the agent loaded the expected skill and workflow.
+- `skill_efficiency`: Checks routing quality, decoy avoidance, and redundant tool usage.
+- `accuracy`: Grades final-answer correctness against the reference answer.
+- `goal_accuracy`: Checks whether the overall user task completed successfully.
+- `behavior_check`: Verifies expected behavior steps, including safety expectations.
+- `token_efficiency`: Compares token usage with and without the skill.
+ + + +## Evaluation Results:
+| Dimension | Num | `claude-code` | `codex` | +|---|---:|---:|---:| +| Security | 2 | 100% (+0%) | 100% (+0%) | +| Correctness | 2 | 100% (+0%) | 95% (+3%) | +| Discoverability | 2 | 100% (+0%) | 70% (-5%) | +| Effectiveness | 2 | 83% (+14%) | 83% (+12%) | +| Efficiency | 2 | 93% (-0%) | 56% (-5%) | + +## Skill Version(s):
+26.08.00 (source: frontmatter)
+ +## Ethical Considerations:
+NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
+ +(For Release on NVIDIA Platforms Only)
+Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail).
diff --git a/plugins/nvidia-skills/skills/cuopt-routing-api-python/skill.oms.sig b/plugins/nvidia-skills/skills/cuopt-routing-api-python/skill.oms.sig new file mode 100644 index 00000000..70d7ec27 --- /dev/null +++ b/plugins/nvidia-skills/skills/cuopt-routing-api-python/skill.oms.sig @@ -0,0 +1 @@ +{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtcm91dGluZy1hcGktcHl0aG9uIiwKICAgICAgImRpZ2VzdCI6IHsKICAgICAgICAic2hhMjU2IjogIjFjNTJlZDRiNGI0NWMyOWQ5YmNlZDE2Yjc5MGQ3YmU3MjQ5MjM1NjQ2NzMwYTE4MjViNzIwZThmNTZkNDNjNzUiCiAgICAgIH0KICAgIH0KICBdLAogICJwcmVkaWNhdGVUeXBlIjogImh0dHBzOi8vbW9kZWxfc2lnbmluZy9zaWduYXR1cmUvdjEuMCIsCiAgInByZWRpY2F0ZSI6IHsKICAgICJzZXJpYWxpemF0aW9uIjogewogICAgICAibWV0aG9kIjogImZpbGVzIiwKICAgICAgImFsbG93X3N5bWxpbmtzIjogZmFsc2UsCiAgICAgICJpZ25vcmVfcGF0aHMiOiBbCiAgICAgICAgIi5naXRodWIiLAogICAgICAgICIuZ2l0aWdub3JlIiwKICAgICAgICAiLmdpdGF0dHJpYnV0ZXMiLAogICAgICAgICIuZ2l0IgogICAgICBdLAogICAgICAiaGFzaF90eXBlIjogInNoYTI1NiIKICAgIH0sCiAgICAicmVzb3VyY2VzIjogWwogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiMzRjZDY0YjIyYmQyMjEyNDc0MDZmZmFkMDhhZjFiYmNkYzE2NWRlODVmYTZkODM3NTVjMWY3OWViMjBjYjBkOSIsCiAgICAgICAgIm5hbWUiOiAiQkVOQ0hNQVJLLm1kIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiNDhjNDc2MzIwYThmN2YxM2VhNmQzOTA5YzRkODkwZjk5ZDk5MmYyMzJiNDZjMTAwNmM3MDE0YTdiZTI5MzIzMyIsCiAgICAgICAgIm5hbWUiOiAiU0tJTEwubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICJjNzAwMmEzMTIxOTgzZjMyOTRlZmJlOGM5NTQxOTQzYmYyNGM4OWEwN2JlYTZhMzIwMDdjNzc0YTJjODA4MDIxIiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvUkVBRE1FLm1kIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiZTBkMWExZmQ3ZDBhZDRlNDU0ZDA4ZjU1ZGU5MWJiZWRlNzhmZjEyMjJkNmE1NDJkNTVhYWFjZjcxYzVhN2U2MiIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL3BkcF9iYXNpYy9SRUFETUUubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICIxNDRkYTFkZjVkZTI4ZDc4NWE5YjQ2N2IzZDE0NDE3ZTcxNmY1MzJhYzliOTg5MDQ2ZWFmN2U0ZjUyOTlhNWZkIiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvcGRwX2Jhc2ljL21vZGVsLnB5IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiMGE4NWFlZjFjMWJlNTk5ODlkZTQwYWE2Y2U5ZmU1NGU3MjBlNzA5NWYwODZiYzZmNjg0ZjJiM2M5ZGEzMzg5NCIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL3ZycF9iYXNpYy9SRUFETUUubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICI1MjQ1Yjc3NDY1YTI2YjY4YWVmYmFhMzI0OWI1MWVmMGRhNDUwNWY0ZTE5NzRjZjZkMGY0NGIxYzc4ZmM4MDcwIiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvdnJwX2Jhc2ljL21vZGVsLnB5IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiZGJhMjAxMTE5ZTRmMGM1YjdkN2IxN2RmOWM3MTFkNDQ4M2UwMTg5MzM1MDhlMzYxZWU5MjllNmM0NGU1NmE0YiIsCiAgICAgICAgIm5hbWUiOiAiZXZhbHMvZXZhbHMuanNvbiIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAiZGlnZXN0IjogIjVlYjM1NzM1NTU5ZDkzNWMyMWUyMGE0OTU5MWQ1NGM5YjZmYTkyNDcyMTdhYTQ3NzZhNWY4OTgzY2JkMjdmODEiLAogICAgICAgICJuYW1lIjogInJlZmVyZW5jZXMvZXhhbXBsZXMubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICI1MDhiZjRhZThjYjViYzdlMjQ5YjM3NzI2MGYxNDIxYjcwZDlkMzQ1YmI1YTZkMTZjNmZhMGI1NmUyNTY4MjViIiwKICAgICAgICAibmFtZSI6ICJyZWZlcmVuY2VzL3NlcnZlcl9leGFtcGxlcy5tZCIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAiZGlnZXN0IjogIjcxOTRjN2FkMmIwMjY2MmU5MTFhNzFmOGIwN2Q2Nzk1NmNiNzdmMDMxZTU3NWEwZTcxZjVjMTZlYjk5ZTMzOWEiLAogICAgICAgICJuYW1lIjogInNraWxsLWNhcmQubWQiCiAgICAgIH0KICAgIF0KICB9Cn0=","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGUCMQC/if57e3mXxcr46MyoN7/Qlrwmk9leJtI83klm2/SuPZXkOfRclZp539nJbCqxcq4CMChXvVkzTj75l5w+zoaUK63MRHUujhIesZqb435AE2hAkKTOIrL4596BL+DxmL+QUA==","keyid":""}]}} \ No newline at end of file diff --git a/plugins/nvidia-skills/skills/cuopt-user-rules/BENCHMARK.md b/plugins/nvidia-skills/skills/cuopt-user-rules/BENCHMARK.md new file mode 100644 index 00000000..0f7d0852 --- /dev/null +++ b/plugins/nvidia-skills/skills/cuopt-user-rules/BENCHMARK.md @@ -0,0 +1,64 @@ +# Evaluation Report + +Evaluation of the `cuopt-user-rules` skill before publication through NVSkills-Eval. + +This benchmark summarizes 3-Tier Evaluation from NVSkills-Eval results for the skill. The goal is to document whether the skill is safe, discoverable, effective, and useful for agents before it is published for broader workflow use. + +## Evaluation Summary + +- Skill: `cuopt-user-rules` +- Evaluation date: 2026-05-29 +- NVSkills-Eval profile: `external` +- Overall verdict: PASS +- Tier 3 live agent evaluation: not available in this report + +## Agents Used + +- Tier 3 agent details were not available in this report. + +## Metrics Used + +Reported benchmark dimensions: + +- Security: checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access. +- Correctness: checks whether the agent follows the expected workflow and produces the correct final output. +- Discoverability: checks whether the agent loads the skill when relevant and avoids using it when irrelevant. +- Effectiveness: checks whether the agent performs measurably better with the skill than without it. +- Efficiency: checks whether the agent uses fewer tokens and avoids redundant work. + +Underlying evaluation signals used in this run: + +- No Tier 3 evaluation signal details were available in this report. + +## Test Tasks + +Tier 3 evaluation task details were not available in this report. + +## Results + +Tier 3 dimension rollup was not available in this report. + +## Tier 1: Static Validation Summary + +Tier 1 validation passed with observations. NVSkills-Eval ran 9 checks and found 8 total findings. + +Top findings: + +- MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (`skills/cuopt-user-rules/SKILL.md`) +- LOW QUALITY/quality_discoverability: No '## Purpose' section (`skills/cuopt-user-rules/SKILL.md`) +- LOW QUALITY/quality_reliability: No prerequisites/requirements documented (`skills/cuopt-user-rules/SKILL.md`) +- LOW QUALITY/quality_reliability: No limitations documented (`skills/cuopt-user-rules/SKILL.md`) +- LOW QUALITY/quality_reliability: No troubleshooting section documented (`skills/cuopt-user-rules/SKILL.md`) + +## Tier 2: Deduplication Summary + +Tier 2 validation passed. NVSkills-Eval ran 2 checks and found 0 total findings. + +Notable observations: + +- Context Deduplication: Collected 1 file(s) +- Inter-Skill Deduplication: Parsed skill 'cuopt-user-rules': 139 char description + +## Publication Recommendation + +The skill is suitable to proceed toward NVSkills-Eval publication based on this benchmark. Skill owners should keep this file with the skill and refresh it when the evaluation dataset, skill behavior, or target agents materially change. diff --git a/plugins/nvidia-skills/skills/cuopt-user-rules/SKILL.md b/plugins/nvidia-skills/skills/cuopt-user-rules/SKILL.md new file mode 100644 index 00000000..94494710 --- /dev/null +++ b/plugins/nvidia-skills/skills/cuopt-user-rules/SKILL.md @@ -0,0 +1,230 @@ +--- +name: cuopt-user-rules +version: "26.08.00" +description: Base rules for end users calling NVIDIA cuOpt (routing/LP/MILP/QP/install/server). Not for cuOpt internals — use cuopt-developer for those. +license: Apache-2.0 +metadata: + author: NVIDIA cuOpt Team + tags: + - cuopt + - user-rules + - guidelines +--- + + + +# cuOpt User Rules + +**Read this when helping someone *use* cuOpt** (calling the SDK, installing, deploying the server). For modifying cuOpt itself, switch to `cuopt-developer`. + +--- + +## Ask Before Assuming + +**Always clarify ambiguous requirements before implementing:** + +- What **language/interface**? +- What problem type? +- What constraints matter? +- What output format? + +**Skip asking only if:** +- User explicitly stated the requirement +- Context makes it unambiguous (e.g., user shows Python code) + +--- + +## Handle Incomplete Questions + +**If a question seems partial or incomplete, ask follow-up questions:** + +- "Could you tell me more about [missing detail]?" +- "What specifically would you like to achieve with this?" +- "Are there any constraints or requirements I should know about?" + +**Common missing information to probe for:** +- Problem size (number of vehicles, locations, variables, constraints) +- Specific constraints (time windows, capacities, precedence) +- Performance requirements (time limits, solution quality) +- Integration context (existing codebase, deployment environment) + +**Don't guess — ask.** A brief clarifying question saves time vs. solving the wrong problem. + +--- + +## Clarify Data Requirements + +**Before generating examples, ask about data:** + +1. **Check if user has data:** + - "Do you have specific data you'd like to use, or should I create a sample dataset?" + - "Can you share the format of your input data?" + +2. **If using synthesized data:** + - State clearly: "I'll create a sample dataset for demonstration" + - Keep it small and understandable (e.g., 5-10 locations, 2-3 vehicles) + - Make values realistic and meaningful + +3. **Always document what you used:** + ``` + "For this example I'm using: + - [X] locations/variables/constraints + - [Key assumptions: e.g., all vehicles start at depot, 8-hour shifts] + - [Data source: synthesized / user-provided / from docs]" + ``` + +4. **State assumptions explicitly:** + - "I'm assuming [X] — let me know if this differs from your scenario" + - List any default values or simplifications made + +--- + +## MUST Verify Understanding + +**Before writing substantial code, you MUST confirm your understanding:** + +``` +"Let me confirm I understand: +- Problem: [restate in your words] +- Constraints: [list them] +- Objective: [minimize/maximize what] +- Interface: [Python/REST/C/CLI] +Is this correct?" +``` + +--- + +## Follow Requirements Exactly + +- Use the **exact** variable names, formats, and structures the user specifies +- Don't add features the user didn't ask for +- Don't change the problem formulation unless asked +- If user provides partial code, extend it—don't rewrite from scratch + +--- + +## Check Results + +After providing a solution, guide the user to verify: + +- **Status check**: Is it `Optimal` / `FeasibleFound` / `SUCCESS`? +- **Constraint satisfaction**: Are all constraints met? +- **Objective value**: Is it reasonable for the problem? + +**Always end with a Result summary** that includes at least: +- Solver status (e.g. Optimal, FeasibleFound, SUCCESS). +- **Objective value with highlight** — easy to spot (bold or code block). Example: **Objective value (min total cost):** <value> or `Objective value: `. +- Briefly what the objective represents (e.g. total cost, total profit). + +Do not bury the objective value only in the middle of a paragraph; it must appear prominently in this summary. Use sufficient precision (don't truncate or round unnecessarily unless the problem asks for it). + +**Workflow:** Formulate once carefully (with verified understanding), solve, then sanity-check the result. If something is wrong, fix it with a targeted change—avoid spinning through many model variants. Decide, implement, verify, then move on. + +Provide diagnostic code snippets when helpful. + +### Post-correction check (mandatory) +If the result required a correction, retry, or workaround to reach this point, you MUST evaluate the skill-evolution workflow (`skills/skill-evolution/SKILL.md`) before moving on. Do not skip this step. + +--- + +## Check Environment First + +**Before writing code or suggesting installation, verify the user's setup:** + +1. **Ask how they access cuOpt:** + - "Do you have cuOpt installed? If so, which interface?" + - "What environment are you using? (local GPU, cloud, Docker, server, etc.)" + +2. **Different packages by language/interface:** + + | Language / Interface | Package | Check | + |----------------------|---------|-------| + | **Python** | `cuopt` (pip/conda) — also pulls in `libcuopt` | `import cuopt` | + | **C** | `libcuopt` (pip/conda) — already present if `cuopt` is installed | `find libcuopt.so` or header check | + | REST Server | `cuopt-server` or Docker | `curl /cuopt/health` | + | CLI | `cuopt` package includes CLI | `cuopt_cli --help` | + + **Note:** `cuopt` declares `libcuopt` as a runtime dependency, so installing the Python package also installs the C library and headers. Installing `libcuopt` on its own does **not** install the Python API. + +3. **If not installed, ask how they want to access:** + - "Would you like help installing cuOpt, or do you have access another way?" + - Options: pip, conda, Docker, cloud instance, existing remote server + +4. **Never assume installation is needed** — the user may: + - Already have it installed + - Be connecting to a remote server + - Prefer a specific installation method + - Only need the C library (not Python) + +5. **Ask before running any verification commands:** + ```python + # Python API check - ask first + import cuopt + print(cuopt.__version__) + ``` + ```bash + # C API check - ask first + find ${CONDA_PREFIX} -name "libcuopt.so" + ``` + ```bash + # Server check - ask first + curl http://localhost:8000/cuopt/health + ``` + +--- + +## Ask Before Running + +**Do not execute commands or code without explicit permission:** + +| Action | Rule | +|--------|------| +| Shell commands | Show command, explain what it does, ask "Should I run this?" | +| Package installs | **Never** run installs yourself — give the exact command, user runs it (see below). | +| Examples/scripts | Show the code first, ask "Would you like me to run this?" | +| File writes | Explain what will change, ask before writing | + +**Exceptions (okay without asking):** +- Read-only commands the user explicitly requested +- Commands the user just provided and asked you to run + +--- + +## No Privileged Operations + +**Never do these without explicit user request AND confirmation:** + +- Use `sudo` or run as root +- Modify system files or configurations +- Add package repositories or keys +- Change firewall, network, or driver settings +- Write files outside the workspace + +--- + +## Never Install Packages Automatically + +> **🔒 MANDATORY — You MUST NOT install, upgrade, or modify packages.** Provide the exact command; the user runs it. No exceptions. + +| Forbidden | What to do instead | +|-----------|--------------------| +| `pip install ...`, `conda install ...`, `apt install ...`, any package manager | Give the exact command and ask the user to run it. Say why the package is needed. | + +**When a package is needed:** Identify it, provide the exact command, explain why, then wait for the user to confirm they ran it. Even if the user says "just install it", give the command and require them to execute it themselves. + +--- + +## Resources + +### Documentation +- [cuOpt User Guide](https://docs.nvidia.com/cuopt/user-guide/latest/introduction.html) +- [API Reference](https://docs.nvidia.com/cuopt/user-guide/latest/api.html) + +### Examples +- [cuopt-examples repo](https://github.com/NVIDIA/cuopt-examples) +- [Google Colab notebooks](https://colab.research.google.com/github/nvidia/cuopt-examples/) + +### Support +- [File a Bug](https://github.com/NVIDIA/cuopt/issues/new?template=bug_report.md) +- [Ask a Question](https://github.com/NVIDIA/cuopt/issues/new?template=submit-question.md) +- [All Issues](https://github.com/NVIDIA/cuopt/issues) diff --git a/plugins/nvidia-skills/skills/cuopt-user-rules/evals/evals.json b/plugins/nvidia-skills/skills/cuopt-user-rules/evals/evals.json new file mode 100644 index 00000000..e20e0fe0 --- /dev/null +++ b/plugins/nvidia-skills/skills/cuopt-user-rules/evals/evals.json @@ -0,0 +1,19 @@ +[ + { + "id": "user-rules-eval-001-clarify-before-code", + "question": "Help me optimize my routing.", + "expected_skill": "cuopt-user-rules", + "expected_script": null, + "ground_truth": "The prompt is incomplete on every dimension. Per the user-rules skill, the agent must ask before assuming. It asks: (a) Language / interface — Python, C, or REST server? (b) Problem type — TSP, VRP, or PDP? (c) Data — does the user have a cost / distance matrix, order locations, fleet definition, or should the agent generate a small sample dataset for demonstration? (d) Constraints — time windows, vehicle capacities, precedence, service times? (e) Problem size — number of locations, vehicles, orders? (f) Performance — time limit, solution-quality target? It does not produce code, does not silently choose Python+VRP and emit a starter script, and does not invent constraint values. If the user later says 'just create a sample dataset', the agent will state clearly what it synthesized (size, depot assumption, time windows used) before producing code.", + "expected_behavior": [ + "Does not produce code on the underspecified prompt", + "Asks about language / interface (Python / C / REST)", + "Asks about problem type (TSP / VRP / PDP)", + "Asks whether the user has data or wants a synthesized sample", + "Asks about constraints (time windows, capacities, precedence, service times)", + "Asks about problem size and performance requirements", + "Does not silently assume Python+VRP defaults and produce a starter script", + "References the user-rules 'ask before assuming' rule" + ] + } +] diff --git a/plugins/nvidia-skills/skills/cuopt-user-rules/skill-card.md b/plugins/nvidia-skills/skills/cuopt-user-rules/skill-card.md new file mode 100644 index 00000000..6c0d4add --- /dev/null +++ b/plugins/nvidia-skills/skills/cuopt-user-rules/skill-card.md @@ -0,0 +1,53 @@ +## Description:
+Base rules for end users calling NVIDIA cuOpt (routing/LP/MILP/QP/install/server). Not for cuOpt internals — use cuopt-developer for those.
+ +This skill is ready for commercial/non-commercial use.
+ +## Owner +NVIDIA
+ +### License/Terms of Use:
+Apache 2.0
+## Use Case:
+Developers and engineers using NVIDIA cuOpt for vehicle routing (VRP/TSP/PDP), linear programming, mixed-integer programming, and quadratic programming tasks across Python, C, CLI, and server interfaces.
+ +### Deployment Geography for Use:
+Global
+ +## Known Risks and Mitigations:
+Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills.
+Mitigation: Review and scan skill before deployment.
+ +## Reference(s):
+- [cuOpt User Guide](https://docs.nvidia.com/cuopt/user-guide/latest/introduction.html)
+- [cuOpt API Reference](https://docs.nvidia.com/cuopt/user-guide/latest/api.html)
+- [cuopt-examples Repository](https://github.com/NVIDIA/cuopt-examples)
+ + +## Skill Output:
+**Output Type(s):** [Configuration instructions, Code, Analysis]
+**Output Format:** [Markdown with inline code blocks]
+**Output Parameters:** [1D]
+**Other Properties Related to Output:** [None]
+ +## Evaluation Tasks:
+Evaluated against 1 internal skill eval case (NVSkills-Eval, profile: external). Overall verdict: PASS.
+ +## Evaluation Metrics Used:
+Reported benchmark dimensions:
+- Security: Checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access.
+- Correctness: Checks whether the agent follows the expected workflow and produces the correct final output.
+- Discoverability: Checks whether the agent loads the skill when relevant and avoids using it when irrelevant.
+- Effectiveness: Checks whether the agent performs measurably better with the skill than without it.
+- Efficiency: Checks whether the agent uses fewer tokens and avoids redundant work.
+ + + +## Skill Version(s):
+26.08.00 (source: frontmatter)
+ +## Ethical Considerations:
+NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
+ +(For Release on NVIDIA Platforms Only)
+Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail).
diff --git a/plugins/nvidia-skills/skills/cuopt-user-rules/skill.oms.sig b/plugins/nvidia-skills/skills/cuopt-user-rules/skill.oms.sig new file mode 100644 index 00000000..50221979 --- /dev/null +++ b/plugins/nvidia-skills/skills/cuopt-user-rules/skill.oms.sig @@ -0,0 +1 @@ +{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtdXNlci1ydWxlcyIsCiAgICAgICJkaWdlc3QiOiB7CiAgICAgICAgInNoYTI1NiI6ICIzZmUzMTU0MjI4OTU3ZjFmZmRiM2NlMTAzODVkMTYxYzhlNmNhYzllODljN2U2NzhlYzY3NTdjNGY5ZWQ1ZWM1IgogICAgICB9CiAgICB9CiAgXSwKICAicHJlZGljYXRlVHlwZSI6ICJodHRwczovL21vZGVsX3NpZ25pbmcvc2lnbmF0dXJlL3YxLjAiLAogICJwcmVkaWNhdGUiOiB7CiAgICAic2VyaWFsaXphdGlvbiI6IHsKICAgICAgImhhc2hfdHlwZSI6ICJzaGEyNTYiLAogICAgICAibWV0aG9kIjogImZpbGVzIiwKICAgICAgImlnbm9yZV9wYXRocyI6IFsKICAgICAgICAiLmdpdCIsCiAgICAgICAgIi5naXRhdHRyaWJ1dGVzIiwKICAgICAgICAiLmdpdGlnbm9yZSIsCiAgICAgICAgIi5naXRodWIiCiAgICAgIF0sCiAgICAgICJhbGxvd19zeW1saW5rcyI6IGZhbHNlCiAgICB9LAogICAgInJlc291cmNlcyI6IFsKICAgICAgewogICAgICAgICJuYW1lIjogIkJFTkNITUFSSy5tZCIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiYmE2MWVhNDYxZGE3MmQ1Yzc0ZGM1MTZjYTZiOGM5NGZhMmZjOGZhYjJlZDM5N2I2Yzk4YTdlYzcwYzhkZjI3NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJuYW1lIjogIlNLSUxMLm1kIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICJmNjQyYjRlODk1M2MwMmMzMWU5OWViZTAyODljZDRmZjlmYmI2NGFlZGUxZjI1YzgwZWM0NzMwZTgyNzQwNTk2IgogICAgICB9LAogICAgICB7CiAgICAgICAgIm5hbWUiOiAiZXZhbHMvZXZhbHMuanNvbiIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiZjI4MWVmNjEwN2I4N2M1MmVlMmFlNGMzZjZkYWUwYTIxYjI3MWExMTRjNjk1Zjc3ZTY2N2M1YjUyMTJlOWMxMSIKICAgICAgfSwKICAgICAgewogICAgICAgICJuYW1lIjogInNraWxsLWNhcmQubWQiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAiZGlnZXN0IjogIjlmNTg3M2I4MDc2NjdmZjU3N2IwZTcwY2I1MWM3NmRhNzdmMmFiZjQ5ZTg1MGI0YmIxYzMyZGY3NTVjNGMwNDEiCiAgICAgIH0KICAgIF0KICB9Cn0=","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGYCMQDcCWcU/JKT7ctopkF8B+wwev0kKsKXB8UjexDwYsO+y3kkqe032WdZCLzgWv30EJUCMQC0R+pvH6cgkwlEpKRjP8RjQxcehew/LWCSuZRcMsJKnMUNJOIHtPP5qyoHkA3Ma58=","keyid":""}]}} \ No newline at end of file