feat: Implement resource requirements for components by Copilot · Pull Request #194 · plugboard-dev/plugboard

Copilot · 2026-01-28T19:44:16Z

Summary

Adds resource specification support for components to enable Ray actor resource allocation and Tuner placement group optimization.

Changes

Core Schema

Added Resource Pydantic class in plugboard_schemas.component with fields: cpu, gpu, memory (integer bytes), resources (custom dict)
Supports full SI prefix notation:
- Decimal SI: n, u, m, k, M, G, T, P, E (e.g., "250m" → 0.25, "5k" → 5000)
- Binary: Ki, Mi, Gi, Ti, Pi, Ei (e.g., "10Mi" → 10,485,760 bytes)
Default: cpu=0.001, others zero
Implements to_ray_options() for Ray actor configuration
Memory stored as integer bytes for precision

Component Integration

Extended Component.__init__() with optional resources parameter
Updated ComponentArgsDict, ComponentArgsSpec for YAML support
Resources auto-exported and reconstructed from configuration

Ray Execution

Modified RayProcess._create_component_actor() to apply resource requirements via ray.remote(**ray_options)
Falls back to default Resource() when unspecified
Import statements organized at top of file

Tuner Optimization

Added Tuner._calculate_placement_bundles() to aggregate component resources
Creates PlacementGroupFactory with tuner overhead (0.5 CPU) + component bundle
Import statements organized at top of file

Documentation

Added section in docs/examples/tutorials/running-in-parallel.md referencing resource requirements example
Python and YAML examples demonstrating resource specification

Usage Example

from plugboard.schemas import Resource

# Python API
component = MyComponent(
    name="worker",
    resources=Resource(cpu="2.5k", gpu=1, memory="512Mi")
)

# YAML config
resources:
  cpu: "500m"
  gpu: 1
  memory: "100Mi"
  resources:
    custom_accelerator: 2

Tests

Unit tests: Resource parsing with all SI prefixes, validation, Ray conversion
Integration tests: Component/ProcessSpec/Tuner scenarios
Manual verification (full test suite requires Ray dependencies)

Original prompt

This section details on the original issue you should resolve

<issue_title>feat: Implement resource requirements for components</issue_title>
<issue_description>### Summary

Users would like to be able to specify custom resource requirements on Component objects. These can be passed to Ray when a RayProcess is used or a Tuner object is run.

Requirements:

Must be able to specify Resource requirements for cpu, gpu, memory, resources. These should be defined as a Pydantic class in plugboard_schemas.

Must be able to specify resources as a numerical value, or as a string (to be validated by Pydantic), e.g. 250m for 0.25 or 10Mi for 1024 * 1024 * 10. resources should be a dictionary of string key, numerical value pairs.

Must be able to convert these requirements to their Ray equivalents, see https://docs.ray.io/en/latest/ray-core/api/doc/ray.actor.ActorClass.options.html, i.e. cpu = "250m" gets converted to num_cpus=0.25.

Must be able to pass a Resource to a Component when instantiating it. This will require changes to the Component and its schema, so that we can specify resource requirements in YAML config.

The default requirement should be {"cpu": 0.001}, to be used if the user does not provide anything.

Resource requirements should be passed to the actor options in RayProcess.

They should be used in the resource placement group inside Tuner.

Example

No response

Alternatives

No response</issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes feat: Implement resource requirements for components #193

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Co-authored-by: toby-coleman <13170610+toby-coleman@users.noreply.github.com>

plugboard-schemas/plugboard_schemas/component.py

plugboard/process/ray_process.py

examples/tutorials/004_using_ray/resources-example.yaml

plugboard/tune/tune.py

toby-coleman

@copilot see comments

…r, move imports to top Co-authored-by: toby-coleman <13170610+toby-coleman@users.noreply.github.com>

…ource-requirements

github-actions · 2026-02-07T14:47:51Z

Benchmark comparison for 140dccf7 (base) vs c14cbaff (PR)


------------------------------------------------------------------------------------------------------------------ benchmark: 2 tests -----------------------------------------------------------------------------------------------------------------
Name (time in ms)                                                                         Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_process_run (main/.benchmarks/Linux-CPython-3.12-64bit/0001_base)     442.5028 (1.0)      455.9622 (1.01)     448.2996 (1.00)     4.9525 (1.26)     447.8442 (1.00)     5.6369 (1.0)           2;0  2.2307 (1.00)          5           1
test_benchmark_process_run (pr/.benchmarks/Linux-CPython-3.12-64bit/0001_pr)         443.6396 (1.00)     452.6386 (1.0)      448.0607 (1.0)      3.9301 (1.0)      447.0895 (1.0)      6.9938 (1.24)          2;0  2.2318 (1.0)           5           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

github-actions · 2026-02-07T14:53:46Z

Benchmark comparison for 140dccf7 (base) vs 2588544c (PR)


------------------------------------------------------------------------------------------------------------------ benchmark: 2 tests -----------------------------------------------------------------------------------------------------------------
Name (time in ms)                                                                         Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_process_run (pr/.benchmarks/Linux-CPython-3.12-64bit/0001_pr)         439.1746 (1.0)      452.4331 (1.0)      446.3794 (1.0)      5.2427 (1.07)     447.1423 (1.0)      8.1439 (1.91)          2;0  2.2402 (1.0)           5           1
test_benchmark_process_run (main/.benchmarks/Linux-CPython-3.12-64bit/0001_base)     447.5662 (1.02)     459.9388 (1.02)     451.3804 (1.01)     4.9127 (1.0)      449.7072 (1.01)     4.2691 (1.0)           1;1  2.2154 (0.99)          5           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

github-actions · 2026-02-07T18:32:47Z

Benchmark comparison for 140dccf7 (base) vs 3301419f (PR)


------------------------------------------------------------------------------------------------------------------ benchmark: 2 tests ------------------------------------------------------------------------------------------------------------------
Name (time in ms)                                                                         Min                 Max                Mean            StdDev              Median                IQR            Outliers     OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_process_run (main/.benchmarks/Linux-CPython-3.12-64bit/0001_base)     448.3365 (1.0)      465.6030 (1.01)     457.8813 (1.0)      6.9427 (4.07)     457.8773 (1.0)      11.0587 (4.78)          2;0  2.1840 (1.0)           5           1
test_benchmark_process_run (pr/.benchmarks/Linux-CPython-3.12-64bit/0001_pr)         457.2382 (1.02)     461.4251 (1.0)      458.7039 (1.00)     1.7056 (1.0)      457.9881 (1.00)      2.3129 (1.0)           1;0  2.1801 (1.00)          5           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

github-actions · 2026-02-07T18:35:48Z

Benchmark comparison for 140dccf7 (base) vs 9b361df5 (PR)


------------------------------------------------------------------------------------------------------------------ benchmark: 2 tests -----------------------------------------------------------------------------------------------------------------
Name (time in ms)                                                                         Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_process_run (main/.benchmarks/Linux-CPython-3.12-64bit/0001_base)     445.6866 (1.0)      452.1979 (1.0)      449.3371 (1.0)      2.5165 (1.0)      449.3286 (1.0)      3.6126 (1.0)           2;0  2.2255 (1.0)           5           1
test_benchmark_process_run (pr/.benchmarks/Linux-CPython-3.12-64bit/0001_pr)         446.5589 (1.00)     457.6603 (1.01)     452.1729 (1.01)     4.8984 (1.95)     453.9259 (1.01)     8.5801 (2.38)          2;0  2.2115 (0.99)          5           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

github-actions · 2026-02-07T18:44:44Z

Benchmark comparison for 140dccf7 (base) vs d7c87c6b (PR)


------------------------------------------------------------------------------------------------------------------ benchmark: 2 tests -----------------------------------------------------------------------------------------------------------------
Name (time in ms)                                                                         Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_process_run (main/.benchmarks/Linux-CPython-3.12-64bit/0001_base)     449.1386 (1.0)      457.5788 (1.00)     453.7119 (1.0)      3.2048 (1.10)     453.2836 (1.0)      4.3319 (1.0)           2;0  2.2040 (1.0)           5           1
test_benchmark_process_run (pr/.benchmarks/Linux-CPython-3.12-64bit/0001_pr)         450.5393 (1.00)     457.4148 (1.0)      453.9127 (1.00)     2.9121 (1.0)      454.0299 (1.00)     5.1117 (1.18)          2;0  2.2031 (1.00)          5           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

github-actions · 2026-02-07T19:48:40Z

Benchmark comparison for 140dccf7 (base) vs a3c9555e (PR)


------------------------------------------------------------------------------------------------------------------ benchmark: 2 tests -----------------------------------------------------------------------------------------------------------------
Name (time in ms)                                                                         Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_process_run (pr/.benchmarks/Linux-CPython-3.12-64bit/0001_pr)         443.6576 (1.0)      454.0091 (1.0)      447.3220 (1.0)      4.2598 (1.0)      446.6230 (1.0)      6.1454 (1.0)           1;0  2.2355 (1.0)           5           1
test_benchmark_process_run (main/.benchmarks/Linux-CPython-3.12-64bit/0001_base)     444.8959 (1.00)     461.4176 (1.02)     450.3497 (1.01)     6.6617 (1.56)     448.6970 (1.00)     8.1958 (1.33)          1;0  2.2205 (0.99)          5           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

github-actions · 2026-02-07T22:16:13Z

Benchmark comparison for 140dccf7 (base) vs c9aa1cea (PR)


------------------------------------------------------------------------------------------------------------------ benchmark: 2 tests -----------------------------------------------------------------------------------------------------------------
Name (time in ms)                                                                         Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_process_run (pr/.benchmarks/Linux-CPython-3.12-64bit/0001_pr)         359.9217 (1.0)      366.7047 (1.0)      363.5997 (1.0)      2.8372 (1.0)      362.6679 (1.0)      4.5080 (1.0)           2;0  2.7503 (1.0)           5           1
test_benchmark_process_run (main/.benchmarks/Linux-CPython-3.12-64bit/0001_base)     364.2002 (1.01)     372.2619 (1.02)     368.2314 (1.01)     3.6488 (1.29)     367.3628 (1.01)     6.7145 (1.49)          2;0  2.7157 (0.99)          5           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

codecov · 2026-02-07T22:22:20Z

Codecov Report

❌ Patch coverage is 85.00000% with 6 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
plugboard/tune/tune.py	84.61%	1 Missing and 3 partials ⚠️
plugboard/component/component.py	66.66%	1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

Initial plan

4530341

Copilot AI assigned Copilot and toby-coleman Jan 28, 2026

Copilot started work on behalf of toby-coleman January 28, 2026 19:44 View session

Copilot AI and others added 4 commits January 28, 2026 19:49

Add Resource class and integrate with Component, RayProcess, and Tuner

d529dee

Co-authored-by: toby-coleman <13170610+toby-coleman@users.noreply.github.com>

Add comprehensive tests for Resource implementation

f59376c

Co-authored-by: toby-coleman <13170610+toby-coleman@users.noreply.github.com>

Add examples for resource requirements in Ray processes

0219833

Co-authored-by: toby-coleman <13170610+toby-coleman@users.noreply.github.com>

Fix type annotations and address code review feedback

3ec7854

Co-authored-by: toby-coleman <13170610+toby-coleman@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add resource requirements for components~~ feat: Implement resource requirements for components Jan 28, 2026

Copilot finished work on behalf of toby-coleman January 28, 2026 20:01

Copilot AI requested a review from toby-coleman January 28, 2026 20:01

toby-coleman reviewed Jan 28, 2026

View reviewed changes

plugboard-schemas/plugboard_schemas/component.py Show resolved Hide resolved

toby-coleman reviewed Jan 28, 2026

View reviewed changes

plugboard-schemas/plugboard_schemas/component.py Outdated Show resolved Hide resolved

toby-coleman reviewed Jan 28, 2026

View reviewed changes

plugboard-schemas/plugboard_schemas/component.py Outdated Show resolved Hide resolved

toby-coleman reviewed Jan 28, 2026

View reviewed changes

plugboard/process/ray_process.py Outdated Show resolved Hide resolved

toby-coleman reviewed Jan 31, 2026

View reviewed changes

examples/tutorials/004_using_ray/resources-example.yaml Show resolved Hide resolved

toby-coleman reviewed Jan 31, 2026

View reviewed changes

plugboard/tune/tune.py Outdated Show resolved Hide resolved

toby-coleman requested changes Jan 31, 2026

View reviewed changes

Copilot started work on behalf of toby-coleman January 31, 2026 09:34 View session

Address code review feedback: generalize suffixes, make memory intege…

9515154

…r, move imports to top Co-authored-by: toby-coleman <13170610+toby-coleman@users.noreply.github.com>

Copilot AI requested a review from toby-coleman January 31, 2026 09:38

Copilot finished work on behalf of toby-coleman January 31, 2026 09:38

toby-coleman added 8 commits February 3, 2026 21:26

Move suffixes to top

c400a97

Rework logic for tune placement bundles

a38aa88

Merge remote-tracking branch 'origin/main' into copilot/implement-res…

573a0e1

…ource-requirements

Docs config improved for schemas

31092ab

Tidy up documentation on resources example

6bf7017

Add resources for smoke tests

bae7c46

Make sure resources are available in runner

a9a4ef5

Remove non-existent cross refs in docs

c14cbaf

Fixup types

2588544

toby-coleman added 2 commits February 7, 2026 18:30

Test that actor is passed the resource requirements

d2e7aa2

Increase timeouts on test CI

3301419

toby-coleman marked this pull request as ready for review February 7, 2026 18:33

toby-coleman approved these changes Feb 7, 2026

View reviewed changes

Typing

9b361df

Fixup test

d7c87c6

Increase tuner test timeout

a3c9555

Correct format for placement groups in Ray tune

c9aa1ce

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Implement resource requirements for components#194

feat: Implement resource requirements for components#194
Copilot wants to merge 21 commits intomainfrom
copilot/implement-resource-requirements

Copilot AI commented Jan 28, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

toby-coleman left a comment

Uh oh!

github-actions bot commented Feb 7, 2026

Uh oh!

github-actions bot commented Feb 7, 2026

Uh oh!

github-actions bot commented Feb 7, 2026

Uh oh!

github-actions bot commented Feb 7, 2026

Uh oh!

github-actions bot commented Feb 7, 2026

Uh oh!

github-actions bot commented Feb 7, 2026

Uh oh!

github-actions bot commented Feb 7, 2026

Uh oh!

codecov bot commented Feb 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Copilot AI commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Example

Alternatives

Comments on the Issue (you are @copilot in this section)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

toby-coleman left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 7, 2026

Uh oh!

github-actions bot commented Feb 7, 2026

Uh oh!

github-actions bot commented Feb 7, 2026

Uh oh!

github-actions bot commented Feb 7, 2026

Uh oh!

github-actions bot commented Feb 7, 2026

Uh oh!

github-actions bot commented Feb 7, 2026

Uh oh!

github-actions bot commented Feb 7, 2026

Uh oh!

codecov bot commented Feb 7, 2026

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Jan 28, 2026 •

edited

Loading