Skip to content

Flexible rank assignments — Phase 1: infrastructure#725

Open
DLWoodruff wants to merge 5 commits into
Pyomo:mainfrom
DLWoodruff:flex-ranks-phase1-infrastructure
Open

Flexible rank assignments — Phase 1: infrastructure#725
DLWoodruff wants to merge 5 commits into
Pyomo:mainfrom
DLWoodruff:flex-ranks-phase1-infrastructure

Conversation

@DLWoodruff
Copy link
Copy Markdown
Collaborator

First implementation phase of the flexible-rank-assignments design
(doc/designs/flexible_rank_assignments.md, merged in #724). Lands the
static, unit-testable building blocks without changing any runtime
behavior at equal rank counts
— the live unequal-rank path is gated
off pending the Phase 2 communicator rework.

What's here

  • mpisppy/utils/rank_apportionment.pyapportion_ranks(ratios, n_proc): largest-remainder (Hare) apportionment of a rank pool across
    cylinders by target ratio, with a floor of one rank per cylinder.

  • mpisppy/cylinders/overlap_map.pyOverlapSegment +
    compute_overlap_segments(): given a local rank's scenarios, a peer
    cylinder's contiguous scenario partition (the slices from
    scen_names_to_ranks), and a per-scenario item count, produce the
    segments that back a local field buffer from the peer's buffers.
    Contiguous same-rank runs coalesce; equal rank counts degenerate to a
    single identity segment. Pure — no MPI/Pyomo.

  • SPWindow.get() — optional item_offset/item_count for
    sub-range reads via MPI_Get displacement. Defaults reproduce the
    original whole-padded-field transfer exactly, so existing callers are
    untouched. This is the primitive multi-source assembly will use.

  • WheelSpinner.run() — reads a per-cylinder rank_ratio dict key
    (default 1.0). All-1.0 is handled exactly as before. A non-uniform
    request is apportioned and reported (global_toc on rank 0), then
    raises NotImplementedError — building communicators for unequal
    per-cylinder counts is not yet wired into the comm/window layer.

  • Fixes two errors in the design doc's overlap worked-example (verified
    against compute_overlap_segments).

Testing

  • 33 new unit tests across test_rank_apportionment.py,
    test_overlap_map.py, test_spwindow_partial_get.py (real
    COMM_SELF RMA window), and test_flexible_rank_gate.py (stub comm).
    All wired into run_coverage.bash and test_pr_and_main.yml.
  • Default (equal-rank) path confirmed unchanged by running
    test_with_cylinders (PH hub + xhatshuffle spoke) end to end.

Not in this phase

Building the actual unequal-rank communicators/window (drop
strata_comm, multi-source get_receive_buffer consuming the overlap
maps) is the next phase; that's why non-uniform ratios raise for now.

🤖 Generated with Claude Code

DLWoodruff and others added 5 commits May 23, 2026 19:05
Add apportion_ranks(ratios, n_proc): largest-remainder (Hare quota)
apportionment of a fixed rank pool across cylinders by target ratio,
with a floor of one rank per cylinder. Counts sum to exactly n_proc and
every cylinder gets at least one rank. Raises when there are more
cylinders than ranks. Pure function; not yet wired into WheelSpinner.

Per doc/designs/flexible_rank_assignments.md, the User Interface
apportionment algorithm.

Tests cover the worked design-doc example, tie-breaking by declaration
order, scale invariance, the floor-of-one rescue, C==np and C>np, and a
property sweep (sum==n_proc, all >=1) across many cases. Wired into
run_coverage.bash and test_pr_and_main.yml.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add OverlapSegment and compute_overlap_segments(): given a local rank's
scenarios, a peer cylinder's contiguous scenario partition (the slices
from scen_names_to_ranks), and a per-scenario item count, produce the
list of segments that back the local field buffer from the peer's
buffers. Contiguous same-rank runs are coalesced; equal rank counts
degenerate to a single identity segment per rank.

Pure (no MPI/Pyomo), so it unit tests directly. Not yet wired into the
communicator. Per doc/designs/flexible_rank_assignments.md, Overlap Maps.

Tests: equal-rank identity (single + multi-item), fewer/more remote
ranks, straddling the peer split, run coalescing, variable per-scenario
item counts, and a tiling invariant. Wired into run_coverage.bash and
test_pr_and_main.yml.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two errors in the 4-hub/2-spoke example, both verified against
compute_overlap_segments:
- Hub rank 2 (scen4,scen5) straddles the spoke's scen5 split, so it
  reads two segments (scen4 from spoke rank 0, scen5 from spoke rank 1),
  not one segment of count 2 from spoke rank 0.
- Hub rank 3 (scen6-9) sits at offsets 1-4 within spoke rank 1's buffer
  (which begins at scen5), so remote_offset is 1, not 0.

Also notes the split is illustrative (chosen to show a straddle) and
states the one-nonant-per-scenario assumption.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add optional item_offset / item_count to SPWindow.get() so a reader can
fetch a sub-range of a field via MPI_Get displacement. Defaults
(item_count=None) reproduce the original whole-padded-field transfer
exactly, so existing callers are unaffected. Partial reads are the
primitive multi-source assembly will use under unequal rank counts
(see overlap_map.py).

Tests exercise a real single-process RMA window (MPI.COMM_SELF):
full read unchanged, prefix/middle/suffix/single-item partial reads,
full-via-count, and out-of-range / dest-size-mismatch assertions. Wired
into run_coverage.bash and test_pr_and_main.yml.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
WheelSpinner.run() now reads a per-cylinder "rank_ratio" key from each
hub/spoke dict (default 1.0). When all ratios are 1.0 the path is exactly
as before. When any differs, it apportions the rank pool
(largest-remainder, floor of one), reports the allocation via global_toc
on rank 0, then raises NotImplementedError -- building communicators for
unequal per-cylinder counts is not yet wired into the comm/window layer.

This lands the apportionment wiring and the dict-key plumbing while
keeping the uniform (default) behavior unchanged; verified by running
test_with_cylinders (PH hub + xhatshuffle spoke) end to end.

Tests use a stub comm to drive the gate without MPI: a non-uniform ratio
raises NotImplementedError; an infeasible floor-of-one raises ValueError
first; explicit all-1.0 ratios skip the gate. Wired into
run_coverage.bash and test_pr_and_main.yml.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@codecov
Copy link
Copy Markdown

codecov Bot commented May 24, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.69%. Comparing base (6662635) to head (109e571).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #725      +/-   ##
==========================================
+ Coverage   72.57%   72.69%   +0.11%     
==========================================
  Files         162      164       +2     
  Lines       20643    20715      +72     
==========================================
+ Hits        14982    15059      +77     
+ Misses       5661     5656       -5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant