feat(BA-4146): Device-based MANUAL mode config

[hhoikoo]

resolves #8428 (BA-4146)

Overview

Changes MANUAL mode resource allocation from slot-based format ({"cuda.mem": 0.5}) to device-based format (cuda = ["cuda0", "cuda1"]), enabling explicit device-to-agent mapping in multi-agent configurations.

Problem Statement

• MANUAL mode used slot-based decimal allocation which didn't align with device-centric resource model
• No explicit control over which physical devices are assigned to which agents
• BEP-1041 migration to device-centric resource allocation was incomplete without MANUAL mode support

Architecture

flowchart TB
    subgraph "Configuration (unified.py)"
        RC[ResourceAllocationConfig]
        RC -->|"devices: Mapping[DeviceName, Sequence[DeviceId]]"| DV[Device Validator]
        DV -->|"Detect old format"| ERR[Validation Error]
        DV -->|"Parse new format"| OK[Valid Config]
    end

    subgraph "Resource Allocation (resources.py)"
        RA[ResourceAllocator.__ainit__]
        RA -->|MANUAL mode| GMA[_generate_manual_assignments]
        RA -->|AUTO_SPLIT| GAS[_generate_auto_split_assignments]
        RA -->|SHARED| GSA[_generate_shared_assignments]
        GMA --> ADA[_apply_device_assignments]
        GAS --> ADA
        GSA --> ADA
    end

    OK --> GMA

Loading

Configuration Format Change

Old slot-based format (rejected):

[resource.allocations]
devices = {"cuda.mem" = 0.5, "cuda.shares" = 2}

New device-based format:

[resource.allocations]
devices = {cuda = ["cuda0", "cuda1"], rocm = ["rocm0"]}

---

Checklist: (if applicable)

• Milestone metadata specifying the target backport version
• Mention to the original issue
• Installer updates including:
  • Fixtures for db schema changes
  • New mandatory config options
• Update of end-to-end CLI integration tests in ai.backend.test
• API server-client counterparts (e.g., manager API -> client SDK)
• Test case(s) to:
  • Demonstrate the difference of before/after
  • Demonstrate the flow of abstract/conceptual models with a concrete implementation
• Documentation
  • Contents in the docs directory
  • docstrings in public interfaces and type annotations

[Copilot]

Pull request overview

Updates agent MANUAL-mode resource allocation to use explicit device ID lists (device-centric) and refactors resource discovery/allocation to operate on per-device assignments across agents.

Changes:

• Switch MANUAL mode allocations.devices from slot-based decimals (cuda.mem, cuda.shares) to device-based mappings (cuda = ["cuda0", ...]).
• Refactor ResourceAllocator to separate global device discovery from per-agent device assignment (AUTO_SPLIT / SHARED / MANUAL).
• Expand/update unit tests for config validation and resource allocation behaviors (including partitioning helpers like natural sort and device distribution).

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/ai/backend/agent/config/unified.py	Changes MANUAL allocations.devices schema to device-based mapping and adds format validation.
src/ai/backend/agent/resources.py	Introduces global device discovery structures and implements device assignment generation/application for allocation modes.
src/ai/backend/agent/types.py	Adds typing for device assignment mappings and a device partitioner protocol.
tests/unit/agent/test_resource_allocation.py	Updates/extends tests to reflect device-based allocation and new partitioning helpers.
tests/unit/agent/test_resources.py	Adds tests for new GlobalDeviceInfo and device discovery helper behavior.
tests/unit/agent/test_config_validation.py	Updates validation tests to reflect device-based MANUAL allocations and old-format rejection.
changes/8440.feature.md	Changelog entry for the device discovery infrastructure refactor.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

Comments suppressed due to low confidence (1)

tests/unit/agent/test_resource_allocation.py:759

• MANUAL mode behavior is now implemented in ResourceAllocator/ResourcePartitioner, but there are no ResourceAllocator MANUAL-mode integration tests in this file anymore (the previous class was removed). Add integration tests that assert per-agent CPU/mem reservations and device visibility in MANUAL mode, and cover failure cases like duplicate device IDs across agents.

        # Memory: shared device, both see same device with divided slots
        assert len(agent1_computers[DeviceName("mem")].alloc_map.device_slots) == 1
        assert len(agent2_computers[DeviceName("mem")].alloc_map.device_slots) == 1

        await allocator.__aexit__(None, None, None)

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.