Context
pkg/agent/executor.go hardcodes the starting port for llama-server processes:
func NewMetalExecutor(...) *MetalExecutor {
return &MetalExecutor{
...
nextPort: 8080, // TODO: Implement proper port allocation
}
}
Problem
When multiple models are deployed simultaneously via the Metal agent, port conflicts can occur. The current implementation increments from 8080 but doesn't check for availability or reclaim ports from stopped processes.
Proposed Solution
- Check port availability before assigning (e.g.,
net.Listen probe)
- Track allocated ports per process and reclaim on stop
- Consider a configurable port range (e.g.,
--port-range 8080-8180)
- Handle the case where all ports in range are exhausted
Location
pkg/agent/executor.go:54
Context
pkg/agent/executor.gohardcodes the starting port for llama-server processes:Problem
When multiple models are deployed simultaneously via the Metal agent, port conflicts can occur. The current implementation increments from 8080 but doesn't check for availability or reclaim ports from stopped processes.
Proposed Solution
net.Listenprobe)--port-range 8080-8180)Location
pkg/agent/executor.go:54