Enable request batching in Ray Deployments

[pthombre]

Pytriton provides built in request batching support. This needs to be manually implemented for Ray serve based deployments.

[marta-sd]

This feature would be useful for NeMo Evaluator - we noticed that evaluation with pytriton serving backend is significantly faster than with ray.

[athitten]

Also adding on, starting with Mbridge the plan was to use Ray as the only deployment path and deprecate Pytriton since its inefficient to maintain both paths. Since Ray is missing batching, we are not able to deprecate Pytriton path yet. Supporting batching with Ray is valuable for large models (mGPU, multi node) including nemotron where having multiple replicas might not be feasible due to compute limitation.

[oyilmaz-nvidia]

@pthombre Just added this. Closing. Thanks.