Add batch request support for generating API payloads

[marckohlbrugge]

Summary

• Adds for_batch_request method to generate request payloads without making API calls
• Currently supports OpenAI; other providers raise NotImplementedError
• Useful for batch processing, testing, and debugging API payloads

Motivation

As discussed, this implements step 1 of batch request support - generating the API payloads. This allows users to:

1. Generate multiple request payloads for batch processing
2. Test request structure without making actual API calls
3. Debug the exact payloads being sent to providers

The remaining steps (combining requests, submitting to batch endpoints, polling, processing results) can be implemented by users based on their specific needs.

Implementation Details

Core Changes

• Added for_batch_request method to Chat class that sets a flag to generate payloads instead of making API calls
• Added render_payload_for_batch_request to Provider base class (raises NotImplementedError by default)
• Implemented batch request formatting for OpenAI provider following their API spec
• All OpenAI-based providers explicitly override to raise NotImplementedError

Usage

# Enable batch request mode
chat = RubyLLM.chat.for_batch_request
chat.ask("What's 2 + 2?")
payload = chat.complete  # Returns payload instead of API call

# Works with all features
chat = RubyLLM.chat
  .with_model('gpt-4')
  .with_tool(MyTool)
  .for_batch_request

Provider Support

• OpenAI: ✅ Fully implemented with proper batch API format
• All Others: Raise NotImplementedError with clear message

Testing

• Added comprehensive test coverage in spec/ruby_llm/chat_batch_request_spec.rb
• All tests passing (54 examples, 0 failures)
• Tests verify OpenAI generates proper payloads, others raise errors

Documentation

• Added docs/batch_requests.md with usage examples and notes
• Clearly documents current limitations and future enhancement possibilities

🤖 Generated with Claude Code

[marckohlbrugge]

I named it for_batch_request (versus e.g. as_batch_request), because technically it doesn't make the batch request. It just formats the request as JSON that's useable for a batch request.

I also explored a more flexible .with_output_format(:batch_request) but decided against it as it seemed premature and I couldn't think of any other use cases.

We could also consider a more flexible .as_hash, .to_h, .without_api_call approach that just returns the payload as a hash. This would require some more work from the user to turn it into a "batch request"-compatible format, but might prove useful for more use cases.

[psdaniel]

Hey @marckohlbrugge, thanks for kicking off this feature! I’ve been evaluating RubyLLM, and the batch support is super helpful for my use case. Are you planning to keep working on it further?

[marckohlbrugge]

@psdaniel Yes, happy to finish it. Would like to get @crmne's input first on the design.

[jankeesvw]

Would love to see this feature!