Skip to content

Commit a5de20c

Browse files
authored
Merge pull request #305 from NYU-RTS/minor-cleanup
minor cleanup
2 parents 0a88e23 + ef8cf3d commit a5de20c

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

docs/genai/04_how_to_guides/04_batch_inference.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Batch Inference
22

3-
When immediate results are not needed, for instance in transforming large datasets of unstructured data with LLMs, batch inference adds convenience while offering lower costs. Typical completion window are 24/48h as LLM inference providers run your workload when the load on the inference server is low. If you are interested in harnessing this feature, reach out to us at [`genai-research-support@nyu.edu`](mailto:genai-research-support@nyu.edu) and we will set up a cloud storage bucket for you.
3+
When immediate results are not needed, for instance in transforming large datasets of unstructured data with LLMs, batch inference adds convenience while offering lower costs. The typical completion window is 24h as LLM inference providers run your workload when the load on the inference server is low. If you are interested in harnessing this feature, reach out to us at [`genai-research-support@nyu.edu`](mailto:genai-research-support@nyu.edu) and we will set up a cloud storage bucket for you.
44

55
:::info
66
Batch processing is only supported for LLMs that can be accessed via the `@vertexai` provider.

0 commit comments

Comments
 (0)