Skip to content

Commit fdd9903

Browse files
authored
Restructure usage section of README (#120)
* Restructure usage section of README * Create separate Providers section and reorganize subheadings * Remove now outdated note about Ollama not supporting guided generation * Bump version to 0.7.0
1 parent a391cc7 commit fdd9903

File tree

1 file changed

+180
-90
lines changed

1 file changed

+180
-90
lines changed

README.md

Lines changed: 180 additions & 90 deletions
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,7 @@ Add this package to your `Package.swift`:
9898

9999
```swift
100100
dependencies: [
101-
.package(url: "https://github.com/mattt/AnyLanguageModel", from: "0.5.0")
101+
.package(url: "https://github.com/mattt/AnyLanguageModel", from: "0.7.0")
102102
]
103103
```
104104

@@ -126,7 +126,7 @@ To enable specific traits, specify them in your package's dependencies:
126126
dependencies: [
127127
.package(
128128
url: "https://github.com/mattt/AnyLanguageModel.git",
129-
from: "0.5.0",
129+
from: "0.7.0",
130130
traits: ["CoreML", "MLX"] // Enable CoreML and MLX support
131131
)
132132
]
@@ -287,26 +287,14 @@ see OWASP's
287287

288288
## Usage
289289

290-
### Apple Foundation Models
291-
292-
Uses Apple's [system language model](https://developer.apple.com/documentation/FoundationModels)
293-
(requires macOS 26 / iOS 26 / visionOS 26 or later).
294-
295-
```swift
296-
let model = SystemLanguageModel.default
297-
let session = LanguageModelSession(model: model)
290+
### Guided Generation
298291

299-
let response = try await session.respond {
300-
Prompt("Explain quantum computing in one sentence")
301-
}
302-
```
303-
304-
> [!NOTE]
305-
> Image inputs are not yet supported by Apple Foundation Models.
306-
307-
`SystemLanguageModel` supports guided generation,
292+
All on-device models — Apple Foundation Models, Core ML, MLX, and llama.cpp —
293+
support guided generation,
308294
letting you request strongly typed outputs using `@Generable` and `@Guide`
309295
instead of parsing raw strings.
296+
Cloud providers (OpenAI, Open Responses, Anthropic, and Gemini)
297+
also support guided generation.
310298
For more details, see
311299
[Generating Swift data structures with guided generation](https://developer.apple.com/documentation/foundationmodels/generating-swift-data-structures-with-guided-generation).
312300

@@ -323,17 +311,125 @@ struct CatProfile {
323311
var profile: String
324312
}
325313

326-
let session = LanguageModelSession(model: .default)
314+
let session = LanguageModelSession(model: model)
327315
let response = try await session.respond(
328316
to: "Generate a cute rescue cat",
329317
generating: CatProfile.self
330318
)
331319
print(response.content)
332320
```
333321

322+
### Image Inputs
323+
324+
Many providers support image inputs,
325+
letting you include images alongside text prompts.
326+
Pass images using the `images:` or `image:` parameter on `respond`:
327+
328+
```swift
329+
let response = try await session.respond(
330+
to: "Describe what you see",
331+
images: [
332+
.init(url: URL(string: "https://example.com/photo.jpg")!),
333+
.init(url: URL(fileURLWithPath: "/path/to/local.png"))
334+
]
335+
)
336+
```
337+
338+
Image support varies by provider:
339+
340+
| Provider | Image Inputs |
341+
| ----------------------- | :-------------: |
342+
| Apple Foundation Models ||
343+
| Core ML ||
344+
| MLX | model-dependent |
345+
| llama.cpp ||
346+
| Ollama | model-dependent |
347+
| OpenAI | yes |
348+
| Open Responses | yes |
349+
| Anthropic | yes |
350+
| Google Gemini | yes |
351+
352+
For MLX and Ollama,
353+
use a vision-capable model
354+
(for example, a VLM or `-vl` variant).
355+
356+
### Tool Calling
357+
358+
Tool calling is supported by all providers except llama.cpp.
359+
Define tools using the `Tool` protocol and pass them when creating a session:
360+
361+
```swift
362+
struct WeatherTool: Tool {
363+
let name = "getWeather"
364+
let description = "Retrieve the latest weather information for a city"
365+
366+
@Generable
367+
struct Arguments {
368+
@Guide(description: "The city to fetch the weather for")
369+
var city: String
370+
}
371+
372+
func call(arguments: Arguments) async throws -> String {
373+
"The weather in \(arguments.city) is sunny and 72°F / 23°C"
374+
}
375+
}
376+
377+
let session = LanguageModelSession(model: model, tools: [WeatherTool()])
378+
379+
let response = try await session.respond {
380+
Prompt("How's the weather in Cupertino?")
381+
}
382+
print(response.content)
383+
```
384+
385+
To observe or control tool execution, assign a delegate on the session:
386+
387+
```swift
388+
actor ToolExecutionObserver: ToolExecutionDelegate {
389+
func didGenerateToolCalls(_ toolCalls: [Transcript.ToolCall], in session: LanguageModelSession) async {
390+
print("Generated tool calls: \(toolCalls)")
391+
}
392+
393+
func toolCallDecision(
394+
for toolCall: Transcript.ToolCall,
395+
in session: LanguageModelSession
396+
) async -> ToolExecutionDecision {
397+
// Return .stop to halt after tool calls, or .provideOutput(...) to bypass execution.
398+
// This is a good place to ask the user for confirmation (for example, in a modal dialog).
399+
.execute
400+
}
401+
402+
func didExecuteToolCall(
403+
_ toolCall: Transcript.ToolCall,
404+
output: Transcript.ToolOutput,
405+
in session: LanguageModelSession
406+
) async {
407+
print("Executed tool call: \(toolCall)")
408+
}
409+
}
410+
411+
session.toolExecutionDelegate = ToolExecutionObserver()
412+
```
413+
414+
## Providers
415+
416+
### Apple Foundation Models
417+
418+
Uses Apple's [system language model](https://developer.apple.com/documentation/FoundationModels)
419+
(requires macOS 26 / iOS 26 / visionOS 26 or later).
420+
421+
```swift
422+
let model = SystemLanguageModel.default
423+
let session = LanguageModelSession(model: model)
424+
425+
let response = try await session.respond {
426+
Prompt("Explain quantum computing in one sentence")
427+
}
428+
```
429+
334430
### Core ML
335431

336-
Run [Core ML](https://developer.apple.com/documentation/coreml) models
432+
Runs [Core ML](https://developer.apple.com/documentation/coreml) models
337433
(requires `CoreML` trait):
338434

339435
```swift
@@ -355,12 +451,9 @@ Enable the trait in Package.swift:
355451
)
356452
```
357453

358-
> [!NOTE]
359-
> Image inputs are not currently supported with `CoreMLLanguageModel`.
360-
361454
### MLX
362455

363-
Run [MLX](https://github.com/ml-explore/mlx-swift) models on Apple Silicon
456+
Runs [MLX](https://github.com/ml-explore/mlx-swift) models on Apple Silicon
364457
(requires `MLX` trait):
365458

366459
```swift
@@ -398,12 +491,9 @@ Enable the trait in Package.swift:
398491
)
399492
```
400493

401-
> [!NOTE]
402-
> MLX supports guided generation (structured output via `@Generable`).
403-
404494
### llama.cpp (GGUF)
405495

406-
Run GGUF quantized models via [llama.cpp](https://github.com/ggml-org/llama.cpp)
496+
Runs GGUF quantized models via [llama.cpp](https://github.com/ggml-org/llama.cpp)
407497
(requires `Llama` trait):
408498

409499
```swift
@@ -451,9 +541,55 @@ let response = try await session.respond(
451541
)
452542
```
453543

454-
> [!NOTE]
455-
> Image inputs are not currently supported with `LlamaLanguageModel`.
456-
> Guided generation (structured output via `@Generable`) is supported.
544+
### Ollama
545+
546+
Run models locally via Ollama's
547+
[HTTP API](https://github.com/ollama/ollama/blob/main/docs/api.md):
548+
549+
```swift
550+
// Default: connects to http://localhost:11434
551+
let model = OllamaLanguageModel(model: "qwen3") // `ollama pull qwen3:8b`
552+
553+
// Custom endpoint
554+
let model = OllamaLanguageModel(
555+
endpoint: URL(string: "http://remote-server:11434")!,
556+
model: "llama3.2"
557+
)
558+
559+
let session = LanguageModelSession(model: model)
560+
let response = try await session.respond {
561+
Prompt("Tell me a joke")
562+
}
563+
```
564+
565+
For local models, make sure you're using a vision‑capable model
566+
(for example, a `-vl` variant).
567+
You can combine multiple images:
568+
569+
```swift
570+
let model = OllamaLanguageModel(model: "qwen3-vl") // `ollama pull qwen3-vl:8b`
571+
let session = LanguageModelSession(model: model)
572+
let response = try await session.respond(
573+
to: "Compare these posters and summarize their differences",
574+
images: [
575+
.init(url: URL(string: "https://example.com/poster1.jpg")!),
576+
.init(url: URL(fileURLWithPath: "/path/to/poster2.jpg"))
577+
]
578+
)
579+
print(response.content)
580+
```
581+
582+
Pass any model-specific parameters using custom generation options:
583+
584+
```swift
585+
var options = GenerationOptions(temperature: 0.8)
586+
options[custom: OllamaLanguageModel.self] = [
587+
"seed": .int(42),
588+
"repeat_penalty": .double(1.2),
589+
"num_ctx": .int(4096),
590+
"stop": .array([.string("###")])
591+
]
592+
```
457593

458594
### OpenAI
459595

@@ -512,28 +648,32 @@ options[custom: OpenAILanguageModel.self] = .init(
512648

513649
### Open Responses
514650

515-
Connects to any API that conforms to the [Open Responses](https://www.openresponses.org) specification (e.g. OpenAI, OpenRouter, or other compatible providers). Base URL is required—use your provider’s endpoint:
651+
Connects to any API that conforms to the
652+
[Open Responses](https://www.openresponses.org) specification
653+
(e.g. OpenAI, OpenRouter, or other compatible providers).
654+
Base URL is required—use your provider’s endpoint:
516655

517656
```swift
518-
// Example: OpenAI
657+
// Example: OpenRouter (https://openrouter.ai/api/v1/)
519658
let model = OpenResponsesLanguageModel(
520-
baseURL: URL(string: "https://api.openai.com/v1/")!,
659+
baseURL: URL(string: "https://openrouter.ai/api/v1/")!,
521660
apiKey: ProcessInfo.processInfo.environment["OPEN_RESPONSES_API_KEY"]!,
522-
model: "gpt-4o-mini"
661+
model: "openai/gpt-4o-mini"
523662
)
524663

525-
// Example: OpenRouter (https://openrouter.ai/api/v1/)
664+
// Example: OpenAI
526665
let model = OpenResponsesLanguageModel(
527-
baseURL: URL(string: "https://openrouter.ai/api/v1/")!,
666+
baseURL: URL(string: "https://api.openai.com/v1/")!,
528667
apiKey: ProcessInfo.processInfo.environment["OPEN_RESPONSES_API_KEY"]!,
529-
model: "openai/gpt-4o-mini"
668+
model: "gpt-4o-mini"
530669
)
531670

532671
let session = LanguageModelSession(model: model)
533672
let response = try await session.respond(to: "Say hello")
534673
```
535674

536-
Custom options support Open Responses–specific fields such as `tool_choice` (including `allowed_tools`) and `extraBody`:
675+
Custom options support Open Responses–specific fields,
676+
such as `tool_choice` (including `allowed_tools`) and `extraBody`:
537677

538678
```swift
539679
var options = GenerationOptions(temperature: 0.8)
@@ -667,56 +807,6 @@ let response = try await session.respond(
667807
> [!TIP]
668808
> Gemini server tools are not available as client tools (`Tool`) for other models.
669809
670-
### Ollama
671-
672-
Run models locally via Ollama's
673-
[HTTP API](https://github.com/ollama/ollama/blob/main/docs/api.md):
674-
675-
```swift
676-
// Default: connects to http://localhost:11434
677-
let model = OllamaLanguageModel(model: "qwen3") // `ollama pull qwen3:8b`
678-
679-
// Custom endpoint
680-
let model = OllamaLanguageModel(
681-
endpoint: URL(string: "http://remote-server:11434")!,
682-
model: "llama3.2"
683-
)
684-
685-
let session = LanguageModelSession(model: model)
686-
let response = try await session.respond {
687-
Prompt("Tell me a joke")
688-
}
689-
```
690-
691-
For local models, make sure you're using a vision‑capable model
692-
(for example, a `-vl` variant).
693-
You can combine multiple images:
694-
695-
```swift
696-
let model = OllamaLanguageModel(model: "qwen3-vl") // `ollama pull qwen3-vl:8b`
697-
let session = LanguageModelSession(model: model)
698-
let response = try await session.respond(
699-
to: "Compare these posters and summarize their differences",
700-
images: [
701-
.init(url: URL(string: "https://example.com/poster1.jpg")!),
702-
.init(url: URL(fileURLWithPath: "/path/to/poster2.jpg"))
703-
]
704-
)
705-
print(response.content)
706-
```
707-
708-
Pass any model-specific parameters using custom generation options:
709-
710-
```swift
711-
var options = GenerationOptions(temperature: 0.8)
712-
options[custom: OllamaLanguageModel.self] = [
713-
"seed": .int(42),
714-
"repeat_penalty": .double(1.2),
715-
"num_ctx": .int(4096),
716-
"stop": .array([.string("###")])
717-
]
718-
```
719-
720810
## Testing
721811

722812
Run the test suite to verify everything works correctly:

0 commit comments

Comments
 (0)