CoreAIChat

CoreAIChat is a minimal macOS demo app that shows how to build a local AI chat experience with SwiftUI and Apple’s Core AI framework.

⚠️ Disclaimer: This app is for demonstration purposes only and is not production-ready. Output generated by the local large language model may be inaccurate, incomplete, or misleading.

Beta notice: This project currently relies on beta versions of Apple's development tools and frameworks. Many components are still under active development, so APIs, behavior, and setup instructions may change.

Setup

Prepare the export tools

Install uv if it is not already available:

brew install uv

Alternatively, use the official installation script:

curl -LsSf https://astral.sh/uv/install.sh | sh

Clone Apple's coreai-models repository and verify the installation:

git clone https://github.com/apple/coreai-models.git && cd coreai-models

Before exporting Gemma 3, accept its license terms on Hugging Face and create an access token. Then authenticate the Hugging Face CLI:

brew install hf
hf auth login --token <YOUR_TOKEN>

Export the model

From the coreai-models directory, run:

uv run coreai.llm.export google/gemma-3-4b-it

This process may take a few minutes to complete, as the model may need to be downloaded first before it can be converted and exported to the Core AI format.

The exported model is written to ./exports/gemma_3_4b_it_4bit_dynamic. This directory should contain the .aimodel package, metadata.json, and a tokenizer directory.

Add the model to Xcode

Open CoreAIChat.xcodeproj. Copy the complete contents of the exported model directory into the existing gemma_3_4b_it_4bit_dynamic folder referenced by the Xcode project.

The resulting directory should contain:

gemma_3_4b_it_4bit_dynamic/
├── gemma_3_4b_it_4bit_dynamic.aimodel/
├── metadata.json
└── tokenizer/

Run the app

Select the CoreAIChat scheme in Xcode and run the app. Loading the model may take a short while, especially on the first launch.

Run a Core AI Language Model

The following example shows how to load an exported Core AI language model, create a session, and generate a response using the Foundation Models framework.

import FoundationModels
import CoreAILanguageModels

let modelUrl: URL? = Bundle.main.url(forResource: "gemma_3_4b_it_4bit_dynamic", withExtension: nil)

let model = try await CoreAILanguageModel(resourcesAt: modelURL!)

let session = LanguageModelSession(model: model)

let response = try await session.respond(to: "Hello")

print(response)

License

Localight is available under the MIT License. See LICENSE for the full license text.