Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ Download the encoderfile CLI tool to build your own model binaries:
curl -fsSL https://raw.githubusercontent.com/mozilla-ai/encoderfile/main/install.sh | sh
```

> **Note for Windows users:** Pre-built binaries are not available for Windows. Please see our guide on [building from source](https://mozilla-ai.github.io/encoderfile/latest/reference/building/) for instructions on building from source.
> **Note for Windows users:** Pre-built binaries are not available for Windows. Please see our guide on [building from source](https://mozilla-ai.github.io/encoderfile/reference/building/) for instructions on building from source.

Move the binary to a location in your PATH:
```bash
Expand All @@ -92,7 +92,7 @@ mv encoderfile ~/.local/bin/

### Option 2: Build CLI Tool from Source

See our guide on [building from source](https://mozilla-ai.github.io/encoderfile/latest/reference/building/) for detailed instructions on building the CLI tool from source.
See our guide on [building from source](https://mozilla-ai.github.io/encoderfile/reference/building/) for detailed instructions on building the CLI tool from source.

Quick build:
```bash
Expand Down Expand Up @@ -310,16 +310,16 @@ Run as a Model Context Protocol server:

## 📚 Documentation

- **[Getting Started Guide](https://mozilla-ai.github.io/encoderfile/latest/getting-started/)** - Step-by-step tutorial
- **[Building Guide](https://mozilla-ai.github.io/encoderfile/latest/reference/building/)** - Build encoderfiles from ONNX models
- **[CLI Reference](https://mozilla-ai.github.io/encoderfile/latest/reference/cli/)** - Complete command-line documentation
- **[API Reference](https://mozilla-ai.github.io/encoderfile/latest/reference/api-reference/)** - REST, gRPC, and MCP API docs
- **[Getting Started Guide](https://mozilla-ai.github.io/encoderfile/getting-started/)** - Step-by-step tutorial
- **[Building Guide](https://mozilla-ai.github.io/encoderfile/reference/building/)** - Build encoderfiles from ONNX models
- **[CLI Reference](https://mozilla-ai.github.io/encoderfile/reference/cli/)** - Complete command-line documentation
- **[API Reference](https://mozilla-ai.github.io/encoderfile/reference/api-reference/)** - REST, gRPC, and MCP API docs

## 🛠️ Building Custom Encoderfiles

Once you have the `encoderfile` CLI tool installed, you can build binaries from any compatible HuggingFace model.

See our guide on [building from source](https://mozilla-ai.github.io/encoderfile/latest/reference/building/) for detailed instructions including:
See our guide on [building from source](https://mozilla-ai.github.io/encoderfile/reference/building/) for detailed instructions including:

- How to export models to ONNX format
- Configuration file options
Expand Down
42 changes: 42 additions & 0 deletions docs/cookbooks/mcp-integration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Using `encoderfile` from agents

The [Model Context Protocol](https://www.anthropic.com/news/model-context-protocol) (MCP) introduced by Anthropic has proven to be a popular method for providing an AI agent with access to a variety of tools. [This Huggingface blog post](https://huggingface.co/blog/Kseniase/mcp) has a nice explanation of MCP.

In the following example we will use Mozilla's own [`any-agent`](https://github.com/mozilla-ai/any-agent) and [`any-llm`](https://github.com/mozilla-ai/any-llm) packages to build a small agent that leverages on capabilities provided by a test encoderfile.


## Build the custom encoderfile and start the server

We will use the existing test config to build an encoderfile using one of the test models by Mozilla.ai. It will detect Personally Identifiable Information (PII) and tag it accordingly, using tags like `B-SURNAME` for, well, surnames, and `O` for non-PII tokens. As we will see, even if the output consists of logits and tags, the underlying LLM is usually robust enough to focus only on the tags and act appropriately.

```sh
curl -fsSL https://raw.githubusercontent.com/mozilla-ai/encoderfile/main/install.sh | sh
encoderfile build -f test_config.yml
```

After building it, we only need to set it up in MCP mode so it will listen to requests. By default it will bind to all interfaces, using port 9100.

```sh
my-model-2.encoderfile mcp
```

## Install Dependencies

For this test, we will need the `any-agent` and `any-llm` Python packages:

```sh
pip install any-agent
pip install any-llm-sdk[mistral]
```

## Write the agent

Now we will write an agent with the appropriate prompt. We instruct the agent to use the provided tool, since the current description is fairly generic, and not use metadata that it might consider useful but is not documented anywhere in the tool itself. We will also instruct it to replace only surnames to showcase that the tags can be extracted appropriately:

```python
--8<-- "examples/agent_mcp_integration/agent_test.py"
```

After some struggling with the call conventions, the LLM finally obtains the information from the encoderfile and acts accordingly:

> `My name is Javier [REDACTED]`
4 changes: 2 additions & 2 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ While **Llamafile** focuses on generative models, **Encoderfile** is purpose-bui
* **Protocol Agnostic:** Runs as a REST API, gRPC microservice, CLI tool, or MCP Server out of the box.
* **Compliance-Friendly:** Deterministic and offline-safe, making it ideal for strict security boundaries.

> **Note for Windows users:** Pre-built binaries are not available for Windows. Please see our guide on [building from source](https://mozilla-ai.github.io/encoderfile/latest/reference/building/) for instructions on building from source.
> **Note for Windows users:** Pre-built binaries are not available for Windows. Please see our guide on [building from source](https://mozilla-ai.github.io/encoderfile/reference/building/) for instructions on building from source.

## Use Cases

Expand All @@ -48,7 +48,7 @@ Encoderfile supports encoder-only transformers for:
- **Token Classification** - Named Entity Recognition, PII detection
- **Sentence Embeddings** - Semantic search, clustering

See our guide on [building from source](https://mozilla-ai.github.io/encoderfile/latest/reference/building/) for detailed instructions on building the CLI tool from source.
See our guide on [building from source](https://mozilla-ai.github.io/encoderfile/reference/building/) for detailed instructions on building the CLI tool from source.

Generation models (GPT, T5) are not supported. See [CLI Reference](reference/cli.md) for complete model type details.

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/api-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -652,7 +652,7 @@ Encoderfile supports Model Context Protocol, allowing integration with MCP-compa
### Connection Details

- **Endpoint:** `/mcp`
- **Transport:** HTTP-based MCP protocol
- **Transport:** HTTP-based MCP protocol (Streamable HTTP only)
- **Port:** Same as HTTP server (default: 8080)

### MCP Tools
Expand Down
66 changes: 66 additions & 0 deletions examples/agent_mcp_integration/agent_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
import os
import shutil
from getpass import getpass

if "MISTRAL_API_KEY" not in os.environ:
print("MISTRAL_API_KEY not found in environment!")
api_key = getpass("Please enter your MISTRAL_API_KEY: ")
os.environ["MISTRAL_API_KEY"] = api_key
print("MISTRAL_API_KEY set for this session!")
else:
print("MISTRAL_API_KEY found in environment.")

# Quick Environment Check (Airbnb tool requires npx/Node.js)\\n",
if not shutil.which("npx"):
print(
"⚠️ Warning: 'npx' was not found in your path. The Airbnb tool requires Node.js/npm to run."
)


from any_agent import AgentConfig, AnyAgent
from any_agent.config import MCPStreamableHttp


async def send_message(message: str) -> str:
"""Display a message to the user and wait for their response.

Args:
message: str
The message to be displayed to the user.

Returns:
str: The response from the user.

"""
if os.environ.get("IN_PYTEST") == "1":
return "2 people, next weekend, low budget. Do not ask for any more information or confirmation."
return input(message + " ")


async def main():
print("Start creating agent")
eftool = MCPStreamableHttp(url="http://localhost:9100/mcp")
try:
agent = await AnyAgent.create_async(
"tinyagent", # See all options in https://mozilla-ai.github.io/any-agent/
AgentConfig(model_id="mistral:mistral-large-latest", tools=[eftool]),
)
except Exception as e:
print(f"❌ Failed to create agent: {e}")
print("Done creating agent")

prompt = """
Use the eftool tool to remove the personal information from this line: "My name is Javier Torres".
Do not use any metadata. The "inputs" param must be a sequence with one string.
Replace each surname, but not given names, with [REDACTED].
"""

agent_trace = await agent.run_async(prompt)
print(agent_trace.final_output)
await agent.cleanup_async()


if __name__ == "__main__":
import asyncio

asyncio.run(main())