Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
183 changes: 183 additions & 0 deletions docs/source/run-workflows/fastmcp-server.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
<!--
SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

# NeMo Agent Toolkit as a FastMCP Server

Model Context Protocol (MCP) is an open protocol developed by Anthropic that standardizes how applications provide context to [LLMs](../build-workflows/llms/index.md). This guide covers how to run NVIDIA NeMo Agent Toolkit workflows as a FastMCP server.

## Decision

NeMo Agent Toolkit supports two MCP server runtimes. Both publish the same workflow tools as MCP. Choose the runtime that matches your deployment stack and organization’s MCP server policy:

- Use `nat mcp serve` for the [MCP SDK server runtime](https://github.com/modelcontextprotocol/python-sdk).
- Use `nat fastmcp server run` for the [FastMCP server runtime](https://github.com/jlowin/fastmcp).
- For the MCP SDK server guide, see [NeMo Agent Toolkit as an MCP Server](./mcp-server.md).
- MCP client commands and configuration require the MCP SDK package (`nvidia-nat-mcp`).

## Installation

Install the `nvidia-nat-fastmcp` package:

```bash
uv pip install nvidia-nat-fastmcp
```

For MCP client commands and configuration, install the `nvidia-nat-mcp` package:

```bash
uv pip install nvidia-nat-mcp
```

## FastMCP Server Usage

Use `nat fastmcp server run` to start a FastMCP server and publish workflow tools.

```bash
nat fastmcp server run --config_file examples/getting_started/simple_calculator/configs/config.yml
```

This starts a FastMCP server on the default host (`localhost`) and port (`9902`) and publishes all workflow tools at `http://localhost:9902/mcp` using streamable-http transport.

You can also specify server settings with CLI flags:

```bash
nat fastmcp server run --config_file examples/getting_started/simple_calculator/configs/config.yml \
--host 0.0.0.0 \
--port 9902 \
--name "My FastMCP Server"
```

### Using Developer Mode

Use `nat fastmcp server dev` to restart the server when files change. This is useful when you iterate on workflow code or configuration.

```bash
nat fastmcp server dev --config_file examples/getting_started/simple_calculator/configs/config.yml \
--watch-path examples/getting_started/simple_calculator/src
```

### Filtering FastMCP Tools

You can publish a subset of tools using the `--tool_names` flag:

```bash
nat fastmcp server run --config_file examples/getting_started/simple_calculator/configs/config.yml \
--tool_names calculator
```

### Mounting at Custom Paths

To mount the server at a custom base path, set `base_path` in the configuration file:

```yaml
general:
front_end:
_type: fastmcp
name: "my_fastmcp_server"
base_path: "/api/v1"
```

With this configuration, the FastMCP server is accessible at `http://localhost:9902/api/v1/mcp`.

## Inspecting and Running MCP Tools Published by a FastMCP Server

Use `nat mcp client` to inspect and run tools exposed by a FastMCP server.

**Note:** The `nat mcp client` commands require the `nvidia-nat-mcp` package. If you encounter an error about missing MCP client functionality, install it with `uv pip install "nvidia-nat[mcp]"`.

### List all tools

```console
$ nat mcp client tool list --url http://localhost:9902/mcp
calculator__divide
calculator__compare
calculator__subtract
calculator__add
calculator__multiply
```

### List a tool with schema

```console
$ nat mcp client tool list --url http://localhost:9902/mcp --tool calculator__multiply --detail
Tool: calculator__multiply
Description: Multiply two or more numbers together.
Input Schema:
{
"properties": {
"numbers": {
"description": "",
"items": {
"type": "number"
},
"title": "Numbers",
"type": "array"
}
},
"required": [
"numbers"
],
"title": "Calculator__MultiplyInputSchema",
"type": "object"
}
```

### Call a tool with JSON arguments

```console
$ nat mcp client tool call calculator__multiply \
--url http://localhost:9902/mcp \
--json-args '{"numbers": [1, 3, 6, 10]}'
180.0
```

### Using the `/debug/tools/list` route (no MCP client required)

```console
$ curl -s http://localhost:9902/debug/tools/list | jq
```

## Integration with MCP Clients

The FastMCP server implements the Model Context Protocol specification, so it works with MCP clients. You can run a workflow that connects to the FastMCP server by pointing an MCP client function group at `http://localhost:9902/mcp`.

Example:

```bash
nat run --config_file examples/MCP/simple_calculator_fastmcp/configs/config-mcp-client.yml \
--input "Is 2 times 2 greater than the current hour?"
```

## Authentication

FastMCP servers can validate bearer tokens using OAuth2 token introspection. Configure `server_auth` in your front end config with the introspection endpoint and client credentials.

See the protected example for a full setup:

- `examples/MCP/simple_calculator_fastmcp_protected`

## Verifying FastMCP Server Health

You can verify server health using the `/health` route or `nat mcp client ping`:

```console
$ curl -s http://localhost:9902/health | jq
```

```console
$ nat mcp client ping --url http://localhost:9902/mcp
```
15 changes: 15 additions & 0 deletions docs/source/run-workflows/mcp-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,21 @@ Model Context Protocol (MCP) is an open protocol developed by Anthropic that sta

This guide will cover how to use NeMo Agent Toolkit as an MCP Server to publish [tools](../build-workflows/functions-and-function-groups/functions.md#agents-and-tools) using MCP. For more information on how to use NeMo Agent Toolkit as an MCP Host with one or more MCP Clients, refer to [MCP Client](../build-workflows/mcp-client.md).

## Decision

NeMo Agent Toolkit supports two MCP server runtimes. Both publish the same workflow tools as MCP. Choose the runtime that matches your deployment stack:

- Use `nat mcp serve` for the [MCP SDK server runtime](https://github.com/modelcontextprotocol/python-sdk).
- Use `nat fastmcp server run` for the [FastMCP server runtime](https://github.com/jlowin/fastmcp). For FastMCP server support, see [NeMo Agent Toolkit as a FastMCP Server](./fastmcp-server.md).

## Installation

Install the `nvidia-nat-mcp` package:

```bash
uv pip install nvidia-nat-mcp
```

## MCP Server Usage

The `nat mcp serve` command can be used to start an MCP server that publishes the [functions](../build-workflows/functions-and-function-groups/functions.md) from your [workflow](../build-workflows/about-building-workflows.md) as MCP tools.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,6 @@ authentication:
_type: mcp_oauth2
server_url: ${CORPORATE_MCP_JIRA_URL}
redirect_uri: ${NAT_REDIRECT_URI:-http://localhost:8000/auth/redirect}
default_user_id: ${NAT_USER_ID}
allow_default_user_id_for_tool_calls: ${ALLOW_DEFAULT_USER_ID_FOR_TOOL_CALLS:-true}

llms:
nim_llm:
Expand Down
75 changes: 75 additions & 0 deletions examples/MCP/simple_calculator_fastmcp/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
<!--
SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

# Simple Calculator - FastMCP

**Complexity:** 🟢 Beginner

This example demonstrates how to run the NVIDIA NeMo Agent Toolkit as a FastMCP server and use those tools from a Model Context Protocol (MCP) client workflow.

This example mirrors the `simple_calculator_mcp` workflow, but it uses the FastMCP server command and defaults to port `9902`. The FastMCP server integration comes from `nvidia-nat-fastmcp`, and the MCP client commands and configuration use `nvidia-nat-mcp`.

## Prerequisites

- **Agent toolkit**: Ensure you have the NVIDIA NeMo Agent Toolkit installed. If you have not already done so, follow the instructions in the [Install Guide](../../../docs/source/get-started/installation.md#install-from-source) to create the development environment and install the toolkit.
- **Base workflow**: This example builds upon the Getting Started [Simple Calculator](../../getting_started/simple_calculator/) example. Make sure you are familiar with the example before proceeding.

## Installation and Setup

If you have not already done so, follow the instructions in the [Install Guide](../../../docs/source/get-started/installation.md#install-from-source) to create the development environment and install the toolkit.

### Install this Workflow

Install this example:

```bash
uv pip install -e examples/MCP/simple_calculator_fastmcp
```

## Run the Workflow

1. Start the FastMCP server:

```bash
nat fastmcp server run --config_file examples/getting_started/simple_calculator/configs/config.yml
```

This starts a FastMCP server on port `9902` with endpoint `/mcp` and uses `streamable-http` transport.

2. Inspect the tools available on the FastMCP server using the MCP client:

```bash
nat mcp client tool list --url http://localhost:9902/mcp
```

Sample output:

```text
calculator__add
calculator__subtract
calculator__multiply
calculator__divide
calculator__compare
```

3. Run the workflow:

```bash
nat run --config_file examples/MCP/simple_calculator_fastmcp/configs/config-mcp-client.yml --input "Is the product of 2 * 4 greater than the current hour of the day?"
```

The client configuration is in `examples/MCP/simple_calculator_fastmcp/configs/config-mcp-client.yml`.
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# This config file shows how to use MCP servers as NeMo Agent Toolkit tools.
# It connects to a local MCP server with stdio transport and a FastMCP server
# using streamable-http transport.

# This config file shows -
# 1. how to use a local MCP server to get the current date and time using stdio transport.
# 2. how to access a FastMCP server using streamable-http transport for math operations.
#
# As the mcp_server_time is running locally ensure that the package "mcp_server_time" is installed
# on your local machine. For example, if you are using pip, you can install it with:
# uv pip install mcp-server-time

function_groups:
mcp_time:
_type: mcp_client
server:
transport: stdio
command: "python"
args: ["-m", "mcp_server_time", "--local-timezone=America/Los_Angeles"]
tool_overrides:
# Optionally override the tool name and description from the MCP server
get_current_time:
alias: get_current_time_mcp_tool
description: "Returns the current date and time"

mcp_math:
_type: mcp_client
server:
transport: streamable-http
url: "http://localhost:9902/mcp"
include:
- calculator__add
- calculator__subtract
- calculator__multiply
- calculator__divide
- calculator__compare

llms:
nim_llm:
_type: nim
model_name: meta/llama-3.1-70b-instruct
temperature: 0.0
max_tokens: 1024

workflow:
_type: react_agent
tool_names:
- mcp_time
- mcp_math
llm_name: nim_llm
verbose: true
retry_parsing_errors: true
max_retries: 3
Loading