From 980987b2c82315a21090120a5e7303ce867f4721 Mon Sep 17 00:00:00 2001 From: SPeak Date: Tue, 10 Mar 2026 20:17:03 +0800 Subject: [PATCH 1/2] Pr/ci docs refresh (#7) * Add multilingual docs structure * Document production readiness gaps * Document isolated-client concurrency model * Expand usage scenarios in examples docs * Add llmapi usage best-practices skill * Release 0.1.0 --- .agents/skills/README.md | 1 + .agents/skills/llmapi-best-practices/SKILL.md | 96 ++++++ .../llmapi-best-practices/references/links.md | 58 ++++ README.md | 23 +- README.zh.hant.md | 123 +++++--- README.zh.md | 122 +++++--- docs/README.md | 27 +- docs/en/README.md | 47 +++ docs/en/advanced.md | 133 +++++++++ docs/en/cpp-api.md | 278 ++++++++++++++++++ docs/en/examples.md | 277 +++++++++++++++++ docs/en/getting-started.md | 120 ++++++++ docs/en/providers.md | 141 +++++++++ docs/getting-started.md | 2 +- docs/zh-hant/README.md | 43 +++ docs/zh-hant/advanced.md | 51 ++++ docs/zh-hant/cpp-api.md | 80 +++++ docs/zh-hant/examples.md | 138 +++++++++ docs/zh-hant/getting-started.md | 81 +++++ docs/zh-hant/providers.md | 46 +++ docs/zh/README.md | 43 +++ docs/zh/advanced.md | 51 ++++ docs/zh/cpp-api.md | 80 +++++ docs/zh/examples.md | 138 +++++++++ docs/zh/getting-started.md | 81 +++++ docs/zh/providers.md | 46 +++ src/client.cppm | 2 + src/llmapi.cppm | 1 + src/tinyhttps/http.cppm | 2 + tests/llmapi/test_client.cpp | 42 ++- xmake.lua | 1 + 31 files changed, 2252 insertions(+), 122 deletions(-) create mode 100644 .agents/skills/llmapi-best-practices/SKILL.md create mode 100644 .agents/skills/llmapi-best-practices/references/links.md create mode 100644 docs/en/README.md create mode 100644 docs/en/advanced.md create mode 100644 docs/en/cpp-api.md create mode 100644 docs/en/examples.md create mode 100644 docs/en/getting-started.md create mode 100644 docs/en/providers.md create mode 100644 docs/zh-hant/README.md create mode 100644 docs/zh-hant/advanced.md create mode 100644 docs/zh-hant/cpp-api.md create mode 100644 docs/zh-hant/examples.md create mode 100644 docs/zh-hant/getting-started.md create mode 100644 docs/zh-hant/providers.md create mode 100644 docs/zh/README.md create mode 100644 docs/zh/advanced.md create mode 100644 docs/zh/cpp-api.md create mode 100644 docs/zh/examples.md create mode 100644 docs/zh/getting-started.md create mode 100644 docs/zh/providers.md diff --git a/.agents/skills/README.md b/.agents/skills/README.md index d21b545..e7eda49 100644 --- a/.agents/skills/README.md +++ b/.agents/skills/README.md @@ -7,6 +7,7 @@ | 技能 | 说明 | |------|------| | [mcpp-style-ref](mcpp-style-ref/SKILL.md) | 面向 mcpp 项目的 Modern/Module C++ (C++23) 命名、模块化与实践规则 | +| [llmapi-best-practices](llmapi-best-practices/SKILL.md) | 面向本仓库 llmapi 的使用最佳实践、文档入口、示例位置、并发/异常/重试建议与相关生态链接 | ## 使用方式 diff --git a/.agents/skills/llmapi-best-practices/SKILL.md b/.agents/skills/llmapi-best-practices/SKILL.md new file mode 100644 index 0000000..f4d9199 --- /dev/null +++ b/.agents/skills/llmapi-best-practices/SKILL.md @@ -0,0 +1,96 @@ +--- +name: llmapi-best-practices +description: 在本仓库中使用 llmapi 的最佳实践。适用于用户询问如何使用 llmapi、如何接入 OpenAI/Anthropic/兼容端点、并发模型、错误处理、文档入口、示例位置、xmake/xlings/mcpplibs-index 安装方式，或需要快速定位本项目相关链接与推荐用法时。 +--- + +# llmapi-best-practices + +用于回答“如何正确使用本项目 `llmapi`”相关问题的技能。 + +## 何时使用 + +在以下场景使用： + +- 用户询问如何集成或调用 `llmapi` +- 用户想知道 OpenAI / Anthropic / 兼容端点的推荐接法 +- 用户询问并发、异常处理、重试边界、生产可用性 +- 用户需要项目文档、示例、仓库、生态链接 +- 用户询问 `mcpplibs` / `mcpplibs-index` / `xlings` 与本项目的关系 + +如果问题是“如何修改 C++ 模块代码风格”，优先改用 `mcpp-style-ref`。 + +## 快速工作流 + +1. 先确认用户要的是“使用方式”还是“改库实现”。 +2. 默认优先引用本仓库文档入口： + - 英文文档：`docs/en/README.md` + - 简中文档：`docs/zh/README.md` + - 繁中文档：`docs/zh-hant/README.md` +3. 解释推荐接法时，优先给当前项目的真实 API： + - 默认 OpenAI 风格：`Client(Config{...})` + - Anthropic：`Client(AnthropicConfig{...})` +4. 解释并发时，使用当前仓库推荐边界： + - 实例隔离，上层并发，不共享 `Client` +5. 解释错误处理时，明确当前建议： + - 当前以异常为主 + - 自动重试更适合由库使用者在上层实现 + +## 推荐回答要点 + +### 1. 默认 API 入口 + +OpenAI 风格默认入口： + +```cpp +auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", +}); +``` + +Anthropic： + +```cpp +auto client = Client(AnthropicConfig{ + .apiKey = std::getenv("ANTHROPIC_API_KEY"), + .model = "claude-sonnet-4-20250514", +}); +``` + +兼容端点： + +```cpp +auto client = Client(Config{ + .apiKey = std::getenv("DEEPSEEK_API_KEY"), + .baseUrl = std::string(URL::DeepSeek), + .model = "deepseek-chat", +}); +``` + +### 2. 并发建议 + +- `Client` 是有状态对象，不要跨线程共享 +- 每个任务 / 线程各自创建一个 `Client` +- 多模型 / 多 provider 并发调用时，各自实例隔离即可 + +### 3. 错误处理建议 + +- 当前主路径是异常模式 +- 如果用户不希望异常外抛，建议在调用点包一层转换成 `optional` / `expected` +- 自动重试策略先由使用者在上层实现，不要默认承诺库内自动重试 + +### 4. 文档和示例定位 + +如果用户要具体用法，优先引导到： + +- 入门：`docs/*/getting-started.md` +- 使用场景：`docs/*/examples.md` +- Provider：`docs/*/providers.md` +- 并发 / 高级用法：`docs/*/advanced.md` +- API：`docs/*/cpp-api.md` + +## 参考文件 + +需要项目和生态链接时，读取： + +- [references/links.md](references/links.md) diff --git a/.agents/skills/llmapi-best-practices/references/links.md b/.agents/skills/llmapi-best-practices/references/links.md new file mode 100644 index 0000000..9e08bb1 --- /dev/null +++ b/.agents/skills/llmapi-best-practices/references/links.md @@ -0,0 +1,58 @@ +# llmapi 相关链接 + +## 仓库与主页 + +- GitHub 仓库：`https://github.com/mcpplibs/llmapi` +- README（英文）：`/home/speak/workspace/github/mcpplibs/llmapi/README.md` +- README（简中）：`/home/speak/workspace/github/mcpplibs/llmapi/README.zh.md` +- README（繁中）：`/home/speak/workspace/github/mcpplibs/llmapi/README.zh.hant.md` + +## 文档入口 + +- 文档导航：`/home/speak/workspace/github/mcpplibs/llmapi/docs/README.md` +- 英文文档主页：`/home/speak/workspace/github/mcpplibs/llmapi/docs/en/README.md` +- 简中文档主页：`/home/speak/workspace/github/mcpplibs/llmapi/docs/zh/README.md` +- 繁中文档主页：`/home/speak/workspace/github/mcpplibs/llmapi/docs/zh-hant/README.md` + +## 使用文档 + +- 英文入门：`/home/speak/workspace/github/mcpplibs/llmapi/docs/en/getting-started.md` +- 英文示例：`/home/speak/workspace/github/mcpplibs/llmapi/docs/en/examples.md` +- 英文 Provider：`/home/speak/workspace/github/mcpplibs/llmapi/docs/en/providers.md` +- 英文高级用法：`/home/speak/workspace/github/mcpplibs/llmapi/docs/en/advanced.md` +- 英文 C++ API：`/home/speak/workspace/github/mcpplibs/llmapi/docs/en/cpp-api.md` + +- 简中入门：`/home/speak/workspace/github/mcpplibs/llmapi/docs/zh/getting-started.md` +- 简中示例：`/home/speak/workspace/github/mcpplibs/llmapi/docs/zh/examples.md` +- 简中 Provider：`/home/speak/workspace/github/mcpplibs/llmapi/docs/zh/providers.md` +- 简中高级用法：`/home/speak/workspace/github/mcpplibs/llmapi/docs/zh/advanced.md` +- 简中 C++ API：`/home/speak/workspace/github/mcpplibs/llmapi/docs/zh/cpp-api.md` + +- 繁中入门：`/home/speak/workspace/github/mcpplibs/llmapi/docs/zh-hant/getting-started.md` +- 繁中示例：`/home/speak/workspace/github/mcpplibs/llmapi/docs/zh-hant/examples.md` +- 繁中 Provider：`/home/speak/workspace/github/mcpplibs/llmapi/docs/zh-hant/providers.md` +- 繁中高级用法：`/home/speak/workspace/github/mcpplibs/llmapi/docs/zh-hant/advanced.md` +- 繁中 C++ API：`/home/speak/workspace/github/mcpplibs/llmapi/docs/zh-hant/cpp-api.md` + +## 示例源码 + +- `examples/hello_mcpp.cpp` +- `examples/basic.cpp` +- `examples/chat.cpp` +- `examples/xmake.lua` + +## 生态与依赖 + +- mcpplibs：`https://github.com/mcpplibs` +- mcpplibs-index：`https://github.com/mcpplibs/mcpplibs-index` +- xlings：`https://github.com/d2learn/xlings` +- xmake：`https://xmake.io/` +- OpenAI API：`https://platform.openai.com/docs/api-reference` +- Anthropic API：`https://docs.anthropic.com/` + +## 推荐回答时的关键事实 + +- 默认 OpenAI 风格入口：`Client(Config{...})` +- Anthropic 入口：`Client(AnthropicConfig{...})` +- 推荐并发模型：实例隔离，上层并发，不共享 `Client` +- 推荐错误处理：异常模式为主，自动重试建议由库使用者在上层实现 diff --git a/README.md b/README.md index 7713330..4e0e1d6 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ | English - [简体中文](README.zh.md) - [繁體中文](README.zh.hant.md) | |:---:| -| [Documentation](docs/) - [C++ API](docs/cpp-api.md) - [Examples](docs/examples.md) | +| [Documentation](docs/README.md) - [English Docs](docs/en/README.md) - [中文文档](docs/zh/README.md) - [繁體中文文件](docs/zh-hant/README.md) | `llmapi` provides a typed `Client` API for chat, streaming, embeddings, tool calls, and conversation persistence. The default config alias `Config` maps to OpenAI-style providers, so the common case does not need an explicit `openai::OpenAI` wrapper. @@ -22,6 +22,23 @@ - Conversation save/load helpers - OpenAI-compatible endpoint support through `openai::Config::baseUrl` +## Production Readiness + +`llmapi` is usable for internal tools, prototypes, and early production experiments, but it should not yet be treated as fully industrial-grade infrastructure. + +Required gaps before that bar: + +- Unified error model across providers and transport +- Retry, backoff, timeout, and idempotency policy +- Request cancellation for long-running and streaming calls +- Logging, metrics, trace hooks, and request correlation +- Hardening of the custom HTTP/TLS transport layer +- Fault-injection, concurrency, and large-scale mock testing +- Stronger API compatibility and versioning guarantees +- More complete production configuration surface +- Explicit thread-safety and concurrency semantics +- Operational documentation for retries, keys, proxies, and failure handling + ## Quick Start ```cpp @@ -79,7 +96,7 @@ xmake run chat ```lua add_repositories("mcpplibs-index https://github.com/mcpplibs/mcpplibs-index.git") -add_requires("llmapi 0.0.2") +add_requires("llmapi 0.1.0") target("demo") set_kind("binary") @@ -89,7 +106,7 @@ target("demo") add_packages("llmapi") ``` -See [docs/getting-started.md](docs/getting-started.md) and [docs/providers.md](docs/providers.md) for more setup detail. +See [docs/en/getting-started.md](docs/en/getting-started.md), [docs/en/providers.md](docs/en/providers.md), and [docs/en/README.md](docs/en/README.md) for more setup and readiness detail. ## License diff --git a/README.zh.hant.md b/README.zh.hant.md index 5e7a099..d4d3f39 100644 --- a/README.zh.hant.md +++ b/README.zh.hant.md @@ -1,86 +1,113 @@ # llmapi -> Modern C++ LLM API client with openai-compatible support +> 使用 C++23 模組建構的現代 LLM 客戶端 [![C++23](https://img.shields.io/badge/C%2B%2B-23-blue.svg)](https://en.cppreference.com/w/cpp/23) [![Module](https://img.shields.io/badge/module-ok-green.svg)](https://en.cppreference.com/w/cpp/language/modules) [![License](https://img.shields.io/badge/license-Apache_2.0-blue.svg)](LICENSE) -[![OpenAI Compatible](https://img.shields.io/badge/OpenAI_API-Compatible-green.svg)](https://platform.openai.com/docs/api-reference) +[![OpenAI Compatible](https://img.shields.io/badge/OpenAI-Compatible-green.svg)](https://platform.openai.com/docs/api-reference) | [English](README.md) - [简体中文](README.zh.md) - 繁體中文 | |:---:| -| [完整文件](docs/) - [C++ API](docs/cpp-api.md) - [C API](docs/c-api.md) - [範例](docs/examples.md) | +| [文件導覽](docs/README.md) - [繁體中文文件](docs/zh-hant/README.md) - [English Docs](docs/en/README.md) - [简体中文文档](docs/zh/README.md) | -簡潔、型別安全的 LLM API 客戶端，使用 C++23 模組。流式介面設計，零成本抽象。支援 OpenAI、Poe、DeepSeek 及相容端點。 +`llmapi` 提供型別化的 `Client` API，涵蓋聊天、串流輸出、embeddings、工具呼叫與對話持久化。預設別名 `Config` 對應 OpenAI 風格設定，常見情況下不需要顯式寫出 `openai::OpenAI(...)`。 -## ✨ 特性 +## 特性 -- **C++23 模組** - `import mcpplibs.llmapi` -- **自動儲存歷史** - 對話歷史自動管理 -- **型別安全串流** - 概念約束的回呼函式 -- **流式介面** - 可鏈式呼叫的方法 -- **提供商無關** - OpenAI、Poe 及相容端點 +- C++23 模組：`import mcpplibs.llmapi` +- 強型別訊息、工具與回應結構 +- 同步、非同步、串流聊天介面 +- OpenAI Provider 支援 embeddings +- 支援儲存 / 載入對話歷史 +- 可透過 `baseUrl` 存取 OpenAI 相容端點 -## 快速開始 +## 生產可用性 + +`llmapi` 目前適合內部工具、原型專案與早期生產試用，但還不應直接視為完整工業級基礎設施。 + +要達到那個標準，至少還需要補齊： +- 統一的 provider / 傳輸層錯誤模型 +- 重試、退避、逾時、冪等策略 +- 長請求與串流請求的取消能力 +- 日誌、指標、trace hook、請求關聯資訊 +- 自研 HTTP/TLS 傳輸層的進一步加固 +- 故障注入、並發、Mock、大規模測試 +- 更強的 API 相容性與版本穩定性承諾 +- 更完整的生產設定面 +- 明確的執行緒安全與並發語義 +- 面向維運的重試、金鑰、代理、故障處理文件 + +## 快速開始 ```cpp -import std; import mcpplibs.llmapi; +import std; int main() { - using namespace mcpplibs; - - llmapi::Client client(std::getenv("OPENAI_API_KEY"), llmapi::URL::Poe); - - client.model("gpt-5") - .system("You are a helpful assistant.") - .user("In one sentence, introduce modern C++. 並給出中文翻譯") - .request([](std::string_view chunk) { - std::print("{}", chunk); - std::cout.flush(); - }); + using namespace mcpplibs::llmapi; + + auto apiKey = std::getenv("OPENAI_API_KEY"); + if (!apiKey) { + std::cerr << "OPENAI_API_KEY not set\n"; + return 1; + } + + auto client = Client(Config{ + .apiKey = apiKey, + .model = "gpt-4o-mini", + }); + client.system("You are a concise assistant."); + auto resp = client.chat("用兩句話解釋 C++23 模組的價值。"); + + std::cout << resp.text() << '\n'; return 0; } ``` -### 模型 / 提供商 +## Provider + +- `Config`：`openai::Config` 的匯出別名，預設走 OpenAI 風格 +- `openai::OpenAI`：OpenAI 聊天、串流、工具呼叫、embeddings +- `AnthropicConfig` / `anthropic::Anthropic`：Anthropic 聊天與串流 + +相容端點範例： ```cpp -llmapi::Client client(apiKey, llmapi::URL::OpenAI); // OpenAI -llmapi::Client client(apiKey, llmapi::URL::Poe); // Poe -llmapi::Client client(apiKey, llmapi::URL::DeepSeek); // Deepseek -llmapi::Client client(apiKey, "https://custom.com"); // 自訂 +auto client = Client(Config{ + .apiKey = std::getenv("DEEPSEEK_API_KEY"), + .baseUrl = std::string(URL::DeepSeek), + .model = "deepseek-chat", +}); ``` -## 🛠️ 建置 +## 建置與執行 ```bash -xmake # 建置 -xmake run basic # 執行範例（需先設定 OPENAI_API_KEY） +xmake +xmake run hello_mcpp +xmake run basic +xmake run chat ``` -## 📦 在建置工具中使用 - -### xmake +## 套件管理使用 ```lua --- 0 - 新增 mcpplibs 索引倉庫 -add_repositories("mcpplibs-index https://github.com/mcpplibs/llmapi.git") - --- 1 - 新增需要的函式庫和版本 -add_requires("llmapi 0.0.2") +add_repositories("mcpplibs-index https://github.com/mcpplibs/mcpplibs-index.git") +add_requires("llmapi 0.1.0") + +target("demo") + set_kind("binary") + set_languages("c++23") + set_policy("build.c++.modules", true) + add_files("src/*.cpp") + add_packages("llmapi") ``` -> More: [mcpplibs-index](https://github.com/mcpplibs/mcpplibs-index) - -### cmake - -``` -todo... -``` +更多內容見 [docs/zh-hant/getting-started.md](docs/zh-hant/getting-started.md)、[docs/zh-hant/providers.md](docs/zh-hant/providers.md) 與 [docs/zh-hant/README.md](docs/zh-hant/README.md)。 -## 📄 授權條款 +## 授權 -Apache-2.0 - 詳見 [LICENSE](LICENSE) +Apache-2.0，詳見 [LICENSE](LICENSE) diff --git a/README.zh.md b/README.zh.md index c1a28db..d0df23b 100644 --- a/README.zh.md +++ b/README.zh.md @@ -1,85 +1,113 @@ # llmapi -> Modern C++ LLM API client with openai-compatible support +> 使用 C++23 模块构建的现代 LLM 客户端 [![C++23](https://img.shields.io/badge/C%2B%2B-23-blue.svg)](https://en.cppreference.com/w/cpp/23) [![Module](https://img.shields.io/badge/module-ok-green.svg)](https://en.cppreference.com/w/cpp/language/modules) [![License](https://img.shields.io/badge/license-Apache_2.0-blue.svg)](LICENSE) -[![OpenAI Compatible](https://img.shields.io/badge/OpenAI_API-Compatible-green.svg)](https://platform.openai.com/docs/api-reference) +[![OpenAI Compatible](https://img.shields.io/badge/OpenAI-Compatible-green.svg)](https://platform.openai.com/docs/api-reference) | [English](README.md) - 简体中文 - [繁體中文](README.zh.hant.md) | |:---:| -| [完整文档](docs/) - [C++ API](docs/cpp-api.md) - [C API](docs/c-api.md) - [示例](docs/examples.md) | +| [文档导航](docs/README.md) - [中文文档](docs/zh/README.md) - [English Docs](docs/en/README.md) - [繁體中文文件](docs/zh-hant/README.md) | -简洁、类型安全的 LLM API 客户端，使用 C++23 模块。流式接口设计，零成本抽象。支持 OpenAI、Poe、DeepSeek 及兼容端点。 +`llmapi` 提供类型化的 `Client` API，覆盖聊天、流式输出、嵌入、工具调用与对话持久化。默认别名 `Config` 对应 OpenAI 风格配置，常见场景不需要显式写 `openai::OpenAI(...)`。 -## ✨ 特性 +## 特性 -- **C++23 模块** - `import mcpplibs.llmapi` -- **自动保存历史** - 对话历史自动管理 -- **类型安全流式** - 概念约束的回调函数 -- **流式接口** - 可链式调用的方法 -- **提供商无关** - OpenAI、Poe 及兼容端点 +- C++23 模块：`import mcpplibs.llmapi` +- 强类型消息、工具与响应结构 +- 同步、异步、流式聊天接口 +- OpenAI Provider 支持 embeddings +- 支持保存 / 加载对话历史 +- 可通过 `baseUrl` 访问 OpenAI 兼容端点 + +## 生产可用性 + +`llmapi` 目前适合内部工具、原型项目和早期生产试用，但还不应直接视为完整工业级基础设施。 + +要达到那个标准，至少还需要补齐： + +- 统一的 provider / 传输层错误模型 +- 重试、退避、超时、幂等策略 +- 长请求和流式请求的取消能力 +- 日志、指标、trace hook、请求关联信息 +- 自研 HTTP/TLS 传输层的进一步加固 +- 故障注入、并发、Mock、大规模测试 +- 更强的 API 兼容性与版本稳定性承诺 +- 更完整的生产配置面 +- 明确的线程安全和并发语义 +- 面向运维的重试、密钥、代理、故障处理文档 ## 快速开始 ```cpp -import std; import mcpplibs.llmapi; +import std; int main() { - using namespace mcpplibs; - - llmapi::Client client(std::getenv("OPENAI_API_KEY"), llmapi::URL::Poe); - - client.model("gpt-5") - .system("You are a helpful assistant.") - .user("In one sentence, introduce modern C++. 并给出中文翻译") - .request([](std::string_view chunk) { - std::print("{}", chunk); - std::cout.flush(); - }); + using namespace mcpplibs::llmapi; + + auto apiKey = std::getenv("OPENAI_API_KEY"); + if (!apiKey) { + std::cerr << "OPENAI_API_KEY not set\n"; + return 1; + } + + auto client = Client(Config{ + .apiKey = apiKey, + .model = "gpt-4o-mini", + }); + + client.system("You are a concise assistant."); + auto resp = client.chat("用两句话解释 C++23 模块的价值。"); + std::cout << resp.text() << '\n'; return 0; } ``` -### 模型 / 提供商 +## Provider + +- `Config`：`openai::Config` 的导出别名，默认走 OpenAI 风格 +- `openai::OpenAI`：OpenAI 聊天、流式、工具调用、embeddings +- `AnthropicConfig` / `anthropic::Anthropic`：Anthropic 聊天与流式 + +兼容端点示例： ```cpp -llmapi::Client client(apiKey, llmapi::URL::OpenAI); // OpenAI -llmapi::Client client(apiKey, llmapi::URL::Poe); // Poe -llmapi::Client client(apiKey, llmapi::URL::DeepSeek); // Deepseek -llmapi::Client client(apiKey, "https://custom.com"); // 自定义 +auto client = Client(Config{ + .apiKey = std::getenv("DEEPSEEK_API_KEY"), + .baseUrl = std::string(URL::DeepSeek), + .model = "deepseek-chat", +}); ``` -## 构建 +## 构建与运行 ```bash -xmake # 构建 -xmake run basic # 运行示例（需要先配置 OPENAI_API_KEY） +xmake +xmake run hello_mcpp +xmake run basic +xmake run chat ``` -## 在构建工具中使用 - -### xmake +## 包管理使用 ```lua --- 0 - 添加 mcpplibs 索引仓库 -add_repositories("mcpplibs-index https://github.com/mcpplibs/llmapi.git") - --- 1 - 添加需要的库和版本 -add_requires("llmapi 0.0.2") +add_repositories("mcpplibs-index https://github.com/mcpplibs/mcpplibs-index.git") +add_requires("llmapi 0.1.0") + +target("demo") + set_kind("binary") + set_languages("c++23") + set_policy("build.c++.modules", true) + add_files("src/*.cpp") + add_packages("llmapi") ``` -> More: [mcpplibs-index](https://github.com/mcpplibs/mcpplibs-index) - -### cmake - -``` -todo... -``` +更多内容见 [docs/zh/getting-started.md](docs/zh/getting-started.md)、[docs/zh/providers.md](docs/zh/providers.md) 与 [docs/zh/README.md](docs/zh/README.md)。 -## 📄 许可证 +## 许可证 -Apache-2.0 - 详见 [LICENSE](LICENSE) +Apache-2.0，详见 [LICENSE](LICENSE) diff --git a/docs/README.md b/docs/README.md index 71ec80d..ceeecd5 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,24 +1,7 @@ -# llmapi Documentation +# Documentation -Current documentation for the `llmapi` C++23 module library. +Select a language: -## Contents - -- [Getting Started](getting-started.md) - install, build, and first request -- [C++ API Guide](cpp-api.md) - types, providers, and `Client

` -- [Examples](examples.md) - chat, streaming, embeddings, and tool flows -- [Providers](providers.md) - OpenAI, Anthropic, and compatible endpoints -- [Advanced Usage](advanced.md) - persistence, async calls, and custom configuration - -## What The Library Provides - -- C++23 modules via `import mcpplibs.llmapi` -- Typed chat messages and multimodal content structs -- Provider concepts for sync, async, streaming, and embeddings -- Built-in OpenAI and Anthropic providers -- OpenAI-compatible endpoint support through configurable base URLs -- Conversation save/load helpers for local session persistence - -## License - -Apache-2.0 - see [LICENSE](../LICENSE) +- [English](en/README.md) +- [简体中文](zh/README.md) +- [繁體中文](zh-hant/README.md) diff --git a/docs/en/README.md b/docs/en/README.md new file mode 100644 index 0000000..f014e96 --- /dev/null +++ b/docs/en/README.md @@ -0,0 +1,47 @@ +# llmapi Documentation + +Current documentation for the `llmapi` C++23 module library. + +Language: + +- English +- [简体中文](../zh/README.md) +- [繁體中文](../zh-hant/README.md) + +## Contents + +- [Getting Started](getting-started.md) - install, build, and first request +- [C++ API Guide](cpp-api.md) - types, providers, and `Client

` +- [Examples](examples.md) - chat, streaming, embeddings, and tool flows +- [Providers](providers.md) - OpenAI, Anthropic, and compatible endpoints +- [Advanced Usage](advanced.md) - persistence, async calls, and custom configuration + +## What The Library Provides + +- C++23 modules via `import mcpplibs.llmapi` +- Typed chat messages and multimodal content structs +- Provider concepts for sync, async, streaming, and embeddings +- Built-in OpenAI and Anthropic providers +- OpenAI-compatible endpoint support through configurable base URLs +- Conversation save/load helpers for local session persistence + +## Production Readiness + +The library is usable for internal tools, prototypes, and early production adoption, but it should not yet be described as fully industrial-grade infrastructure. + +Required gaps before that claim: + +- Unified error model across providers and transport +- Retry, backoff, timeout, and idempotency policy +- Request cancellation for long-running and streaming calls +- Logging, metrics, trace hooks, and request correlation +- Hardening of the custom HTTP/TLS transport layer +- Fault-injection, concurrency, and large-scale mock testing +- Stronger API compatibility and versioning guarantees +- More complete production configuration surface +- Explicit thread-safety and concurrency semantics +- Operational documentation for retries, keys, proxies, and failure handling + +## License + +Apache-2.0 - see [LICENSE](../../LICENSE) diff --git a/docs/en/advanced.md b/docs/en/advanced.md new file mode 100644 index 0000000..88ed033 --- /dev/null +++ b/docs/en/advanced.md @@ -0,0 +1,133 @@ +# Advanced Usage + +Advanced patterns using the current API surface. + +## Conversation Persistence + +Each `chat()` and `chat_stream()` call appends the user message and the assistant response to the in-memory `Conversation`. + +```cpp +auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", +}); + +client.system("You are helpful."); +client.chat("Remember that I prefer concise answers."); +client.save_conversation("session.json"); + +auto restored = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", +}); +restored.load_conversation("session.json"); +``` + +## Manual Message Control + +You can seed few-shot history or inject tool results directly: + +```cpp +client.clear(); +client.add_message(Message::system("Translate English to Chinese.")); +client.add_message(Message::user("hello")); +client.add_message(Message::assistant("你好")); +auto resp = client.chat("goodbye"); +``` + +## Streaming + +Streaming returns a full `ChatResponse` after the callback finishes, so you can combine live output with post-processing. + +```cpp +std::string collected; +auto resp = client.chat_stream("Tell a story about templates.", [&](std::string_view chunk) { + collected += chunk; + std::cout << chunk; +}); + +std::cout << "\nstop reason=" << static_cast(resp.stopReason) << '\n'; +``` + +## Async API + +```cpp +auto task = client.chat_async("Explain coroutines briefly."); +auto resp = task.get(); +std::cout << resp.text() << '\n'; +``` + +## Concurrency Model + +Use instance isolation for concurrency: + +- `Client` is stateful and not thread-safe +- `tinyhttps::HttpClient` is also not thread-safe +- create one `Client` per task or per thread +- do not share a single `Client` across concurrent callers + +This works well for calling multiple providers in parallel because each client owns its own provider, conversation, and transport state. + +```cpp +auto futureA = std::async(std::launch::async, [&] { + auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", + }); + return client.chat("summarize this"); +}); + +auto futureB = std::async(std::launch::async, [&] { + auto client = Client(AnthropicConfig{ + .apiKey = std::getenv("ANTHROPIC_API_KEY"), + .model = "claude-sonnet-4-20250514", + }); + return client.chat("translate this"); +}); +``` + +## Tool Calling Loop + +The provider surfaces requested tools via `ChatResponse::tool_calls()`. You then append a tool result and continue the conversation. + +```cpp +auto params = ChatParams{ + .tools = std::vector{{ + .name = "get_weather", + .description = "Return weather for a city", + .inputSchema = R"({"type":"object","properties":{"city":{"type":"string"}},"required":["city"]})", + }}, + .toolChoice = ToolChoice::Auto, +}; + +auto first = client.chat("What's the weather in Tokyo?", params); +for (const auto& call : first.tool_calls()) { + client.add_message(Message{ + .role = Role::Tool, + .content = std::vector{ + ToolResultContent{ + .toolUseId = call.id, + .content = R"({"temperature":"22C","condition":"sunny"})", + }, + }, + }); +} + +auto final = client.provider().chat(client.conversation().messages, params); +``` + +## Compatible Endpoints + +```cpp +auto provider = openai::OpenAI({ + .apiKey = std::getenv("DEEPSEEK_API_KEY"), + .baseUrl = std::string(URL::DeepSeek), + .model = "deepseek-chat", +}); +``` + +## See Also + +- [C++ API Reference](cpp-api.md) +- [Examples](examples.md) +- [Providers](providers.md) diff --git a/docs/en/cpp-api.md b/docs/en/cpp-api.md new file mode 100644 index 0000000..4719c89 --- /dev/null +++ b/docs/en/cpp-api.md @@ -0,0 +1,278 @@ +# C++ API Reference + +Reference for the current public module interface. + +## Namespace + +```cpp +import mcpplibs.llmapi; +using namespace mcpplibs::llmapi; +``` + +## Exported Modules + +- `mcpplibs.llmapi` +- `mcpplibs.llmapi:types` +- `mcpplibs.llmapi:url` +- `mcpplibs.llmapi:coro` +- `mcpplibs.llmapi:provider` +- `mcpplibs.llmapi:client` +- `mcpplibs.llmapi:openai` +- `mcpplibs.llmapi:anthropic` +- `mcpplibs.llmapi:errors` + +## Core Types + +Important exported structs and enums: + +- `Role` +- `Message` +- `TextContent`, `ImageContent`, `AudioContent` +- `ToolDef`, `ToolCall`, `ToolUseContent`, `ToolResultContent` +- `ChatParams` +- `ChatResponse` +- `EmbeddingResponse` +- `Conversation` +- `Usage` +- `ResponseFormat` + +## Provider Concepts + +```cpp +template +concept Provider = requires(P p, const std::vector& messages, const ChatParams& params) { + { p.name() } -> std::convertible_to; + { p.chat(messages, params) } -> std::same_as; + { p.chat_async(messages, params) } -> std::same_as>; +}; +``` + +```cpp +template +concept StreamableProvider = Provider

&& requires( + P p, + const std::vector& messages, + const ChatParams& params, + std::function cb +) { + { p.chat_stream(messages, params, cb) } -> std::same_as; + { p.chat_stream_async(messages, params, cb) } -> std::same_as>; +}; +``` + +```cpp +template +concept EmbeddableProvider = Provider

&& requires( + P p, + const std::vector& inputs, + std::string_view model +) { + { p.embed(inputs, model) } -> std::same_as; +}; +``` + +## `Client

` + +`Client` is a class template that owns a provider instance and a `Conversation`. + +```cpp +template +class Client; +``` + +### Construction + +```cpp +auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", +}); +``` + +`Config` is an exported alias for `openai::Config`, so the default path uses an OpenAI-style provider without explicitly writing `openai::OpenAI`. + +### Configuration + +```cpp +Client& default_params(ChatParams params) +``` + +Stores default parameters used by `chat()`, `chat_async()`, and `chat_stream()`. + +### Conversation Management + +```cpp +Client& system(std::string_view content) +Client& user(std::string_view content) +Client& add_message(Message msg) +Client& clear() +``` + +### Synchronous Chat + +```cpp +ChatResponse chat(std::string_view userMessage) +ChatResponse chat(std::string_view userMessage, ChatParams params) +``` + +`chat()` appends the user message, sends the full conversation, stores the assistant text response, and returns the parsed `ChatResponse`. + +### Async Chat + +```cpp +Task chat_async(std::string_view userMessage) +``` + +### Streaming Chat + +```cpp +ChatResponse chat_stream( + std::string_view userMessage, + std::function callback +) +``` + +Available only when `P` satisfies `StreamableProvider`. + +```cpp +Task chat_stream_async( + std::string_view userMessage, + std::function callback +) +``` + +### Embeddings + +```cpp +EmbeddingResponse embed(const std::vector& inputs, std::string_view model) +``` + +Available only when `P` satisfies `EmbeddableProvider`. + +### Accessors + +```cpp +const Conversation& conversation() const +Conversation& conversation() +void save_conversation(std::string_view filePath) const +void load_conversation(std::string_view filePath) +const P& provider() const +P& provider() +``` + +### Thread-Safety + +- `Client

` is stateful and not thread-safe +- use one client per task or thread +- do not share one client across concurrent callers + +## Provider Config Types + +```cpp +openai::Config { + std::string apiKey; + std::string baseUrl { "https://api.openai.com/v1" }; + std::string model; + std::string organization; + std::optional proxy; + std::map customHeaders; +} +``` + +```cpp +anthropic::Config { + std::string apiKey; + std::string baseUrl { "https://api.anthropic.com/v1" }; + std::string model; + std::string version { "2023-06-01" }; + int defaultMaxTokens { 4096 }; + std::optional proxy; + std::map customHeaders; +} +``` + +## `ChatParams` + +```cpp +struct ChatParams { + std::optional temperature; + std::optional topP; + std::optional maxTokens; + std::optional> stop; + std::optional> tools; + std::optional toolChoice; + std::optional responseFormat; + std::optional extraJson; +}; +``` + +## `ChatResponse` + +```cpp +struct ChatResponse { + std::string id; + std::string model; + std::vector content; + StopReason stopReason; + Usage usage; + + std::string text() const; + std::vector tool_calls() const; +}; +``` + +## `Conversation` + +```cpp +struct Conversation { + std::vector messages; + + void push(Message msg); + void clear(); + int size() const; + void save(std::string_view filePath) const; + static Conversation load(std::string_view filePath); +}; +``` + +## Complete Example + +```cpp +import mcpplibs.llmapi; +import std; + +int main() { + using namespace mcpplibs::llmapi; + + auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", + }); + + client.default_params(ChatParams{ + .temperature = 0.2, + }); + + client.system("Be concise."); + + auto resp1 = client.chat("What is C++?"); + std::cout << resp1.text() << '\n'; + + auto resp2 = client.chat_stream("Give me one example of a C++23 feature.", [](std::string_view chunk) { + std::cout << chunk; + }); + std::cout << "\nmessages=" << client.conversation().size() << '\n'; + + return 0; +} +``` + +## Error Handling + +```cpp +try { + auto resp = client.chat("Hello"); +} catch (const std::runtime_error& e) { + std::cerr << "Error: " << e.what() << '\n'; +} +``` diff --git a/docs/en/examples.md b/docs/en/examples.md new file mode 100644 index 0000000..7959272 --- /dev/null +++ b/docs/en/examples.md @@ -0,0 +1,277 @@ +# Examples + +Practical examples using the current `Client` API. + +## Minimal Chat + +```cpp +import mcpplibs.llmapi; +import std; + +int main() { + using namespace mcpplibs::llmapi; + + auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", + }); + + client.system("You are a helpful assistant."); + auto resp = client.chat("In one sentence, explain C++23 modules."); + std::cout << resp.text() << '\n'; +} +``` + +## Streaming Response + +```cpp +import mcpplibs.llmapi; +import std; + +int main() { + using namespace mcpplibs::llmapi; + + auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", + }); + + std::string streamed; + client.chat_stream("Write a 3-line poem about templates.", [&](std::string_view chunk) { + streamed += chunk; + std::cout << chunk; + }); + std::cout << "\n\nCollected " << streamed.size() << " bytes\n"; +} +``` + +## Multi-Turn Conversation + +```cpp +import mcpplibs.llmapi; +import std; + +int main() { + using namespace mcpplibs::llmapi; + + auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", + }); + + client.system("Reply briefly."); + + auto resp1 = client.chat("What is the capital of France?"); + auto resp2 = client.chat("What is its population roughly?"); + + std::cout << resp1.text() << '\n'; + std::cout << resp2.text() << '\n'; + std::cout << "Messages stored: " << client.conversation().size() << '\n'; +} +``` + +## Save And Load Conversation + +```cpp +import mcpplibs.llmapi; +import std; + +int main() { + using namespace mcpplibs::llmapi; + + auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", + }); + + client.chat("Remember that my favorite language is C++."); + client.save_conversation("conversation.json"); + + auto restored = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", + }); + restored.load_conversation("conversation.json"); + + auto resp = restored.chat("What language do I like?"); + std::cout << resp.text() << '\n'; +} +``` + +## Tool Calling + +```cpp +import mcpplibs.llmapi; +import std; + +int main() { + using namespace mcpplibs::llmapi; + + auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", + }); + + auto params = ChatParams{ + .tools = std::vector{{ + .name = "get_temperature", + .description = "Get the temperature for a city", + .inputSchema = R"({"type":"object","properties":{"city":{"type":"string"}},"required":["city"]})", + }}, + .toolChoice = ToolChoice::Auto, + }; + + auto resp = client.chat("What's the temperature in Tokyo?", params); + for (const auto& call : resp.tool_calls()) { + std::cout << call.name << ": " << call.arguments << '\n'; + } +} +``` + +## Embeddings + +```cpp +import mcpplibs.llmapi; +import std; + +int main() { + using namespace mcpplibs::llmapi; + + auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", + }); + + auto embedding = client.embed( + {"hello world", "modern c++"}, + "text-embedding-3-small" + ); + + std::cout << "vectors: " << embedding.embeddings.size() << '\n'; + std::cout << "dimension: " << embedding.embeddings[0].size() << '\n'; +} +``` + +## Exception Mode + +The library currently reports failures by throwing exceptions. This is the recommended direct style when you want errors to propagate naturally. + +```cpp +import mcpplibs.llmapi; +import std; + +int main() { + using namespace mcpplibs::llmapi; + + try { + auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", + }); + + auto resp = client.chat("Explain RAII in one paragraph."); + std::cout << resp.text() << '\n'; + } catch (const ApiError& e) { + std::cerr << "API error: status=" << e.statusCode << " body=" << e.body << '\n'; + return 2; + } catch (const ConnectionError& e) { + std::cerr << "Connection error: " << e.what() << '\n'; + return 3; + } catch (const std::exception& e) { + std::cerr << "Unexpected error: " << e.what() << '\n'; + return 4; + } +} +``` + +## No-Exception Style At Call Site + +If your application prefers not to let exceptions escape, wrap the call and convert the result to `std::optional`, `std::expected`, or your own result type. + +```cpp +import mcpplibs.llmapi; +import std; + +std::optional safe_chat(std::string_view prompt) { + using namespace mcpplibs::llmapi; + + try { + auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", + }); + return client.chat(prompt).text(); + } catch (...) { + return std::nullopt; + } +} +``` + +## Recommended Retry At The Application Layer + +Retry policy is currently best implemented by the library user because retryability depends on business semantics. + +```cpp +import mcpplibs.llmapi; +import std; + +std::string chat_with_retry(std::string_view prompt) { + using namespace mcpplibs::llmapi; + + for (int attempt = 0; attempt < 3; ++attempt) { + try { + auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", + }); + return client.chat(prompt).text(); + } catch (const ConnectionError&) { + } catch (const ApiError& e) { + if (e.statusCode != 429 && (e.statusCode < 500 || e.statusCode >= 600)) { + throw; + } + } + + std::this_thread::sleep_for(std::chrono::milliseconds(200 * (1 << attempt))); + } + + throw std::runtime_error("retry limit exceeded"); +} +``` + +## Parallel Use With Isolated Clients + +The recommended concurrency model is one client per task or thread. + +```cpp +import mcpplibs.llmapi; +import std; + +int main() { + using namespace mcpplibs::llmapi; + + auto futureA = std::async(std::launch::async, [] { + auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", + }); + return client.chat("Summarize modules.").text(); + }); + + auto futureB = std::async(std::launch::async, [] { + auto client = Client(AnthropicConfig{ + .apiKey = std::getenv("ANTHROPIC_API_KEY"), + .model = "claude-sonnet-4-20250514", + }); + return client.chat("Translate 'hello world' to Japanese.").text(); + }); + + std::cout << futureA.get() << '\n'; + std::cout << futureB.get() << '\n'; +} +``` + +## See Also + +- [C++ API Reference](cpp-api.md) +- [Providers Configuration](providers.md) diff --git a/docs/en/getting-started.md b/docs/en/getting-started.md new file mode 100644 index 0000000..27332d6 --- /dev/null +++ b/docs/en/getting-started.md @@ -0,0 +1,120 @@ +# Getting Started + +## Prerequisites + +- **C++ Compiler**: GCC 14+, Clang 18+, or MSVC 2022+ with C++23 support +- **Build System**: [xmake](https://xmake.io/) 3.0.0+ +- **Dependencies**: `mbedtls` is resolved automatically by xmake + +## Installation + +### Using xmake Package Manager + +Add to your `xmake.lua`: + +```lua +add_repositories("mcpplibs-index git@github.com:mcpplibs/mcpplibs-index.git") +add_requires("llmapi 0.1.0") + +target("myapp") + set_kind("binary") + set_languages("c++23") + set_policy("build.c++.modules", true) + add_files("src/*.cpp") + add_packages("llmapi") +``` + +### Building from Source + +```bash +git clone https://github.com/mcpplibs/llmapi.git +cd llmapi + +xmake +xmake run hello_mcpp +xmake run basic +xmake run chat +``` + +## First Example + +Create `main.cpp`: + +```cpp +import mcpplibs.llmapi; +import std; + +int main() { + using namespace mcpplibs::llmapi; + + auto apiKey = std::getenv("OPENAI_API_KEY"); + if (!apiKey) { + std::cerr << "OPENAI_API_KEY not set\n"; + return 1; + } + + auto client = Client(Config{ + .apiKey = apiKey, + .model = "gpt-4o-mini", + }); + + client.system("You are a helpful assistant."); + auto resp = client.chat("Hello, introduce yourself in one sentence."); + + std::cout << resp.text() << '\n'; + return 0; +} +``` + +Build and run: + +```bash +xmake +xmake run hello_mcpp +``` + +## Environment Setup + +Set the provider-specific API key you plan to use: + +```bash +export OPENAI_API_KEY="sk-..." +export ANTHROPIC_API_KEY="sk-ant-..." +export DEEPSEEK_API_KEY="..." +``` + +## Switching Providers + +OpenAI: + +```cpp +auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", +}); +``` + +Anthropic: + +```cpp +auto client = Client(anthropic::Anthropic({ + .apiKey = std::getenv("ANTHROPIC_API_KEY"), + .model = "claude-sonnet-4-20250514", +})); +``` + +Compatible endpoint through the OpenAI provider: + +```cpp +auto client = Client(Config{ + .apiKey = std::getenv("DEEPSEEK_API_KEY"), + .baseUrl = std::string(URL::DeepSeek), + .model = "deepseek-chat", +}); +``` + +## Next Steps + +- [C++ API Guide](cpp-api.md) - Learn the full C++ API +- [Examples](examples.md) - See more examples +- [Providers](providers.md) - Configure different providers diff --git a/docs/en/providers.md b/docs/en/providers.md new file mode 100644 index 0000000..c9e8352 --- /dev/null +++ b/docs/en/providers.md @@ -0,0 +1,141 @@ +# Providers Configuration + +How to configure the providers that exist in the current codebase. + +## URL Constants + +`mcpplibs::llmapi::URL` exposes common base URLs: + +```cpp +import mcpplibs.llmapi; +using namespace mcpplibs::llmapi; + +URL::OpenAI +URL::Anthropic +URL::DeepSeek +URL::OpenRouter +URL::Poe +``` + +## OpenAI + +Use `openai::OpenAI` for OpenAI chat, streaming, tool calls, and embeddings. + +```cpp +auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", +}); +``` + +- Get keys from [OpenAI Platform](https://platform.openai.com/api-keys) +- Set `OPENAI_API_KEY` + +## Anthropic + +Use `anthropic::Anthropic` for Anthropic chat and streaming. + +```cpp +auto client = Client(anthropic::Anthropic({ + .apiKey = std::getenv("ANTHROPIC_API_KEY"), + .model = "claude-sonnet-4-20250514", +})); +``` + +- Get keys from [Anthropic Console](https://console.anthropic.com/) +- Set `ANTHROPIC_API_KEY` + +## OpenAI-Compatible Endpoints + +The OpenAI provider accepts a custom `baseUrl`, so DeepSeek, OpenRouter, Poe, local gateways, and self-hosted OpenAI-compatible services can all reuse `openai::OpenAI`. + +```cpp +auto client = Client(openai::OpenAI({ + .apiKey = std::getenv("DEEPSEEK_API_KEY"), + .baseUrl = std::string(URL::DeepSeek), + .model = "deepseek-chat", +})); +``` + +You can also use a literal endpoint: + +```cpp +auto client = Client(openai::OpenAI({ + .apiKey = std::getenv("OPENROUTER_API_KEY"), + .baseUrl = "https://openrouter.ai/api/v1", + .model = "openai/gpt-4o-mini", +})); +``` + +## Custom Headers And Proxy + +```cpp +auto provider = openai::OpenAI({ + .apiKey = std::getenv("OPENAI_API_KEY"), + .baseUrl = std::string(URL::OpenAI), + .model = "gpt-4o-mini", + .proxy = "http://127.0.0.1:7890", + .customHeaders = { + {"X-Trace-Id", "demo-request"}, + }, +}); +``` + +## Environment Variables + +Typical setup: + +```bash +export OPENAI_API_KEY="sk-..." +export ANTHROPIC_API_KEY="sk-ant-..." +export DEEPSEEK_API_KEY="..." +export OPENROUTER_API_KEY="..." +``` + +## Provider Capabilities + +| Provider | Chat | Streaming | Embeddings | Notes | +|----------|------|-----------|------------|-------| +| `openai::OpenAI` | yes | yes | yes | Also works with compatible endpoints | +| `anthropic::Anthropic` | yes | yes | no | Anthropic Messages API | + +## Troubleshooting + +### Connection Issues + +```cpp +try { + auto resp = client.chat("test"); +} catch (const std::runtime_error& e) { + std::cerr << "Network error: " << e.what() << '\n'; +} +``` + +### Authentication Errors + +```cpp +auto key = std::getenv("OPENAI_API_KEY"); +if (!key || std::string(key).empty()) { + std::cerr << "OPENAI_API_KEY not set\n"; +} +``` + +### Model Not Found + +```cpp +auto openaiClient = Client(openai::OpenAI({ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", +})); + +auto anthropicClient = Client(anthropic::Anthropic({ + .apiKey = std::getenv("ANTHROPIC_API_KEY"), + .model = "claude-sonnet-4-20250514", +})); +``` + +## See Also + +- [Getting Started](getting-started.md) +- [C++ API Reference](cpp-api.md) +- [Examples](examples.md) diff --git a/docs/getting-started.md b/docs/getting-started.md index 098c4c8..27332d6 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -14,7 +14,7 @@ Add to your `xmake.lua`: ```lua add_repositories("mcpplibs-index git@github.com:mcpplibs/mcpplibs-index.git") -add_requires("llmapi 0.0.2") +add_requires("llmapi 0.1.0") target("myapp") set_kind("binary") diff --git a/docs/zh-hant/README.md b/docs/zh-hant/README.md new file mode 100644 index 0000000..4d68715 --- /dev/null +++ b/docs/zh-hant/README.md @@ -0,0 +1,43 @@ +# llmapi 中文文件 + +`llmapi` 是一個基於 C++23 模組的 LLM 客戶端函式庫。 + +語言： + +- [English](../en/README.md) +- [简体中文](../zh/README.md) +- 繁體中文 + +## 內容導覽 + +- [快速開始](getting-started.md) +- [C++ API 參考](cpp-api.md) +- [範例](examples.md) +- [Provider 設定](providers.md) +- [進階用法](advanced.md) + +## 核心能力 + +- `import mcpplibs.llmapi` +- 型別化訊息、工具與回應結構 +- Provider 概念約束 +- 內建 OpenAI / Anthropic 支援 +- OpenAI 相容端點支援 +- 對話儲存與還原 + +## 生產可用性 + +這個函式庫已可用於內部工具、原型專案與早期生產試用，但還不應被定義為「完整工業級基礎設施」。 + +要達到那個標準，至少還需要補齊： + +- 統一的 provider / 傳輸層錯誤模型 +- 重試、退避、逾時、冪等策略 +- 長請求與串流請求的取消能力 +- 日誌、指標、trace hook、請求關聯資訊 +- 自研 HTTP/TLS 傳輸層的進一步加固 +- 故障注入、並發、Mock、大規模測試 +- 更強的 API 相容性與版本穩定性承諾 +- 更完整的生產設定面 +- 明確的執行緒安全與並發語義 +- 面向維運的重試、金鑰、代理、故障處理文件 diff --git a/docs/zh-hant/advanced.md b/docs/zh-hant/advanced.md new file mode 100644 index 0000000..251f8f6 --- /dev/null +++ b/docs/zh-hant/advanced.md @@ -0,0 +1,51 @@ +# 進階用法 + +## 對話持久化 + +```cpp +client.chat("記住我偏好簡潔回答。"); +client.save_conversation("session.json"); +``` + +## 手動注入訊息 + +```cpp +client.add_message(Message::system("Translate English to Chinese.")); +client.add_message(Message::user("hello")); +client.add_message(Message::assistant("你好")); +``` + +## 非同步介面 + +```cpp +auto task = client.chat_async("簡要解釋 coroutine。"); +auto resp = task.get(); +``` + +## 並發模型 + +建議使用「實例隔離、上層並發」的方式： + +- `Client` 是有狀態物件，不保證執行緒安全 +- `tinyhttps::HttpClient` 也不保證執行緒安全 +- 每個任務或執行緒各自建立一個 `Client` +- 不要把同一個 `Client` 共享給多個並發呼叫方 + +這種方式天然適合多模型 / 多 provider 並發呼叫，因為每個實例都持有獨立的 provider、對話與傳輸狀態。 + +## 工具呼叫流程 + +```cpp +auto first = client.chat("東京天氣如何？", params); +for (const auto& call : first.tool_calls()) { + client.add_message(Message{ + .role = Role::Tool, + .content = std::vector{ + ToolResultContent{ + .toolUseId = call.id, + .content = R"({"temperature":"22C","condition":"sunny"})", + }, + }, + }); +} +``` diff --git a/docs/zh-hant/cpp-api.md b/docs/zh-hant/cpp-api.md new file mode 100644 index 0000000..be17c79 --- /dev/null +++ b/docs/zh-hant/cpp-api.md @@ -0,0 +1,80 @@ +# C++ API 參考 + +## 匯出模組 + +- `mcpplibs.llmapi` +- `mcpplibs.llmapi:types` +- `mcpplibs.llmapi:url` +- `mcpplibs.llmapi:coro` +- `mcpplibs.llmapi:provider` +- `mcpplibs.llmapi:client` +- `mcpplibs.llmapi:openai` +- `mcpplibs.llmapi:anthropic` + +## 關鍵別名 + +- `Config` -> `openai::Config` +- `AnthropicConfig` -> `anthropic::Config` +- `OpenAI` -> `openai::OpenAI` +- `Anthropic` -> `anthropic::Anthropic` + +## Client 入口 + +```cpp +auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", +}); +``` + +```cpp +auto client = Client(AnthropicConfig{ + .apiKey = std::getenv("ANTHROPIC_API_KEY"), + .model = "claude-sonnet-4-20250514", +}); +``` + +## 常用方法 + +```cpp +client.system(...) +client.user(...) +client.add_message(...) +client.clear() + +client.chat(...) +client.chat_async(...) +client.chat_stream(...) +client.chat_stream_async(...) + +client.embed(...) +client.save_conversation(...) +client.load_conversation(...) +client.conversation() +client.provider() +``` + +## ChatParams + +`ChatParams` 支援： + +- `temperature` +- `topP` +- `maxTokens` +- `stop` +- `tools` +- `toolChoice` +- `responseFormat` +- `extraJson` + +## ChatResponse + +常用欄位與輔助方法： + +- `id` +- `model` +- `content` +- `stopReason` +- `usage` +- `text()` +- `tool_calls()` diff --git a/docs/zh-hant/examples.md b/docs/zh-hant/examples.md new file mode 100644 index 0000000..a582fd5 --- /dev/null +++ b/docs/zh-hant/examples.md @@ -0,0 +1,138 @@ +# 範例 + +## 最小聊天 + +```cpp +auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", +}); + +client.system("You are a helpful assistant."); +auto resp = client.chat("請用一句話解釋 C++23 模組。"); +std::cout << resp.text() << '\n'; +``` + +## 串流輸出 + +```cpp +std::string collected; +client.chat_stream("寫一首關於模板的三行短詩。", [&](std::string_view chunk) { + collected += chunk; + std::cout << chunk; +}); +``` + +## 多輪對話 + +```cpp +auto resp1 = client.chat("法國首都是哪裡？"); +auto resp2 = client.chat("它大概有多少人口？"); +``` + +## 儲存與還原對話 + +```cpp +client.chat("記住我最喜歡的語言是 C++。"); +client.save_conversation("conversation.json"); + +auto restored = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", +}); +restored.load_conversation("conversation.json"); +``` + +## 工具呼叫 + +```cpp +auto params = ChatParams{ + .tools = std::vector{{ + .name = "get_temperature", + .description = "Get the temperature for a city", + .inputSchema = R"({"type":"object","properties":{"city":{"type":"string"}},"required":["city"]})", + }}, + .toolChoice = ToolChoice::Auto, +}; + +auto resp = client.chat("東京現在幾度？", params); +``` + +## Embeddings + +```cpp +auto embedding = client.embed( + {"hello world", "modern c++"}, + "text-embedding-3-small" +); +``` + +## 例外模式 + +目前函式庫以拋出例外作為失敗回報方式。對於希望讓錯誤自然向上傳遞的呼叫鏈，這是最直接的使用方式。 + +```cpp +try { + auto resp = client.chat("請用一段話解釋 RAII。"); + std::cout << resp.text() << '\n'; +} catch (const ApiError& e) { + std::cerr << "API error: status=" << e.statusCode << " body=" << e.body << '\n'; +} catch (const ConnectionError& e) { + std::cerr << "Connection error: " << e.what() << '\n'; +} +``` + +## 無例外模式 + +如果上層不希望讓例外繼續外拋，可以在呼叫點自行包一層，轉成 `optional`、`expected` 或業務自己的結果型別。 + +```cpp +std::optional safe_chat(std::string_view prompt) { + try { + return client.chat(prompt).text(); + } catch (...) { + return std::nullopt; + } +} +``` + +## 建議的上層重試 + +目前更建議由函式庫使用者在上層實作重試，因為是否可重試取決於業務語義。 + +```cpp +for (int attempt = 0; attempt < 3; ++attempt) { + try { + return client.chat(prompt).text(); + } catch (const ConnectionError&) { + } catch (const ApiError& e) { + if (e.statusCode != 429 && (e.statusCode < 500 || e.statusCode >= 600)) { + throw; + } + } + + std::this_thread::sleep_for(std::chrono::milliseconds(200 * (1 << attempt))); +} +``` + +## 並發使用 + +建議模式是「實例隔離、上層並發」，也就是每個任務 / 執行緒各自建立一個 `Client`。 + +```cpp +auto futureA = std::async(std::launch::async, [] { + auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", + }); + return client.chat("總結一下 modules。").text(); +}); + +auto futureB = std::async(std::launch::async, [] { + auto client = Client(AnthropicConfig{ + .apiKey = std::getenv("ANTHROPIC_API_KEY"), + .model = "claude-sonnet-4-20250514", + }); + return client.chat("把 hello world 翻譯成日語。").text(); +}); +``` diff --git a/docs/zh-hant/getting-started.md b/docs/zh-hant/getting-started.md new file mode 100644 index 0000000..13c0e7e --- /dev/null +++ b/docs/zh-hant/getting-started.md @@ -0,0 +1,81 @@ +# 快速開始 + +## 環境需求 + +- 編譯器：GCC 14+、Clang 18+ 或 MSVC 2022+ +- 建置工具：`xmake` +- 依賴：`mbedtls` 由 xmake 自動處理 + +## 使用 xmake 引入 + +```lua +add_repositories("mcpplibs-index git@github.com:mcpplibs/mcpplibs-index.git") +add_requires("llmapi 0.1.0") + +target("myapp") + set_kind("binary") + set_languages("c++23") + set_policy("build.c++.modules", true) + add_files("src/*.cpp") + add_packages("llmapi") +``` + +## 從原始碼建置 + +```bash +git clone https://github.com/mcpplibs/llmapi.git +cd llmapi + +xmake +xmake run hello_mcpp +``` + +## 第一個範例 + +```cpp +import mcpplibs.llmapi; +import std; + +int main() { + using namespace mcpplibs::llmapi; + + auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", + }); + + client.system("You are a helpful assistant."); + auto resp = client.chat("請用一句話介紹現代 C++。"); + std::cout << resp.text() << '\n'; +} +``` + +## 切換 Provider + +OpenAI： + +```cpp +auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", +}); +``` + +Anthropic： + +```cpp +auto client = Client(AnthropicConfig{ + .apiKey = std::getenv("ANTHROPIC_API_KEY"), + .model = "claude-sonnet-4-20250514", +}); +``` + +相容端點： + +```cpp +auto client = Client(Config{ + .apiKey = std::getenv("DEEPSEEK_API_KEY"), + .baseUrl = std::string(URL::DeepSeek), + .model = "deepseek-chat", +}); +``` diff --git a/docs/zh-hant/providers.md b/docs/zh-hant/providers.md new file mode 100644 index 0000000..30a8d2b --- /dev/null +++ b/docs/zh-hant/providers.md @@ -0,0 +1,46 @@ +# Provider 設定 + +## URL 常數 + +```cpp +URL::OpenAI +URL::Anthropic +URL::DeepSeek +URL::OpenRouter +URL::Poe +``` + +## OpenAI + +```cpp +auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", +}); +``` + +## Anthropic + +```cpp +auto client = Client(AnthropicConfig{ + .apiKey = std::getenv("ANTHROPIC_API_KEY"), + .model = "claude-sonnet-4-20250514", +}); +``` + +## OpenAI 相容端點 + +```cpp +auto client = Client(Config{ + .apiKey = std::getenv("DEEPSEEK_API_KEY"), + .baseUrl = std::string(URL::DeepSeek), + .model = "deepseek-chat", +}); +``` + +## 能力對照 + +| Provider | 聊天 | 串流 | Embeddings | +|----------|------|------|------------| +| `openai::OpenAI` | 是 | 是 | 是 | +| `anthropic::Anthropic` | 是 | 是 | 否 | diff --git a/docs/zh/README.md b/docs/zh/README.md new file mode 100644 index 0000000..54baa70 --- /dev/null +++ b/docs/zh/README.md @@ -0,0 +1,43 @@ +# llmapi 中文文档 + +`llmapi` 是一个基于 C++23 模块的 LLM 客户端库。 + +语言： + +- [English](../en/README.md) +- 简体中文 +- [繁體中文](../zh-hant/README.md) + +## 内容导航 + +- [快速开始](getting-started.md) +- [C++ API 参考](cpp-api.md) +- [示例](examples.md) +- [Provider 配置](providers.md) +- [高级用法](advanced.md) + +## 核心能力 + +- `import mcpplibs.llmapi` +- 类型化消息、工具和响应结构 +- Provider 概念约束 +- OpenAI / Anthropic 内置支持 +- OpenAI 兼容端点支持 +- 对话保存与恢复 + +## 生产可用性 + +这个库已经可以用于内部工具、原型项目和早期生产试用，但还不应被定义为“完整工业级基础设施”。 + +要达到那个标准，至少还需要补齐： + +- 统一的 provider / 传输层错误模型 +- 重试、退避、超时、幂等策略 +- 长请求和流式请求的取消能力 +- 日志、指标、trace hook、请求关联信息 +- 自研 HTTP/TLS 传输层的进一步加固 +- 故障注入、并发、Mock、大规模测试 +- 更强的 API 兼容性与版本稳定性承诺 +- 更完整的生产配置面 +- 明确的线程安全和并发语义 +- 面向运维的重试、密钥、代理、故障处理文档 diff --git a/docs/zh/advanced.md b/docs/zh/advanced.md new file mode 100644 index 0000000..b686f02 --- /dev/null +++ b/docs/zh/advanced.md @@ -0,0 +1,51 @@ +# 高级用法 + +## 对话持久化 + +```cpp +client.chat("记住我偏好简洁回答。"); +client.save_conversation("session.json"); +``` + +## 手动注入消息 + +```cpp +client.add_message(Message::system("Translate English to Chinese.")); +client.add_message(Message::user("hello")); +client.add_message(Message::assistant("你好")); +``` + +## 异步接口 + +```cpp +auto task = client.chat_async("简要解释 coroutine。"); +auto resp = task.get(); +``` + +## 并发模型 + +推荐使用“实例隔离，上层并发”的方式： + +- `Client` 是有状态对象，不保证线程安全 +- `tinyhttps::HttpClient` 也不保证线程安全 +- 每个任务或线程单独创建一个 `Client` +- 不要把同一个 `Client` 共享给多个并发调用方 + +这种方式天然适合多个模型 / 多个 provider 并发调用，因为每个实例都持有独立的 provider、会话和传输状态。 + +## 工具调用循环 + +```cpp +auto first = client.chat("东京天气如何？", params); +for (const auto& call : first.tool_calls()) { + client.add_message(Message{ + .role = Role::Tool, + .content = std::vector{ + ToolResultContent{ + .toolUseId = call.id, + .content = R"({"temperature":"22C","condition":"sunny"})", + }, + }, + }); +} +``` diff --git a/docs/zh/cpp-api.md b/docs/zh/cpp-api.md new file mode 100644 index 0000000..74665d7 --- /dev/null +++ b/docs/zh/cpp-api.md @@ -0,0 +1,80 @@ +# C++ API 参考 + +## 导出模块 + +- `mcpplibs.llmapi` +- `mcpplibs.llmapi:types` +- `mcpplibs.llmapi:url` +- `mcpplibs.llmapi:coro` +- `mcpplibs.llmapi:provider` +- `mcpplibs.llmapi:client` +- `mcpplibs.llmapi:openai` +- `mcpplibs.llmapi:anthropic` + +## 关键别名 + +- `Config` -> `openai::Config` +- `AnthropicConfig` -> `anthropic::Config` +- `OpenAI` -> `openai::OpenAI` +- `Anthropic` -> `anthropic::Anthropic` + +## Client 入口 + +```cpp +auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", +}); +``` + +```cpp +auto client = Client(AnthropicConfig{ + .apiKey = std::getenv("ANTHROPIC_API_KEY"), + .model = "claude-sonnet-4-20250514", +}); +``` + +## 常用方法 + +```cpp +client.system(...) +client.user(...) +client.add_message(...) +client.clear() + +client.chat(...) +client.chat_async(...) +client.chat_stream(...) +client.chat_stream_async(...) + +client.embed(...) +client.save_conversation(...) +client.load_conversation(...) +client.conversation() +client.provider() +``` + +## ChatParams + +`ChatParams` 支持： + +- `temperature` +- `topP` +- `maxTokens` +- `stop` +- `tools` +- `toolChoice` +- `responseFormat` +- `extraJson` + +## ChatResponse + +常用字段与辅助方法： + +- `id` +- `model` +- `content` +- `stopReason` +- `usage` +- `text()` +- `tool_calls()` diff --git a/docs/zh/examples.md b/docs/zh/examples.md new file mode 100644 index 0000000..2ff2dff --- /dev/null +++ b/docs/zh/examples.md @@ -0,0 +1,138 @@ +# 示例 + +## 最小聊天 + +```cpp +auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", +}); + +client.system("You are a helpful assistant."); +auto resp = client.chat("请用一句话解释 C++23 模块。"); +std::cout << resp.text() << '\n'; +``` + +## 流式输出 + +```cpp +std::string collected; +client.chat_stream("写一首关于模板的三行短诗。", [&](std::string_view chunk) { + collected += chunk; + std::cout << chunk; +}); +``` + +## 多轮对话 + +```cpp +auto resp1 = client.chat("法国首都是哪里？"); +auto resp2 = client.chat("它大概有多少人口？"); +``` + +## 保存与恢复对话 + +```cpp +client.chat("记住我最喜欢的语言是 C++。"); +client.save_conversation("conversation.json"); + +auto restored = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", +}); +restored.load_conversation("conversation.json"); +``` + +## 工具调用 + +```cpp +auto params = ChatParams{ + .tools = std::vector{{ + .name = "get_temperature", + .description = "Get the temperature for a city", + .inputSchema = R"({"type":"object","properties":{"city":{"type":"string"}},"required":["city"]})", + }}, + .toolChoice = ToolChoice::Auto, +}; + +auto resp = client.chat("东京现在多少度？", params); +``` + +## Embeddings + +```cpp +auto embedding = client.embed( + {"hello world", "modern c++"}, + "text-embedding-3-small" +); +``` + +## 异常模式 + +当前库以抛异常作为失败报告方式。对于希望让错误直接上传的调用链，这是最直接的使用方式。 + +```cpp +try { + auto resp = client.chat("请用一段话解释 RAII。"); + std::cout << resp.text() << '\n'; +} catch (const ApiError& e) { + std::cerr << "API error: status=" << e.statusCode << " body=" << e.body << '\n'; +} catch (const ConnectionError& e) { + std::cerr << "Connection error: " << e.what() << '\n'; +} +``` + +## 无异常模式 + +如果你的上层不希望异常继续外抛，可以在调用点自行包一层，转换成 `optional`、`expected` 或业务自己的结果类型。 + +```cpp +std::optional safe_chat(std::string_view prompt) { + try { + return client.chat(prompt).text(); + } catch (...) { + return std::nullopt; + } +} +``` + +## 推荐的上层重试 + +当前更推荐由库使用者在上层实现重试，因为是否可重试取决于业务语义。 + +```cpp +for (int attempt = 0; attempt < 3; ++attempt) { + try { + return client.chat(prompt).text(); + } catch (const ConnectionError&) { + } catch (const ApiError& e) { + if (e.statusCode != 429 && (e.statusCode < 500 || e.statusCode >= 600)) { + throw; + } + } + + std::this_thread::sleep_for(std::chrono::milliseconds(200 * (1 << attempt))); +} +``` + +## 并发使用 + +推荐模式是“实例隔离，上层并发”，也就是每个任务 / 线程各自创建一个 `Client`。 + +```cpp +auto futureA = std::async(std::launch::async, [] { + auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", + }); + return client.chat("总结一下 modules。").text(); +}); + +auto futureB = std::async(std::launch::async, [] { + auto client = Client(AnthropicConfig{ + .apiKey = std::getenv("ANTHROPIC_API_KEY"), + .model = "claude-sonnet-4-20250514", + }); + return client.chat("把 hello world 翻译成日语。").text(); +}); +``` diff --git a/docs/zh/getting-started.md b/docs/zh/getting-started.md new file mode 100644 index 0000000..afce254 --- /dev/null +++ b/docs/zh/getting-started.md @@ -0,0 +1,81 @@ +# 快速开始 + +## 环境要求 + +- 编译器：GCC 14+、Clang 18+ 或 MSVC 2022+ +- 构建工具：`xmake` +- 依赖：`mbedtls` 由 xmake 自动处理 + +## 使用 xmake 引入 + +```lua +add_repositories("mcpplibs-index git@github.com:mcpplibs/mcpplibs-index.git") +add_requires("llmapi 0.1.0") + +target("myapp") + set_kind("binary") + set_languages("c++23") + set_policy("build.c++.modules", true) + add_files("src/*.cpp") + add_packages("llmapi") +``` + +## 从源码构建 + +```bash +git clone https://github.com/mcpplibs/llmapi.git +cd llmapi + +xmake +xmake run hello_mcpp +``` + +## 第一个例子 + +```cpp +import mcpplibs.llmapi; +import std; + +int main() { + using namespace mcpplibs::llmapi; + + auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", + }); + + client.system("You are a helpful assistant."); + auto resp = client.chat("请用一句话介绍现代 C++。"); + std::cout << resp.text() << '\n'; +} +``` + +## 切换 Provider + +OpenAI： + +```cpp +auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", +}); +``` + +Anthropic： + +```cpp +auto client = Client(AnthropicConfig{ + .apiKey = std::getenv("ANTHROPIC_API_KEY"), + .model = "claude-sonnet-4-20250514", +}); +``` + +兼容端点： + +```cpp +auto client = Client(Config{ + .apiKey = std::getenv("DEEPSEEK_API_KEY"), + .baseUrl = std::string(URL::DeepSeek), + .model = "deepseek-chat", +}); +``` diff --git a/docs/zh/providers.md b/docs/zh/providers.md new file mode 100644 index 0000000..259703b --- /dev/null +++ b/docs/zh/providers.md @@ -0,0 +1,46 @@ +# Provider 配置 + +## URL 常量 + +```cpp +URL::OpenAI +URL::Anthropic +URL::DeepSeek +URL::OpenRouter +URL::Poe +``` + +## OpenAI + +```cpp +auto client = Client(Config{ + .apiKey = std::getenv("OPENAI_API_KEY"), + .model = "gpt-4o-mini", +}); +``` + +## Anthropic + +```cpp +auto client = Client(AnthropicConfig{ + .apiKey = std::getenv("ANTHROPIC_API_KEY"), + .model = "claude-sonnet-4-20250514", +}); +``` + +## OpenAI 兼容端点 + +```cpp +auto client = Client(Config{ + .apiKey = std::getenv("DEEPSEEK_API_KEY"), + .baseUrl = std::string(URL::DeepSeek), + .model = "deepseek-chat", +}); +``` + +## 能力对比 + +| Provider | 聊天 | 流式 | Embeddings | +|----------|------|------|------------| +| `openai::OpenAI` | 是 | 是 | 是 | +| `anthropic::Anthropic` | 是 | 是 | 否 | diff --git a/src/client.cppm b/src/client.cppm index 02f5f38..1eb38e7 100644 --- a/src/client.cppm +++ b/src/client.cppm @@ -17,6 +17,8 @@ private: ChatParams defaultParams_; public: + // Thread-safety: Client instances are intentionally stateful and not synchronized. + // Use one Client per task/thread and avoid sharing a Client across threads. explicit Client(P provider) : provider_(std::move(provider)) {} explicit Client(openai::Config config) requires std::same_as diff --git a/src/llmapi.cppm b/src/llmapi.cppm index a8766e4..6c1d7bc 100644 --- a/src/llmapi.cppm +++ b/src/llmapi.cppm @@ -14,6 +14,7 @@ import std; import mcpplibs.llmapi.nlohmann.json; namespace mcpplibs::llmapi { + export inline constexpr std::string_view VERSION { "0.1.0" }; export using OpenAI = openai::OpenAI; export using Config = openai::Config; export using Anthropic = anthropic::Anthropic; diff --git a/src/tinyhttps/http.cppm b/src/tinyhttps/http.cppm index d6d39c7..0fc167f 100644 --- a/src/tinyhttps/http.cppm +++ b/src/tinyhttps/http.cppm @@ -219,6 +219,8 @@ static bool iequals(std::string_view a, std::string_view b) { export class HttpClient { public: + // Thread-safety: HttpClient owns a mutable connection pool and is not synchronized. + // Keep each instance isolated to a single caller/task unless you add external locking. explicit HttpClient(HttpClientConfig config = {}) : config_(std::move(config)) {} diff --git a/tests/llmapi/test_client.cpp b/tests/llmapi/test_client.cpp index e148252..003edb7 100644 --- a/tests/llmapi/test_client.cpp +++ b/tests/llmapi/test_client.cpp @@ -7,9 +7,15 @@ import std; using namespace mcpplibs::llmapi; struct FullMockProvider { + std::string prefix { "reply to: " }; + int delayMs { 0 }; + std::string_view name() const { return "full_mock"; } ChatResponse chat(const std::vector& msgs, const ChatParams&) { + if (delayMs > 0) { + std::this_thread::sleep_for(std::chrono::milliseconds(delayMs)); + } std::string lastContent; if (!msgs.empty()) { auto& c = msgs.back().content; @@ -18,7 +24,7 @@ struct FullMockProvider { } } return ChatResponse { - .content = { TextContent { "reply to: " + lastContent } }, + .content = { TextContent { prefix + lastContent } }, .stopReason = StopReason::EndOfTurn, .usage = { .inputTokens = 10, .outputTokens = 5, .totalTokens = 15 }, }; @@ -96,6 +102,40 @@ int main() { assert(client2.conversation().size() == 2); std::filesystem::remove("/tmp/test_client_conv.json"); + // Test 8: isolated clients can be used concurrently without sharing conversation state + auto futureA = std::async(std::launch::async, [] { + auto isolatedClient = Client(FullMockProvider{ + .prefix = "openai-like: ", + .delayMs = 10, + }); + isolatedClient.system("provider a"); + auto resp = isolatedClient.chat("hello from a"); + return std::pair{ + resp.text(), + isolatedClient.conversation().size(), + }; + }); + + auto futureB = std::async(std::launch::async, [] { + auto isolatedClient = Client(FullMockProvider{ + .prefix = "anthropic-like: ", + .delayMs = 10, + }); + isolatedClient.system("provider b"); + auto resp = isolatedClient.chat("hello from b"); + return std::pair{ + resp.text(), + isolatedClient.conversation().size(), + }; + }); + + auto [textA, sizeA] = futureA.get(); + auto [textB, sizeB] = futureB.get(); + assert(textA == "openai-like: hello from a"); + assert(textB == "anthropic-like: hello from b"); + assert(sizeA == 3); + assert(sizeB == 3); + println("test_client: ALL PASSED"); return 0; } diff --git a/xmake.lua b/xmake.lua index 92ac765..1cc3e86 100644 --- a/xmake.lua +++ b/xmake.lua @@ -1,4 +1,5 @@ set_languages("c++23") +set_version("0.1.0") set_policy("build.c++.modules", true) add_requires("mbedtls 3.6.1") From ed186d0ed4deef8719b438d334431b02824f478c Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Tue, 10 Mar 2026 20:21:47 +0800 Subject: [PATCH 2/2] Move production planning out of README --- README.md | 19 +------------------ README.zh.hant.md | 19 +------------------ README.zh.md | 19 +------------------ docs/en/README.md | 18 +----------------- docs/en/roadmap.md | 16 ++++++++++++++++ docs/zh-hant/README.md | 18 +----------------- docs/zh-hant/roadmap.md | 16 ++++++++++++++++ docs/zh/README.md | 18 +----------------- docs/zh/roadmap.md | 16 ++++++++++++++++ 9 files changed, 54 insertions(+), 105 deletions(-) create mode 100644 docs/en/roadmap.md create mode 100644 docs/zh-hant/roadmap.md create mode 100644 docs/zh/roadmap.md diff --git a/README.md b/README.md index 4e0e1d6..207d3eb 100644 --- a/README.md +++ b/README.md @@ -22,23 +22,6 @@ - Conversation save/load helpers - OpenAI-compatible endpoint support through `openai::Config::baseUrl` -## Production Readiness - -`llmapi` is usable for internal tools, prototypes, and early production experiments, but it should not yet be treated as fully industrial-grade infrastructure. - -Required gaps before that bar: - -- Unified error model across providers and transport -- Retry, backoff, timeout, and idempotency policy -- Request cancellation for long-running and streaming calls -- Logging, metrics, trace hooks, and request correlation -- Hardening of the custom HTTP/TLS transport layer -- Fault-injection, concurrency, and large-scale mock testing -- Stronger API compatibility and versioning guarantees -- More complete production configuration surface -- Explicit thread-safety and concurrency semantics -- Operational documentation for retries, keys, proxies, and failure handling - ## Quick Start ```cpp @@ -106,7 +89,7 @@ target("demo") add_packages("llmapi") ``` -See [docs/en/getting-started.md](docs/en/getting-started.md), [docs/en/providers.md](docs/en/providers.md), and [docs/en/README.md](docs/en/README.md) for more setup and readiness detail. +See [docs/en/getting-started.md](docs/en/getting-started.md), [docs/en/providers.md](docs/en/providers.md), [docs/en/roadmap.md](docs/en/roadmap.md), and [docs/en/README.md](docs/en/README.md) for more setup and planning detail. ## License diff --git a/README.zh.hant.md b/README.zh.hant.md index d4d3f39..078c8a1 100644 --- a/README.zh.hant.md +++ b/README.zh.hant.md @@ -22,23 +22,6 @@ - 支援儲存 / 載入對話歷史 - 可透過 `baseUrl` 存取 OpenAI 相容端點 -## 生產可用性 - -`llmapi` 目前適合內部工具、原型專案與早期生產試用，但還不應直接視為完整工業級基礎設施。 - -要達到那個標準，至少還需要補齊： - -- 統一的 provider / 傳輸層錯誤模型 -- 重試、退避、逾時、冪等策略 -- 長請求與串流請求的取消能力 -- 日誌、指標、trace hook、請求關聯資訊 -- 自研 HTTP/TLS 傳輸層的進一步加固 -- 故障注入、並發、Mock、大規模測試 -- 更強的 API 相容性與版本穩定性承諾 -- 更完整的生產設定面 -- 明確的執行緒安全與並發語義 -- 面向維運的重試、金鑰、代理、故障處理文件 - ## 快速開始 ```cpp @@ -106,7 +89,7 @@ target("demo") add_packages("llmapi") ``` -更多內容見 [docs/zh-hant/getting-started.md](docs/zh-hant/getting-started.md)、[docs/zh-hant/providers.md](docs/zh-hant/providers.md) 與 [docs/zh-hant/README.md](docs/zh-hant/README.md)。 +更多內容見 [docs/zh-hant/getting-started.md](docs/zh-hant/getting-started.md)、[docs/zh-hant/providers.md](docs/zh-hant/providers.md)、[docs/zh-hant/roadmap.md](docs/zh-hant/roadmap.md) 與 [docs/zh-hant/README.md](docs/zh-hant/README.md)。 ## 授權 diff --git a/README.zh.md b/README.zh.md index d0df23b..d8c44b9 100644 --- a/README.zh.md +++ b/README.zh.md @@ -22,23 +22,6 @@ - 支持保存 / 加载对话历史 - 可通过 `baseUrl` 访问 OpenAI 兼容端点 -## 生产可用性 - -`llmapi` 目前适合内部工具、原型项目和早期生产试用，但还不应直接视为完整工业级基础设施。 - -要达到那个标准，至少还需要补齐： - -- 统一的 provider / 传输层错误模型 -- 重试、退避、超时、幂等策略 -- 长请求和流式请求的取消能力 -- 日志、指标、trace hook、请求关联信息 -- 自研 HTTP/TLS 传输层的进一步加固 -- 故障注入、并发、Mock、大规模测试 -- 更强的 API 兼容性与版本稳定性承诺 -- 更完整的生产配置面 -- 明确的线程安全和并发语义 -- 面向运维的重试、密钥、代理、故障处理文档 - ## 快速开始 ```cpp @@ -106,7 +89,7 @@ target("demo") add_packages("llmapi") ``` -更多内容见 [docs/zh/getting-started.md](docs/zh/getting-started.md)、[docs/zh/providers.md](docs/zh/providers.md) 与 [docs/zh/README.md](docs/zh/README.md)。 +更多内容见 [docs/zh/getting-started.md](docs/zh/getting-started.md)、[docs/zh/providers.md](docs/zh/providers.md)、[docs/zh/roadmap.md](docs/zh/roadmap.md) 与 [docs/zh/README.md](docs/zh/README.md)。 ## 许可证 diff --git a/docs/en/README.md b/docs/en/README.md index f014e96..e0707ce 100644 --- a/docs/en/README.md +++ b/docs/en/README.md @@ -15,6 +15,7 @@ Language: - [Examples](examples.md) - chat, streaming, embeddings, and tool flows - [Providers](providers.md) - OpenAI, Anthropic, and compatible endpoints - [Advanced Usage](advanced.md) - persistence, async calls, and custom configuration +- [Roadmap](roadmap.md) - future production and infrastructure improvements ## What The Library Provides @@ -25,23 +26,6 @@ Language: - OpenAI-compatible endpoint support through configurable base URLs - Conversation save/load helpers for local session persistence -## Production Readiness - -The library is usable for internal tools, prototypes, and early production adoption, but it should not yet be described as fully industrial-grade infrastructure. - -Required gaps before that claim: - -- Unified error model across providers and transport -- Retry, backoff, timeout, and idempotency policy -- Request cancellation for long-running and streaming calls -- Logging, metrics, trace hooks, and request correlation -- Hardening of the custom HTTP/TLS transport layer -- Fault-injection, concurrency, and large-scale mock testing -- Stronger API compatibility and versioning guarantees -- More complete production configuration surface -- Explicit thread-safety and concurrency semantics -- Operational documentation for retries, keys, proxies, and failure handling - ## License Apache-2.0 - see [LICENSE](../../LICENSE) diff --git a/docs/en/roadmap.md b/docs/en/roadmap.md new file mode 100644 index 0000000..ff8a3c7 --- /dev/null +++ b/docs/en/roadmap.md @@ -0,0 +1,16 @@ +# Roadmap + +`llmapi` is already usable for internal tools, prototypes, and early production experiments. The items below describe the main gaps before it should be treated as fully industrial-grade infrastructure. + +## Production And Infrastructure Priorities + +- Unified error model across providers and transport +- Retry, backoff, timeout, and idempotency policy +- Request cancellation for long-running and streaming calls +- Logging, metrics, trace hooks, and request correlation +- Hardening of the custom HTTP/TLS transport layer +- Fault-injection, concurrency, and large-scale mock testing +- Stronger API compatibility and versioning guarantees +- More complete production configuration surface +- Explicit thread-safety and concurrency semantics +- Operational documentation for retries, keys, proxies, and failure handling diff --git a/docs/zh-hant/README.md b/docs/zh-hant/README.md index 4d68715..2d3acb4 100644 --- a/docs/zh-hant/README.md +++ b/docs/zh-hant/README.md @@ -15,6 +15,7 @@ - [範例](examples.md) - [Provider 設定](providers.md) - [進階用法](advanced.md) +- [未來規劃](roadmap.md) ## 核心能力 @@ -24,20 +25,3 @@ - 內建 OpenAI / Anthropic 支援 - OpenAI 相容端點支援 - 對話儲存與還原 - -## 生產可用性 - -這個函式庫已可用於內部工具、原型專案與早期生產試用，但還不應被定義為「完整工業級基礎設施」。 - -要達到那個標準，至少還需要補齊： - -- 統一的 provider / 傳輸層錯誤模型 -- 重試、退避、逾時、冪等策略 -- 長請求與串流請求的取消能力 -- 日誌、指標、trace hook、請求關聯資訊 -- 自研 HTTP/TLS 傳輸層的進一步加固 -- 故障注入、並發、Mock、大規模測試 -- 更強的 API 相容性與版本穩定性承諾 -- 更完整的生產設定面 -- 明確的執行緒安全與並發語義 -- 面向維運的重試、金鑰、代理、故障處理文件 diff --git a/docs/zh-hant/roadmap.md b/docs/zh-hant/roadmap.md new file mode 100644 index 0000000..1ac0797 --- /dev/null +++ b/docs/zh-hant/roadmap.md @@ -0,0 +1,16 @@ +# 未來規劃 + +`llmapi` 已可用於內部工具、原型專案與早期生產試用。下面這些事項，是它邁向完整工業級基礎設施之前最主要的差距。 + +## 生產與基礎設施優先項 + +- 統一的 provider / 傳輸層錯誤模型 +- 重試、退避、逾時、冪等策略 +- 長請求與串流請求的取消能力 +- 日誌、指標、trace hook、請求關聯資訊 +- 自研 HTTP/TLS 傳輸層的進一步加固 +- 故障注入、並發、Mock、大規模測試 +- 更強的 API 相容性與版本穩定性承諾 +- 更完整的生產設定面 +- 明確的執行緒安全與並發語義 +- 面向維運的重試、金鑰、代理、故障處理文件 diff --git a/docs/zh/README.md b/docs/zh/README.md index 54baa70..8e53b7f 100644 --- a/docs/zh/README.md +++ b/docs/zh/README.md @@ -15,6 +15,7 @@ - [示例](examples.md) - [Provider 配置](providers.md) - [高级用法](advanced.md) +- [未来规划](roadmap.md) ## 核心能力 @@ -24,20 +25,3 @@ - OpenAI / Anthropic 内置支持 - OpenAI 兼容端点支持 - 对话保存与恢复 - -## 生产可用性 - -这个库已经可以用于内部工具、原型项目和早期生产试用，但还不应被定义为“完整工业级基础设施”。 - -要达到那个标准，至少还需要补齐： - -- 统一的 provider / 传输层错误模型 -- 重试、退避、超时、幂等策略 -- 长请求和流式请求的取消能力 -- 日志、指标、trace hook、请求关联信息 -- 自研 HTTP/TLS 传输层的进一步加固 -- 故障注入、并发、Mock、大规模测试 -- 更强的 API 兼容性与版本稳定性承诺 -- 更完整的生产配置面 -- 明确的线程安全和并发语义 -- 面向运维的重试、密钥、代理、故障处理文档 diff --git a/docs/zh/roadmap.md b/docs/zh/roadmap.md new file mode 100644 index 0000000..d69373c --- /dev/null +++ b/docs/zh/roadmap.md @@ -0,0 +1,16 @@ +# 未来规划 + +`llmapi` 已经可以用于内部工具、原型项目和早期生产试用。下面这些事项，是它迈向完整工业级基础设施之前最主要的差距。 + +## 生产与基础设施优先项 + +- 统一的 provider / 传输层错误模型 +- 重试、退避、超时、幂等策略 +- 长请求和流式请求的取消能力 +- 日志、指标、trace hook、请求关联信息 +- 自研 HTTP/TLS 传输层的进一步加固 +- 故障注入、并发、Mock、大规模测试 +- 更强的 API 兼容性与版本稳定性承诺 +- 更完整的生产配置面 +- 明确的线程安全和并发语义 +- 面向运维的重试、密钥、代理、故障处理文档