diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 0000000..602f68f --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,42 @@ +--- +name: Bug 报告 +about: 报告一个 bug 帮助我们改进 +title: '[BUG] ' +labels: bug +assignees: '' +--- + +## 描述 + +简要描述 bug。 + +## 复现步骤 + +1. 执行 '...' +2. 输入 '...' +3. 看到 '...' + +## 期望行为 + +描述你期望的正确行为。 + +## 实际行为 + +描述实际发生的情况。 + +## 环境信息 + +- OS: [e.g., Windows 11, macOS 14, Ubuntu 22.04] +- Python 版本: [e.g., 3.12.0] +- jojo-code 版本: [e.g., 0.2.0] +- 安装方式: [e.g., pip, uv, source] + +## 错误日志 + +``` +粘贴相关的错误日志 +``` + +## 补充信息 + +其他可能有用的信息（截图、配置文件等）。 diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 0000000..bb89fcd --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,27 @@ +--- +name: 功能请求 +about: 建议一个新功能 +title: '[FEATURE] ' +labels: enhancement +assignees: '' +--- + +## 描述 + +简要描述你想要的功能。 + +## 使用场景 + +解释为什么需要这个功能，以及你会如何使用它。 + +## 建议实现 + +如果你有想法，描述可能的实现方式。 + +## 替代方案 + +描述你目前使用的替代方案（如果有）。 + +## 补充信息 + +其他上下文信息。 diff --git a/.github/ISSUE_TEMPLATE/question.md b/.github/ISSUE_TEMPLATE/question.md new file mode 100644 index 0000000..c04e01f --- /dev/null +++ b/.github/ISSUE_TEMPLATE/question.md @@ -0,0 +1,19 @@ +--- +name: 问题咨询 +about: 使用中遇到的问题 +title: '[QUESTION] ' +labels: question +assignees: '' +--- + +## 问题描述 + +描述你的问题。 + +## 已尝试的解决方案 + +描述你已经尝试过的方法。 + +## 相关文档 + +如果有相关的文档或链接，请提供。 diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 0000000..d5540a7 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,33 @@ +## 描述 + +简要描述此 PR 的变更内容。 + +## 变更类型 + +- [ ] Bug 修复 +- [ ] 新功能 +- [ ] 重构 +- [ ] 文档更新 +- [ ] 测试 +- [ ] 其他 + +## 相关 Issue + +Closes #(issue 编号) + +## 测试 + +- [ ] 已运行 `uv run ruff check src/ tests/` +- [ ] 已运行 `uv run ruff format src/ tests/` +- [ ] 已运行 `uv run pytest tests/ -v` +- [ ] 已添加新功能的测试 + +## 截图（如适用） + +## 检查清单 + +- [ ] 代码遵循项目风格指南 +- [ ] 已添加必要的文档 +- [ ] 已添加必要的测试 +- [ ] 所有测试通过 +- [ ] 已更新 CHANGELOG.md（如适用） diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..e028e8d --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,64 @@ +# Changelog + +All notable changes to this project will be documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), +and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [Unreleased] + +### Added +- Dynamic system prompt generation from ToolRegistry +- Tool call retry mechanism with exponential backoff +- Tool result size limiting (50KB max) to prevent context overflow +- Configurable max_iterations in AgentState +- grep_search context_lines parameter for showing surrounding lines +- read_file start_line/end_line parameters for line range reading +- Enhanced health check endpoint with CPU, memory, uptime info +- CLI --version flag +- LLM call error handling in thinking_node +- CONTRIBUTING.md contribution guide +- GitHub Issue templates (bug report, feature request, question) +- PR template +- CHANGELOG.md +- examples/README.md +- Comprehensive test coverage for security modules (ssrf, rule, denial) +- Comprehensive test coverage for memory modules (short_term, long_term, retriever, types) +- Comprehensive test coverage for MCP client +- Comprehensive test coverage for built-in plugins (code_review, test_generator, git) +- Comprehensive test coverage for core modules (database, monitoring, webhook) + +### Fixed +- datetime.utcnow() deprecation warning in audit.py +- PytestCollectionWarning for TestCaseEvaluator in evaluator.py +- Indentation bug in denial.py AdaptivePermissionMixin.check_with_denial_tracking +- UnboundLocalError in long_term.py (redundant local Path import) +- MockDatabaseBackend.fetch_one id filtering logic +- AlertManager.check using rule.condition instead of non-existent rule.evaluate +- Webhook.trigger returning results for successful local handlers + +### Changed +- System prompt is now dynamically generated from available tools +- Tool execution now supports automatic retry on transient failures + +## [0.1.0] - 2026-05-29 + +### Added +- Initial release +- LangGraph-based agent loop with thinking/execute nodes +- 40+ built-in tools (file, shell, git, search, web, HTTP, code analysis, performance, data, docs, system) +- Textual TUI with chat, header, footer, input area, status bar widgets +- WebSocket (JSON-RPC 2.0) + SSE streaming server +- REST API endpoints for agents, conversations, metrics +- Plugin system with 3 built-in plugins (code_review, git, test_generator) +- Security: permission management, audit logging, SSRF protection, command/path guards +- Memory: conversation memory with token counting and compression +- Short-term and long-term memory with retrieval +- AgentOps: metrics collection, evaluation, dashboard, reporting +- Model abstraction: OpenAI, Anthropic Claude, OpenAI-compatible APIs +- MCP client support +- Skills system +- Task execution framework +- Session persistence +- Docker support (experimental) +- CLI commands: TUI, server management, config, plugin management, setup wizard diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..57a12df --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,224 @@ +# 贡献指南 + +感谢你对 jojo-code 项目的关注！本文档将帮助你了解如何参与贡献。 + +## 开发环境搭建 + +### 前置要求 + +- Python 3.11+ +- [uv](https://docs.astral.sh/uv/) 包管理器 +- Git + +### 搭建步骤 + +```bash +# 1. Fork 并克隆仓库 +git clone https://github.com/YOUR_USERNAME/jojo-code.git +cd jojo-code + +# 2. 安装依赖 +uv sync + +# 3. 运行测试确认环境正常 +uv run pytest tests/ -v + +# 4. 配置环境变量 +cp .env.example .env +# 编辑 .env 填入你的 API Key +``` + +## 开发流程 + +### 1. 创建分支 + +```bash +git checkout -b feature/your-feature-name +# 或 +git checkout -b fix/your-bug-fix +``` + +### 2. 编写代码 + +- 遵循项目代码风格（见下方） +- 为新功能添加测试 +- 确保所有测试通过 + +### 3. 提交代码 + +```bash +# 运行 lint 检查 +uv run ruff check src/ tests/ +uv run ruff format src/ tests/ + +# 运行测试 +uv run pytest tests/ -v + +# 提交 +git add -A +git commit -m "feat: 添加新功能 XXX" +``` + +### 4. 创建 Pull Request + +- 标题清晰描述变更内容 +- 描述中说明变更原因和实现方式 +- 关联相关 Issue + +## 代码风格 + +### Python 规范 + +- 行长度：100 字符 +- 使用 type hints +- 函数和类必须有 docstring +- 使用 `ruff` 进行 lint 和格式化 + +### 提交信息规范 + +使用 [Conventional Commits](https://www.conventionalcommits.org/) 格式： + +``` +(): + +[optional body] + +[optional footer] +``` + +类型（type）： +- `feat`: 新功能 +- `fix`: 修复 bug +- `docs`: 文档更新 +- `style`: 代码格式调整（不影响功能） +- `refactor`: 重构 +- `test`: 添加或修改测试 +- `chore`: 构建、工具等杂项 + +示例： +``` +feat(tools): 添加 YAML 解析工具 +fix(auth): 修复 token 过期未刷新的问题 +docs(readme): 更新安装说明 +test(security): 补充 SSRF 防护测试 +``` + +## 添加新工具 + +1. 在 `src/jojo_code/tools/` 创建工具文件 +2. 使用 `@tool` 装饰器定义工具 +3. 在 `registry.py` 的 `_register_default_tools()` 中注册 +4. 添加对应的测试文件 `tests/test_tools/test_xxx_tools.py` + +示例： + +```python +from langchain_core.tools import tool + +@tool +def my_new_tool(param1: str, param2: int = 10) -> str: + """工具描述（会显示给 LLM）。 + + Args: + param1: 参数1描述 + param2: 参数2描述 + + Returns: + 返回值描述 + """ + # 实现逻辑 + return f"结果: {param1}" +``` + +## 添加新插件 + +1. 在 `src/jojo_code/plugins/` 创建插件文件 +2. 继承 `BasePlugin` 类 +3. 实现 `get_tools()` 方法 +4. 添加测试 + +示例： + +```python +from jojo_code.plugin.base import BasePlugin, PluginMetadata, PluginPermission + +class MyPlugin(BasePlugin): + metadata = PluginMetadata( + name="my-plugin", + version="0.1.0", + description="我的插件", + ) + permission = PluginPermission.RESTRICTED + + def get_tools(self): + return [my_tool_1, my_tool_2] +``` + +## 测试指南 + +### 运行测试 + +```bash +# 运行所有测试 +uv run pytest tests/ -v + +# 运行特定模块测试 +uv run pytest tests/test_tools/ -v + +# 运行带覆盖率 +uv run pytest tests/ -v --cov=src/jojo_code --cov-report=html + +# 跳过 E2E 测试（需要真实 API） +uv run pytest tests/ -v --ignore=tests/test_e2e +``` + +### 编写测试 + +- 使用 `pytest` 和 `pytest-asyncio` +- 测试文件命名：`test_xxx.py` +- 测试类命名：`TestXxx` +- 测试方法命名：`test_xxx` +- 使用 `tmp_path` fixture 处理临时文件 +- Mock 外部依赖（API 调用、文件系统等） + +## Issue 规范 + +### Bug 报告 + +```markdown +**描述**: 简要描述 bug + +**复现步骤**: +1. 执行 '...' +2. 看到 '...' + +**期望行为**: 描述你期望的正确行为 + +**实际行为**: 描述实际发生的情况 + +**环境**: +- OS: [e.g., Windows 11] +- Python: [e.g., 3.12] +- jojo-code: [e.g., 0.2.0] +``` + +### 功能请求 + +```markdown +**描述**: 简要描述你想要的功能 + +**使用场景**: 解释为什么需要这个功能 + +**建议实现**: 如果有想法，描述可能的实现方式 +``` + +## 行为准则 + +- 尊重所有参与者 +- 接受建设性批评 +- 专注于对社区最有利的事情 +- 对他人表示同理心 + +## 问题？ + +如有疑问，请在 [GitHub Issues](https://github.com/YOUR_USERNAME/jojo-code/issues) 提问。 diff --git a/docs/docker.md b/docs/docker.md index 5bc8667..d6032c6 100644 --- a/docs/docker.md +++ b/docs/docker.md @@ -75,3 +75,113 @@ docker compose down # 或 docker stop jojo-code ``` + +## 多阶段构建 + +对于生产环境，推荐使用多阶段构建减小镜像体积： + +```dockerfile +# 构建阶段 +FROM python:3.11-slim AS builder +WORKDIR /app +COPY pyproject.toml . +RUN pip install --no-cache-dir uv && uv pip install --system --no-cache . + +# 运行阶段 +FROM python:3.11-slim +WORKDIR /app +COPY --from=builder /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages +COPY --from=builder /usr/local/bin /usr/local/bin +COPY src/ src/ +EXPOSE 8080 +CMD ["python", "-m", "jojo_code.server"] +``` + +## 常见问题排查 + +### 容器启动后立即退出 + +**症状**: `docker ps` 看不到容器，`docker ps -a` 显示 Exited。 + +**排查步骤**: +```bash +# 查看容器日志 +docker logs jojo-code + +# 检查退出码 +docker inspect jojo-code --format='{{.State.ExitCode}}' +``` + +**常见原因**: +- API Key 未设置或格式错误 +- 端口已被占用（检查 `lsof -i :8080`） +- 挂载目录权限不足 + +### 无法连接到 LLM API + +**症状**: 容器运行中但 Agent 无响应。 + +**排查步骤**: +```bash +# 进入容器检查网络 +docker exec -it jojo-code sh +curl -s https://api.openai.com/v1/models -H "Authorization: Bearer $OPENAI_API_KEY" + +# 检查 DNS 解析 +nslookup api.openai.com +``` + +**常见原因**: +- 代理未配置：设置 `HTTP_PROXY` / `HTTPS_PROXY` 环境变量 +- API Key 无效或已过期 +- 网络策略限制出站连接 + +### 挂载目录内容不可见 + +**症状**: Agent 看不到工作目录中的文件。 + +**排查步骤**: +```bash +# 验证挂载点 +docker inspect jojo-code --format='{{json .Mounts}}' | jq + +# 进入容器检查 +docker exec -it jojo-code ls -la /workspace +``` + +**常见原因**: +- 路径拼写错误（Windows 用户注意路径分隔符） +- 目录为空 +- SELinux 限制：添加 `:z` 后缀，如 `-v /path:/workspace:z` + +### 内存不足 (OOM) + +**症状**: 容器被 OOM Killer 终止。 + +**解决方案**: +```bash +# 增加内存限制 +docker run -m 2g --memory-swap 4g jojo-code + +# 或在 docker-compose.yml 中设置 +services: + jojo-code: + deploy: + resources: + limits: + memory: 2G +``` + +### 权限问题 + +**症状**: Agent 无法读写挂载的文件。 + +**解决方案**: +```bash +# 使用与宿主机相同的用户 ID 运行 +docker run --user $(id -u):$(id -g) jojo-code + +# 或在 Dockerfile 中创建匹配的用户 +RUN useradd -u 1000 -m jojo +USER jojo +``` diff --git a/docs/memory-system.md b/docs/memory-system.md new file mode 100644 index 0000000..5e6a127 --- /dev/null +++ b/docs/memory-system.md @@ -0,0 +1,331 @@ +# Memory System Guide + +> jojo-code provides a tiered memory system that manages conversation history, session context, and persistent knowledge across sessions. + +## Overview + +The memory system has three layers: + +- **ConversationMemory** - Message history with token counting and automatic compression +- **ShortTermMemory** - Current session memory with role-based message management +- **LongTermMemory** - Persistent storage across sessions with search and cleanup +- **MemoryRetriever** - Unified search interface across both memory tiers + +## Architecture + +``` +SessionMemory (unified API) +├── ShortTermMemory (current session) +│ ├── Messages (HumanMessage / AIMessage / SystemMessage) +│ ├── Token counting +│ └── Auto-compression +├── LongTermMemory (persistent) +│ ├── Per-session storage (~/.jojo-code/memory/) +│ ├── Keyword search +│ └── Automatic cleanup +└── MemoryRetriever (search) + ├── Current session search + ├── History search + └── Cross-session retrieval +``` + +## Quick Start + +### Using SessionMemory (Recommended) + +```python +from jojo_code.memory import SessionMemory + +# Create a session +memory = SessionMemory(session_id="my-session") + +# Add messages +memory.add_message("Hello, how can I help?", role="user") +memory.add_message("I can help you with coding tasks.", role="ai") +memory.add_message("You are a helpful assistant.", role="system") + +# Get context for the LLM +context = memory.get_context() +print(f"Messages: {len(context)}") + +# Get only recent messages +recent = memory.get_context(max_messages=5) + +# Search across all memory +results = memory.search("coding", scope="all") + +# Get statistics +stats = memory.get_stats() +print(f"Session: {stats['session_id']}") +print(f"Messages: {stats['current_messages']}") +print(f"Tokens: {stats['current_tokens']}") +print(f"History sessions: {stats['history_sessions']}") +``` + +### Using ShortTermMemory Directly + +```python +from jojo_code.memory import ShortTermMemory + +# Create short-term memory +stm = ShortTermMemory(session_id="session-001", max_tokens=50000) + +# Add messages with convenience methods +stm.add_user_message("What is Python?") +stm.add_ai_message("Python is a programming language.") +stm.add_system_message("You are a coding assistant.") + +# Get messages by role +user_msgs = stm.get_messages_by_role("user") +ai_msgs = stm.get_messages_by_role("ai") + +# Search messages +results = stm.search("Python") + +# Check token count +tokens = stm.token_count() +print(f"Current tokens: {tokens}") + +# Get recent messages +last_3 = stm.get_last_n(3) + +# Convert to memory items for long-term storage +items = stm.to_memory_items() +``` + +### Using LongTermMemory Directly + +```python +from jojo_code.memory import LongTermMemory + +# Create long-term memory (defaults to ~/.jojo-code/memory/) +ltm = LongTermMemory(storage_dir="./my_memory", max_items=5000) + +# Add memories +item = ltm.add( + content="Learned that Python 3.12 has improved error messages", + session_id="session-001", + tags=["python", "learning"], + metadata={"importance": "high"}, +) + +# Add message-style memories +ltm.add_message("The project uses pytest for testing", role="user", session_id="session-001") + +# Search across sessions +results = ltm.search("Python", limit=5) +for result in results: + print(f"Score: {result.score:.2f} - {result.matched_content}") + +# Get session memories +memories = ltm.get_session_memories("session-001", limit=10) + +# List all sessions +sessions = ltm.list_sessions() + +# Get statistics +stats = ltm.get_stats() +print(f"Total items: {stats['total_items']}") +print(f"Sessions: {stats['sessions']}") + +# Cleanup old memories (older than 30 days) +cleaned = ltm.cleanup(retention_days=30) +print(f"Cleaned {cleaned} expired items") + +# Delete a session +ltm.delete_session("old-session") +``` + +### Using MemoryRetriever + +```python +from jojo_code.memory import MemoryRetriever, ShortTermMemory, LongTermMemory + +# Create with custom memory instances +stm = ShortTermMemory() +ltm = LongTermMemory(storage_dir="./memory") +retriever = MemoryRetriever(short_term=stm, long_term=ltm) + +# Search all memory +results = retriever.search("Python", scope="all") +print(f"Current session matches: {len(results['current_session'])}") +print(f"History matches: {len(results['history'])}") + +# Search only current session +current_results = retriever.search_current_session("error") + +# Search only history +history_results = retriever.search_history("deployment", session_id="deploy-001") + +# Get recent memories across both tiers +recent = retriever.get_recent_memories(limit=10) + +# Save current session to long-term memory +retriever.save_current_session() + +# Load a historical session +history = retriever.load_session("session-001") + +# Get all session IDs +all_sessions = retriever.get_all_sessions() +``` + +## ConversationMemory (Legacy API) + +The `ConversationMemory` class provides a simpler API with automatic persistence: + +```python +from pathlib import Path +from jojo_code.memory import ConversationMemory +from langchain_core.messages import HumanMessage, AIMessage + +# Create with auto-save +memory = ConversationMemory( + max_tokens=100000, + storage_path=Path(".jojo-code/conversation.json"), + auto_save=True, +) + +# Add messages +memory.add_message(HumanMessage(content="Hello")) +memory.add_message(AIMessage(content="Hi there!")) + +# Check token count +print(f"Tokens: {memory.token_count()}") + +# Get context +context = memory.get_context() + +# Manual save/load +memory.save() +memory.load() + +# Clear memory +memory.clear() +``` + +## Memory Types + +### MemoryItem + +The fundamental unit of memory storage: + +```python +from jojo_code.memory import MemoryItem, MemoryType +from datetime import datetime + +item = MemoryItem( + id="unique-id", + content="The project uses FastAPI", + memory_type=MemoryType.LONG_TERM, + session_id="session-001", + created_at=datetime.now(), + metadata={"role": "user", "importance": "high"}, + tags=["tech-stack", "fastapi"], +) + +# Serialize +data = item.to_dict() + +# Deserialize +restored = MemoryItem.from_dict(data) +``` + +### SearchResult + +Search results include relevance scoring: + +```python +from jojo_code.memory import SearchResult + +# SearchResult contains: +# - item: MemoryItem - the matched memory +# - score: float - relevance score (0.0 to 1.0) +# - matched_content: str - snippet showing the match +``` + +## Automatic Compression + +When token count exceeds `max_tokens`, memory is automatically compressed: + +1. System messages are always preserved +2. Recent messages (default: 20) are kept +3. Older messages are replaced with a summary placeholder + +```python +from jojo_code.memory import ShortTermMemory + +stm = ShortTermMemory(max_tokens=1000) # Small limit for demo + +# Add many messages... +for i in range(100): + stm.add_user_message(f"Message {i}") + stm.add_ai_message(f"Response {i}") + +# Memory is automatically compressed when tokens exceed 1000 +print(f"Messages after compression: {stm.message_count}") +``` + +## Configuration + +### LongTermMemory Configuration + +```python +from jojo_code.memory import LongTermMemory + +ltm = LongTermMemory( + storage_dir="~/.jojo-code/memory", # Storage directory + max_items=10000, # Max items per session + retention_days=90, # Auto-cleanup after N days +) +``` + +### Storage Structure + +Long-term memory stores data in a directory structure: + +``` +~/.jojo-code/memory/ +├── session-001/ +│ └── memory.json +├── session-002/ +│ └── memory.json +└── archived_session-003/ + └── memory.json +``` + +Each `memory.json` contains: + +```json +{ + "session_id": "session-001", + "updated_at": "2026-05-29T10:00:00", + "items": [ + { + "id": "uuid", + "content": "memory content", + "memory_type": "long_term", + "session_id": "session-001", + "created_at": "2026-05-29T10:00:00", + "metadata": {}, + "tags": [] + } + ] +} +``` + +## Best Practices + +1. **Use SessionMemory for most cases** - It provides a unified API over both memory tiers +2. **Set appropriate max_tokens** - Balance between context length and memory usage +3. **Tag important memories** - Tags improve search relevance +4. **Use metadata for filtering** - Store role, importance, and other structured data +5. **Clean up regularly** - Use `cleanup()` to remove expired memories +6. **Save sessions before switching** - Call `retriever.save_current_session()` to persist +7. **Search before adding duplicates** - Check if similar memories already exist + +## See Also + +- [Session System](session-system.md) - Session persistence and recovery +- [Plugin System](plugin-system.md) - Extending memory with plugins +- [Models System](models-system.md) - Model selection and configuration diff --git a/docs/models-system.md b/docs/models-system.md new file mode 100644 index 0000000..653c93f --- /dev/null +++ b/docs/models-system.md @@ -0,0 +1,233 @@ +# Models System Guide + +> jojo-code supports multiple LLM providers through a unified model registry with preset configurations, custom model registration, and factory functions for quick model creation. + +## Overview + +The models system is built around three components: + +- **ModelProvider** - Enum of supported providers (OpenAI, Anthropic, Custom, Local) +- **ModelCapability** - Enum of model capabilities (chat, function calling, vision, etc.) +- **ModelRegistry** - Central registry for model discovery, filtering, and management + +## Architecture + +``` +ModelRegistry +├── Preset Models (PRESET_MODELS) +│ ├── OpenAI: gpt-4o, gpt-4o-mini, gpt-4-turbo +│ ├── Anthropic: claude-sonnet-4, claude-opus-4, claude-3.5-sonnet, claude-3-haiku +│ └── LongCat: Flash-Chat, Flash-Thinking +├── Custom Models (user-registered) +└── Custom Providers (factory functions) + +Factory Functions +├── create_model(name) -> BaseChatModel +├── create_fast_model() -> BaseChatModel +├── create_smart_model() -> BaseChatModel +└── create_cheap_model() -> BaseChatModel +``` + +## Quick Start + +### Listing Available Models + +```python +from jojo_code.models import ModelRegistry + +registry = ModelRegistry() + +# List all models +all_models = registry.list_models() +print(f"Total models: {len(all_models)}") + +# Filter by provider +from jojo_code.models import ModelProvider +openai_models = registry.list_models(provider=ModelProvider.OPENAI) +anthropic_models = registry.list_models(provider=ModelProvider.ANTHROPIC) + +# Filter by capability +from jojo_code.models import ModelCapability +vision_models = registry.list_models(capability=ModelCapability.VISION) + +# Filter by tags +fast_models = registry.list_fast() +cheap_models = registry.list_cheap() +smart_models = registry.list_smart() +``` + +### Getting Model Information + +```python +from jojo_code.models import ModelRegistry + +registry = ModelRegistry() + +# Get a specific model +info = registry.get("gpt-4o") +print(f"Name: {info.display_name}") +print(f"Provider: {info.provider.value}") +print(f"Context length: {info.context_length}") +print(f"Capabilities: {[c.value for c in info.capabilities]}") +print(f"Cost (input): ${info.cost_per_1k_input}/1K tokens") +print(f"Cost (output): ${info.cost_per_1k_output}/1K tokens") +print(f"Tags: {info.tags}") +``` + +### Using Factory Functions + +```python +from jojo_code.models import create_model, create_fast_model, create_smart_model + +# Create a specific model +model = create_model("gpt-4o") + +# Create by category +fast_model = create_fast_model() # e.g., gpt-4o-mini +smart_model = create_smart_model() # e.g., claude-opus-4 +cheap_model = create_cheap_model() # e.g., gpt-4o-mini +``` + +### Registering Custom Models + +```python +from jojo_code.models import ModelRegistry, ModelInfo, ModelProvider, ModelCapability + +registry = ModelRegistry() + +# Define a custom model +custom = ModelInfo( + name="my-local-model", + provider=ModelProvider.LOCAL, + display_name="My Local Model", + description="A locally hosted model", + context_length=32000, + capabilities=[ModelCapability.CHAT, ModelCapability.STREAMING], + cost_per_1k_input=0.0, + cost_per_1k_output=0.0, + tags=["local", "free"], +) + +# Register it +registry.register(custom) + +# Now it appears in listings +local_models = registry.list_models(provider=ModelProvider.LOCAL) +print(f"Local models: {[m.name for m in local_models]}") +``` + +## Model Providers + +| Provider | Value | Description | API | +|----------|-------|-------------|-----| +| `OPENAI` | `"openai"` | OpenAI GPT models | OpenAI API | +| `ANTHROPIC` | `"anthropic"` | Anthropic Claude models | Anthropic API | +| `CUSTOM` | `"custom"` | OpenAI-compatible APIs | Custom base URL | +| `LOCAL` | `"local"` | Locally hosted models | Local endpoint | + +## Model Capabilities + +| Capability | Value | Description | +|------------|-------|-------------| +| `CHAT` | `"chat"` | Basic conversation | +| `FUNCTION_CALLING` | `"function_calling"` | Tool/function use | +| `VISION` | `"vision"` | Image understanding | +| `STREAMING` | `"streaming"` | Streaming responses | +| `JSON_MODE` | `"json_mode"` | Structured JSON output | +| `REASONING` | `"reasoning"` | Extended reasoning / chain-of-thought | + +## Preset Models + +### OpenAI + +| Model | Context | Capabilities | Tags | Cost (in/out per 1K) | +|-------|---------|--------------|------|----------------------| +| `gpt-4o` | 128K | chat, function_calling, vision, streaming, json_mode | - | $0.005 / $0.015 | +| `gpt-4o-mini` | 128K | chat, function_calling, streaming | fast, cheap | $0.00015 / $0.0006 | +| `gpt-4-turbo` | 128K | chat, function_calling, vision, streaming | - | $0.01 / $0.03 | + +### Anthropic + +| Model | Context | Capabilities | Tags | Cost (in/out per 1K) | +|-------|---------|--------------|------|----------------------| +| `claude-sonnet-4-20250514` | 200K | chat, function_calling, vision, streaming | - | $0.003 / $0.015 | +| `claude-opus-4-20250514` | 200K | chat, function_calling, vision, streaming, reasoning | smart, reasoning | $0.015 / $0.075 | +| `claude-3-5-sonnet-20240620` | 200K | chat, function_calling, vision, streaming | - | $0.003 / $0.015 | +| `claude-3-haiku-20240307` | 200K | chat, function_calling, vision, streaming | fast, cheap | $0.00025 / $0.00125 | + +### LongCat (Custom) + +| Model | Context | Capabilities | Tags | Cost (in/out per 1K) | +|-------|---------|--------------|------|----------------------| +| `LongCat-Flash-Chat` | 128K | chat, function_calling, streaming | fast, cheap, chinese | $0.0001 / $0.0003 | +| `LongCat-Flash-Thinking-2601` | 128K | chat, function_calling, streaming, reasoning | reasoning, chinese | $0.0002 / $0.0006 | + +## Registry Statistics + +```python +registry = ModelRegistry() +stats = registry.get_stats() + +print(f"Total models: {stats['total']}") +print("By provider:") +for provider, count in stats["by_provider"].items(): + print(f" {provider}: {count}") +``` + +## Global Registry + +A global singleton registry is available for convenience: + +```python +from jojo_code.models import get_model_registry, set_model_registry + +# Get the global registry (creates one if needed) +registry = get_model_registry() + +# Replace with a custom registry +custom_registry = ModelRegistry() +set_model_registry(custom_registry) +``` + +## Custom Providers + +Register factory functions for custom model creation: + +```python +from jojo_code.models import ModelRegistry, ModelProvider + +registry = ModelRegistry() + +# Register a factory for local models +def create_local_model(): + # Your local model creation logic + pass + +registry.register_custom_provider(ModelProvider.LOCAL, create_local_model) +``` + +## Configuration + +Models are configured via environment variables or `~/.jojo-code/config.json`: + +```bash +# .env +OPENAI_API_KEY=sk-... +OPENAI_BASE_URL=https://api.openai.com/v1 +ANTHROPIC_API_KEY=sk-ant-... +JOJO_CODE_MODEL=gpt-4o +``` + +## Best Practices + +1. **Use factory functions for quick setup** - `create_fast_model()`, `create_smart_model()`, `create_cheap_model()` +2. **Check capabilities before use** - Not all models support function calling or vision +3. **Consider cost when choosing** - Use `list_cheap()` for high-volume tasks +4. **Register custom models for non-standard APIs** - Extends the registry without modifying presets +5. **Use tags for filtering** - Tags like "fast", "cheap", "smart" help with model selection +6. **Set appropriate context_length** - Prevents token overflow errors + +## See Also + +- [Memory System](memory-system.md) - Managing conversation context across sessions +- [Security System](security-system.md) - Controlling model access and permissions diff --git a/docs/plugin-system.md b/docs/plugin-system.md new file mode 100644 index 0000000..0fd6120 --- /dev/null +++ b/docs/plugin-system.md @@ -0,0 +1,276 @@ +# Plugin System Guide + +> jojo-code provides a flexible plugin system that lets you extend the agent with custom tools, hooks, and lifecycle management. + +## Overview + +The plugin system is built around four key components: + +- **BasePlugin** - Abstract base class for all plugins +- **PluginRegistry** - Singleton registry for plugin management +- **PluginDiscovery** - Automatic plugin detection from files and directories +- **HookDispatcher** - Event-driven hook system for lifecycle integration + +## Quick Start + +### Creating a Plugin + +```python +from jojo_code.plugin.base import BasePlugin, PluginMetadata, PluginPermission +from jojo_code.plugin.hooks import HOOK_BEFORE_TOOL_CALL, HOOK_AFTER_TOOL_CALL + +class MyPlugin(BasePlugin): + metadata = PluginMetadata( + name="my-plugin", + version="1.0.0", + description="A custom plugin", + author="your-name", + tags=["custom"], + ) + permission = PluginPermission.UNTRUSTED + + def on_load(self) -> None: + print("Plugin loaded!") + + def on_unload(self) -> None: + print("Plugin unloaded!") + + def get_tools(self) -> list: + """Return custom tools (LangChain BaseTool instances).""" + return [] + + def get_hooks(self) -> dict: + """Return hook handlers.""" + return { + HOOK_BEFORE_TOOL_CALL: self._before_call, + HOOK_AFTER_TOOL_CALL: self._after_call, + } + + def _before_call(self, tool_name: str, args: dict) -> None: + print(f"About to call: {tool_name}") + + def _after_call(self, tool_name: str, result: str) -> None: + print(f"Finished: {tool_name}") +``` + +### Registering a Plugin + +```python +from jojo_code.plugin.registry import PluginRegistry +from jojo_code.plugin.hooks import HookDispatcher + +registry = PluginRegistry.get_instance() +dispatcher = HookDispatcher() +registry.set_dispatcher(dispatcher) + +plugin = MyPlugin() +registry.register("my-plugin", plugin) + +# List registered plugins +print(registry.list_plugins()) + +# Unregister when done +registry.unregister("my-plugin") +``` + +## Permission Levels + +| Level | Access | Use Case | +|-------|--------|----------| +| `UNTRUSTED` | No filesystem or network access | Safe utility plugins | +| `RESTRICTED` | Limited filesystem, no network | File processing plugins | +| `TRUSTED` | Full access | Development tools | + +## Sandbox Configuration + +```python +from jojo_code.plugin.base import PluginSandbox + +sandbox = PluginSandbox( + restricted=True, + allowed_paths=["src/**", "tests/**"], + allowed_urls=["https://api.example.com"], + max_memory_mb=100, +) +``` + +## Available Hooks + +| Hook | Arguments | Description | +|------|-----------|-------------| +| `HOOK_BEFORE_TOOL_CALL` | `tool_name, args` | Before tool execution | +| `HOOK_AFTER_TOOL_CALL` | `tool_name, result` | After tool execution | +| `HOOK_BEFORE_AGENT_RUN` | `messages` | Before agent loop starts | +| `HOOK_AFTER_AGENT_RUN` | `messages, result` | After agent loop ends | +| `HOOK_ON_ERROR` | `error` | When an error occurs | + +## Plugin Discovery + +Plugins can be discovered from: + +1. **Directories** - Scan a directory for plugin classes +2. **Files** - Load a specific plugin file +3. **Entry Points** - Python package entry points + +```python +from pathlib import Path +from jojo_code.plugin.discovery import PluginDiscovery + +discovery = PluginDiscovery() + +# From directory +plugins = discovery.discover(Path("plugins/")) + +# From single file +plugins = discovery.discover(Path("my_plugin.py")) +``` + +## CLI Commands + +```bash +# List all plugins +jojo-code plugin list + +# Enable/disable plugins +jojo-code plugin enable +jojo-code plugin disable + +# Show plugin details +jojo-code plugin info +``` + +## Built-in Plugins + +jojo-code ships with several built-in plugins: + +- **code_review** - Code quality and security analysis +- **git** - Git workflow integration +- **test_generator** - Automatic test generation + +## Plugin Configuration + +Plugins can be configured via `plugin.yaml`: + +```yaml +plugins: + - name: code-review + enabled: true + config: + security_patterns: + - "eval(" + - "exec(" + severity_threshold: medium + + - name: test-generator + enabled: true + config: + framework: pytest + include_type_hints: true +``` + +## Core Plugin Manager + +In addition to the `plugin/` package above, jojo-code provides a standalone `PluginManager` in `core/plugin.py` for file-based plugin discovery and lifecycle management. + +### Key Differences + +| Feature | `plugin/` (PluginRegistry) | `core/plugin.py` (PluginManager) | +|---------|---------------------------|----------------------------------| +| Discovery | Class-based, in-process | File/directory scanning | +| Lifecycle | `on_load` / `on_unload` | `on_load` / `on_unload` / `on_enable` / `on_disable` | +| Version check | No | Yes, via `semver` | +| Config | Via `PluginSandbox` | Per-plugin `config.json` | +| Hooks | `HookDispatcher` | Built-in `register_hook` / `trigger_hook` | + +### Using PluginManager + +```python +from jojo_code.core.plugin import PluginManager, PluginContext + +# Create a manager (auto-creates ~/.jojo-code/plugins/) +manager = PluginManager() + +# Discover and load all plugins +await manager.load_all_plugins() + +# Or load a specific plugin +from pathlib import Path +plugin = await manager.load_plugin(Path("~/.jojo-code/plugins/my-plugin")) + +# Enable/disable +await manager.enable_plugin("my-plugin") +await manager.disable_plugin("my-plugin") + +# Unload +await manager.unload_plugin("my-plugin") +``` + +### Plugin File Structure + +A file-based plugin must have a `PLUGIN_METADATA` dict and optional lifecycle hooks: + +```python +# my-plugin/plugin.py +PLUGIN_METADATA = { + "name": "my-plugin", + "version": "1.0.0", + "description": "My custom plugin", + "author": "your-name", + "min_jojo_code_version": "0.2.0", +} + +async def on_load(): + print("Plugin loaded!") + +async def on_unload(): + print("Plugin unloaded!") + +async def on_enable(): + print("Plugin enabled!") + +async def on_disable(): + print("Plugin disabled!") +``` + +### PluginContext + +`PluginContext` gives plugins access to core features: + +```python +from jojo_code.core.plugin import get_plugin_manager, PluginContext + +manager = get_plugin_manager() +ctx = PluginContext("my-plugin", manager) + +# Register commands and tools +ctx.register_command("my-cmd", my_handler) +ctx.register_tool("my-tool", MyToolClass) + +# Emit events +ctx.emit_event("data_ready", {"rows": 100}) + +# Read/write per-plugin config +value = ctx.get_config("api_key", default="") +ctx.set_config("api_key", "sk-xxx") +``` + +### Creating a Plugin Template + +Use `create_plugin_template` to scaffold a new plugin: + +```python +from jojo_code.core.plugin import create_plugin_template +create_plugin_template("my-plugin", "1.0.0", "your-name") +``` + +This creates `plugins/my-plugin/` with `plugin.py`, `config.json`, and `README.md`. + +## Best Practices + +1. **Start with UNTRUSTED** - Use the lowest permission level that works +2. **Validate inputs** - Check arguments in hook handlers +3. **Handle errors gracefully** - Hooks should not crash the agent +4. **Use sandboxing** - Restrict filesystem and network access when possible +5. **Keep hooks fast** - Slow hooks block the agent loop +6. **Version your plugins** - Use `min_jojo_code_version` to prevent loading on incompatible versions +7. **Use `PluginContext` for config** - Avoid hardcoding paths; use `get_config` / `set_config` diff --git a/docs/security-system.md b/docs/security-system.md new file mode 100644 index 0000000..5c29726 --- /dev/null +++ b/docs/security-system.md @@ -0,0 +1,381 @@ +# Security System Guide + +> jojo-code provides a comprehensive security system for controlling tool access, managing permissions, and auditing operations. + +## Overview + +The security system is built around these components: + +- **PermissionManager** - Central coordinator for all permission checks +- **EnhancedPermissionManager** - Extended manager with rule engine and denial tracking +- **BaseGuard** - Abstract base class for permission guards +- **PathGuard** - Filesystem path access control +- **CommandGuard** - Shell command access control +- **RuleEngine** - Flexible rule-based permission matching +- **DenialTracker** - Tracks repeated denials to prevent abuse +- **AuditLogger** - Logs all permission decisions + +## Quick Start + +### Basic Permission Configuration + +```python +from jojo_code.security import PermissionConfig, PermissionManager + +# Development mode - relaxed permissions +config = PermissionConfig.development() +manager = PermissionManager(config) + +# Production mode - strict permissions +config = PermissionConfig.production() +manager = PermissionManager(config) + +# Custom configuration +config = PermissionConfig( + workspace_root=Path("."), + allowed_paths=["src/**", "tests/**"], + denied_paths=[".env", ".git/**", "*.pem"], + confirm_on_write=["**/*.py"], + shell_enabled=True, + allowed_commands=["ls", "cat", "grep", "pytest"], + denied_commands=["rm -rf", "sudo"], + max_tool_calls=100, + audit_log=True, +) +manager = PermissionManager(config) +``` + +### Checking Permissions + +```python +# Check a tool call +result = manager.check("read_file", {"path": "src/main.py"}) + +if result.allowed: + print("Operation allowed") +elif result.needs_confirm: + print("User confirmation required") +elif result.denied: + print(f"Denied: {result.reason}") +``` + +## Permission Levels + +| Level | Description | Behavior | +|-------|-------------|----------| +| `ALLOW` | Auto-approved | Tool executes immediately | +| `CONFIRM` | Needs user approval | Prompts user for confirmation | +| `DENY` | Blocked | Tool call is rejected | + +## Permission Modes + +| Mode | Description | Use Case | +|------|-------------|----------| +| `AUTO` | Low-risk auto-approved, high-risk needs confirm | Default mode | +| `MANUAL` | All write operations need confirmation | Careful development | +| `BYPASS` | All operations allowed (dangerous!) | Testing only | + +```python +from jojo_code.security import PermissionMode + +# Set mode +manager.set_mode("auto") # AUTO mode +manager.set_mode("manual") # MANUAL mode +manager.set_mode("bypass") # BYPASS mode (use with caution!) +``` + +## Guards + +### PathGuard + +Controls filesystem access based on path patterns: + +```python +from jojo_code.security import PathGuard + +guard = PathGuard( + workspace_root=Path("/project"), + allowed_patterns=["src/**", "tests/**"], + denied_patterns=[".env", "secrets/**"], + confirm_patterns=["**/*.py"], + allow_outside=False, +) +``` + +### CommandGuard + +Controls shell command execution: + +```python +from jojo_code.security import CommandGuard + +guard = CommandGuard( + enabled=True, + allowed_commands=["ls", "cat", "grep"], + denied_commands=["rm -rf", "sudo", "curl"], + default=PermissionLevel.CONFIRM, + max_timeout=120, + allow_network=False, +) +``` + +### Custom Guards + +Create custom guards by extending `BaseGuard`: + +```python +from jojo_code.security import BaseGuard, PermissionLevel, PermissionResult + +class NetworkGuard(BaseGuard): + BLOCKED_TOOLS = {"web_search", "http_request"} + + @property + def name(self) -> str: + return "network_guard" + + def check(self, tool_name: str, args: dict) -> PermissionResult: + if tool_name in self.BLOCKED_TOOLS: + return PermissionResult( + PermissionLevel.DENY, tool_name, args, + reason="Network access blocked", + ) + return PermissionResult(PermissionLevel.ALLOW, tool_name, args) + +# Add to manager +manager.guards.append(NetworkGuard()) +``` + +## Rule Engine + +The rule engine provides flexible pattern matching for permissions. + +### Creating Rules + +```python +from jojo_code.security import PermissionRule, RuleAction, RuleMatchType + +# Exact match +rule = PermissionRule( + name="allow_read", + tool_pattern="read_file", + match_type=RuleMatchType.EXACT, + action=RuleAction.ALLOW, + priority=10, +) + +# Glob match +rule = PermissionRule( + name="deny_writes", + tool_pattern="write_*", + match_type=RuleMatchType.GLOB, + action=RuleAction.DENY, + priority=50, +) + +# Regex match +rule = PermissionRule( + name="ask_git", + tool_pattern=r"git_\w+", + match_type=RuleMatchType.REGEX, + action=RuleAction.ASK, + priority=30, +) + +# With argument patterns +rule = PermissionRule( + name="deny_rm_rf", + tool_pattern="run_command", + args_pattern={"command": "rm -rf *"}, + action=RuleAction.DENY, + priority=100, +) +``` + +### Rule Actions + +| Action | Description | +|--------|-------------| +| `ALLOW` | Automatically allow the operation | +| `DENY` | Block the operation | +| `ASK` | Continue to next check (fall through) | + +### Rule Priority + +Rules are evaluated in priority order (highest first). If no rule matches, the default action is used. + +### RuleFactory + +Pre-built rule templates: + +```python +from jojo_code.security import RuleFactory + +# Dangerous command rules +rules = RuleFactory.deny_dangerous_commands() +# Includes: deny_rm_rf, deny_sudo, deny_chmod_777 + +# Write confirmation rules +rules = RuleFactory.require_confirmation_for_writes() +# Includes: ask_write_file, ask_edit_file, ask_run_command, ask_delete_file +``` + +## Enhanced Permission Manager + +Combines base permissions, rule engine, and denial tracking: + +```python +from jojo_code.security import ( + EnhancedPermissionConfig, + EnhancedPermissionManager, +) + +def confirm_callback(result): + """User confirmation callback.""" + response = input(f"Allow {result.tool_name}? [y/N] ") + return response.lower() == "y" + +config = EnhancedPermissionConfig( + base=PermissionConfig( + workspace_root=Path("."), + denied_paths=[".env"], + audit_log=False, + ), + enable_rule_engine=True, + default_rule_action="ask", + enable_denial_tracking=True, + denial_threshold=3, + denial_window_seconds=300, + confirm_callback=confirm_callback, +) + +manager = EnhancedPermissionManager(config) +``` + +### Check Flow + +1. **Rule Engine** - Check if any rule matches +2. **Base Permission** - Check path/command guards +3. **Denial Tracking** - Check if threshold exceeded +4. **Confirm Callback** - Ask user if needed + +## Denial Tracking + +Tracks repeated denials to prevent abuse: + +```python +from jojo_code.security import DenialTracker + +tracker = DenialTracker( + threshold=3, # Max denials before blocking + window_seconds=300, # Time window (5 minutes) +) + +# Record a denial +tracker.record("run_command", {"command": "rm -rf /"}, "Dangerous command") + +# Check threshold +if tracker.is_threshold_exceeded("run_command", {"command": "rm -rf /"}): + print("Too many denied attempts!") + +# Get stats +stats = tracker.get_stats() +print(f"Total denials: {stats['total_denials']}") +``` + +## Risk Assessment + +Built-in risk assessment for tool calls: + +```python +from jojo_code.security import assess_risk, RiskLevel + +risk = assess_risk("write_file", {"path": "config.json"}) +# Returns: "medium" + +risk = assess_risk("run_command", {"command": "rm -rf /"}) +# Returns: "critical" +``` + +Risk levels: `low`, `medium`, `high`, `critical` + +## Audit Logging + +All permission decisions are logged: + +```python +config = PermissionConfig( + audit_log=True, + audit_log_path=Path(".jojo-code/audit.log"), +) +manager = PermissionManager(config) + +# After operations, get audit log +log = manager.get_audit_log() +for entry in log: + print(f"{entry['timestamp']} - {entry['tool']}: {entry['result']}") +``` + +## YAML Configuration + +Load configuration from YAML: + +```yaml +# security.yaml +workspace: + root: "." + allow_outside: false + +file: + allowed_paths: + - "src/**" + - "tests/**" + denied_paths: + - ".env" + - ".git/**" + - "*.pem" + confirm_on_write: + - "**/*.py" + +shell: + enabled: true + allowed_commands: + - "ls" + - "cat" + - "grep" + denied_commands: + - "rm -rf" + - "sudo" + default: "confirm" + max_timeout: 120 + allow_network: false + +global: + max_tool_calls: 100 + audit_log: true + audit_log_path: ".jojo-code/audit.log" +``` + +```python +from pathlib import Path +from jojo_code.security import PermissionConfig + +config = PermissionConfig.from_yaml(Path("security.yaml")) +``` + +## Best Practices + +1. **Use AUTO mode** - Provides good balance of safety and convenience +2. **Deny dangerous commands** - Always block `rm -rf /`, `sudo`, etc. +3. **Enable audit logging** - Essential for debugging and compliance +4. **Set reasonable limits** - Use `max_tool_calls` to prevent runaway agents +5. **Use path guards** - Restrict access to sensitive files (.env, secrets) +6. **Test with MANUAL mode** - Verify all operations before production +7. **Never use BYPASS in production** - Only for testing +8. **Create custom guards** - Extend `BaseGuard` for domain-specific rules +9. **Use denial tracking** - Prevent repeated failed attempts +10. **Configure per-environment** - Different configs for dev/staging/prod + +## See Also + +- [Plugin System](plugin-system.md) - Extending security with plugins +- [Memory System](memory-system.md) - Memory management and retrieval diff --git a/docs/session-system.md b/docs/session-system.md new file mode 100644 index 0000000..d33df6f --- /dev/null +++ b/docs/session-system.md @@ -0,0 +1,219 @@ +# Session System Guide + +> jojo-code provides a session management system for creating, persisting, and recovering conversation sessions with full message history. + +## Overview + +The session system is built around two core components: + +- **Session** - A conversation container with messages, metadata, and timestamps +- **SessionManager** - Handles CRUD operations and JSON persistence on disk + +## Architecture + +``` +SessionManager +├── create_session(user_id, metadata) -> Session +├── get_session(session_id) -> Session | None +├── add_message(session_id, role, content) +├── save_session(session) +├── recover_session(session_id) -> Session | None +└── Storage: {storage_dir}/{session_id}.json + +Session +├── id: str (UUID) +├── user_id: str | None +├── created_at: float (UTC timestamp) +├── last_seen_at: float (UTC timestamp) +├── messages: list[Message] +└── metadata: dict[str, str] + +Message +├── role: str ("user" | "assistant" | "system") +├── content: str +└── timestamp: float (UTC timestamp) +``` + +## Quick Start + +### Creating and Using Sessions + +```python +from jojo_code.session.manager import SessionManager + +# Create a manager (auto-creates the storage directory) +manager = SessionManager(storage_dir="./sessions") + +# Create a new session +session = manager.create_session(user_id="user-1", metadata={"lang": "en"}) +print(f"Session ID: {session.id}") + +# Add messages +manager.add_message(session.id, "user", "How do I read a file in Python?") +manager.add_message(session.id, "assistant", "Use open() or pathlib.") +manager.add_message(session.id, "user", "Which is better?") + +# Retrieve the session +loaded = manager.get_session(session.id) +print(f"Messages: {len(loaded.messages)}") +for msg in loaded.messages: + print(f" [{msg.role}] {msg.content}") +``` + +### Session Recovery + +Sessions persist to disk as JSON files, so they survive process restarts: + +```python +# In a new process, recover the session +manager = SessionManager(storage_dir="./sessions") +session = manager.recover_session(session_id) +if session: + print(f"Recovered {len(session.messages)} messages") +``` + +### Working with Metadata + +Metadata stores arbitrary key-value pairs alongside the session: + +```python +session = manager.create_session( + user_id="user-1", + metadata={ + "source": "cli", + "mode": "build", + "project": "my-app", + }, +) + +# Metadata persists across reloads +loaded = manager.get_session(session.id) +print(loaded.metadata["mode"]) # "build" +``` + +## Session Data Model + +### Session Fields + +| Field | Type | Default | Description | +|-------|------|---------|-------------| +| `id` | `str` | UUID4 | Unique session identifier | +| `user_id` | `str \| None` | `None` | Optional user identifier | +| `created_at` | `float` | now (UTC) | Creation timestamp | +| `last_seen_at` | `float` | now (UTC) | Last activity timestamp | +| `messages` | `list[Message]` | `[]` | Conversation messages | +| `metadata` | `dict[str, str]` | `{}` | Arbitrary metadata | + +### Message Fields + +| Field | Type | Default | Description | +|-------|------|---------|-------------| +| `role` | `str` | - | `"user"`, `"assistant"`, or `"system"` | +| `content` | `str` | - | Message text | +| `timestamp` | `float` | now (UTC) | Message creation time | + +### Serialization + +Sessions and messages support dict serialization for JSON persistence: + +```python +# Session -> dict +data = session.to_dict() + +# dict -> Session +session = Session.from_dict(data) + +# Message -> dict +msg_data = message.to_dict() + +# dict -> Message +message = Message.from_dict(msg_data) +``` + +## Storage Format + +Each session is stored as a JSON file at `{storage_dir}/{session_id}.json`: + +```json +{ + "id": "550e8400-e29b-41d4-a716-446655440000", + "user_id": "user-1", + "created_at": 1717000000.0, + "last_seen_at": 1717000100.0, + "messages": [ + { + "role": "user", + "content": "Hello", + "timestamp": 1717000000.0 + }, + { + "role": "assistant", + "content": "Hi! How can I help?", + "timestamp": 1717000050.0 + } + ], + "metadata": { + "source": "cli" + } +} +``` + +## Error Handling + +The session manager is resilient to corrupted data: + +- **Missing session**: `get_session()` returns `None` +- **Corrupted JSON**: `get_session()` returns `None` (does not crash) +- **Empty file**: `get_session()` returns `None` +- **Missing message in unknown session**: `add_message()` raises `ValueError` + +```python +# Safe retrieval +session = manager.get_session(session_id) +if session is None: + print("Session not found or corrupted") + +# Safe message addition +try: + manager.add_message(session_id, "user", "hello") +except ValueError as e: + print(f"Error: {e}") +``` + +## Integration with Memory System + +The session system works alongside the memory system. Use sessions for conversation persistence and memory for searchable knowledge: + +```python +from jojo_code.session.manager import SessionManager +from jojo_code.memory import SessionMemory + +# Session for persistence +session_mgr = SessionManager(storage_dir="./sessions") +session = session_mgr.create_session(user_id="user-1") + +# Memory for search and retrieval +memory = SessionMemory(session_id=session.id) + +# Add a message to both +session_mgr.add_message(session.id, "user", "How do I test Python code?") +memory.add_message("How do I test Python code?", role="user") + +# Memory supports search; sessions support recovery +results = memory.search("test") +recovered = session_mgr.recover_session(session.id) +``` + +## Best Practices + +1. **Use meaningful user_id values** - Helps with multi-user scenarios +2. **Store context in metadata** - Project name, mode, language preferences +3. **Use recover_session for clarity** - It is an alias for get_session with clearer intent +4. **Set appropriate storage_dir** - Use absolute paths in production +5. **Handle None returns** - Always check for None when retrieving sessions +6. **Use SessionMemory for search** - Sessions are for persistence, memory is for retrieval + +## See Also + +- [Memory System](memory-system.md) - Conversation memory with search and retrieval +- [Plugin System](plugin-system.md) - Extending session behavior with plugins diff --git a/docs/skills-system.md b/docs/skills-system.md new file mode 100644 index 0000000..e24b862 --- /dev/null +++ b/docs/skills-system.md @@ -0,0 +1,199 @@ +# Skills System Guide + +> Skills are reusable, composable capabilities that the agent can invoke. They wrap common operations into named, documented units with metadata and categories. + +## Overview + +The skills system provides: + +- **SkillDefinition** - Metadata-rich skill descriptors +- **SkillManager** - Registration, discovery, and execution +- **@skill decorator** - Quick skill creation from functions +- **Built-in skills** - 10+ pre-built skills for common tasks + +## Quick Start + +### Creating a Skill with the @skill Decorator + +```python +from jojo_code.skills.base import skill +from jojo_code.skills.types import SkillCategory + +@skill( + name="my_skill", + description="Does something useful", + category=SkillCategory.CODE, + tags=["custom", "utility"], + examples=["Run my_skill on file.py"], +) +def my_skill(file_path: str) -> str: + """Process a file. + + Args: + file_path: Path to the file + + Returns: + Processing result + """ + return f"Processed {file_path}" +``` + +### Creating a Skill with BaseSkill + +```python +from jojo_code.skills.base import BaseSkill +from jojo_code.skills.types import SkillCategory + +class MyCustomSkill(BaseSkill): + def __init__(self): + super().__init__( + name="custom", + description="A custom skill", + category=SkillCategory.CUSTOM, + ) + + def execute(self, *args, **kwargs): + return "result" + + def validate(self, *args, **kwargs): + return True +``` + +### Registering Skills with SkillManager + +```python +from jojo_code.skills.manager import SkillManager + +manager = SkillManager() + +# Register a decorated skill +manager.register(my_skill._skill_def) + +# Or register a function directly +skill_id = manager.register_function( + my_skill, + name="my_skill", + description="Does something useful", +) + +# Execute a skill +result = manager.execute(skill_id, "file.py") +print(result.output) + +# Search for skills +matches = manager.search("file") + +# List by category +from jojo_code.skills.types import SkillCategory +code_skills = manager.list_skills(category=SkillCategory.CODE) +``` + +## Skill Categories + +| Category | Description | Examples | +|----------|-------------|----------| +| `WEB` | Web operations | search, fetch | +| `DATA` | Data processing | JSON, CSV, math | +| `CODE` | Code operations | analysis, formatting | +| `FILE` | File operations | read, write | +| `SYSTEM` | System operations | shell commands | +| `SEARCH` | Search operations | text search, translation | +| `CUSTOM` | User-defined | anything else | + +## Built-in Skills + +jojo-code provides these built-in skills: + +### Web Skills +- **web_search** - Search the web for information +- **web_fetch** - Fetch web page content + +### File Skills +- **read_file** - Read file contents +- **write_file** - Write content to a file + +### System Skills +- **run_command** - Execute shell commands + +### Code Skills +- **analyze_code** - Analyze code quality and structure + +### Data Skills +- **format_json** - Format JSON data +- **validate_json** - Validate JSON syntax +- **calculate** - Evaluate math expressions + +### Search Skills +- **translate** - Translate text between languages + +## Skill Metadata + +Each skill carries rich metadata: + +```python +@skill( + name="example", # Skill name + description="An example", # Human-readable description + category=SkillCategory.CODE, # Category for organization + tags=["example", "demo"], # Searchable tags + version="1.0.0", # Semantic version + author="your-name", # Author + examples=["Run example"], # Usage examples + requires=["read_file"], # Dependencies + scope=SkillScope.GLOBAL, # Visibility scope +) +``` + +## Skill Scopes + +| Scope | Description | +|-------|-------------| +| `GLOBAL` | Available everywhere (default) | +| `SESSION` | Current session only | +| `PROJECT` | Current project only | + +## Skill Results + +Execution results are wrapped in `SkillResult`: + +```python +result = manager.execute(skill_id, "input") + +print(result.success) # bool +print(result.output) # Any - the result data +print(result.error) # str or None +print(result.duration_ms) # float - execution time +print(result.metadata) # dict - extra info +``` + +## Adapting Skills to LangChain Tools + +Skills can be converted to LangChain `BaseTool` instances for use with the agent: + +```python +from jojo_code.skills.base import create_skill_tool + +# Create a LangChain tool from a skill +tool = create_skill_tool(my_skill) +result = tool.invoke({"input": "value"}) +``` + +## CLI Integration + +```bash +# Skills are automatically loaded and available to the agent +jojo-code + +# The agent can invoke skills by name during conversation +# "Search for Python tutorials" -> triggers web_search skill +# "Format this JSON" -> triggers format_json skill +``` + +## Best Practices + +1. **Write clear descriptions** - The agent uses descriptions to choose skills +2. **Add examples** - Help the agent understand when to use the skill +3. **Use appropriate categories** - Makes skills discoverable +4. **Handle errors gracefully** - Return error strings, never raise exceptions +5. **Keep skills focused** - One skill = one capability +6. **Add tags** - Improve searchability diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 0000000..86bf087 --- /dev/null +++ b/examples/README.md @@ -0,0 +1,156 @@ +# jojo-code Examples + +This directory contains example scripts and configuration files to help you get started with jojo-code. + +## Files + +### new_tools_demo.py + +A comprehensive demonstration script showcasing jojo-code's built-in tools: + +- **Code Analysis Tools**: Analyze Python files, find dependencies, check code style, suggest refactoring +- **Git Integration Tools**: Git status, diff, log, blame, branch info +- **Performance Tools**: Profile Python files, analyze function complexity, suggest optimizations + +Run the demo: + +```bash +uv run python examples/new_tools_demo.py +``` + +### new_tools_demo_latest.py + +An updated version of the demo script based on the latest master branch. Use this for the most current tool demonstrations. + +```bash +uv run python examples/new_tools_demo_latest.py +``` + +### plugin_development.py + +A comprehensive example showing how to create custom plugins for jojo-code: + +- **Plugin Lifecycle**: Implementing `on_load` and `on_unload` methods +- **Hook System**: Registering `before_tool_call` and `after_tool_call` hooks +- **Custom Tools**: Providing tools via `get_tools()` +- **Security Plugins**: Auditing dangerous operations +- **Plugin Discovery**: Discovering plugins from files and directories + +Run the demo: + +```bash +uv run python examples/plugin_development.py +``` + +### tool_creation.py + +A guide to creating custom tools for jojo-code: + +- **@tool Decorator**: Using LangChain's `@tool` decorator +- **Error Handling**: Graceful error handling in tools +- **File Tools**: Creating tools that work with the filesystem +- **ToolRegistry Integration**: Registering and managing custom tools +- **Tool Metadata**: Inspecting tool names, descriptions, and parameters + +Run the demo: + +```bash +uv run python examples/tool_creation.py +``` + +### advanced_usage.py + +An advanced example demonstrating sub-agents, plugins, and custom tools working together: + +- **Custom Tools**: Creating tools with `@tool` decorator (word frequency, code stats, dict merge) +- **Sub-Agents**: Creating and managing sub-agents with `AgentRegistry` and `AgentTool` +- **Built-in Agents**: Using `BuiltInAgents` factory (code_review, research, debug) +- **Plugins**: Creating plugins with hooks for metrics tracking and input validation +- **Integrated Workflow**: All components working together in a pipeline + +Run the demo: + +```bash +uv run python examples/advanced_usage.py +``` + +### security_config.py + +A comprehensive security configuration example demonstrating: + +- **Permission Configurations**: Development, production, and custom permission setups +- **Permission Manager**: Checking permissions for various tool operations +- **Rule Engine**: Creating custom rules with pattern matching (glob, regex, prefix) +- **RuleFactory**: Pre-built rule templates for dangerous commands and write confirmations +- **Enhanced Permission Manager**: Combining rules, denial tracking, and confirm callbacks +- **Denial Tracking**: Automatic detection of repeated denied requests +- **Permission Modes**: AUTO, MANUAL, and BYPASS modes with different behaviors +- **Custom Guards**: Creating custom permission guards (e.g., NetworkGuard) + +Run the demo: + +```bash +uv run python examples/security_config.py +``` + +### plugin.yaml + +Example plugin configuration file for jojo-code. This file shows how to: + +- Enable/disable specific plugins (`code-review`, `test-generator`, `git`) +- Configure plugin-specific settings +- Set security patterns for code review +- Configure test generation options (framework, type hints) + +To use this configuration: + +1. Copy `plugin.yaml` to your project root +2. Modify the settings as needed +3. jojo-code will automatically detect and load the configuration + +## Creating Your Own Examples + +To add new examples: + +1. Create a new Python file in this directory +2. Import tools from `jojo_code.tools.*` +3. Use the `.invoke()` method to call tools +4. Add clear comments explaining what the example demonstrates +5. Update this README with a description of your example + +Example template: + +```python +"""Description of what this example demonstrates.""" + +from jojo_code.tools.some_tools import some_tool + +def main(): + # Call the tool using .invoke() + result = some_tool.invoke("your input") + print(result) + +if __name__ == "__main__": + main() +``` + +## Tool Categories + +jojo-code provides 40+ tools organized into categories: + +| Category | Example Tools | Description | +|----------|---------------|-------------| +| File | `read_file`, `write_file`, `edit_file` | File operations | +| Shell | `run_shell_command` | Execute shell commands | +| Git | `git_status`, `git_diff`, `git_log` | Git integration | +| Search | `ripgrep`, `find_files` | Code search | +| Web | `web_search`, `web_fetch` | Web access | +| Code Analysis | `analyze_python_file`, `find_dependencies` | Code analysis | +| Performance | `profile_file`, `analyze_complexity` | Performance profiling | +| Data | `csv_analyze`, `json_query` | Data processing | + +## More Resources + +- [Main Documentation](../README.md) +- [Project Structure](../CLAUDE.md) +- [Plugin System](../src/jojo_code/plugin/) diff --git a/examples/advanced_usage.py b/examples/advanced_usage.py new file mode 100644 index 0000000..04bcf03 --- /dev/null +++ b/examples/advanced_usage.py @@ -0,0 +1,482 @@ +"""Advanced Usage Example - jojo-code + +Demonstrates sub-agents, plugins, and custom tools working together. +Shows how to build complex workflows with jojo-code's extensibility features. + +Usage: + uv run python examples/advanced_usage.py +""" + +import json +import logging +import sys +from pathlib import Path +from typing import Any + +# Add project src to path for direct execution +sys.path.insert(0, str(Path(__file__).parent.parent / "src")) + +from langchain_core.tools import tool + +from jojo_code.agent.sub import AgentRegistry +from jojo_code.agent.tool import AgentTool, BuiltInAgents +from jojo_code.plugin.base import BasePlugin, PluginMetadata, PluginPermission +from jojo_code.plugin.hooks import HOOK_BEFORE_TOOL_CALL +from jojo_code.plugin.registry import PluginRegistry +from jojo_code.tools.registry import ToolRegistry + +logging.basicConfig(level=logging.INFO) +logger = logging.getLogger(__name__) + + +# ============================================================================= +# Part 1: Custom Tools +# ============================================================================= + + +@tool +def word_frequency(text: str, top_n: int = 10) -> str: + """Analyze word frequency in text and return the most common words. + + Args: + text: The text to analyze + top_n: Number of top words to return (default 10) + + Returns: + JSON string with word frequencies + """ + words = text.lower().split() + freq: dict[str, int] = {} + for w in words: + cleaned = w.strip(".,!?;:\"'()[]{}") + if cleaned: + freq[cleaned] = freq.get(cleaned, 0) + 1 + + sorted_words = sorted(freq.items(), key=lambda x: x[1], reverse=True)[:top_n] + return json.dumps(dict(sorted_words), indent=2) + + +@tool +def code_snippet_stats(code: str) -> str: + """Analyze a code snippet and return statistics. + + Args: + code: The code snippet to analyze + + Returns: + JSON string with code statistics + """ + lines = code.split("\n") + total = len(lines) + blank = sum(1 for line in lines if not line.strip()) + comments = sum(1 for line in lines if line.strip().startswith("#")) + imports = sum(1 for line in lines if line.strip().startswith(("import ", "from "))) + functions = sum(1 for line in lines if line.strip().startswith("def ")) + classes = sum(1 for line in lines if line.strip().startswith("class ")) + + return json.dumps({ + "total_lines": total, + "blank_lines": blank, + "comment_lines": comments, + "import_lines": imports, + "functions": functions, + "classes": classes, + "code_lines": total - blank - comments, + }, indent=2) + + +@tool +def merge_dicts(dict1_json: str, dict2_json: str) -> str: + """Merge two JSON dictionaries. Values from dict2 override dict1. + + Args: + dict1_json: First JSON dictionary + dict2_json: Second JSON dictionary (takes precedence) + + Returns: + Merged JSON dictionary + """ + try: + d1 = json.loads(dict1_json) + d2 = json.loads(dict2_json) + merged = {**d1, **d2} + return json.dumps(merged, indent=2, ensure_ascii=False) + except json.JSONDecodeError as e: + return f"Error: Invalid JSON - {e}" + + +def demo_custom_tools(): + """Demonstrate custom tool creation and usage.""" + print("=" * 60) + print("Part 1: Custom Tools") + print("=" * 60) + + # Register custom tools + registry = ToolRegistry() + registry.register(word_frequency) + registry.register(code_snippet_stats) + registry.register(merge_dicts) + + # Test word frequency + sample_text = "the quick brown fox jumps over the lazy dog the fox" + print(f"\n1. Word frequency analysis of: '{sample_text}'") + result = word_frequency.invoke({"text": sample_text, "top_n": 5}) + print(f" {result}") + + # Test code snippet stats + sample_code = '''import os +import sys + +# Main function +def main(): + """Entry point.""" + print("Hello, World!") + +class MyClass: + def method(self): + pass +''' + print("\n2. Code snippet statistics:") + result = code_snippet_stats.invoke({"code": sample_code}) + print(f" {result}") + + # Test merge dicts + dict1 = '{"name": "jojo", "version": "1.0"}' + dict2 = '{"version": "2.0", "author": "team"}' + print("\n3. Merging dictionaries:") + result = merge_dicts.invoke({"dict1_json": dict1, "dict2_json": dict2}) + print(f" {result}") + + # List registered tools + all_tools = registry.list_tools() + print(f"\n4. Total tools in registry: {len(all_tools)}") + + +# ============================================================================= +# Part 2: Sub-Agents +# ============================================================================= + + +def demo_sub_agents(): + """Demonstrate sub-agent creation and execution.""" + print("\n" + "=" * 60) + print("Part 2: Sub-Agents") + print("=" * 60) + + # Create an agent registry + registry = AgentRegistry() + + # Create agents with different configurations + print("\n1. Creating sub-agents...") + + code_agent = registry.create_agent( + name="code_analyst", + description="Analyzes code and provides improvement suggestions", + tools=["read_file", "grep_search", "glob_search"], + system_prompt=( + "You are a code analysis expert. " + "Review code for quality, security, and best practices." + ), + ) + print(f" - Created: {code_agent.config.name}") + + research_agent = registry.create_agent( + name="researcher", + description="Searches and summarizes information", + tools=["web_search", "read_file"], + system_prompt=( + "You are a research assistant. " + "Search for information and provide concise summaries." + ), + ) + print(f" - Created: {research_agent.config.name}") + + debug_agent = registry.create_agent( + name="debugger", + description="Helps debug code issues", + tools=["read_file", "run_command", "grep_search"], + system_prompt=( + "You are a debugging expert. " + "Help identify and fix code issues systematically." + ), + ) + print(f" - Created: {debug_agent.config.name}") + + # List all registered agents + agents = registry.list_agents() + print(f"\n2. Registered agents: {agents}") + + # Demonstrate AgentTool wrapping + print("\n3. Creating AgentTool wrappers...") + code_tool = AgentTool( + agent=code_agent, + name="agent_code_analyst", + description="Delegate code analysis tasks to a specialist agent", + ) + print(f" - Tool name: {code_tool.name}") + print(f" - Tool description: {code_tool.description}") + + # Show built-in agent factory + print("\n4. Built-in agent factories:") + review_tool = BuiltInAgents.code_review_agent() + print(f" - Code review agent: {review_tool.name}") + + research_tool = BuiltInAgents.research_agent() + print(f" - Research agent: {research_tool.name}") + + debug_tool = BuiltInAgents.debug_agent() + print(f" - Debug agent: {debug_tool.name}") + + # Demonstrate shared state + print("\n5. Sub-agent shared state:") + code_agent.set_shared_state("project_root", "/path/to/project") + code_agent.set_shared_state("language", "python") + print(f" - project_root: {code_agent.get_shared_state('project_root')}") + print(f" - language: {code_agent.get_shared_state('language')}") + + # Clean up + for name in agents: + registry.unregister(name) + print(f"\n6. Cleaned up {len(agents)} agents") + + +# ============================================================================= +# Part 3: Plugin System +# ============================================================================= + + +class MetricsPlugin(BasePlugin): + """A plugin that tracks tool call metrics. + + Demonstrates: + - Plugin lifecycle (on_load, on_unload) + - Hook registration for tool call tracking + - State management within a plugin + """ + + metadata = PluginMetadata( + name="metrics", + version="1.0.0", + description="Tracks tool call counts and timing", + author="jojo-code", + tags=["metrics", "monitoring"], + ) + permission = PluginPermission.UNTRUSTED + + def __init__(self): + super().__init__() + self._call_counts: dict[str, int] = {} + self._total_calls = 0 + + def on_load(self) -> None: + logger.info("[MetricsPlugin] Loaded - tracking tool calls") + + def on_unload(self) -> None: + logger.info(f"[MetricsPlugin] Unloaded - total calls: {self._total_calls}") + + def get_hooks(self) -> dict[str, Any]: + return { + HOOK_BEFORE_TOOL_CALL: self._track_call, + } + + def _track_call(self, tool_name: str, args: dict) -> None: + self._call_counts[tool_name] = self._call_counts.get(tool_name, 0) + 1 + self._total_calls += 1 + + def get_stats(self) -> dict: + return { + "total_calls": self._total_calls, + "by_tool": dict(self._call_counts), + } + + +class ValidationPlugin(BasePlugin): + """A plugin that validates tool arguments before execution. + + Demonstrates: + - Input validation via hooks + - Blocking dangerous operations + - Custom validation rules + """ + + metadata = PluginMetadata( + name="validation", + version="1.0.0", + description="Validates tool arguments for safety", + author="jojo-code", + tags=["security", "validation"], + ) + permission = PluginPermission.UNTRUSTED + + BLOCKED_PATTERNS = ["rm -rf", "DROP TABLE", "DELETE FROM"] + + def on_load(self) -> None: + logger.info("[ValidationPlugin] Loaded - enforcing validation rules") + + def on_unload(self) -> None: + logger.info("[ValidationPlugin] Unloaded") + + def get_hooks(self) -> dict[str, Any]: + return { + HOOK_BEFORE_TOOL_CALL: self._validate, + } + + def _validate(self, tool_name: str, args: dict) -> None: + """Check for dangerous patterns in tool arguments.""" + for key, value in args.items(): + if isinstance(value, str): + for pattern in self.BLOCKED_PATTERNS: + if pattern.lower() in value.lower(): + logger.warning( + f"[ValidationPlugin] Blocked pattern '{pattern}' " + f"in {tool_name}.{key}" + ) + + +def demo_plugins(): + """Demonstrate plugin system usage.""" + print("\n" + "=" * 60) + print("Part 3: Plugin System") + print("=" * 60) + + # Create plugin registry + registry = PluginRegistry.get_instance() + + # Create and register plugins + print("\n1. Registering plugins...") + metrics = MetricsPlugin() + validation = ValidationPlugin() + + registry.register("metrics", metrics) + registry.register("validation", validation) + + # List plugins + print("\n2. Registered plugins:") + for name in registry.list_plugins(): + p = registry.get(name) + print(f" - {name}: {p.metadata.description}") + + # Simulate hook dispatch + print("\n3. Simulating tool calls through hooks...") + from jojo_code.plugin.hooks import HookDispatcher + + dispatcher = HookDispatcher() + registry.set_dispatcher(dispatcher) + + # Simulate some tool calls + calls = [ + ("read_file", {"path": "/tmp/test.py"}), + ("run_command", {"command": "ls -la"}), + ("read_file", {"path": "/tmp/other.py"}), + ("write_file", {"path": "/tmp/out.txt", "content": "data"}), + ("run_command", {"command": "echo hello"}), + ] + + for tool_name, args in calls: + dispatcher.dispatch(HOOK_BEFORE_TOOL_CALL, tool_name, args) + + # Show metrics + print("\n4. Metrics collected:") + stats = metrics.get_stats() + print(f" Total calls: {stats['total_calls']}") + for tool_name, count in stats["by_tool"].items(): + print(f" - {tool_name}: {count}") + + # Test validation plugin + print("\n5. Validation plugin test:") + dispatcher.dispatch( + HOOK_BEFORE_TOOL_CALL, + "run_command", + {"command": "rm -rf /important/data"}, + ) + print(" (Warning logged for dangerous command)") + + # Unregister plugins + print("\n6. Cleaning up...") + registry.unregister("metrics") + registry.unregister("validation") + registry.clear() + + +# ============================================================================= +# Part 4: Integrated Workflow +# ============================================================================= + + +def demo_integrated_workflow(): + """Demonstrate all components working together.""" + print("\n" + "=" * 60) + print("Part 4: Integrated Workflow") + print("=" * 60) + + # Set up tool registry with custom tools + tool_registry = ToolRegistry() + tool_registry.register(word_frequency) + tool_registry.register(code_snippet_stats) + + # Set up agent with custom tools + agent_registry = AgentRegistry() + agent = agent_registry.create_agent( + name="analyst", + description="Analyzes text and code", + tools=["word_frequency", "code_snippet_stats"], + ) + + # Set up plugin for monitoring + plugin_registry = PluginRegistry.get_instance() + metrics = MetricsPlugin() + plugin_registry.register("metrics", metrics) + + print("\n1. Workflow setup complete:") + print(f" - Custom tools: {len(tool_registry.list_tools())}") + print(f" - Agents: {agent_registry.list_agents()}") + print(f" - Plugins: {plugin_registry.list_plugins()}") + + # Demonstrate the workflow + print("\n2. Running analysis workflow...") + + # Step 1: Analyze text + text = "Python is a great language. Python is versatile. Python is popular." + result = word_frequency.invoke({"text": text, "top_n": 3}) + print(f" Text analysis: {result}") + + # Step 2: Analyze code + code = "import os\ndef main():\n pass\n" + result = code_snippet_stats.invoke({"code": code}) + print(f" Code analysis: {result}") + + # Show agent config + print("\n3. Agent configuration:") + print(f" Name: {agent.config.name}") + print(f" Tools: {agent.config.tools}") + print(f" Model: {agent.config.model}") + print(f" Max iterations: {agent.config.max_iterations}") + + # Clean up + agent_registry.unregister("analyst") + plugin_registry.clear() + print("\n4. Workflow complete!") + + +# ============================================================================= +# Main +# ============================================================================= + + +def main(): + """Run all advanced usage demos.""" + print("jojo-Code Advanced Usage Example") + print("Demonstrating sub-agents, plugins, and custom tools\n") + + demo_custom_tools() + demo_sub_agents() + demo_plugins() + demo_integrated_workflow() + + print("\n" + "=" * 60) + print("All advanced demos completed!") + print("=" * 60) + + +if __name__ == "__main__": + main() diff --git a/examples/basic_usage.py b/examples/basic_usage.py new file mode 100644 index 0000000..71c0dda --- /dev/null +++ b/examples/basic_usage.py @@ -0,0 +1,267 @@ +"""jojo-Code basic usage example. + +Demonstrates the main features of jojo-Code: +- AgentOps tracing and metrics +- Data export (JSON and Markdown) +- CLI dashboard + +Run this script from the project root: + uv run python examples/basic_usage.py +""" + +import sys +import tempfile +from datetime import datetime +from pathlib import Path + +# Add project src to path for direct execution +sys.path.insert(0, str(Path(__file__).parent.parent / "src")) + +from jojo_code.ops import ( + Dashboard, + Exporter, + MetricsEngine, + SpanStatus, + SpanType, + Trace, +) +from jojo_code.ops.models import Span + + +def create_sample_traces() -> list[Trace]: + """Create sample traces to demonstrate the ops system.""" + traces = [] + + # Trace 1: Successful file read task + trace1 = Trace( + id="trace-001", + session_id="demo-session", + task="Read README.md and summarize", + start_time=datetime(2026, 1, 1, 10, 0, 0), + end_time=datetime(2026, 1, 1, 10, 0, 2), + status=SpanStatus.COMPLETED, + ) + trace1.spans = [ + Span( + trace_id="trace-001", + type=SpanType.THINKING, + name="thinking", + status=SpanStatus.COMPLETED, + start_time=datetime(2026, 1, 1, 10, 0, 0), + end_time=datetime(2026, 1, 1, 10, 0, 0, 500000), + ), + Span( + trace_id="trace-001", + type=SpanType.TOOL_CALL, + name="read_file", + status=SpanStatus.COMPLETED, + start_time=datetime(2026, 1, 1, 10, 0, 1), + end_time=datetime(2026, 1, 1, 10, 0, 1, 200000), + ), + Span( + trace_id="trace-001", + type=SpanType.THINKING, + name="thinking", + status=SpanStatus.COMPLETED, + start_time=datetime(2026, 1, 1, 10, 0, 1, 500000), + end_time=datetime(2026, 1, 1, 10, 0, 2), + ), + ] + traces.append(trace1) + + # Trace 2: Task with multiple tool calls + trace2 = Trace( + id="trace-002", + session_id="demo-session", + task="Search for TODO comments and list them", + start_time=datetime(2026, 1, 1, 10, 5, 0), + end_time=datetime(2026, 1, 1, 10, 5, 5), + status=SpanStatus.COMPLETED, + ) + trace2.spans = [ + Span( + trace_id="trace-002", + type=SpanType.THINKING, + name="thinking", + status=SpanStatus.COMPLETED, + start_time=datetime(2026, 1, 1, 10, 5, 0), + end_time=datetime(2026, 1, 1, 10, 5, 1), + ), + Span( + trace_id="trace-002", + type=SpanType.TOOL_CALL, + name="grep_search", + status=SpanStatus.COMPLETED, + start_time=datetime(2026, 1, 1, 10, 5, 1), + end_time=datetime(2026, 1, 1, 10, 5, 2), + ), + Span( + trace_id="trace-002", + type=SpanType.TOOL_CALL, + name="read_file", + status=SpanStatus.COMPLETED, + start_time=datetime(2026, 1, 1, 10, 5, 2), + end_time=datetime(2026, 1, 1, 10, 5, 3), + ), + Span( + trace_id="trace-002", + type=SpanType.TOOL_CALL, + name="read_file", + status=SpanStatus.COMPLETED, + start_time=datetime(2026, 1, 1, 10, 5, 3), + end_time=datetime(2026, 1, 1, 10, 5, 4), + ), + Span( + trace_id="trace-002", + type=SpanType.THINKING, + name="thinking", + status=SpanStatus.COMPLETED, + start_time=datetime(2026, 1, 1, 10, 5, 4), + end_time=datetime(2026, 1, 1, 10, 5, 5), + ), + ] + traces.append(trace2) + + # Trace 3: Failed task + trace3 = Trace( + id="trace-003", + session_id="demo-session", + task="Write to a protected file", + start_time=datetime(2026, 1, 1, 10, 10, 0), + end_time=datetime(2026, 1, 1, 10, 10, 3), + status=SpanStatus.FAILED, + ) + trace3.spans = [ + Span( + trace_id="trace-003", + type=SpanType.THINKING, + name="thinking", + status=SpanStatus.COMPLETED, + start_time=datetime(2026, 1, 1, 10, 10, 0), + end_time=datetime(2026, 1, 1, 10, 10, 1), + ), + Span( + trace_id="trace-003", + type=SpanType.TOOL_CALL, + name="write_file", + status=SpanStatus.FAILED, + error="PermissionError: Access denied", + start_time=datetime(2026, 1, 1, 10, 10, 1), + end_time=datetime(2026, 1, 1, 10, 10, 2), + ), + Span( + trace_id="trace-003", + type=SpanType.ERROR, + name="error", + status=SpanStatus.FAILED, + start_time=datetime(2026, 1, 1, 10, 10, 2), + end_time=datetime(2026, 1, 1, 10, 10, 3), + ), + ] + traces.append(trace3) + + return traces + + +def demo_metrics(traces: list[Trace]) -> None: + """Demonstrate metrics calculation.""" + print("=" * 60) + print("Metrics Engine Demo") + print("=" * 60) + + engine = MetricsEngine(traces) + summary = engine.calculate() + + print(f"\nTotal traces: {summary.total_traces}") + print(f"Completed: {summary.completed_traces}") + print(f"Failed: {summary.failed_traces}") + print(f"Task success rate: {summary.task_success_rate:.1%}") + print(f"Tool success rate: {summary.tool_success_rate:.1%}") + print(f"Avg thinking rounds: {summary.avg_thinking_rounds:.1f}") + print(f"Avg duration: {summary.avg_duration_ms:.0f}ms") + + print("\nTool usage ranking:") + for tool, count in engine.get_tool_usage_ranking(): + print(f" {tool}: {count}") + + print("\nPerformance stats:") + stats = engine.get_performance_stats() + for key, value in stats.items(): + print(f" {key}: {value}ms") + + +def demo_export(traces: list[Trace]) -> None: + """Demonstrate data export.""" + print("\n" + "=" * 60) + print("Export Demo") + print("=" * 60) + + with tempfile.TemporaryDirectory() as tmpdir: + # Export traces as JSON + json_path = str(Path(tmpdir) / "traces.json") + Exporter.export_traces_json(traces, json_path) + print(f"\nExported {len(traces)} traces to {json_path}") + + # Generate Markdown report + engine = MetricsEngine(traces) + summary = engine.calculate() + md_path = str(Path(tmpdir) / "report.md") + report = Exporter.export_summary_markdown( + total_traces=summary.total_traces, + completed_traces=summary.completed_traces, + failed_traces=summary.failed_traces, + avg_thinking_rounds=summary.avg_thinking_rounds, + avg_tool_calls=summary.avg_tool_calls, + avg_duration_ms=summary.avg_duration_ms, + tool_success_rate=summary.tool_success_rate, + task_success_rate=summary.task_success_rate, + tool_usage=summary.tool_usage, + error_types=summary.error_types, + output_path=md_path, + ) + print(f"Generated Markdown report at {md_path}") + print(f"Report length: {len(report)} characters") + + +def demo_dashboard(traces: list[Trace]) -> None: + """Demonstrate CLI dashboard.""" + print("\n" + "=" * 60) + print("Dashboard Demo") + print("=" * 60) + + dashboard = Dashboard() + + # Show traces list + print("\n--- Traces List ---") + dashboard.show_traces_list(traces) + + # Show metrics + engine = MetricsEngine(traces) + summary = engine.calculate() + print("\n--- Metrics Summary ---") + dashboard.show_metrics(summary) + + # Show individual trace metrics + print("\n--- Single Trace Metrics ---") + tm = engine.calculate_trace_metrics(traces[0]) + dashboard.show_trace_metrics(tm) + + +def main(): + """Run all demos.""" + print("jojo-Code Basic Usage Example") + print("Demonstrating AgentOps features\n") + + traces = create_sample_traces() + + demo_metrics(traces) + demo_export(traces) + demo_dashboard(traces) + + print("\n" + "=" * 60) + print("All demos completed successfully!") + print("=" * 60) + + +if __name__ == "__main__": + main() diff --git a/examples/custom_plugin.py b/examples/custom_plugin.py new file mode 100644 index 0000000..11b785b --- /dev/null +++ b/examples/custom_plugin.py @@ -0,0 +1,184 @@ +"""Custom Plugin Example - jojo-code + +This example shows how to create a custom plugin with tools, +hooks, and proper lifecycle management. + +Usage: + uv run python examples/custom_plugin.py +""" + +import logging +import time +from datetime import UTC +from typing import Any + +from jojo_code.plugin.base import BasePlugin, PluginMetadata, PluginPermission, PluginSandbox +from jojo_code.plugin.hooks import ( + HOOK_AFTER_TOOL_CALL, + HOOK_BEFORE_TOOL_CALL, + HookDispatcher, +) +from jojo_code.plugin.registry import PluginRegistry + +logging.basicConfig(level=logging.INFO, format="%(name)s - %(message)s") +logger = logging.getLogger(__name__) + + +# ============================================================================= +# Step 1: Define your plugin class +# ============================================================================= + + +class TimerPlugin(BasePlugin): + """A plugin that times tool calls and provides a timestamp tool. + + Demonstrates: + - Plugin metadata and permissions + - Providing custom tools via get_tools() + - Hook-based lifecycle monitoring + - Sandbox configuration + """ + + metadata = PluginMetadata( + name="timer", + version="1.0.0", + description="Times tool calls and provides timestamp utilities", + author="jojo-code-example", + tags=["utility", "timing"], + ) + + permission = PluginPermission.UNTRUSTED + sandbox = PluginSandbox(restricted=True) + + def __init__(self) -> None: + self._timings: dict[str, float] = {} + + def on_load(self) -> None: + logger.info("[TimerPlugin] Loaded - ready to time tool calls") + + def on_unload(self) -> None: + logger.info("[TimerPlugin] Unloaded - collected %d timing records", len(self._timings)) + + def get_tools(self) -> list: + """Return custom tools provided by this plugin.""" + from langchain_core.tools import tool + + @tool + def current_timestamp() -> str: + """Get the current Unix timestamp and formatted datetime. + + Returns: + A string with the current timestamp in multiple formats. + """ + now = time.time() + from datetime import datetime + + dt = datetime.fromtimestamp(now, tz=UTC) + return f"Unix: {now:.3f}\nUTC: {dt.isoformat()}\nLocal: {datetime.now().isoformat()}" + + @tool + def timing_report() -> str: + """Get a report of all timed tool calls. + + Returns: + A formatted report of tool call durations. + """ + if not self._timings: + return "No tool calls have been timed yet." + + lines = ["Tool Call Timings:", "-" * 40] + for name, duration in sorted(self._timings.items()): + lines.append(f" {name}: {duration:.3f}s") + total = sum(self._timings.values()) + lines.append("-" * 40) + lines.append(f" Total: {total:.3f}s ({len(self._timings)} calls)") + return "\n".join(lines) + + return [current_timestamp, timing_report] + + def get_hooks(self) -> dict[str, Any]: + """Register hook handlers for timing.""" + return { + HOOK_BEFORE_TOOL_CALL: self._start_timer, + HOOK_AFTER_TOOL_CALL: self._stop_timer, + } + + def _start_timer(self, tool_name: str, args: dict) -> None: + """Record the start time of a tool call.""" + self._timings[f"{tool_name}_start"] = time.time() + logger.info("[TimerPlugin] Starting timer for: %s", tool_name) + + def _stop_timer(self, tool_name: str, result: str) -> None: + """Record the duration of a tool call.""" + start_key = f"{tool_name}_start" + start_time = self._timings.pop(start_key, None) + if start_time is not None: + duration = time.time() - start_time + self._timings[tool_name] = duration + logger.info("[TimerPlugin] %s took %.3fs", tool_name, duration) + + +# ============================================================================= +# Step 2: Register and use the plugin +# ============================================================================= + + +def main() -> None: + print("=" * 60) + print("Custom Plugin Example: TimerPlugin") + print("=" * 60) + + # Get the registry singleton + registry = PluginRegistry.get_instance() + + # Set up hook dispatcher so hooks actually work + dispatcher = HookDispatcher() + registry.set_dispatcher(dispatcher) + + # Create and register the plugin + plugin = TimerPlugin() + registry.register("timer", plugin) + + # Verify it's registered + print(f"\nRegistered plugins: {registry.list_plugins()}") + + # Access the plugin + loaded = registry.get("timer") + assert loaded is plugin + print(f"Plugin metadata: {loaded.metadata.name} v{loaded.metadata.version}") + print(f"Permission: {loaded.permission.value}") + + # Get tools from the plugin + tools = loaded.get_tools() + print(f"\nTools provided: {[t.name for t in tools]}") + + # Use the timestamp tool + print("\n--- Using current_timestamp tool ---") + timestamp_tool = next(t for t in tools if t.name == "current_timestamp") + result = timestamp_tool.invoke({}) + print(result) + + # Simulate some timed operations + print("\n--- Simulating timed tool calls ---") + dispatcher.dispatch(HOOK_BEFORE_TOOL_CALL, "read_file", {"path": "test.py"}) + time.sleep(0.05) # Simulate work + dispatcher.dispatch(HOOK_AFTER_TOOL_CALL, "read_file", "file contents...") + + dispatcher.dispatch(HOOK_BEFORE_TOOL_CALL, "run_command", {"command": "ls"}) + time.sleep(0.03) # Simulate work + dispatcher.dispatch(HOOK_AFTER_TOOL_CALL, "run_command", "output...") + + # Get timing report + print("\n--- Using timing_report tool ---") + report_tool = next(t for t in tools if t.name == "timing_report") + print(report_tool.invoke({})) + + # Clean up + registry.unregister("timer") + print(f"\nRemaining plugins: {registry.list_plugins()}") + registry.clear() + print("Done!") + + +if __name__ == "__main__": + main() diff --git a/examples/deployment.py b/examples/deployment.py new file mode 100644 index 0000000..5231507 --- /dev/null +++ b/examples/deployment.py @@ -0,0 +1,334 @@ +"""Deployment Example - jojo-code + +This example demonstrates how to deploy jojo-code in different environments: +- Local CLI mode +- WebSocket server mode +- Docker container deployment + +Usage: + uv run python examples/deployment.py +""" + +import json +import os +import sys +from pathlib import Path + +# Add project src to path for direct execution +sys.path.insert(0, str(Path(__file__).parent.parent / "src")) + + +# ============================================================================= +# Example 1: Configuration Setup +# ============================================================================= + + +def demo_configuration(): + """Show how to configure jojo-code for different environments.""" + print("=" * 60) + print("Configuration Setup") + print("=" * 60) + + # Configuration options + configs = { + "development": { + "description": "Local development with debug logging", + "env": { + "OPENAI_API_KEY": "sk-dev-key", + "OPENAI_BASE_URL": "https://api.example.com/v1", + "JOJO_CODE_MODEL": "gpt-4o-mini", + "JOJO_CODE_LOG_LEVEL": "DEBUG", + }, + }, + "staging": { + "description": "Staging environment with standard settings", + "env": { + "OPENAI_API_KEY": "sk-staging-key", + "OPENAI_BASE_URL": "https://api.example.com/v1", + "JOJO_CODE_MODEL": "gpt-4o", + "JOJO_CODE_LOG_LEVEL": "INFO", + "JOJO_CODE_PORT": "8080", + }, + }, + "production": { + "description": "Production environment with strict settings", + "env": { + "OPENAI_API_KEY": "sk-prod-key", + "OPENAI_BASE_URL": "https://api.example.com/v1", + "JOJO_CODE_MODEL": "gpt-4o", + "JOJO_CODE_LOG_LEVEL": "WARNING", + "JOJO_CODE_LOG_FORMAT": "json", + "JOJO_CODE_PORT": "8080", + "JOJO_CODE_HOST": "0.0.0.0", + }, + }, + } + + for env_name, config in configs.items(): + print(f"\n--- {env_name.upper()} ---") + print(f" Description: {config['description']}") + print(" Environment variables:") + for key, value in config["env"].items(): + # Mask API keys for display + display_value = value if "KEY" not in key else value[:8] + "..." + print(f" {key}={display_value}") + + +# ============================================================================= +# Example 2: Docker Configuration +# ============================================================================= + + +def demo_docker_config(): + """Generate Docker-related configuration files.""" + print("\n" + "=" * 60) + print("Docker Configuration") + print("=" * 60) + + # Dockerfile content + dockerfile = """FROM python:3.12-slim + +WORKDIR /app + +# Install uv for fast dependency management +RUN pip install uv + +# Copy dependency files first for layer caching +COPY pyproject.toml uv.lock ./ +RUN uv sync --no-dev + +# Copy source code +COPY src/ src/ +COPY examples/ examples/ + +# Expose server port +EXPOSE 8080 + +# Health check +HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \\ + CMD curl -f http://localhost:8080/health || exit 1 + +# Default: start WebSocket server +CMD ["uv", "run", "python", "-m", "jojo_code.server.ws_server"] +""" + + # docker-compose.yml content + docker_compose = """version: "3.8" + +services: + jojo-code: + build: . + container_name: jojo-code + ports: + - "8080:8080" + volumes: + - ./:/workspace + - jojo-data:/root/.jojo-code + environment: + - OPENAI_API_KEY=${OPENAI_API_KEY} + - OPENAI_BASE_URL=${OPENAI_BASE_URL} + - JOJO_CODE_MODEL=${JOJO_CODE_MODEL:-gpt-4o-mini} + - JOJO_CODE_LOG_LEVEL=INFO + - JOJO_CODE_LOG_FORMAT=json + restart: unless-stopped + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:8080/health"] + interval: 30s + timeout: 5s + retries: 3 + +volumes: + jojo-data: +""" + + print("\n--- Dockerfile ---") + print(dockerfile) + + print("--- docker-compose.yml ---") + print(docker_compose) + + +# ============================================================================= +# Example 3: nginx Reverse Proxy +# ============================================================================= + + +def demo_nginx_config(): + """Generate nginx reverse proxy configuration.""" + print("\n" + "=" * 60) + print("nginx Reverse Proxy Configuration") + print("=" * 60) + + nginx_config = """server { + listen 443 ssl http2; + server_name jojo.example.com; + + # TLS configuration + ssl_certificate /etc/nginx/ssl/cert.pem; + ssl_certificate_key /etc/nginx/ssl/key.pem; + ssl_protocols TLSv1.2 TLSv1.3; + ssl_ciphers HIGH:!aNULL:!MD5; + + # WebSocket endpoint + location /ws { + proxy_pass http://127.0.0.1:8080; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; + proxy_read_timeout 86400; + proxy_send_timeout 86400; + } + + # SSE endpoints + location /sse/ { + proxy_pass http://127.0.0.1:8080; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header Connection ''; + proxy_http_version 1.1; + chunked_transfer_encoding off; + proxy_buffering off; + proxy_cache off; + } + + # REST API and health check + location / { + proxy_pass http://127.0.0.1:8080; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + } +} + +# HTTP to HTTPS redirect +server { + listen 80; + server_name jojo.example.com; + return 301 https://$host$request_uri; +} +""" + + print("\n--- /etc/nginx/sites-available/jojo-code ---") + print(nginx_config) + + +# ============================================================================= +# Example 4: Systemd Service +# ============================================================================= + + +def demo_systemd_config(): + """Generate systemd service configuration.""" + print("\n" + "=" * 60) + print("systemd Service Configuration") + print("=" * 60) + + systemd_config = """[Unit] +Description=jojo-code AI Coding Agent Server +After=network.target +Wants=network-online.target + +[Service] +Type=simple +User=jojo +Group=jojo +WorkingDirectory=/opt/jojo-code +EnvironmentFile=/opt/jojo-code/.env +ExecStart=/usr/local/bin/uv run python -m jojo_code.server.ws_server +Restart=on-failure +RestartSec=5 +StandardOutput=journal +StandardError=journal + +# Security hardening +NoNewPrivileges=true +ProtectSystem=strict +ProtectHome=true +ReadWritePaths=/opt/jojo-code /root/.jojo-code +PrivateTmp=true + +# Resource limits +LimitNOFILE=65536 +MemoryMax=2G + +[Install] +WantedBy=multi-user.target +""" + + print("\n--- /etc/systemd/system/jojo-code.service ---") + print(systemd_config) + + print("--- Usage commands ---") + print(" sudo systemctl daemon-reload") + print(" sudo systemctl enable jojo-code") + print(" sudo systemctl start jojo-code") + print(" sudo systemctl status jojo-code") + print(" sudo journalctl -u jojo-code -f") + + +# ============================================================================= +# Example 5: Health Check Script +# ============================================================================= + + +def demo_health_check(): + """Demonstrate health check and monitoring.""" + print("\n" + "=" * 60) + print("Health Check and Monitoring") + print("=" * 60) + + health_script = """#!/bin/bash +# jojo-code health check script +# Usage: ./health_check.sh [host:port] + +HOST=${1:-localhost:8080} +URL="http://${HOST}/health" + +response=$(curl -s -f "$URL" 2>/dev/null) +if [ $? -ne 0 ]; then + echo "CRITICAL: jojo-code server is not responding" + exit 2 +fi + +status=$(echo "$response" | python3 -c "import sys,json; print(json.load(sys.stdin)['status'])") +if [ "$status" = "healthy" ]; then + echo "OK: jojo-code server is healthy" + echo "$response" | python3 -m json.tool + exit 0 +else + echo "WARNING: jojo-code server status is $status" + exit 1 +fi +""" + + print("\n--- health_check.sh ---") + print(health_script) + + +# ============================================================================= +# Main +# ============================================================================= + + +def main(): + """Run all deployment demos.""" + print("jojo-Code Deployment Examples") + print("Showing configuration for different deployment scenarios\n") + + demo_configuration() + demo_docker_config() + demo_nginx_config() + demo_systemd_config() + demo_health_check() + + print("\n" + "=" * 60) + print("All deployment examples completed!") + print("=" * 60) + + +if __name__ == "__main__": + main() diff --git a/examples/mcp_integration.py b/examples/mcp_integration.py new file mode 100644 index 0000000..4b9d694 --- /dev/null +++ b/examples/mcp_integration.py @@ -0,0 +1,348 @@ +"""MCP Integration Example - jojo-code + +Demonstrates how to connect to MCP (Model Context Protocol) servers, +discover tools, call them, and manage resources. + +MCP allows jojo-code to use external tool servers that implement the +Model Context Protocol standard. This example shows both stdio and +HTTP transport modes. + +Usage: + uv run python examples/mcp_integration.py + +Note: This example uses mocked transport for demonstration. + In production, replace with real MCP server commands. +""" + +import asyncio +import logging +import sys +from pathlib import Path + +# Add project src to path for direct execution +sys.path.insert(0, str(Path(__file__).parent.parent / "src")) + +from jojo_code.mcp.client import ( + MCPClient, + MCPClientManager, + MCPConfig, + get_mcp_manager, +) + +logging.basicConfig(level=logging.INFO) +logger = logging.getLogger(__name__) + + +# ============================================================================= +# Part 1: Basic MCP Client Usage +# ============================================================================= + + +async def demo_basic_client(): + """Demonstrate basic MCP client lifecycle.""" + print("=" * 60) + print("Part 1: Basic MCP Client") + print("=" * 60) + + # Create a configuration for a stdio-based MCP server + config = MCPConfig( + name="filesystem", + url="npx -y @modelcontextprotocol/server-filesystem /tmp", + transport="stdio", + timeout=30.0, + retry=3, + ) + + print("\n1. Server config:") + print(f" Name: {config.name}") + print(f" URL: {config.url}") + print(f" Transport: {config.transport}") + print(f" Timeout: {config.timeout}s") + + # Create the client + client = MCPClient(config) + print(f"\n2. Client created, connected: {client.is_connected}") + + # In a real scenario, you would call: + # await client.connect() + # tools = client.list_tools() + # result = await client.call_tool("read_file", {"path": "/tmp/hello.txt"}) + # await client.close() + + print("\n3. (Skipping actual connection - requires real MCP server)") + print(" To connect: await client.connect()") + print(" To call: await client.call_tool('tool_name', {args})") + print(" To close: await client.close()") + + +# ============================================================================= +# Part 2: HTTP Transport +# ============================================================================= + + +async def demo_http_client(): + """Demonstrate HTTP-based MCP client.""" + print("\n" + "=" * 60) + print("Part 2: HTTP Transport") + print("=" * 60) + + # Configuration for an HTTP MCP server + config = MCPConfig( + name="remote-tools", + url="http://localhost:8080/mcp", + transport="http", + auth={"Authorization": "Bearer your-token-here"}, + timeout=60.0, + ) + + print("\n1. HTTP server config:") + print(f" Name: {config.name}") + print(f" URL: {config.url}") + print(f" Transport: {config.transport}") + print(f" Auth: {config.auth}") + + client = MCPClient(config) + print(f"\n2. Client created, connected: {client.is_connected}") + + print("\n3. (Skipping actual connection - requires running HTTP server)") + print(" To connect: await client.connect()") + print(" HTTP requests will be sent to:", config.url) + + +# ============================================================================= +# Part 3: Multi-Server Management +# ============================================================================= + + +async def demo_multi_server(): + """Demonstrate managing multiple MCP servers.""" + print("\n" + "=" * 60) + print("Part 3: Multi-Server Management") + print("=" * 60) + + manager = MCPClientManager() + + # Add multiple servers with different transports + servers = [ + MCPConfig( + name="filesystem", + url="npx -y @modelcontextprotocol/server-filesystem /home/user", + transport="stdio", + ), + MCPConfig( + name="database", + url="http://localhost:8081/mcp", + transport="http", + auth={"Authorization": "Bearer db-token"}, + ), + MCPConfig( + name="search", + url="http://localhost:8082/mcp", + transport="http", + ), + ] + + print("\n1. Adding servers to manager:") + for config in servers: + manager.add_server(config) + print(f" + {config.name} ({config.transport})") + + print(f"\n2. Registered servers: {manager.list_servers()}") + + # Get a specific client + fs_client = manager.get_client("filesystem") + print(f"\n3. Retrieved 'filesystem' client: {fs_client is not None}") + + # In a real scenario: + # await manager.connect_all() # Connect all servers + # await manager.close_all() # Close all connections + + print("\n4. (Skipping connect_all/close_all - requires real servers)") + print(" To connect all: await manager.connect_all()") + print(" To close all: await manager.close_all()") + + +# ============================================================================= +# Part 4: Working with Tools +# ============================================================================= + + +async def demo_tool_workflow(): + """Demonstrate a typical tool discovery and invocation workflow.""" + print("\n" + "=" * 60) + print("Part 4: Tool Workflow (Simulated)") + print("=" * 60) + + config = MCPConfig( + name="demo-server", + url="demo-server --stdio", + transport="stdio", + ) + client = MCPClient(config) + + # Simulate having discovered tools + from jojo_code.mcp.client import MCPTool + + client._tools = { + "read_file": MCPTool( + name="read_file", + description="Read a file from the filesystem", + input_schema={ + "type": "object", + "properties": { + "path": {"type": "string", "description": "File path to read"}, + }, + "required": ["path"], + }, + ), + "write_file": MCPTool( + name="write_file", + description="Write content to a file", + input_schema={ + "type": "object", + "properties": { + "path": {"type": "string", "description": "File path to write"}, + "content": {"type": "string", "description": "Content to write"}, + }, + "required": ["path", "content"], + }, + ), + "list_directory": MCPTool( + name="list_directory", + description="List files in a directory", + input_schema={ + "type": "object", + "properties": { + "path": {"type": "string", "description": "Directory path"}, + }, + "required": ["path"], + }, + ), + } + + print("\n1. Discovered tools:") + for tool in client.list_tools(): + print(f" - {tool.name}: {tool.description}") + if tool.input_schema: + required = tool.input_schema.get("required", []) + props = tool.input_schema.get("properties", {}) + params = ", ".join( + f"{k}{'*' if k in required else ''}" + for k in props + ) + print(f" Parameters: ({params})") + + # Look up a specific tool + tool = client.get_tool("read_file") + print(f"\n2. Lookup 'read_file': {tool.name if tool else 'not found'}") + + # Simulate calling a tool + print("\n3. Simulating tool call:") + print(' await client.call_tool("read_file", {"path": "/tmp/hello.txt"})') + print(" Result: {contents: [{text: 'Hello, World!'}]}") + + +# ============================================================================= +# Part 5: Resource Management +# ============================================================================= + + +async def demo_resources(): + """Demonstrate MCP resource listing and reading.""" + print("\n" + "=" * 60) + print("Part 5: Resource Management (Simulated)") + print("=" * 60) + + from jojo_code.mcp.client import MCPResource + + # Simulate discovered resources + resources = [ + MCPResource( + uri="file:///home/user/project/README.md", + name="README.md", + description="Project README file", + mime_type="text/markdown", + ), + MCPResource( + uri="file:///home/user/project/config.json", + name="config.json", + description="Project configuration", + mime_type="application/json", + ), + MCPResource( + uri="https://api.example.com/schema", + name="API Schema", + description="OpenAPI schema", + mime_type="application/json", + ), + ] + + print("\n1. Available resources:") + for res in resources: + print(f" - {res.name} ({res.mime_type})") + print(f" URI: {res.uri}") + if res.description: + print(f" Description: {res.description}") + + print("\n2. Reading a resource:") + print(' content = await client.read_resource("file:///home/user/project/README.md")') + print(" Result: '# My Project\\n\\nWelcome to the project.'") + + +# ============================================================================= +# Part 6: Convenience Functions +# ============================================================================= + + +async def demo_convenience(): + """Demonstrate the global convenience functions.""" + print("\n" + "=" * 60) + print("Part 6: Convenience Functions") + print("=" * 60) + + # add_mcp_server is a shorthand for creating config + adding to manager + print("\n1. Quick server addition:") + print(' client = add_mcp_server("quick", "http://localhost:9090/mcp")') + print(" This creates an MCPConfig and adds it to the global manager.") + + # get_mcp_manager returns the singleton manager + print("\n2. Global manager access:") + print(" manager = get_mcp_manager()") + print(" Returns the same MCPClientManager instance every time.") + + # Clean up the global manager (don't leave test data) + manager = get_mcp_manager() + for name in manager.list_servers(): + manager.remove_server(name) + print("\n3. Cleaned up global manager.") + + +# ============================================================================= +# Main +# ============================================================================= + + +async def main(): + """Run all MCP integration demos.""" + print("jojo-Code MCP Integration Example") + print("Demonstrating MCP protocol client capabilities\n") + + await demo_basic_client() + await demo_http_client() + await demo_multi_server() + await demo_tool_workflow() + await demo_resources() + await demo_convenience() + + print("\n" + "=" * 60) + print("All MCP demos completed!") + print("=" * 60) + print("\nTo use with a real MCP server:") + print(" 1. Install an MCP server (e.g., @modelcontextprotocol/server-filesystem)") + print(" 2. Create config with the server command") + print(" 3. Call await client.connect()") + print(" 4. Use client.list_tools() and client.call_tool()") + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/examples/memory_usage.py b/examples/memory_usage.py new file mode 100644 index 0000000..3af47f7 --- /dev/null +++ b/examples/memory_usage.py @@ -0,0 +1,354 @@ +"""jojo-Code memory system usage example. + +Demonstrates the memory management features: +- ShortTermMemory for session-scoped memory +- LongTermMemory for persistent cross-session storage +- MemoryRetriever for unified search +- SessionMemory as the recommended unified API + +Run this script from the project root: + uv run python examples/memory_usage.py +""" + +import sys +import tempfile +from pathlib import Path + +# Add project src to path for direct execution +sys.path.insert(0, str(Path(__file__).parent.parent / "src")) + + +def demo_session_memory() -> None: + """Demonstrate the SessionMemory unified API.""" + from jojo_code.memory import SessionMemory + + print("=" * 60) + print("SessionMemory Demo") + print("=" * 60) + + with tempfile.TemporaryDirectory() as tmpdir: + # Create a session with persistent long-term storage + memory = SessionMemory( + session_id="demo-session", + max_tokens=50000, + storage_dir=tmpdir, + ) + + # Add messages with different roles + memory.add_message("Hello, I need help with a Python project.", role="user") + memory.add_message( + "I'd be happy to help! What kind of project are you working on?", + role="ai", + ) + memory.add_message("It's a web scraper using BeautifulSoup.", role="user") + memory.add_message( + "Great choice! BeautifulSoup is excellent for HTML parsing.", + role="ai", + ) + memory.add_message("You are a Python expert assistant.", role="system") + + # Get context + context = memory.get_context() + print(f"\nTotal messages: {len(context)}") + for msg in context: + role = msg.__class__.__name__.replace("Message", "") + print(f" [{role}] {msg.content[:60]}...") + + # Get recent messages only + recent = memory.get_context(max_messages=3) + print(f"\nLast 3 messages: {len(recent)}") + + # Search across all memory + results = memory.search("Python", scope="all") + print("\nSearch results for 'Python':") + print(f" Current session: {len(results['current_session'])}") + print(f" History: {len(results['history'])}") + + # Get statistics + stats = memory.get_stats() + print("\nSession statistics:") + for key, value in stats.items(): + print(f" {key}: {value}") + + +def demo_short_term_memory() -> None: + """Demonstrate ShortTermMemory features.""" + from jojo_code.memory import ShortTermMemory + + print("\n" + "=" * 60) + print("ShortTermMemory Demo") + print("=" * 60) + + # Create short-term memory + stm = ShortTermMemory(session_id="short-demo", max_tokens=100000) + + # Add messages using convenience methods + stm.add_user_message("What is the best Python testing framework?") + stm.add_ai_message("pytest is the most popular Python testing framework.") + stm.add_user_message("How do I install it?") + stm.add_ai_message("Run: pip install pytest") + stm.add_system_message("You are a helpful assistant.") + + print(f"\nSession ID: {stm.session_id}") + print(f"Created at: {stm.created_at}") + print(f"Message count: {stm.message_count}") + + # Get messages by role + user_msgs = stm.get_messages_by_role("user") + ai_msgs = stm.get_messages_by_role("ai") + system_msgs = stm.get_messages_by_role("system") + print(f"\nUser messages: {len(user_msgs)}") + print(f"AI messages: {len(ai_msgs)}") + print(f"System messages: {len(system_msgs)}") + + # Search messages + results = stm.search("pytest") + print(f"\nSearch for 'pytest': {len(results)} matches") + for msg in results: + print(f" - {msg.content[:60]}...") + + # Case-sensitive search + results = stm.search("Python", case_sensitive=True) + print(f"\nCase-sensitive search for 'Python': {len(results)} matches") + + # Token count + tokens = stm.token_count() + print(f"\nToken count: {tokens}") + + # Get recent messages + last_2 = stm.get_last_n(2) + print(f"\nLast 2 messages: {len(last_2)}") + for msg in last_2: + print(f" - {msg.content[:60]}...") + + # Convert to memory items + items = stm.to_memory_items() + print(f"\nMemory items: {len(items)}") + for item in items: + print(f" [{item.metadata.get('role', 'unknown')}] {item.content[:50]}...") + + +def demo_long_term_memory() -> None: + """Demonstrate LongTermMemory features.""" + from jojo_code.memory import LongTermMemory + + print("\n" + "=" * 60) + print("LongTermMemory Demo") + print("=" * 60) + + with tempfile.TemporaryDirectory() as tmpdir: + # Create long-term memory + ltm = LongTermMemory( + storage_dir=tmpdir, + max_items=1000, + retention_days=30, + ) + + # Add memories with metadata and tags + ltm.add( + content="Learned that pytest supports parametrize for data-driven tests", + session_id="session-001", + tags=["python", "testing", "pytest"], + metadata={"importance": "high"}, + ) + + ltm.add( + content="BeautifulSoup uses parsers like lxml or html.parser", + session_id="session-001", + tags=["python", "scraping"], + metadata={"importance": "medium"}, + ) + + ltm.add( + content="FastAPI is a modern web framework for building APIs", + session_id="session-002", + tags=["python", "web", "api"], + metadata={"importance": "high"}, + ) + + ltm.add_message( + "The project uses uv for dependency management", + role="user", + session_id="session-002", + ) + + # List sessions + sessions = ltm.list_sessions() + print(f"\nSessions: {sessions}") + + # Get session memories + memories = ltm.get_session_memories("session-001") + print(f"\nSession-001 memories: {len(memories)}") + for mem in memories: + print(f" [{', '.join(mem.tags)}] {mem.content[:60]}...") + + # Search across sessions + results = ltm.search("Python", limit=5) + print(f"\nSearch for 'Python': {len(results)} results") + for result in results: + print(f" Score: {result.score:.2f} - {result.matched_content[:60]}...") + + # Search in specific session + results = ltm.search("testing", session_id="session-001") + print(f"\nSearch 'testing' in session-001: {len(results)} results") + + # Get statistics + stats = ltm.get_stats() + print("\nLong-term memory statistics:") + for key, value in stats.items(): + print(f" {key}: {value}") + + # Cleanup (nothing should be cleaned since data is fresh) + cleaned = ltm.cleanup(retention_days=90) + print(f"\nCleaned {cleaned} expired items") + + # Delete a session + deleted = ltm.delete_session("session-001") + print(f"Deleted session-001: {deleted}") + print(f"Remaining sessions: {ltm.list_sessions()}") + + +def demo_memory_retriever() -> None: + """Demonstrate MemoryRetriever unified search.""" + from jojo_code.memory import ( + LongTermMemory, + MemoryRetriever, + ShortTermMemory, + ) + + print("\n" + "=" * 60) + print("MemoryRetriever Demo") + print("=" * 60) + + with tempfile.TemporaryDirectory() as tmpdir: + # Create memory instances + stm = ShortTermMemory(session_id="retriever-demo") + ltm = LongTermMemory(storage_dir=tmpdir) + + # Populate short-term memory + stm.add_user_message("How do I use pytest fixtures?") + stm.add_ai_message("Fixtures are defined with @pytest.fixture decorator.") + stm.add_user_message("Can fixtures be scoped?") + stm.add_ai_message("Yes, fixtures can be function, class, module, or session scoped.") + + # Populate long-term memory + ltm.add( + content="pytest conftest.py is used for shared fixtures", + session_id="old-session", + tags=["pytest", "fixtures"], + ) + ltm.add( + content="unittest is Python's built-in testing framework", + session_id="old-session", + tags=["python", "testing"], + ) + + # Create retriever + retriever = MemoryRetriever(short_term=stm, long_term=ltm) + + # Search all memory + results = retriever.search("pytest", scope="all") + print("\nSearch 'pytest' (all):") + print(f" Current session: {len(results['current_session'])}") + print(f" History: {len(results['history'])}") + + # Search current session only + current = retriever.search_current_session("fixtures") + print(f"\nSearch 'fixtures' (current): {len(current)}") + + # Search history only + history = retriever.search_history("testing") + print(f"Search 'testing' (history): {len(history)}") + + # Get recent memories + recent = retriever.get_recent_memories(limit=5) + print(f"\nRecent memories: {len(recent)}") + for mem in recent: + print(f" [{mem.memory_type.value}] {mem.content[:60]}...") + + # Save current session to long-term + retriever.save_current_session() + print("\nSaved current session to long-term memory") + + # Verify it was saved + all_sessions = retriever.get_all_sessions() + print(f"All sessions: {all_sessions}") + + +def demo_conversation_memory() -> None: + """Demonstrate ConversationMemory (legacy API).""" + from langchain_core.messages import AIMessage, HumanMessage + + from jojo_code.memory import ConversationMemory + + print("\n" + "=" * 60) + print("ConversationMemory Demo (Legacy API)") + print("=" * 60) + + with tempfile.TemporaryDirectory() as tmpdir: + storage_path = Path(tmpdir) / "conversation.json" + + # Create with auto-save + memory = ConversationMemory( + max_tokens=100000, + storage_path=storage_path, + auto_save=True, + ) + + # Add messages + memory.add_message(HumanMessage(content="What is LangGraph?")) + memory.add_message( + AIMessage(content="LangGraph is a library for building stateful agents.") + ) + memory.add_message(HumanMessage(content="How does it work?")) + memory.add_message( + AIMessage(content="It uses a state machine graph to manage agent flow.") + ) + + print(f"\nMessages: {len(memory.messages)}") + print(f"Token count: {memory.token_count()}") + print(f"Storage path: {memory.storage_path}") + + # Get context + context = memory.get_context() + print(f"\nContext messages: {len(context)}") + + # Get recent messages + last_2 = memory.get_last_n_messages(2) + print(f"Last 2 messages: {len(last_2)}") + + # Save and reload + memory.save() + print(f"Saved to: {storage_path}") + print(f"File exists: {storage_path.exists()}") + + # Create new instance and load + memory2 = ConversationMemory( + max_tokens=100000, + storage_path=storage_path, + ) + print(f"Loaded messages: {len(memory2.messages)}") + + # Clear + memory2.clear() + print(f"After clear: {len(memory2.messages)} messages") + + +def main() -> None: + """Run all memory system demos.""" + print("jojo-Code Memory System Usage Example") + print("Demonstrating memory management features\n") + + demo_session_memory() + demo_short_term_memory() + demo_long_term_memory() + demo_memory_retriever() + demo_conversation_memory() + + print("\n" + "=" * 60) + print("All memory demos completed successfully!") + print("=" * 60) + + +if __name__ == "__main__": + main() diff --git a/examples/monitoring.py b/examples/monitoring.py new file mode 100644 index 0000000..868da7f --- /dev/null +++ b/examples/monitoring.py @@ -0,0 +1,300 @@ +"""jojo-Code monitoring example. + +Demonstrates how to use the AgentOps monitoring system for: +- Real-time trace collection during agent execution +- Metrics calculation and analysis +- Evaluation with built-in evaluators +- Report generation +- CLI dashboard visualization + +Run this script from the project root: + uv run python examples/monitoring.py +""" + +import sys +import tempfile +from pathlib import Path + +# Add project src to path for direct execution +sys.path.insert(0, str(Path(__file__).parent.parent / "src")) + +from jojo_code.ops import ( + Collector, + CompositeEvaluator, + Dashboard, + Exporter, + MetricsEngine, + PerformanceEvaluator, + PlanningEvaluator, + ReportGenerator, + SpanStatus, +) +from jojo_code.ops.config import OpsConfig + + +def demo_collector() -> list: + """Demonstrate real-time trace collection.""" + print("=" * 60) + print("Collector Demo - Simulating Agent Execution") + print("=" * 60) + + config = OpsConfig(enabled=True, persist_traces=False, real_time_display=False) + collector = Collector(config) + + # Simulate task 1: Successful file operations + collector.start_trace("Read and analyze README.md", session_id="session-001") + + # Thinking phase + collector.record_thinking("I need to read the README first") + + # Tool call: read file + collector.record_tool_call( + "read_file", + {"path": "README.md"}, + result="# Project README\nA coding agent built with LangGraph.", + ) + + # Another thinking phase + collector.record_thinking("Now I'll search for key sections") + + # Tool call: search + collector.record_tool_call( + "grep_search", + {"pattern": "## ", "path": "README.md"}, + result="## Overview\n## Installation\n## Usage", + ) + + collector.end_trace(status=SpanStatus.COMPLETED) + print("\n [Trace 1] Completed successfully") + + # Simulate task 2: Task with errors + collector.start_trace("Write configuration file", session_id="session-001") + + collector.record_thinking("I'll write the config file") + collector.record_tool_call( + "write_file", + {"path": "/etc/app/config.yaml", "content": "debug: true"}, + error="PermissionError: Access denied to /etc/app/", + ) + collector.record_error("PermissionError: Cannot write to protected directory") + + collector.end_trace(status=SpanStatus.FAILED) + print(" [Trace 2] Failed with permission error") + + # Simulate task 3: Complex multi-step task + collector.start_trace("Refactor authentication module", session_id="session-002") + + collector.record_thinking("Planning the refactoring approach") + collector.record_tool_call( + "read_file", {"path": "src/auth.py"}, result="class Auth: ..." + ) + collector.record_tool_call( + "read_file", {"path": "tests/test_auth.py"}, result="def test_auth: ..." + ) + collector.record_tool_call( + "grep_search", {"pattern": "import auth"}, result="3 files found" + ) + collector.record_thinking("Now I'll make the changes") + collector.record_tool_call( + "write_file", + {"path": "src/auth.py", "content": "class AuthService: ..."}, + ) + collector.record_tool_call( + "run_command", + {"command": "pytest tests/test_auth.py"}, + result="All tests passed", + ) + + collector.end_trace(status=SpanStatus.COMPLETED) + print(" [Trace 3] Completed with multiple tool calls") + + traces = collector.get_all_traces() + print(f"\n Total traces collected: {len(traces)}") + return traces + + +def demo_metrics_engine(traces: list) -> None: + """Demonstrate metrics calculation and analysis.""" + print("\n" + "=" * 60) + print("Metrics Engine Demo") + print("=" * 60) + + engine = MetricsEngine(traces) + summary = engine.calculate() + + print(f"\n Total traces: {summary.total_traces}") + print(f" Completed: {summary.completed_traces}") + print(f" Failed: {summary.failed_traces}") + print(f" Task success rate:{summary.task_success_rate:.1%}") + print(f" Tool success rate:{summary.tool_success_rate:.1%}") + print(f" Avg thinking: {summary.avg_thinking_rounds:.1f} rounds") + print(f" Avg duration: {summary.avg_duration_ms:.0f}ms") + + print("\n Tool usage ranking:") + for tool, count in engine.get_tool_usage_ranking(): + print(f" {tool}: {count} calls") + + print("\n Performance stats:") + stats = engine.get_performance_stats() + for key, value in stats.items(): + print(f" {key}: {value}ms") + + # Filtering + completed = engine.filter_by_status(SpanStatus.COMPLETED) + print(f"\n Completed traces: {len(completed)}") + + session_traces = engine.filter_by_session("session-001") + print(f" Session-001 traces: {len(session_traces)}") + + +def demo_evaluation(traces: list) -> None: + """Demonstrate the evaluation system.""" + print("\n" + "=" * 60) + print("Evaluation Demo") + print("=" * 60) + + # Planning evaluator + planning_eval = PlanningEvaluator( + max_thinking_rounds=5, + min_tool_success_rate=0.8, + max_errors=0, + ) + + print("\n Planning Evaluation Results:") + for trace in traces: + score = planning_eval.evaluate(trace) + print(f" [{trace.id}] {score.result.value}: {score.score:.2f} - {score.reason}") + + # Performance evaluator + perf_eval = PerformanceEvaluator( + max_duration_ms=30000, + max_thinking_rounds=5, + max_tool_calls=10, + ) + + print("\n Performance Evaluation Results:") + for trace in traces: + score = perf_eval.evaluate(trace) + print(f" [{trace.id}] {score.result.value}: {score.score:.2f} - {score.reason}") + + # Composite evaluator + composite = CompositeEvaluator( + evaluators=[planning_eval, perf_eval], + weights=[0.6, 0.4], + ) + + print("\n Composite Evaluation Results:") + for trace in traces: + score = composite.evaluate(trace) + print(f" [{trace.id}] {score.result.value}: {score.score:.2f} - {score.reason}") + + +def demo_report_generation(traces: list) -> None: + """Demonstrate report generation.""" + print("\n" + "=" * 60) + print("Report Generation Demo") + print("=" * 60) + + planning_eval = PlanningEvaluator() + + with tempfile.TemporaryDirectory() as tmpdir: + # Generate individual trace reports + print("\n Generating per-trace evaluation reports...") + for trace in traces: + score = planning_eval.evaluate(trace) + report_path = str(Path(tmpdir) / f"report-{trace.id}.md") + report = ReportGenerator.generate_evaluation_report( + trace, score, output_path=report_path + ) + print(f" [{trace.id}] Report: {len(report)} chars -> {report_path}") + + # Generate summary report + engine = MetricsEngine(traces) + summary = engine.calculate() + + scores = [planning_eval.evaluate(t) for t in traces] + summary_path = str(Path(tmpdir) / "summary-report.md") + summary_report = ReportGenerator.generate_summary_report( + summary, + evaluation_scores=scores, + output_path=summary_path, + ) + print(f"\n Summary report: {len(summary_report)} chars -> {summary_path}") + + # Export traces as JSON + json_path = str(Path(tmpdir) / "traces.json") + Exporter.export_traces_json(traces, json_path) + print(f" Traces JSON exported -> {json_path}") + + # Export summary as standalone Markdown + md_path = str(Path(tmpdir) / "metrics-report.md") + Exporter.export_summary_markdown( + total_traces=summary.total_traces, + completed_traces=summary.completed_traces, + failed_traces=summary.failed_traces, + avg_thinking_rounds=summary.avg_thinking_rounds, + avg_tool_calls=summary.avg_tool_calls, + avg_duration_ms=summary.avg_duration_ms, + tool_success_rate=summary.tool_success_rate, + task_success_rate=summary.task_success_rate, + tool_usage=summary.tool_usage, + error_types=summary.error_types, + output_path=md_path, + ) + print(f" Metrics report exported -> {md_path}") + + +def demo_dashboard(traces: list) -> None: + """Demonstrate CLI dashboard.""" + print("\n" + "=" * 60) + print("Dashboard Demo") + print("=" * 60) + + dashboard = Dashboard() + engine = MetricsEngine(traces) + summary = engine.calculate() + + print("\n --- Traces List ---") + dashboard.show_traces_list(traces) + + print("\n --- Metrics Summary ---") + dashboard.show_metrics(summary) + + print("\n --- Single Trace Metrics ---") + tm = engine.calculate_trace_metrics(traces[0]) + dashboard.show_trace_metrics(tm) + + print("\n --- Status Messages ---") + dashboard.print_success("Agent started successfully") + dashboard.print_info("Processing 3 tasks...") + dashboard.print_warning("Memory usage at 85%") + dashboard.print_error("Connection timeout to LLM API") + + +def main(): + """Run all monitoring demos.""" + print("jojo-Code Monitoring Example") + print("Demonstrating the AgentOps monitoring system\n") + + # Collect traces + traces = demo_collector() + + # Analyze metrics + demo_metrics_engine(traces) + + # Evaluate traces + demo_evaluation(traces) + + # Generate reports + demo_report_generation(traces) + + # Show dashboard + demo_dashboard(traces) + + print("\n" + "=" * 60) + print("All monitoring demos completed!") + print("=" * 60) + + +if __name__ == "__main__": + main() diff --git a/examples/performance_tuning.py b/examples/performance_tuning.py new file mode 100644 index 0000000..9316afb --- /dev/null +++ b/examples/performance_tuning.py @@ -0,0 +1,426 @@ +"""Performance Tuning Example - jojo-code + +Demonstrates how to optimize jojo-code for different workloads: +- Model selection and token budgeting +- Rate limiting and quota management +- Caching strategies +- Concurrency control +- AgentOps metrics for identifying bottlenecks + +Usage: + uv run python examples/performance_tuning.py +""" + +import asyncio +import sys +import time +from datetime import datetime +from pathlib import Path + +# Add project src to path for direct execution +sys.path.insert(0, str(Path(__file__).parent.parent / "src")) + + +# ============================================================================= +# Part 1: Model Selection & Token Budgeting +# ============================================================================= + + +def demo_model_selection(): + """Show how model choice affects speed and cost.""" + print("=" * 60) + print("Part 1: Model Selection & Token Budgeting") + print("=" * 60) + + # Model characteristics for common workloads + models = { + "gpt-4o-mini": { + "speed": "fast", + "cost_per_1k_input": 0.00015, + "cost_per_1k_output": 0.0006, + "best_for": "Simple tasks, code formatting, Q&A", + }, + "gpt-4o": { + "speed": "medium", + "cost_per_1k_input": 0.0025, + "cost_per_1k_output": 0.01, + "best_for": "Complex reasoning, code review, architecture", + }, + "claude-sonnet-4-20250514": { + "speed": "medium", + "cost_per_1k_input": 0.003, + "cost_per_1k_output": 0.015, + "best_for": "Long context, nuanced analysis, writing", + }, + } + + print("\nModel comparison for a typical 10-round agent session:") + print(f"{'Model':<28} {'Est. Cost':<12} {'Best For'}") + print("-" * 70) + + for name, info in models.items(): + # Estimate: ~2k input + ~1k output per round, 10 rounds + est_cost = (info["cost_per_1k_input"] * 2 + info["cost_per_1k_output"]) * 10 + print(f" {name:<26} ${est_cost:<10.4f} {info['best_for']}") + + print("\nRecommendation:") + print(" - Use gpt-4o-mini for routine tasks (formatting, simple Q&A)") + print(" - Use gpt-4o/claude-sonnet for complex multi-step tasks") + print(" - Set max_tokens to limit output length and cost") + + +# ============================================================================= +# Part 2: Rate Limiting & Quota Management +# ============================================================================= + + +async def demo_rate_limiting(): + """Demonstrate rate limiting for API protection.""" + print("\n" + "=" * 60) + print("Part 2: Rate Limiting & Quota Management") + print("=" * 60) + + from jojo_code.core.ratelimit import ( + LimitAlgorithm, + QuotaConfig, + QuotaManager, + RateLimitConfig, + RateLimiter, + ) + + # Scenario 1: Token bucket for bursty workloads + print("\n1. Token Bucket (bursty workload):") + config = RateLimitConfig(requests=5, window=10, algorithm=LimitAlgorithm.TOKEN_BUCKET) + limiter = RateLimiter(config) + + allowed = 0 + denied = 0 + for _ in range(8): + if await limiter.check(): + allowed += 1 + else: + denied += 1 + print(f" 8 rapid requests: {allowed} allowed, {denied} denied") + + # Scenario 2: Sliding window for steady workloads + print("\n2. Sliding Window (steady workload):") + config = RateLimitConfig(requests=3, window=5, algorithm=LimitAlgorithm.SLIDING_WINDOW) + limiter = RateLimiter(config) + + allowed = 0 + for _ in range(5): + if await limiter.check(): + allowed += 1 + print(f" 5 requests in quick succession: {allowed} allowed") + + # Scenario 3: Quota management for budget control + print("\n3. Quota Management (budget control):") + quota_mgr = QuotaManager() + quota_mgr.register_quota("daily_tokens", QuotaConfig(limit=100000, period=86400)) + quota_mgr.register_quota("api_calls", QuotaConfig(limit=500, period=3600)) + + await quota_mgr.consume_quota("daily_tokens", amount=45000) + await quota_mgr.consume_quota("api_calls", amount=120) + + remaining_tokens = await quota_mgr.get_remaining("daily_tokens") + remaining_calls = await quota_mgr.get_remaining("api_calls") + print(f" Daily tokens remaining: {remaining_tokens:,}") + print(f" API calls remaining: {remaining_calls}") + + print("\nTuning tips:") + print(" - Set rate limits slightly below your API provider's limits") + print(" - Use quotas to prevent runaway costs") + print(" - Monitor usage with AgentOps metrics") + + +# ============================================================================= +# Part 3: Concurrency Control +# ============================================================================= + + +async def demo_concurrency(): + """Show how to control concurrent agent executions.""" + print("\n" + "=" * 60) + print("Part 3: Concurrency Control") + print("=" * 60) + + # Simulate concurrent task execution with semaphore + print("\n1. Semaphore-based concurrency limiting:") + + results = [] + + async def simulated_task(task_id: int, sem: asyncio.Semaphore): + async with sem: + start = time.monotonic() + await asyncio.sleep(0.05) # Simulate work + elapsed = time.monotonic() - start + results.append((task_id, elapsed)) + return task_id + + # Run 10 tasks with max 3 concurrent + sem = asyncio.Semaphore(3) + start = time.monotonic() + await asyncio.gather(*[simulated_task(i, sem) for i in range(10)]) + total = time.monotonic() - start + + print(f" 10 tasks, max 3 concurrent: {total:.2f}s total") + print(f" Completed: {len(results)} tasks") + + # Show the tradeoff + print("\n2. Concurrency vs. rate limit tradeoff:") + configs = [ + ("Conservative", 1, "Safe for all API tiers"), + ("Balanced", 3, "Good default for most workloads"), + ("Aggressive", 10, "Requires high API tier, fast hardware"), + ] + print(f" {'Profile':<16} {'Concurrency':<14} {'Notes'}") + print(" " + "-" * 55) + for name, conc, note in configs: + print(f" {name:<16} {conc:<14} {note}") + + +# ============================================================================= +# Part 4: Caching Strategies +# ============================================================================= + + +def demo_caching(): + """Demonstrate caching patterns for repeated operations.""" + print("\n" + "=" * 60) + print("Part 4: Caching Strategies") + print("=" * 60) + + # Simple LRU cache simulation + cache: dict[str, tuple[str, float]] = {} + cache_ttl = 300 # 5 minutes + + def cached_search(query: str) -> str: + """Simulate a cached web search.""" + now = time.time() + if query in cache: + result, ts = cache[query] + if now - ts < cache_ttl: + return f"[CACHE HIT] {result}" + + # Simulate API call + result = f"Results for: {query}" + cache[query] = (result, now) + return f"[API CALL] {result}" + + # Demonstrate cache hits + queries = ["python async", "langgraph tutorial", "python async"] + + print("\n1. Response caching:") + for q in queries: + result = cached_search(q) + print(f" '{q}' -> {result}") + + print(f"\n Cache size: {len(cache)} entries") + + # Cache configuration recommendations + print("\n2. Caching recommendations by workload:") + configs = [ + ("Code analysis", "TTL=10min", "Files change frequently"), + ("Web search", "TTL=5min", "Results are relatively fresh"), + ("Documentation lookup", "TTL=1h", "Docs change rarely"), + ("Config reading", "TTL=30s", "Config may change during session"), + ] + print(f" {'Workload':<22} {'TTL':<14} {'Reason'}") + print(" " + "-" * 60) + for workload, ttl, reason in configs: + print(f" {workload:<22} {ttl:<14} {reason}") + + +# ============================================================================= +# Part 5: AgentOps Metrics for Bottleneck Identification +# ============================================================================= + + +def demo_bottleneck_analysis(): + """Use AgentOps metrics to find performance bottlenecks.""" + print("\n" + "=" * 60) + print("Part 5: Bottleneck Identification with AgentOps") + print("=" * 60) + + from jojo_code.ops import MetricsEngine, SpanStatus, SpanType, Trace + from jojo_code.ops.models import Span + + # Simulate a session with various timings + trace = Trace( + id="perf-trace", + session_id="perf-session", + task="Refactor database module", + start_time=datetime(2026, 1, 1, 10, 0, 0), + end_time=datetime(2026, 1, 1, 10, 2, 30), + status=SpanStatus.COMPLETED, + ) + + # Simulate spans with realistic timings + spans = [ + Span( + trace_id="perf-trace", type=SpanType.THINKING, name="thinking", + status=SpanStatus.COMPLETED, + start_time=datetime(2026, 1, 1, 10, 0, 0), + end_time=datetime(2026, 1, 1, 10, 0, 3), # 3s thinking + ), + Span( + trace_id="perf-trace", type=SpanType.TOOL_CALL, name="read_file", + status=SpanStatus.COMPLETED, + start_time=datetime(2026, 1, 1, 10, 0, 3), + end_time=datetime(2026, 1, 1, 10, 0, 4), # 1s file read + ), + Span( + trace_id="perf-trace", type=SpanType.TOOL_CALL, name="grep_search", + status=SpanStatus.COMPLETED, + start_time=datetime(2026, 1, 1, 10, 0, 4), + end_time=datetime(2026, 1, 1, 10, 0, 8), # 4s grep (slow!) + ), + Span( + trace_id="perf-trace", type=SpanType.THINKING, name="thinking", + status=SpanStatus.COMPLETED, + start_time=datetime(2026, 1, 1, 10, 0, 8), + end_time=datetime(2026, 1, 1, 10, 0, 15), # 7s thinking + ), + Span( + trace_id="perf-trace", type=SpanType.TOOL_CALL, name="run_command", + status=SpanStatus.COMPLETED, + start_time=datetime(2026, 1, 1, 10, 0, 15), + end_time=datetime(2026, 1, 1, 10, 1, 15), # 60s command (very slow!) + ), + Span( + trace_id="perf-trace", type=SpanType.TOOL_CALL, name="run_command", + status=SpanStatus.FAILED, + error="TimeoutError: Command exceeded 30s limit", + start_time=datetime(2026, 1, 1, 10, 1, 15), + end_time=datetime(2026, 1, 1, 10, 1, 45), # 30s timeout + ), + Span( + trace_id="perf-trace", type=SpanType.THINKING, name="thinking", + status=SpanStatus.COMPLETED, + start_time=datetime(2026, 1, 1, 10, 1, 45), + end_time=datetime(2026, 1, 1, 10, 2, 0), # 15s thinking (recovery) + ), + Span( + trace_id="perf-trace", type=SpanType.TOOL_CALL, name="read_file", + status=SpanStatus.COMPLETED, + start_time=datetime(2026, 1, 1, 10, 2, 0), + end_time=datetime(2026, 1, 1, 10, 2, 1), # 1s + ), + Span( + trace_id="perf-trace", type=SpanType.TOOL_CALL, name="write_file", + status=SpanStatus.COMPLETED, + start_time=datetime(2026, 1, 1, 10, 2, 1), + end_time=datetime(2026, 1, 1, 10, 2, 2), # 1s + ), + ] + trace.spans = spans + + # Analyze with MetricsEngine + engine = MetricsEngine([trace]) + summary = engine.calculate() + + print("\n1. Session summary:") + print(f" Total duration: {summary.avg_duration_ms / 1000:.0f}s") + print(f" Thinking rounds: {summary.avg_thinking_rounds:.0f}") + print(f" Tool calls: {summary.avg_tool_calls:.0f}") + print(f" Tool success rate: {summary.tool_success_rate:.0%}") + + print("\n2. Tool usage breakdown:") + for tool, count in sorted(summary.tool_usage.items(), key=lambda x: -x[1]): + print(f" {tool:<16} {count} calls") + + print("\n3. Errors encountered:") + for error, count in summary.error_types.items(): + print(f" {error:<50} x{count}") + + print("\n4. Performance recommendations based on this trace:") + print(" - run_command took 60s + 30s timeout = 90s (60% of session)") + print(" -> Set shorter command timeouts, or use background tasks") + print(" - grep_search took 4s on large codebase") + print(" -> Use glob_search first to narrow file scope") + print(" - 3 thinking rounds total, last one was 15s (error recovery)") + print(" -> Improve error handling to reduce recovery time") + + +# ============================================================================= +# Part 6: Configuration Templates +# ============================================================================= + + +def demo_config_templates(): + """Show recommended configurations for different workloads.""" + print("\n" + "=" * 60) + print("Part 6: Recommended Configurations") + print("=" * 60) + + configs = { + "Quick Q&A": { + "model": "gpt-4o-mini", + "max_iterations": 10, + "max_tokens": 50000, + "description": "Fast responses for simple questions", + }, + "Code Review": { + "model": "gpt-4o", + "max_iterations": 30, + "max_tokens": 100000, + "description": "Thorough analysis with moderate depth", + }, + "Large Refactor": { + "model": "claude-sonnet-4-20250514", + "max_iterations": 50, + "max_tokens": 200000, + "description": "Long context for understanding entire modules", + }, + "CI/CD Pipeline": { + "model": "gpt-4o-mini", + "max_iterations": 15, + "max_tokens": 50000, + "description": "Cost-effective for automated checks", + }, + } + + for name, config in configs.items(): + print(f"\n {name}:") + print(f" Model: {config['model']}") + print(f" Max iterations: {config['max_iterations']}") + print(f" Max tokens: {config['max_tokens']:,}") + print(f" Use case: {config['description']}") + + print("\n To apply a configuration:") + print(" jojo-code config set model gpt-4o") + print(" jojo-code config set max_iterations 30") + print(" # Or set via environment variables:") + print(" export JOJO_CODE_MODEL=gpt-4o") + print(" export JOJO_CODE_MAX_ITERATIONS=30") + + +# ============================================================================= +# Main +# ============================================================================= + + +def main(): + """Run all performance tuning demos.""" + print("jojo-Code Performance Tuning Guide") + print("Optimizing for different workloads\n") + + demo_model_selection() + asyncio.run(demo_rate_limiting()) + asyncio.run(demo_concurrency()) + demo_caching() + demo_bottleneck_analysis() + demo_config_templates() + + print("\n" + "=" * 60) + print("Performance tuning guide complete!") + print("=" * 60) + print("\nNext steps:") + print(" 1. Run 'jojo-code --trace' to collect real performance data") + print(" 2. Use 'jojo-code --metrics' to analyze bottlenecks") + print(" 3. Adjust model and limits based on your workload") + print(" 4. Set up rate limiting for production deployments") + + +if __name__ == "__main__": + main() diff --git a/examples/plugin_development.py b/examples/plugin_development.py new file mode 100644 index 0000000..8f9669f --- /dev/null +++ b/examples/plugin_development.py @@ -0,0 +1,312 @@ +"""Plugin Development Example - jojo-code + +This example demonstrates how to create a custom plugin for jojo-code. +Plugins can provide tools, hook into lifecycle events, and extend functionality. + +Usage: + uv run python examples/plugin_development.py +""" + +import logging +from typing import Any + +from jojo_code.plugin.base import BasePlugin, PluginMetadata, PluginPermission, PluginSandbox +from jojo_code.plugin.hooks import ( + HOOK_AFTER_TOOL_CALL, + HOOK_BEFORE_TOOL_CALL, + HookDispatcher, +) +from jojo_code.plugin.registry import PluginRegistry + +logging.basicConfig(level=logging.INFO) +logger = logging.getLogger(__name__) + + +# ============================================================================= +# Example 1: Simple Plugin with Hooks +# ============================================================================= + + +class LoggingPlugin(BasePlugin): + """A plugin that logs all tool calls for debugging. + + This plugin demonstrates: + - Defining plugin metadata + - Implementing lifecycle methods (on_load, on_unload) + - Registering hook handlers + """ + + metadata = PluginMetadata( + name="logging", + version="1.0.0", + description="Logs all tool calls for debugging purposes", + author="jojo-code", + tags=["debug", "logging"], + ) + + # Plugins are untrusted by default (no file/network access) + permission = PluginPermission.UNTRUSTED + + def on_load(self) -> None: + """Called when the plugin is loaded into the registry.""" + logger.info("[LoggingPlugin] Plugin loaded - will log tool calls") + + def on_unload(self) -> None: + """Called when the plugin is unloaded from the registry.""" + logger.info("[LoggingPlugin] Plugin unloaded") + + def get_hooks(self) -> dict[str, Any]: + """Return hook handlers for lifecycle events. + + Available hooks: + - before_tool_call: Called before a tool executes + - after_tool_call: Called after a tool executes + - before_agent_run: Called before agent starts + - after_agent_run: Called after agent finishes + - on_error: Called when an error occurs + """ + return { + HOOK_BEFORE_TOOL_CALL: self._on_before_tool_call, + HOOK_AFTER_TOOL_CALL: self._on_after_tool_call, + } + + def _on_before_tool_call(self, tool_name: str, args: dict) -> None: + """Log before tool execution.""" + logger.info(f"[LoggingPlugin] About to call: {tool_name}({args})") + + def _on_after_tool_call(self, tool_name: str, result: str) -> None: + """Log after tool execution.""" + preview = result[:100] + "..." if len(result) > 100 else result + logger.info(f"[LoggingPlugin] {tool_name} returned: {preview}") + + +# ============================================================================= +# Example 2: Plugin with Custom Tools +# ============================================================================= + + +class GreetingPlugin(BasePlugin): + """A plugin that provides a greeting tool. + + This plugin demonstrates: + - Providing custom tools via get_tools() + - Using PluginPermission levels + - Using PluginSandbox for restricted access + """ + + metadata = PluginMetadata( + name="greeting", + version="1.0.0", + description="Provides a greeting tool that says hello in different languages", + author="jojo-code", + tags=["example", "utility"], + ) + + permission = PluginPermission.TRUSTED + sandbox = PluginSandbox( + restricted=False, + max_memory_mb=100, + ) + + def on_load(self) -> None: + logger.info("[GreetingPlugin] Plugin loaded") + + def on_unload(self) -> None: + logger.info("[GreetingPlugin] Plugin unloaded") + + def get_tools(self) -> list: + """Return custom tools provided by this plugin. + + Tools must be LangChain BaseTool instances. You can use the @tool + decorator or create Tool objects manually. + """ + from langchain_core.tools import tool + + @tool + def greet(name: str, language: str = "en") -> str: + """Say hello to someone in a specified language. + + Args: + name: The name of the person to greet + language: Language code (en, zh, ja, es). Defaults to 'en'. + + Returns: + A greeting message + """ + greetings = { + "en": f"Hello, {name}!", + "zh": f"你好, {name}!", + "ja": f"こんにちは, {name}!", + "es": f"¡Hola, {name}!", + } + return greetings.get(language, greetings["en"]) + + return [greet] + + +# ============================================================================= +# Example 3: Security-Aware Plugin +# ============================================================================= + + +class SecurityAuditPlugin(BasePlugin): + """A plugin that audits potentially dangerous operations. + + This plugin demonstrates: + - Using CONFIRM permission level + - Denying dangerous operations via hooks + - Using PluginSandbox with path restrictions + """ + + metadata = PluginMetadata( + name="security-audit", + version="1.0.0", + description="Audits and blocks dangerous tool calls", + author="jojo-code", + tags=["security", "audit"], + ) + + permission = PluginPermission.RESTRICTED + sandbox = PluginSandbox( + restricted=True, + allowed_paths=["src/**", "tests/**"], + allowed_urls=["https://api.example.com"], + ) + + # Patterns that should trigger warnings + DANGEROUS_PATTERNS = ["rm -rf", "sudo", "DROP TABLE", "DELETE FROM"] + + def on_load(self) -> None: + logger.info("[SecurityAuditPlugin] Security audit plugin loaded") + + def on_unload(self) -> None: + logger.info("[SecurityAuditPlugin] Security audit plugin unloaded") + + def get_hooks(self) -> dict[str, Any]: + return { + HOOK_BEFORE_TOOL_CALL: self._check_dangerous_call, + } + + def _check_dangerous_call(self, tool_name: str, args: dict) -> None: + """Check if a tool call might be dangerous.""" + # Check shell commands + if tool_name == "run_command": + command = args.get("command", "") + for pattern in self.DANGEROUS_PATTERNS: + if pattern.lower() in command.lower(): + logger.warning( + f"[SecurityAuditPlugin] DANGEROUS command detected: " + f"'{command}' contains '{pattern}'" + ) + + # Check file writes to sensitive paths + if tool_name == "write_file": + path = args.get("path", "") + sensitive = [".env", ".git/config", "id_rsa", ".pem"] + for s in sensitive: + if s in path: + logger.warning( + f"[SecurityAuditPlugin] Sensitive file write: {path}" + ) + + +# ============================================================================= +# Demo: Register and Use Plugins +# ============================================================================= + + +def demo_plugin_registry(): + """Demonstrate plugin registration and lifecycle.""" + print("=" * 60) + print("Plugin Registry Demo") + print("=" * 60) + + # Get the singleton registry + registry = PluginRegistry.get_instance() + + # Create and register plugins + logging_plugin = LoggingPlugin() + greeting_plugin = GreetingPlugin() + security_plugin = SecurityAuditPlugin() + + # Set up hook dispatcher + dispatcher = HookDispatcher() + registry.set_dispatcher(dispatcher) + + # Register plugins (triggers on_load) + print("\n1. Registering plugins...") + registry.register("logging", logging_plugin) + registry.register("greeting", greeting_plugin) + registry.register("security-audit", security_plugin) + + # List registered plugins + print("\n2. Registered plugins:") + for name in registry.list_plugins(): + plugin = registry.get(name) + print(f" - {name}: {plugin.metadata.description}") + + # Get tools from plugins + print("\n3. Tools from greeting plugin:") + tools = greeting_plugin.get_tools() + for t in tools: + print(f" - {t.name}: {t.description}") + + # Test the greeting tool + print("\n4. Testing greeting tool:") + if tools: + result = tools[0].invoke({"name": "World", "language": "zh"}) + print(f" Result: {result}") + + # Demonstrate hook dispatching + print("\n5. Dispatching hooks:") + dispatcher.dispatch(HOOK_BEFORE_TOOL_CALL, "run_command", {"command": "ls -la"}) + dispatcher.dispatch(HOOK_BEFORE_TOOL_CALL, "run_command", {"command": "rm -rf /"}) + dispatcher.dispatch(HOOK_AFTER_TOOL_CALL, "read_file", "file content here") + + # Unregister a plugin (triggers on_unload) + print("\n6. Unregistering logging plugin...") + registry.unregister("logging") + + print("\n7. Remaining plugins:") + for name in registry.list_plugins(): + print(f" - {name}") + + # Clean up + registry.clear() + + +def demo_plugin_discovery(): + """Demonstrate plugin discovery from files.""" + from pathlib import Path + + from jojo_code.plugin.discovery import PluginDiscovery + + print("\n" + "=" * 60) + print("Plugin Discovery Demo") + print("=" * 60) + + discovery = PluginDiscovery() + + # Discover from a directory + plugins_dir = Path("plugins") + if plugins_dir.exists(): + plugins = discovery.discover(plugins_dir) + print(f"\nDiscovered {len(plugins)} plugins from {plugins_dir}") + for p in plugins: + print(f" - {p.metadata.name} v{p.metadata.version}") + else: + print(f"\n{plugins_dir} directory not found (expected for demo)") + + # Discover from a single file + single_file = Path("examples/plugin_development.py") + if single_file.exists(): + plugins = discovery.discover(single_file) + print(f"\nDiscovered {len(plugins)} plugins from {single_file}") + for p in plugins: + print(f" - {p.metadata.name} v{p.metadata.version}") + + +if __name__ == "__main__": + demo_plugin_registry() + demo_plugin_discovery() + print("\nDone!") diff --git a/examples/security_config.py b/examples/security_config.py new file mode 100644 index 0000000..a66a7e9 --- /dev/null +++ b/examples/security_config.py @@ -0,0 +1,437 @@ +"""Security Configuration Example - jojo-code + +Demonstrates how to configure the security system in jojo-code: +- Permission modes (AUTO, MANUAL, BYPASS) +- Rule engine with custom rules +- Denial tracking to prevent repeated requests +- Path and command guards +- Enhanced permission manager with confirm callbacks + +Usage: + uv run python examples/security_config.py +""" + +import sys +from pathlib import Path +from typing import Any + +# Add project src to path for direct execution +sys.path.insert(0, str(Path(__file__).parent.parent / "src")) + +from jojo_code.security import ( + BaseGuard, + EnhancedPermissionConfig, + EnhancedPermissionManager, + PermissionConfig, + PermissionLevel, + PermissionManager, + PermissionResult, + PermissionRule, + RuleAction, + RuleEngine, + RuleFactory, +) + +# ============================================================================= +# Part 1: Basic Permission Configuration +# ============================================================================= + + +def demo_basic_permission_config(): + """Demonstrate basic permission configuration.""" + print("=" * 60) + print("Part 1: Basic Permission Configuration") + print("=" * 60) + + # Development mode - relaxed permissions + print("\n1. Development mode (relaxed):") + dev_config = PermissionConfig.development() + print(f" Shell enabled: {dev_config.shell_enabled}") + print(f" Denied commands: {dev_config.denied_commands}") + print(f" Shell default: {dev_config.shell_default}") + + # Production mode - strict permissions + print("\n2. Production mode (strict):") + prod_config = PermissionConfig.production() + print(f" Allowed paths: {prod_config.allowed_paths}") + print(f" Denied paths: {prod_config.denied_paths}") + print(f" Allowed commands: {prod_config.allowed_commands}") + print(f" Denied commands: {prod_config.denied_commands}") + print(f" Max tool calls: {prod_config.max_tool_calls}") + print(f" Audit log: {prod_config.audit_log}") + + # Custom configuration + print("\n3. Custom configuration:") + custom_config = PermissionConfig( + workspace_root=Path("."), + allowed_paths=["src/**", "tests/**", "*.md"], + denied_paths=[".env", ".git/**", "*.pem", "*.key"], + confirm_on_write=["**/*.py"], + shell_enabled=True, + allowed_commands=["ls", "cat", "grep", "pytest"], + denied_commands=["rm -rf", "sudo", "curl", "wget"], + shell_default=PermissionLevel.CONFIRM, + max_timeout=120, + allow_network=False, + max_tool_calls=50, + audit_log=True, + ) + print(f" Workspace: {custom_config.workspace_root}") + print(f" Confirm on write: {custom_config.confirm_on_write}") + print(f" Max timeout: {custom_config.max_timeout}") + + +# ============================================================================= +# Part 2: Permission Manager Usage +# ============================================================================= + + +def demo_permission_manager(): + """Demonstrate permission manager usage.""" + print("\n" + "=" * 60) + print("Part 2: Permission Manager Usage") + print("=" * 60) + + config = PermissionConfig( + workspace_root=Path("."), + denied_paths=[".env", "*.pem"], + denied_commands=["rm -rf", "sudo"], + audit_log=False, + ) + manager = PermissionManager(config) + + # Check various operations + print("\n1. Checking permissions:") + + # Read a file (should be allowed) + result = manager.check("read_file", {"path": "src/main.py"}) + print(f" read_file src/main.py: {result.level.value}") + + # Read a denied path + result = manager.check("read_file", {"path": ".env"}) + print(f" read_file .env: {result.level.value} ({result.reason})") + + # Run a denied command + result = manager.check("run_command", {"command": "sudo ls"}) + print(f" run_command sudo ls: {result.level.value} ({result.reason})") + + # Switch to manual mode + print("\n2. Switching to manual mode:") + manager.set_mode("manual") + print(f" Mode: {manager.mode.value}") + + # Check mode effects + result = manager.check("read_file", {"path": "src/main.py"}) + print(f" read_file in manual mode: {result.level.value}") + + +# ============================================================================= +# Part 3: Rule Engine +# ============================================================================= + + +def demo_rule_engine(): + """Demonstrate rule engine configuration.""" + print("\n" + "=" * 60) + print("Part 3: Rule Engine") + print("=" * 60) + + engine = RuleEngine() + + # Add custom rules + print("\n1. Adding custom rules:") + + # Allow all read operations + engine.add_rule( + PermissionRule( + name="allow_reads", + tool_pattern="read_*", + action=RuleAction.ALLOW, + priority=10, + description="Allow all read operations", + ) + ) + print(" Added: allow_reads (read_* -> ALLOW)") + + # Deny dangerous commands + engine.add_rule( + PermissionRule( + name="deny_rm_rf", + tool_pattern="run_command", + args_pattern={"command": "rm -rf *"}, + action=RuleAction.DENY, + priority=100, + description="Deny rm -rf commands", + ) + ) + print(" Added: deny_rm_rf (run_command with rm -rf -> DENY)") + + # Ask for write operations + engine.add_rule( + PermissionRule( + name="ask_writes", + tool_pattern="write_*", + action=RuleAction.ASK, + priority=50, + description="Ask before writing files", + ) + ) + print(" Added: ask_writes (write_* -> ASK)") + + # Test rule matching + print("\n2. Testing rule matching:") + + test_cases = [ + ("read_file", {"path": "test.py"}), + ("read_directory", {"path": "."}), + ("write_file", {"path": "output.txt", "content": "data"}), + ("run_command", {"command": "rm -rf /"}), + ("run_command", {"command": "ls -la"}), + ] + + for tool_name, args in test_cases: + action = engine.check(tool_name, args) + print(f" {tool_name}({args}) -> {action.value}") + + # List rules + print("\n3. All rules:") + for rule in engine.list_rules(): + print(f" [{rule.priority}] {rule.name}: {rule.tool_pattern} -> {rule.action.value}") + + # Use RuleFactory + print("\n4. Using RuleFactory:") + dangerous_rules = RuleFactory.deny_dangerous_commands() + print(f" Dangerous command rules: {len(dangerous_rules)}") + for rule in dangerous_rules: + print(f" - {rule.name}: {rule.description}") + + write_rules = RuleFactory.require_confirmation_for_writes() + print(f" Write confirmation rules: {len(write_rules)}") + for rule in write_rules: + print(f" - {rule.name}: {rule.description}") + + +# ============================================================================= +# Part 4: Enhanced Permission Manager +# ============================================================================= + + +def demo_enhanced_manager(): + """Demonstrate enhanced permission manager.""" + print("\n" + "=" * 60) + print("Part 4: Enhanced Permission Manager") + print("=" * 60) + + # Custom confirm callback + def my_confirm_callback(result: PermissionResult) -> bool: + """Simulate user confirmation.""" + print(f" [Callback] Confirm: {result.tool_name}?") + # In real usage, this would prompt the user + return True # Auto-approve for demo + + # Create enhanced config + config = EnhancedPermissionConfig( + base=PermissionConfig( + workspace_root=Path("."), + denied_paths=[".env"], + denied_commands=["sudo"], + audit_log=False, + ), + enable_rule_engine=True, + default_rule_action="ask", + enable_denial_tracking=True, + denial_threshold=3, + denial_window_seconds=300, + confirm_callback=my_confirm_callback, + ) + + manager = EnhancedPermissionManager(config) + + # Add custom rules + print("\n1. Adding rules to enhanced manager:") + manager.add_rule( + tool_pattern="read_*", + action="allow", + name="allow_reads", + priority=10, + ) + manager.add_rule( + tool_pattern="run_command", + action="deny", + args_pattern={"command": "rm *"}, + name="deny_rm", + priority=100, + ) + print(f" Total rules: {len(manager.list_rules())}") + + # Check permissions + print("\n2. Checking permissions:") + + result = manager.check("read_file", {"path": "test.py"}) + print(f" read_file: {result.level.value}") + + result = manager.check("run_command", {"command": "rm -rf /"}) + print(f" rm -rf: {result.level.value} ({result.reason})") + + # Get stats + print("\n3. Manager statistics:") + stats = manager.get_stats() + print(f" Mode: {stats['mode']}") + print(f" Rules count: {stats['rules_count']}") + print(f" Denial tracker: {stats.get('denial_tracker', 'N/A')}") + + +# ============================================================================= +# Part 5: Denial Tracking +# ============================================================================= + + +def demo_denial_tracking(): + """Demonstrate denial tracking.""" + print("\n" + "=" * 60) + print("Part 5: Denial Tracking") + print("=" * 60) + + config = EnhancedPermissionConfig( + base=PermissionConfig( + workspace_root=Path("."), + audit_log=False, + ), + enable_denial_tracking=True, + denial_threshold=3, + ) + manager = EnhancedPermissionManager(config) + + # Simulate repeated denials + print("\n1. Simulating repeated denials:") + args = {"path": "/etc/passwd"} + + for i in range(4): + result = manager.check("read_file", args) + print(f" Attempt {i + 1}: {result.level.value}") + if result.denied and "连续拒绝" in (result.reason or ""): + print(f" -> Threshold exceeded after {i + 1} attempts") + + # Get denial stats + print("\n2. Denial statistics:") + stats = manager.get_stats() + tracker_stats = stats.get("denial_tracker", {}) + print(f" Total denials: {tracker_stats.get('total_denials', 0)}") + print(f" Tools tracked: {tracker_stats.get('tools_tracked', 0)}") + print(f" Threshold: {tracker_stats.get('threshold', 0)}") + + +# ============================================================================= +# Part 6: Permission Modes +# ============================================================================= + + +def demo_permission_modes(): + """Demonstrate permission modes.""" + print("\n" + "=" * 60) + print("Part 6: Permission Modes") + print("=" * 60) + + modes = ["auto", "manual", "bypass"] + + for mode_name in modes: + print(f"\n--- {mode_name.upper()} mode ---") + config = PermissionConfig( + workspace_root=Path("."), + mode=mode_name, + audit_log=False, + ) + manager = PermissionManager(config) + + # Test various operations + test_cases = [ + ("read_file", {"path": "test.py"}), + ("write_file", {"path": "output.txt", "content": "data"}), + ("run_command", {"command": "ls"}), + ] + + for tool_name, args in test_cases: + result = manager.check(tool_name, args) + print(f" {tool_name}: {result.level.value}") + + +# ============================================================================= +# Part 7: Custom Guards +# ============================================================================= + + +class NetworkGuard(BaseGuard): + """Custom guard that blocks network operations.""" + + BLOCKED_TOOLS = {"web_search", "web_fetch", "http_request"} + + @property + def name(self) -> str: + return "network_guard" + + def check(self, tool_name: str, args: dict[str, Any]) -> PermissionResult: + if tool_name in self.BLOCKED_TOOLS: + return PermissionResult( + PermissionLevel.DENY, + tool_name, + args, + reason=f"Network tool {tool_name} is blocked by policy", + ) + return PermissionResult(PermissionLevel.ALLOW, tool_name, args) + + +def demo_custom_guards(): + """Demonstrate custom guards.""" + print("\n" + "=" * 60) + print("Part 7: Custom Guards") + print("=" * 60) + + config = PermissionConfig( + workspace_root=Path("."), + audit_log=False, + ) + manager = PermissionManager(config) + + # Add custom guard + print("\n1. Adding NetworkGuard:") + manager.guards.append(NetworkGuard()) + print(f" Total guards: {len(manager.guards)}") + + # Test with custom guard + print("\n2. Testing with NetworkGuard:") + + result = manager.check("read_file", {"path": "test.py"}) + print(f" read_file: {result.level.value}") + + result = manager.check("web_search", {"query": "Python docs"}) + print(f" web_search: {result.level.value} ({result.reason})") + + result = manager.check("http_request", {"url": "https://api.example.com"}) + print(f" http_request: {result.level.value} ({result.reason})") + + +# ============================================================================= +# Main +# ============================================================================= + + +def main(): + """Run all security configuration demos.""" + print("jojo-Code Security Configuration Example") + print("Demonstrating permission system, rules, and guards\n") + + demo_basic_permission_config() + demo_permission_manager() + demo_rule_engine() + demo_enhanced_manager() + demo_denial_tracking() + demo_permission_modes() + demo_custom_guards() + + print("\n" + "=" * 60) + print("All security demos completed!") + print("=" * 60) + + +if __name__ == "__main__": + main() diff --git a/examples/session_management.py b/examples/session_management.py new file mode 100644 index 0000000..61978f1 --- /dev/null +++ b/examples/session_management.py @@ -0,0 +1,284 @@ +"""jojo-Code session management usage example. + +Demonstrates the session management features: +- Creating and retrieving sessions +- Adding messages with different roles +- Using metadata for session context +- Session persistence and recovery +- Integrating with the memory system + +Run this script from the project root: + uv run python examples/session_management.py +""" + +import sys +import tempfile +from pathlib import Path + +# Add project src to path for direct execution +sys.path.insert(0, str(Path(__file__).parent.parent / "src")) + + +def demo_basic_session() -> None: + """Demonstrate basic session CRUD operations.""" + from jojo_code.session.manager import SessionManager + + print("=" * 60) + print("Basic Session Management Demo") + print("=" * 60) + + with tempfile.TemporaryDirectory() as tmpdir: + manager = SessionManager(storage_dir=tmpdir) + + # Create a session + session = manager.create_session(user_id="demo-user") + print(f"\nCreated session: {session.id}") + print(f" User ID: {session.user_id}") + print(f" Created at: {session.created_at}") + + # Add messages + manager.add_message(session.id, "user", "What is Python?") + manager.add_message( + session.id, + "assistant", + "Python is a high-level programming language known for its readability.", + ) + manager.add_message(session.id, "user", "How do I install packages?") + manager.add_message( + session.id, + "assistant", + "Use pip: pip install . Or use uv for faster installs.", + ) + + # Retrieve and display + loaded = manager.get_session(session.id) + print(f"\nSession has {len(loaded.messages)} messages:") + for msg in loaded.messages: + print(f" [{msg.role}] {msg.content[:60]}...") + + +def demo_session_with_metadata() -> None: + """Demonstrate sessions with metadata for context tracking.""" + from jojo_code.session.manager import SessionManager + + print("\n" + "=" * 60) + print("Session with Metadata Demo") + print("=" * 60) + + with tempfile.TemporaryDirectory() as tmpdir: + manager = SessionManager(storage_dir=tmpdir) + + # Create a session with rich metadata + session = manager.create_session( + user_id="developer-1", + metadata={ + "project": "my-web-app", + "language": "python", + "mode": "build", + "source": "cli", + }, + ) + + manager.add_message(session.id, "user", "Set up a FastAPI project") + manager.add_message( + session.id, + "assistant", + "I'll help you set up a FastAPI project. Let me create the structure.", + ) + + # Metadata persists across reloads + loaded = manager.get_session(session.id) + print("\nSession metadata:") + for key, value in loaded.metadata.items(): + print(f" {key}: {value}") + print(f"\nMessages: {len(loaded.messages)}") + + +def demo_session_recovery() -> None: + """Demonstrate session persistence and recovery across manager instances.""" + from jojo_code.session.manager import SessionManager + + print("\n" + "=" * 60) + print("Session Recovery Demo") + print("=" * 60) + + with tempfile.TemporaryDirectory() as tmpdir: + # First manager instance creates and populates the session + manager1 = SessionManager(storage_dir=tmpdir) + session = manager1.create_session(user_id="user-1") + manager1.add_message(session.id, "user", "Remember this: my API key is abc123") + manager1.add_message(session.id, "assistant", "Noted. I will remember that.") + print(f"\nCreated session with {len(manager1.get_session(session.id).messages)} messages") + + # Simulate process restart: new manager instance + manager2 = SessionManager(storage_dir=tmpdir) + + # Recover the session + recovered = manager2.recover_session(session.id) + if recovered: + print(f"Recovered session: {recovered.id}") + print(f" User: {recovered.user_id}") + print(f" Messages: {len(recovered.messages)}") + for msg in recovered.messages: + print(f" [{msg.role}] {msg.content}") + + # Add more messages to the recovered session + manager2.add_message(session.id, "user", "What was my API key?") + manager2.add_message(session.id, "assistant", "Your API key is abc123.") + + # Verify in a third manager instance + manager3 = SessionManager(storage_dir=tmpdir) + final = manager3.get_session(session.id) + print(f"\nFinal message count: {len(final.messages)}") + + +def demo_multiple_sessions() -> None: + """Demonstrate managing multiple sessions simultaneously.""" + from jojo_code.session.manager import SessionManager + + print("\n" + "=" * 60) + print("Multiple Sessions Demo") + print("=" * 60) + + with tempfile.TemporaryDirectory() as tmpdir: + manager = SessionManager(storage_dir=tmpdir) + + # Create multiple sessions for different tasks + code_session = manager.create_session( + user_id="dev", + metadata={"task": "code-review"}, + ) + manager.add_message(code_session.id, "user", "Review this function for bugs") + manager.add_message(code_session.id, "assistant", "I see two issues...") + + docs_session = manager.create_session( + user_id="dev", + metadata={"task": "documentation"}, + ) + manager.add_message(docs_session.id, "user", "Write a README for my project") + manager.add_message( + docs_session.id, + "assistant", + "Here is a draft README...", + ) + + test_session = manager.create_session( + user_id="dev", + metadata={"task": "testing"}, + ) + manager.add_message(test_session.id, "user", "Write tests for utils.py") + + # List all sessions + print("\nActive sessions: 3") + for sid, task in [ + (code_session.id, "code-review"), + (docs_session.id, "documentation"), + (test_session.id, "testing"), + ]: + loaded = manager.get_session(sid) + print(f" [{task}] {loaded.id[:8]}... ({len(loaded.messages)} messages)") + + +def demo_error_handling() -> None: + """Demonstrate error handling in session management.""" + from jojo_code.session.manager import SessionManager + + print("\n" + "=" * 60) + print("Error Handling Demo") + print("=" * 60) + + with tempfile.TemporaryDirectory() as tmpdir: + manager = SessionManager(storage_dir=tmpdir) + + # Missing session returns None + result = manager.get_session("nonexistent-id") + print(f"\nGet missing session: {result}") + + # recover_session also returns None for missing sessions + result = manager.recover_session("nonexistent-id") + print(f"Recover missing session: {result}") + + # add_message raises ValueError for missing sessions + try: + manager.add_message("nonexistent", "user", "hello") + except ValueError as e: + print(f"Add message to missing session: {e}") + + # Corrupted JSON returns None + session = manager.create_session() + path = manager._path(session.id) + with open(path, "w") as f: + f.write("not valid json {{{") + + result = manager.get_session(session.id) + print(f"Get corrupted session: {result}") + + +def demo_session_with_memory() -> None: + """Demonstrate integrating sessions with the memory system.""" + from jojo_code.memory import SessionMemory + from jojo_code.session.manager import SessionManager + + print("\n" + "=" * 60) + print("Session + Memory Integration Demo") + print("=" * 60) + + with tempfile.TemporaryDirectory() as tmpdir: + # Create session manager and memory + session_mgr = SessionManager(storage_dir=tmpdir) + session = session_mgr.create_session(user_id="user-1") + + memory = SessionMemory( + session_id=session.id, + max_tokens=50000, + storage_dir=tmpdir, + ) + + # Add messages to both systems + messages = [ + ("user", "How do I read a CSV file in Python?"), + ("assistant", "Use pandas: pd.read_csv('file.csv')"), + ("user", "What about large files?"), + ("assistant", "Use chunksize parameter for memory-efficient reading."), + ] + + for role, content in messages: + session_mgr.add_message(session.id, role, content) + memory.add_message(content, role=role) + + # Session provides persistence + recovered = session_mgr.get_session(session.id) + print(f"\nSession messages: {len(recovered.messages)}") + + # Memory provides search + results = memory.search("CSV", scope="all") + print("Memory search for 'CSV':") + print(f" Current session: {len(results['current_session'])}") + print(f" History: {len(results['history'])}") + + # Get memory stats + stats = memory.get_stats() + print("\nMemory stats:") + for key, value in stats.items(): + print(f" {key}: {value}") + + +def main() -> None: + """Run all session management demos.""" + print("jojo-Code Session Management Example") + print("Demonstrating session lifecycle features\n") + + demo_basic_session() + demo_session_with_metadata() + demo_session_recovery() + demo_multiple_sessions() + demo_error_handling() + demo_session_with_memory() + + print("\n" + "=" * 60) + print("All session management demos completed successfully!") + print("=" * 60) + + +if __name__ == "__main__": + main() diff --git a/examples/tool_creation.py b/examples/tool_creation.py new file mode 100644 index 0000000..9e72ac8 --- /dev/null +++ b/examples/tool_creation.py @@ -0,0 +1,302 @@ +"""Tool Creation Example - jojo-code + +This example demonstrates how to create custom tools for jojo-code. +Tools are LangChain BaseTool instances that the agent can invoke. + +Usage: + uv run python examples/tool_creation.py +""" + +import json +import subprocess +from pathlib import Path + +from langchain_core.tools import tool + +from jojo_code.tools.registry import ToolRegistry + + +# ============================================================================= +# Example 1: Simple Tool with @tool Decorator +# ============================================================================= + + +@tool +def word_count(text: str) -> str: + """Count the number of words, lines, and characters in text. + + Args: + text: The text to analyze + + Returns: + JSON string with word, line, and character counts + """ + words = len(text.split()) + lines = text.count("\n") + (1 if text else 0) + chars = len(text) + + return json.dumps({ + "words": words, + "lines": lines, + "characters": chars, + }, indent=2) + + +@tool +def json_pretty(json_string: str) -> str: + """Pretty-print a JSON string with indentation. + + Args: + json_string: A JSON string to format + + Returns: + Formatted JSON string, or error message if invalid + """ + try: + data = json.loads(json_string) + return json.dumps(data, indent=2, ensure_ascii=False) + except json.JSONDecodeError as e: + return f"Error: Invalid JSON - {e}" + + +# ============================================================================= +# Example 2: File-Based Tool +# ============================================================================= + + +@tool +def file_stats(path: str) -> str: + """Get statistics about a file (size, line count, extension). + + Args: + path: Path to the file + + Returns: + JSON string with file statistics + """ + file_path = Path(path) + + if not file_path.exists(): + return f"Error: File not found: {path}" + + if not file_path.is_file(): + return f"Error: Not a file: {path}" + + stat = file_path.stat() + content = file_path.read_text(encoding="utf-8", errors="replace") + lines = content.count("\n") + (1 if content else 0) + + return json.dumps({ + "path": str(file_path.absolute()), + "size_bytes": stat.st_size, + "lines": lines, + "extension": file_path.suffix, + "name": file_path.name, + }, indent=2) + + +# ============================================================================= +# Example 3: Tool with Error Handling +# ============================================================================= + + +@tool +def safe_divide(a: float, b: float) -> str: + """Divide two numbers safely, handling division by zero. + + Args: + a: The numerator + b: The denominator + + Returns: + Result as string, or error message + """ + if b == 0: + return "Error: Division by zero is not allowed" + + result = a / b + return f"{a} / {b} = {result}" + + +# ============================================================================= +# Example 4: Tool with External Process +# ============================================================================= + + +@tool +def check_python_version() -> str: + """Check the Python version and environment info. + + Returns: + Python version and path information + """ + try: + result = subprocess.run( + ["python", "--version"], + capture_output=True, + text=True, + timeout=10, + ) + version = result.stdout.strip() or result.stderr.strip() + + import sys + + return json.dumps({ + "version": version, + "executable": sys.executable, + "platform": sys.platform, + "prefix": sys.prefix, + }, indent=2) + except Exception as e: + return f"Error checking Python version: {e}" + + +# ============================================================================= +# Example 5: Tool with Optional Parameters +# ============================================================================= + + +@tool +def truncate_text(text: str, max_length: int = 100, suffix: str = "...") -> str: + """Truncate text to a maximum length with optional suffix. + + Args: + text: The text to truncate + max_length: Maximum length (default: 100) + suffix: Suffix to add when truncated (default: "...") + + Returns: + Truncated text + """ + if len(text) <= max_length: + return text + + return text[: max_length - len(suffix)] + suffix + + +# ============================================================================= +# Demo: Register and Use Custom Tools +# ============================================================================= + + +def demo_basic_tools(): + """Demonstrate basic tool usage.""" + print("=" * 60) + print("Basic Tool Usage Demo") + print("=" * 60) + + # Use tools directly via .invoke() + print("\n1. Word count tool:") + result = word_count.invoke({"text": "Hello world\nThis is a test"}) + print(f" {result}") + + print("\n2. JSON pretty-print tool:") + result = json_pretty.invoke({"json_string": '{"name":"jojo","version":"1.0"}'}) + print(f" {result}") + + print("\n3. Safe divide tool:") + result = safe_divide.invoke({"a": 10, "b": 3}) + print(f" {result}") + + result = safe_divide.invoke({"a": 10, "b": 0}) + print(f" {result}") + + print("\n4. Truncate text tool:") + result = truncate_text.invoke({ + "text": "This is a very long text that should be truncated", + "max_length": 20, + }) + print(f" {result}") + + +def demo_file_tools(): + """Demonstrate file-based tools.""" + print("\n" + "=" * 60) + print("File Tool Demo") + print("=" * 60) + + # Analyze a real file + target = Path("pyproject.toml") + if target.exists(): + print(f"\n1. File stats for {target}:") + result = file_stats.invoke({"path": str(target)}) + print(f" {result}") + else: + print(f"\n1. {target} not found, skipping file stats demo") + + # Test with nonexistent file + print("\n2. File stats for nonexistent file:") + result = file_stats.invoke({"path": "/nonexistent/file.txt"}) + print(f" {result}") + + +def demo_tool_registry(): + """Demonstrate ToolRegistry integration.""" + print("\n" + "=" * 60) + print("ToolRegistry Integration Demo") + print("=" * 60) + + # Create a fresh registry (in production, use get_tool_registry()) + registry = ToolRegistry() + + # Register custom tools + print("\n1. Registering custom tools...") + registry.register(word_count) + registry.register(json_pretty) + registry.register(safe_divide) + registry.register(file_stats) + registry.register(truncate_text) + registry.register(check_python_version) + + # List all tools + all_tools = registry.list_tools() + print(f"\n2. Total tools registered: {len(all_tools)}") + print(f" Custom tools: word_count, json_pretty, safe_divide, file_stats, truncate_text, check_python_version") + + # Execute a tool through the registry + print("\n3. Executing word_count via registry:") + result = registry.execute("word_count", {"text": "Hello World"}) + print(f" {result}") + + # Execute another tool + print("\n4. Executing json_pretty via registry:") + result = registry.execute("json_pretty", {"json_string": '{"key": "value"}'}) + print(f" {result}") + + # Get LangChain tools list (for binding to LLM) + lc_tools = registry.get_langchain_tools() + print(f"\n5. LangChain tools count: {len(lc_tools)}") + print(f" These can be bound to an LLM with: llm.bind_tools(tools)") + + # Unregister a tool + print("\n6. Unregistering word_count...") + registry.unregister("word_count") + + remaining = registry.list_tools() + print(f" Remaining tools: {len(remaining)}") + + +def demo_tool_info(): + """Show tool metadata.""" + print("\n" + "=" * 60) + print("Tool Metadata Demo") + print("=" * 60) + + tools = [word_count, json_pretty, safe_divide, file_stats, truncate_text] + + for t in tools: + print(f"\n Tool: {t.name}") + print(f" Description: {t.description}") + # Access args schema + if hasattr(t, "args_schema") and t.args_schema: + schema = t.args_schema.model_json_schema() + props = schema.get("properties", {}) + print(f" Parameters: {', '.join(props.keys())}") + + +if __name__ == "__main__": + demo_basic_tools() + demo_file_tools() + demo_tool_registry() + demo_tool_info() + print("\nDone!") diff --git a/pyproject.toml b/pyproject.toml index 989ef25..e7a5328 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "jojo-code" -version = "0.1.0" +version = "0.2.0" description = "A coding agent powered by jojo AI - Python CLI + LangGraph Core" readme = "README.md" license = "MIT" diff --git "a/wiki/01-\351\241\271\347\233\256\347\273\223\346\236\204.md" "b/wiki/01-\351\241\271\347\233\256\347\273\223\346\236\204.md" index 35b92de..8212ae5 100644 --- "a/wiki/01-\351\241\271\347\233\256\347\273\223\346\236\204.md" +++ "b/wiki/01-\351\241\271\347\233\256\347\273\223\346\236\204.md" @@ -4,30 +4,91 @@ ``` jojo-code/ -├── src/jojo_code/ # 源代码 -│ ├── agent/ # Agent 核心（LangGraph 状态机） -│ │ ├── graph.py # 状态图定义 -│ │ ├── nodes.py # 节点实现 -│ │ └── state.py # 状态定义 -│ ├── cli/ # 命令行界面 -│ │ ├── main.py # CLI 主入口 -│ │ ├── console.py # 控制台输出 -│ │ └── enhanced.py # 增强功能 -│ ├── core/ # 核心模块 -│ │ ├── llm.py # LLM 客户端 -│ │ ├── config.py # 配置管理 -│ │ └── ... # 其他核心功能 -│ ├── memory/ # 记忆管理 -│ │ └── conversation.py # 对话记忆 -│ └── tools/ # 工具系统 -│ ├── registry.py # 工具注册中心 -│ ├── file_tools.py # 文件操作 -│ ├── shell_tools.py # Shell 命令 -│ └── search_tools.py # 搜索工具 -├── tests/ # 测试代码 -├── wiki/ # 项目文档 -├── pyproject.toml # 项目配置 -└── README.md # 项目说明 +├── src/jojo_code/ # 源代码 +│ ├── agent/ # Agent 核心（LangGraph 状态机） +│ │ ├── graph.py # 状态图定义 +│ │ ├── nodes.py # 节点实现 +│ │ ├── state.py # 状态定义 +│ │ ├── sub.py # SubAgent 子代理 +│ │ ├── tool.py # AgentTool 代理工具 +│ │ └── modes.py # Plan/Build 模式 +│ ├── cli/ # 命令行界面（Textual TUI） +│ │ ├── main.py # CLI 主入口 +│ │ ├── console.py # 控制台输出 +│ │ ├── widgets/ # TUI 组件 +│ │ └── enhanced.py # 增强功能 +│ ├── context/ # 项目上下文 +│ │ ├── project.py # 项目检测和 AGENTS.md 解析 +│ │ └── init.py # /init 命令实现 +│ ├── core/ # 核心模块 +│ │ ├── llm.py # LLM 客户端 +│ │ └── config.py # Pydantic Settings 配置管理 +│ ├── mcp/ # MCP 客户端 +│ │ └── client.py # MCP 客户端实现 +│ ├── memory/ # 记忆管理 +│ │ ├── conversation.py # 对话记忆、Token 计数、自动压缩 +│ │ ├── short_term.py # 短期记忆 +│ │ └── long_term.py # 长期记忆 +│ ├── models/ # 模型提供商抽象 +│ ├── ops/ # AgentOps 运维体系 +│ │ ├── config.py # Ops 配置 +│ │ ├── models.py # Trace/Span 数据结构 +│ │ ├── collector.py # 数据收集器 +│ │ ├── metrics.py # 指标计算引擎 +│ │ ├── evaluator.py # 评估引擎 +│ │ ├── exporter.py # 数据导出 +│ │ ├── dashboard.py # CLI 监控面板 +│ │ └── report.py # 报告生成器 +│ ├── plugin/ # 插件系统 +│ │ ├── base.py # BasePlugin 基类 +│ │ ├── hooks.py # Hook 生命周期系统 +│ │ ├── registry.py # 插件注册表 +│ │ ├── discovery.py # 插件发现 +│ │ ├── loader.py # 插件加载器 +│ │ └── config.py # 插件配置 +│ ├── plugins/ # 内置插件 +│ │ ├── code_review/ # 代码审查插件 +│ │ ├── git/ # Git 插件 +│ │ └── test_generator/ # 测试生成插件 +│ ├── security/ # 安全模块 +│ │ ├── permission.py # 权限级别和结果 +│ │ ├── guards.py # 守卫基类 +│ │ ├── path_guard.py # 路径守卫 +│ │ ├── command_guard.py # 命令守卫 +│ │ └── manager.py # 权限管理器 +│ ├── server/ # WebSocket/SSE 服务器 +│ │ └── app.py # FastAPI 应用 +│ ├── session/ # 会话管理 +│ │ ├── models.py # Session 数据模型 +│ │ └── manager.py # 会话管理器 +│ ├── skills/ # 技能系统 +│ ├── task/ # 任务执行框架 +│ └── tools/ # 工具系统 +│ ├── registry.py # 工具注册中心 +│ ├── file_tools.py # 文件操作 +│ ├── shell_tools.py # Shell 命令 +│ ├── search_tools.py # 搜索工具 +│ ├── git_tools.py # Git 集成 +│ ├── code_analysis_tools.py # 代码分析 +│ ├── performance_tools.py # 性能分析 +│ ├── web_tools.py # Web 搜索 +│ ├── web_fetch_tools.py # 网页抓取 +│ ├── http_tools.py # HTTP 请求 +│ ├── data_tools.py # 数据处理 +│ ├── doc_tools.py # 文档工具 +│ └── system_tools.py # 系统工具 +├── tests/ # 测试代码 +│ ├── test_agent/ # Agent 测试 +│ ├── test_cli/ # CLI 测试 +│ ├── test_core/ # Core 测试 +│ ├── test_memory/ # Memory 测试 +│ ├── test_ops/ # Ops 测试 +│ ├── test_tools/ # Tools 测试 +│ └── conftest.py # pytest fixtures +├── examples/ # 示例脚本 +├── wiki/ # 项目文档 +├── pyproject.toml # 项目配置 +└── README.md # 项目说明 ``` ## 核心文件说明 @@ -70,6 +131,18 @@ jojo-code/ | [tools/shell_tools.py](../src/jojo_code/tools/shell_tools.py) | run_command 执行 Shell 命令 | | [tools/search_tools.py](../src/jojo_code/tools/search_tools.py) | grep_search, glob_search 搜索工具 | +### Ops 层（AgentOps 运维体系） + +| 文件 | 职责 | +|------|------| +| [ops/models.py](../src/jojo_code/ops/models.py) | Span/Trace 数据结构定义 | +| [ops/collector.py](../src/jojo_code/ops/collector.py) | 数据收集器，在 Agent 循环中埋点 | +| [ops/metrics.py](../src/jojo_code/ops/metrics.py) | 指标计算引擎，聚合统计数据 | +| [ops/evaluator.py](../src/jojo_code/ops/evaluator.py) | 评估引擎，评估 Agent 行为质量 | +| [ops/exporter.py](../src/jojo_code/ops/exporter.py) | 数据导出（JSON/Markdown） | +| [ops/dashboard.py](../src/jojo_code/ops/dashboard.py) | CLI 监控面板（Rich 表格输出） | +| [ops/report.py](../src/jojo_code/ops/report.py) | 评估报告和汇总报告生成 | + ## 架构层次 ``` diff --git "a/wiki/06-\345\274\200\345\217\221\346\214\207\345\215\227.md" "b/wiki/06-\345\274\200\345\217\221\346\214\207\345\215\227.md" index abea21c..1edc5ef 100644 --- "a/wiki/06-\345\274\200\345\217\221\346\214\207\345\215\227.md" +++ "b/wiki/06-\345\274\200\345\217\221\346\214\207\345\215\227.md" @@ -79,10 +79,21 @@ uv publish src/jojo_code/ ├── __init__.py # 包入口 ├── __main__.py # python -m jojo-code 入口 -├── agent/ # Agent 核心 -├── cli/ # 命令行界面 -├── core/ # 核心模块 +├── agent/ # Agent 核心（LangGraph 状态机） +├── cli/ # 命令行界面（Textual TUI） +├── context/ # 项目上下文 +├── core/ # 核心模块（LLM、配置、异常） +├── mcp/ # MCP 协议客户端 ├── memory/ # 记忆管理 +├── models/ # 模型提供商抽象 +├── ops/ # AgentOps 运维体系 +├── plugin/ # 插件系统 +├── plugins/ # 内置插件 +├── security/ # 安全模块 +├── server/ # WebSocket/SSE 服务器 +├── session/ # 会话管理 +├── skills/ # 技能系统 +├── task/ # 任务执行框架 └── tools/ # 工具系统 ``` @@ -90,12 +101,20 @@ src/jojo_code/ ``` tests/ -├── conftest.py # pytest 配置和 fixtures -├── test_agent.py # Agent 测试 -├── test_cli/ # CLI 子测试 -├── test_core/ # Core 子测试 -├── test_memory/ # Memory 子测试 -└── test_tools/ # Tools 子测试 +├── conftest.py # pytest 配置和 fixtures +├── test_agent/ # Agent 子测试 +├── test_cli/ # CLI 子测试 +├── test_core/ # Core 子测试 +├── test_mcp/ # MCP 客户端测试 +├── test_memory/ # Memory 子测试 +├── test_ops/ # Ops 子测试 +├── test_plugin/ # 插件系统测试 +├── test_security/ # 安全模块测试 +├── test_server/ # 服务器测试 +├── test_session/ # 会话管理测试 +├── test_skills/ # 技能系统测试 +├── test_task/ # 任务框架测试 +└── test_tools/ # Tools 子测试 ``` ## 代码风格 diff --git a/wiki/10-AgentOps.md b/wiki/10-AgentOps.md new file mode 100644 index 0000000..636e37c --- /dev/null +++ b/wiki/10-AgentOps.md @@ -0,0 +1,153 @@ +# AgentOps - Agent 运维体系 + +## 概述 + +AgentOps 是 jojo-Code 的运维监控子系统，提供 Agent 执行的追踪、指标统计、数据导出和 CLI 监控面板能力。 + +## 模块结构 + +``` +src/jojo_code/ops/ +├── __init__.py # 模块入口，导出主要类 +├── models.py # Span/Trace 数据结构 +├── collector.py # 数据收集器 +├── metrics.py # 指标计算引擎 +├── evaluator.py # 评估引擎 +├── exporter.py # 数据导出 (JSON/Markdown) +├── dashboard.py # CLI 监控面板 (Rich) +├── config.py # 配置管理 +└── report.py # 报告生成 +``` + +## 核心概念 + +### Trace（追踪） + +一次完整的 Agent 执行过程，包含多个 Span。每个 Trace 记录任务描述、开始/结束时间、执行状态。 + +```python +from jojo_code.ops import Trace, SpanStatus + +trace = Trace(task="读取 README.md", session_id="s1") +# trace.status, trace.duration_ms, trace.thinking_count 等属性 +``` + +### Span（跨度） + +Agent 执行的一个步骤，类型包括： + +| SpanType | 说明 | +|----------|------| +| `THINKING` | Agent 思考 | +| `TOOL_CALL` | 工具调用 | +| `OBSERVE` | 观察结果 | +| `ERROR` | 错误 | + +每个 Span 记录输入、输出、状态、耗时等信息。 + +### Metrics（指标） + +从 Trace 和 Span 聚合计算的统计数据，由 `MetricsEngine` 计算。 + +## 使用方式 + +### 数据收集 + +```python +from jojo_code.ops import Collector, SpanType + +collector = Collector() + +# 开始追踪 +trace = collector.start_trace("读取 README.md") + +# 记录工具调用 +span = collector.start_span(SpanType.TOOL_CALL, "read_file", {"path": "README.md"}) +collector.end_span(span, output_data="文件内容") + +# 结束追踪 +collector.end_trace() +``` + +### 指标计算 + +```python +from jojo_code.ops import MetricsEngine + +engine = MetricsEngine(traces) +summary = engine.calculate() + +print(summary.task_success_rate) # 任务成功率 +print(summary.tool_success_rate) # 工具成功率 +print(summary.avg_thinking_rounds) # 平均思考轮数 +print(summary.tool_usage) # 工具使用统计 +``` + +### 数据导出 + +```python +from jojo_code.ops import Exporter + +# 导出为 JSON +Exporter.export_traces_json(traces, "traces.json") + +# 生成 Markdown 报告 +report = Exporter.export_summary_markdown( + total_traces=10, + completed_traces=8, + failed_traces=2, + avg_thinking_rounds=3.5, + avg_tool_calls=5.0, + avg_duration_ms=1500.0, + tool_success_rate=0.95, + task_success_rate=0.8, + tool_usage={"read_file": 10, "write_file": 5}, + output_path="report.md", +) +``` + +### CLI 监控面板 + +```python +from jojo_code.ops import Dashboard + +dashboard = Dashboard() +dashboard.show_current_trace(trace) # 显示当前 Trace +dashboard.show_metrics(summary) # 显示指标汇总 +dashboard.show_traces_list(traces) # 显示 Trace 列表 +dashboard.show_summary_report(summary) # 显示完整报告 +``` + +## 评估系统 + +内置三种评估器： + +| 评估器 | 说明 | +|--------|------| +| `PlanningEvaluator` | 评估规划质量（轮数、成功率、重复调用） | +| `TestCaseEvaluator` | 根据预设 test case 评估 | +| `PerformanceEvaluator` | 性能指标评估 | +| `CompositeEvaluator` | 组合多个评估器 | + +```python +from jojo_code.ops import PlanningEvaluator, CompositeEvaluator + +evaluator = PlanningEvaluator() +score = evaluator.evaluate(trace) +print(score.score) # 0.0 - 1.0 +print(score.reason) # 评估原因 +``` + +## 配置 + +在 `.env` 中配置： + +```bash +JOJO_CODE_OPS_ENABLED=true +JOJO_CODE_OPS_TRACE_DIR=.jojo-code/traces +``` + +## 相关文档 + +- [AgentOps 功能设计](../docs/agentops-feature-design.md) +- [AgentOps 系统设计](../docs/agentops-system-design.md) diff --git "a/wiki/11-\346\234\215\345\212\241\345\231\250\351\203\250\347\275\262.md" "b/wiki/11-\346\234\215\345\212\241\345\231\250\351\203\250\347\275\262.md" new file mode 100644 index 0000000..991dc18 --- /dev/null +++ "b/wiki/11-\346\234\215\345\212\241\345\231\250\351\203\250\347\275\262.md" @@ -0,0 +1,279 @@ +# 服务器部署 + +## 概述 + +jojo-Code 提供多种部署方式：本地 CLI、WebSocket 服务器、Docker 容器化部署。本文档涵盖所有部署场景。 + +## 部署架构 + +``` +┌─────────────────────────────────────────────────────────────┐ +│ 部署方式选择 │ +├───────────────┬──────────────────┬──────────────────────────┤ +│ 本地 CLI │ WebSocket 服务 │ Docker 容器 │ +│ │ │ │ +│ uv run │ jojo-code │ docker compose up │ +│ jojo-code │ server start │ │ +│ │ │ │ +│ 适合开发 │ 适合团队共享 │ 适合生产环境 │ +└───────────────┴──────────────────┴──────────────────────────┘ +``` + +## 方式一：本地 CLI + +最简单的使用方式，直接在终端运行： + +```bash +# 安装 +uv sync + +# 配置 +cp .env.example .env +# 编辑 .env 填入 API Key + +# 运行 +uv run jojo-code +``` + +### 配置说明 + +在 `.env` 文件中配置： + +```env +# OpenAI 兼容 API +OPENAI_API_KEY=sk-xxx +OPENAI_BASE_URL=https://api.example.com/v1 + +# 或 Anthropic Claude +ANTHROPIC_API_KEY=sk-ant-xxx + +# 模型选择 +JOJO_CODE_MODEL=gpt-4o-mini +``` + +### CLI 命令 + +```bash +jojo-code # 启动 TUI +jojo-code setup # 交互式配置向导 +jojo-code config show # 显示当前配置 +jojo-code config set # 设置配置项 +``` + +## 方式二：WebSocket 服务器 + +将 jojo-Code 作为后台服务运行，支持多客户端通过 WebSocket 连接。 + +### 启动服务 + +```bash +# 前台运行 +jojo-code server start + +# 后台守护进程 +jojo-code server start -d + +# 检查状态 +jojo-code server status + +# 停止服务 +jojo-code server stop +``` + +### 环境变量 + +| 变量 | 说明 | 默认值 | +|------|------|--------| +| `JOJO_CODE_HOST` | 监听地址 | `0.0.0.0` | +| `JOJO_CODE_PORT` | 监听端口 | `8080` | +| `JOJO_CODE_LOG_FILE` | 日志文件 | `server.log` | +| `JOJO_CODE_LOG_LEVEL` | 日志级别 | `INFO` | +| `JOJO_CODE_LOG_FORMAT` | 日志格式 | `json` | + +### API 端点 + +| 端点 | 协议 | 说明 | +|------|------|------| +| `/ws` | WebSocket | JSON-RPC 2.0 聊天端点 | +| `/sse/chat` | SSE | 流式聊天（GET + message 参数） | +| `/sse/events` | SSE | 通用事件订阅 | +| `/health` | HTTP | 健康检查 | +| `/api/info` | HTTP | 服务器信息 | +| `/api/agents` | HTTP | Agent 管理 | +| `/api/conversations` | HTTP | 对话管理 | +| `/api/metrics` | HTTP | 指标查询 | + +### WebSocket 协议 + +使用 JSON-RPC 2.0 协议： + +```json +// 请求 +{"jsonrpc": "2.0", "id": 1, "method": "chat", "params": {"message": "hello"}} + +// 响应 +{"jsonrpc": "2.0", "id": 1, "result": {"type": "content", "text": "..."}} + +// 流式响应：同一 id 发送多个 result，最后发送 {"type": "done"} +``` + +### SSE 使用 + +```bash +# 流式聊天 +curl -N "http://localhost:8080/sse/chat?message=hello" + +# 事件订阅 +curl -N "http://localhost:8080/sse/events" +``` + +### 远程连接 + +从另一台机器连接到服务器： + +```bash +# 设置远程服务器地址 +jojo-code config set server ws://your-server:8080/ws + +# 启动 TUI +jojo-code +``` + +## 方式三：Docker 部署 + +### 快速启动 + +```bash +# 使用 docker-compose（推荐） +docker compose up -d + +# 或使用 docker run +docker run -d \ + --name jojo-code \ + -p 8080:8080 \ + -v $(pwd):/workspace \ + -e OPENAI_API_KEY=your-api-key \ + jojo-code +``` + +### 验证服务 + +```bash +# 健康检查 +curl http://localhost:8080/health + +# 查看日志 +docker logs jojo-code +``` + +### 挂载项目目录 + +```bash +docker run -d \ + -p 8080:8080 \ + -v /path/to/your/project:/workspace \ + -e OPENAI_API_KEY=xxx \ + jojo-code +``` + +### 停止服务 + +```bash +docker compose down +# 或 +docker stop jojo-code +``` + +## 生产环境建议 + +### 安全配置 + +1. **不要暴露公网**：WebSocket 服务默认监听 `0.0.0.0`，生产环境应通过反向代理暴露 +2. **启用 HTTPS**：使用 nginx 或 Caddy 做 TLS 终端 +3. **限制访问**：配置 CORS 和速率限制 + +### nginx 反向代理配置 + +```nginx +server { + listen 443 ssl; + server_name jojo.example.com; + + ssl_certificate /path/to/cert.pem; + ssl_certificate_key /path/to/key.pem; + + location /ws { + proxy_pass http://127.0.0.1:8080; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + proxy_set_header Host $host; + proxy_read_timeout 86400; + } + + location / { + proxy_pass http://127.0.0.1:8080; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + } +} +``` + +### 日志管理 + +```bash +# 配置日志 +export JOJO_CODE_LOG_FILE=/var/log/jojo-code/server.log +export JOJO_CODE_LOG_LEVEL=INFO +export JOJO_CODE_LOG_FORMAT=json + +# 日志轮转（logrotate） +# /etc/logrotate.d/jojo-code +# /var/log/jojo-code/*.log { +# daily +# rotate 14 +# compress +# missingok +# notifempty +# } +``` + +### 健康检查与监控 + +```bash +# 健康检查端点 +curl http://localhost:8080/health + +# 返回示例: +# { +# "status": "healthy", +# "version": "0.2.0", +# "platform": "Linux", +# "python_version": "3.12.0", +# "cpu_percent": 15.2, +# "memory": {"total_gb": 16.0, "available_gb": 8.5, "percent": 46.9}, +# "uptime_seconds": 3600 +# } +``` + +## 故障排查 + +### 常见问题 + +| 问题 | 原因 | 解决方案 | +|------|------|----------| +| 连接被拒绝 | 服务未启动 | `jojo-code server start` | +| API Key 错误 | 环境变量未配置 | 检查 `.env` 文件 | +| 端口被占用 | 其他进程占用 8080 | 修改 `JOJO_CODE_PORT` | +| 内存不足 | 消息历史过长 | 配置 `max_tokens` 限制 | +| 响应缓慢 | LLM API 延迟 | 检查网络或切换模型 | + +### 调试模式 + +```bash +# 启用详细日志 +export JOJO_CODE_LOG_LEVEL=DEBUG + +# 前台运行查看输出 +jojo-code server start +``` diff --git "a/wiki/12-\346\217\222\344\273\266\347\263\273\347\273\237.md" "b/wiki/12-\346\217\222\344\273\266\347\263\273\347\273\237.md" new file mode 100644 index 0000000..6fb80e4 --- /dev/null +++ "b/wiki/12-\346\217\222\344\273\266\347\263\273\347\273\237.md" @@ -0,0 +1,355 @@ +# 插件系统 + +## 概述 + +jojo-Code 的插件系统允许通过插件扩展 Agent 的功能，包括自定义工具、生命周期钩子（Hooks）和安全沙箱配置。 + +插件系统由以下组件组成： + +| 组件 | 文件 | 职责 | +|------|------|------| +| BasePlugin | `plugin/base.py` | 插件基类，定义生命周期和元数据 | +| PluginRegistry | `plugin/registry.py` | 插件注册表，管理所有已加载的插件 | +| PluginLoader | `plugin/loader.py` | 插件加载器，从模块/文件/类加载插件 | +| PluginDiscovery | `plugin/discovery.py` | 插件发现，自动扫描目录和入口点 | +| PluginConfig | `plugin/config.py` | 插件配置管理，支持 YAML/TOML/环境变量 | +| HookDispatcher | `plugin/hooks.py` | 钩子分发器，管理生命周期事件 | + +## 快速开始 + +### 1. 创建一个最简单的插件 + +```python +from jojo_code.plugin import BasePlugin, PluginMetadata + +class MyPlugin(BasePlugin): + metadata = PluginMetadata( + name="my-plugin", + version="1.0.0", + description="My first plugin", + ) + + def on_load(self) -> None: + print("Plugin loaded!") + + def on_unload(self) -> None: + print("Plugin unloaded!") +``` + +### 2. 注册插件 + +```python +from jojo_code.plugin import PluginRegistry + +registry = PluginRegistry.get_instance() +registry.register("my-plugin", MyPlugin()) +``` + +### 3. 使用插件 + +```python +plugin = registry.get("my-plugin") +print(plugin.metadata.name) # "my-plugin" +``` + +## BasePlugin 基类 + +所有插件必须继承 `BasePlugin` 并实现 `on_load()` 和 `on_unload()` 方法。 + +### 必须实现的方法 + +| 方法 | 说明 | +|------|------| +| `on_load()` | 插件加载时调用 | +| `on_unload()` | 插件卸载时调用 | + +### 可选方法 + +| 方法 | 说明 | +|------|------| +| `get_tools()` | 返回插件提供的工具列表 | +| `get_hooks()` | 返回钩子处理器字典 | + +### 插件元数据 + +```python +PluginMetadata( + name="my-plugin", # 插件名称（必填） + version="1.0.0", # 版本号（必填） + description="Description", # 描述（必填） + author="Author", # 作者 + tags=["tag1", "tag2"], # 标签 + license="MIT", # 许可证 + home_url="https://...", # 主页 +) +``` + +### 权限级别 + +```python +from jojo_code.plugin import PluginPermission + +class MyPlugin(BasePlugin): + permission = PluginPermission.UNTRUSTED # 无文件/网络访问 + permission = PluginPermission.RESTRICTED # 有限文件访问，无网络 + permission = PluginPermission.TRUSTED # 完全访问 +``` + +### 沙箱配置 + +```python +from jojo_code.plugin import PluginSandbox + +class MyPlugin(BasePlugin): + sandbox = PluginSandbox( + restricted=True, + allowed_paths=["src/**", "tests/**"], + allowed_urls=["https://api.example.com"], + max_memory_mb=100, + ) +``` + +## 提供自定义工具 + +插件可以通过 `get_tools()` 方法向 Agent 注册工具： + +```python +from jojo_code.plugin import BasePlugin, PluginMetadata + +class GreetingPlugin(BasePlugin): + metadata = PluginMetadata( + name="greeting", + version="1.0.0", + description="Provides a greeting tool", + ) + + def on_load(self) -> None: + pass + + def on_unload(self) -> None: + pass + + def get_tools(self) -> list: + from langchain_core.tools import tool + + @tool + def greet(name: str, language: str = "en") -> str: + """Say hello to someone in a specified language. + + Args: + name: The name of the person to greet + language: Language code (en, zh, ja, es). Defaults to 'en'. + + Returns: + A greeting message + """ + greetings = { + "en": f"Hello, {name}!", + "zh": f"你好, {name}!", + "ja": f"こんにちは, {name}!", + "es": f"¡Hola, {name}!", + } + return greetings.get(language, greetings["en"]) + + return [greet] +``` + +## 钩子系统 + +钩子允许插件拦截和响应 Agent 生命周期事件。 + +### 可用钩子 + +| 钩子名 | 触发时机 | +|--------|----------| +| `before_tool_call` | 工具执行前 | +| `after_tool_call` | 工具执行后 | +| `before_agent_run` | Agent 运行前 | +| `after_agent_run` | Agent 运行后 | +| `on_error` | 发生错误时 | + +### 使用钩子装饰器 + +```python +from jojo_code.plugin import hook + +@hook("before_tool_call") +def log_tool_call(tool_name: str, args: dict) -> None: + print(f"Calling {tool_name} with {args}") +``` + +### 在插件中注册钩子 + +```python +from jojo_code.plugin import BasePlugin, PluginMetadata +from jojo_code.plugin.hooks import HOOK_BEFORE_TOOL_CALL, HOOK_AFTER_TOOL_CALL + +class LoggingPlugin(BasePlugin): + metadata = PluginMetadata(name="logging", version="1.0.0", description="") + + def on_load(self) -> None: + pass + + def on_unload(self) -> None: + pass + + def get_hooks(self) -> dict: + return { + HOOK_BEFORE_TOOL_CALL: self._before_tool, + HOOK_AFTER_TOOL_CALL: self._after_tool, + } + + def _before_tool(self, tool_name: str, args: dict) -> None: + print(f"[LOG] Before: {tool_name}") + + def _after_tool(self, tool_name: str, result: str) -> None: + print(f"[LOG] After: {tool_name} -> {result[:50]}") +``` + +### HookDispatcher + +```python +from jojo_code.plugin import HookDispatcher + +dispatcher = HookDispatcher() + +# 注册处理器 +dispatcher.register("my_hook", my_handler) + +# 分发事件 +results = dispatcher.dispatch("my_hook", arg1, arg2) + +# 检查是否有处理器 +dispatcher.has_handlers("my_hook") # True + +# 注销处理器 +dispatcher.unregister("my_hook", my_handler) +``` + +## 插件加载 + +### 从类加载 + +```python +from jojo_code.plugin import PluginLoader + +loader = PluginLoader() +plugin = loader.load_from_class(MyPlugin) +``` + +### 从文件加载 + +```python +plugin = loader.load_from_file("/path/to/my_plugin.py") +``` + +### 从模块加载 + +```python +plugin = loader.load_from_module("my_package.my_plugin") +``` + +## 插件发现 + +PluginDiscovery 可以自动扫描目录中的插件： + +```python +from pathlib import Path +from jojo_code.plugin import PluginDiscovery + +discovery = PluginDiscovery() + +# 从目录发现 +plugins = discovery.discover(Path("plugins/")) + +# 从单个文件发现 +plugins = discovery.discover(Path("my_plugin.py")) + +# 从入口点发现（setuptools entry_points） +plugins = discovery.discover_entry_points() +``` + +### 入口点注册 + +在 `pyproject.toml` 中注册插件： + +```toml +[project.entry-points."jojo_code.plugins"] +my-plugin = "my_package.plugin:MyPlugin" +``` + +## 插件配置 + +### 配置来源（优先级从高到低） + +1. 环境变量 +2. `plugin.yaml` +3. `pyproject.toml` + +### plugin.yaml 示例 + +```yaml +plugins: + enabled: + - code-review + - git + - my-custom-plugin + +plugin_settings: + code-review: + severity: high + my-custom-plugin: + api_key: "your-key" + timeout: 30 +``` + +### pyproject.toml 示例 + +```toml +[tool.jojo-code.plugins] +enabled = ["code-review", "git"] +``` + +### 环境变量 + +```bash +# 启用的插件列表（逗号分隔） +export JOJO_PLUGINS_ENABLED="plugin-a,plugin-b" + +# 单个插件启用/禁用 +export JOJO_PLUGIN_MYPLUGIN_ENABLED="true" + +# 插件配置（JSON 格式） +export JOJO_PLUGIN_MYPLUGIN_CONFIG='{"key": "value"}' +``` + +### 程序化配置 + +```python +from jojo_code.plugin import get_plugin_config + +config = get_plugin_config() +config.set("my_key", "my_value") +config.is_plugin_enabled("my-plugin") # True/False +config.get_plugin_setting("my-plugin", "timeout", default=30) +``` + +## 内置插件 + +jojo-Code 自带三个内置插件： + +| 插件 | 说明 | +|------|------| +| code-review | 代码审查（安全、质量、风格） | +| test-generator | 测试生成（单元测试、fixtures、mocks） | +| git | Git 操作（status、log、diff、branch） | + +## 最佳实践 + +1. **单一职责**：每个插件只做一件事 +2. **清晰的元数据**：填写完整的 `PluginMetadata` +3. **适当的权限**：使用最小必要权限级别 +4. **错误处理**：在钩子中捕获异常，避免影响 Agent 运行 +5. **资源清理**：在 `on_unload()` 中释放资源 +6. **类型注解**：使用 Python 类型注解 +7. **文档字符串**：工具的 docstring 会被 LLM 看到 diff --git "a/wiki/13-\346\212\200\350\203\275\347\263\273\347\273\237.md" "b/wiki/13-\346\212\200\350\203\275\347\263\273\347\273\237.md" new file mode 100644 index 0000000..4bf584d --- /dev/null +++ "b/wiki/13-\346\212\200\350\203\275\347\263\273\347\273\237.md" @@ -0,0 +1,214 @@ +# 技能系统 + +## 概述 + +技能（Skills）是可复用、可组合的能力单元，Agent 可以按名称调用。每个技能携带丰富的元数据（分类、标签、示例），支持从函数快速创建，也可通过类自定义。 + +## 模块结构 + +``` +src/jojo_code/skills/ +├── __init__.py # 模块入口 +├── base.py # BaseSkill 基类、@skill 装饰器 +├── manager.py # SkillManager 注册和执行 +├── types.py # SkillCategory、SkillScope、SkillResult 类型 +└── builtins.py # 内置技能 +``` + +## 快速开始 + +### 使用 @skill 装饰器 + +```python +from jojo_code.skills.base import skill +from jojo_code.skills.types import SkillCategory + +@skill( + name="my_skill", + description="Does something useful", + category=SkillCategory.CODE, + tags=["custom", "utility"], + examples=["Run my_skill on file.py"], +) +def my_skill(file_path: str) -> str: + """Process a file. + + Args: + file_path: Path to the file + + Returns: + Processing result + """ + return f"Processed {file_path}" +``` + +### 继承 BaseSkill + +```python +from jojo_code.skills.base import BaseSkill +from jojo_code.skills.types import SkillCategory + +class MyCustomSkill(BaseSkill): + def __init__(self): + super().__init__( + name="custom", + description="A custom skill", + category=SkillCategory.CUSTOM, + ) + + def execute(self, *args, **kwargs): + return "result" + + def validate(self, *args, **kwargs): + return True +``` + +### 注册和执行 + +```python +from jojo_code.skills.manager import SkillManager + +manager = SkillManager() + +# 注册装饰器技能 +manager.register(my_skill._skill_def) + +# 或直接注册函数 +skill_id = manager.register_function( + my_skill, + name="my_skill", + description="Does something useful", +) + +# 执行技能 +result = manager.execute(skill_id, "file.py") +print(result.output) + +# 搜索技能 +matches = manager.search("file") + +# 按分类列出 +from jojo_code.skills.types import SkillCategory +code_skills = manager.list_skills(category=SkillCategory.CODE) +``` + +## 技能分类 + +| 分类 | 说明 | 示例 | +|------|------|------| +| `WEB` | Web 操作 | 搜索、抓取 | +| `DATA` | 数据处理 | JSON、CSV、数学 | +| `CODE` | 代码操作 | 分析、格式化 | +| `FILE` | 文件操作 | 读取、写入 | +| `SYSTEM` | 系统操作 | Shell 命令 | +| `SEARCH` | 搜索操作 | 文本搜索、翻译 | +| `CUSTOM` | 用户自定义 | 任意 | + +## 内置技能 + +### Web 技能 +- **web_search** - 搜索网页信息 +- **web_fetch** - 抓取网页内容 + +### 文件技能 +- **read_file** - 读取文件内容 +- **write_file** - 写入文件 + +### 系统技能 +- **run_command** - 执行 Shell 命令 + +### 代码技能 +- **analyze_code** - 分析代码质量和结构 + +### 数据技能 +- **format_json** - 格式化 JSON 数据 +- **validate_json** - 验证 JSON 语法 +- **calculate** - 计算数学表达式 + +### 搜索技能 +- **translate** - 翻译文本 + +## 技能元数据 + +每个技能携带丰富元数据： + +```python +@skill( + name="example", # 技能名称 + description="An example", # 人类可读描述 + category=SkillCategory.CODE, # 分类 + tags=["example", "demo"], # 可搜索标签 + version="1.0.0", # 语义版本 + author="your-name", # 作者 + examples=["Run example"], # 使用示例 + requires=["read_file"], # 依赖 + scope=SkillScope.GLOBAL, # 可见范围 +) +``` + +## 技能范围 + +| 范围 | 说明 | +|------|------| +| `GLOBAL` | 全局可用（默认） | +| `SESSION` | 仅当前会话 | +| `PROJECT` | 仅当前项目 | + +## 执行结果 + +执行结果包装在 `SkillResult` 中： + +```python +result = manager.execute(skill_id, "input") + +print(result.success) # bool - 是否成功 +print(result.output) # Any - 结果数据 +print(result.error) # str or None - 错误信息 +print(result.duration_ms) # float - 执行时间（毫秒） +print(result.metadata) # dict - 额外信息 +``` + +## 转换为 LangChain 工具 + +技能可以转换为 LangChain `BaseTool` 实例供 Agent 使用： + +```python +from jojo_code.skills.base import create_skill_tool + +# 从技能创建 LangChain 工具 +tool = create_skill_tool(my_skill) +result = tool.invoke({"input": "value"}) +``` + +## CLI 集成 + +```bash +# 技能自动加载并可用 +jojo-code + +# Agent 可以在对话中按名称调用技能 +# "搜索 Python 教程" -> 触发 web_search 技能 +# "格式化这个 JSON" -> 触发 format_json 技能 +``` + +## 测试 + +```bash +# 运行技能系统测试 +uv run pytest tests/test_skills/ -v +``` + +## 最佳实践 + +1. **写清晰的描述** - Agent 根据描述选择技能 +2. **添加示例** - 帮助 Agent 理解何时使用 +3. **使用合适的分类** - 便于发现 +4. **优雅处理错误** - 返回错误字符串，不抛出异常 +5. **保持技能聚焦** - 一个技能 = 一种能力 +6. **添加标签** - 提高可搜索性 + +## 相关文档 + +- [工具系统](03-工具系统.md) - 技能与工具的关系 +- [插件系统](12-插件系统.md) - 插件可以提供技能 +- [开发指南](06-开发指南.md) - 如何添加新技能 diff --git "a/wiki/14-MCP\347\263\273\347\273\237.md" "b/wiki/14-MCP\347\263\273\347\273\237.md" new file mode 100644 index 0000000..950b39e --- /dev/null +++ "b/wiki/14-MCP\347\263\273\347\273\237.md" @@ -0,0 +1,230 @@ +# MCP (Model Context Protocol) 系统 + +## 概述 + +MCP 系统允许 jojo-code 连接到外部 MCP 服务器，发现并调用其提供的工具和资源。支持 stdio 和 HTTP/SSE 两种传输方式。 + +## 模块结构 + +``` +src/jojo_code/mcp/ +└── client.py # MCP 客户端实现 +``` + +## 核心概念 + +### MCPConfig（服务器配置） + +```python +from jojo_code.mcp.client import MCPConfig + +config = MCPConfig( + name="my-server", # 服务器名称 + url="npx mcp-server", # 服务器地址或命令 + transport="stdio", # 传输方式: stdio/http/sse + auth={"token": "xxx"}, # 认证信息（可选） + timeout=30.0, # 超时时间（秒） + retry=3, # 重试次数 +) +``` + +### MCPTool（工具定义） + +```python +from jojo_code.mcp.client import MCPTool + +tool = MCPTool( + name="read_file", + description="Read a file from disk", + input_schema={"type": "object", "properties": {"path": {"type": "string"}}}, +) +``` + +### MCPResource（资源定义） + +```python +from jojo_code.mcp.client import MCPResource + +resource = MCPResource( + uri="file:///home/user/README.md", + name="README.md", + description="Project README", + mime_type="text/markdown", +) +``` + +## 使用方式 + +### 基本连接 + +```python +import asyncio +from jojo_code.mcp.client import MCPClient, MCPConfig + +async def main(): + config = MCPConfig( + name="filesystem-server", + url="npx -y @modelcontextprotocol/server-filesystem /tmp", + transport="stdio", + ) + client = MCPClient(config) + + # 连接（自动发现工具） + await client.connect() + + # 列出可用工具 + for tool in client.list_tools(): + print(f" {tool.name}: {tool.description}") + + # 调用工具 + result = await client.call_tool("read_file", {"path": "/tmp/hello.txt"}) + print(result) + + # 关闭连接 + await client.close() + +asyncio.run(main()) +``` + +### HTTP 传输 + +```python +config = MCPConfig( + name="remote-server", + url="http://localhost:8080/mcp", + transport="http", + auth={"Authorization": "Bearer token123"}, +) +client = MCPClient(config) +await client.connect() +``` + +### 资源管理 + +```python +# 列出资源 +resources = await client.list_resources() +for res in resources: + print(f" {res.name} ({res.mime_type}): {res.uri}") + +# 读取资源 +content = await client.read_resource("file:///home/user/README.md") +print(content) +``` + +## MCPClientManager（多服务器管理） + +管理多个 MCP 服务器连接： + +```python +from jojo_code.mcp.client import MCPClientManager, MCPConfig + +manager = MCPClientManager() + +# 添加多个服务器 +manager.add_server(MCPConfig(name="fs", url="npx mcp-fs-server", transport="stdio")) +manager.add_server(MCPConfig(name="db", url="http://localhost:8081/mcp", transport="http")) + +# 连接所有 +await manager.connect_all() + +# 获取特定客户端 +fs_client = manager.get_client("fs") + +# 列出所有服务器 +print(manager.list_servers()) # ["fs", "db"] + +# 关闭所有 +await manager.close_all() +``` + +## 全局便捷函数 + +```python +from jojo_code.mcp.client import get_mcp_manager, add_mcp_server + +# 快速添加服务器 +client = add_mcp_server("my-server", "npx mcp-server", transport="stdio") + +# 获取全局管理器 +manager = get_mcp_manager() +``` + +## JSON-RPC 协议 + +MCP 客户端使用 JSON-RPC 2.0 协议与服务器通信： + +### 请求格式 + +```json +{ + "jsonrpc": "2.0", + "id": 1234567890.123, + "method": "tools/list", + "params": {} +} +``` + +### 支持的方法 + +| 方法 | 说明 | +|------|------| +| `tools/list` | 列出可用工具 | +| `tools/call` | 调用工具 | +| `resources/list` | 列出可用资源 | +| `resources/read` | 读取资源内容 | + +## 错误处理 + +```python +from jojo_code.core.exceptions import NetworkError + +try: + result = await client.call_tool("nonexistent", {}) +except NetworkError as e: + print(f"MCP 调用失败: {e}") +except ValueError as e: + print(f"配置错误: {e}") +``` + +## 配置示例 + +在 `.env` 中配置 MCP 服务器： + +```bash +# 不需要特殊环境变量，通过代码配置 +``` + +在代码中配置： + +```python +from jojo_code.mcp.client import add_mcp_server + +# 文件系统服务器 +add_mcp_server( + "filesystem", + "npx -y @modelcontextprotocol/server-filesystem /home/user/project", + transport="stdio", +) + +# 数据库服务器 +add_mcp_server( + "database", + "http://localhost:8081/mcp", + transport="http", + auth={"Authorization": "Bearer db-token"}, +) +``` + +## 测试 + +```bash +# 运行 MCP 测试 +uv run pytest tests/test_mcp/ -v +``` + +## 相关文档 + +- [项目结构](01-项目结构.md) - MCP 模块在项目中的位置 +- [工具系统](03-工具系统.md) - MCP 工具与内置工具的关系 +- [插件系统](12-插件系统.md) - 插件如何集成 MCP 工具 diff --git "a/wiki/15-\344\274\232\350\257\235\347\263\273\347\273\237.md" "b/wiki/15-\344\274\232\350\257\235\347\263\273\347\273\237.md" new file mode 100644 index 0000000..ac8f9f7 --- /dev/null +++ "b/wiki/15-\344\274\232\350\257\235\347\263\273\347\273\237.md" @@ -0,0 +1,279 @@ +# 会话系统 + +## 概述 + +会话系统管理 Agent 与用户之间的对话会话，提供会话的创建、持久化、恢复和消息管理功能。 + +## 模块结构 + +``` +src/jojo_code/session/ +├── __init__.py # 模块入口 +├── models.py # Session 和 Message 数据模型 +└── manager.py # SessionManager 会话管理器 +``` + +## 核心概念 + +### Message（消息） + +消息是会话中的单条记录，包含角色、内容和时间戳。 + +```python +from jojo_code.session.models import Message + +msg = Message(role="user", content="Hello") +print(msg.role) # "user" +print(msg.content) # "Hello" +print(msg.timestamp) # Unix 时间戳（float） +``` + +#### 支持的角色 + +| 角色 | 说明 | +|------|------| +| `user` | 用户输入 | +| `assistant` | Agent 回复 | +| `system` | 系统消息 | + +#### 序列化 + +```python +# 转换为字典 +d = msg.to_dict() +# {"role": "user", "content": "Hello", "timestamp": 1700000000.0} + +# 从字典恢复 +msg = Message.from_dict(d) +``` + +### Session（会话） + +会话是一次完整的对话过程，包含多个消息和元数据。 + +```python +from jojo_code.session.models import Session + +session = Session(id="session-001", user_id="user-1") +session.add_message("user", "Hello") +session.add_message("assistant", "Hi! How can I help?") + +print(len(session.messages)) # 2 +print(session.last_seen_at) # 更新的时间戳 +``` + +#### 数据模型 + +| 字段 | 类型 | 说明 | +|------|------|------| +| `id` | `str` | 会话唯一标识 | +| `user_id` | `str \| None` | 用户标识（可选） | +| `created_at` | `float` | 创建时间戳 | +| `last_seen_at` | `float` | 最后活跃时间戳 | +| `messages` | `list[Message]` | 消息列表 | +| `metadata` | `dict[str, str]` | 自定义元数据 | + +#### 序列化 + +```python +# 转换为字典（用于 JSON 持久化） +d = session.to_dict() + +# 从字典恢复 +session = Session.from_dict(d) +``` + +## SessionManager（会话管理器） + +`SessionManager` 提供会话的 CRUD 操作和 JSON 文件持久化。 + +### 初始化 + +```python +from jojo_code.session.manager import SessionManager + +manager = SessionManager(storage_dir="./sessions") +``` + +初始化时会自动创建 `storage_dir` 目录（如果不存在）。 + +### 创建会话 + +```python +# 基本创建 +session = manager.create_session() + +# 带用户标识 +session = manager.create_session(user_id="user-1") + +# 带元数据 +session = manager.create_session( + user_id="user-1", + metadata={"source": "cli", "version": "1.0"}, +) +``` + +每次创建会话会自动生成 UUID 作为 `session_id`，并立即保存到磁盘。 + +### 获取会话 + +```python +session = manager.get_session(session_id) +if session is None: + print("Session not found") +``` + +返回 `None` 表示会话不存在或 JSON 文件损坏。 + +### 添加消息 + +```python +manager.add_message(session_id, "user", "Hello") +manager.add_message(session_id, "assistant", "Hi there!") +``` + +添加消息会自动保存到磁盘。如果会话不存在，抛出 `ValueError`。 + +### 保存会话 + +```python +session.user_id = "modified" +manager.save_session(session) +``` + +直接保存会话对象到磁盘，可用于更新元数据或用户信息。 + +### 恢复会话 + +```python +session = manager.recover_session(session_id) +``` + +`recover_session` 是 `get_session` 的语义别名，用于明确表示"从持久化存储恢复会话"。 + +## 存储格式 + +会话以 JSON 文件形式存储，文件名为 `{session_id}.json`。 + +```json +{ + "id": "550e8400-e29b-41d4-a716-446655440000", + "user_id": "user-1", + "created_at": 1700000000.0, + "last_seen_at": 1700000060.0, + "messages": [ + { + "role": "user", + "content": "Hello", + "timestamp": 1700000000.0 + }, + { + "role": "assistant", + "content": "Hi! How can I help?", + "timestamp": 1700000005.0 + } + ], + "metadata": { + "source": "cli" + } +} +``` + +## 错误处理 + +### 损坏的 JSON 文件 + +如果 JSON 文件损坏（格式错误、截断、空文件），`get_session` 返回 `None` 而不是抛出异常。这是设计决策（Bug #7 修复），确保系统在文件损坏时不会崩溃。 + +```python +session = manager.get_session("corrupted-id") +# 返回 None，不抛出异常 +``` + +### 会话不存在 + +```python +# get_session 返回 None +session = manager.get_session("nonexistent") # None + +# add_message 抛出 ValueError +manager.add_message("nonexistent", "user", "msg") # ValueError +``` + +## 使用示例 + +### 基本对话流程 + +```python +from jojo_code.session.manager import SessionManager + +manager = SessionManager(storage_dir="./sessions") + +# 创建会话 +session = manager.create_session(user_id="cli-user") + +# 模拟对话 +manager.add_message(session.id, "user", "Read README.md") +manager.add_message(session.id, "assistant", "Here is the content of README.md...") + +# 恢复会话（例如重启后） +recovered = manager.recover_session(session.id) +print(f"Session has {len(recovered.messages)} messages") +``` + +### 多会话管理 + +```python +# 不同用户的独立会话 +s1 = manager.create_session(user_id="alice") +s2 = manager.create_session(user_id="bob") + +manager.add_message(s1.id, "user", "Help me with Python") +manager.add_message(s2.id, "user", "Help me with JavaScript") + +# 获取特定会话 +alice_session = manager.get_session(s1.id) +print(alice_session.messages[0].content) # "Help me with Python" +``` + +### 跨管理器持久化 + +```python +# 在进程 A 中创建会话 +sm1 = SessionManager(storage_dir="./shared_sessions") +session = sm1.create_session(user_id="u1") +sm1.add_message(session.id, "user", "Hello from process A") + +# 在进程 B 中恢复会话 +sm2 = SessionManager(storage_dir="./shared_sessions") +loaded = sm2.get_session(session.id) +print(loaded.messages[0].content) # "Hello from process A" +``` + +## 目录结构 + +``` +sessions/ +├── 550e8400-e29b-41d4-a716-446655440000.json +├── 6ba7b810-9dad-11d1-80b4-00c04fd430c8.json +└── ... +``` + +## 与记忆系统的关系 + +会话系统（`session/`）和记忆系统（`memory/`）是互补的： + +| 特性 | 会话系统 | 记忆系统 | +|------|----------|----------| +| 聚焦 | 会话 CRUD 和持久化 | Token 计数、压缩、检索 | +| 存储 | JSON 文件 | JSON 文件 / 内存 | +| API | `SessionManager` | `ConversationMemory`, `ShortTermMemory`, `LongTermMemory` | +| 消息格式 | `Message` dataclass | LangChain `BaseMessage` | + +会话系统适合管理独立的对话记录，记忆系统适合管理 Agent 的上下文窗口和长期知识。 + +## 相关文档 + +- [记忆系统](05-记忆系统.md) - 对话记忆和 Token 管理 +- [项目结构](01-项目结构.md) - session 模块在项目中的位置 +- [Agent 架构](02-Agent架构.md) - Agent 如何使用会话 diff --git a/wiki/Home.md b/wiki/Home.md new file mode 100644 index 0000000..31cbd8a --- /dev/null +++ b/wiki/Home.md @@ -0,0 +1,85 @@ +# jojo-Code Wiki + +Welcome to the jojo-Code wiki. jojo-Code is a coding agent built with LangGraph, featuring a Textual TUI, WebSocket server, plugin system, and 40+ tools. + +## Quick Start + +```bash +# Install dependencies +uv sync + +# Configure API Key +cp .env.example .env + +# Run CLI +uv run jojo-code +``` + +## Architecture + +``` +CLI Layer (cli/) -> Server (server/) -> Agent Loop (agent/) -> Tool Layer (tools/) + WebSocket/SSE LangGraph StateGraph 40+ tools +``` + +## Core Documentation + +| Document | Description | +|----------|-------------| +| [Project Structure](01-项目结构.md) | Directory layout, core files, architecture layers | +| [Agent Architecture](02-Agent架构.md) | LangGraph state machine design, node implementations, execution flow | +| [Tool System](03-工具系统.md) | Tool registration, tool definitions, adding new tools | +| [LLM Configuration](04-LLM配置.md) | API configuration, supported models, priority rules | +| [Memory System](05-记忆系统.md) | Conversation memory, token counting, auto-compression | +| [Development Guide](06-开发指南.md) | Environment setup, common commands, code style | +| [Permission System](07-权限系统.md) | Path isolation, command filtering, user confirmation, audit logging | +| [Server Deployment](11-服务器部署.md) | WebSocket server, Docker deployment, production configuration | + +## Subsystems + +| Document | Description | +|----------|-------------| +| [MCP System](14-MCP系统.md) | MCP protocol client: connect to servers, call tools, manage resources | +| [Plugin System](12-插件系统.md) | Plugin development: lifecycle, hook system, permission control, discovery | +| [Skills System](13-技能系统.md) | Skill definition and management: decorators, categories, execution, LangChain integration | +| [Session System](15-会话系统.md) | Session management: Session/Message models, persistence, recovery | + +## Design Documents + +| Document | Description | +|----------|-------------| +| [AgentOps](10-AgentOps.md) | AgentOps system: tracing, metrics, export, dashboard | +| [Tool Permission Design](tool-permission-design.md) | Detailed design for tool permission restrictions | +| [Development Plan](DEV_PLAN.md) | Core capability development roadmap | + +## Review and Process + +| Document | Description | +|----------|-------------| +| [Code Review Record](08-代码审查记录.md) | PR #3 code review report | +| [PR Interaction Enhancement](09-PR交互增强.md) | CLI interaction enhancement details | +| [Enhancement Summary](ENHANCEMENT_SUMMARY.md) | Tool enhancement feature summary | +| [Final Status Report](FINAL_STATUS.md) | Enhancement completion status | +| [PR Description](PR_DESCRIPTION.md) | PR description for enhanced tool set | +| [Submit Guide](SUBMIT_GUIDE.md) | Pre-submission checklist and guide | + +## Key Concepts + +### Agent Loop + +``` +Thinking -> Tool Call -> Execute -> Observe -> Thinking -> ... +``` + +### Three-Layer Architecture + +``` +CLI Layer -> Agent Loop -> Tool Layer +``` + +### Core Technologies + +- **LangGraph**: State machine driven Agent loop +- **LangChain**: Tool definitions and LLM abstraction +- **Pydantic**: Configuration and state management +- **Textual**: Terminal UI framework diff --git a/wiki/README.md b/wiki/README.md index e2110c6..8260f0f 100644 --- a/wiki/README.md +++ b/wiki/README.md @@ -13,13 +13,15 @@ jojo-Code 是一个基于 LangGraph 构建的迷你编程 Agent，用于学习 A | [05-记忆系统](05-记忆系统.md) | 对话记忆、Token 计数、自动压缩 | | [06-开发指南](06-开发指南.md) | 环境设置、常用命令、代码风格 | | [07-权限系统](07-权限系统.md) | 路径隔离、命令过滤、用户确认、审计日志 | +| [11-服务器部署](11-服务器部署.md) | WebSocket 服务器、Docker 部署、生产环境配置 | | [08-代码审查记录](08-代码审查记录.md) | PR #3 代码审查报告 | | [09-PR交互增强](09-PR交互增强.md) | PR #3 CLI 交互增强功能说明 | +| [10-AgentOps](10-AgentOps.md) | AgentOps 运维体系：追踪、指标、导出、监控面板 | +| [12-插件系统](12-插件系统.md) | 插件开发：生命周期、钩子系统、权限控制、发现加载 | +| [13-技能系统](13-技能系统.md) | 技能定义与管理：装饰器、分类、执行、LangChain 集成 | +| [14-MCP系统](14-MCP系统.md) | MCP 客户端：连接外部服务器、工具发现、资源管理 | +| [15-会话系统](15-会话系统.md) | 会话管理：Session/Message 模型、持久化、恢复 | | [DEV_PLAN](DEV_PLAN.md) | 核心能力开发计划 | -| [TASK_PLAN_MODE](TASK_PLAN_MODE.md) | Plan 模式任务需求 | -| [TASK_PROJECT_CONTEXT](TASK_PROJECT_CONTEXT.md) | 项目上下文任务需求 | -| [TASK_SESSION_RECOVERY](TASK_SESSION_RECOVERY.md) | 会话恢复任务需求 | -| [TASK_WEB_SEARCH](TASK_WEB_SEARCH.md) | Web 搜索任务需求 | | [ENHANCEMENT_SUMMARY](ENHANCEMENT_SUMMARY.md) | 工具增强功能总结 | | [FINAL_STATUS](FINAL_STATUS.md) | 最终状态报告 | | [PR_DESCRIPTION](PR_DESCRIPTION.md) | PR 描述文档 | @@ -58,4 +60,4 @@ CLI Layer → Agent Loop → Tool Layer - **LangGraph**: 状态机驱动的 Agent 循环 - **LangChain**: 工具定义和 LLM 抽象 - **Pydantic**: 配置和状态管理 -- **Rich/Prompt Toolkit**: 交互式 CLI +- **Textual**: 终端 UI 框架