Introduce Testing Plan and Implement Runtime Module Tests

.gitignore

@@ -74,6 +74,15 @@ build/
 .pytest_cache/
 _version.py
 
+# Test coverage and reports
+htmlcov/
+.coverage
+coverage.xml
+test-report.html
+test-results.xml
+*.cover
+.hypothesis/
+
 # Virtual environments
 .venv/
 venv/

tests/README.md

@@ -0,0 +1,27 @@
+# Microsoft Agent 365 SDK Tests
+
+Unit and integration tests for the Microsoft Agent 365-Python SDK. This test suite ensures reliability, maintainability, and quality across all modules including runtime, tooling, notifications, and observability extensions.
+
+## Usage
+
+For detailed instructions on running tests and generating coverage reports, see:
+
+- **[Test Plan](TEST_PLAN.md)** - Comprehensive testing strategy and implementation roadmap
+- **[Running Tests](RUNNING_TESTS.md)** - Complete guide for installation, running tests, generating coverage reports, and troubleshooting
+
+## Support
+
+For issues, questions, or feedback:
+
+- File issues in the [GitHub Issues](https://github.com/microsoft/Agent365-python/issues) section
+- See the [main documentation](../README.md) for more information
+
+## Trademarks
+
+*Microsoft, Windows, Microsoft Azure and/or other Microsoft products and services referenced in the documentation may be either trademarks or registered trademarks of Microsoft in the United States and/or other countries. The licenses for this project do not grant you rights to use any Microsoft names, logos, or trademarks. Microsoft's general trademark guidelines can be found at http://go.microsoft.com/fwlink/?LinkID=254653.*
+
+## License
+
+Copyright (c) Microsoft Corporation. All rights reserved.
+
+Licensed under the MIT License - see the [LICENSE](../LICENSE.md) file for details.

tests/RUNNING_TESTS.md

@@ -0,0 +1,159 @@
+# Running Unit Tests for Agent365-Python SDK
+
+This guide covers setting up and running tests.
+
+---
+
+## Prerequisites
+
+### 1. Create Virtual Environment
+
+```powershell
+# Create virtual environment using uv
+uv venv
+
+# Activate the virtual environment
+.\.venv\Scripts\Activate.ps1
+```
+
+### 2. Configure Python Environment
+
+1. Press `Ctrl+Shift+P`
+2. Type "Python: Select Interpreter"
+3. Choose the `.venv` interpreter from the list
+
+### 3. Install Dependencies
+
+```powershell
+# Test framework and reporting
+uv pip install pytest pytest-asyncio pytest-mock pytest-cov pytest-html wrapt
+
+# SDK core libraries
+uv pip install -e libraries/microsoft-agents-a365-runtime -e libraries/microsoft-agents-a365-notifications -e libraries/microsoft-agents-a365-observability-core -e libraries/microsoft-agents-a365-tooling
+
+# Framework extension libraries
+uv pip install -e libraries/microsoft-agents-a365-observability-extensions-langchain -e libraries/microsoft-agents-a365-observability-extensions-openai -e libraries/microsoft-agents-a365-observability-extensions-semantickernel -e libraries/microsoft-agents-a365-observability-extensions-agentframework -e libraries/microsoft-agents-a365-tooling-extensions-agentframework -e libraries/microsoft-agents-a365-tooling-extensions-azureaifoundry -e libraries/microsoft-agents-a365-tooling-extensions-openai -e libraries/microsoft-agents-a365-tooling-extensions-semantickernel
+```
+
+---
+
+## Test Structure
+
+> **Note:** This structure will be updated as new tests are added.
+
+```plaintext
+tests/
+├── runtime/                           # Runtime tests
+├── observability/                     # Observability tests
+├── tooling/                           # Tooling tests
+└── notifications/                     # Notifications tests
+```
+
+---
+
+## Running Tests in VS Code (Optional)
+
+### Test Explorer
+
+1. Click the beaker icon in the Activity Bar or press `Ctrl+Shift+P` → "Test: Focus on Test Explorer View"
+2. Click the play button to run tests (all/folder/file/individual)
+3. Right-click → "Debug Test" to debug with breakpoints
+
+### Command Palette
+
+- `Test: Run All Tests`
+- `Test: Run Tests in Current File`
+- `Test: Debug Tests in Current File`
+
+---
+
+## Running Tests from Command Line
+
+```powershell
+# Run all tests
+python -m pytest tests/
+
+# Run specific module/file
+python -m pytest tests/runtime/
+python -m pytest tests/runtime/test_environment_utils.py
+
+# Run with options
+python -m pytest tests/ -v                    # Verbose
+python -m pytest tests/ -x                    # Stop on first failure
+python -m pytest tests/ -k "environment"      # Pattern matching
+python -m pytest --lf                         # Re-run failed tests
+```
+
+---
+
+## Generating Reports
+
+### HTML Reports
+
+```powershell
+# Coverage report
+python -m pytest tests/ --cov=libraries --cov-report=html -v
+
+# View reports
+start htmlcov\index.html
+
+# Test report (requires: uv pip install pytest-html)
+python -m pytest tests/ --html=test-report.html --self-contained-html
+
+# View reports
+start test-report.html
+
+# Combined (requires: uv pip install pytest-html)
+python -m pytest tests/ --cov=libraries --cov-report=html --html=test-report.html --self-contained-html -v
+
+# View reports
+start htmlcov\index.html
+```
+
+### CI/CD Reports
+
+```powershell
+# XML reports for CI/CD pipelines
+python -m pytest tests/ --cov=libraries --cov-report=xml --junitxml=test-results.xml
+
+# View reports
+start test-results.xml
+start coverage.xml
+```
+
+---
+
+## Troubleshooting
+
+### Common Issues
+
+| Issue | Solution |
+|-------|----------|
+| **Test loading failed** | Clean pyproject.toml, reinstall packages, restart VS Code |
+| **ImportError: No module named 'pytest'** | `uv pip install pytest pytest-asyncio pytest-mock` |
+| **ImportError: No module named 'microsoft_agents_a365'** | `uv pip install -e .` |
+| **Tests not discovered** | Refresh Test Explorer or restart VS Code |
+
+### Fix Steps
+
+If tests fail to discover or import errors occur:
+
+**1. Clean pyproject.toml**
+
+```powershell
+$content = Get-Content "pyproject.toml" -Raw
+$fixed = $content -replace "`r`n", "`n"
+$fixed | Set-Content "pyproject.toml" -NoNewline
+```
+
+**2. Reinstall packages**
+
+```powershell
+uv pip install -e .
+```
+
+**3. Restart VS Code**
+
+- Close completely and reopen
+- Wait for Python extension to reload
+- Refresh Test Explorer

tests/TEST_PLAN.md

@@ -0,0 +1,146 @@
+# Test Plan for Agent365-Python SDK
+
+> **Note:** This plan is under active development. Keep updating as testing progresses.
+
+**Version:** 1.0  
+**Date:** November 24, 2025  
+**Status:** Draft
+
+---
+
+## Overview
+
+### Current State
+- ✅ Unit tests exist for `observability` and `runtime` modules
+- ❌ Missing tests for `tooling` and `notifications` modules
+- ❌ No integration tests or CI/CD automation
+
+### Goals
+- Achieve **80%+ code coverage** across all modules
+- Implement integration tests for cross-module functionality
+- Integrate testing into CI/CD pipeline with coverage enforcement
+
+---
+
+## Testing Strategy
+
+**Framework:** `unittest` + `pytest` runner  
+**Coverage:** `pytest-cov`  
+**Mocking:** `unittest.mock`  
+**Async:** `pytest-asyncio`
+
+**Test Pattern:** AAA (Arrange → Act → Assert)  
+**Naming Convention:** `test_<method>_<condition>_<expected_result>`
+
+---
+
+## Implementation Roadmap
+
+| Phase | Deliverables | Priority |
+|-------|-------------|----------|
+| 1.1 | Runtime unit tests | ✅ Complete |
+| 1.2 | Tooling unit tests | HIGH |
+| 1.3 | Notifications unit tests | HIGH |
+| 1.4 | Expand observability tests | MEDIUM |
+| 1.5 | Tooling extension tests | LOW |
+| 2 | Integration tests | MEDIUM |
+| 3 | CI/CD automation | HIGH |
+
+---
+
+## Phase 1: Unit Tests
+
+### 1.1 Runtime Module
+**Priority:** HIGH
+
+| Module | Test File | Status |
+|--------|-----------|--------|
+| `power_platform_api_discovery.py` | `test_power_platform_api_discovery.py` | ✅ Complete |
+| `utility.py` | `test_utility.py` | ✅ Complete |
+| `environment_utils.py` | `test_environment_utils.py` | ✅ Complete |
+| `version_utils.py` | `test_version_utils.py` | ✅ Complete |
+
+---
+
+### 1.2 Tooling Module
+**Priority:** HIGH
+
+| Module | Test File | Status |
+|--------|-----------|--------|
+| `utils/utility.py` | `test_utility.py` | ❌ Missing |
+| `models/mcp_server_config.py` | `test_mcp_server_config.py` | ❌ Missing |
+| `services/mcp_tool_server_configuration_service.py` | `test_mcp_tool_server_configuration_service.py` | ❌ Missing |
+
+---
+
+### 1.3 Notifications Module
+**Priority:** HIGH
+
+| Module | Test File | Status |
+|--------|-----------|--------|
+| `models/agent_lifecycle_event.py` | `test_agent_lifecycle_event.py` | ❌ Missing |
+| `models/agent_notification_activity.py` | `test_agent_notification_activity.py` | ❌ Missing |
+| `models/email_reference.py` | `test_email_reference.py` | ❌ Missing |
+| `agent_notification.py` | `test_agent_notification.py` | ❌ Missing |
+
+---
+
+### 1.4 Observability Extensions
+**Priority:** MEDIUM
+
+| Extension | Status |
+|-----------|--------|
+| `agentframework` | ✅ Expand existing |
+| `langchain` | ✅ Expand existing |
+| `openai` | ✅ Expand existing |
+| `semantickernel` | ✅ Expand existing |
+
+---
+
+### 1.5 Tooling Extensions
+**Priority:** LOW
+
+| Extension | Status |
+|-----------|--------|
+| Agent Framework | ❌ Missing |
+| Azure AI Foundry | ❌ Missing |
+| OpenAI | ❌ Missing |
+| Semantic Kernel | ❌ Missing |
+
+---
+
+## Phase 2: Integration Tests
+
+**Priority:** MEDIUM
+
+| Integration | Status |
+|-------------|--------|
+| Runtime + Observability | ❌ Missing |
+| Tooling + Runtime | ❌ Missing |
+| Notifications + Runtime | ❌ Missing |
+| Agent Framework full flow | ❌ Missing |
+| LangChain full flow | ❌ Missing |
+
+---
+
+## Phase 3: CI/CD Integration
+
+**Priority:** HIGH
+
+| Component | Status |
+|-----------|--------|
+| GitHub Actions workflow | ❌ Missing |
+| Python matrix (3.9-3.12) | ❌ Missing |
+| Coverage enforcement (80%+) | ❌ Missing |
+| Codecov integration | ❌ Missing |
+| PR blocking on failures | ❌ Missing |
+
+---
+
+## Success Criteria
+
+- ✅ 80%+ code coverage for all modules
+- ✅ All tests pass independently
+- ✅ Full suite completes in < 30 seconds (unit) / < 5 minutes (full)
+- ✅ Automated test execution on all PRs
+- ✅ Coverage reports visible and enforced