Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
fb67fdc
CLI-13: Add LRU caching to CommandRegistry.get_suggestions()
dugshub Sep 27, 2025
4430677
CLI-13: Add comprehensive tests for command suggestion caching
dugshub Sep 27, 2025
f30dea7
CLI-15: Define pytest markers in pyproject.toml
dugshub Sep 27, 2025
e70540e
CLI-15: Add automatic test marking based on file location
dugshub Sep 27, 2025
bcdb739
CLI-15: Add explicit parser markers to parser tests
dugshub Sep 27, 2025
028a70c
CLI-15: Add executor markers to subprocess tests
dugshub Sep 27, 2025
f5d9e3b
CLI-15: Add design markers to design system tests
dugshub Sep 27, 2025
0ac910a
CLI-16: Update CLAUDE.md with parser system documentation
dugshub Sep 27, 2025
1b34e66
fix: Add runtime_checkable to Parser protocol
dugshub Sep 27, 2025
27b743d
style: Fix formatting in test_pipeline.py
dugshub Sep 27, 2025
421b901
feat(core): Add semantic types for parser system (CLI-11)
dugshub Sep 28, 2025
7f04b37
feat(parser): Add SemanticParseResult and SemanticContext
dugshub Sep 28, 2025
ecbbe7c
feat(parser): Add semantic error handling and command registry
dugshub Sep 28, 2025
f23966e
feat(parser): Add semantic parser and pipeline implementation
dugshub Sep 28, 2025
ba2f3af
test(core): Add comprehensive tests for semantic types
dugshub Sep 28, 2025
3f712ef
test(parser): Add tests for semantic parser components
dugshub Sep 28, 2025
3302fcb
test(integration): Add end-to-end tests for semantic type flow
dugshub Sep 28, 2025
5148627
fix(tests): Ensure newline at end of file in conftest.py
dugshub Sep 30, 2025
89cfa10
feat: Address PR feedback for semantic types implementation
dugshub Sep 30, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 17 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,9 @@ PYTHONPATH=src python3 -m pytest tests/unit/ui/design/ -v

# Test execution components
PYTHONPATH=src python3 -m pytest tests/unit/execution/ -v

# Test parser components
PYTHONPATH=src python3 -m pytest tests/unit/ui/parser/ -v
```

## Architecture Overview
Expand All @@ -60,6 +63,7 @@ src/cli_patterns/
├── execution/ # Runtime engine and subprocess execution
├── ui/ # User interface components
│ ├── design/ # Design system (themes, tokens, components)
│ ├── parser/ # Command parsing system
│ └── screens/ # Screen implementations (future)
└── cli.py # Main entry point
```
Expand All @@ -71,12 +75,20 @@ The UI uses a comprehensive design system with:
- **Component Registry**: Centralized component registration and theming
- **Rich Integration**: Built on Rich library for terminal rendering

### Parser System
The command parsing system provides flexible input interpretation:
- **Protocol-based**: All parsers implement the `Parser` protocol for consistency
- **Pipeline Architecture**: `ParserPipeline` routes input to appropriate parsers
- **Command Registry**: Manages command metadata and provides fuzzy-matching suggestions
- **Multiple Paradigms**: Supports text commands, shell pass-through (!), and extensible for more

### Key Protocols
- `WizardConfig`: Complete wizard definition
- `SessionState`: Runtime state management
- `ActionExecutor`: Protocol for action execution
- `OptionCollector`: Protocol for option collection
- `NavigationController`: Protocol for navigation
- `Parser`: Protocol for command parsers with consistent input interpretation

## Type System Requirements

Expand All @@ -103,19 +115,20 @@ Key testing focus areas:

### Completed Components
- Design system (themes, tokens, components, registry)
- Subprocess executor with async execution and themed output
- Subprocess executor with async execution and themed output (CLI-9)
- Interactive shell with prompt_toolkit (CLI-7)
- Command parser system with composable architecture (CLI-8)
- Command registry with fuzzy matching and suggestions
- Basic type definitions and protocols structure

### In Progress (Branch CLI-9)
- Subprocess executor enhancements
### In Progress
- Test coverage improvements
- Integration testing

### Pending Implementation
- Core wizard models and validation
- YAML/JSON definition loaders
- Python decorator system
- Interactive shell with prompt_toolkit
- Navigation controller
- Session state management
- CLI entry point
Expand Down
292 changes: 292 additions & 0 deletions docs/semantic-types.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,292 @@
# Semantic Types Guide

## Overview

CLI Patterns uses semantic types to provide compile-time type safety and prevent type confusion in the parser system. These types use Python's `NewType` feature to create distinct types at compile time with zero runtime overhead.

## Core Semantic Types

### Command and Parser Types
- `CommandId`: Unique identifier for commands (e.g., "help", "run", "test")
- `OptionKey`: Command option keys (e.g., "--verbose", "--output")
- `FlagName`: Boolean flag names (e.g., "--debug", "--quiet")
- `ArgumentValue`: String argument values
- `ParseMode`: Parser mode identifiers (e.g., "text", "shell", "semantic")
- `ContextKey`: Context variable keys for parser state

## When to Use Semantic Types

### Use Semantic Types When:
- Defining command identifiers in registries
- Passing option keys between parser components
- Storing parser context values
- Building parse results with typed fields
- Creating command suggestions and error messages

### Use Regular Strings When:
- Displaying user-facing messages
- Working with raw input before parsing
- Interfacing with external libraries
- Performance-critical hot paths (though overhead is minimal)

## Migration Guide

### Converting Existing Code

#### Before (using plain strings):
```python
def register_command(name: str, handler: Callable) -> None:
commands[name] = handler

def parse_command(input: str) -> tuple[str, dict[str, str]]:
command = input.split()[0]
options = parse_options(input)
return command, options
```

#### After (using semantic types):
```python
from cli_patterns.core.parser_types import CommandId, OptionKey, make_command_id, make_option_key

def register_command(name: CommandId, handler: Callable) -> None:
commands[name] = handler

def parse_command(input: str) -> tuple[CommandId, dict[OptionKey, str]]:
command = make_command_id(input.split()[0])
options = {make_option_key(k): v for k, v in parse_options(input).items()}
return command, options
```

### Conversion Methods

The semantic parser components provide bidirectional conversion:

```python
from cli_patterns.ui.parser.semantic_result import SemanticParseResult
from cli_patterns.ui.parser.types import ParseResult

# Convert from regular to semantic
regular_result = ParseResult(command="help", options={"verbose": "true"})
semantic_result = SemanticParseResult.from_parse_result(regular_result)

# Convert from semantic to regular
regular_again = semantic_result.to_parse_result()
```

## Factory Functions

Use factory functions to create semantic types:

```python
from cli_patterns.core.parser_types import (
make_command_id,
make_option_key,
make_flag_name,
make_argument_value,
make_parse_mode,
make_context_key
)

# Creating semantic types
cmd = make_command_id("help")
opt = make_option_key("--verbose")
flag = make_flag_name("--debug")
arg = make_argument_value("output.txt")
mode = make_parse_mode("semantic")
ctx_key = make_context_key("current_command")
```

## Type Aliases for Collections

Use provided type aliases for better readability:

```python
from cli_patterns.core.parser_types import CommandList, OptionDict, FlagSet

# Type-safe collections
commands: CommandList = [make_command_id("help"), make_command_id("run")]
options: OptionDict = {make_option_key("--output"): "file.txt"}
flags: FlagSet = {make_flag_name("--verbose"), make_flag_name("--debug")}
```

## Working with SemanticParseResult

The `SemanticParseResult` class provides a type-safe parse result:

```python
from cli_patterns.ui.parser.semantic_result import SemanticParseResult
from cli_patterns.core.parser_types import make_command_id, make_option_key

# Creating a semantic parse result
result = SemanticParseResult(
command=make_command_id("test"),
options={make_option_key("--coverage"): "true"},
arguments=["test_file.py"],
mode=make_parse_mode("text")
)

# Accessing typed fields
cmd: CommandId = result.command
opts: dict[OptionKey, str] = result.options
```

## Error Handling with Semantic Types

The `SemanticParseError` provides rich error context:

```python
from cli_patterns.ui.parser.semantic_errors import SemanticParseError
from cli_patterns.core.parser_types import make_command_id, make_option_key

# Creating semantic errors
error = SemanticParseError(
message="Unknown option",
command=make_command_id("test"),
invalid_option=make_option_key("--unknown"),
valid_options=[make_option_key("--verbose"), make_option_key("--output")],
suggestions=[make_command_id("test")]
)

# Accessing semantic fields
cmd: CommandId = error.command
invalid: OptionKey = error.invalid_option
valid: list[OptionKey] = error.valid_options
```

## Best Practices

### 1. Use Factory Functions
Always use factory functions rather than direct type casting:
```python
# Good
cmd = make_command_id("help")

# Avoid
cmd = CommandId("help") # Works but less clear
```

### 2. Maintain Type Consistency
Keep semantic types throughout your parser pipeline:
```python
def process_command(cmd: CommandId) -> CommandId:
# Process and return same type
return cmd

# Don't mix types unnecessarily
def process_command(cmd: CommandId) -> str: # Avoid unless needed
return str(cmd)
```

### 3. Use Type Aliases
Leverage type aliases for complex types:
```python
from cli_patterns.core.parser_types import CommandList, OptionDict

def get_commands() -> CommandList:
return [make_command_id("help"), make_command_id("run")]

def get_options() -> OptionDict:
return {make_option_key("--verbose"): "true"}
```

### 4. Document Type Conversions
When converting between semantic and regular types, document why:
```python
# Convert to string for display to user
display_name = str(command_id)

# Convert from user input to semantic type
command_id = make_command_id(user_input.strip())
```

## Extending the Type System

To add new semantic types:

1. Define the type in `core/parser_types.py`:
```python
from typing import NewType

# Define new semantic type
ConfigKey = NewType('ConfigKey', str)

# Add factory function
def make_config_key(value: str) -> ConfigKey:
return ConfigKey(value)

# Add type alias if needed
ConfigDict = dict[ConfigKey, str]
```

2. Add conversion support if needed:
```python
class SemanticConfig:
def __init__(self, config: dict[ConfigKey, str]):
self.config = config

@classmethod
def from_dict(cls, config: dict[str, str]) -> 'SemanticConfig':
return cls({make_config_key(k): v for k, v in config.items()})

def to_dict(self) -> dict[str, str]:
return {str(k): v for k, v in self.config.items()}
```

## Performance Considerations

Semantic types have **zero runtime overhead** because:
- `NewType` creates aliases at compile time only
- No runtime type checking or validation
- Type information is erased after compilation
- Factory functions are simple identity functions

## IDE Support

Semantic types improve IDE experience:
- Autocomplete shows only valid operations for each type
- Type checking catches mixing of incompatible types
- Better documentation through meaningful type names
- Refactoring tools understand type relationships

## Troubleshooting

### Common Issues

1. **Type Mismatch Errors**
```python
# Error: Cannot assign str to CommandId
cmd: CommandId = "help" # ❌

# Fix: Use factory function
cmd: CommandId = make_command_id("help") # ✅
```

2. **Missing Conversions**
```python
# Error: dict[str, str] not compatible with OptionDict
options: OptionDict = {"--verbose": "true"} # ❌

# Fix: Convert keys to semantic types
options: OptionDict = {make_option_key("--verbose"): "true"} # ✅
```

3. **JSON Serialization**
```python
import json
from cli_patterns.core.parser_types import CommandId, make_command_id

cmd = make_command_id("help")

# Semantic types serialize as strings
json_str = json.dumps({"command": cmd}) # Works fine

# Deserialize needs conversion
data = json.loads(json_str)
cmd = make_command_id(data["command"]) # Convert back to semantic type
```

## Further Reading

- [Python NewType Documentation](https://docs.python.org/3/library/typing.html#newtype)
- [MyPy Documentation on NewType](https://mypy.readthedocs.io/en/stable/more_types.html#newtypes)
- [CLI Patterns Architecture Guide](../CLAUDE.md)
14 changes: 8 additions & 6 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -102,12 +102,14 @@ testpaths = [
"tests",
]
markers = [
"unit: Unit tests for individual components",
"integration: Integration tests for component interactions",
"parser: Tests for parser system components",
"executor: Tests for execution/subprocess components",
"design: Tests for design system components",
"slow: Tests that take significant time to run",
"unit: Unit tests (tests/unit/)",
"integration: Integration tests (tests/integration/)",
"slow: Tests taking longer than 0.5 seconds",
"parser: Parser component tests",
"executor: Executor/execution component tests",
"design: Design system tests",
"ui: UI component tests",
"asyncio: Async tests (already in use)"
]

[dependency-groups]
Expand Down
Loading
Loading