(outdated) feat: add comprehensive pdd setup prompts for all LLM providers (#480)#483
Closed
niti-go wants to merge 17 commits intopromptdriven:mainfrom
Closed
(outdated) feat: add comprehensive pdd setup prompts for all LLM providers (#480)#483niti-go wants to merge 17 commits intopromptdriven:mainfrom
niti-go wants to merge 17 commits intopromptdriven:mainfrom
Conversation
…ptdriven#480) Create 7 new prompts to transform pdd setup into a comprehensive system bootstrapper that supports dynamic provider discovery, API key management, and interactive model selection across all LLM providers. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Delete model_selector_python.prompt and its example: interactive tier selection removed; adding a provider now auto-loads all its models - Delete api_key_validator_python.prompt and its example: replaced by model_tester which tests individual models via litellm.completion() directly instead of steering llm_invoke with PDD_MODEL_DEFAULT - Create model_tester_python.prompt and example: new "Test a model" menu option using litellm.completion() with direct api_key param, showing diagnostics, timing, and cost per call - Create cli_detector_python.prompt and example: new "Detect CLI tools" menu option leveraging get_available_agents() from agentic_common, cross-referencing API keys with installed CLIs - Rewrite setup_tool_python.prompt: new 6-option menu (Add provider, Remove models, Test model, Detect CLI, Init .pddrc, Done) with sub-menus, replacing the old 5-option flow that ended in model selection - Rewrite provider_manager_python.prompt: new functions add_api_key (auto-loads all models), remove_models_by_provider (comments out keys), and remove_individual_models, replacing add_or_fix_keys/remove_provider - Trim all prompts per prompting guide: remove implementation patterns, keep behavioral requirements, target 10-30% prompt-to-code ratio
- Store files in /pdd/setup, rather than main directory /pdd - Modify utils.py command to run pdd setup at the new correct file path - Still needs refinement on prompts + code to further improve interface, but is functional - Still needs example files and test files
This comment was marked as outdated.
This comment was marked as outdated.
- Show LiteLLM registry of providers and models when adding models rather than data/llm_model.csv - Add search feature to search for a provider - Ask user for API keys based on the models they want to add
- Fixed misleading "local — no key required" message: Changed to "(no key configured)" in yellow, since an empty api_key field doesn't necessarily mean local—it could mean the user skipped entering the env var name, and litellm will fall back to its own defaults.
- Expanded provider → API key mapping: Added 30 new providers to PROVIDER_API_KEY_MAP (including moonshot → MOONSHOT_API_KEY), so the setup flow now knows which env var to prompt for when users add models from lesser-known providers.
- Fixed a shell escaping bug in provider_manager.py discovered by the new tests: changed _save_key_to_api_env to use shlex.quote() instead of simple double-quoting, which was breaking API keys containing special characters like $, ", ', and backticks
- Immediate session availability: API keys are now loaded into os.environ when saved, allowing users to test models immediately within the same pdd setup session without restarting their shell
- Automatic shell RC integration: The source line (source ~/.pdd/api-env.{shell}) is automatically added to the user's shell startup file (~/.zshrc, ~/.bashrc, etc.) with shell-appropriate syntax, so new terminal sessions have the API keys available without manual configuration
- Created 4 comprehensive test files (165 tests total) for the new setup modules: test_api_key_scanner.py, test_litellm_registry.py, test_provider_manager.py, and test_cli_detector.py — including rigorous shell execution tests that actually run bash/zsh to verify API key escaping works correctly
- Starts by boostrapping agentic CLI - Then utomatically scans for API keys, populates llm_model.csv, checks for local LLMs, initializes .pddrc, and tests a model - Removed /setup directory, all setup-related files now exist in root /pdd directory - Users mostly just need to hit 'enter' to finish setup - Add testing files
I split this into a 4-step agentic task, but later I realized it's more reliable to do setup deterministically right now.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR restructures the PDD setup from a menu-driven system to a two-phase automatic configuration flow designed for minimal user friction. Phase 1 bootstraps an agentic CLI (Claude/Gemini/Codex) interactively. Phase 2 runs deterministic auto-configuration that scans API keys, configures models, checks local LLMs, and initializes .pddrc.
Changes:
- Replaces manual menu system with automatic CLI detection and streamlined configuration
- Adds 7 new Python modules for modular setup functionality
- Implements comprehensive API key scanning with source transparency
- Adds local LLM detection (Ollama, LM Studio)
- Complete test coverage rewrite from integration tests to unit tests
Reviewed changes
Copilot reviewed 34 out of 34 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| pdd/provider_manager.py | New module for provider management with registry search and CSV operations |
| pdd/cli_detector.py | New CLI detection and bootstrapping with interactive installation |
| pdd/litellm_registry.py | New LiteLLM registry wrapper for provider/model discovery |
| pdd/model_tester.py | New interactive model testing with diagnostics |
| pdd/pddrc_initializer.py | New .pddrc file creation with language detection |
| pdd/api_key_scanner.py | New API key scanner with multi-source detection |
| pdd/core/utils.py | Updated import path for setup_tool module |
| tests/* | Complete rewrite of tests from integration to unit tests |
| pdd/prompts/* | New prompt files for all modules |
| context/* | New example files demonstrating module usage |
| docs/* | Updated documentation for new setup flow |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Adds support for many more LiteLLM-supported providers (Vertex AI, AWS Bedrock, Azure, etc.) - The api_key column now supports pipe-delimited fields (e.g. VERTEXAI_PROJECT|VERTEXAI_LOCATION|GOOGLE_APPLICATION_CREDENTIALS) for providers whose auth requires multiple credentials - Updated pdd setup documentation - Update llm_invoke api_key handling to support the new pipe-delimited credentials format and generalized to remove provider-specific logic
niti-go
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This draft PR is outdated and will be deleted soon, see #544 for the new PR.
Summary (as of 2/17)
Restructures pdd setup from a menu-driven interactive menu into a streamlined automatic configuration flow. Makes it easier for users to setup quickly with little to no manual configuration required. Allows users to set up local LLMs and other less common providers.
Phase 1 — CLI Bootstrap: Detects installed agentic CLI tools (claude, gemini, codex), cross-references with available API keys, and offers to install a matching CLI if one isn't found. Required for pdd fix, pdd change, and pdd bug to work.
Phase 2 — Auto-Configuration (4 deterministic steps):
New Python Modules
New Example Files (context/)
The PR also has corresponding test and prompt files.
Documentation