11# Recursive Requirements Scan and Creation Tool (RRSCT)
22
3- This tool provides a ** Qt6 / PySide6 GUI** to recursively scan Python projects for dependencies and automatically create a clean, deduplicated ` requirements.txt ` file.
3+ This tool provides a ** Qt6 / PySide6 GUI** to recursively scan Python projects for dependencies and automatically create a clean, deduplicated ` requirements.txt ` file. It also includes a ** robust package installer ** that supports multiple package managers.
44
55It scans:
66- ** First 50 lines** of all ` .py ` files for ` import ` and ` from ` statements.
77- Any existing ` requirements.txt ` files.
88
9- Features:
9+ ## Features
10+
11+ ### Requirements Generation
1012- ✅ ** GUI Interface** built with Qt6 / PySide6
1113- ✅ ** Recursive scanning** of directories
1214- ✅ ** Exclude standard library** option
1315- ✅ ** Automatic cleanup** : deduplication, removing local paths, handling version conflicts
1416- ✅ ** PyPI validation** : only valid packages remain
17+ - ✅ ** In-memory storage** : Generate requirements without saving to file
18+ - ✅ ** Individual error handling** : One file's failure won't stop the entire scan
1519- ✅ ** Log panel** inside GUI for warnings and debug messages
1620
21+ ### Package Installation
22+ - ✅ ** Multiple package manager support** : pip, pipx, uv, poetry, conda, pip3
23+ - ✅ ** Install from file or memory** : No need to save before installing
24+ - ✅ ** Virtual environment detection** : Automatically detects venv vs global installation
25+ - ✅ ** Individual package installation** : Each package is installed separately
26+ - ✅ ** Robust error handling** : One failed package won't stop the rest
27+ - ✅ ** Real-time progress tracking** : Live updates during installation
28+ - ✅ ** Detailed installation summary** : Success, failed, and skipped packages
29+
1730---
1831
1932## Installation
@@ -35,18 +48,53 @@ Run the script:
3548python rrsct.py
3649```
3750
38- ### Steps:
51+ ### Generating Requirements
52+
39531 . ** Select Source Directory** → Folder containing your Python project(s)
40- 2 . ** Select Destination** → Where to save the generated ` requirements.txt `
41- 3 . Optionally check ** "Exclude standard libraries"**
42- 4 . Click ** Generate Master Requirements**
54+ 2 . ** (Optional) Select Destination** → Where to save the generated ` requirements.txt `
55+ - If no destination is selected, requirements are stored in memory only
56+ 3 . ** (Optional) Select Package Manager** → Choose from detected package managers
57+ 4 . Optionally check ** "Exclude standard libraries"** (recommended)
58+ 5 . Click ** Generate Master Requirements**
4359
4460The tool will:
4561- Scan ` .py ` and ` requirements.txt ` files recursively
62+ - Process each file individually (failures won't stop the scan)
4663- Deduplicate dependencies
47- - Validate packages on PyPI
48- - Show logs in the GUI
49- - Write the final list to ` requirements.txt `
64+ - Validate packages on PyPI (each package validated separately)
65+ - Show detailed logs in the GUI
66+ - Store requirements in memory and optionally save to file
67+
68+ ### Installing Requirements
69+
70+ 1 . Click ** Install Requirements from File**
71+ 2 . If you have generated requirements in memory:
72+ - ** Yes** → Install directly from memory (no file needed)
73+ - ** No** → Select a requirements file to install from
74+ - ** Cancel** → Do nothing
75+ 3 . Confirm the installation (shows virtual environment status)
76+ 4 . Watch real-time progress as each package installs
77+
78+ The installer will:
79+ - Detect if you're in a virtual environment or global Python
80+ - Install each package individually with the selected package manager
81+ - Continue even if individual packages fail
82+ - Provide a detailed summary of successes and failures
83+
84+ ---
85+
86+ ## Package Manager Support
87+
88+ | Package Manager | Install Command | Notes |
89+ | -----------------| -----------------| -------|
90+ | ** pip** | ` pip install <package> ` | Standard Python package installer |
91+ | ** pip3** | ` pip3 install <package> ` | Python 3 specific pip |
92+ | ** uv** | ` uv pip install <package> ` | Fast Python package installer |
93+ | ** poetry** | ` poetry add <package> ` | Dependency management tool |
94+ | ** conda** | ` conda install -y <package> ` | Cross-platform package manager |
95+ | ** pipx** | N/A | Skipped (for applications only, not libraries) |
96+
97+ The tool automatically detects which package managers are installed on your system.
5098
5199---
52100
@@ -55,62 +103,189 @@ The tool will:
55103The generated ` requirements.txt ` will:
56104- Contain ** only valid third-party dependencies**
57105- Deduplicate multiple versions, keeping the most restrictive or latest version
58- - Be directly usable with:
106+ - Include a header with metadata (package manager, total packages)
107+ - Be directly usable with any package manager
59108
60- ``` bash
61- cat requirements.txt | xargs -n 1 pip install
62- Get-Content requirements.txt | ForEach-Object { pip install $_ }
109+ Example output:
110+ ```
111+ # Generated by Master Requirements Generator
112+ # Package Manager: pip
113+ # Total packages: 4
114+
115+ numpy==1.26.4
116+ pandas==2.2.3
117+ requests>=2.31.0
118+ PySide6==6.7.0
63119```
64120
65121---
66122
67- # # Example
123+ ## Error Handling
124+
125+ This tool is designed with ** robust error handling** to prevent cascading failures:
126+
127+ ### During Generation:
128+ - Each file is processed independently
129+ - File read errors don't stop the scan
130+ - Invalid import lines are skipped with warnings
131+ - Each package validation happens separately
132+ - Network errors during PyPI checks won't halt the process
133+
134+ ### During Installation:
135+ - Each package is installed separately
136+ - Failed installations don't stop the rest
137+ - 2-minute timeout per package prevents hanging
138+ - Detailed error messages for each failure
139+ - Summary report at the end
68140
69- Sample log output in the GUI:
141+ ---
142+
143+ ## Examples
144+
145+ ### Sample Log Output
70146```
71- Skipping invalid line: random_text_here
72- Package not on PyPI: custom_package
73- Could not read file: some_binary_file.txt
147+ Detected package managers: pip, uv, conda
148+ ✓ Source directory selected: /path/to/project
149+ Starting scan with package manager: pip
150+ Found 150 files to process
151+ Found 45 unique imports/requirements
152+ Excluded 20 standard library modules
153+ Cleaned and merged to 40 packages
154+ Validating packages on PyPI...
155+ ✓ Validated: numpy
156+ ✓ Validated: pandas
157+ ✗ Package not found on PyPI: custom_local_package
158+ ✓ SUCCESS: Requirements generated!
159+ Total packages: 38
74160```
75161
76- Final ` requirements.txt ` example:
162+ ### Installation Log
77163```
78- numpy==1.26.4
79- pandas==2.2.3
80- requests> =2.31.0
81- PySide6==6.7.0
164+ Installing to: virtual environment
165+ Python: /path/to/venv/bin/python
166+ Package manager: pip
167+
168+ Installing from memory
169+ Found 38 packages to install
170+
171+ [1/38] Installing numpy...
172+ ✓ Successfully installed numpy
173+ [2/38] Installing pandas...
174+ ✓ Successfully installed pandas
175+ [3/38] Installing invalid-package...
176+ ✗ Failed to install invalid-package: No matching distribution found
177+
178+ ============================================================
179+ Installation Summary:
180+ ✓ Success: 36
181+ ✗ Failed: 2
182+ ⚠ Skipped: 0
183+ ============================================================
82184```
83185
84186---
85187
86188## Options
87189
88- | Option | Description |
89- | -----------------------------| ------------------------------------------------------|
90- | ** Exclude standard libraries** | Skips built-in Python modules like ` os` , ` sys` , etc. |
91- | ** Log Panel** | Shows skipped packages, errors, and warnings. |
190+ | Option | Description |
191+ | --------| -------------|
192+ | ** Select Source Directory** | Required: The root folder to scan for Python files |
193+ | ** Select Destination** | Optional: Where to save requirements.txt (omit to use memory only) |
194+ | ** Package Manager** | Choose which tool to use for installation (auto-detected) |
195+ | ** Exclude standard libraries** | Recommended: Skips built-in Python modules like ` os ` , ` sys ` , etc. |
196+ | ** Log Panel** | Real-time display of all operations, warnings, and errors |
197+
198+ ---
199+
200+ ## Workflow Examples
201+
202+ ### Quick Generate & Install (No File)
203+ 1 . Select source directory
204+ 2 . Click "Generate Master Requirements" (no destination needed)
205+ 3 . Click "Install Requirements from File"
206+ 4 . Choose "Yes" to install from memory
207+ 5 . Done! No file created.
208+
209+ ### Generate, Save, & Install Later
210+ 1 . Select source directory
211+ 2 . Select destination file
212+ 3 . Click "Generate Master Requirements"
213+ 4 . Requirements saved to file AND stored in memory
214+ 5 . Install from memory or file anytime
215+
216+ ### Install Existing Requirements File
217+ 1 . Click "Install Requirements from File"
218+ 2 . If you have memory requirements, click "No" to use a file
219+ 3 . Select the requirements.txt file
220+ 4 . Confirm installation
221+ 5 . Watch the progress
92222
93223---
94224
95225## Dependencies
96226
97- - [PySide6](https://pypi.org/project/PySide6/) - Qt6 Python bindings
98- - [requests](https://pypi.org/project/requests/) - for PyPI validation
99- - [packaging](https://pypi.org/project/packaging/) - for version parsing
227+ - [ PySide6] ( https://pypi.org/project/PySide6/ ) - Qt6 Python bindings for GUI
228+ - [ requests] ( https://pypi.org/project/requests/ ) - HTTP library for PyPI validation
229+ - [ packaging] ( https://pypi.org/project/packaging/ ) - Core utilities for version parsing
100230
101231Install all at once:
102232
103233``` bash
104234pip install PySide6 requests packaging
105235```
106236
237+ Or use the requirements.txt in this folder:
238+
239+ ``` bash
240+ pip install -r requirements.txt
241+ ```
242+
243+ ---
244+
245+ ## Troubleshooting
246+
247+ ### "No package managers detected"
248+ - Ensure at least pip is installed: ` pip --version `
249+ - Restart the tool after installing a package manager
250+
251+ ### "Permission denied" errors during installation
252+ - Use a virtual environment: ` python -m venv venv `
253+ - Or run with appropriate permissions
254+
255+ ### Installation hangs
256+ - Each package has a 2-minute timeout
257+ - Check your internet connection
258+ - Some packages may take longer to build
259+
260+ ### PyPI validation fails
261+ - Check your internet connection
262+ - Packages are still included if validation fails
263+ - Validation errors are logged but don't stop the process
264+
265+ ---
266+
267+ ## Advanced Usage
268+
269+ ### Custom Package Managers
270+ The tool automatically detects installed package managers. To add support for a new one, modify the ` detect_package_managers() ` function.
271+
272+ ### Encoding Issues
273+ The tool tries multiple encodings (utf-8-sig, utf-8, latin-1, cp1252) when reading files. Most encoding issues are handled automatically.
274+
275+ ### Large Projects
276+ - Processing hundreds of files works efficiently
277+ - Each file is processed independently
278+ - Progress is shown during generation and installation
279+
107280---
108281
109282## Future Enhancements
110283
111- - Save log output to a file
112- - Group dependencies by source (` .py` vs ` requirements.txt` )
113- - Export results in multiple formats (CSV, JSON)
284+ - Export results in multiple formats (CSV, JSON)
285+ - Batch file processing options
286+ - Custom package repository support
287+ - Requirements.txt diff viewer
288+ - Dependency graph visualization
114289
115290---
116291
0 commit comments