Skip to content

Commit ee89247

Browse files
committed
Enhanced RRSCT tool and created modern Qt GUI for background/foreground remover
- RRSCT: Added multi-package manager support (pip, pipx, uv, poetry, conda) - RRSCT: Individual error handling to prevent cascading failures - RRSCT: Install requirements button with in-memory installation - RRSCT: Comprehensive README documentation - Remove Background: Created modern Qt GUI with dark theme - Remove Background: 4 operation modes (remove/replace BG/FG) - Remove Background: 3 replacement types (color, image, gradient) for both BG and FG - Remove Background: Full type safety with strict linting compliance - Remove Background: Comprehensive README with usage guide and troubleshooting
1 parent cba6cae commit ee89247

File tree

8 files changed

+2095
-143
lines changed

8 files changed

+2095
-143
lines changed

Recursive Requirements Scan and Creation Tool/README.md

Lines changed: 208 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,32 @@
11
# Recursive Requirements Scan and Creation Tool (RRSCT)
22

3-
This tool provides a **Qt6 / PySide6 GUI** to recursively scan Python projects for dependencies and automatically create a clean, deduplicated `requirements.txt` file.
3+
This tool provides a **Qt6 / PySide6 GUI** to recursively scan Python projects for dependencies and automatically create a clean, deduplicated `requirements.txt` file. It also includes a **robust package installer** that supports multiple package managers.
44

55
It scans:
66
- **First 50 lines** of all `.py` files for `import` and `from` statements.
77
- Any existing `requirements.txt` files.
88

9-
Features:
9+
## Features
10+
11+
### Requirements Generation
1012
-**GUI Interface** built with Qt6 / PySide6
1113
-**Recursive scanning** of directories
1214
-**Exclude standard library** option
1315
-**Automatic cleanup**: deduplication, removing local paths, handling version conflicts
1416
-**PyPI validation**: only valid packages remain
17+
-**In-memory storage**: Generate requirements without saving to file
18+
-**Individual error handling**: One file's failure won't stop the entire scan
1519
-**Log panel** inside GUI for warnings and debug messages
1620

21+
### Package Installation
22+
-**Multiple package manager support**: pip, pipx, uv, poetry, conda, pip3
23+
-**Install from file or memory**: No need to save before installing
24+
-**Virtual environment detection**: Automatically detects venv vs global installation
25+
-**Individual package installation**: Each package is installed separately
26+
-**Robust error handling**: One failed package won't stop the rest
27+
-**Real-time progress tracking**: Live updates during installation
28+
-**Detailed installation summary**: Success, failed, and skipped packages
29+
1730
---
1831

1932
## Installation
@@ -35,18 +48,53 @@ Run the script:
3548
python rrsct.py
3649
```
3750

38-
### Steps:
51+
### Generating Requirements
52+
3953
1. **Select Source Directory** → Folder containing your Python project(s)
40-
2. **Select Destination** → Where to save the generated `requirements.txt`
41-
3. Optionally check **"Exclude standard libraries"**
42-
4. Click **Generate Master Requirements**
54+
2. **(Optional) Select Destination** → Where to save the generated `requirements.txt`
55+
- If no destination is selected, requirements are stored in memory only
56+
3. **(Optional) Select Package Manager** → Choose from detected package managers
57+
4. Optionally check **"Exclude standard libraries"** (recommended)
58+
5. Click **Generate Master Requirements**
4359

4460
The tool will:
4561
- Scan `.py` and `requirements.txt` files recursively
62+
- Process each file individually (failures won't stop the scan)
4663
- Deduplicate dependencies
47-
- Validate packages on PyPI
48-
- Show logs in the GUI
49-
- Write the final list to `requirements.txt`
64+
- Validate packages on PyPI (each package validated separately)
65+
- Show detailed logs in the GUI
66+
- Store requirements in memory and optionally save to file
67+
68+
### Installing Requirements
69+
70+
1. Click **Install Requirements from File**
71+
2. If you have generated requirements in memory:
72+
- **Yes** → Install directly from memory (no file needed)
73+
- **No** → Select a requirements file to install from
74+
- **Cancel** → Do nothing
75+
3. Confirm the installation (shows virtual environment status)
76+
4. Watch real-time progress as each package installs
77+
78+
The installer will:
79+
- Detect if you're in a virtual environment or global Python
80+
- Install each package individually with the selected package manager
81+
- Continue even if individual packages fail
82+
- Provide a detailed summary of successes and failures
83+
84+
---
85+
86+
## Package Manager Support
87+
88+
| Package Manager | Install Command | Notes |
89+
|-----------------|-----------------|-------|
90+
| **pip** | `pip install <package>` | Standard Python package installer |
91+
| **pip3** | `pip3 install <package>` | Python 3 specific pip |
92+
| **uv** | `uv pip install <package>` | Fast Python package installer |
93+
| **poetry** | `poetry add <package>` | Dependency management tool |
94+
| **conda** | `conda install -y <package>` | Cross-platform package manager |
95+
| **pipx** | N/A | Skipped (for applications only, not libraries) |
96+
97+
The tool automatically detects which package managers are installed on your system.
5098

5199
---
52100

@@ -55,62 +103,189 @@ The tool will:
55103
The generated `requirements.txt` will:
56104
- Contain **only valid third-party dependencies**
57105
- Deduplicate multiple versions, keeping the most restrictive or latest version
58-
- Be directly usable with:
106+
- Include a header with metadata (package manager, total packages)
107+
- Be directly usable with any package manager
59108

60-
```bash
61-
cat requirements.txt | xargs -n 1 pip install
62-
Get-Content requirements.txt | ForEach-Object { pip install $_ }
109+
Example output:
110+
```
111+
# Generated by Master Requirements Generator
112+
# Package Manager: pip
113+
# Total packages: 4
114+
115+
numpy==1.26.4
116+
pandas==2.2.3
117+
requests>=2.31.0
118+
PySide6==6.7.0
63119
```
64120

65121
---
66122

67-
## Example
123+
## Error Handling
124+
125+
This tool is designed with **robust error handling** to prevent cascading failures:
126+
127+
### During Generation:
128+
- Each file is processed independently
129+
- File read errors don't stop the scan
130+
- Invalid import lines are skipped with warnings
131+
- Each package validation happens separately
132+
- Network errors during PyPI checks won't halt the process
133+
134+
### During Installation:
135+
- Each package is installed separately
136+
- Failed installations don't stop the rest
137+
- 2-minute timeout per package prevents hanging
138+
- Detailed error messages for each failure
139+
- Summary report at the end
68140

69-
Sample log output in the GUI:
141+
---
142+
143+
## Examples
144+
145+
### Sample Log Output
70146
```
71-
Skipping invalid line: random_text_here
72-
Package not on PyPI: custom_package
73-
Could not read file: some_binary_file.txt
147+
Detected package managers: pip, uv, conda
148+
✓ Source directory selected: /path/to/project
149+
Starting scan with package manager: pip
150+
Found 150 files to process
151+
Found 45 unique imports/requirements
152+
Excluded 20 standard library modules
153+
Cleaned and merged to 40 packages
154+
Validating packages on PyPI...
155+
✓ Validated: numpy
156+
✓ Validated: pandas
157+
✗ Package not found on PyPI: custom_local_package
158+
✓ SUCCESS: Requirements generated!
159+
Total packages: 38
74160
```
75161

76-
Final `requirements.txt` example:
162+
### Installation Log
77163
```
78-
numpy==1.26.4
79-
pandas==2.2.3
80-
requests>=2.31.0
81-
PySide6==6.7.0
164+
Installing to: virtual environment
165+
Python: /path/to/venv/bin/python
166+
Package manager: pip
167+
168+
Installing from memory
169+
Found 38 packages to install
170+
171+
[1/38] Installing numpy...
172+
✓ Successfully installed numpy
173+
[2/38] Installing pandas...
174+
✓ Successfully installed pandas
175+
[3/38] Installing invalid-package...
176+
✗ Failed to install invalid-package: No matching distribution found
177+
178+
============================================================
179+
Installation Summary:
180+
✓ Success: 36
181+
✗ Failed: 2
182+
⚠ Skipped: 0
183+
============================================================
82184
```
83185

84186
---
85187

86188
## Options
87189

88-
| Option | Description |
89-
|-----------------------------|------------------------------------------------------|
90-
| **Exclude standard libraries** | Skips built-in Python modules like `os`, `sys`, etc. |
91-
| **Log Panel** | Shows skipped packages, errors, and warnings. |
190+
| Option | Description |
191+
|--------|-------------|
192+
| **Select Source Directory** | Required: The root folder to scan for Python files |
193+
| **Select Destination** | Optional: Where to save requirements.txt (omit to use memory only) |
194+
| **Package Manager** | Choose which tool to use for installation (auto-detected) |
195+
| **Exclude standard libraries** | Recommended: Skips built-in Python modules like `os`, `sys`, etc. |
196+
| **Log Panel** | Real-time display of all operations, warnings, and errors |
197+
198+
---
199+
200+
## Workflow Examples
201+
202+
### Quick Generate & Install (No File)
203+
1. Select source directory
204+
2. Click "Generate Master Requirements" (no destination needed)
205+
3. Click "Install Requirements from File"
206+
4. Choose "Yes" to install from memory
207+
5. Done! No file created.
208+
209+
### Generate, Save, & Install Later
210+
1. Select source directory
211+
2. Select destination file
212+
3. Click "Generate Master Requirements"
213+
4. Requirements saved to file AND stored in memory
214+
5. Install from memory or file anytime
215+
216+
### Install Existing Requirements File
217+
1. Click "Install Requirements from File"
218+
2. If you have memory requirements, click "No" to use a file
219+
3. Select the requirements.txt file
220+
4. Confirm installation
221+
5. Watch the progress
92222

93223
---
94224

95225
## Dependencies
96226

97-
- [PySide6](https://pypi.org/project/PySide6/) - Qt6 Python bindings
98-
- [requests](https://pypi.org/project/requests/) - for PyPI validation
99-
- [packaging](https://pypi.org/project/packaging/) - for version parsing
227+
- [PySide6](https://pypi.org/project/PySide6/) - Qt6 Python bindings for GUI
228+
- [requests](https://pypi.org/project/requests/) - HTTP library for PyPI validation
229+
- [packaging](https://pypi.org/project/packaging/) - Core utilities for version parsing
100230

101231
Install all at once:
102232

103233
```bash
104234
pip install PySide6 requests packaging
105235
```
106236

237+
Or use the requirements.txt in this folder:
238+
239+
```bash
240+
pip install -r requirements.txt
241+
```
242+
243+
---
244+
245+
## Troubleshooting
246+
247+
### "No package managers detected"
248+
- Ensure at least pip is installed: `pip --version`
249+
- Restart the tool after installing a package manager
250+
251+
### "Permission denied" errors during installation
252+
- Use a virtual environment: `python -m venv venv`
253+
- Or run with appropriate permissions
254+
255+
### Installation hangs
256+
- Each package has a 2-minute timeout
257+
- Check your internet connection
258+
- Some packages may take longer to build
259+
260+
### PyPI validation fails
261+
- Check your internet connection
262+
- Packages are still included if validation fails
263+
- Validation errors are logged but don't stop the process
264+
265+
---
266+
267+
## Advanced Usage
268+
269+
### Custom Package Managers
270+
The tool automatically detects installed package managers. To add support for a new one, modify the `detect_package_managers()` function.
271+
272+
### Encoding Issues
273+
The tool tries multiple encodings (utf-8-sig, utf-8, latin-1, cp1252) when reading files. Most encoding issues are handled automatically.
274+
275+
### Large Projects
276+
- Processing hundreds of files works efficiently
277+
- Each file is processed independently
278+
- Progress is shown during generation and installation
279+
107280
---
108281

109282
## Future Enhancements
110283

111-
- Save log output to a file
112-
- Group dependencies by source (`.py` vs `requirements.txt`)
113-
- Export results in multiple formats (CSV, JSON)
284+
- Export results in multiple formats (CSV, JSON)
285+
- Batch file processing options
286+
- Custom package repository support
287+
- Requirements.txt diff viewer
288+
- Dependency graph visualization
114289

115290
---
116291

0 commit comments

Comments
 (0)