Skip to content

Commit 3524826

Browse files
committed
[v0.1.6] 2025-10-06
- Reconfigured wheel builds to keep all generated files in `pyhelios_build` directory - Fixed README.md badge so wheel builds will show 'passing' - Added missing files in `pyhelios/runtime/` directory to git control
1 parent f5cebfd commit 3524826

9 files changed

Lines changed: 449 additions & 242 deletions

File tree

.gitignore

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -9,16 +9,12 @@ build/
99
*.egg-info/
1010
*.egg
1111

12-
# Generated assets and libraries for wheel packaging (created by prepare_wheel.py)
13-
pyhelios/assets/build/
14-
# Track Python source files in plugins but ignore binary libraries
15-
pyhelios/plugins/*.so
16-
pyhelios/plugins/*.dll
17-
pyhelios/plugins/*.dylib
18-
1912
# Auto-generated version file (setuptools-scm)
2013
pyhelios/_version.py
2114

15+
# Note: All other generated files (libraries, assets) are kept in pyhelios_build/
16+
# and are NOT in the source tree, so they don't need gitignore entries
17+
2218
# Auto-generated documentation
2319
docs/generated/
2420

@@ -46,4 +42,8 @@ docs/generated/
4642
*.~*
4743

4844
# Auto-synced WeberPennTree resources (automatically maintained)
49-
/plugins/
45+
/plugins/
46+
47+
# MCP memory server files (local development only)
48+
.aim/
49+
.mcp.json

CLAUDE.md

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -83,9 +83,16 @@ PyHelios now supports flexible plugin selection for customized builds based on y
8383

8484
#### Clean Helios-Style Project Structure
8585

86-
Following standard Helios C++ project conventions:
87-
- **pyhelios_build/** - Source files (CMakeLists.txt, main.cpp, etc.)
88-
- **pyhelios_build/build/** - All generated build artifacts including shared libraries for Python import
86+
Following standard Helios C++ project conventions with strict separation of source and generated files:
87+
- **pyhelios_build/** - Build source files (CMakeLists.txt, main.cpp, etc.)
88+
- **pyhelios_build/build/** - ALL generated build artifacts (libraries, assets, CMake cache)
89+
- **pyhelios/** - Python source code ONLY (no generated files)
90+
91+
**CRITICAL BUILD ARTIFACT POLICY:**
92+
- ALL generated files (libraries, compiled assets, build outputs) MUST remain in `pyhelios_build/`
93+
- NEVER copy generated files into the source tree (`pyhelios/`)
94+
- Wheel packaging uses custom build commands to copy from `pyhelios_build/` to temporary build directories
95+
- This prevents accidental deletion of source files and maintains clean git status
8996

9097
#### Plugin Selection Options
9198

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
<br><br>
22

3-
[![Build Wheels](https://github.com/PlantSimulationLab/PyHelios/actions/workflows/build-wheels.yml/badge.svg)](https://github.com/PlantSimulationLab/PyHelios/actions/workflows/build-wheels.yml) [![Test Linux](https://github.com/PlantSimulationLab/PyHelios/actions/workflows/pytest-linux.yml/badge.svg?branch=master)](https://github.com/PlantSimulationLab/PyHelios/actions/workflows/pytest-linux.yml) [![Test Windows](https://github.com/PlantSimulationLab/PyHelios/actions/workflows/pytest-windows.yml/badge.svg?branch=master)](https://github.com/PlantSimulationLab/PyHelios/actions/workflows/pytest-windows.yml) [![Test MacOS](https://github.com/PlantSimulationLab/PyHelios/actions/workflows/pytest-macos.yml/badge.svg?branch=master)](https://github.com/PlantSimulationLab/PyHelios/actions/workflows/pytest-macos.yml)
3+
[![Build Wheels](https://github.com/PlantSimulationLab/PyHelios/actions/workflows/build-wheels.yml/badge.svg?event=push)](https://github.com/PlantSimulationLab/PyHelios/actions/workflows/build-wheels.yml) [![Test Linux](https://github.com/PlantSimulationLab/PyHelios/actions/workflows/pytest-linux.yml/badge.svg?branch=master)](https://github.com/PlantSimulationLab/PyHelios/actions/workflows/pytest-linux.yml) [![Test Windows](https://github.com/PlantSimulationLab/PyHelios/actions/workflows/pytest-windows.yml/badge.svg?branch=master)](https://github.com/PlantSimulationLab/PyHelios/actions/workflows/pytest-windows.yml) [![Test MacOS](https://github.com/PlantSimulationLab/PyHelios/actions/workflows/pytest-macos.yml/badge.svg?branch=master)](https://github.com/PlantSimulationLab/PyHelios/actions/workflows/pytest-macos.yml)
44
[![PyPI version](https://img.shields.io/pypi/v/pyhelios3d.svg)](https://pypi.org/project/pyhelios3d/)
55

66
<div align="center">

build_scripts/prepare_wheel.py

Lines changed: 48 additions & 145 deletions
Original file line numberDiff line numberDiff line change
@@ -152,19 +152,21 @@ def find_windows_dll_dependencies(lib_path, project_root):
152152

153153
def copy_assets_for_packaging(project_root):
154154
"""
155-
Copy Helios assets to pyhelios/assets/build for packaging in wheels.
156-
155+
Organize Helios assets in pyhelios_build/build/assets_for_wheel for packaging in wheels.
156+
157+
This keeps all generated files in pyhelios_build/ and out of the source tree.
158+
157159
Args:
158160
project_root: Path to project root directory
159161
"""
160-
print("\nCopying assets for packaging...")
161-
162+
print("\nOrganizing assets for packaging...")
163+
162164
# Source directories
163165
build_dir = project_root / 'pyhelios_build' / 'build'
164166
helios_core = project_root / 'helios-core'
165-
166-
# Destination: pyhelios/assets/build
167-
dest_assets_dir = project_root / 'pyhelios' / 'assets' / 'build'
167+
168+
# Destination: pyhelios_build/build/assets_for_wheel (keeps generated files out of source tree)
169+
dest_assets_dir = project_root / 'pyhelios_build' / 'build' / 'assets_for_wheel'
168170
dest_assets_dir.mkdir(parents=True, exist_ok=True)
169171

170172
total_copied = 0
@@ -310,9 +312,12 @@ def copy_assets_for_packaging(project_root):
310312
print(f"[OK] Assets packaged in {dest_assets_dir}")
311313

312314
def build_and_prepare(build_args):
313-
"""Build native libraries and copy them to pyhelios/plugins for packaging."""
315+
"""Build native libraries and organize assets for wheel packaging.
316+
317+
All generated files remain in pyhelios_build/ - nothing is copied to source tree.
318+
"""
314319
print("Building native libraries for wheel...")
315-
320+
316321
script_dir = Path(__file__).parent
317322
project_root = script_dir.parent
318323

@@ -355,43 +360,13 @@ def build_and_prepare(build_args):
355360
print(f"Build stderr: {e.stderr}")
356361
sys.exit(1)
357362

358-
# Copy built libraries to packaging location
363+
# Validate built libraries exist (but don't copy them to source tree)
359364
build_lib_dir = project_root / 'pyhelios_build' / 'build' / 'lib'
360-
plugins_dir = project_root / 'pyhelios' / 'plugins'
361-
365+
362366
if not build_lib_dir.exists():
363-
print(f"Warning: Build directory {build_lib_dir} does not exist")
364-
return
365-
366-
# Verify source plugins directory exists (should contain Python files)
367-
if not plugins_dir.exists():
368-
print(f"ERROR: Source plugins directory not found: {plugins_dir}")
369-
print("The pyhelios/plugins/ directory with Python source files must exist")
370-
sys.exit(1)
371-
372-
# Verify it has the required Python files
373-
py_files = list(plugins_dir.glob('*.py'))
374-
if not py_files:
375-
print(f"ERROR: No Python files found in plugins directory: {plugins_dir}")
376-
print("The plugins directory must contain __init__.py and other Python source files")
367+
print(f"ERROR: Build directory {build_lib_dir} does not exist")
377368
sys.exit(1)
378-
379-
print(f"Found {len(py_files)} Python source files in plugins directory")
380-
381-
# Remove any existing binary libraries from the source directory
382-
# (These should not be there, but clean up just in case)
383-
system = platform.system()
384-
if system == 'Windows':
385-
old_libs = list(plugins_dir.glob('*.dll'))
386-
elif system == 'Darwin':
387-
old_libs = list(plugins_dir.glob('*.dylib'))
388-
else:
389-
old_libs = list(plugins_dir.glob('*.so*'))
390-
391-
for old_lib in old_libs:
392-
print(f"Removing old library: {old_lib.name}")
393-
old_lib.unlink()
394-
369+
395370
# Platform-specific library extensions
396371
system = platform.system()
397372
if system == 'Windows':
@@ -400,13 +375,13 @@ def build_and_prepare(build_args):
400375
patterns = ['*.dylib']
401376
else: # Linux and others
402377
patterns = ['*.so', '*.so.*']
403-
404-
# Find and validate libraries
378+
379+
# Find and validate libraries (they stay in pyhelios_build/build/lib/)
405380
found_libraries = []
406381
for pattern in patterns:
407382
for lib_file in build_lib_dir.glob(pattern):
408383
found_libraries.append(lib_file)
409-
384+
410385
if not found_libraries:
411386
print(f"ERROR: No libraries found matching patterns {patterns} in {build_lib_dir}")
412387
print("Available files in build directory:")
@@ -416,111 +391,38 @@ def build_and_prepare(build_args):
416391
except:
417392
print(" (cannot list directory contents)")
418393
sys.exit(1)
419-
420-
# Copy and validate libraries
421-
copied_count = 0
394+
395+
# Validate libraries (but keep them in build directory)
396+
validated_count = 0
422397
failed_count = 0
423-
398+
424399
for lib_file in found_libraries:
425-
print(f"\nProcessing library: {lib_file.name}")
426-
427-
# Validate library before copying
400+
print(f"\nValidating library: {lib_file.name}")
401+
402+
# Validate library
428403
if not validate_library(lib_file):
429-
print(f"[WARNING] Skipping invalid library: {lib_file.name}")
404+
print(f"[WARNING] Invalid library: {lib_file.name}")
430405
failed_count += 1
431406
continue
432-
433-
# Copy library
434-
dest_file = plugins_dir / lib_file.name
435-
try:
436-
shutil.copy2(lib_file, dest_file)
437-
print(f"[OK] Copied: {lib_file.name} -> {dest_file}")
438-
439-
# Verify copied file
440-
if not dest_file.exists() or dest_file.stat().st_size != lib_file.stat().st_size:
441-
print(f"[ERROR] Copy verification failed for {lib_file.name}")
442-
failed_count += 1
443-
continue
444-
445-
copied_count += 1
446-
447-
except (OSError, PermissionError) as e:
448-
print(f"[ERROR] Failed to copy {lib_file.name}: {e}")
449-
failed_count += 1
450-
407+
408+
validated_count += 1
409+
451410
# Summary
452-
print(f"\n=== Library Copy Summary ===")
453-
print(f"Found: {len(found_libraries)} libraries")
454-
print(f"Copied: {copied_count} libraries")
411+
print(f"\n=== Library Validation Summary ===")
412+
print(f"Found: {len(found_libraries)} libraries in {build_lib_dir}")
413+
print(f"Validated: {validated_count} libraries")
455414
print(f"Failed: {failed_count} libraries")
456-
457-
if copied_count == 0:
458-
print("ERROR: No libraries were successfully copied!")
415+
416+
if validated_count == 0:
417+
print("ERROR: No valid libraries were found!")
459418
sys.exit(1)
460419
elif failed_count > 0:
461-
print(f"WARNING: {failed_count} libraries could not be copied")
462-
# Continue anyway - some failures might be acceptable
463-
464-
print(f"[OK] Successfully prepared {copied_count} libraries for packaging")
465-
466-
# Windows-specific: Bundle additional DLL dependencies
467-
if system == 'Windows' and copied_count > 0:
468-
print(f"\n=== Windows DLL Dependency Bundling ===")
469-
470-
# Find the main library (usually libhelios.dll or helios.dll)
471-
main_library = None
472-
for lib_file in found_libraries:
473-
if 'helios' in lib_file.name.lower() and lib_file.suffix == '.dll':
474-
main_library = lib_file
475-
break
476-
477-
if main_library:
478-
# Find all required DLL dependencies
479-
dependencies = find_windows_dll_dependencies(main_library, project_root)
480-
481-
dependency_copied = 0
482-
dependency_failed = 0
483-
484-
for dep_dll in dependencies:
485-
try:
486-
dest_dll = plugins_dir / dep_dll.name
487-
488-
# Skip if already exists (avoid overwriting main libraries)
489-
if dest_dll.exists():
490-
print(f"[SKIP] Dependency already bundled: {dep_dll.name}")
491-
continue
492-
493-
shutil.copy2(dep_dll, dest_dll)
494-
print(f"[OK] Bundled dependency: {dep_dll.name}")
495-
dependency_copied += 1
496-
497-
except (OSError, PermissionError) as e:
498-
print(f"[ERROR] Failed to bundle critical dependency {dep_dll.name}: {e}")
499-
print(f"This dependency is required for libhelios.dll to load properly on Windows systems")
500-
print(f"without development tools installed. The wheel will not work correctly.")
501-
dependency_failed += 1
502-
503-
print(f"\n=== Dependency Bundle Summary ===")
504-
print(f"Found: {len(dependencies)} DLL dependencies")
505-
print(f"Bundled: {dependency_copied} dependencies")
506-
print(f"Failed: {dependency_failed} dependencies")
507-
508-
# Fail-fast: If critical dependencies are missing, the wheel is broken
509-
if len(dependencies) > 0 and dependency_copied == 0:
510-
print(f"[ERROR] CRITICAL: No Windows DLL dependencies were bundled!")
511-
print(f"This means the wheel will fail to load on systems without development tools.")
512-
print(f"Required dependencies: {[dep.name for dep in dependencies]}")
513-
print(f"The wheel build cannot continue with missing critical dependencies.")
514-
sys.exit(1)
515-
elif dependency_failed > 0:
516-
print(f"[ERROR] CRITICAL: {dependency_failed} critical dependencies could not be bundled!")
517-
print(f"The wheel will not work properly on clean Windows systems.")
518-
print(f"All dependencies must be bundled for the wheel to function correctly.")
519-
sys.exit(1)
520-
else:
521-
print(f"[OK] All {dependency_copied} Windows DLL dependencies bundled successfully")
522-
else:
523-
print(f"[WARNING] Could not find main Helios library for dependency analysis")
420+
print(f"WARNING: {failed_count} libraries could not be validated")
421+
422+
print(f"[OK] Libraries remain in build directory for wheel packaging: {build_lib_dir}")
423+
424+
# Note: Windows DLL dependencies will be handled by setup.py during wheel packaging
425+
# No need to copy them to source tree
524426

525427
# Copy assets for packaging
526428
copy_assets_for_packaging(project_root)
@@ -538,9 +440,10 @@ def main():
538440
print()
539441
print("This script:")
540442
print(" 1. Calls build_scripts/build_helios.py with the provided arguments")
541-
print(" 2. Copies built libraries to pyhelios/plugins/ for wheel packaging")
542-
print(" 3. Copies required assets to pyhelios/assets/build/")
543-
print(" 4. Validates libraries can be loaded properly")
443+
print(" 2. Validates built libraries in pyhelios_build/build/lib/")
444+
print(" 3. Organizes assets in pyhelios_build/build/assets_for_wheel/")
445+
print(" 4. ALL files remain in pyhelios_build/ (NOT copied to source tree)")
446+
print(" 5. setup.py copies files from pyhelios_build/ during wheel creation")
544447
print()
545448
print("Common build arguments:")
546449
print(" --buildmode {debug,release,relwithdebinfo} CMake build type")

docs/CHANGELOG.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
# Changelog
22

3+
# [v0.1.6] 2025-10-06
4+
5+
- Reconfigured wheel builds to keep all generated files in `pyhelios_build` directory
6+
- Fixed README.md badge so wheel builds will show 'passing'
7+
- Added missing files in `pyhelios/runtime/` directory to git control
8+
39
# [v0.1.5] 2025-10-06
410

511
🚨++ New Plug-in Integrated ++ 🚨

docs/plugin_integration_guide.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -995,10 +995,12 @@ plugin_asset_dirs = {
995995
- **Exclude**: Source files (`.blend`, development tools, documentation)
996996
- **Verify**: Check helios-core/plugins/yourplugin/assets/ for directory structure
997997

998-
**Why This Is Required**: The working directory context manager looks for assets via the asset manager, which searches multiple paths including `pyhelios/assets/build/plugins/yourplugin/`. During wheel builds, assets must be copied to this location by `prepare_wheel.py` or the plugin will fail with "assets not found" errors.
998+
**Why This Is Required**: The working directory context manager looks for assets via the asset manager. During wheel builds, `prepare_wheel.py` organizes assets in `pyhelios_build/build/assets_for_wheel/`, then `setup.py`'s custom build command copies them into the wheel. This keeps the source tree (`pyhelios/`) clean of all generated files.
999999

10001000
**INTEGRATION RULE**: New plugins should follow the **Stage 1 pattern** (CMake + working directory context manager) like WeberPennTree, Visualizer, and PlantArchitecture, **AND** must be added to `plugin_asset_dirs` in `prepare_wheel.py` for wheel distribution.
10011001

1002+
**BUILD ARTIFACT POLICY**: All generated files (libraries, assets, build outputs) remain in `pyhelios_build/` and are NEVER copied to the source tree. Wheel packaging uses custom build commands to copy from `pyhelios_build/build/` to temporary build directories.
1003+
10021004
### 6.4 Plugin-Specific Asset Considerations
10031005

10041006
Some plugins may require special asset handling:

pyhelios/runtime/__init__.py

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
"""
2+
PyHelios runtime capability detection.
3+
4+
This module provides runtime detection of hardware and software capabilities
5+
for PyHelios functionality, used primarily for test skipping and graceful
6+
degradation when hardware is not available.
7+
"""
8+
9+
from .gpu_detector import is_gpu_runtime_available, get_gpu_runtime_info
10+
11+
__all__ = [
12+
'is_gpu_runtime_available',
13+
'get_gpu_runtime_info'
14+
]

0 commit comments

Comments
 (0)