fix: resolve pipeline reload module caching issue

[hectorc98]

Fix: Pipeline Reload Module Caching Issue

Problem

The pipeline reload API (POST /pipelines/reload) was returning HTTP 200 responses but not actually reloading updated pipeline code. Users had to restart the entire pod to see pipeline changes take effect, which defeats the purpose of the reload functionality.

Root Cause

Python's module import system caches modules in sys.modules by name. When load_module_from_path() was called during pipeline reload:

1. importlib.util.spec_from_file_location() would create a module spec
2. spec.loader.exec_module() would execute the module
3. However, if the module name already existed in sys.modules, Python would return the cached version instead of re-executing the updated file content

This meant that pipeline code changes were read from disk but the cached module code continued to execute.

Solution

Added sys.modules cache clearing in load_module_from_path() before loading modules:

# Clear module from sys.modules cache to ensure fresh reload
# This fixes the issue where pipeline reload API would return 200 but
# continue using cached module code instead of reloading updated files
if module_name in sys.modules:
    del sys.modules[module_name]

Impact

• ✅ Pipeline reload API now works correctly - code changes are immediately active after calling /pipelines/reload
• ✅ No more pod restarts required - development workflow is much smoother
• ✅ Backward compatible - no breaking changes to existing functionality
• ✅ Minimal code change - targeted fix with clear documentation

Testing

• Verified all module loading flows through the fixed load_module_from_path() function
• Confirmed this is the only location in the codebase that needed the fix
• Created test script demonstrating the caching issue and fix
• All reload paths (API endpoint, file upload, file deletion) benefit from this change

Files Changed

• main.py: Added sys.modules cache clearing and documentation

Validation Steps

1. Deploy updated pipelines server
2. Upload or modify a pipeline file
3. Call POST /pipelines/reload
4. Verify updated pipeline code executes (without pod restart)

This fix resolves a significant development experience issue where pipeline changes appeared to be ignored by the reload API.