-
Notifications
You must be signed in to change notification settings - Fork 4
Open
Labels
advancedenhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is neededperformance
Description
Description:
Optimize the accorr (adjusted circular correlation) metric computation, which is currently a performance bottleneck.
Context:
The accorr metric in analyses.py is slow for large datasets. Users have reported long computation times that limit practical use. We need to explore optimization strategies.
Current implementation location: hypyp/analyses.py
Optimization strategies to explore:
-
Pre-computing
- Identify repeated computations that can be cached
- Pre-compute frequency band signals once, reuse across metrics
-
Parallelization
- Use
joblibfor parallel computation across epochs/channels - Consider
numbaJIT compilation for inner loops - Explore
numpyvectorization improvements
- Use
-
Algorithm optimization
- Profile current code to find bottlenecks
- Review circular statistics libraries (e.g.,
pycircstat,astropy.stats) - Consider approximations for large datasets
Tasks:
- Profile current
accorrimplementation (usecProfileorline_profiler) - Identify top 3 bottlenecks
- Implement at least one optimization strategy
- Benchmark: before vs. after (varying data sizes)
- Ensure numerical accuracy is preserved (compare outputs)
- Add tests for optimized code
Acceptance Criteria:
- Profiling report identifying bottlenecks
- At least 2x speedup on benchmark dataset
- Benchmark script included in PR
- Unit tests pass (numerical accuracy preserved)
- Documentation of optimization approach
Benchmark template:
import time
import numpy as np
from hypyp import analyses
# Generate test data of varying sizes
sizes = [(10, 32, 1000), (50, 64, 2000), (100, 64, 5000)]
for n_epochs, n_channels, n_samples in sizes:
data = generate_test_data(n_epochs, n_channels, n_samples)
start = time.time()
result = analyses.compute_sync(data, metric='accorr')
elapsed = time.time() - start
print(f"Size {n_epochs}x{n_channels}x{n_samples}: {elapsed:.2f}s")Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
advancedenhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is neededperformance