In individual/correlation.ipynb (cell 1), per-sample heart_rate series for Baseline/Test_01/Test_02/Test_03 — different lengths, different absolute timestamps — are placed into one DataFrame via reset_index(drop=True) and correlated with method='spearman'.
This aligns sample i of one session with sample i of another, even though those samples occur at unrelated wall-clock times and the sessions have different row counts (shorter columns get NaN-padded). There's no correspondence between the i-th beat of baseline and the i-th beat of a test session, and within-session samples are heavily autocorrelated, so they aren't independent observations. Any "HR correlation between sessions" read off this matrix is an artifact of arbitrary positional alignment.
Fix: don't correlate raw sample streams positionally. Either (a) reduce each session to per-question / per-window summary statistics and correlate those, or (b) resample/align both series onto a common time grid (merge_asof on timestamps) first — and note that within-session samples aren't independent.
In
individual/correlation.ipynb(cell 1), per-sampleheart_rateseries for Baseline/Test_01/Test_02/Test_03 — different lengths, different absolute timestamps — are placed into one DataFrame viareset_index(drop=True)and correlated withmethod='spearman'.This aligns sample i of one session with sample i of another, even though those samples occur at unrelated wall-clock times and the sessions have different row counts (shorter columns get NaN-padded). There's no correspondence between the i-th beat of baseline and the i-th beat of a test session, and within-session samples are heavily autocorrelated, so they aren't independent observations. Any "HR correlation between sessions" read off this matrix is an artifact of arbitrary positional alignment.
Fix: don't correlate raw sample streams positionally. Either (a) reduce each session to per-question / per-window summary statistics and correlate those, or (b) resample/align both series onto a common time grid (
merge_asofon timestamps) first — and note that within-session samples aren't independent.