[Feedback] Small peaks at residual plots

[hazeljhuang]

Feedback

Hi Dr.Aharoni, I'm Hazel working with Dr. Hecht! I used the fluorescence data analyzed with CaImAn from Dr. Hecht's recordings and tested it on CaTune. I went over the tutorial and tuned the decay time, sparsity, and rise time so that the green deconvolved traces fit with the rise of the raw traces and mostly do not contain multiple peaks in the spread of an event. The residuals are noisy in general but they have high peaks whenever there is a peak event in raw trace. I attached a few examples here.
figure 1 - medium decay time and sparsity, fit line captures most changes in raw traces but residuals are in pattern

In my understanding this is because the orange fit traces in my settings do not have enough magnitude at most peaks. I found that some ways to decrease this magnitude difference are to decrease the decay time and increase sparsity, but the peak pattern sometimes still exists.
figure 2 - small decay time, residuals more like noise but still have a few peaks during the raw trace's peak

This might also be my insufficient understanding to the algorithm/tuning process but hope this feedback helps! I would really appreciate to know if the parameters I used are appropriate or any suggestions to further tune it!

Additional Context

No response

[daharoni]

Hi @hazeljhuang , thanks so much for the feedback. Can you try the following and report back:

• It looks like you have your rise-time set to 1ms, the lowest value possible. See about raising that up to somewhere between 20 to 100ms while also decreasing your sparsity value, and lowering your decay time a bit more. Usually sparsity of over 2 starts to push the fitted peak below the actual peak and should be avoided. See around which sparsity value your fit's peaks start dropping down. That is a good indicator of where not to go higher than.

In your second image, the decay time looks pretty good but you still want to try a slower rise-time and a lower sparsity.

If you think you get better results with these adjustments please post them here. Also, let me know what wording in the tutorial would help get these points across.

[hazeljhuang]

Hi Dr. Aharoni! Thank you for the advice. I tried increasing rise time and lowering decay time/sparsity, and I think it helps distinguish noise and events - the residual plot looks a little more "noisy" now and more peaks are included. The tiny positive bumps still occur sometimes but they look more aligned with the other noise. It might be helpful if anywhere in the "Understanding Parameters" or "Guided Workflow" tutorial can include a rough range of reasonable decay time/rise time (eg. better not to be 1ms) /sparsity (eg. around 2). I know this may vary significantly across data but even a little bit of numerical range will be of great practical guidance for users!

The "Guided Tuning Workflow" tutorial step 5 could also add one more sentence comparing the significance of maintaining magnitude of event peaks with reducing noise artifacts after the last sentence "Stop before real events start vanishing" (ie, "sometimes you will end up with a few noise artifact when sparsity starts to influence real events but it is acceptable ....") so that users won't keep increasing lambda without realizing the event peak magnitude is also an important measure!

Also in the "Advanced Techniques" tutorial the "Residual Pattern Analysis" mentions "Systematic positive bumps after peaks: decay too short". I'm not sure if my data applies to this case as I need to lower the decay time to reduce those systematic bumps. Increasing decay time will end up having smaller magnitude in the fit line and larger positive bumps in residual patterns. One thing I'm also uncertain about in this tutorial is how much attention users should put on to the residual plots. In my case tuning does not completely eliminate the transient positive bumps after events (but helps make them more like noise), and I'm not sure if this belongs to any kind of the artifacts mentioned later. If it is another acceptable artifact it will be helpful to include it so that users won't be too worried about underfitting.

Lastly I found that when decay time is set too short (less than 100ms for my data) or when rise time is too long (greater than 100ms for my data), the model will run infinitely trying to fit the data. Maybe we can include a note in one of the tutorial as a reminder to not set too extreme values.

These might be really trivial but hope it helps!