Add server-side analysis functions

[JBorrow]

Add server-side analysis of things like per-band variance, medians, means, and so forth.

[JBorrow]

Can also cache this within the lightcurve response?

[JBorrow]

Please list below the requests for summary statistics and how to compute them

[davecl42]

Max flux/ min flux; max flux/median flux; flux ratios between different bands (actually being able to do a plot of how flux ration changes with time would be interesting to see if the colour of the sources change - is this something that could go into the timeline plot?); some way of getting a timescale of variability - not sure how but this might be useful for identifying fast vs. slow sources; will try to think of more :-)

[axf295]

• similar to what dave was suggesting - (most recent week mean) / (mean of previous weeks) ; i'm thinking just something simple like mean(lc[-10:])/mean(lc[:-10])
  of course, we'd want to do some error bar cutting of bad data and upper/lower limit estimation, etc.

• we can store spectral indices in the database (or calc it on the fly). having mean or median spectral index would be nice for cutting sources w the api. you can use https://github.com/simonsobs/sotrplib/blob/main/sotrplib/utils/utils.py#L657 and https://github.com/simonsobs/sotrplib/blob/main/sotrplib/utils/utils.py#L666 although i don't love the formatting of this, it does both the calculation of spectral index between two band and the general fit of spectral index for multiple bands as well as error propagation.

• if we get polarization in the database, polarization amplitude will be a natural one to add. this may be a bit more complicated since we'll have to subtract monopole leakage, etc but to start it is just sqrt(Q^2+U^2)/I
  could also look for polarization angle swings if we're measuring the EVPA

I don't have a favorite variability metric so I'll let others decide on that.

[JBorrow]

I'm gonna need you to define what spectral index is and how to calculate it

[davecl42]

The spectral index is given by:

S_\nu \propto \nu ^ \alpha

where S_\nu is the flux in Jy at frequency \nu and \alpha is the spectral index. Thus something that has a positive alpha has a flux that rises with frequency.

Some conventions, and I think this applies a lot in the radio, have the relation inverted so that:

S_\nu \propto \nu ^ -\alpha

so that a positive value of \alpha means that flux rises with decreasing frequency. I think this is the convention usually used in the radio. We should decide on one and stick with it and have that properly documented.

Most non-thermal sources have spectra that rise with decreasing frequency, but thermal sources, at long wavelengths, do the opposite. Looking at the spectral index will give us an initial idea of the nature of a source - thermal or non-thermal - and can be a way of identifying odd sources that have eg. spectra peaking in the GHz region (so called GPS sources).

Hope this helps, and apologies for the delay!

[axf295]

Thanks Dave!

Sorry I should have replied in the thread, but I edited my comment to provide the lines in the sotrplib code where there is a function to calculate the spectral index (inherited from the ACT folks). looks like they used the S \propto \nu^\alpha.

[davecl42]

Glad it's sorted. Thanks!

[diveshjain-phy]

Hi @JBorrow ! I was looking into this. I am assuming these statistics are not calculated ahead of time. The analysis functions get an input on source ID, band name, and time range and then all these statistics are calculated and returned.