Fast Counting

[trezitorul]

I am working on implementing a scanner and am just doing point by point measurement for simplicity. This works beautifully, but it is relatively slow since it seems that the buffer being read updates pretty slowly (~50-100ms+). Is there a way to speed up the buffer update speed. Ideally, I would like to read out integrated counts at a rate of 0.1 to 1ms if possible. So something long the lines of get_counts(integration_time) where the whole function call takes ~integration_time.

I know that the obvious solution would be to hook up and program the scanner to output TTL signals to use as markers, however, I figured I would check if there is some setting or configuration that would allow for more rapid readout from the PH330 since that approach is very simple and implemented.

[tpoint75]

The PH330 supports the conti mode, that do exactly what you want. But unfortunately it is not available in snAPI atm. You have to use the PH330.dll directly to use it. In Uniharp there is the adjustment mode. But you need it, to make images and therefor you need the data for each pixel?

[tpoint75]

But instead of start and stop measurement, you can read the data on the fly and use the clearMeasure function after the next pixel is arrived?!!
https://picoquant.github.io/snAPI/snAPI.Main.html#snAPI.Main.Histogram.clearMeasure

[trezitorul]

Hi Torsten, thank you for the reply! I actually implemented exactly what you suggested earlier where I start the measurement (either time-trace or unfold and wait for some time, and then collect the signal from the buffer, unfortunately, the buffer still takes around 10-100ms to read out and if you read out to quickly it just returns the same buffer. But also great idea with clear measure since then if I do use markers, I can collect a chunk of data and then wipe the buffer for the next chunk of data.

Before working on implementing all the hardware and software for the markers, are there any settings that might speed up the buffer readout? Or is there a mode that is faster (*unfold vs. raw vs. time trace vs. histogram?)

[tpoint75]

Hi Samuel, the buffer read out must be instant. I assume there was a Pause in you script that reads the memory block only after each 100 ms like here https://github.com/PicoQuant/snAPI/blob/main/demos/Demo_HistogramRefresh.py#L34 ?

[tpoint75]

If I should check your code you can open a support request here: https://www.picoquant.com/contact/support and ask directly for my help..

[trezitorul]

Hi Torsten, my apologies for the delay, I was out travelling this week. I also submitted a support request, but figured I would also add my testing here for community benefit from our discussion.

To be clear the buffer read out is nearly instantaneous, however, it does not receive new data from the PH330 as frequently. The maximum rate the buffer from the PH330 can be read out seems to be approximately every 0.02 seconds. I wrote a simple little script to test the maximum readout speed, see below:
`
from snAPI.Main import *
import time
sn = snAPI()
sn.getDevice()
sn.initDevice(MeasMode.T2)

sn.unfold.measure(acqTime=0)
time.sleep(1)#Gives time for measurement to start
data = sn.unfold.getData()
start_t=time.perf_counter()
i=0
prev_num=0
delay=0.0001#This slows down the read out rate of the internal buffer incase reading it too fast is the problem, this can be set to 0 with no real change.
times=[]
num_data=[]
read_times=[]
total_start=time.perf_counter()
while True:
delay_start=time.perf_counter()
while delay>=time.perf_counter()-delay_start:#A method of setting a delay with very accurate timing up to ~10KHz
pass
if sn.unfold.numRead()!=prev_num:
read_times.append(time.perf_counter()-start_t)
print(sn.unfold.numRead())
prev_num=sn.unfold.numRead()
start_t=time.perf_counter()
i=0
i=i+1
num_data.append(sn.unfold.numRead())
times.append(time.perf_counter()-total_start)
`

If this is run, then the number of events vs. time is plotted we get a plot like this:

[tpoint75]

Hi Samuel,

this is very interesting. Now I understand your problem. When data arrives over the USB bus, it is also sent in blocks. SnAPI collects the data in different processing threads and also processes it in blocks. This is done for performance reasons. Accessing the USB bus takes some time, so we can't transmit each event individually. The size of the blocks received by snAPI can change depending on the data rate (count rate) or your PC's performance.
To process the data effectively, you also need to collect it in an input buffer. A loop or a callback should check the buffer for new data. If there are data, it should check the arrival-times and truncates the required portion. Only this portion should be processed by the routine. The rest must remain in the buffer and be supplemented with new data later. It might be more efficient to use a circular buffer for data collection.

I hope that helps,
best regards,
Torsten.

[trezitorul]

Hello Torsten,

Perfect, this is exactly what I was suspecting is going on. One quick question, you mention that I should be writing a loop or callback to check for new data. I am unclear how that should be done, isn't the code uploaded which is looping waiting for the numRead() value essentially performing this role? In that case is the ultimate speed that data gets pulled into the snAPI buffer around 0.02 seconds? I just want to make sure I understand your suggestion. Although, it sounds more that what needs to be done is to collect multiple points of data into the counts buffer, and then perform some kind of synchronization to sync the buffer and scan head. Any suggestions on synchronizing the computer clock and the PH330 clock if that approach is needed?

[tpoint75]

Sure, your "while True:" exactly does this, but you don't have to do this that often. The buffers should be big enough to do it every second or even the rate in which your view should be updated. If you do it rarely the data blocks becomes bigger..
Btw. if you use unfold.getBlock() you should start the measurement with sn.unfold.startBlock(), where you can set the acqTime and the (buffer) size. See https://picoquant.github.io/snAPI/snAPI.Main.html#snAPI.Main.Unfold.startBlock.
Normally the scan head has some signals for new line and new picture that are connected to the marker inputs: see https://picoquant.github.io/snAPI/snAPI.Main.html#snAPI.Main.Unfold.startBlock. This markers and the timestamps (iinside a line) can be used to split the data afterwards to get the pixel informations. https://picoquant.github.io/snAPI/snAPI.Main.html#snAPI.Main.Unfold.isMarker
I'm not sure, what you want to measure. Is the Scan Head moving all the time or do you want to move it to a new position, doe a measurement, move to next next position, do a measurement and so on. I this case, i would keep the repetition rate of the while loop high and connect only a signal to the marker input, when the scan head has reached the position.
Don't change the clock input of the harp device. It will worsen you timing jitter, instead of having a better clock then the internal.

[trezitorul]

Thank you Torsten! I am hoping to move position to position to allow for different data streams (lifetime, counts, spectra) to be connected easily using the same code and these are slow enough to allow for this. So in conclusion the max read out rate of the buffer is slow enough to where trying to read the buffer per pixel has a maximum read out rate of approximately 50-100hz. So the new plan is basically to connect the markers from the scan head to the PH330 and then use those to post-process the buffer into individual pixels.

One quick question:

1. Any suggestions for synchronizing the start of the measurement block with the start of the scan. My initial thought was to emit 2 or 3 markers rapidly to signal the start of the measurement, and then to emit a single pulse per pixel location.

[tpoint75]

We have 4 different markers. Isn't that enough decode all?

[trezitorul]

I totally forgot that there are multiple Marker channels, that works perfectly. Thank you again for all your help!