Length not divisible by 3!

[donpdonp]

it looks like last night's recording has hit an edge-case.

python cms50dplus.py -s '2015-09-28 22:00:00'  RECORDED /dev/ttyUSB0 "out-`date --iso-8601=seconds`.csv"
Saving recorded data...
Please wait as the latest session is downloaded...
Traceback (most recent call last):
  File "cms50dplus.py", line 327, in <module>
    dumpRecordedData(args.starttime, args.serialport, args.output)
  File "cms50dplus.py", line 302, in dumpRecordedData
    for recordedData in oximeter.getRecordedData(starttime):
  File "cms50dplus.py", line 251, in getRecordedData
    raise Exception("Length not divisible by 3!")
Exception: Length not divisible by 3!

I adjusted the error message to show the length
"Length 253 not divisible by 3!"

any ideas whats going on with this?

[atbrask]

Hmm.. I think I've seen this before, but not very often. And I wonder if this is a bug in the device or if my understanding of the protocol is a bit flawed. Basically, what happens is this: Before the device dumps its data, it first reports the total length in bytes. Each measurement is three bytes, so this total length must be divisible by 3 to be valid. An alternative approach would be to ignore the reported length and just keep listening for data until the device finishes and shuts up. But I think it would be preferable to try to fix this properly. Could you provide me with the first 20-30 bytes of the returned data? Just insert a loop right after line 231 that dumps the data to the terminal.

[donpdonp]

i put in a loop after the preamble and after the length, but also printing the length bytes. i hope its useful

--- a/cms50dplus.py
+++ b/cms50dplus.py
@@ -245,10 +245,17 @@ class CMS50Dplus(object):
             if ((lena & 0x80) == 0 or (lenb & 0x80) == 0 or (lenc & 0x80) != 0):
                 raise Exception("Corrupted length in header!")

+            print "lena {0}".format(lena)
+            print "lenb {0}".format(lenb)
+            print "lenc {0}".format(lenc)
+
+            for x in range(3):
+              print "data: {0}".format(self.getByte())
+
             length = (((lena & 0x7f) << 14) | ((lenb & 0x7f) << 7) | lenc) + 1

             if length % 3 != 0:
-                raise Exception("Length not divisible by 3!")
+                print "Length {0} not divisible by 3!".format(length)

Output

$ PYTHONPATH=./python-dateutil-2.4.2/:./six-1.9.0/ python cms50dplus.py -s '2015-09-28 22:00:00'  RECORDED /dev/ttyUSB0 "out-`date --iso-8601=seconds`.csv"
Saving recorded data...
Please wait as the latest session is downloaded...
lena 128
lenb 129
lenc 124
data: 240
data: 128
data: 0
Length 253 not divisible by 3!
Number of measurements: 84 (0h1m24s)
Got 83 measurements...Traceback (most recent call last):
  File "cms50dplus.py", line 334, in <module>
    dumpRecordedData(args.starttime, args.serialport, args.output)
  File "cms50dplus.py", line 309, in dumpRecordedData
    for recordedData in oximeter.getRecordedData(starttime):
  File "cms50dplus.py", line 276, in getRecordedData
    raise Exception("Timeout during download!")
Exception: Timeout during download!

[atbrask]

Hmm.. So it only sends back 83 measurements? That's 249 bytes. In that case lenc should have been 120 instead of 124. The following 3 bytes look as expected, so it's looking at the right bytes for the length. Perhaps the solution will be to only treat this length as a guideline for the progress indication during the download, and then just continue fetching data until the timeout happens.

[donpdonp]

I'm trying to save this data set because I wore it all night. It should have 30,000 or more readings. If its possible to read 'until end of file' that would be good, id be happy to try it on my device.

maybe the dump to console was wrong somehow. i mean to dump more than 3 characters on the extra loop i added.

[atbrask]

That's weird. Your dump definitely doesn't indicate that much data. Have you tried running the bundled Windows software that comes with the oximeter?

[donpdonp]

i havent, i got this device because i found your script via googling. i use it with a rasperryPi. well maybe i'll have to start over with this data then.

[atbrask]

Hmm ok. Try replacing the for-loop in line 266 with a while True: and the exception in line 269 with a break. That should cause it to read until the timeout happens (without raising the timeout exception). I don't have my oximeter with me right here, so I haven't tested this change. If it still only returns 83 or 84 readings, then I'm afraid you are out of luck with your big dataset.. :-(
(Unless the bundled software says otherwise.)

[donpdonp]

i'll give it a try. thanks for the help!

[gabemaroz]

I have also gotten this error. I commented out the exception on line 251 and it says the number of measurements was 21843 for about 6 hours of time, which seems about right. However, I am still unable to download the readings even with the fix from Oct 7, 2015.

Any ideas?

[atbrask]

21843 measurements does indeed sound about right (6 hours, 4 minutes and 3 seconds). Changing lines 251, 266 and 269 as suggested above should have fixed most of this problem... What error do you get? Is it "Invalid data packet" in line 108? Or is it just hanging?

[gabemaroz]

I'll give it another go tonight since I have used the CMS50D+ after that time, which I assume overwrites the previous recording. Will report back with more info in the event that the error occurs.

[btoews]

I hit this bug too. Applying the patch from above, I get this output:

$ python ./cms50dplus/cms50dplus.py -s "2016-11-11 21:00:00" RECORDED /dev/tty.SLAB_USBtoUART ../ox.csv
Saving recorded data...
Please wait as the latest session is downloaded...
lena 133
lenb 157
lenc 117
data: 240
data: 188
data: 93
Length 85750 not divisible by 3!
Number of measurements: 28583 (7h56m23s)
Got 26344 measurements...Traceback (most recent call last):
  File "./cms50dplus/cms50dplus.py", line 335, in <module>
    dumpRecordedData(args.starttime, args.serialport, args.output)
  File "./cms50dplus/cms50dplus.py", line 310, in dumpRecordedData
    for recordedData in oximeter.getRecordedData(starttime):
  File "./cms50dplus/cms50dplus.py", line 277, in getRecordedData
    raise Exception("Timeout during download!")
Exception: Timeout during download!

[btoews]

Stopping reading the data early seems to be an okay workaround. I just changed for i in range(length): to for i in range(length - 10):...