Too Many Digits Will Mess Up Motor Offset Y

[madgrizzle]

If you enter too many digits in the Motor Height Offset setting, I think the front end of it gets truncated. I entered a lot of digits (just what the cal routine I'm working on outputted) and it sent them to the controller. However, the controller thought 0,0 was way high up on the plywood. I changed the offset to reduce the number of digits after the decimal point, and it immediately found 0,0 to be 0,0.

[blurfl]

How many digits are we talking about here?

[blurfl]

This might belong in the Firmware repository - GC correctly sends the whole string $3-425.01234567890123456789 but the firmware mishandles it (NutsAndBolts.cpp:readFloat() as a suspect?)

[blurfl]

Firmware:NutsAndBolts:readFloat() accumulates the digits of a value as a 32bit long with a range of -2,147,483,648 to 2,147,483,647. It keeps track of the sign separately, so it could use an unsigned long to better effect. The approach breaks down, though, if the value has too many digits.

[BarbourSmith]

I think we had a similar issue with sending the chain compensation factor. The offending code is copied straight from GRBL. The solution last time was to just send fewer digits

[blurfl]

I think it goes deeper than that - try this:
Set MotorOffsetHeightInMM to the default, 463
Now look at the value of $3 returned by $$: $3=463.00000000 (motor height, mm)
Now set it to 463.1 and check $$ again: $3=463.10000610 (motor height, mm)

I think there's a wrinkle in readFloat() but I haven't put my finger on it. I suppose it could be an issue with Arduino's printing the value of the float, haven't checked that yet.

[BarbourSmith]

That looks like the normal error in representing numbers in floating point to me 😞

[blurfl]

the normal error in representing numbers in floating point

You're right. I guess limiting the number of digits past the decimal point would be the solution. Which end of the transaction (firmware...), how many digits (3...), truncate or round (round...)?