Improve precision of float numbers

[Druzyek]

(04-17-2019 05:07 AM)deetee Wrote:  Finally I'm close to give up and do it with the intrinsic double format - even if it's 7-decimal-precision only.

Don't give up! You'll get it if you keep working at it. I think you can get really far if you decide to store BCD bytes like I mentioned. It should also make your input code a little simpler and save you some space. As far as input goes, there are two ways I have tried. One is to let the user enter whatever they like then scan the input to make sure it is a valid number at the end. The other is to keep track of what has already been entered and ignore invalid characters. For example, if the user has already entered a decimal point, ignore the input if they press it again. I went with the second way for my last project and it simplifies things a little. Another big thing that might make your code a lot smaller is only storing one BCD digit per byte instead of two. Numbers will take up twice as much memory in RAM, but you will save some code space by not having to pack and unpack the bytes.

How much flash space do you have on your chip? I have been able to implement BCD math functions on an MSP430 pretty compactly in assembly. The same code might be bigger or smaller on an AVR, so you can't compare directly, but I wouldn't be surprised if you could also implement some pretty decent math functions in just a few K of flash on an AVR too.

I have been keeping tables for each math function to compare different versions and see the size/speed trade off in case I need to go with a slower version to fit everything in flash (so far that hasn't been the case).
   

I'm still working on interface code, so the final version might take up a lot more flash, but the math functions are done. As you can see, they are pretty small. I bet you could do the same on an AVR.
   
   

EDIT: Can I make the images full size?