Post Reply 
[Free42] Numbers reformatted in program mode
07-23-2018, 08:21 AM
Post: #8
RE: [Free42] Numbers reformatted in program mode
With BCD coded reals, there is no digit conversion required Smile
A normalisation step would have to be done, so digit entry would be slightly slower than storing the number natively.

It must have been fun dealing with binary floating point numbers and decimal fractions that change the number. 0.1 being converted to 0.1000000000000000055511151231257827021181583404541015625 when stored as an IEEE 64 bit number. I guess digits after the 12th can be rounded away which gives a solution.


Pauli
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: [Free42] Numbers reformatted in program mode - Paul Dale - 07-23-2018 08:21 AM



User(s) browsing this thread: 1 Guest(s)