Post Reply 
Let's vote for suggestions and bugs
09-23-2018, 02:53 PM
Post: #69
RE: Let's vote for suggestions and bugs
(09-23-2018 01:45 PM)CyberAngel Wrote:  Great!
but... the decimal is still "over-accurate", not rounded to 14.4 decimals.

I am sure calculator can do roundings for you ...
Probably not 14.4 decimals, but 14 should be no problem Big Grin

FYI, you don't really get back 48-bits precision (~ 14.4 decimals).
The last bit may be totally off, if chopped 5 bits are all ones.
In other words, last bit carry little information about precision.

Using Pi as example, truncated 48-bits float = 0x0.c90fdaa22168P+2
However, rounded 48-bits is much better, float = 0x0.c90fdaa22169P+2 (more than Pi times closer to Pi)
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: Let's vote for suggestions and bugs - Albert Chan - 09-23-2018 02:53 PM
BREAK still reboot the calc - Oulan - 07-21-2019, 09:12 AM



User(s) browsing this thread: 11 Guest(s)