Let's vote for suggestions and bugs
|
09-23-2018, 02:53 PM
Post: #69
|
|||
|
|||
RE: Let's vote for suggestions and bugs
(09-23-2018 01:45 PM)CyberAngel Wrote: Great! I am sure calculator can do roundings for you ... Probably not 14.4 decimals, but 14 should be no problem FYI, you don't really get back 48-bits precision (~ 14.4 decimals). The last bit may be totally off, if chopped 5 bits are all ones. In other words, last bit carry little information about precision. Using Pi as example, truncated 48-bits float = 0x0.c90fdaa22168P+2 However, rounded 48-bits is much better, float = 0x0.c90fdaa22169P+2 (more than Pi times closer to Pi) |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 11 Guest(s)