Round off error calculation example
|
07-23-2017, 07:03 PM
Post: #7
|
|||
|
|||
RE: Round off error calculation example
(07-23-2017 06:32 PM)Mark Hardman Wrote:(07-23-2017 05:07 PM)Dieter Wrote: BTW the Windows calculator works with 30+ digits, so it returns 1 here. ;-) Let's not forget the WP34s (double precision mode), and even the 31s internally uses (at least) 34 digits. BTW, Free42 has 34 digit BCD precision, but as far as I remember not all functions deliver results with that accuracy. This also is not guaranteed for the WP34s. Dieter |
|||
« Next Oldest | Next Newest »
|
Messages In This Thread |
Round off error calculation example - Gamo - 07-23-2017, 10:16 AM
RE: Round off error calculation example - Arno K - 07-23-2017, 10:29 AM
RE: Round off error calculation example - Joe Horn - 07-23-2017, 11:57 AM
RE: Round off error calculation example - Csaba Tizedes - 07-23-2017, 12:24 PM
RE: Round off error calculation example - Dieter - 07-23-2017, 05:07 PM
RE: Round off error calculation example - Mark Hardman - 07-23-2017, 06:32 PM
RE: Round off error calculation example - Dieter - 07-23-2017 07:03 PM
RE: Round off error calculation example - Gamo - 07-24-2017, 01:17 AM
|
User(s) browsing this thread: 1 Guest(s)