Post Reply 
Algorithm accuracy vs calculator precision
11-12-2024, 05:44 PM
Post: #2
RE: Algorithm accuracy vs calculator precision
I certainly share your view. I think Steve will hopefully chime in with a Prof Kahan quote about the algorithms of the HPs being superior even though the digits were less. I guess HP persisted with 10 digits for a while also when other manufacturers offered more, and maybe they felt that having better algorithms was more useful than digits.

From a user perspective I'd prefer less digits if better accuracy, especially if accuracy could be guaranteed.

Hidden precision is somewhat relevant though because it allows built-in functions to utilise more digits. That may or may not be beneficial to the accueacy though, and really hiding the precision means "I am not allowing you to use this precision in your calculations or your programs, just for my built-in functions". which is an odd stance.

When I went on my dive of TVM calculators I noticed very varied results depending on the algorithm. I plotted a 'digit efficiency' graph to show how well the calculators did for their precision. I'll dig out my latest graph and post it here later.
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: Algorithm accuracy vs calculator precision - dm319 - 11-12-2024 05:44 PM



User(s) browsing this thread: 2 Guest(s)