Post Reply 
Should I replace my TI-89 Swiss Army Knife on the table with the HP Prime?
08-26-2022, 06:27 AM
Post: #14
RE: Should I replace my TI-89 Swiss Army Knife on the table with the HP Prime?
(07-26-2022 11:40 AM)Nigel (UK) Wrote:  
(1) In Home mode, try 10.1^600/10.1^599. The correct answer is 10.1, but the calculator returns 1. Both numerator and denominator overflow and evaluate to 9.99999999999e499. This caught me out a few times with expressions that included factorials of large numbers not throwing an error, but evaluating to an incorrect answer. I think others have noticed this too. In CAS mode there’s no problem - both top and bottom evaluate to \(\infty\) and the result is undefined.

This has indeed been noticed. An enhancement request (to report numerical overflow) is in the bug tracker (ticket 52). Addressing this (once it gets near the “top of the work items”) will take some time as it involves careful review and adjustment of the numerical routines (to incorporate flags for numerical events, such as overflow, at a minimum). (And adding automated tests etc.)

Within the CAS numerical environment, overflow can lead to things like 10.1^306/10.1^307 being (silently) quite far from 1/10.1. (Or 10.1^300/10.1^400 being far from 1/10.1^100; A/B with A staying within the range of finite floating-point values and B bubbling over.)
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: Should I replace my TI-89 Swiss Army Knife on the table with the HP Prime? - jte - 08-26-2022 06:27 AM



User(s) browsing this thread: 1 Guest(s)