Crash when Solve for Integral with 15C
|
09-02-2017, 01:40 PM
Post: #24
|
|||
|
|||
RE: Crash when Solve for Integral with 15C
(09-02-2017 05:53 AM)Dieter Wrote: The result is about 6 E+91. FIX 2 means two digits after the decimal point, so that's actually 94 significant digits, in other words: it's (more than) full machine precision you're demanding. On the other hand SCI 4 sets only five significant digits which is returned much faster. Thanks Dieter. All this makes sense to me, as is so often the case, after the explanation. Though I initially would have guessed that the 15C would treat FIX 2 as 2 significant places, even if it must shift to SCI mode, just as it does when the displayed number must be so shifted. Though subtle, this is a significant difference and highlights how the machine handles the numbers internally (vs. display) and so is an interesting exercise. I will read the referenced section in the 15C AF HB. Again! AFAIK, this is the best treatment of such topics in any HP manual; can you recommend others I may have missed? --Bob Prosperi |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 2 Guest(s)