How does one raise a number to a power in the 9100A?
|
03-12-2018, 09:31 PM
(This post was last modified: 03-12-2018 09:36 PM by Dieter.)
Post: #12
|
|||
|
|||
RE: How does one raise a number to a power in the 9100A?
(03-12-2018 02:28 PM)toml_12953 Wrote: Here's the calculation in Floating point mode Interesting. The ln is slighty low (last digit rounded down instead of up), the multiplication then has a result which is 1 ULP high. Finally the exponentiation is significantly off: e^220,8210700 is 7,968419489 E+95. So while the result looks very good, it actually is caused by the error of the exponential function which has only 7 correct significant digits. Which only by chance is close to the correct final result. Dieter |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 1 Guest(s)