Numerical accuracy on the HP Prime (in Python, Home, and CAS)
|
07-26-2024, 04:27 PM
Post: #4
|
|||
|
|||
RE: Numerical accuracy on the HP Prime (in Python, Home, and CAS)
1e50 mod 2*pi can not be computed with any significant digit using "normal" precision. The reason is that the relative error rounding any floating point number will be say 1e-17 (it's 2^(-53) for double precision inside MicroPython, 2^(-48) for Xcas/CAS normal precision), that's an absolute error of 1e33 for 1e50, and that's much greater than 2*pi.
This is something that everyone doing scientific computation should know, if it's not teached in a course during highschool or first years of University, there is a real problem. |
|||
« Next Oldest | Next Newest »
|
Messages In This Thread |
Numerical accuracy on the HP Prime (in Python, Home, and CAS) - ftneek - 07-25-2024, 06:40 PM
RE: Numerical accuracy on the HP Prime (in Python, Home, and CAS) - Nigel (UK) - 07-25-2024, 08:52 PM
RE: Numerical accuracy on the HP Prime (in Python, Home, and CAS) - ftneek - 07-25-2024, 10:17 PM
RE: Numerical accuracy on the HP Prime (in Python, Home, and CAS) - parisse - 07-26-2024 04:27 PM
RE: Numerical accuracy on the HP Prime (in Python, Home, and CAS) - ftneek - 07-26-2024, 07:01 PM
|
User(s) browsing this thread: 1 Guest(s)