linsolve bug
|
08-25-2023, 04:34 PM
Post: #5
|
|||
|
|||
RE: linsolve bug
Some of linsolve accuracy issue may be my upgrade from XCas 1.5.0 to 1.9.0
Float precision is reduced, from 53-bits mantissa IEEE double, to 48 (what HP Prime Cas has) Worse, XCas 1.9.0 now round the same as HP Prime Cas too ... truncation! Cas> h := 2^48 - 1.0 Cas> 2.96+h-h → 1.03125 // same as XCas 1.9.0 Cas> 2.97+h-h → 3.03125 // same as XCas 1.9.0 With ill-conditioned matrix, lost of precision and losing round-to-nearest matters! Code: /* sx[k] = sum(X.^k), syx[k] = sum(Y.*X.^k) */ XCas 1.5.0 --> [-5.46969697020, 22010.2878808] XCas 1.9.0 --> [-5.46452227167, 21989.5114662] |
|||
« Next Oldest | Next Newest »
|
Messages In This Thread |
linsolve bug - Albert Chan - 08-24-2023, 01:57 PM
RE: linsolve bug - parisse - 08-24-2023, 02:24 PM
RE: linsolve bug - Albert Chan - 08-24-2023, 04:12 PM
RE: linsolve bug - Albert Chan - 08-28-2023, 01:03 PM
RE: linsolve bug - parisse - 08-25-2023, 03:19 PM
RE: linsolve bug - Albert Chan - 08-25-2023 04:34 PM
RE: linsolve bug - parisse - 08-25-2023, 06:58 PM
RE: linsolve bug - Albert Chan - 08-26-2023, 02:57 PM
RE: linsolve bug - Albert Chan - 08-25-2023, 08:24 PM
RE: linsolve bug - parisse - 08-27-2023, 06:47 AM
RE: linsolve bug - Albert Chan - 08-27-2023, 09:15 AM
|
User(s) browsing this thread: 2 Guest(s)