HP Prime complex division problem in CAS
|
11-28-2020, 02:00 AM
Post: #22
|
|||
|
|||
RE: HP Prime complex division problem in CAS
For IEEE double, we cannot scale to DBL_MAX/2 = 0x1.fffffffffffffp+1022
With round-to-nearest, halfway-to-even, it is possible scaled parts is slightly bigger. To ensure no overflow, we reduced it a bit, to say, 0x1p+1022 Also, to flip the parts, we can use x/y = (x*i)/(y*i) Here is compdiv_improved, with scaling. Code: function cdiv(xr, xi, yr, yi) lua> cdiv(1e154, 2e154, 2e154, 1e154) 0.8 0.6000000000000001 lua> cdiv(1e307, 1e-307, 1e204, 1e-204) 1e+103 -1.0000000000000001e-305 Example from improved_cdiv_v0.3.pdf, "4.7 A failure of the robust algorithm", page 35. lua> cdiv(0x1p-912, 0x1p-1029, 0x1p-122, 0x1p46) 5e-324 -4.1045368012983762e-289 With scaling, we get the correctly rounded parts. Without scaling, we lost the real part, same as compdiv_robust |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 1 Guest(s)