Unexpected result calculating the determinant of a singular matrix (42S)

10212019, 12:48 PM
Post: #11




RE: Unexpected result calculating the determinant of a singular matrix (42S)
(10212019 11:58 AM)John Keith Wrote: The HP 50 in approximate mode and with flag 54 (the "cheat" flag) set*, returns 0 for both Dave's and Thomas's matrices. I'm surprised that the 42s gives an inexact result, I assumed the internal code for matrix math was essentially the same as the 48/49 series. Approximately true. However, I've been working on some number theory stuff (which I'll post if I get it in shape for public use) that use really big (greater than 2^64) integers. Some seemingly integer stuff ends up being floating point. Conversions of big numbers out of binary comes to mine. I've got most things to work. Reducing some things with big integer multiple of things like Sqrt(2) or (Sqrt(5)1)/2 and the like need careful handling. Generally FLOOR and CEIL work well. FXND can be a problem as I found some case (I can't reproduce it) where would convert a number to a floating point. That's for theory. In practice, I can just keep the numerators and denominators of stuff separate and use really close rational approximations for the irrational numbers in the final step. 

« Next Oldest  Next Newest »

User(s) browsing this thread: 1 Guest(s)