Unexpected result calculating the determinant of a singular matrix (42S)

10212019, 06:12 PM
Post: #17




RE: Unexpected result calculating the determinant of a singular matrix (42S)
(10212019 12:48 PM)ttw Wrote: However, I've been working on some number theory stuff (which I'll post if I get it in shape for public use) that use really big (greater than 2^64) integers. Some seemingly integer stuff ends up being floating point. Conversions of big numbers out of binary comes to mine. I've got most things to work. Conversion of binary numbers to reals will lose precision if the value is greater than 10^12. To convert large binary numbers to exact integers, you can try >STR 3. OVER SIZE 1.  SUB OBJ> in decimal mode. I can't see how FXND would return reals as long as you are in exact mode (check flags 3 and 105). If you want floatingpoint numbers with more than 12 digits you will have to use LongFloat. I am also interested in number theory and I would like to see what you come up with. Apologies to others for taking this thread offtopic. 

« Next Oldest  Next Newest »

User(s) browsing this thread: 1 Guest(s)