Unexpected result calculating the determinant of a singular matrix (42S)

[John Keith]

(10-21-2019 12:48 PM)ttw Wrote:  However, I've been working on some number theory stuff (which I'll post if I get it in shape for public use) that use really big (greater than 2^64) integers. Some seemingly integer stuff ends up being floating point. Conversions of big numbers out of binary comes to mine. I've got most things to work.

Reducing some things with big integer multiple of things like Sqrt(2) or (Sqrt(5)-1)/2 and the like need careful handling. Generally FLOOR and CEIL work well. FXND can be a problem as I found some case (I can't reproduce it) where would convert a number to a floating point.

Conversion of binary numbers to reals will lose precision if the value is greater than 10^12. To convert large binary numbers to exact integers, you can try
->STR 3. OVER SIZE 1. - SUB OBJ-> in decimal mode.

I can't see how FXND would return reals as long as you are in exact mode (check flags -3 and -105). If you want floating-point numbers with more than 12 digits you will have to use LongFloat.

I am also interested in number theory and I would like to see what you come up with.

Apologies to others for taking this thread off-topic.