CAS: (2^10)! is undefined?

[parisse]

The fact that large integers must be limited in size can of course not be avoided, the limit being smaller on calculators than on PC. The CAS code is based on Giac which is also open-source licensed, you can therefore find useful informations in the source code if you don't want to make a small dichotomoy to find the limit, look in global.cc for FACTORIAL_SIZE_LIMIT (it's certainly possible to raise a little bit the limit, but the biginteger library used on the Prime is not very efficient compared to GMP or MPIR). BTW, the largest factorial that the nspire can compute exactly seems to be 446!

Once you can not compute exactly (and you can't convert to a floating point number), returning undef is a good option, because you are sure that it will remain so in further computation. Introducing an overflow constant would not bring much additional information, but would increase the code complexity because we would have to deal with arithmetic operations like overflow+1=overflow (ok), overflow*0= well is it 0 or undefined? 1/overflow= hum, should I define underflow also? Then, should I defined +overflow and -overflow...?

@sasa: I'm in general open to constructive discussions that can help improve Giac, but please consider that I'm not really a newcomer : I'm coding CAS software since 25 years, I think I know a little bit how to do it. Of course, I may miss something that someone could point to me, but your last post about solving z=sqrt(i) giving "unexpected" results does not help considering your comments very seriously.