RPNcalc+ free colorful web calculator

[EdS2]

(11-10-2020 10:53 AM)radulle Wrote:  I used basic JS number type which is double precision floating point number IEEE 754 and JS Math object. I'm not really sure how best to handle this without introducing some other issues. I might better implement some JS library that handles it if there are too many edge cases. BTW: How did you come to this test?

I suspect it will be rather tricky to make a binary calculator comply with decimal expectations. There are relatively few calculators which use binary, which for me is a valid implementation choice, but there will be oddities which some people will regard as bugs. The same kinds of things happen in decimal, but people tend to expect those errors. It's a perennial problem.

As double precision gives about 16 decimal digits, you could perhaps round to and display only 12 or 14. But you will have to deal with catastrophic cancellation somehow: when subtraction gives a result much smaller than the inputs, you will in effect need to round much more dramatically.

It would be good if there were a JS library to do this. Perhaps see here.