Floating point bug in Canon F-789SGA

[Gerald H]

(11-30-2014 07:32 PM)Peter Van Roy Wrote:  I recently bought a Canon F-789SGA calculator because its specification intriguingly mentions that it has 18 digit precision. Tests show that it actually has 20 digit internal precision, and transcendental functions actually achieve this precision. That is the nice part.

Unfortunately, the calculator has bugs in its floating point operations. It took me a while to figure out what was going on, since there are several bugs that interact in unobvious ways. The main bug is an error in floating point addition. When doing a floating point addition A+B, if the difference in exponents between the two numbers A and B is 16, 17, 18, or 19, and if the lower four digits of the larger number are 0000, then the smaller number is not added in. That is, the lower four digits of the result stay 0000. For example, doing the addition 1 + 1E-16 gives exactly 1 instead of 1.0000000000000001. Floating point subtraction has a similar bug. There are other irregularities in the floating point operations, in particular, the value of the last (20th) digit also plays a role.

My guess to the origin of this bug is that the calculator is an improved version of a Casio calculator with 16 digit precision. Possibly the internal precision was increased without proper testing.

Has anyone else found similar bugs in these or other recent calculators?

(12-09-2014 07:38 PM)Peter Van Roy Wrote:  

(12-06-2014 06:37 AM)Gerald H Wrote:  Can't find the referenced text in "User Instructions" E-IM-2725 page 11 nor in f789sga.pdf E-IE-455 page 11.

It's on the bottom of page 10 in E-IE-455, the first entry in the first table in section Input Range and Error Messages. It's page 11 when viewing the pdf document but numbered as page 10 on the page itself.

No - in the pdf on page 10 "Nr of digits for internal calc : Up to 18", spec met by trig funcs & everything else too, as "up to" is a guarantee of nothing.

In "Calculation Examples" paper document, example #6, page 4,
Nr digits for internal prec 18 BUT also +/- 1 at the 10th digit.

You can accuse Canon of sloppy, unclear documentation but I don't think the claim of calc not fulfilling specs is warranted.