Accurate Normal Distribution for the HP67/97

[Dieter]

(12-16-2018 07:53 PM)John Keith Wrote:  I certainly did not make that detailed of an analysis but it seemed to me intuitively that Albert's method double x = (float) z discards exactly half of the significant bits. Similarly -6 RND discards half of the significant (BCD) digits in a 12-digit calculator.

IEEE 754 single precision numbers are stored with a 23 bit mantissa, compared to 52 bits in double precision. So actually more than half of the mantissa bits are discarded. That's about 7 vs. 15-16 decimal digits.

Rounding to a certain number of significant digits (RND with negative argument) is the more flexible method. So stay with –6 RND, this seems to produce results close to the theoretical optimum:

(12-16-2018 07:53 PM)John Keith Wrote:  Nonetheless, the 1-Exp method implemented as above had errors as large as 34 ULPs with some inputs, whereas the 2-Exp method never had more than 1 ULP of error in all of the inputs I tried.

That's as good as it gets.

Dieter