Machine epsilon of the HP Prime

[Albert Chan]

(09-18-2022 08:11 AM)EdS2 Wrote:  BBC Basic with a 32 bit mantissa, the program as written returns 2^-33 as the epsilon,
whereas I think the right answer is 2^-31.

Your "right" answer was the variant definition, machine epsilon = ULP of 1.

1. = 2^31 ULP           → ε = ULP = 2^-31

OP use formal definition, u = ε/2 = 2^-32

Round-to-nearest, halfway-to-even, 1. + u == 1., for any even base

>>> from gmpy2 import *
>>> get_context().precision = 32 # binary32
>>> one = mpfr(1)
>>> one + 2**-31 # 1 + ε
mpfr('1.0000000005',32)
>>> one + 2**-32 # 1 + u
mpfr('1.0',32)

10-digits decimal, correct rounding:

C:\> rpn =10 1e-9 1+
1.000000001
C:\> rpn =10 1e-9 2/ 1+
1