Post Reply 
Machine epsilon of the HP Prime
09-18-2022, 08:11 AM
Post: #3
RE: Machine epsilon of the HP Prime
I fear there might be a couple of problems with this approach...

I should say that I'm taking machine epsilon to be the same as a unit in the last place (an ULP). If that's wrong, please tell me!

First, I would have thought the correct value of epsilon is not the final u, but the one before that.

Second, on a binary machine with correct rounding, we can add half an ULP and find that it makes a difference, because half an ULP may round upwards.

So, in the case of BBC Basic with a 32 bit mantissa, the program as written returns 2^-33 as the epsilon, whereas I think the right answer is 2^-31.

(Actually, I'm thinking now that even a decimal machine with correct rounding should round up when half an ULP is added... but then again, in this sense I think I'd have to say the majority of HP machines don't tend to round this way. With a decimal machine, halving the candidate epsilon each time might be asking for trouble: at the very least, the final epsilon found will be close to a power of two, but it ought to be an exact power of ten.)
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: Machine epsilon of the HP Prime - EdS2 - 09-19-2022, 01:05 PM
RE: Machine epsilon of the HP Prime - EdS2 - 09-18-2022 08:11 AM
RE: Machine epsilon of the HP Prime - EdS2 - 09-19-2022, 06:31 AM



User(s) browsing this thread: 1 Guest(s)