Post Reply 
New firmware for the Swissmicros DM15L
10-22-2024, 12:39 PM (This post was last modified: 10-22-2024 12:47 PM by Johnp_g.)
Post: #126
RE: New firmware for the Swissmicros DM15L
I found another weird rounding error!

in FIX04

1.00005 ENTER

Displays as

2.0000

change to FIX05

now shows

1.00005

In FIX03

displays as

1.000

Staying in FIX03

1.0005 ENTER

displays as

2.000

It looks as though if there's a digit in the "n+1" position, in FIX0n (or SCI0n) that the rounding up happens to the whole number, not to the decimal portion. This seems to be the case whatever n is chosen for FIX/SCI/ENG.

I found it while going through the HP15C User Manual examples.

on P.22 "Numeric Functions"

in FIX04

3.4012 e^x

displays as 31.0000 instead of 30.0001

weirder still..... with 31.0000 in the display pressing ENTER

puts 31.0000 in Y and X

add them "+"

display now shows

60.0002

so the underlying number was still actually 30.0001 but was displayed as 31.0000, even when ENTER duplicated it and raised it to Y register!

very confusing!

Cheers

John
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: New firmware for the Swissmicros DM15L - Johnp_g - 10-22-2024 12:39 PM



User(s) browsing this thread: 8 Guest(s)