Post Reply 
New firmware for the Swissmicros DM15L
12-20-2024, 02:45 PM
Post: #153
RE: New firmware for the Swissmicros DM15L
Weird behaviour with big numbers...

I was reminding myself of the maximum possible number that could be handled, by finding the factorial of larger and larger numbers until the calculator returned INF...

170 f x! gives 7.257416E306
171 f X1 gives INF

I guessed the max number possible was around 1E307...

so entering

1 EEX 307 ENTER gives 1.0000000E307

but multiplying by 10 doesn't give 1.000000E308, nor does it give "INF", it returns

0

(no decimal places displayed)

dividing now by 10

10 /

gets back 1.000000E307

It looks as though numbers between

2.0000 E 307 and 1.700 E 308 are displayed as

0

but are still available for calculation, and only numbers from 1.800E308 are shown as "INF" and are no longer valid for calculations.

A minor niggle - I don't recall ever needing to work with numbers of this magnitude.

Cheers,

John
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: New firmware for the Swissmicros DM15L - Johnp_g - 12-20-2024 02:45 PM



User(s) browsing this thread: 8 Guest(s)