New firmware for the Swissmicros DM15L
|
12-20-2024, 02:45 PM
Post: #153
|
|||
|
|||
RE: New firmware for the Swissmicros DM15L
Weird behaviour with big numbers...
I was reminding myself of the maximum possible number that could be handled, by finding the factorial of larger and larger numbers until the calculator returned INF... 170 f x! gives 7.257416E306 171 f X1 gives INF I guessed the max number possible was around 1E307... so entering 1 EEX 307 ENTER gives 1.0000000E307 but multiplying by 10 doesn't give 1.000000E308, nor does it give "INF", it returns 0 (no decimal places displayed) dividing now by 10 10 / gets back 1.000000E307 It looks as though numbers between 2.0000 E 307 and 1.700 E 308 are displayed as 0 but are still available for calculation, and only numbers from 1.800E308 are shown as "INF" and are no longer valid for calculations. A minor niggle - I don't recall ever needing to work with numbers of this magnitude. Cheers, John |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 8 Guest(s)