A quick precision test
|
06-06-2014, 12:04 PM
(This post was last modified: 06-06-2014 12:07 PM by Claudio L..)
Post: #58
|
|||
|
|||
RE: A quick precision test
(06-06-2014 11:16 AM)pito Wrote:(06-06-2014 12:57 AM)Claudio L. Wrote: .. 00 2.09858469968759412260293728493047158318234204531375910645959950911100\ 455087648653787132935200077099580084905028343200684*10^-36 000 2.09858469968759412260293728493047158318234204531375910645959950911100\ 455087648653787132935200077099580084905028343200684*10^-36 0000 2.09858469968755291048739331660132323148430199128683081113869140777373\ 772224262464875860587211945559288771556897645713391*10^-36 00000 2.09858469968755291048739331660132323148430199128683081113869140777373\ 772224262464875860587211945559288771556897645713391*10^-36 .. 000000000000000 2.09858469968755291048747229615389016692898991768376891330235232863694\ 116237367172368967960981143453674385805543780051439*10^-36 I put the "good" digits in BOLD. The last one seems to give good 33 digits (well, the last one off by 1 per rounding, but let's not start on that subject again...) I think it was Paul's error: he asked Alpha "with 120 decimal digits", he should've written "with 120 decimal correctly calculated and correctly rounded digits". See? it was human error, not Alpha. Claudio |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 11 Guest(s)