newRPL: [UPDATED April 27-2017] Firmware for testing available for download
|
10-22-2015, 06:16 PM
Post: #77
|
|||
|
|||
RE: newRPL: [UPDATED Oct-19-2015] Firmware for testing available for download
But 527.67 x (5/9) does produce 293.15 ...
on a 12-digit machine: 527.67 x 0.555555555556 = 293.15000000023452 I see you multiply by 0.555555555555, that is not the correctly rounded 5/9 Alternatively you may multiply by 5 and then divide by 9 - only the latter will produce a rounding error. Cheers, Werner 41CV†,42S,48GX,49G,DM42,DM41X,17BII,15CE |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 2 Guest(s)