H->HMS conversion HP-15C vs. HP42S vs HP67
|
09-07-2018, 03:43 PM
Post: #23
|
|||
|
|||
RE: H->HMS conversion HP-15C vs. HP42S vs HP67
(09-07-2018 07:07 AM)Pekis Wrote: And with my funny one-liners: Hi, Pekis Above work (mostly) in Excel, because Excel does some behind-the-scene rounding. Example: if angle = 2° 1' , X = 2.01 (IEEE double of 2.0099999...) DMS->DEG formula: Python: (502.499999... - 200 - 120) / 90 = 2.0277777... (wrong!) Excel : (502.5 - 201 - 120) / 90 = 2.0166666... (all terms rounded implicitly) I would rather do explicit scaling / rounding, before conversion (see post #17): X = 2.0099999 ... --> Y = 20100.00 (scaled and rounded to hundredth of second) DMS->DEG: (Y - 40*int(Y/100) - 2400*int(Y/10000)) / 3600 = (20100 - 8040 - 4800) / 3600 ~ 2.0166666... |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 1 Guest(s)