WP 34S conversion error

[Dieter]

(10-09-2014 09:37 PM)Paul Dale Wrote:  As is documented in the manual, the 34S uses the NIST definitions for units where ever possible:
http://physics.nist.gov/Pubs/SP811/appenB9.html#LENGTH

Unless printed in bold, the values on that page are generally rounded to seven significant digits. So they are "close enough" but not exact. I think the 34s constants should not be limited in that way, if a more exact value is available.

Dieter