Post Reply 
Some assorted HP67/97 questions
06-12-2016, 10:17 PM
Post: #72
RE: Some assorted HP67/97 questions
(06-10-2016 08:56 AM)PANAMATIK Wrote:  Reading DATA values bug is fixed! Smile

Almost. ;-)

Values below 1 with 10 significant digits are loaded with just 9 decimals. For instance a DATA entry of 0.01512345678 is read back as 0.015123456, and 0.001234512345 is truncated to 0.001234512, i.e. at nine digits after the decimal point.

On the other hand scientific notation in the DATA section seems to work fine, all ten digits are transferred. So the easiest fix is writing the DATA values always this way.

Dieter
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
Some assorted HP67/97 questions - Dieter - 06-04-2016, 11:14 AM
Larger bitmaps - striegel - 06-08-2016, 11:38 PM
Not my photo - striegel - 06-09-2016, 08:18 PM
Another image tweak - striegel - 06-09-2016, 11:08 PM
RE: Some assorted HP67/97 questions - Gene - 06-06-2016, 04:44 PM
RE: Some assorted HP67/97 questions - Gene - 06-09-2016, 03:47 PM
RE: Some assorted HP67/97 questions - Gene - 06-07-2016, 12:33 PM
Image belongs to HP - striegel - 06-12-2016, 11:54 PM
RE: Some assorted HP67/97 questions - Dieter - 06-12-2016 10:17 PM
RE: Some assorted HP67/97 questions - Gene - 06-15-2016, 04:19 PM



User(s) browsing this thread: 1 Guest(s)