Post Reply 
newRPL: Number formatting on screen
07-12-2015, 03:02 AM
Post: #16
RE: newRPL: Number formatting on screen
(07-12-2015 01:16 AM)Helix Wrote:  To be consistent with the 50g behavior, for the "SCI" and "ENG" modes, this should be the total number of significant figures minus 1.

For example, 10.12345 with 3 digits:
Standard mode (FIX): 10.123
SCI mode: 1.012E1 (4 significant figures)
ENG mode: 10.12E0 (4 significant figures)

It's not very important, however.

I had to go and check it on the calculator to believe it. I never paid any attention to it, but you are right, if you select 4 it shows 5 significant figures. I guess it makes sense for SCI, since you get the same number of figures after the decimal dot as in standard mode, but in ENG...
anyway, it's easy to implement, so I don't have a problem showing one extra digit.


(07-12-2015 01:16 AM)Helix Wrote:  I think it's not necessary, because we could get the same display with the following settings:
big number cut-off: 1000
big number format: ENG
small number cut-off: 1
small number format: ENG
mid-range number format: standard

You are right again, but I implemented it today before reading your post... now it will take me extra work to remove it, so I guess that option will stay there.

I have a generic question about total significant figures:
If I have a number, let's say 99.9999, and I want to display only 4 total significant figures, so 99.99 rounded goes to 100, but what is more correct: 100.0 or 100.00?
I'm inclined to think 100.00, since the "real" digits you are using are 99.99, the fact that there's a virtual extra digit coming from the rounding doesn't change that your 4 digits are nn.nn, so 1mm.mm seems more appropriate for 4 digits, but is it?
100.0 has 4 significant figures, but does that "virtual" 1 count as "significant" or not? I think not simply because it's not real, but I have no way of knowing if I'm right or wrong about it.
Any thoughts?
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: newRPL: Number formatting on screen - Claudio L. - 07-12-2015 03:02 AM



User(s) browsing this thread: 1 Guest(s)