newRPL: Number formatting on screen
|
07-12-2015, 03:02 AM
Post: #16
|
|||
|
|||
RE: newRPL: Number formatting on screen
(07-12-2015 01:16 AM)Helix Wrote: To be consistent with the 50g behavior, for the "SCI" and "ENG" modes, this should be the total number of significant figures minus 1. I had to go and check it on the calculator to believe it. I never paid any attention to it, but you are right, if you select 4 it shows 5 significant figures. I guess it makes sense for SCI, since you get the same number of figures after the decimal dot as in standard mode, but in ENG... anyway, it's easy to implement, so I don't have a problem showing one extra digit. (07-12-2015 01:16 AM)Helix Wrote: I think it's not necessary, because we could get the same display with the following settings: You are right again, but I implemented it today before reading your post... now it will take me extra work to remove it, so I guess that option will stay there. I have a generic question about total significant figures: If I have a number, let's say 99.9999, and I want to display only 4 total significant figures, so 99.99 rounded goes to 100, but what is more correct: 100.0 or 100.00? I'm inclined to think 100.00, since the "real" digits you are using are 99.99, the fact that there's a virtual extra digit coming from the rounding doesn't change that your 4 digits are nn.nn, so 1mm.mm seems more appropriate for 4 digits, but is it? 100.0 has 4 significant figures, but does that "virtual" 1 count as "significant" or not? I think not simply because it's not real, but I have no way of knowing if I'm right or wrong about it. Any thoughts? |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 1 Guest(s)