Post Reply 
REQUEST: Significant Digit Calculator
02-29-2020, 08:56 AM (This post was last modified: 02-29-2020 01:55 PM by StephenG1CMZ.)
Post: #11
RE: REQUEST: Significant Digit Calculator
To avoid the loss of precision in examples like 5/2 =2 to 1 sigfig, one could keep the additional digits but indicate the possible limitation of accuracy visually (for example, by changing the colour, or using bold).

So 5/2 is 2.[/red]5
Or 0.625*1.0 is 0.6[/red]25

That avoids the inaccurate 2 but gives a reminder that the result may exceed the precision of the input values, where needed.

Stephen Lewkowicz (G1CMZ)
https://my.numworks.com/python/steveg1cmz
Visit this user's website Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: REQUEST: Significant Digit Calculator - StephenG1CMZ - 02-29-2020 08:56 AM



User(s) browsing this thread: 1 Guest(s)