Post Reply 
Improving FIX mode
10-20-2023, 02:00 PM (This post was last modified: 10-25-2023 07:29 PM by Albert Chan.)
Post: #7
RE: Improving FIX mode
(10-20-2023 11:17 AM)Pyjam Wrote:  It took me a while to figure it out too.
DIG is the minimum number of significant digits you want to see, otherwise it goes to SCI.
In FIX 3 and DIG 1, it can go down to 0.00n, and in FIX 4 to 0.000n, and so on.

In Fix 3 and Dig 0, it can go down to 0.00n too!

What is 0 significant digit? It is just too confusing.

From https://github.com/c3d/DB48X-on-DM42/iss...1765870801
Fix 4 and Dig 0: 0.000055 --> 0.0001
Fix 4 and Dig 1: 0.000055 --> 5.5000e-5

c3d Wrote:With Dig 1, the number in level 4 [0.000055] changes, because we want
at least one real significant digit to show, and the 1 there is not significant (it’s rounding)

What is real significant digit? Round up from 0 to 1 not real?

https://www.chemteam.info/SigFigs/SigFigRules.html Wrote:Rule#1: Non-zero digits are always significant.

Even without rounding up, 0 is a significant digit.

https://speleotrove.com/decimal/decifaq2.html Wrote:The number zero is a special case; because it requires one digit
to indicate its presence it is usually considered to have a precision of 1.
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
Improving FIX mode - c3d - 10-16-2023, 12:08 PM
RE: Improving FIX mode - Albert Chan - 10-16-2023, 01:28 PM
RE: Improving FIX mode - c3d - 10-16-2023, 08:27 PM
RE: Improving FIX mode - Pyjam - 10-20-2023, 08:07 AM
RE: Improving FIX mode - Albert Chan - 10-20-2023, 10:59 AM
RE: Improving FIX mode - c3d - 10-20-2023, 05:07 PM
RE: Improving FIX mode - c3d - 10-20-2023, 04:51 PM
RE: Improving FIX mode - c3d - 10-20-2023, 05:58 PM
RE: Improving FIX mode - Pyjam - 10-20-2023, 11:17 AM
RE: Improving FIX mode - Albert Chan - 10-20-2023 02:00 PM
RE: Improving FIX mode - c3d - 10-21-2023, 08:53 AM
RE: Improving FIX mode - Pyjam - 10-21-2023, 10:24 AM
RE: Improving FIX mode - c3d - 10-21-2023, 11:26 AM
RE: Improving FIX mode - Pyjam - 10-20-2023, 02:25 PM
RE: Improving FIX mode - agarza - 10-20-2023, 02:56 PM
RE: Improving FIX mode - Pyjam - 10-20-2023, 03:38 PM
RE: Improving FIX mode - agarza - 10-20-2023, 04:39 PM
RE: Improving FIX mode - Pyjam - 10-20-2023, 05:36 PM



User(s) browsing this thread: 4 Guest(s)