Unexpected result from STRING command
|
05-07-2023, 06:27 PM
Post: #2
|
|||
|
|||
RE: Unexpected result from STRING command
For reference, I compiled results with different HFormat settings. I just left HDigits at 3 for all these tests.
I set variable num:=0.123456789, and then used STRING(num,mode) to get these results. Bolded lines look incorrect to me. The first result with HFormat set to 0, and then specifying fixed mode (2) adds an extra two zeroes at the end, certianly looks like a bug to me. I would expect 0.123456789 for all strings where mode was specified as standard (1), but htis only happens when HFormat is 0. Looks like a bug to me. HFormat = 0 (Standard) HDigits = 3 0.123456789 - no argument 0.123456789 - Mode = 0 (current) 0.123456789 - Mode = 1 (standard) 0.12345678900 - Mode = 2 (fixed) 1.23456789000ᴇ−1 - Mode = 3 (scientific) 123.456789000ᴇ−3 - Mode = 4 (engineering) 0.123456789 - Mode = 5 (floating) 0.123456789 - Mode = 6 (rounded) HFormat = 1 (Fixed) HDigits = 3 0.123 - no argument 0.123 - Mode = 0 (current) 0.1235 - Mode = 1 (standard) 0.123 - Mode = 2 (fixed) 1.235ᴇ−1 - Mode = 3 (scientific) 123.5ᴇ−3 - Mode = 4 (engineering) 0.1235 - Mode = 5 (floating) 0.123 - Mode = 6 (rounded) HFormat = 2 (Scientific) HDigits = 3 1.235ᴇ−1 - no argument 1.235ᴇ−1 - Mode = 0 (current) 0.1235 - Mode = 1 (standard) 0.123 - Mode = 2 (fixed) 1.235ᴇ−1 - Mode = 3 (scientific) 123.5ᴇ−3 - Mode = 4 (engineering) 0.1235 - Mode = 5 (floating) 0.123 - Mode = 6 (rounded) HFormat = 3 (Engineering) HDigits = 3 123.5ᴇ−3 - no argument 123.5ᴇ−3 - Mode = 0 (current) 0.1235 - Mode = 1 (standard) 0.123 - Mode = 2 (fixed) 1.235ᴇ−1 - Mode = 3 (scientific) 123.5ᴇ−3 - Mode = 4 (engineering) 0.1235 - Mode = 5 (floating) 0.123 - Mode = 6 (rounded) HFormat = 4 (Floating) HDigits = 3 0.1235 - no argument 0.1235 - Mode = 0 (current) 0.1235 - Mode = 1 (standard) 0.123 - Mode = 2 (fixed) 1.235ᴇ−1 - Mode = 3 (scientific) 123.5ᴇ−3 - Mode = 4 (engineering) 0.1235 - Mode = 5 (floating) 0.123 - Mode = 6 (rounded) HFormat = 5 (Rounded) HDigits = 3 0.123 - no argument 0.123 - Mode = 0 (current) 0.1235 - Mode = 1 (standard) 0.123 - Mode = 2 (fixed) 1.235ᴇ−1 - Mode = 3 (scientific) 123.5ᴇ−3 - Mode = 4 (engineering) 0.1235 - Mode = 5 (floating) 0.123 - Mode = 6 (rounded) |
|||
« Next Oldest | Next Newest »
|
Messages In This Thread |
Unexpected result from STRING command - Jacob Wall - 05-02-2023, 03:31 AM
RE: Unexpected result from STRING command - Jacob Wall - 05-07-2023 06:27 PM
|
User(s) browsing this thread: 1 Guest(s)