Post Reply 
Unexpected result from STRING command
05-02-2023, 03:31 AM
Post: #1
Unexpected result from STRING command
Code:
STRING(0.201168,1)
Results in "0.2012" when the current Number Format is set to "Fixed".

I thought the standard mode argument (1) would specify that the STRING command return all digits regardless of current Number Format.

When the calculator is switched to "Standard" Number Format, then the resulting string is "0.201168".

Anyone encounter this or have some sort of explanation as to why?
Visit this user's website Find all posts by this user
Quote this message in a reply
05-07-2023, 06:27 PM
Post: #2
RE: Unexpected result from STRING command
For reference, I compiled results with different HFormat settings. I just left HDigits at 3 for all these tests.

I set variable num:=0.123456789, and then used STRING(num,mode) to get these results. Bolded lines look incorrect to me.

The first result with HFormat set to 0, and then specifying fixed mode (2) adds an extra two zeroes at the end, certianly looks like a bug to me.

I would expect 0.123456789 for all strings where mode was specified as standard (1), but htis only happens when HFormat is 0. Looks like a bug to me.

HFormat = 0 (Standard) HDigits = 3
0.123456789 - no argument
0.123456789 - Mode = 0 (current)
0.123456789 - Mode = 1 (standard)
0.12345678900 - Mode = 2 (fixed)
1.23456789000ᴇ−1 - Mode = 3 (scientific)
123.456789000ᴇ−3 - Mode = 4 (engineering)
0.123456789 - Mode = 5 (floating)
0.123456789 - Mode = 6 (rounded)

HFormat = 1 (Fixed) HDigits = 3
0.123 - no argument
0.123 - Mode = 0 (current)
0.1235 - Mode = 1 (standard)
0.123 - Mode = 2 (fixed)
1.235ᴇ−1 - Mode = 3 (scientific)
123.5ᴇ−3 - Mode = 4 (engineering)
0.1235 - Mode = 5 (floating)
0.123 - Mode = 6 (rounded)

HFormat = 2 (Scientific) HDigits = 3
1.235ᴇ−1 - no argument
1.235ᴇ−1 - Mode = 0 (current)
0.1235 - Mode = 1 (standard)
0.123 - Mode = 2 (fixed)
1.235ᴇ−1 - Mode = 3 (scientific)
123.5ᴇ−3 - Mode = 4 (engineering)
0.1235 - Mode = 5 (floating)
0.123 - Mode = 6 (rounded)

HFormat = 3 (Engineering) HDigits = 3
123.5ᴇ−3 - no argument
123.5ᴇ−3 - Mode = 0 (current)
0.1235 - Mode = 1 (standard)
0.123 - Mode = 2 (fixed)
1.235ᴇ−1 - Mode = 3 (scientific)
123.5ᴇ−3 - Mode = 4 (engineering)
0.1235 - Mode = 5 (floating)
0.123 - Mode = 6 (rounded)

HFormat = 4 (Floating) HDigits = 3
0.1235 - no argument
0.1235 - Mode = 0 (current)
0.1235 - Mode = 1 (standard)
0.123 - Mode = 2 (fixed)
1.235ᴇ−1 - Mode = 3 (scientific)
123.5ᴇ−3 - Mode = 4 (engineering)
0.1235 - Mode = 5 (floating)
0.123 - Mode = 6 (rounded)

HFormat = 5 (Rounded) HDigits = 3
0.123 - no argument
0.123 - Mode = 0 (current)
0.1235 - Mode = 1 (standard)
0.123 - Mode = 2 (fixed)
1.235ᴇ−1 - Mode = 3 (scientific)
123.5ᴇ−3 - Mode = 4 (engineering)
0.1235 - Mode = 5 (floating)
0.123 - Mode = 6 (rounded)
Visit this user's website Find all posts by this user
Quote this message in a reply
Post Reply 




User(s) browsing this thread: 2 Guest(s)