Be quiet, NaN

[Albert Chan]

I was adding a feature to my old luajit 1.1.8, having randomseed() return its state.
(So that I can save it and restore later ...)

I had patched luajit using xoroshiro128+.
With 128 bits state, my first thought were to return 2 64-bits IEEE double.
(luajit (still) does not support 64-bits integer)

But, if the returned double happens to be signaling NaN, it change the bits, and turn it quiet.

gcc:1> union {double d; uint64_t u;} h
gcc:2> h.u = 0x7ff0000000000001ULL /* signaling NaN */
gcc:3> printf("%I64x", h.u)
7ff0000000000001
gcc:4> h.d = h.d /* just copied itself change the bits ! */
gcc:5> printf("%I64x", h.u)
7ff8000000000001

The fix is hard, see In which the CPU changes my data under my nose

---

I gave up 2 IEEE double approach, and return 4 32-bits integers.

Below is the example that tripped my 2 IEEE double approach.
Last 64 bits = 7ff4d423272049a1, silently converted to 7ffc ...

lua> randomseed(599)
1122989783       -1462116390       2146751523       656427425

lua> state = { randomseed() }            -- save state
lua> random(), random(), random()
0.2792170183740439       0.4552087223210315       0.9415057196559153
lua> ; randomseed(unpack(state))       -- restore state
lua> random(), random(), random()
0.2792170183740439       0.4552087223210315       0.9415057196559153

[mpark]

Wow, interesting! Thanks, Albert.

[cruff]

Was your runtime support perhaps compiled with '-fno-signaling-nans'?

[Albert Chan]

(10-14-2020 09:22 PM)mpark Wrote:  Wow, interesting! Thanks, Albert.

This is one nasty bug

The fix is ugly, and not worth the hassle. I am quoting from Geoff Chappell post (for float)
Signaling NaN (float, double) becomes quiet NaN when returned from function (both x86 and x64)

Quote:Compiled for 32-bit Windows, GenerateNaN will return your NaN through the floating-point register ST(0). This loads your pattern into the FPU. If exceptions - the signal of your "signaling NaN' - are masked, then your NaN is silently converted from 0x7F800001 to 0x7FC00001. I don't think there's anything you can do about this except to unmask the exception and get the signal. It's the FPU architecture. But if you do want the signal, the CRT has the _controlfp_s routine, which may help. (I confess to having never ever had any real-world use for this.)

Compiled for 64-bit Windows, the pattern is returned through XMM0 without being an FPU operand.
So you get no conversion.

(10-14-2020 10:46 PM)cruff Wrote:  Was your runtime support perhaps compiled with '-fno-signaling-nans'?

Yes, I think that was gcc default setting.
https://gcc.gnu.org/onlinedocs/gcc/Optim...ze-Options