Dedekind Sums

[johnb]

(02-26-2024 02:28 PM)Eddie W. Shore Wrote:  

Code:

...
a:=a-FLOOR(a)-0.5;
...
b:=b-FLOOR(b)-0.5;
...
s:=s+a*b; 
...
RETURN s; 
...

I don't have a prime, so I don't recall whether it uses a IEEE-754 binary floating point, or a BCD representation, so I may be off-base here. If it's any "binary, mantissa+exponent" representation, then 0.5 is not exactly the same thing as 1/2.

You might squeeze an extra ULP or two out of your answer by computing 'a' and 'b' as twice their values and subtract 1, then return (s/2).

Oops. LOL! Let this be a lesson to my fellow amateur mathematicians: do not attempt what you think are reversible transformations in the real domain when discontinuous functions are involved!

FLOOR(2x) / 2 ≠ FLOOR(x)


[crawling back into my hole in the ground...]