wp34s - FIB problem (DBLON)

[Paul Dale]

(11-23-2018 10:45 AM)Gerson W. Barbosa Wrote:  So the only problem is it is not accurate enough in double precision mode for certain applications. Well, not really a problem once we are aware of that.

We deliberately made no guarantees about accuracy in double precision. That the FIB function is in XROM and working at double precision pretty much guarantees it will be off in the last place or so. Flash was full so I had to pull the C implementation that worked in higher precision.

I agree ROUNDI is better for your purposes.


Pauli