Post Reply 
Rounding errors when mimicking Mod() with FP()
03-14-2016, 08:14 AM
Post: #4
RE: Rounding errors when mimicking Mod() with FP()
Hello

Similar rounding errors are inevitable when working with floating point numbers.
Various floating point (FP) systems will behave differently depending on their implementation (binary/decimal, number of internal/external digits used for precision).
Some system will try to hide the "problems" (which itself causes other issues), other will not. Some will hide it by hiding the real value of numbers (which means that subsequent calculations are technically OK, but yield a 'strange' result), some by modifying the numbers...

Fixing a given system for a given case usually breaks it for another case :-)

Here, you want to do exact arithmetics. This can be done using binary integers or the CAS.

use #10d^R->B(X) MOD #13d for example for F1(X) and you get the result that you want (warning, the function is only defined for integer X.). You can go to the num setup, set step to 1 and got to NUM view to see the data.

irem(10^X,13) should also do what you want (this is a CAS function, integer remener).

Cyrille

Although I work for the HP calculator group, the views and opinions I post here are my own. I do not speak for HP.
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: Rounding errors when mimicking Mod() with FP() - cyrille de brébisson - 03-14-2016 08:14 AM



User(s) browsing this thread: 1 Guest(s)