Post Reply 
@Thomas Klemm -> CORDIC Article
06-04-2014, 06:37 PM
Post: #27
RE: @Thomas Klemm -> CORDIC Article
(06-04-2014 06:30 PM)pito Wrote:  WP34s calc and various other arbitrary precision decimal fp libs/wrappers are using the IBM's decNumber library (which sets the standards). There are some settings for the rounding inside (context something), I never messed with them in detail, though. Maybe the difference between the results is related to the proper rounding settings know how within the library..

Yes, but decNumber doesn't have sin, cos, tan, so everybody has their own implementations. We should all have correctly rounded results, so in theory all programs using a fixed precision should give the exact same number (not 9.0).
The ones that use adaptive precision or use higher internal precision and the rounding is for display only (not for intermediate results) would be able to show 9.0.

What we can do, is use Wolfram Alpha to do all the steps, but typing the 34 digit rounded result from the previous answer to get the final result with rounding. Same as I did in the stack.

Claudio
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
@Thomas Klemm -> CORDIC Article - Tugdual - 05-31-2014, 08:00 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-01-2014, 08:28 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-02-2014, 10:13 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-02-2014, 10:58 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-02-2014, 11:05 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-02-2014, 11:20 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-03-2014, 02:02 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-04-2014, 02:52 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-04-2014, 06:06 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-04-2014, 06:30 PM
RE: @Thomas Klemm -> CORDIC Article - Claudio L. - 06-04-2014 06:37 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-04-2014, 06:45 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-04-2014, 07:16 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-04-2014, 07:30 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-04-2014, 07:51 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-04-2014, 08:12 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-04-2014, 08:34 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-04-2014, 09:21 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-04-2014, 09:33 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-04-2014, 09:41 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-04-2014, 09:50 PM
RE: @Thomas Klemm -> CORDIC Article - pito - 06-04-2014, 10:46 PM



User(s) browsing this thread: 1 Guest(s)