CAS precision
|
10-16-2015, 03:16 PM
(This post was last modified: 10-16-2015 03:21 PM by Han.)
Post: #3
|
|||
|
|||
RE: CAS precision
Does using return(param) within a subroutine truncate any digits? One of the issues I have found is that matrix multiplication (row vector times column vector) gives me a slightly different result than calling a subroutine that does the multiplication and returns the inner product (the former is more accurate than the latter it seems). That is, t:=b*v; seems more accurate in the long run (which I use later on to for something else) than t:=inprod(b,v); where inprod(); is a cas routine whose code is:
Code: inprod(b,v):= Graph 3D | QPI | SolveSys |
|||
« Next Oldest | Next Newest »
|
Messages In This Thread |
CAS precision - Han - 10-12-2015, 07:19 PM
RE: CAS precision - parisse - 10-13-2015, 06:44 AM
RE: CAS precision - Han - 10-16-2015 03:16 PM
RE: CAS precision - parisse - 10-17-2015, 04:05 PM
|
User(s) browsing this thread: 1 Guest(s)