CAS precision

[Han]

Does a CAS program behave like built-in CAS commands in the way it handles floating point numbers and intermediate results?

[parisse]

Yes, it should.

[Han]

Does using return(param) within a subroutine truncate any digits? One of the issues I have found is that matrix multiplication (row vector times column vector) gives me a slightly different result than calling a subroutine that does the multiplication and returns the inner product (the former is more accurate than the latter it seems). That is, t:=b*v; seems more accurate in the long run (which I use later on to for something else) than t:=inprod(b,v); where inprod(); is a cas routine whose code is:

Code:

inprod(b,v):=
BEGIN
  local d,k;
  local s;

  d:=dim(v);
  s:=0;
  d:=d[1];

  for k from 1 to d do
    s:=s + b[1,k]*v[k,1];
  end;
  return(s);
END;

[parisse]

This might happen because the intermediate conversions double <-> giac::gen are not exactly the same in both processes. Every time a double is stored in a giac::gen, the 5 less significant bits are replaced by the type field, and in the other direction the 5 less significant bits are set to 0.