Precision problem with matrix inversion

[Anders]

(07-15-2015 04:59 AM)cyrille de brébisson Wrote:  Hello,

I can only speak of the numerical mode here.
The matrix inversion and multiplication have been coded with 3 conflicting constraints:
- High precision
- 0 result elements should be 0
- n^3 (not n^4) and memory constraint

Of course, it is not possible to have all 3 at the same time.
- Calculations ARE done in 15 digit precision on a row by row, but are stored in 12 digit (for memory)
- small elements (smaller in magnitude than epsilon*min number) are transformed into 0
- n^3 algorithm

This leads to the behavior that you observed.
If you do want full precision, that is what the CAS is there for.

Cyrille

So:
1) I suggest this is a bug in the RPN implementation
2) Solving linear equation system through a vector / matrix division is maybe the most basic linear algebraic function that an engineer or a student performs, so it needs to work also in RPN mode
3) HP matrix capable RPN calculators have been able to perform vector - matrix division correctly with sufficiently high precision for ~30 years until now with Prime (I know of HP 28C, HP 28S, 48SX, 50G), so of course it is possible to implement this correctly. These calculators had 2KBs+ of RAM (HP 28C had 2KB) and Prime have 3-4 magnitudes more RAM with 32MB. The "memory constraint" argument does not hold.
4) There are many books and papers written on algorithms to solve vector and matrix division (and matrix inversion) with sufficient precision ranging from O(n^2.3727) to O(n^3). There are also iterative models to improve the result until error < epsilon. In other words, this is a very well research and understood problem. Why would you use an O(n^4) algorithm? (O=Ordo)
5) Suggesting to use CAS mode, which is likely disabled in Exam mode, is not helpful for students if you want them to use RPN (or are you saying we should stop using RPN?)

In general I think HP should thank users (like Maro) for bringing these cases to their attention and address it in an update. You are basically receiving free testing, helping you to make a better product.