Bit's WP 34S and 31S patches and custom binaries (version: r3802 20150805-1)
|
05-26-2015, 07:05 AM
Post: #29
|
|||
|
|||
RE: Bit's WP 34S and 31S patches and custom binaries (version: r3747 20150207-1)
There are ways to verify the stack/volatiles clash.
Reduce the maximum matrix size and the inversion code will use less RAM. This would identify the issue but not help fix it (unless smaller matrices are okay 10x10 is pretty large). Another option would be to rework the Doolittle LU decomposition to use decimal64's for the intermediate matrix rather than decimal128's. Again, this will show if memory is running out and provide a solution where the accuracy is reduced but the matrix operations still work up to 10x10. - Pauli |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 1 Guest(s)