HP Prime crashes on undef-undef*i in CAS

[lyuka]

Severe bug:
Calculation results of undef-undef*i itself won't stall the calculator, but,
if you try to PRINT the result containing undef-undef * i,
the calculator will stall and reboot.
In these case, using cat befor PRINT will be a workaround for the problem.

The + operator also seems to have some potential problems.

Code:

#CAS
print_undef_test():=
BEGIN
  LOCAL x, y, z;
  LOCAL g, k;
  LOCAL s;
  g := MAXREAL / 2;
  PRINT("print_undef_test:"); // Printing a string is OK.
  PRINT("(2 * g + i * g ) / (g + i * k * g), g="+ g); 
  //There is a bug that a 0 is added at the begging when printing a string and numbers in Emu,
  FOR k FROM 0.0 TO 2.0 STEP 0.5 DO
    x := 2 + i; y := 1 + i * k;
    PRINT("k="+k+":Ref.-> " + (x / y)); 
    //however, printing a string after a number is OK.
    z := (x * g) / (y * g); 
    // Calculation results of undef-undef*i itself won't stall the calculator, but,
    // PRINT(k + ":#/# -> "+ z);
    // tring to print a result that is undef-undef*i will stall the calculator to reboot.
    // s := k + ":#/# -> "+ z;
    // Using + operator to a string and undef-undef*i will stall the Emu to reboot,
    // but it's OK for Physical HP Prime.
    // PRINT(s);
    // however, printing the 's' above will stall the calculator to reboot.
    // PRINT(z);
    // Tring to print only undef-undef*i also stall the calculator to reboot.
    PRINT(cat("k=",k,":#/# -> ", z));
    // Using cat befor PRINT is a workaround for the problem above.
    // The + operator seems to have some potential problems.
  END;
  return z;
END;
#END

The version of the Physical HP Prime is:


Another bug in the Emu is that a 0 is added at the begging when printing a string and numbers.


The version of the HP Prime Emu is:

[toml_12953]

(11-29-2020 12:23 AM)lyuka Wrote:  Severe bug:
Calculation results of undef-undef*i itself won't stall the calculator, but,
if you try to PRINT the result containing undef-undef * i,
the calculator will stall and reboot.
In these case, using cat befor PRINT will be a workaround for the problem.

IMHO, even if there's a workaround, no sequence of commands should ever be allowed to crash or even reset the calculator. If one is found, it should be fixed ASAP with an out-of-band OS release or patch. That's the way the old HP would have done it (see the original HP-35 bug)

[rprosperi]

(11-29-2020 04:49 PM)toml_12953 Wrote:  IMHO, even if there's a workaround, no sequence of commands should ever be allowed to crash or even reset the calculator. If one is found, it should be fixed ASAP with an out-of-band OS release or patch. That's the way the old HP would have done it (see the original HP-35 bug)

Yeah... no, they're gone. Not forgotten for sure, but definitely gone.

As for this problem, there can be no doubt that nothing should be allowed to crash the machine, but that said, the subject command is meaningless so unlikely to ever be encountered (except by folks reading this thread) though it's creative and impressive that lyuka thought to explore it. BTW, the lowly 36-year old HP-71B handles this and properly reports this (NAN-NAN*i) as invalid.

[Steve Simpkin]

Yes I have to agree with Bob on this. This particular problem is unlikely to impact anyone. Don't get me wrong. I am glad that a bug was identified and hopefully it will be fixed in a future update. In the meantime, if it hurts to do "this", then don't do "this".

My memories of how the old HP would handle this type of issue vary. When I received my HP-28C, HP-28S and HP-48SX, they each included an errata sheet with a list of bugs and their workarounds. Some of theses bugs would be fixed in a later run of calculators but there was no offer to replace my "flawed" unit. With software in ROM, it just wasn't practical.

Here is a list of bugs for the various ROM versions of the HP-48 series over its production run (see section 3.5). Some of these bugs caused a crash and a complete memory loss. Yeah, avoid doing those things.
https://www.hpcalc.org/hp48/docs/faq/48faq-3.html

[Valentin Albillo]

.
Hi, Bob:

(11-29-2020 05:53 PM)rprosperi Wrote:  

(11-29-2020 04:49 PM)toml_12953 Wrote:  IMHO, even if there's a workaround, no sequence of commands should ever be allowed to crash or even reset the calculator. If one is found, it should be fixed ASAP with an out-of-band OS release or patch. That's the way the old HP would have done it (see the original HP-35 bug)

Yeah... no, they're gone. Not forgotten for sure, but definitely gone.

Absolutely. As I tell in this recent post, said HP engineers of old took every precaution to avoid problems even in the presence of extreme values or conditions, even for situations deemed impossible to produce. Just in case.

Gone forever indeed.

Quote:As for this problem, there can be no doubt that nothing should be allowed to crash the machine,

Absolutely. Since exception handling was made available in all major programming languages many years ago there's no excuse whatsoever for allowing the machine to crash. None.

Every exception must be handled (gracefully, if possible) and certainly such simple exceptions as the one we're discussing here (printing a complex Undef value) can and should be handled. That is, if it is the case that this should be considered an exception, which I doubt, it should simply print "Undef".

The sheer number of crashes the Prime users encounter all the time for the most inane operations is simply unacceptable, period. That's why I consider the Prime a broken toy. You simply can't trust the results you get (that is, if you get any results instead of simply a crash and reset) unless extensively checking them, thus making it impossible to use it professionally or for any serious task without checking all the time all the results.

I'd never rely on it and I'll never willingly accept to be a paying guinea pig for anyone, left to discover and suffer all the miriad bugs lurking in the thing.

Quote:but that said, the subject command is meaningless so unlikely to ever be encountered (except by folks reading this thread)

Not so. At all, as I explained in that post, but I'll summarize here:

Imagine you're conducting some vibration analysis of a large structure using a program written for the Prime (yeah, as if !). Said analysis requires finding the eigenvalues of a sizable matrix, which in turn may require finding the (complex) roots of a high-degree polynomial.

The root-finding procedure requires the computation of complex divisions involving very large values, as the ones seen in this and the other recent thread, which due to the bug in complex division can result in Undef + i Undef results, which when printed by the vibration analysis program will crash the machine.

Why did the crash occur ? Because the complex division was performed using a faulty implementation that didn't correctly deal with large values (not "normal entries", as someone called them in that thread), and then the bug wasn't corrected, and further the resulting exception wasn't handled properly, if at all. So, you correctly use some engineering program and you get a crashed machine. Not a correct result, not an exception, not a warning, not a message. Just a crash out of the blue.

Thus, the "Don't do it, man" doesn't cut it, sorry, because the user wasn't trying to do a "meaningless" operation like "Gee, let's see if this thing can print an Undef", but a valid engineering computation, which results in attempting to print such an Undef due to a bug in the complex division implementation (not in the user's program), which results in a crash.

"Don't do it" doesn't cut it. Checking in development time for all kinds of values in range and using the correct algorithms so that the users don't have to suffer the consequences of not doing so, at least fixing the blatant bugs in the most basic operations when the checks are omitted, and at the very least mandatorily handling exceptions adequately instead of crashing the machine is what cuts it.

Quote:BTW, the lowly 36-year old HP-71B handles this and properly reports this (NAN-NAN*i) as invalid.

Of course it does ! What else did you expect from the HP engineers of old ?

But I take offence at that "lowly". Please take it back.

Best regards. V.

[parisse]

(11-30-2020 07:17 PM)Valentin Albillo Wrote:  Imagine you're conducting some vibration analysis of a large structure using a program written for the Prime (yeah, as if !). Said analysis requires finding the eigenvalues of a sizable matrix, which in turn may require finding the (complex) roots of a high-degree polynomial.

The root-finding procedure requires the computation of complex divisions involving very large values, as the ones seen in this and the other recent thread, which due to the bug in complex division can result in Undef + i Undef results, which when printed by the vibration analysis program will crash the machine.

This just shows that you know very little about linear algebra and numeric analysis. Nobody will ever compute the characteristic polynomial of a large matrix to find its eigenvalues. The reason is somewhat obvious: the coefficients of the resulting polynomial are bad conditionned: for example for a 150x150 matrix with 2 digits coefficients, the leading coefficient is +/-1 while the constant coefficient (the determinant) is about 400 digits large and does not fit in a double.
In fact, proot works the converse way, if you want to find the roots of a polynomial, you get them by computing the eigenvalues of the companion matrix and this works very well for reasonably conditionned polynomials of degree up to a few hundreds.
This should relativize your opinion about my code.

[rprosperi]

(11-30-2020 07:17 PM)Valentin Albillo Wrote:  [snip]

Quote:but that said, the subject command is meaningless so unlikely to ever be encountered (except by folks reading this thread)

Not so. At all, as I explained in that post, but I'll summarize here:

Imagine you're conducting some vibration analysis of a large structure using a program written for the Prime (yeah, as if !). Said analysis requires finding the eigenvalues of a sizable matrix, which in turn may require finding the (complex) roots of a high-degree polynomial.

The root-finding procedure requires the computation of complex divisions involving very large values, as the ones seen in this and the other recent thread, which due to the bug in complex division can result in Undef + i Undef results, which when printed by the vibration analysis program will crash the machine.

My tongue may have been too firmly planted in my cheek - I was inferring (too gently it seems) that it's pretty unlikely anyone would be even attempting any such calculations with a Prime, so the risk of encountering it is remote.

(11-30-2020 07:17 PM)Valentin Albillo Wrote:  But I take offence at that "lowly". Please take it back.

Best regards. V.

You are absolutely right, Valentin, that was needlessly insensitive and inaccurate. I probably should have gone with my original 'vintage HP-71B...' but I wanted to stress it's extreme age and got careless. My humble apologies, both to Valentin and the 71B. The 71B has been my favorite machine since 1984 and it remains so today.

[Valentin Albillo]

.
Hi again, Bob:

(11-30-2020 10:47 PM)rprosperi Wrote:  

(11-30-2020 07:17 PM)Valentin Albillo Wrote:  [snip]
Not so. At all, as I explained in that post, but I'll summarize here: [...] The root-finding procedure requires the computation of complex divisions involving very large values, as the ones seen in this and the other recent thread, which due to the bug in complex division can result in Undef + i Undef results, which when printed by the vibration analysis program will crash the machine.

My tongue may have been too firmly planted in my cheek - I was inferring (too gently it seems) that it's pretty unlikely anyone would be even attempting any such calculations with a Prime, so the risk of encountering it is remote.

But that's precisely the question, and the point I want to make:

The firmware developer's job is not to make assumptions on what kinds of computations the device's users will be performing using it, nor whether said computations are more or less likely to be encountered, in the developer's subjective opinion, as long as said computations are within the specifications.

The firmware developer's job is to make sure that all intended functionality works as specified as long as the parameters are within range, taking care of both "normal values" and extreme values, without disregarding the latter because in his/her subjective opinion they're less "likely" to be encountered, possibly causing crashes when they eventually are.

Else, they should include a caveat which reads:

"This device will work as specified only for normal users doing normal computations on normal values, where "normal" should be taken as whatever the developer subjectively considers it should be. Failure to heed this caveat may result in a crashed machine."

Do you see how utterly ridiculous that would be ?

Quote:

(11-30-2020 07:17 PM)Valentin Albillo Wrote:  But I take offence at that "lowly". Please take it back.

You are absolutely right, Valentin, that was needlessly insensitive and inaccurate. I probably should have gone with my original 'vintage HP-71B...' but I wanted to stress it's extreme age and got careless. My humble apologies, both to Valentin and the 71B. The 71B has been my favorite machine since 1984 and it remains so today.

Nah, think nothing of it, I wasn't offended at all, much less at you. You're not the only one to do the "tongue-in-cheek" thing, you know ...

Thanks and Best regards, Bob.
V.

[parisse]

Actually, it *is* very unlikely that someone will 1/ obtain undef-undef*i and 2/ print it with the print command.
Proof: nobody reported that bug in 7 years.
Now that it is reported we will of course do our best to fix it.

I'm sorry to tell you that it is impossible to prevent all possible crashes, especially considering the increased complexity of a CAS calculator firmware. Exception handling will not save you from memory allocation errors.

The best way I know to run crashproof software is 1/ select a software that is used by a lot of users 2/ wait several years (of course this means that you will not have access to the latest functionnalities). Even if you consider your software to be reliable, save your work frequently. And if you plan to use the result of a (CAS) computation for something sensitive, check it, numerically, graphically or/and with a different kernel.

[Albert Chan]

(12-01-2020 07:59 AM)parisse Wrote:  Now that it is reported we will of course do our best to fix it.

Thank you, Parisse.

I see 2 ways to fix this.
1). fix PRINT to handle undef - undef*i
2). convert undef - undef*i to plain undef

I like the 2nd way, because it is more consistent, and we can easily test for it

CAS> z = 0/0      → undef
CAS> z == undef      → 1
CAS> type(z)      → DOM_IDENT

CAS> z = 1e108+1e200*i
CAS> z = z/z      → undef
CAS> z == undef      → 0
CAS> type(z)      → DOM_FLOAT

CAS> z = 1e109+1e200*i
CAS> z = z/z      → undef-undef*i
CAS> z == undef      → 0
CAS> z == undef-undef*i      → 0
CAS> type(z)      → DOM_COMPLEX

Last test RHS is "simplified" to plain undef. But, z is stuck with "complex" undef.
Internally converting all types of undef's to "plain" undef seems better.

What do you think ?

[parisse]

Well, it's not the CAS code that prints, the CAS print command calls a conversion from a CAS object to a Home object and then the Home object is printed (BTW, this may be the reason why there is a crash, converting objects is of course harder to code than when I code for CAS staying in CAS code and HP codes for Home staying in Home code).

[lyuka]

I'm sorry for the long sentence and a little sentimental.

This is what I encountered in a short period of time because of the only thing I tried
to get started with HP Prime, which my wife just gave me as a birthday present.

When explaining Widler's current source to my students, I needed Lambert's W
function to explain the change in current due to the degeneration resistor.

I wanted to plot that graph with the latest graphing calculator, HP Prime.
It's a common function for scientists and engineers, so I thought it must be
included in HP Prime, a 21st century graphing calculator with CAS.
But it didn't have Lambert's W function.

Then I wrote Lambert's W function as a programming exercise in HP Prime.
In HOME, it may be due to the size of the machine epsilon, but the behavior of
convergence by the iterative method such as Halley's method was not good.
In addition to that, I gave up using it in HOME because the standard rounding
is not an even rounding but a rounding towards zero (which accumulates errors).

I wrote the LambertW function in CAS because it is still better than HOME
because it can be used like a degraded version of IEEE 754 64-bit in CAS.
I referred to LambertW in GSL, but the initial values ​​weren't given very well,
so I decided to create another initial value approximation function.

Since the LambertW function is a multi-valued complex function, we need to
make sure that the approximation function does not jump to other branches,
not only at singularities and near zero, but also when given large values.

When I tried to plot it on HP Prime for an overview, HP Prime stalled and restarted.
As a result, the mathematical formulas I wrote experimentally were lost.

I tried to confirm by plotting instead of analytically because it is difficult to confirm
analytically when the formula to be used has not been decided yet.
What I expect from a graphing calculator is that it helps me when I think about it,
but it didn't come true.

When I was investigating to find out the reason why the plot failed, there was a place
where the required digit was temporarily doubled or tripled during the calculation.
But even though it was in the MAXREAL value range, HP Prime seemed to stall there.

When examined individually, it seems that the number of digits that can be used for
complex division is only half that of MAXREAL. I tentatively wrote a complex division
by Smith's method to deal with this.
Calculators that fail to divide complex numbers are like computer beginners trying to
make them. Fortunately, however, I found that the product of the reciprocals of
complex numbers can be used without being so bad.

HP Prime stalled and restarted when I tried to PRINT the calculated value in the
middle of each step of the calculation to see where it exceeded that digit in the formula.
I want to do my job and I don't really want to spend time doing things like these.

Apparently, HP Prime stalls and restarts when I try to PRINT what the calculation
result is undef-undef * i due to poor complex division. PRINT doesn't seem to know
how to behave with a given object. But CAT seems to know it to some extent.

As a professional in electronic circuit design, I always think it is necessary to make
sure that what I design does not behave abnormally in extreme situations, and I
always design it that way. It's a concept called failsafe or foolproof that anyone
involved in product design must know.

However, even though there are no written documents or specifications, there are
developers who say
"Please do not use it in extreme situations" or
"There is no problem because the user did not report it".
I am very sad as a technical professional.

There may be consistency issues with those developed separately, but as
development managers, we take responsibility for knowing what we are using and how.

There is no offense. But I'm disappointed as one of the generations who felt the
[(hp)] logo was a sign of trust.
HP calculators may no longer be professional tools, but more like broken educational toys.

Lyuka

[parisse]

I'm not sure you really realize the amount of work inside the CAS of the Prime. As I already said, unfortunately it is impossible to be crash-proof, especially if you are using the software to do things that are not in it's main focus. As you can guess, the CAS main target is, well, CAS.
This does not mean I do not program numerics and many numeric commands of the Prime call Giac/Xcas code, and in fact LambertW *is* available in Giac/Xcas, but it's unfortunately not available on the Prime (it's not the only command in this situation, and before you ask, it's not my decision).

It's easy to criticize especially if you don't know what is behind some decisions, it's much harder to do the work. We do our best to fix bugs, and I usually publish fixed versions of Giac/Xcas a couple of weeks after someone report a bug (things are of course different for Prime firmwares). But my time is finite (and Prime devel ressources are finite as well). I can not invest too much time in extensive testings on numerics, I have other priorities: Giac/Xcas is not a replacement for scilab/matlab or R, it's a CAS.

Now I think I have lost too much time in this thread. If your next messages are still just complains about the way I/we code, without any sign of recognition of the work I/we do ("broken educational toys"), I will just ignore them (and your bug reports will be ignored).

Goodbye.

[Gene]

Thread closed.

Lyuka please move on to something else... preferably not complaining about the PRIME/CAS, since I have had more than one complaint.

thank you. Let's find some joy in what our tools / toys ** can ** do.