Some integrals with problematic evaluation

[Tim Wessman]

(03-23-2016 10:28 PM)quinyu Wrote:  Sure. Starting from the basic assumption that the HP Prime was meant to be aimed mainly at education (which might be wrong, but from the advertising that it had, that's how it came across):

Nope, not wrong. Even with the areas we'd like to improve in the UI I think it is the most powerful calculator out there though.

Quote: while it might do things, it does not have any step-by-step mode in places where it would matter the most, not even as an optional switch/configuration setting.

Well, that is a very recent development in xcas and is still "experimental" at this point unless I'm mistaken.

Quote:This can be integration, solving of general or differential equations, you name it. The calculator spits out a result, do what you will with it - in this respect, the 50g is actually more helpful with the proper settings.

Are you running with the "simplification" setting in CAS settings on "maximum"? Basically, "none" will do nothing. "Minimum" calls the xcas "regroup" command, and "maximum" calls the simplify command. My default xcas setting seems to be "regoup" now.

Quote:Simplify... well, it might simplify or not, it's hard to measure the "simplicity" of an expression, although if I look at a tree representation of it, it would likely be possible to select the tree with smallest average depth,

Yeah, it is tricky. I think the problem here is it wants to put everything over a single denominator. The "normal" command probably does a better job in most cases.

[quinyu]

Yeah well, I'm not trying to argue that it's a very strong calculator; it is. In some particular cases the Nspire CX CAS beats it though (recent find: integrating the following: x^n/(a^5+x^5), where n=0..3, gives a partially integrated result. I admit the Xcas [online version] doesn't give a closed form either, in that it returns "rootof" at a number of places, so if the Prime CAS is based on that, I can't expect it to solve them either. The Nspire CX CAS returns closed forms which are identical or a constant off from the expected. I am sure this will be fixed sooner rather than later, though.)

I couldn't find a changelog for xcas, so I don't know how recent this feature is. But one way or the other, it is definitely something that students will love once it's included in the Prime.

As of my simplification setting, I kept it at "minimum", being that the cases where "maximum" did any better was not worth it. Maybe I should just leave it at "none", in the end...

[quinyu]

As of complexity, I found a page which contains some observations (here) that might be worth considering - basically, the expression size (number of nodes required to represent it in a tree diagram) and expression depth (length of longest path from root to leaf) are the two measures that you're trying to minimize. Not sure if it is trivial to do so.

A more in-depth look at perceived complexity is in this paper, and although they focused on people who were visually impaired, I expect the results to be also applicable to people with normal vision, at least as a starting point.

[parisse]

Step by step was not working when the last firmware was released, and it is still in early testing stages today, you can't expect to have it inside the Prime today.
I just checked with the web version of Xcas with Firefox, int(x^1/(a^5+x^5)), int(x^2/(a^5+x^5)), int(x^3/(a^5+x^5)) return the antiderivative without any rootof. Make sure to clear your browser cache.
Simplify does not mean you will get a simpler answer, because that can not be well defined, sometimes you need a factored expression, sometimes an expanded one, sometimes with sin, sometimes with cos, etc. That's why Giac provides a lot of rewriting commands. The simplify command should perhaps be renamed, it is a more powerful version of normal in the sense that it handles relation between transcendental functions like sin^2+cos^2=1. normal handles rational expressions and algebraic extensions, ratnormal handles only rational expressions. regroup is completly different, it does not rewrite the expression as an irreducible fraction, it just collects some terms without modifying much the structure of the expression.
You will always find some expressions that are better simplified or that return a simpler integral on one CAS than on an other one. I don't know for sure what algorithms the nspire is using because their CAS is not opensource, but I think they do more table-lookups than I do (they have probably lookup for common US textbooks exercices) and did not implement more advanced algorithms like Hermite reduction for rational fractions or the Risch algorithm : for example int(1/(x^4+1)^10) is slow or int((2x^2+1)*exp(x^2)) is left as is.

[quinyu]

I'm not entirely sure about the Nspire's CAS basis, but (at least according to the developer of that system) it is based on Derive (ref: here, at the bottom of page.) In theory, Derive is/was capable to show steps of the calculation as well, but the Nspire doesn't do so. Whether or not it will later on... no clue, probably if enough students complain about it, they will.
The two integrals that you showed, I tried to time them: on the last CAS version available to me (4.2.0.532) and overclocked to 210MHz base/CPU, 52MHz AHB, the first integral evaluates in a little less than 16s. The second, as you say, is not evaluated, just split into the sum of two integrals. In comparison, the HP Prime results are effectively instantaneous. For added interest: the 50G also solves both integrals. The Classpad II does the first one in ~7s, the second in ~3s. I did not bother actually checking the result from the first, too long for me to compare by hand, but there were no integrals left in it on the CX, 50G or CP. Derive (v6.10) can solve the first as well as the second, and from the displayed steps I can definitely see that it is looking for patterns. Some of the patterns are pretty curious, but they seem to work in the end. So if Derive can solve the second, I'm unsure why the CX cannot, unless they took some earlier branch (that couldn't) as codebase, or trimmed down the available rules...

[quinyu]

Found an integral that essentially freezes the Prime as well as the web-based or binary version of Xcas:

Code:

int(sqrt(sqrt(sec(x)+1)-sqrt(sec(x)-1)))

The Prime tells me

Code:

Warning, integration of abs or sign assumes constant sign by intervals (correct if the argument is real):
Check [abs(cos(x))]

While it's true that the integral is nasty, it does have a closed form. It's more the issue of freezing/locking up that's to be corrected, at this point; also, there is no hourglass visible, but I do assume that the calculator remains busy one way or another... might be a completely unrelated bug, or perhaps not.

[parisse]

Unfortunately the Prime and the web version of Xcas do not have exceptions enabled, this make computations difficult to interrupt. Of course, I could have a closer look and probably find a workaround for this specific integral, but this will not solve the real problem, and you would find another problematic integral. Therefore if you are running very complicated integrals on the Prime, take a paperclip with you, or run it on the binary version of Xcas where there is a working STOP button that can cancel a computation (by clean exception throw or by thread cancellation).

[quinyu]

Now a general response to all manufacturers of CAS calculators: Look here, guys. You take > US$100 for your calculators. Is it such a big thing to ask to pack at some convenient place, such as the battery cover, or perhaps the calculator cover itself, a reset pin, so that the user doesn't need to carry a paper clip at all times? I mean... oh okay. Your calcs don't actually ever need one. I get it. *duh*

[quinyu]

A curious one that sort of locks up the calculator:

Code:

Int(1/(((x+1)^2)^(1/3)-sqrt(x+1)),x)

Still interruptable by clicking On, doesn't show no hourglass; Enter is pressed in console, where a warning about Check [abs(x)] is given. With assume(x>-1), it works and gives the good result. However, without any assumption,

Code:

Int(1/(1+sqrt(x+1)),x)

calculates just fine. What's the issue here?

[parisse]

If you don't make assumptions, it takes 30s on the emulator on my desktop (2s in Xcas natively), while with assumptions, you get the answer in 0.14s.
By the way, how many integrals did you test?

[quinyu]

Well, another motivation not to use/trust the emulator for testing. I have waited for at least 15 minutes on the real metal, pressing the occasional enter when the screen started to dim, as to prevent auto power off. For your interest: the 50g has no problems with it. The Classpad II, apart from putting the absolute value in the logarithm, doesn't have any issues either. The Nspire CX CAS however refuses to do it, no clue why... seeing that Derive can do it.

As of your question to the number of integrals tested: I'm done with the first three chapters of Timofeev's book (that counts 81+90+14=185, minus the few that I didn't feel there was a point checking, there being too many factors a, b, c, a', b', etc.), just started the fourth chapter, of which I checked 7 so far, the last one being the one I've just reported. I will proceed with the rest, as my time permits.

A not buggy but slightly interesting case was with the integral Int((x^(3/2)-3*x^(3/5))*(4*x^(3/2)-1/3*x^(2/3)),x). On the 50g, it created truly giant factors, because of wanting to bring everything to a common denominator; the answer was in fact correct, but almost unreadable. On the contrary, the Prime didn't do that, but refused to add up powers, so the answer had terms such as x^(1/10)*x^(2/3)*x^3. Is there any manner to unify powers? I know there can be arguments why not to ever do that, but presuming someone wants to undertake that risk?

[parisse]

The 50 is much less rigorous than giac, if the sign of an expression had to be computed inside such kind of integration, I think that the variable was assumed to be near 0. This is normal considering the relative speed of the 49 and a desktop of today. The Prime CAS is a port of giac, it's not a CAS specifically designed to run on a calc. It has some limitations, speed of course (relative to the desktop) but also the OS. You can do a lot more on the Prime than on the 50, but for complex computations, you must help the system (assume) or go on the desktop.
About testing on the real Prime, I generally don't do that, it's much easier to copy/paste on the emulator. And if it takes half a minute on the emulator, then I know it's not very useful to test on the Prime since long computations are likely to have a much more defavorable speed ratio because memory would probably be almost exhausted on the Prime.

[quinyu]

I don't know about the memory management strategies used in the HP Prime OS/CAS; but if it differs from that of the emulator's, then perhaps it would be preferable to call it a simulator, ultimately; from what you just said, it sounds that it doesn't exactly replicate the underlying state of the real device (as memory constraints, for example.) While it is justifiable not to replicate everything, such as system clock timing during calculations, so that you get your results quicker, but it's not justifiable to make the software on the PC behave differently than the device would in real life, as far as the results are concerned; it's cool, I'm not trying to put blame on you with it, I'm just saying maybe it wasn't such a great idea for them to allow different memory usage - as long as we're not getting expansion possibilities for the real world device and could shove in a 64Gb piece of DDR3 memory or something of the sort... for one thing, it can possibly mislead the people wanting to buy the device, believing that their calculator would behave identical as the simulator.

But since the calculator manufacturers are still focusing on selling their hardware, as opposed to simulators/emulators (though the emus are a nice addition), I judged it'd be better to test on the actual raw iron. The proof of the pudding, you see... or "c'est à l'usage qu'on peut juger" from what I could find. Much like how it is with engineering - ultimately, it boils down to "is it doable in real life, without having to use space age technology?" And the other reason why I'm focusing on testing the real-life units now is that ultimately, that's where people would likely have problems in real-life situations. If you are using a computer to emulate them, it's not that bad - but for any real usage, you'd fire up Mathematica, xcas/giac, Maple, Matlab, wxMaxima or something similar on that computer instead (your preferences may vary.)

[parisse]

(04-02-2016 01:20 PM)quinyu Wrote:  If you are using a computer to emulate them, it's not that bad - but for any real usage, you'd fire up Mathematica, xcas/giac, Maple, Matlab, wxMaxima or something similar on that computer instead (your preferences may vary.)

The main use of emulators (as I see it) is to explain students how to use the calc. As you say if you want to do real usage, you will use a desktop, for the same reason you will probably not enter very complicated integrals on a calc.

[quinyu]

Sure, we fully agree on this one. The question is, so to say, when does the user start to "see the break," that is, realise when he should do that switch. The term "complicated" in itself is a very subjective term. And then - most users, when they see that the calculator/CAS does return a result that isn't a symbolic explosion and on first glance looks plausible, they prefer to assume that the result is correct. You, or me for that matter, know better, but the "average" user doesn't, which can lead to a number of unfortunate consequences.

When the calculator says (in one form or the other) "cannot do," they surely make a switch, but not otherwise, since in their judgement the handheld/calculator is still capable of returning a (what they think) correct answer. Extreme long waiting times might influence this judgement of course, suggesting it might be better to switch over to some other system, with a better hardware foundation. In one manner, I respect the Classpad II for this, for it dares to return the original integral as an answer, indicating "sorry, man, this is above my level."

Of course, unlike with PCs, the hardware in the calculators varies much more wildly, so actual comparison of what the raw iron could do are close to impossible, never mind what the programs on it are able to do. Using FLOPS or something similar is close to being irrelevant here, I feel. And while it's certainly nice for the calculator users to get a result quickly, my personal feeling is that it's more important for the users to be able to trust their calculators that if/when it gives an answer, it's actually a correct answer. Let them be aware that the system has its limits - but let them know when the question they asked of the system can be answered (correctly, mind you) by the same system, and that they are getting that result.

Before assuming that I'm praising it to high heavens, there was a case on the CPII where I needed to switch to Complex mode to evaluate an integral, even though there was nothing related to complex numbers either in the integrated function or the corresponding result. My emotion at that time was that it was underhanded. How are you supposed to deduce by looking at an integral that the calculator will only be able to handle it properly in complex mode, when it has only perfectly real functions and terms in it? You can't. So this is where I feel tricked, cheated, call it what you will... but I'll cut this rant here, it's already longer than I originally intended it to be.

[parisse]

(04-02-2016 08:04 PM)quinyu Wrote:  Sure, we fully agree on this one. The question is, so to say, when does the user start to "see the break," that is, realise when he should do that switch. The term "complicated" in itself is a very subjective term. And then - most users, when they see that the calculator/CAS does return a result that isn't a symbolic explosion and on first glance looks plausible, they prefer to assume that the result is correct. You, or me for that matter, know better, but the "average" user doesn't, which can lead to a number of unfortunate consequences.

It should be the job of math teachers to explain to students when they can trust a calculator, and when they should not. Unfortunately many math teachers do not want to learn that (they say it is not math ... a good excuse not to learn).

Quote:When the calculator says (in one form or the other) "cannot do," they surely make a switch, but not otherwise, since in their judgement the handheld/calculator is still capable of returning a (what they think) correct answer. Extreme long waiting times might influence this judgement of course, suggesting it might be better to switch over to some other system, with a better hardware foundation. In one manner, I respect the Classpad II for this, for it dares to return the original integral as an answer, indicating "sorry, man, this is above my level."

This could only be done by being very conservative. In other words by adding more and more checks, you ease use for students who do not want to learn a bit of maths, but in the same time you will add limits for students who are accepting to learn a bit of maths. My opinion is that it is not the right way to go.

[quinyu]

Once again, we agree that many math teachers simply don't take the time to understand what's going on under the surface. Nevertheless, for the general high school teacher, it might well be a bit too high expectation to know and fully understand for example the Hermite-Kronecker-Brioschi characterization (which in itself isn't a particularly complicated one, at least in terms of programming), and know the associated caveats when it may or may not be used, or even the path to reduce a quintic equation to a form suitable for the method. From that point on, conveying the same bulk of information to the high school students might border on applied sadism. And strictly speaking, the above-mentioned method is still in the field of algebra, not that of calculus; and not something that'd make its way into the Journal of Symbolic Computation.

But I believe I understand your point of view as well; limitations, particularly ones without a good reason, can often be very annoying, calculators or otherwise. Nevertheless, maybe there's a middle way, allowing the user himself to limit what the calculator is doing for him. In the HP, or any other handheld, the methods are hidden behind smoke and mirrors (and for this reason, I particularly praise this recent development in xcas.) Expose that interface - let it be configurable what the calculator itself is willing to do or not. For the students who have insight, this can be a very practical way of taking things at their own pace, or teachers who want to influence more precisely what may happen in exam mode - right now it is rather coarse-grained. I already know (or expect the student to know) how to solve a standard quadratic equation - let the calculator do that for me, it's just boring number crunching from that point on. I didn't yet learn (or teach) how to calculate derivatives of an expression containing hyperbolic functions - turn it off.

The whole point of computers in general was to take the task of boring number crunching off the shoulder of humans who can instead focus on the parts of the task requiring intuition or insight (which computers lack.) The math teacher likely expects no rote learning from the students (excepting perhaps such basic things as a multiplication table or some basic identities), but the development of the skill to see where a method or another might be useful, regardless of the field. In a manner, the handful of "tutors" that are included in Maple are a good step in that direction. The tutor isn't going to tell you what to do with the integrand (unless you ask); you can apply any of the methods listed, if you so choose, and if it's at all applicable. In a while, the student learns to see which option is viable in any given situation.

And while Maple certainly can do integration by simply using the command "int," in my point of view, the real satisfaction comes from gaining the insight and applying it properly; and perhaps you can ultimately arrive to a neater form than that produced automatically by the CAS. Perhaps not; but you can compare and learn from it, particularly if you see the stepwise approach, assuming it's human-compatible. I wouldn't call it human-compatible, for example, to spot that a part of your infinite series is actually rewritable as the sum of three Taylor series and simplify it accordingly. Maybe, when you've been in that particular field for several years, it's doable, but for everyone else... it's not their game.

[parisse]

(04-03-2016 04:37 PM)quinyu Wrote:  Once again, we agree that many math teachers simply don't take the time to understand what's going on under the surface. Nevertheless, for the general high school teacher, it might well be a bit too high expectation to know and fully understand for example the Hermite-Kronecker-Brioschi characterization (which in itself isn't a particularly complicated one, at least in terms of programming),

?? Never heard of that.
I don't think math teachers should learn very specialized algorithms, but they should learn when a problem can be solved by a CAS with full confidence, and what they can expect from a CAS. Things like you can not expect a CAS to make a particular case when simplifying x^2/x. Or know that you can use a CAS with full confidence when computing with polynomials and rational fractions. Or the difference between exact and approx representations...