To Jeff

[komame]

Jeff,

Is there anything we can do as a community to make HP Prime better? Perhaps we can develop some data, files, such as refining on-calc help or other things that would somehow improve the work on the new firmware version? Personally, I'd be willing to work on fixing known bugs, or even analyze the code and perform low-level optimization (I think there's a lot that can be improved in PPL and the user interface), but I understand that there's no chance of access to things like source code (C++) at least until working indirectly or directly for HP. With the fantastic hardware that HP Prime has, we could work wonders....
Let us know if there's anything we can do.

Piotr

[David Hayden]

I don't know if it's still true, but a few years ago, middle- and high-school textbooks often had calculator exercises with instructions for the TI-83 family. I've always thought that it would be useful for the community to create supplements for HP calculators so that students and teachers were more comfortable using a Prime with the textbook. For this to work, someone would have to do a little research to find popular textbooks to work on and we'd probably need some reassurance from HP, Royal or Moravian that they would publicize the supplements.

For the calculator itself, we can test and report bugs, including improvements to the help files and manual.

Dave

[jte]

(09-18-2023 06:54 PM)komame Wrote:  Jeff,

Is there anything we can do as a community to make HP Prime better? Perhaps we can develop some data, files, such as refining on-calc help or other things that would somehow improve the work on the new firmware version? Personally, I’d be willing to work on fixing known bugs, or even analyze the code and perform low-level optimization (I think there’s a lot that can be improved in PPL and the user interface), but I understand that there’s no chance of access to things like source code (C++) at least until working indirectly or directly for HP. With the fantastic hardware that HP Prime has, we could work wonders....
Let us know if there’s anything we can do.

Piotr

Piotr,

Thanks for this question.

If anyone reading this wants to contribute with a substantial amount of translation work, I’d recommend contacting Klaas (and including, in the message, some background information on oneself).

For very small translation hiccups (minor irritants noticed): these could be sent to me or reported on the bug tracker site I’ve created for the project. (If an account is wanted for that, those interested should contact me.) And, of course, other software bugs can be reported on the bug tracker I’ve set up.

I know that you, Piotr, are interested in benchmarking and efficiency. I do plan on setting up some more automated testing. If benchmark programs are collected / created, that could help; they could be used to monitor performance impacts of changes. ideally these would be reflective of real-world / common / typical use ["in classroom" can certainly count!]. (Even failing that, broad coverage could help guide development work.) Note that a variety of automated testing is already being done; it’s just that I’m interested in increasing the amount being done.

Jeff

[jte]

(09-19-2023 08:43 PM)David Hayden Wrote:  I don't know if it's still true, but a few years ago, middle- and high-school textbooks often had calculator exercises with instructions for the TI-83 family. I've always thought that it would be useful for the community to create supplements for HP calculators so that students and teachers were more comfortable using a Prime with the textbook. For this to work, someone would have to do a little research to find popular textbooks to work on and we'd probably need some reassurance from HP, Royal or Moravian that they would publicize the supplements.

For the calculator itself, we can test and report bugs, including improvements to the help files and manual.

Dave

Dave,

Thanks for the solid comments. That textbooks include instructions for certain families of calculators is certainly true. There is a lot of momentum in the educational market. Many teachers are often stretched for time and would appreciate useful materials. One of the challenges in a project of this sort is getting all the parts to fit together: instructors, students, calculators, supporting materials (lesson plans, textbooks, manuals, videos, ...; workshops, conferences, ...), and education departments / boards / ministries. I've been involved a bit with some of this (as some has connections with software development). (But this is still somewhat outside my wheelhouse for this project, so please keep that in mind — I might not be exactly on-target with my reply.)

For those reading, I just did a quick Google search for HP Prime workshop / activities / courses and more than a few links came up... Here are just a few:

• Mathematical Exploration with HP Prime (a 3-page PDF from Moravia, put together by Chris Olley; a very to-the-point example of how to use the Advanced Graphing app in a lesson), (I’d tend to think these sort of proto-lesson plans could be quite appreciated by busy teachers)
• HP Prime Advanced Workshop (a 24-page PDF from HP; shows how the Geometry app, PPL, and the 1-var Stats app can be used in an education setting),
• HP Prime Algebra Workshop Materials (a 21-page PDF from HP, put together by GT Springer; shows how the Advanced Graphing app, HOME, the Spreadsheet app, and the Geometry app can be used in an education setting), and
• HP Prime Documentation Files (a web page with links to webinars, materials from summer institutes, workshops, ...), and
• Expanded HP Prime Finite Math Course (a post on this forum about an HP Prime-oriented learning center).
  

(This is from a quick off-the-cuff search... There’s much more out there; this isn’t meant to be representative in any way, or to be read as me endorsing specific third-party materials or groups.) Putting together materials and getting it out there should certainly help! (Perhaps I’m showing my age what with all the PDF links... social media and video-sharing sites are certainly “a thing”. )

(And, yes, reporting bugs is good. Myself, I've recently been busy mostly with tasks that wouldn't be directly reflected in the bug tracker. But I will be looking at bugs in the bug tracker; even just knowing of them can help improve overall development efficiency.)

Jeff

[komame]

Jeff,

(09-21-2023 05:06 AM)jte Wrote:  [...] I do plan on setting up some more automated testing. If benchmark programs are collected / created, that could help; they could be used to monitor performance impacts of changes. ideally these would be reflective of real-world / common / typical use ["in classroom" can certainly count!]. (Even failing that, broad coverage could help guide development work.) [...]

Could you provide an example of a test? Preferably one that already exists. Is it required in the form of a PPL program only, or could it be a single command performing a complex (time-consuming) task? What about Python? Please provide some context.

Piotr

[jte]

(09-24-2023 06:01 AM)komame Wrote:  Jeff,

(09-21-2023 05:06 AM)jte Wrote:  [...] I do plan on setting up some more automated testing. If benchmark programs are collected / created, that could help; they could be used to monitor performance impacts of changes. ideally these would be reflective of real-world / common / typical use ["in classroom" can certainly count!]. (Even failing that, broad coverage could help guide development work.) [...]

Could you provide an example of a test? Preferably one that already exists. Is it required in the form of a PPL program only, or could it be a single command performing a complex (time-consuming) task? What about Python? Please provide some context.

Piotr

Piotr,

Here’s a quick roughing-up of a format. Each benchmark would be a PPL program that contains a “BENCHMARK” PPL function (that takes no parameters). PPL source comments would be used to state benchmark information (this information would be used when generating reports). An abstract description is as follows:

Code:

// bench    name: <text>
// bench  abbrev: <text>
// bench timeout: <integer>
// bench numiter: <integer>
// bench  author: <text>
// bench contact: <text>
// bench   birth: <date> 
// bench     mod: <date> 
// bench version: <version number>
// bench description
<multiline text>
// bench history
<multiline text>

With the <…> as follows:

• <text>: string ended by newline
• <integer>: [1-9][0-9]*
• <date>: [2-9][0-9][0-9][0-9]/[0-9][0-9]/[0-9][0-9]
• <version number>: [1-9][0-9]*
• <version number>: [1-9][0-9]*.[0-9]+
• <multiline text>: sequence of lines, each line starting with “//”, ended by blank line (pretty printer would trim out the same number of spaces after each line’s “//” as there were after the “//” line with the bench tag just before the multiline text)
  

and the meanings as follows:

• name: name of benchmark
• abbrev: shorter name of benchmark
• timeout: number of seconds before benchmark is aborted (default could be 30)
• numiter: number of iterations performed by the benchmark (charts would divide elapsed time by this; reports could be “per iteration”)
• author: name of benchmark author; this line could be repeated for multiple authors (one name per line of this sort)
• contact: contact information for benchmark author; this line is for the last-listed author (before the contact line) and should have one form of contact (e.g., email address); for multiple types of contact information, multiple “bench contact” lines could be used
• birth: publication date for first version of benchmark
• mod: publication date for current version of benchmark
• version: benchmark version number; the latest version would be used in automated benchmarking (with A.B > a.b if A > a or A = a and B > b)
• description: description of benchmark
• history: version history for benchmark
  

There only need be one space between “bench” and the bench tag (“name”, “timeout”, …).

Automated processing would generally trim out extra spaces, something along the lines of the following, where “{ }” denotes ignored spaces:

Code:

//{ }bench { }name{ }:{ }<text>{ }
//{ }bench { }abbrev{ }:{ }<text>{ }
//{ }bench { }timeout{ }:{ }<integer>{ }
//{ }bench { }numiter{ }:{ }<integer>{ }
//{ }bench { }author{ }:{ }<text>{ }
//{ }bench { }contact{ }:{ }<text>{ }
//{ }bench { }birth{ }:{ }<date>{ }
//{ }bench { }mod{ }:{ }<date>{ }
//{ }bench { }version{ }:{ }<version number>{ }
//{ }bench { }description{ }
//{ }bench { }history{ }

<date>: [2-9][0-9][0-9][0-9]{ }/{ }[0-9][0-9]{ }/{ }[0-9][0-9]
<version number>: [1-9][0-9]*
<version number>: [1-9][0-9]*{ }.{ }[0-9]+

Order of “// bench” lines generally doesn’t matter, “bench contact” being an exception as it associates with the last “bench author” that came before it.
“// bench” lines should generally be given at most once, with the exceptions being “bench author” and “bench contact”.

An example follows:

Code:

// bench    name: Sum of Random List
// bench  abbrev: SoRL
// bench timeout: 40
// bench numiter: 100
// bench  author: John Smith
// bench contact: jsmith@…
// bench   birth: 2023/09/24
// bench     mod: 2023/09/24
// bench version: 1
// bench description
// Creates and sums up a list of 1,000 random numbers from 0 to 1.

// bench history
// 2023/09/24: first version

EXPORT BENCHMARK()
BEGIN
 FOR I FROM 1 TO 100 DO
  ΣLIST(MAKELIST(RANDOM(1),X,1,1000))
 END;
END;

[komame]

(09-24-2023 11:02 PM)jte Wrote:  Piotr,

Here’s a quick roughing-up of a format. Each benchmark would be a PPL program that contains a “BENCHMARK” PPL function (that takes no parameters). PPL source comments would be used to state benchmark information (this information would be used when generating reports). An abstract description is as follows:
[...]

Thanks for the example. I'll try to prepare something. However, I also see a certain inconvenience with the descriptions available under the 'Help' button. For many instructions, there is no documentation, and the descriptions that are there are often outdated; there is a lack of descriptions for all the syntax, and the examples provided there don't work at all (for example, for ROTATE [the Demo_ROTATE subroutine cannot be executed]). As for translations, we can use artificial intelligence. The quality of Chat-GPT's translations is already good enough that I believe there shouldn't be a major issue with it. It easily distinguishes between code and content to translate, etc. Moreover, we can verify this by translating it back (perhaps even using a different tool).

Should we also report such errors in the descriptions under the 'Help' button in the bug tracker for now? There could be quite a lot of them...

Piotr

[KlaasKuperus]

(09-18-2023 06:54 PM)komame Wrote:  Jeff,

Is there anything we can do as a community to make HP Prime better? Perhaps we can develop some data, files, such as refining on-calc help or other things that would somehow improve the work on the new firmware version? Personally, I'd be willing to work on fixing known bugs, or even analyze the code and perform low-level optimization (I think there's a lot that can be improved in PPL and the user interface), but I understand that there's no chance of access to things like source code (C++) at least until working indirectly or directly for HP. With the fantastic hardware that HP Prime has, we could work wonders....
Let us know if there's anything we can do.

Piotr

Did I mention before I love this community?!

Please send me a mail with your details and capacities and I am sure we can find a way to work together. My address is klaas.kuperus@moravia-consulting.com.

[Tyann]

Bonjour Jte

Comme je sais que vous avez un peu suivi mes échanges avec Komame pour le programme de dictionnaire, je voulais attitrer vôtre attention sur un problème de la Prime.
J'ai publié mon application sur un site français 'TI-Planet' et avant j'ai testé le chargement de l'appli pour voir si tout se passait bien, et j'ai rencontré un problème en essayant de supprimer l'application, la prime semble rentré dans une boucle et ne pas en sortir :
au bout de 40 minutes la machine s'est éteinte pour alimentation faible, lorsque je l'ai rallumé après l'avoir connecté au PC et validé par 'ok' alors l'appli était bien effacée.
J'ai remarqué aussi que l'effacement de mes variables AVars prenait beaucoup de temps.
Les AVars pèsent 9,33 Mo dans l'appli.


Hello Jte

As I know you've been following my exchanges with Komame about the dictionary programme, I wanted to draw your attention to a problem with Prime.
I published my application on a French site 'TI-Planet' and before I tested the loading of the application to see if everything went well, and I encountered a problem when trying to delete the application, the Prime seems to enter a loop and not leave:
after 40 minutes the machine shut down due to low power, but when I turned it back on after connecting it to the PC and pressing 'ok', the app had been deleted.
I also noticed that deleting my AVars variables took a long time.
The AVars weigh 9.33 MB in the app.

[komame]

Jeff,

I have another question regarding the creation of these test programs. I know that what you showed is just an example, but I have a certain observation I'd like to share. Each loop takes some time by itself, even when it's completely empty. I conducted tests of empty loop execution for different firmware versions, and for example, in version 14588, the REPEAT loop is slower than WHILE, whereas in version 14730, it's the opposite. The FOR loop can also change its performance by 20% between two firmware versions.
If the code inside the loop executes relatively quickly in a single iteration, this could lead to incorrect conclusions because the time taken for iterating the loop itself may make you think that the code inside the loop started to run faster/slower, when in fact it's the internal loop processing that has changed. In such a situation, wouldn't it be better to reduce the number of iterations by a factor of 5 but at the same time duplicate the code inside the loop 5 times to minimize the impact of the iterations themselves?

My measurnments [ms]:

\begin{array}{|c|c|}
\hline \textbf{Firmware}& \textbf{REPEAT }& \textbf{WHILE} & \textbf{FOR}\\\hline
\textbf{14588} & 5953 & 6335 & 1685 \\\hline
\textbf{14730} & 7474 & 6955 & 1980 \\\hline
\end{array}
Perhaps it doesn't matter much, but I preferred to ask to avoid doing something senseless.

And the last question:
Should I split these programs into separate files or do you want one large text file?

[jte]

(10-01-2023 06:30 AM)komame Wrote:  Jeff,

I have another question regarding the creation of these test programs. I know that what you showed is just an example, but I have a certain observation I'd like to share. Each loop takes some time by itself, even when it's completely empty. I conducted tests of empty loop execution for different firmware versions, and for example, in version 14588, the REPEAT loop is slower than WHILE, whereas in version 14730, it's the opposite. The FOR loop can also change its performance by 20% between two firmware versions.
If the code inside the loop executes relatively quickly in a single iteration, this could lead to incorrect conclusions because the time taken for iterating the loop itself may make you think that the code inside the loop started to run faster/slower, when in fact it's the internal loop processing that has changed. In such a situation, wouldn't it be better to reduce the number of iterations by a factor of 5 but at the same time duplicate the code inside the loop 5 times to minimize the impact of the iterations themselves?

My measurnments [ms]:

\begin{array}{|c|c|}
\hline \textbf{Firmware}& \textbf{REPEAT }& \textbf{WHILE} & \textbf{FOR}\\\hline
\textbf{14588} & 5953 & 6335 & 1685 \\\hline
\textbf{14730} & 7474 & 6955 & 1980 \\\hline
\end{array}
Perhaps it doesn't matter much, but I preferred to ask to avoid doing something senseless.

And the last question:
Should I split these programs into separate files or do you want one large text file?

Piotr,

Thanks for the ongoing dialogue. Loop unrolling is certainly a common approach that is considered for improving performance (whether if done by hand or by the interpreter / compiler). If some benchmarks were in the benchmark collection with different amounts of unrolling, those could help give some insight as to matters such as those you brought up. (And similarly, with different types of loop structures.)

There are many possibilities in symbolic processing of code that could be considered in the future: constant lifting (moving calculations of constants out of loops --- or even non-constant expressions which do not depend on looping etc.), function inlining / expansion (moving the body of called functions into their caller), strength reduction (replacing more expensive function calls with less expensive ones --- based on knowledge of arguments), etc. The graphing engines in their processing do what can be seen as variants of these (viewed through the lens of a general interpreter / compiler); it is easier to do these sorts of things in the graphers as the code there punts to the general-purpose PPL interpreter when things like loops are in the expression to be graphed (or, with the Advanced Graphing app, reject the input). Even some benchmarks comparing the trade-offs here might illuminate the sort of changes that could be beneficial.

The bigger picture is: what sort of code is typical for a PPL program? This is why I mentioned "real world" earlier. (If optimizations like the above are already being done, other alternatives are to be considered... Or: do these sorts of tricks apply to typical PPL code?)

I'd prefer the benchmarks to be one per file. (If not delivered that way, that's one of the first things I'd do --- separate them into files, one file per benchmark. I'd think this would make collection maintenance simpler.)

Jeff

my 2¢

[komame]

I have a feeling that your response went further than the scope of my question, or perhaps we misunderstood each other. I wasn't asking about optimizations but rather about a scenario where a simple example from the 'real world,' which executes quickly in a single iteration, has its timing disrupted by the internal handling of the loop iteration itself, which can constitute a significant portion of the measurement time. In such a case, it would be difficult to determine whether a change in development affected the measured operation or just the loop handling. If the measured operation takes a long time, the impact of loop handling will be minimal/negligible. However, if that operation runs very quickly (in a time not much longer than the loop handling), the impact of loop handling on the measurement will be significant. Hence, my suggestion was to, instead of writing

Code:

EXPORT BENCHMARK()
BEGIN
 FOR I FROM 1 TO 100 DO
  ΣLIST(MAKELIST(RANDOM(1),X,1,1000))
 END;
END;

write it as

Code:

EXPORT BENCHMARK()
BEGIN
 FOR I FROM 1 TO 20 DO
  ΣLIST(MAKELIST(RANDOM(1),X,1,1000))
  ΣLIST(MAKELIST(RANDOM(1),X,1,1000))
  ΣLIST(MAKELIST(RANDOM(1),X,1,1000))
  ΣLIST(MAKELIST(RANDOM(1),X,1,1000))
  ΣLIST(MAKELIST(RANDOM(1),X,1,1000))
 END;
END;

which would minimize the impact of the loop on the measurement.

That's why I showed measurements for the variability in the speed of empty loops (without code inside) between different firmware versions because in the scenario described above, these changes can have a significant impact on measurement results and lead to incorrect conclusions (for example, you make a change in the firmware, and then you might think that due to that change, the operation being measured inside the loop takes longer, even though it operates at the same speed; it's just that the loop itself runs slower internally). Reducing the number of loop iterations while maintaining the same number of executions of the measured operation will result in a smaller impact of the loop handling on the measurement results.
However, perhaps I'm being overly precise with this...

[jte]

(10-01-2023 06:09 PM)komame Wrote:  I have a feeling that your response went further than the scope of my question…

(Note: after writing the bulk of this, I had an aha! moment: see the blue text below.)

more like: wandered off? Not an unfair call. There are a variety of types of benchmarks. A question to ask is: what is the purpose of the benchmark? If we're aiming to improve user experience, considering the chunks of code users will typically run is likely where to start. — or, more likely, approximations thereof: chunks of code we'd think similar to those users would run.

(10-01-2023 06:09 PM)komame Wrote:  ⋮
I wasn't asking about optimizations but rather about a scenario where a simple example from the 'real world,' which executes quickly in a single iteration, has its timing disrupted by the internal handling of the loop iteration itself, which can constitute a significant portion of the measurement time. In such a case, it would be difficult to determine whether a change in development affected the measured operation or just the loop handling.

My comments regarding optimization has connections with several things, two being: 1. do PPL authors optimize their code using techniques like those I mentioned (e.g., loop unrolling [or, more precisely, are such techniques applied to code hotspots]) --- if so, benchmark sets including loop unrolled code would help the benchmark results better reflect "real world" use; and 2. will such optimization techniques improve run times of typical PPL code (e.g., how beneficial would automatic loop unrolling be?)?

If the benchmark collection included typical PPL code whose performance was impacted by loop overhead, and improvements could be made to reduce loop overhead, that would seem to be a good thing to make happen. (Or, at least, a good thing to monitor. And such benchmarks would be reflective of real-world use. So adjusting the system to benefit such code [to improve those benchmarks] would seem entirely appropriate.)

If we're focused too much on one particular line, changes aimed at improving performance may well improve the performance of that one line, but at the cost of others. (At an abstract level, it's a bit like compression: performance is improved for some programs, but at the cost of others. Usually "the others" are not the sort of programs that are of concern [e.g., execute extremely quickly in any case], although this line of reasoning can become less abstract and more practical as performance improves.)

My plan was to have benchmarks run in the standard way as standard PPL programs (apps are also a possibility) on physical calculators so as to get measurements of the "whole picture" (well, as regards the performance of PPL programs on HP Primes). If we want to aim at code snippets, some other approaches are possible (do the looping outside of the PPL interpreter --- the graphing engines do this sort of thing, and also carry some interpreter state information across evaluations to avoid setup costs).

(10-01-2023 06:09 PM)komame Wrote:  ⋮

Code:

EXPORT BENCHMARK()
BEGIN
 FOR I FROM 1 TO 100 DO
  ΣLIST(MAKELIST(RANDOM(1),X,1,1000))
 END;
END;

write it as

Code:

EXPORT BENCHMARK()
BEGIN
 FOR I FROM 1 TO 20 DO
  ΣLIST(MAKELIST(RANDOM(1),X,1,1000))
  ΣLIST(MAKELIST(RANDOM(1),X,1,1000))
  ΣLIST(MAKELIST(RANDOM(1),X,1,1000))
  ΣLIST(MAKELIST(RANDOM(1),X,1,1000))
  ΣLIST(MAKELIST(RANDOM(1),X,1,1000))
 END;
END;

This is precisely what I meant by loop unrolling (it is an example of loop unrolling), in case I wasn't clear.

(10-01-2023 06:09 PM)komame Wrote:  ⋮
That's why I showed measurements for the variability in the speed of empty loops (without code inside) between different firmware versions because in the scenario described above, these changes can have a significant impact on measurement results and lead to incorrect conclusions (for example, you make a change in the firmware, and then you might think that due to that change, the operation being measured inside the loop takes longer, even though it operates at the same speed; it's just that the loop itself runs slower internally). Reducing the number of loop iterations while maintaining the same number of executions of the measured operation will result in a smaller impact of the loop handling on the measurement results.
However, perhaps I'm being overly precise with this...

As an abstract comment: automation (automated benchmarking) can help us get a better handle on variability as it is more practical to run more tests. (Both across revisions, but also how other factors influence performance.)

If there are a substantial number of benchmark programs, my intuition is that performance changes involving things like loop overhead would show up across a large swath of the results. Thinking on it now, it seems that if loop unrolling is to be used, having that explicitly mentioned in the specially-formatted benchmark comment lines would be proper. I was going to write a bit more about this, but then...

My aha! moment: I was assuming the goal wasn't to measure tiny snippets of fast code --- code small enough that putting it into a loop would substantially adjust its performance. (That the time spent for the entire "real world" use being measured is substantially affected by wrapping it in a loop that executes once.)

[komame]

Thank you for the additional explanations, Jeff.
I think everything is clear now. I'll need some time, and I'll get back to you when I manage to prepare some meaningful benchmars/test cases.

PK

[komame]

Jeff,

I've just sent you the benchmarks I prepared. I await your feedback.