To Jeff

[jte]

(10-01-2023 06:30 AM)komame Wrote:  Jeff,

I have another question regarding the creation of these test programs. I know that what you showed is just an example, but I have a certain observation I'd like to share. Each loop takes some time by itself, even when it's completely empty. I conducted tests of empty loop execution for different firmware versions, and for example, in version 14588, the REPEAT loop is slower than WHILE, whereas in version 14730, it's the opposite. The FOR loop can also change its performance by 20% between two firmware versions.
If the code inside the loop executes relatively quickly in a single iteration, this could lead to incorrect conclusions because the time taken for iterating the loop itself may make you think that the code inside the loop started to run faster/slower, when in fact it's the internal loop processing that has changed. In such a situation, wouldn't it be better to reduce the number of iterations by a factor of 5 but at the same time duplicate the code inside the loop 5 times to minimize the impact of the iterations themselves?

My measurnments [ms]:

\begin{array}{|c|c|}
\hline \textbf{Firmware}& \textbf{REPEAT }& \textbf{WHILE} & \textbf{FOR}\\\hline
\textbf{14588} & 5953 & 6335 & 1685 \\\hline
\textbf{14730} & 7474 & 6955 & 1980 \\\hline
\end{array}
Perhaps it doesn't matter much, but I preferred to ask to avoid doing something senseless.

And the last question:
Should I split these programs into separate files or do you want one large text file?

Piotr,

Thanks for the ongoing dialogue. Loop unrolling is certainly a common approach that is considered for improving performance (whether if done by hand or by the interpreter / compiler). If some benchmarks were in the benchmark collection with different amounts of unrolling, those could help give some insight as to matters such as those you brought up. (And similarly, with different types of loop structures.)

There are many possibilities in symbolic processing of code that could be considered in the future: constant lifting (moving calculations of constants out of loops --- or even non-constant expressions which do not depend on looping etc.), function inlining / expansion (moving the body of called functions into their caller), strength reduction (replacing more expensive function calls with less expensive ones --- based on knowledge of arguments), etc. The graphing engines in their processing do what can be seen as variants of these (viewed through the lens of a general interpreter / compiler); it is easier to do these sorts of things in the graphers as the code there punts to the general-purpose PPL interpreter when things like loops are in the expression to be graphed (or, with the Advanced Graphing app, reject the input). Even some benchmarks comparing the trade-offs here might illuminate the sort of changes that could be beneficial.

The bigger picture is: what sort of code is typical for a PPL program? This is why I mentioned "real world" earlier. (If optimizations like the above are already being done, other alternatives are to be considered... Or: do these sorts of tricks apply to typical PPL code?)

I'd prefer the benchmarks to be one per file. (If not delivered that way, that's one of the first things I'd do --- separate them into files, one file per benchmark. I'd think this would make collection maintenance simpler.)

Jeff

my 2¢