About calculator benchmark (8 queens) and fast devices. MS challenge #2
05-03-2017, 09:18 PM (This post was last modified: 05-04-2017 03:27 PM by Helix.)
Post: #9
 Helix Member Posts: 295 Joined: Dec 2013
RE: About calculator benchmark (8 queens) and fast devices. MS challenge #2
(05-02-2017 05:18 PM)pier4r Wrote:  I thought that the benchmark, here, was not maintained since long time. Instead it has many new results! I wonder how the maintainer tracks the new results (Xerses seems registered here, just not so active). Kudos to him.

I agree that Xerxes has made a nice work in maintaining this page, and furthermore his presentation is very clean and simple to interpret at first glance.

(05-02-2017 05:18 PM)pier4r Wrote:  Said that, I believe that executions that are getting under 1 seconds are increasingly limited by the overhead of just 'starting' the computation. Therefore I would suggest to expand the benchmark for faster devices, for example nqueens with n=9, to see how much the real computation takes on the "fast" device.

The main limitation of this benchmark in my opinion is the use of integers exclusively, which greatly favors some languages. As soon as there are some calculations with reals, this benchmark is grossly misleading.

For example, if I take the HP 50G with User RPL as a reference, then this benchmark gives the following increases of speed :
HP Prime : 65x
HP 200LX with Turbo Pascal : 213x
HP 50G with newRPL : 219x

Now, if I consider this very simple Calculator Performance Index, which uses calculations on reals, the results are dramatically different :
HP 200LX with Turbo Pascal : 5.5x
HP 50G with newRPL (12 digits) : 17.5x
HP Prime : 48x

And finally, the Savage Benchmark, which relies heavily on transcendental functions, gives the following results:
HP 200LX with Turbo Pascal : 1.5x
HP 50G with newRPL (12 digits) : 4.3x
HP Prime : 114x

I like a lot the "Calculator Performance Index", which is probably rather representative of usual scientific programs. The table of benchmarks is not as complete as the Xerxes Table, but it is instructive.

Jean-Charles
 « Next Oldest | Next Newest »