Summation based benchmark for calculators

[pier4r]

(12-24-2017 02:32 AM)AlexFekken Wrote:  Simpson's rule is for student's only. ... But perhaps it explains your obsession (in this thread) with summations.
...
i.e. the opposite of combining a bunch of ad hoc operations into a single benchmark and then hoping that your benchmark becomes popular.

...
But I would certainly *avoid* calling transcendental functions (whose execution time would probably dominate the total execution time) and then suggest that I am testing the performance of a summation loop...

I do perceive the above sentences as confrontational and belittling. If this would be the case, I don't understand why and I think it is not needed. I mean, whatever the topic is, if it bugs you, you can just ignore it.

Said that, some answers.

First, thanks for your input about what you think should be a benchmark. Anyway I disagree with it. Not technically though. Technically you may be also right, but because what you suggest is quite time consuming since a benchmark like the one in this thread is based on voluntary participation. If you design a task that is time consuming, likely no one (I myself!) will participate. With no data, there is no benchmark.
You yourself said "Lots of words, encouraging you to do lots of work, by someone who isn't going to do that work. ". That confirms my point above.

Second. A benchmark is limited in scope. I can benchmark whatever, if someone uses the calculator for a well defined tasks that is not included, even partially, in the benchmark, the benchmark is useless. I think everyone knows this. In this thread I wanted to collect the time needed to repeat some functions from the set of trig , ln/exp, power functions. That's it. And I thank everyone for it!
This also leads to the third point.

Third: my apparent obsession with summation (thanks for jumping to conclusion!). I see the summation as an easy way to have a loop that tests the given instructions. A loop that can be found also in scientific calculators that are not programmable . If you find another way to let non programmable scientific calculators do a loop without summation or integration, I am interested, seriously.
With a summation, a user can quickly press (if the function is available): Sum function, the function as argument of the sum, the variable, extremes. Execute it. Report the time. End of the task.

4th. In connection to the third point. "Simpson's rule is for student's only.". This sounds like "if you are using a decimal approximation of pi with less than 109 digits, your are doing it wrong". If that sounds good to you, good to know.
Aside from that, I used the Simpson's rule because the only loop available in a sharp 506w is the numeric integration, and it uses the Simpson's rule (the image posted previously is taken from the 506w manual). Then I tried to adapt the formula in the numeric integration so to have the closes possible similarity between the summation formula and the numeric integration on the 506w.

I hope my answers helps to clarify why I did this or that.

Going back to the benchmark topic. Is there anyone willing to do the test with a:
- a 41 version
- 41 CL
- dm41 --> covered by grbanks
- hp 42s
- 35s --> covered by grbanks
- 12C (recent)
- 15C (LE got covered by Gilles)
- 71B
- 67
- 34S

Thanks for your help!

The rest got mostly covered (although having confirmations of the timings, by independent tests, is always good). Well aside from 50g hpgcc and nspire lua, but in those cases the benchmark becomes time consuming, since it is not immediate to program with lua/hpgcc in the calculator.

edit: thanks for the additional results Gilles!
edit2: thanks for the additional results, Grsbanks!

[klesl]

According to the pier4r 's recommendation I set the settings to approximate and display digits - float 6
Ti-Nspire CX CAS - approx. 2.5s, 1395.35

n=10000
Ti-Nspire CX CAS - approx. 24.5s, 13955.9

[grsbanks]

(12-24-2017 01:31 PM)klesl Wrote:  According to the pier4r 's recommendation I set the settings to approximate and display digits - float 6
Ti-Nspire CX CAS - approx. 2.5s, 1395.35

n=10000
Ti-Nspire CX CAS - approx. 24.5s, 13955.9

I can corroborate these timings, although I get 22.5s for n=10^4

n=10^5 => 222s

[pier4r]

added and fixed some entries thanks to the results of grsbanks! (in some cases timings were confirmed, but information was added)

Interesting observations from him

Quote:I can confirm that the DM41 is much slower than the DM15/DM11 (I used my DM11L this time around). The DM41 really is a slow machine. Plenty of features, sure, but dog slow. In fact it's the slowest of them all and really not something you want to use for intensive number crunching.

Regarding the Prime, I'm guessing that it generates a list under the hood and then runs 'ΣLIST()' on it (when using the built in sum function rather than a hp ppl program, note by Pier). Now, the Prime cannot manage lists longer than 10000 elements and if you try and use the summation function on a series with more than 10000 terms then it hangs and you need to press the Reset button on the back.

Given the results so far I would say.

- Impressive the 991ex, pretty quick for its price. In general the flagship casio products are all quite competitive.
- the prime is on par with the nspire when using the internal sum function but the limit of 10000 entries sounds bad (although few would use more than 10000 entries).
- the prime hp ppl is impressive. 10 times faster than any other calculator reported. (I guess lua with the nspire would reach a similar speed, but it is unusable from the calculator)
- the hp 50g summation is more advanced than a userRPL for loop. Well done. Plus it still holds its ground after years. For example in comparison versus the new primz.
- newRPL pushes the 50g greatly, well done.
- the dm42 is fricking fast. Especially compared to other swissmicro products.
- the first nspire (from 2006) is still competitive. Therefore the main systems did not change much in the nspire.
- I learned that the ti 89 has quintillions of apps (really, quintillions), but in terms of speed the real competitor was the 48 series. The 50g has almost the double of the speed.
- hp 35s . Suprisingly slow! I was expecting the speed of the 15 LE.

Still missing:
- a 41 version
- 41 CL
- 12C (recent)
- 15C
- 71B
- 67
- 34S
- Casio classpad (if possible 300 . We got the 400)
- Some sharpr PC
- other interesting calculators (the casio fx5008p for example)

[lrdheat]

WP 34S double on, fix 2: ~185 seconds
Double off, fix 2: ~168 seconds

[lrdheat]

CASIO Prizm fox-CG10: ~24.5 seconds (looks like CG50 is about twice as fast with this task)

[Paul Dale]

(12-24-2017 11:15 PM)lrdheat Wrote:  WP 34S double on, fix 2: ~185 seconds
Double off, fix 2: ~168 seconds

Firmware version? Version 2 firmware is a fair bit faster than version 3.
Using the build in Sigma command or a programmed loop? The latter will be faster.


Pauli

[lrdheat]

Version 3.3 3774

My CASIO fx-CG10 report was using it's summation function.

[emece67]

(12-24-2017 09:09 AM)pier4r Wrote:  Going back to the benchmark topic. Is there anyone willing to do the test with a:
- a 41 version
- 41 CL
- dm41 --> covered by grbanks
- hp 42s
- 35s --> covered by grbanks
- 12C (recent)
- 15C (LE got covered by Gilles)

• hp 15C
  • 10: 32'', 13.71183502
  • 100: 320'', 139.2971874
    
• DM15L (@12 MHz)
  • 10: 6'', 13.71183502
  • 100: 63'', 139.2971874
    
• DM15L (@48 MHz)
  • 100: 14'', 139.2971874
  • 1000: 133'', 1395.346288
    

- 71B
- 67
- 34S

• wp34s (DSE-based loop, DBLOFF)
  • 10: 3'', 13.71183501670439
  • 100: 26'', 139.2971870459241
  • 1000: 256'', 1395.346287743423
    
• wp34s (DSE-based loop, DBLON)
  • 10: 3'', 13.71183501670437880652763283584306
  • 100: 27'', 139.2971870459242385751150019615150
  • 1000: 285'', 1395.346287743423256291575365067091
    
• wp34s (\(\Sigma\) command, DBLOFF)
  • 10: 3'', 13.71183501670438
  • 100: 30'', 139.2971870459242
  • 1000: 303'', 1395.346287743423
    
• wp34s (\(\Sigma\) command, DBLON)
  • 10: 3'', 13.71183501670437880652763283584306
  • 100: 34'', 139.2971870459242385751150019615149
  • 1000: 332'', 1395.346287743423256291575365067093
    

Amazed by the really small (~10 %) increase in time in the wp34s when switched from 16 to 34 digits.

[Paul Dale]

(12-25-2017 12:46 AM)emece67 Wrote:  Amazed by the really small (~10 %) increase in time in the wp34s when switched from 16 to 34 digits.

The 34S computes in beyond double precision internally regardless of the mode set. My feeling is that the reduction in memory accesses (copying mostly) account for the bulk of the difference between the two. However, there are a few functions where being in single precision allows some short cuts.


Pauli

[pier4r]

(12-24-2017 11:15 PM)lrdheat Wrote:  .

(12-24-2017 11:17 PM)lrdheat Wrote:  .

(12-25-2017 12:46 AM)emece67 Wrote:  .

Thanks! (and Merry Christmas!)

edit:
It is also interesting to see the difference between systems in this test (that is more math heavy) compared to the 8 queen test (that is more add/sub + memory heavy).

In this test newRPL is 5.5 times faster than userRPL on the 50g (6s vs 33s). A bit less when the 50g uses the sum function.
In the 8 queen test newRPL is 216 times faster than userRPL (0.106s vs 22.6s).

Still missing:
- a 41 version
- 41 CL
- 12C (recent and old)
- 71B
- 67
- Casio classpad (if possible 300 . We got the 400)
- Some sharp PC
- other capable (programmable or sum function) calculators (the casio fx5008p for example)

[emece67]

(12-25-2017 08:25 AM)pier4r Wrote:  Still missing:
- a 41 version

• hp 41CV (plain, fullnut, no turbo)
  • 10: 16'', 13.71183502
  • 100: 158'', 139.2971874
    

- 41 CL
- 12C (recent and old)
- 71B
- 67
- Casio classpad (if possible 300 . We got the 400)
- Some sharp PC
- other capable (programmable or sum function) calculators (the casio fx5008p for example)

Twice as fast as a 15C, slightly faster than expected.

[klesl]

Casio fx-9700GE - approx. 1m 16s, 1395.34628774
Casio ClassPad 330-A - approx. 18s, 1395.346288
Casio fx-5800P - approx. 2m 46s, 1395.346288
Casio fx-7400GII - approx. 20s, 1395.346288
Casio fx-9860G Slim (hacked to 9860GII OS 2.04) - approx. 23s, 1395.346288

HP-40gs - approx. 34s, 1395.3462877

Canon X Mark I Pro - between 14m a 14m 50s, 1395.346288

Notice: The ClassPad is labeled on the back side as CLASSPAD300PLS, the 330-A is written on the box

[pier4r]

Thanks for the new entries!

Observation. As discussed in this thread the 35s, fx5800p and dm42 are the top3 options for a scientific calculator with ability to store many formulas. Anyway I am impressed by the difference of speed between the 35s and the fx5800p (note, it may be that for normal usage this difference will be never noticed). The dm42 is just amazing but it is also 4 time the price of 35s and fx5800p (and again, we are talking only about speed, not common operations).

Question to emece67. Is your 15C much older than your 41CV ?

Still missing
- 41 CL
- 12C (recent and old)
- 71B
- 67
- Some sharp PC
- other capable (programmable or sum function) calculators

[emece67]

(12-25-2017 02:25 PM)pier4r Wrote:  Question to emece67. Is your 15C much older than your 41CV ?

Cannot say, the 41 I used has its CPU board changed. Originally it was a 41C, now it is a 41CV. I do not know the serial number of the donor machine.

Regards.

[klesl]

All my previous tests used sum function.

n=1000

Casio Basic, loop "For To Next"

Casio fx-50F PLUS - approx. 10m 33s, 1395.346288
Casio fx-3650PII - approx. 9m 22s, 1395.346288
----------------------------------------
Integral calculation

Sharp EL-W506X - approx. 64s, 1394.866122
Sharp EL-506X - approx. 97s, 1394.026202, n=500

[BobVA]

HP-67. 100 terms (139.2925695) took 5m 40s.

[pier4r]

(12-25-2017 06:16 PM)klesl Wrote:  -
Sharp EL-506X - approx. 97s, 1394.026202, n=500

(12-25-2017 07:24 PM)BobVA Wrote:  -

Both included. Thanks and Merry Christmas /happy holidays.


Still missing
- 41 CL
- 12C (recent and old)
- 71B
- Some sharp PC
- other capable (programmable or sum function) calculators

[BobVA]

HP-41CL / x50 speed: 100 iterations (139.2926) 15s

[mfleming]

(12-26-2017 03:45 AM)BobVA Wrote:  HP-41CL / x50 speed: 100 iterations (139.2926) 15s

I get something different. 41CL, v5 board, TURBO 50. Using the Sigma+ function to accumulate intermediate values because it has the advantage of returning the index X each invocation. Twelve instructions from the loop header label to the bottom GTO with no instructions other than the 41C/CV/CX defaults:

10 13.711835 0.55 seconds
100 139.297187 5.13 seconds
1,000 1395.346260 48.31 seconds
10,000 13955.84859 7 minutes 48.24 seconds

Oddly, this quick hack turned out to be faster than dedicated register counter and accumulator versions. I guess that's the speed advantage of MCODE over FOCAL programs, eh?

Interesting problem to play with!
~Mark