List Commands Library for 50g

[pier4r]

(09-29-2017 09:39 PM)DavidM Wrote:  Wouldn't it be easier to simply keep the keys and values in their own lists? That way you achieve the above with { key or value list } <target> LCNT/MPOS/etc.



Give it some thought. I think you'll find that keeping the keys and values as separate lists has some nice benefits, and KSORT provides some nice flexibility by allowing you to execute a user-supplied program on the bundled list.

Yes, as I said, I did not play around with KSORT. Having the two list separated is ok and has advantages (I can even input the bundled list and then build somehow the separated lists).

(09-29-2017 11:30 PM)John Keith Wrote:  Not to be disagreeable, but I have to vote for 2a for several reasons:


IMHO, version 2a would result in less stack manipulation than version 2b in most cases.

John

Then I change my vote to 2a too. As wrote, I don't mind much, I thought that 2b was the most stack friendly solution but it happens that I am wrong. Thanks for the explanations.

PS: little request. If you (as 'you all', not John) are going to reply generically to a long post (that is, not to specific paragraphs), could you please shorten the quoted part?

[pier4r]

After a quick look in listExt and goferlist I did not find it, so request (maybe I already requested it, I don't remember).

What about such a command (for list of integers/reals or even just reals):

L3: input list
L2: list positions to increment
L1: increments to apply (same size of the position lists)

example
L3: { 0 0 1 0 2 }
L2: { 1 2 5 }
L1: { 3 2 1 }
---
L1: { 3 2 1 0 3 }

Or is it too better to use the built in ADD ? Like:
L2: { 0 0 1 0 2 }
L1: { 3 2 0 0 1 } (maybe a bit cumbersome to create?)
ADD

Of course, given the increment, one then would request for subtractions as well (and in the best case multiplication, division). This let me think that maybe the built in operations are already good, since they are already there.


Also random pick from a list.

One could do:
(short, but maybe not always quick with large lists)
{list} LSHUF HEAD

or (way longer but pinpointing)
{list} DUP SIZE RAND * 1 + IP GET

What about something like:
{list} LRPICK
(List random pick)

Pick a random element from the list. Returns:
L2: position of the element
L1: value

{list} n LRPICKM
(List random pick multiple)

Pick randomly 'n' times from the list (so the list can be even smaller than n). Produces in output 2 lists
L2: {positions}
L1: {values}

{list} n LRPICKMU
(List random pick multiple unique)
Pick randomly 'n' times from the list without picking from the same position, so n should be at most the size of the list. Produces in output 2 lists
L2: {positions}
L1: {values}
This one can be approximated with:
{list} LSHUF n LFRST

Approximated because one has no positions.


<Spoiled>I want more high quality commands like the others in listExt that simplify programs and avoid that I write my own little libraries.
</spoiled>
(at the end it is my fault, I could have collected various short and fast programs already while reading the forum. Actually I should do it, to provide myself and others with a valid library for lazy people like me)

[John Keith]

(10-01-2017 12:56 PM)pier4r Wrote:  or (way longer but pinpointing)
{list} DUP SIZE RAND * 1 + IP GET

{ list } DUP SIZE RAND * CEIL GET is shorter and faster.

To pick multiple random items, use the above code in a loop, combine the random indices with \->LIST, and use the resulting list with LPICK. Or, just use LSHUF followed by SUB.

John

[pier4r]

(10-01-2017 02:21 PM)John Keith Wrote:  { list } DUP SIZE RAND * CEIL GET is shorter and faster.

To pick multiple random items, use the above code in a loop, combine the random indices with \->LIST, and use the resulting list with LPICK. Or, just use LSHUF followed by SUB.

John

Thanks. Anyway it is exactly my point. If I have to do it by myself I do it obviously, but putting it in a library would save (for every user and for every place it is used) development/testing time.

[DavidM]

(10-01-2017 12:56 PM)pier4r Wrote:  Or is it too better to use the built in ADD ? Like:
L2: { 0 0 1 0 2 }
L1: { 3 2 0 0 1 } (maybe a bit cumbersome to create?)
ADD

Of course, given the increment, one then would request for subtractions as well (and in the best case multiplication, division). This let me think that maybe the built in operations are already good, since they are already there.

Reading through your description, I kept having the feeling that there were simply too many potential variations for this to be turned into a generic function. It seemed like you were beginning to come to that same conclusion in your request .

One thing I'd point out, though, is that creating the list identified above as L1 could be done fairly easily from the originals in your example as follows:

Code:

\<<
  { 0 0 1 0 2 }
  { 1 2 5 }
  { 3 2 1 }
  PICK3 SIZE
  \-> pos val sz
  \<<
    0 sz NDUPN \->LIST
    pos val 2 \<< PUT \>> DOLIST
  \>>
\>>

(after which you can apply whatever function you need: ADD, -, *, etc.).

(10-01-2017 12:56 PM)pier4r Wrote:  Also random pick from a list. ...

Once again, it seems to me that this type of thing is best left for your own specific uses. There's lots of room for interpretation here, with lots of perspectives. With or without replacement? Should the list be randomized once, or after each selection? ...and more. Each variation may indicate a different approach for implementation.

While the LSHUF command performs reasonably well (especially in the most recent version), it is a "big hammer" in that it shuffles the entire list. You may not actually need all of that shuffling going on to do what you need. If you simply need 3 random elements from a list of 3000, it hardly makes sense to shuffle the entire list just to do 1 3 SUB. And that approach assumes "without replacement", which may or may not be appropriate for any given situation.

(10-01-2017 12:56 PM)pier4r Wrote:  I could have collected various short and fast programs already while reading the forum. Actually I should do it, to provide myself and others with a valid library for lazy people like me)

I think that's a good approach.

(10-01-2017 03:45 PM)pier4r Wrote:  If I have to do it by myself I do it obviously, but putting it in a library would save (for every user and for every place it is used) development/testing time.

But that assumes that the command in question has the same input/output that everyone else would need. I don't see that being the case here.

[pier4r]

(10-01-2017 04:50 PM)DavidM Wrote:  But that assumes that the command in question has the same input/output that everyone else would need. I don't see that being the case here.

well, I tried

[Thomas Okken]

(10-01-2017 02:21 PM)John Keith Wrote:  

(10-01-2017 12:56 PM)pier4r Wrote:  or (way longer but pinpointing)
{list} DUP SIZE RAND * 1 + IP GET

{ list } DUP SIZE RAND * CEIL GET is shorter and faster.

It’s not equivalent, though. With CEIL it is possible to get 0, and there is a correspondingly slightly lower probability of getting the maximum value. 1 + IP is the way to go; correctness trumps speed.

UPDATE: I just checked the manual (the 48SX manual from the MoHPC documentation set) and it says that RAND returns values in the range 0 <= x <= 1.

If that is true, then neither of the above code sequences are correct, because 1 + IP could return SIZE + 1. Seems weird though, I don’t think I’ve ever seen a random number function with that kind of range, it’s usually 0 <= x < 1 or some range of integers. Is the 48SX (and other RPL calculators?) weird in this regard, or is it a typo in the manual?

UPDATE 2: The Programmer's Reference Manual says 0 <= x <= 1, and the Owner's Manual says 0 <= x < 1.

[DavidM]

(10-01-2017 05:32 PM)pier4r Wrote:  well, I tried

Yes, nothing wrong with that.

I think it's fair to say that I'm really starting to think more in terms of refining/finalizing the existing command set now as opposed to expanding it. Unless there's something really compelling, I'm more inclined to defer new commands to some future version so that the existing set can be finalized.

There's a handful of optimizations I still need to do with the existing code base, a couple of which only came to my attention recently. A couple others have been on my TODO list for a while, but kept getting pushed down by higher priority items. There's also a couple of design decisions I'd like to get some feedback on before finalizing things. The first of those will be posted momentarily...

[DavidM]

As has been pointed out by Joe Horn and John Keith in another thread, real arguments (as opposed to exact integers) are usually better performers when passed to built-in commands for processing.

This brings up an important point about many of the ListExt commands. Previously, I had been thinking that numeric output for many of the commands should be tied to the current mode of the calculator: Exact mode would have the commands output exact integers, Approximate would cause reals instead. That seemed to be a coherent approach that respected a user's mode preference.

That said, I've also wanted to keep an eye on performance of the library's routines. In that sense, real numbers are almost always going to be the hands-down winners. Just as an example:

In exact mode, 5000 LSEQ returns the result as a list of exact integers in about 8.75 seconds.
In approximate mode, 5000 LSEQ gives the same result as a list of reals in about 1.29 seconds.

Furthermore, subsequent operations you may want to perform using those results will almost always be faster if they are reals.

My current thinking is that I should return reals for commands that aren't already specifically designated to return integers (such as NL→I), regardless of mode. If integers are needed, it's easy enough for the user to use the built-in R→I command on the result list.

Does anyone have a problem with that? Am I overlooking any significant issues with that approach?

[pier4r]

for me even if the list commands works only (and always) in real mode would be already fine. The fact that the commands are consistent with the input is a very nice feature though (and I wonder how much work is behind that!). If someone is in exact mode, maybe there is a reason (or the user will find out on his own).

[rprosperi]

(10-01-2017 05:48 PM)Thomas Okken Wrote:  UPDATE 2: The Programmer's Reference Manual says 0 <= x <= 1, and the Owner's Manual says 0 <= x < 1.

Both the 48G AUR and the 50g AUR state RAND's results will be in the range 0 <= x < 1. The 50g User Guide and User Manual do not specify the range, just stating 'between 0 and 1'. I'd guess the 48 Programmer's Ref Manual (which was very early and I don't believe was updated) was incorrect and was fixed in later manual versions.

[Werner]

RAND will never return 0 or 1. I remember from many, many winters ago. Alas, that is all I remember. So I had to look it up:

Random thoughts on HP48 random functions

Read the whole thread. Interesting stuff.
Basically, it boils down to the fact that a whole class of Random Number Generators use

x(n+1) := (x(n)*a + c) mod m

eg. the early Random Number generator used for the 41c used
c=.211327, a=9821, m=1. (This is from memory, but I think I'm right ;-)
Now, the algorithm on the 48 uses c=0, and thus can never return 0.

Cheers, Werner

[Didier Lachieze]

I’ve looked at the string functions in this library and I have a question about how numbers with a fractional part should be handled.

For example the description of I→NL ( Integer to Number List ) states that: " A real number will be treated as IP(number)", but in the examples 15.8 is rounded to 16: 15.8 I→NL => { 1 6 }

Same for NL→I, which description states that "Fractional parts of numbers are ignored", but in the example 2,718 is rounded to 3: { 3.14 2.718 } NL→I => 33

Same for NL→S which according to the description is working with absolute value of the integer part of the numbers, but in the example 67.6 is converted to "D" (ASCII 68)

[John Keith]

The built-in CHAR command rounds to the nearest integer (e.g. 67.6 CHAR returns "D") so I think that is desirable for the Library's string commands.

John

[DavidM]

(10-02-2017 12:10 PM)Didier Lachieze Wrote:  I’ve looked at the string functions in this library and I have a question about how numbers with a fractional part should be handled. ...

Thanks for pointing out the error in the documentation! The commands previously used a truncation approach, but in order to match how most built-in commands work, they were changed to use a "nearest integer" approach instead. I apparently only changed the examples in the documentation without changing the descriptive text. I've added this to my list for updates.

[DavidM]

LSPLT/LRSPL

I'm moving forward now with the assumption that option 2a was the consensus for how the result should be presented for LSPLT and LRPSL.

Reals vs. Exact Integers in results

Also, having only received 1 confirming response to my question about always returning real results (as opposed to exact integers while in Exact mode), I'm moving forward with changing all affected commands to only return real results for indexes, counts, and char codes (there's quite a few that this applies to).

These changes will be in the next release, which I'm now thinking will be a "beta" as opposed to "release candidate" due to the nature and quantity of changes that the library is undergoing.

[HP67]

(10-01-2017 05:48 PM)Thomas Okken Wrote:  UPDATE 2: The Programmer's Reference Manual says 0 <= x <= 1, and the Owner's Manual says 0 <= x < 1.

I have an SX put away. If I have to I can run a program indefinitely until it produces a 1 or drains the batteries

[Joe Horn]

(10-03-2017 07:34 PM)HP67 Wrote:  

(10-01-2017 05:48 PM)Thomas Okken Wrote:  UPDATE 2: The Programmer's Reference Manual says 0 <= x <= 1, and the Owner's Manual says 0 <= x < 1.

I have an SX put away. If I have to I can run a program indefinitely until it produces a 1 or drains the batteries

No need to do that. As Werner explained above, the RPL RAND function CANNOT return either 0 or 1, ever, guaranteed. Even the 50g AUR manual is wrong about this.

[brickviking]

(10-03-2017 08:31 PM)Joe Horn Wrote:  ... As Werner explained above, the RPL RAND function CANNOT return either 0 or 1, ever, guaranteed. Even the 50g AUR manual is wrong about this.

Hm. So how do I actually GET a value between 0 and 1 inclusive from a random function, at least in my 50G? I'm even more surprised that the venerable AUR has an error!

(Post 109)

[Thomas Okken]

(10-02-2017 08:58 AM)Werner Wrote:  eg. the early Random Number generator used for the 41c used
c=.211327, a=9821, m=1. (This is from memory, but I think I'm right ;-)

You're absolutely right, and I know that one from memory as well. :-D

That is also the RNG that I use in Free42. This thread got me thinking, though: maybe I should replace that RNG? Even if its properties are generally OK, the RNG in the real HP-42S generates numbers with 12 significant digits, while the one from the HP-41C manual generates numbers with only 6 significant digits.

And now I discover that the HP-42S RAN function behaves exactly like RAND in the 48G! Give it the same seed, and the same numbers pop out.

The behavior for seed 0 is different, but still, it looks like I've stumbled upon another way to make Free42 a bit truer to the original. Neat!