newRPL: Alpha demo 0.9 released [UPDATED 2017-10-25]

[The Shadow]

(08-26-2017 01:14 AM)Claudio L. Wrote:  Yes, I'll add those. I can't see why you would want to hold alpha for those but it's OK. I can see why you wanted that for the arrow, though, makes perfect sense as it's part of the name but even if you are typing a formula you press alpha only for the name, then you can type the symbols without alpha.

Now that you mention it, those aren't really all that necessary except for pi and the line feed and possibly the comma. You're right, the brackets and so on have no real reason to hold alpha for.

[Claudio L.]

(08-26-2017 04:40 AM)The Shadow Wrote:  Simple. You generate as many sets of 18 digits as you need to fit the precision and stick them together. (In my RPL implementation, I've just been adding them together as strings.) Any excess gets rounded off as usual. The only thing you have to watch out for is to pad the front of a set of 18 with leading 0's if necessary.

I'd have to get 16 digits out of the 18, which is 2 groups of 8 (reals internally store 8 digit groups). That means a random 2000 digit number would need 120 calls to this RNG. It better be fast!

(08-26-2017 04:40 AM)The Shadow Wrote:  Incidentally, I've been encountering some oddities with the editor. It seems that when programs get past a certain size, you get weird errors. Like, a variable I've been using all along suddenly gets changed to 'INVALID_COMMAND', or I suddenly get an error of 'Invalid word' when I try to exit. I haven't done anything I know of to justify these things.

This shouldn't happen, no matter what you do (nothing you do justifies data corruption). Can you try to find a sequence of events to reproduce it? I need your guidance to track it. The INVALID_COMMAND is the string the decompiler uses when an object is not valid. Most likely one of the commands you used wrote to an object that was moved during a garbage collection. If the command is not coded properly, the object moves, and some other object takes its place and will get corrupted. What commands were you trying when this happened?
To fix these things, turn the calc off and back on. The calculator runs MEMFIX every time it wakes up, scanning for invalid objects. If an invalid object is found, it will be replaced with the BINT zero (you'll lose the damaged object, but it won't be invalid anymore).

(08-26-2017 04:40 AM)The Shadow Wrote:  EDIT: Oh, and I coded a quick version of what we called ->NFMT above (though I called it ->NFS to emphasize it produces a string) and it is very useful. It's especially nice for getting all the digits of a number, and to shed those pesky dots marking uncertainty. In fact I've got a little program XDOT that does nothing except get rid of dots. (It's just << "#.A#" ->NFS STR-> >>) Might make a useful command, similar to oldRPL R->I, though working on non-integers. Even better, it works on lists of numbers all at once.

It's been in the bug tracker for a while, to write a new command that marks any number as approximate, or as exact. Not too useful so it's sitting there.

(08-26-2017 04:40 AM)The Shadow Wrote:  By the way, some commands mark uncertainty when they shouldn't. For example, sqrt(4) gives 2. rather than 2

Numeric algorithms will always introduce rounding, even if they produce the exact result with 32 digits. The square root algorithm is no exception. The result might be the integer number 2, but after many iterations of numbers that weren't exact, so it can't be exact. In order to verify the result is exact, you'd have to take the rounded result, square it and compare with the original exact argument. This would slow things down a lot, just for an extra trailing dot...\

(08-26-2017 04:40 AM)The Shadow Wrote:  EDIT: Just found some oddities with complex arithmetic. If at 32 digits of precision you do:

(16 16) (1 1) 5 ^ /

You get (-4 1.25E-31) instead of (-4 0).

Complex numbers require a lot of operations to produce a result, so there can (and will) be rounding errors. In this case, (1 1) 5 ^ is done through LN 5 * EXP, which is causing the rounding errors. I could detect the special case of an integer exponent and do it with multiplications to mitigate the effect, but in the general case this is normal and expected.
I could also increase the precision by 8 digits to reduce the rounding errors on the final results, paying a price in speed.

[Claudio L.]

(08-26-2017 07:01 PM)The Shadow Wrote:  Now that you mention it, those aren't really all that necessary except for pi and the line feed and possibly the comma. You're right, the brackets and so on have no real reason to hold alpha for.

Talking about Pi, I think alpha-hold+shift could/should be the greek letters and symbols.

[The Shadow]

(08-26-2017 07:04 PM)Claudio L. Wrote:  

(08-26-2017 07:01 PM)The Shadow Wrote:  Now that you mention it, those aren't really all that necessary except for pi and the line feed and possibly the comma. You're right, the brackets and so on have no real reason to hold alpha for.

Talking about Pi, I think alpha-hold+shift could/should be the greek letters and symbols.

That's not a bad idea. (Just out of curiosity, why 'pi0' instead of just 'pi'?)

By the way, I accidentally discovered that half-space or whatever it's called on AH-SPC. It's very nice-looking when used as a digit separator in SETLOCALE! Much better than the comma, and takes up less space too. It has tripped me up a couple times when programming, though. It looks close enough to a space that it's not an obvious issue.

Quote:Numeric algorithms will always introduce rounding, even if they produce the exact result with 32 digits. The square root algorithm is no exception. The result might be the integer number 2, but after many iterations of numbers that weren't exact, so it can't be exact. In order to verify the result is exact, you'd have to take the rounded result, square it and compare with the original exact argument. This would slow things down a lot, just for an extra trailing dot...\

I see your point. Never mind.

Quote:Complex numbers require a lot of operations to produce a result, so there can (and will) be rounding errors. In this case, (1 1) 5 ^ is done through LN 5 * EXP, which is causing the rounding errors. I could detect the special case of an integer exponent and do it with multiplications to mitigate the effect, but in the general case this is normal and expected.
I could also increase the precision by 8 digits to reduce the rounding errors on the final results, paying a price in speed.

Again, I see your point. I may just do all my Gaussian integer stuff in matrix representations, I sometimes have to do that anyway.

Quote:This shouldn't happen, no matter what you do (nothing you do justifies data corruption). Can you try to find a sequence of events to reproduce it? I need your guidance to track it. The INVALID_COMMAND is the string the decompiler uses when an object is not valid. Most likely one of the commands you used wrote to an object that was moved during a garbage collection. If the command is not coded properly, the object moves, and some other object takes its place and will get corrupted. What commands were you trying when this happened?
To fix these things, turn the calc off and back on. The calculator runs MEMFIX every time it wakes up, scanning for invalid objects. If an invalid object is found, it will be replaced with the BINT zero (you'll lose the damaged object, but it won't be invalid anymore).

Thinking back, both times this happened, the variable was being used in the test part of a WHILE loop.

Like, in the more recent case I had:

WHILE n p / <blah> REPEAT <blah> 'n' STO <blah> END

The n in both places got changed to INVALID_COMMAND. When I tried to retype in the n's, I got an 'Invalid word' error.

Both times, it happened in a long program, multiple flow structures deep, and only on the innermost structure. (Which, as mentioned, was a WHILE in both cases.)

[The Shadow]

Forgot this bit:

Quote:I'd have to get 16 digits out of the 18, which is 2 groups of 8 (reals internally store 8 digit groups). That means a random 2000 digit number would need 120 calls to this RNG. It better be fast!

It IS fast. Though I don't know what the speed would translate to on the calculator. My RPL implementation is slow, but of course as a native command it would be much faster.

For most everyday purposes (simulating dice, whatever) 16 digits is fine. The only applications I can think of in which one would need 32+ digit random numbers, you absolutely want the RNG to introduce as little bias as possible. This one does that, in spades. And while other highly-unbiased algorithms are available, I doubt there's anything that will produce 2000 digits at one go.

You might actually want two different commands, one low-quality and one high-quality. Called perhaps RAND and RANDPREC.

[Vtile]

Hmm... I see a dilemma here... How precise is randomness. :/

[Claudio L.]

(08-26-2017 07:29 PM)The Shadow Wrote:  That's not a bad idea. (Just out of curiosity, why 'pi0' instead of just 'pi'?)

It's not a constant per se, but a command that returns the value of pi to twice the system precision. Only needed temporarily to verify how the trig functions were working (when angles are passed close to pi, you need pi to twice the system precision to get any significant digits from the difference), so I didn't want to call it the same as an actual pi constant. Then I never removed it, though it might eventually be removed when the actual constant pi is implemented. Consider it a temporary patch.

(08-26-2017 07:29 PM)The Shadow Wrote:  By the way, I accidentally discovered that half-space or whatever it's called on AH-SPC. It's very nice-looking when used as a digit separator in SETLOCALE! Much better than the comma, and takes up less space too. It has tripped me up a couple times when programming, though. It looks close enough to a space that it's not an obvious issue.

It's the default separator of the fractional part, but looks good anywhere. I read somewhere that ISO recommends using the Unicode Thin Space character as digit grouping, so I did. And yes, it's bound to Alpha-hold-space, we need to put that in the wiki.

(08-26-2017 07:29 PM)The Shadow Wrote:  Thinking back, both times this happened, the variable was being used in the test part of a WHILE loop.

Like, in the more recent case I had:

WHILE n p / <blah> REPEAT <blah> 'n' STO <blah> END

The n in both places got changed to INVALID_COMMAND. When I tried to retype in the n's, I got an 'Invalid word' error.

Both times, it happened in a long program, multiple flow structures deep, and only on the innermost structure. (Which, as mentioned, was a WHILE in both cases.)

Ahhhh, then there's no memory corruption anywhere! That's good news, no bugs in any command, but the compiler itself. It seems something in your code threw off the decompiler's variable name tracking.
Long story: The first time you do LSTO or create locals with -> << >> the compiler creates the variables and track their names. Next time it finds the name, it doesn't include the name but a special GETLAM/PUTLAM style opcode. The decompiler needs to do the reverse: It analyzes the binary code and tracks the names of the variables you create, so when it finds those weird PUTLAM/GETLAM opcodes it decompiles them as the actual variable name. It seems you hit a case where the decompiler is losing track of the names somehow, so it couldn't decompile the GETLAM into anything, leaving an INVALID_COMMAND in its place.

** EDIT: Could you find a specific piece of your code that fails consistently? a test case would help a lot **

The "Invalid word" is a different story, it's only generated when compiling symbolics, when no library wanted to accept a certain token. For example, you get it when you introduce an invalid character in a symbolic, like double quotes:

'MYFUNC"' --> Invalid word

In this case, MYFUNC" is an invalid name for a variable, or function, or command, hence called a "word", I'm all ears if you can think of a better error message than "invalid word".
Sometimes if you don't close the ticks on a variable name and starts compiling your code as a symbolic you might also get an invalid word error.

[The Shadow]

(08-26-2017 11:44 PM)Claudio L. Wrote:  Ahhhh, then there's no memory corruption anywhere! That's good news, no bugs in any command, but the compiler itself.

Well, it's good that there's no bugs in the OS, but it's bad because I don't know what to do differently. In both cases I had to painfully rewrite the thing from scratch, though the judicious use of cutting and pasting did help.

[Claudio L.]

(08-26-2017 11:59 PM)The Shadow Wrote:  Well, it's good that there's no bugs in the OS, but it's bad because I don't know what to do differently. In both cases I had to painfully rewrite the thing from scratch, though the judicious use of cutting and pasting did help.

There's a bug in the decompiler tracking of variables, I just need to find it.
What to do? Easy: Always save your sources as a string in a variable (with comments). Then just compile with STR-> and that way you don't ever lose the sources if the decompiler fails.

[The Shadow]

Something weird is going on with EVAL. And after a lot of experimentation after getting very strange answers, I think I've figured out what.

If I type in '2/3', then press EVAL, I get something that *looks* exactly like '2/3', but if I do OBJ-> on it, rather than:

2
3
2
/

I get:

2
INV(3)
2
*

I would have expected the latter to look like:

'2*INV(3)', not '2/3'. So my FXND algorithm is getting very confused.

[The Shadow]

Something else. I generated a long list of Gaussian primes in the first quadrant - in other words, a long list of both complex numbers and integers. (The list is 1514 entries long.)

139 is in the list. I did 139 POS, and got back 0. Then I did (139 0) POS, and got a result. When I did GET on the result, I got 139. I've tried this with smaller lists, but so far I haven't gotten the same behavior.

[Claudio L.]

(08-29-2017 04:25 AM)The Shadow Wrote:  Something weird is going on with EVAL. And after a lot of experimentation after getting very strange answers, I think I've figured out what.

If I type in '2/3', then press EVAL, I get something that *looks* exactly like '2/3', but if I do OBJ-> on it, rather than:

2
3
2
/

I get:

2
INV(3)
2
*

I would have expected the latter to look like:

'2*INV(3)', not '2/3'. So my FXND algorithm is getting very confused.

It's not weird at all once you understand some basic concepts of expressions in newRPL:

1. Expressions remain as entered by the user until operated upon.
2. Operations in expressions produce new expressions always in canonical form (more on this below).
   

Canonical form enforces a few internal rules that make manipulating an expression a lot simpler:

• Negative numbers don't exist. They are replaced with unary minus operator on a positive number.
• Subtractions don't exist. They are replaced with additions of negative terms.
• Divisions don't exist. They are replaced with multiplication of INVerted factors.
• Addition trees are flattened: The + operator can take multiple arguments.
• Multiplication trees are flattened: The * operator can take multiple arguments.
  

And a few other rules that I can't recall.
So when you type: '2+3+1'
you generate an object with a tree structure (+ (+ 2 3) 1) because for the parser + is a binary operator, but for the (future) CAS to operate properly, it flattens the tree to (+ 2 3 1), which makes it simpler to simplify terms, search for expressions, cancel out terms, etc.
Your fraction '2/3' is converted after EVAL to '2*INV(3)' per the rules above. What you get from OBJ-> is one level expansion of the main operator. Perhaps OBJ-> should convert to canonical form always before expanding, so you get consistent results every time.

Other than that, everything works as designed.

[Claudio L.]

(08-29-2017 05:47 AM)The Shadow Wrote:  Something else. I generated a long list of Gaussian primes in the first quadrant - in other words, a long list of both complex numbers and integers. (The list is 1514 entries long.)

139 is in the list. I did 139 POS, and got back 0. Then I did (139 0) POS, and got a result. When I did GET on the result, I got 139. I've tried this with smaller lists, but so far I haven't gotten the same behavior.

I'll take a deep look, size of the list doesn't really matter. Any chance you could copy/paste the list with the simulator (or SDSTO to an SD card with your 50g) and send it to me via PM or email? I want to see what's in that list that made the == operator fail, although the same function that compares (139 0) to 139 does the comparison of the real part of 139.

[The Shadow]

(08-29-2017 03:18 PM)Claudio L. Wrote:  you generate an object with a tree structure (+ (+ 2 3) 1) because for the parser + is a binary operator, but for the (future) CAS to operate properly, it flattens the tree to (+ 2 3 1), which makes it simpler to simplify terms, search for expressions, cancel out terms, etc.
Your fraction '2/3' is converted after EVAL to '2*INV(3)' per the rules above. What you get from OBJ-> is one level expansion of the main operator. Perhaps OBJ-> should convert to canonical form always before expanding, so you get consistent results every time.

Other than that, everything works as designed.

Interesting. I'll have to rethink how I do things a bit, then. ->NUM followed by ->Q should do the trick.

Will we get access to the full tree, like the old ->LST and ->ALG commands? Very useful at times.

I'll recreate the list and email it to you.

[The Shadow]

Some suggested extended types:

Vectors as opposed to matrices.

Vectors and matrices that contain symbolics as opposed to those don't.

Vectors and matrices that contain complex numbers as opposed to those that only contain reals.

Algebraics that contain undefined variables (and thus can't be ->NUM'd) as opposed to those that don't. (Alternatively or in addition, algebraics that contain identifiers as opposed to those that only contain numbers and operators.)

Also, I just discovered that '0' counts as non-zero for purposes of IF, but not for NOT. I don't know if this is intended, but it's definitely unexpected.

To clarify, IF '0' THEN 1 ELSE 0 END and IF '0' NOT THEN 1 ELSE 0 END both return 1. You can fix it by doing EVAL or ->NUM on the '0' first, of course, but it's still unexpected. I'm guessing that any algebraic counts as "something"?

Likewise, IF 'X' THEN 1 ELSE 0 END returns 1, while oldRPL gives an undefined name error.

EDIT: I also just discovered that trying to find the inverse of the matrix:

[ [ 0 1 ] [ -1 0 ] ]

gives an 'Undefined result' error. If I then try to invert

[ [ '0' '1' ] [ '-1' '0' ] ]

I get

[ [ '0/0' '0/0'] [ '1' '0' ] ]

which may give some insight as to why this is happening.

EDIT:

I've emailed you two programs in which SAME, or possibly IFERR, is acting very oddly.

[Claudio L.]

(08-30-2017 05:42 PM)The Shadow Wrote:  Some suggested extended types:

Vectors as opposed to matrices.

Vectors and matrices that contain symbolics as opposed to those don't.

Vectors and matrices that contain complex numbers as opposed to those that only contain reals.

Algebraics that contain undefined variables (and thus can't be ->NUM'd) as opposed to those that don't. (Alternatively or in addition, algebraics that contain identifiers as opposed to those that only contain numbers and operators.)

That's in the queue. The matrix library is incomplete and in much need of a revision. It was written before TYPE was implemented, now it's time to revisit and provide all that information.
Soon I'll attack that library and make a big push in terms of new commands implemented and better algorithms. I'll keep these suggestions in mind. Some might be time consuming, like the last one. There's no way to know if it can be ->NUM'ed unless you actually do it, but there will be a way to identify if a matrix is numeric, symbolic but with all numeric constants (in other words, numeric with fractions or non-rationals).

(08-30-2017 05:42 PM)The Shadow Wrote:  Also, I just discovered that '0' counts as non-zero for purposes of IF, but not for NOT. I don't know if this is intended, but it's definitely unexpected.

To clarify, IF '0' THEN 1 ELSE 0 END and IF '0' NOT THEN 1 ELSE 0 END both return 1. You can fix it by doing EVAL or ->NUM on the '0' first, of course, but it's still unexpected. I'm guessing that any algebraic counts as "something"?

Likewise, IF 'X' THEN 1 ELSE 0 END returns 1, while oldRPL gives an undefined name error.

In newRPL the definition is that FALSE is real or complex zero, everything else is true. The IF command only checks if the argument is false, because in the modular architecture, the idea was that IF should be independent of the object types, so adding new objects makes them automatically work well with IF. So far it worked well, but on symbolics there's an incompatibility, since classic RPL evaluates the symbolic (and newRPL should too).
I'm still debating how to resolve this issue. I think to keep IF independent of object types, it should ->NUM its argument before checking if it's false, but this might make it non-atomic as ->NUM might execute RPL code (specifically on lists it does MAP to each element). There's also the possibility of adding an overloadable unary operator ISFALSE to all object types. This way it each object can define its own behavior. For example an empty list or a list containing all zeros could be false too, but that's something the list library should define.
Anyway, it will be fixed eventually, but it needs some more thought.

(08-30-2017 05:42 PM)The Shadow Wrote:  EDIT: I also just discovered that trying to find the inverse of the matrix:

[ [ 0 1 ] [ -1 0 ] ]

gives an 'Undefined result' error. If I then try to invert

[ [ '0' '1' ] [ '-1' '0' ] ]

I get

[ [ '0/0' '0/0'] [ '1' '0' ] ]

which may give some insight as to why this is happening.

EDIT:

I've emailed you two programs in which SAME, or possibly IFERR, is acting very oddly.

The algorithm for matrix inversion doesn't do partial pivoting yet, so any zeros in the diagonal will make it fail. Like I said, the matrix library is next in line for an overhaul.

I'll look at those programs., and the list you sent me. Will keep you updated.

[Claudio L.]

(08-30-2017 05:42 PM)The Shadow Wrote:  I've emailed you two programs in which SAME, or possibly IFERR, is acting very oddly.

Found it! The SAME cross operator between a real and an integer had a really dumb bug. It was doing: A=argument from level 2, B=argument from level 1, if B is the SAME as C then return 1, else 0. Of course, C was a real register that could contain anything from previous calculations.
In other words, SAME would result in 1 or 0 more or less randomly, depending on previous operations (but only when one argument was real and the other was an integer, it worked fine between 2 integers or between 2 reals). With SAME fixed, not only your program will work fine, but also the problem of POS not finding an item should be instantly resolved (POS uses the SAME operator to compare objects).

I'll see if I can push an updated ROM tomorrow, as this is quite critical.

[The Shadow]

Had an idea to make user menus even more flexible and (in some common use cases) more compact. Unfortunately, I'm guessing it would be non-trivial to implement. But I'll throw it out there in case I'm wrong.

Basically, recycle the key code format. Since the actual key will vary, you leave it off. So "..L" would mean "long press on the menu key".

"..P" would mean "do this when pressed, but don't include other VAR menu stuff unless explicitly mentioned with keycodes"

So, let's say you have a program called PROG. { PROG } would, as now, produce a key that works like a VAR menu key: Left shift does STO and right shift does RCL.

But frequently, all you want is a menu which runs PROG, without worrying about the VAR baggage. Currently you have to express that as { << PROG >> << >> << >> } and you don't get the neat flexibility in different modes.

With this new version, you'd just say { "..P" { PROG } }. You don't have to add key combinations that don't do anything, and you keep the mode flexibility.

Unfortunately, as I said, I suspect this would be a lot of work, so I don't have any expectations.

[Claudio L.]

(09-01-2017 05:52 PM)The Shadow Wrote:  Had an idea to make user menus even more flexible and (in some common use cases) more compact. Unfortunately, I'm guessing it would be non-trivial to implement. But I'll throw it out there in case I'm wrong.

Basically, recycle the key code format. Since the actual key will vary, you leave it off. So "..L" would mean "long press on the menu key".

"..P" would mean "do this when pressed, but don't include other VAR menu stuff unless explicitly mentioned with keycodes"

So, let's say you have a program called PROG. { PROG } would, as now, produce a key that works like a VAR menu key: Left shift does STO and right shift does RCL.

But frequently, all you want is a menu which runs PROG, without worrying about the VAR baggage. Currently you have to express that as { << PROG >> << >> << >> } and you don't get the neat flexibility in different modes.

With this new version, you'd just say { "..P" { PROG } }. You don't have to add key combinations that don't do anything, and you keep the mode flexibility.

Unfortunately, as I said, I suspect this would be a lot of work, so I don't have any expectations.

Sorry, I'm not following. If you want PROG so that it doesn't act as a variable just use { { "PROG" << PROG >> } }, no need to add empty programs, or even keycodes.
The long press is reserved for the OS to show help or preview the variable, the alpha plane is reserved for the alpha keys, etc. So there are only a few shift planes that are allowed for a menu key, and those are available to be redefined in the menu definition. Other shift planes and modes are not part of the menu system, but can be redefined with a custom key assignment via ASNKEY.
With a command that tells you which item is visible in the menu from a program, you could write code that "moves" with the menu using ASNKEY.

[pier4r]

little idea: what about inviting The shadow to the wiki, since at the moment I am not so active while he is?