newRPL: Alpha demo 0.9 released [UPDATED 2017-10-25]

[The Shadow]

After installing the latest ROM, one of my programs had a >= sign changed to INVALID_COMMAND. Not sure if that's a fluke or what.

[Claudio L.]

(10-31-2017 04:38 AM)The Shadow Wrote:  After installing the latest ROM, one of my programs had a >= sign changed to INVALID_COMMAND. Not sure if that's a fluke or what.

I forgot to mention that in the rush! There was a bad #define for that operator, it wasn't being classified properly as a binary test operator so I had to fix its definition, which changed the compiled opcode.
This means any program compiled previously would show invalid command, you need to recompile from source or manually replace the INVALID_COMMAND with the proper symbol on the editor.
Sorry for the inconvenience, I fixed so many things I forgot to list that one.

As usual, nothing escapes your sight...

[pier4r]

Is "the shadow" already involved in the newrpl wiki? If not, would it be possible to invite him?

[Claudio L.]

(10-31-2017 05:56 PM)pier4r Wrote:  Is "the shadow" already involved in the newrpl wiki? If not, would it be possible to invite him?

Everyone is invited to collaborate in the wiki, just need to ask me for credentials.

But our friend is having fun coding, testing and more important, bug hunting. That's his preferred way of helping the project and is doing a terrific job at it, I don't want (or need) that to change.

I'd love to have more people writing the wiki, but not at the expense of losing our #1 bug hunter, so let's take the opportunity to invite writers willing to write, testers willing to test, advocacy helpers willing to talk about the project and make video tutorials, etc. But each person has to do whatever is more fun to them without peer pressure (or "pier" pressure, see what I did there?).

[pier4r]

(10-31-2017 08:01 PM)Claudio L. Wrote:  (or "pier" pressure, see what I did there?).

Me making pressure? Nah. I give orders!

Yes I agree, one does what he finds fun. I was just assuming that the shadow is using newRPL so much to have also the ability to spot missing points or errata in the wiki easily.

[Gilles59]

Hi !

I get a bad crash with this kind of sequence :
v0.9 Alpha , Build 938

Code:

 << 2 DUP2 MOD NOT << KILL >> IFT >>

try 8 and the program for example.
EDIT : I try again and it works nows :/ But i'm sure there was a problem.

I noticed in a more complex example that with a HALT instead of KILL, the stacks seems inconsistant.

This is a program that give the first divisor of an integer or the number himself if prime (idea is to be fast no small) :

Code:


« 2   DUP2 MOD NOT « KILL » IFT
  1 + DUP2 MOD NOT « KILL » IFT
  2 + DUP2 MOD NOT « KILL » IFT
  2 + DUP2 MOD NOT « KILL » IFT
  DO
     4 + DUP2 MOD NOT « KILL » IFT
     2 + DUP2 MOD NOT « KILL » IFT
     4 + DUP2 MOD NOT « KILL » IFT
     2 + DUP2 MOD NOT « KILL » IFT
     4 + DUP2 MOD NOT « KILL » IFT
     6 + DUP2 MOD NOT « KILL » IFT
     2 + DUP2 MOD NOT « KILL » IFT
     6 + DUP2 MOD NOT « KILL » IFT
  UNTIL  DUP2 SQ <  END
  DROP
»

Select a nomber to test (ex : 999999937 )
After the KILL the stack is not correct state (if im not wrong ?) . Same thing with HALT. But if you generate an error, like 0 INV instead of KILL or HALT the stak is fine.

Note that with few modifications NewRPL return 999999937 as Prime in ~0.6s

I also notice another problem. I have difficulty to do a "Lshif CONT" sequence after a HALT. I dont think it is a physical keyboard problem but perhaps ? Did you also notice this ?

[Claudio L.]

(11-01-2017 04:48 PM)Gilles59 Wrote:  Hi !

I get a bad crash with this kind of sequence :
v0.9 Alpha , Build 938

Code:

 << 2 DUP2 MOD NOT << KILL >> IFT >>

try 8 and the program for example.
EDIT : I try again and it works nows :/ But i'm sure there was a problem.

I could not reproduce a crash either on my calc (same build) or the simulator. I even debugged step by step with the simulator and couldn't find anything wrong. Doesn't mean it's bug free, but at least nothing obvious. When you say "bad crash": what was it? data abort?

(11-01-2017 04:48 PM)Gilles59 Wrote:  I noticed in a more complex example that with a HALT instead of KILL, the stacks seems inconsistant.

...

After the KILL the stack is not correct state (if im not wrong ?) . Same thing with HALT. But if you generate an error, like 0 INV instead of KILL or HALT the stak is fine.

There's a difference between newRPL and classic RPL in everything that has to do with halting and continuing code. KILL does not terminate the current program, but a previously halted one. This difference is because of the more advanced breakpoints in newRPL, and the possibility to use RPL handlers for hardware interrupts like timers. The interrupt handler should be able to kill the main program without killing the handler.
The inconsistencies you noticed is probably because you expected the code to stop after KILL, but it continued running and changed the stack.
But in reality, you are using KILL as EXIT. I just implemented an EXIT command that ends the innermost loop (works like break), or innermost program. Will come out in the next update for you to test.

(11-01-2017 04:48 PM)Gilles59 Wrote:  Note that with few modifications NewRPL return 999999937 as Prime in ~0.6s

Actually, your code could be much faster using NEXTPRIME to find the next divisor to test. Start from 2 and test the division with all the next prime factors until sqrt(N).

(11-01-2017 04:48 PM)Gilles59 Wrote:  I also notice another problem. I have difficulty to do a "Lshif CONT" sequence after a HALT. I dont think it is a physical keyboard problem but perhaps ? Did you also notice this ?

I just verified that CONT is not bound to LS-On (or any key). No wonder you had "difficulty" :-). I'll add it for the next update.

[Claudio L.]

(11-01-2017 04:48 PM)Gilles59 Wrote:  This is a program that give the first divisor of an integer or the number himself if prime (idea is to be fast no small) :

Here's a similarly fast without using KILL, and it returns all factors:

Code:


«
2 WHILE DUP2 SQ ≥ REPEAT
     DUP2 IF IDIV2 THEN 
              DROP NEXTPRIME
              ELSE
              ROT DROP OVER
              END
   END
DROP
»

[The Shadow]

Claudio L. Wrote:I just implemented an EXIT command that ends the innermost loop (works like break), or innermost program. Will come out in the next update for you to test.

Well! This will definitely require some rethinking of algorithms!

Quote:As usual, nothing escapes your sight...

Who knows what bugs lurk in the heart of code?

The Shadow knows!

[Claudio L.]

ROMs were updated.

However, I'm not sure what changed in this release, but something moved in ROM memory (I upgraded the system, I think it upgraded my arm-none-eabi-gcc package to a newer gcc version) and my calc flattened my entire directory tree. Nothing was lost, the calc runs a memory recovery procedure every time it's turned on, which detected whatever was invalid and repaired it automatically, but everything was recovered in HOME.
I strongly recommend those upgrading to use SDARCHIVE before flashing (I wish I had followed my own recommendation...), then if things get messed up SDRESTORE should bring everything back correctly. This is recommended always, since SDARCHIVE/SDRESTORE uses a file format that is independent of the ROM version, so if things move within the ROM it doesn't matter.

I'll comment on what's new in this ROM tomorrow, just wanted to give a heads-up in case somebody notices the files were updated and tries to flash them.

[Gilles59]

Hi, here a problem, both with Build 916 (émulator) and Build 948 (HP50g)

No matter what is this program for (problem 141 of the Turing challenge : http://turing.nymphomath.ch/ ):

Code:

«
  0 
  10 1000 FOR 'n' n 
    DO
      →STR UTF8→ -48 SWAP + ΠLIST 
    UNTIL DUP 10 < END
    6 == + 
  NEXT
»

This program get "out of memory" error both on the calc or emulateur with more loops (change 1000 with 10000 or more). The limit is highter with the emulator.

EDIT : the same program with a MEM DROP after ΠLIST works without problem...

[Claudio L.]

(11-04-2017 04:31 PM)Gilles59 Wrote:  This program get "out of memory" error both on the calc or emulateur with more loops (change 1000 with 10000 or more). The limit is highter with the emulator.

EDIT : the same program with a MEM DROP after ΠLIST works without problem...

Thanks for the report. This one was a tough bug. UTF8-> was to blame.
This is a corner case I've never thought of, and I'll have to review all the commands to make sure it isn't happening anywhere else.
The bug itself (for those interested, others feel free to skip the technical details) is quite simple but very hard to catch: newRPL has pointers that are automatically updated during GC, so the code doesn't have to worry if a memory block moves. In this case, there were 2 pointers: one to the character being processed by UTF8, and one to the end of the string, used only to compare and end the loop.
Turns out that the end-of-string pointer actually points to the byte after the end of the string, but that's just outside the block of memory allocated for the string. When the garbage collector moved the block, that end pointer didn't move with the string, since it was pointing into the next object, which was moved to a different location.
With the pointers now separated in memory, the UTF8-> loop was processing more than 59000 characters, and running out of memory trying to create that huge list.

It's a simple fix, the end pointer needs to point to the last used byte, to make sure it's within the same object and gets moved together with the object.
But now I wonder in how many other places this is happening in the code... it'll take me a few days to review all libraries.

Thanks a lot for the report, this one was an eye opener.

[Claudio L.]

(11-04-2017 05:17 AM)Claudio L. Wrote:  I'll comment on what's new in this ROM tomorrow, just wanted to give a heads-up in case somebody notices the files were updated and tries to flash them.

Few things to play with in this ROM:

• STO and RCL support for absolute paths using list of lists {{ }} for the paths.
• New EXIT command needs thorough testing (see below for explanation)
• IFT and IFTE were updated to use the ISTRUE operator, need thorough testing.
• New system flags to allow keyboard handler chaining.
  

The EXIT command:

• Will exit the current program, returning to the caller (not completely stopping RPL like KILL on the 50g)
• Inside a loop, it exits the loop
• Not affected by IF/CASE/IFERR, so it exits the program or loop in which the IF/CASE statement is in.
  

The new keyboard handler flags (I know who's going to like this
Flag -46 (FL_DODEFAULTKEY), is set to zero by the system right before calling a custom key handler. If the handler sets this flag to 1, the system will execute the default key handler immediately after.
Flag -45 (FL_NEXTCUSTOMKEY), is set to zero by the system right before calling a custom key handler. If the handler sets this flag to 1, the system will continue to look for another handler for the same key and run it.

An example:
<< 1 -46 SF >> "+" 0 ASNKEY
This handler puts a 1 on the stack, then sets flag -46 to indicate the default handler should be executed. Put a number on the stack, and pressing + will do << 1 >> << + >>, so it will be increased.

Now define another handler to chain the previous one:
<< 2 / -45 SF >> "+" 0 ASNKEY
Putting a number 4 on the stack and pressing + will do << 2 / >> << 1 >> << + >>, resulting in 3. The first handler will chain to the second one, which will chain to the default key.
To clean up this you'll need to remove both handlers with "+" DELKEY "+" DELKEY, since handlers will be erased one by one.
I think this makes for a huge improvement in key handler flexibility.

EDIT: In case you mess up your key definitions badly, you can disable custom keys through the menu (System/Keys) and programmatically setting flag -4.

[The Shadow]

Quote:The new keyboard handler flags (I know who's going to like this Smile

Ooh! Ooh! Is it me?

Those flags are amazing. I'm kind of so overloaded with ideas to use them, nothing is actually coming coherently to mind yet.

[Claudio L.]

I updated the ROMs yet again. We are now up to build 950, this one incorporated the fix for UTF8-> and SREPL, same bug that Gilles reported. Those were the only commands affected.

[The Shadow]

Any new news, Claudio? I haven't had much time for coding the last couple weeks, so no new bug reports.

[Claudio L.]

(11-21-2017 03:30 AM)The Shadow Wrote:  Any new news, Claudio? I haven't had much time for coding the last couple weeks, so no new bug reports.

I know, I went quiet for a while, here's what I've been up to:
I decided to try and write a USB driver, which of course didn't work at all. This triggered an obsessive bug hunt that lasted a couple of weeks until I finally got it working this past weekend. Debugging USB is a painful and slow process. Each trial and error involves rebuilding the firmware, copying to SD card, flashing and testing. It also forces me to work only whenever I'm in a desktop setup, so opportunities to work on it are few.
My 50g with newRPL is finally recognized well under Windows and Linux, I'm currently testing data transfer from the PC to the calc, once that works well I'm going to work on data from the calc to the PC. The driver I wrote is HID so it doesn't need any custom drivers on any operating system.
USB comms will work different from the stock firmware. Upon connecting, the calculator continues to work normally, but will receive data in the background. When the calculator is idle in the stack, any data received will be (compiled if needed and) executed instantly, so you'll be able to control your calc remotely. A flag will allow you to disable remote execution if not wanted. It's something like the old Kermit Server mode, where you could send commands to the calc, except the calc is available to the user while receiving data.
Programs will be able to disable this automatic service and wait for data themselves, so coders will have full control to write their own application specific data transfers.
As far as commands, in my head there will be a USBON, USBOFF (in case you want to disable USB altogether, for example if you only want to get power from USB, but on second thought since it's all transparent there's no real need to turn it off), USBSEND/USBRECV to transfer arbitrary blocks of data, USBARCHIVE/USBRESTORE taking the place of SDARCHIVE/SDRESTORE on the 39gs/40gs.

The PC simulator will be retrofitted with the ability to link to a calculator and send/receive commands or objects straight to the stack, sync directories, etc.

In my wishlist is also using this USB mechanism to do firmware updates, independently of the bootloader.

Lots of work to do, and until it all works together, there isn't much I can release.

[Francois Lanciault]

(11-21-2017 04:06 AM)Claudio L. Wrote:  

(11-21-2017 03:30 AM)The Shadow Wrote:  Any new news, Claudio? I haven't had much time for coding the last couple weeks, so no new bug reports.

My 50g with newRPL is finally recognized well under Windows and Linux, I'm currently testing data transfer from the PC to the calc, once that works well I'm going to work on data from the calc to the PC. The driver I wrote is HID so it doesn't need any custom drivers on any operating system.
USB comms will work different from the stock firmware. Upon connecting, the calculator continues to work normally, but will receive data in the background. When the calculator is idle in the stack, any data received will be (compiled if needed and) executed instantly, so you'll be able to control your calc remotely. A flag will allow you to disable remote execution if not wanted. It's something like the old Kermit Server mode, where you could send commands to the calc, except the calc is available to the user while receiving data.
Programs will be able to disable this automatic service and wait for data themselves, so coders will have full control to write their own application specific data transfers.

For me this is one of the most exciting feature to date!!

Can't wait to try it!

François

[pier4r]

That's amazing work Claudio! I wish I could have 1/1000 of your perseverance! So actually I can develop on the PC and let the calculator do the intensive work easier (I admit I was worried of pushing back and forth the sd card. Little mechanical springs can just break)

[Claudio L.]

(11-21-2017 07:17 PM)pier4r Wrote:  That's amazing work Claudio! I wish I could have 1/1000 of your perseverance! So actually I can develop on the PC and let the calculator do the intensive work easier (I admit I was worried of pushing back and forth the sd card. Little mechanical springs can just break)

Your fears are unfounded. My calculator is from 2006, I had it since short after the official launch of the 50g. I used the SD card slot extensively, still in perfect shape. The slot doesn't break, but the SD card contacts may wear off and you'll need a new card. Happened to me with one card. I had to use sand paper to "revive" the scratched contact, transferrred my data out and got a new card.
But the slot still works.