Threaded Mode | Linear Mode

JoJo1973 · 06-23-2019, 11:57 AM

(06-23-2019 02:38 AM)Claudio L. Wrote: The file looks OK. Did you use newRPL Desktop to flash it or SD card? Looks like your calc jumped out of place. Try flashing again, and put new batteries just in case. When flashing with newRPL desktop it doesn't check the battery voltage. Perhaps it wasn't sufficient to write to flash successfully.
Also, try the download again, it's not the first time a download gets corrupted.

I don't think it's a batteries issue because I've been able to reflash it back to build 1255 successfully with the same batteries. Anyway I tried with a fresh pack and got the same result. All my attempts were made using SD card.

I never attempted before using the PC version but now decided a try: using build 1255 the progress bar advanced to 20-25% then it became all black, returned white with an animation and the calc reset to a blank screen.

I panicked a bit because afterwards the calc seemed unresponsive to reset and I feared having bricked it. However I've been able to resurrect it and reflashed it to 1255.

To ensure that there was no problem with my cable I used the PC simulator to flash build 1255 on itself: this time the progress bar grew up to 50% (I assume now it's calibrated to 2MB RAM) and everything went OK - I'd exclude a cable problem.

I've redownloaded the file and checked it against the copy I've used. The files are identical and their checksum is:

newrplfw.bin (application/octet-stream) - 1045356 bytes
SHA-256 3c112f108f5e512d0f93237762777c01a41918e4d013a0e2984ee28136e272f6

Claudio L. · 06-23-2019, 05:51 PM

(06-23-2019 11:57 AM)JoJo1973 Wrote:
(06-23-2019 02:38 AM)Claudio L. Wrote: The file looks OK. Did you use newRPL Desktop to flash it or SD card? Looks like your calc jumped out of place. Try flashing again, and put new batteries just in case. When flashing with newRPL desktop it doesn't check the battery voltage. Perhaps it wasn't sufficient to write to flash successfully.
Also, try the download again, it's not the first time a download gets corrupted.

I don't think it's a batteries issue because I've been able to reflash it back to build 1255 successfully with the same batteries. Anyway I tried with a fresh pack and got the same result. All my attempts were made using SD card.

I never attempted before using the PC version but now decided a try: using build 1255 the progress bar advanced to 20-25% then it became all black, returned white with an animation and the calc reset to a blank screen.

I panicked a bit because afterwards the calc seemed unresponsive to reset and I feared having bricked it. However I've been able to resurrect it and reflashed it to 1255.

To ensure that there was no problem with my cable I used the PC simulator to flash build 1255 on itself: this time the progress bar grew up to 50% (I assume now it's calibrated to 2MB RAM) and everything went OK - I'd exclude a cable problem.

I've redownloaded the file and checked it against the copy I've used. The files are identical and their checksum is:

newrplfw.bin (application/octet-stream) - 1045356 bytes
SHA-256 3c112f108f5e512d0f93237762777c01a41918e4d013a0e2984ee28136e272f6

I downloaded the file on a different PC from the same link in this forum, flashed using SD card and it works just fine here. Try clearing your RAM completely (On-A-F and all 3 shifts+clear).

Claudio L. · 06-23-2019, 05:57 PM

(06-23-2019 11:57 AM)JoJo1973 Wrote: I panicked a bit because afterwards the calc seemed unresponsive to reset and I feared having bricked it. However I've been able to resurrect it and reflashed it to 1255.

No need to panic, I have safeguards in place that if the program requests to erase or write to the bootloader it will reset the calc immediately. My calculator gave its life to make me realize the safeguards were needed. Now is next to impossible to brick a calc during firmware upgrade.
The blank screen is normal since you don't have a firmware, just press + and - during reset to bring up the menu.

JoJo1973 · 06-23-2019, 07:53 PM

(06-23-2019 05:57 PM)Claudio L. Wrote:
(06-23-2019 11:57 AM)JoJo1973 Wrote: I panicked a bit because afterwards the calc seemed unresponsive to reset and I feared having bricked it. However I've been able to resurrect it and reflashed it to 1255.

My calculator gave its life to make me realize the safeguards were needed. Now is next to impossible to brick a calc during firmware upgrade.

Fallen in the line of duty :,(

The memory clean did the trick! Build 1261 installed successfully!

Claudio L. · 06-24-2019, 01:34 PM

(06-23-2019 07:53 PM)JoJo1973 Wrote: The memory clean did the trick! Build 1261 installed successfully!

Great to hear! Some memory corruption must've happened while you were bug-hunting and crashing the calc on purpose to reproduce them. However, your service is crucial to the project so please don't stop doing it :-), just backup often to different files to make sure you don't lose anything.

JoJo1973 · 06-24-2019, 10:51 PM

(06-24-2019 01:34 PM)Claudio L. Wrote: Great to hear! Some memory corruption must've happened while you were bug-hunting and crashing the calc on purpose to reproduce them. However, your service is crucial to the project so please don't stop doing it :-), just backup often to different files to make sure you don't lose anything.

Oh, I've found some interesting ways to crash the calc Wink

)) , in fact there's one I want to verify with the new build... I'll keep you informed! Wink

BTW I probably found the culprit of the memory corruption: a formerly good dirobject that now hangs the calc when I enter it; curiously I can retrieve its contents if I use a program to enter it and RCL a variable in it.

I tried to PGDIR it, but the calc errors with an Out of Memory error. It's not a problem since I can wipe out the memory and have plenty of backups: one of them will be surely ok.

My question is if is it possible in the future to add a command like UserRPL PINIT that returns the memory to a sane state so that a corrupted object can be safely purged (recovery of the corrupted object wouldn't be important)?

Claudio L. · 06-25-2019, 03:06 AM

(06-24-2019 10:51 PM)JoJo1973 Wrote:
(06-24-2019 01:34 PM)Claudio L. Wrote: Great to hear! Some memory corruption must've happened while you were bug-hunting and crashing the calc on purpose to reproduce them. However, your service is crucial to the project so please don't stop doing it :-), just backup often to different files to make sure you don't lose anything.

Oh, I've found some interesting ways to crash the calc )) , in fact there's one I want to verify with the new build... I'll keep you informed!

BTW I probably found the culprit of the memory corruption: a formerly good dirobject that now hangs the calc when I enter it; curiously I can retrieve its contents if I use a program to enter it and RCL a variable in it.

I tried to PGDIR it, but the calc errors with an Out of Memory error. It's not a problem since I can wipe out the memory and have plenty of backups: one of them will be surely ok.

My question is if is it possible in the future to add a command like UserRPL PINIT that returns the memory to a sane state so that a corrupted object can be safely purged (recovery of the corrupted object wouldn't be important)?

There's no need for a specific command. Every time you turn the calculator on, or restore a backup, the entire memory is scanned and every single object is checked for validity. Of course, validity depends on the object type: a real number for example, as long as it has consistent headers is deemed valid, if some digits got corrupted it can't be detected (I'd have to waste a lot of memory in storing a CRC for every object). Any invalid object is purged (but this is extremely rare, hardly ever happened). Directory entries are scanned and checked for validity names as well. Invalid names with valid objects are given an 8-letter name.
As I find more creative ways to corruption memory, I also find ways to improve the sanity checks.
Now I don't want this to sound like newRPL is unstable, memory corruption is extremely rare, and mostly due to testing unfinished, buggy code that 90% of the time never gets to the final user. In your case, you are trying to crash it on purpose to help the project, and your success means you might corrupt your memory every now and then.
Because backups do sanity checks, most of the time recovering from backup fixes any issues, even if the backup was done after the corruption happened.

I'll see if I can harden detection of malformed directory structures during the sanity checks, can you describe in more detail which things crash your bad directory and which ones work OK? This way I can at least try to guess what could be wrong and add a sanity check for it.
For example, you mentioned entering the directory crashes. I need to know exactly what happens in all of the following cases:
Entering from the menu?
Typing the name then doing EVAL?
Executing the unquoted name within a program?
Going into the bad directory then doing UPDIR?
In the parent directory, type the name quoted, then do RCL, do we get a DirObject? If so, go to HOME, type another name and do STO. This does a deep copy of the entire subtree, does it crash? If not, is the copy having the same issues as the original?
Going into the directory then doing PATH, does it crash?

To avoid annoying our readers, perhaps you should create the issue in the bug tracker and add all this info.

JoJo1973 · 06-25-2019, 11:21 PM

(06-25-2019 03:06 AM)Claudio L. Wrote:
(06-24-2019 10:51 PM)JoJo1973 Wrote: Oh, I've found some interesting ways to crash the calc )) , in fact there's one I want to verify with the new build... I'll keep you informed!

BTW I probably found the culprit of the memory corruption: a formerly good dirobject that now hangs the calc when I enter it; curiously I can retrieve its contents if I use a program to enter it and RCL a variable in it.

I tried to PGDIR it, but the calc errors with an Out of Memory error. It's not a problem since I can wipe out the memory and have plenty of backups: one of them will be surely ok.

My question is if is it possible in the future to add a command like UserRPL PINIT that returns the memory to a sane state so that a corrupted object can be safely purged (recovery of the corrupted object wouldn't be important)?

There's no need for a specific command. Every time you turn the calculator on, or restore a backup, the entire memory is scanned and every single object is checked for validity. Of course, validity depends on the object type: a real number for example, as long as it has consistent headers is deemed valid, if some digits got corrupted it can't be detected (I'd have to waste a lot of memory in storing a CRC for every object). Any invalid object is purged (but this is extremely rare, hardly ever happened). Directory entries are scanned and checked for validity names as well. Invalid names with valid objects are given an 8-letter name.
As I find more creative ways to corruption memory, I also find ways to improve the sanity checks.
Now I don't want this to sound like newRPL is unstable, memory corruption is extremely rare, and mostly due to testing unfinished, buggy code that 90% of the time never gets to the final user. In your case, you are trying to crash it on purpose to help the project, and your success means you might corrupt your memory every now and then.
Because backups do sanity checks, most of the time recovering from backup fixes any issues, even if the backup was done after the corruption happened.

I'll see if I can harden detection of malformed directory structures during the sanity checks, can you describe in more detail which things crash your bad directory and which ones work OK? This way I can at least try to guess what could be wrong and add a sanity check for it.
For example, you mentioned entering the directory crashes. I need to know exactly what happens in all of the following cases:
Entering from the menu?
Typing the name then doing EVAL?
Executing the unquoted name within a program?
Going into the bad directory then doing UPDIR?
In the parent directory, type the name quoted, then do RCL, do we get a DirObject? If so, go to HOME, type another name and do STO. This does a deep copy of the entire subtree, does it crash? If not, is the copy having the same issues as the original?
Going into the directory then doing PATH, does it crash?

To avoid annoying our readers, perhaps you should create the issue in the bug tracker and add all this info.

Ok, I'm preparing a bug report answering all the question provided. I can confirm that newRPL is pretty robust: it's likely that the object stayed corrupted for a while, and overall the system was unaffected and worked very reliably.

Moreover, as you remember I failed to flash build 1261 but succeeded to reflash build 1255 after each failed attempt: every time I recovered programs and settings without any loss (I can safely assume that it was the pre-existence of the corrupted dir the cause of the failures) - quite an accomplishment!

Claudio L. · 06-26-2019, 01:58 PM

(06-25-2019 11:21 PM)JoJo1973 Wrote: Moreover, as you remember I failed to flash build 1261 but succeeded to reflash build 1255 after each failed attempt: every time I recovered programs and settings without any loss (I can safely assume that it was the pre-existence of the corrupted dir the cause of the failures) - quite an accomplishment!

Thanks!, I put a lot of effort to make sure TTRM messages are a thing of the past, there's no flashy graphics advertising that effort, no section on the wiki, it's completely transparent to the final user with no delay or bragging messages (like "we just scanned your computer and found no security threats!", which I find most annoying). There's just a sad file called 'sanity.c' buried in the middle of the source code. I'm glad somebody noticed that after a bad crash, it used to be normal to lose it all, and now is not.

Claudio L. · 07-26-2019, 04:30 PM

All unofficial ROMs were updated to build 1280.

Many bugs were ironed out in this release, I truly appreciate the work and dedication of our forum friend JoJo1973 for submitting report after report with all the inconsistencies and bugs he could find. We need more of that to make newRPL a really polished product.

This release saw operations with angles completely reworked so that evaluating symbolic expressions would produce the correct, expected result even when the user does some questionable operations on angles.

I updated the wiki explaining how operations on angles work so it's clear what this modification does. It's not foolproof and it may cause unexpected behavior especially in RPL sequences. If you use symbolic expressions, everything should work as expected.

The Shadow · 07-27-2019, 03:07 PM

Looks great, Claudio! Could you tell us more about the new PACKDIR objects and editable directory trees?

Claudio L. · 07-28-2019, 02:05 AM

(07-27-2019 03:07 PM)The Shadow Wrote: Looks great, Claudio! Could you tell us more about the new PACKDIR objects and editable directory trees?

Basically you provide a name of a directory or a path (double list, same format used for STO) and do PACKDIR.
You'll see for yourself, but basically the syntax is:

Code:

DIRECTORY

'Var1' OBJECT

'Var2' OBJECT

...

ENDDIR

Of course, OBJECT can also be a subdirectory (same DIRECTORY ... ENDDIR syntax)
This packed directory can be transferred as a single object on the USB or stored in an SD card for quick backup, could also be used to distribute software of you don't want to create a library, etc.

You can simply move somewhere else in your path, and store this under a new name to get the entire directory tree restored. You cannot store it to a temporary variable since directories are not supported there. I could expand all vars it into a local environment, but no subdirectories would be allowed, so it's not really a tree, and it's not really faster than using a global directory. For now I decided to only allow packed directories to be expanded into global variables.
It's nothing new or exciting, just that it was nice to be able to edit entire directories as source code on a PC rather than individual variables, and newRPL lacked that feature because of the internal format of the directories.

Regarding memory usage: PACKDIR needs to include a copy of all objects and their names so if you try to pack HOME for example you need to have more than 50% of free memory available. Unpacking on the other hand does not make another copy, the objects in the directory are still pointing to within the packed directory object. This is good because it requires very little extra memory, bad because even if you purge all variables but one, the entire packed directory object cannot be purged by the garbage collector.

Claudio L. · 07-29-2019, 02:15 PM

(07-26-2019 04:30 PM)Claudio L. Wrote: All unofficial ROMs were updated to build 1280.

Many bugs were ironed out in this release, I truly appreciate the work and dedication of our forum friend JoJo1973 for submitting report after report with all the inconsistencies and bugs he could find. We need more of that to make newRPL a really polished product.

This release saw operations with angles completely reworked so that evaluating symbolic expressions would produce the correct, expected result even when the user does some questionable operations on angles.

I updated the wiki explaining how operations on angles work so it's clear what this modification does. It's not foolproof and it may cause unexpected behavior especially in RPL sequences. If you use symbolic expressions, everything should work as expected.

All ROMs were updated to build 1282. If you downloaded 1280 in the last couple of days, you are urged to update to 1282 due to a regression bug in 1280 that affect expressions with a multiplication by 1.

Once more, thank you to our bug hunter in chief JoJo1973 for caching this one quickly.

JoJo1973 · 07-29-2019, 10:49 PM

(07-28-2019 02:05 AM)Claudio L. Wrote: Basically you provide a name of a directory or a path (double list, same format used for STO) and do PACKDIR.

Wait, wait, is there a specific syntax for paths? Can you provide an example? What do you mean by double list? Which commands support them, apart from those mentioned above?

Claudio L. · (This post was last modified: 07-30-2019 02:19 PM by Claudio L..)

(07-29-2019 10:49 PM)JoJo1973 Wrote:
(07-28-2019 02:05 AM)Claudio L. Wrote: Basically you provide a name of a directory or a path (double list, same format used for STO) and do PACKDIR.

Wait, wait, is there a specific syntax for paths? Can you provide an example? What do you mean by double list? Which commands support them, apart from those mentioned above?

Yes, there's a special syntax for paths. The reason being that I wanted RCL and STO to be able to do list processing just like every other command, for consistency.
The problem is, because RCL and STO "normally" would interpret a list as a path instead of as a list of names, list processing was not possible.
So I tweaked the syntax a little bit:
Paths must be expressed as list of lists: { { HOME 'MyDIR' } }, so if STO receives a list of lists, it will consider it a path and will store a single object in level 2 into the single variable given by the path. Otherwise it will consider it a list of names and will take a list in level 2 with the same number of elements, performing a multiple STO into the current directory.
There's a couple of limitations to this syntax:
* The first name of the list of names can't be a path, others can be paths (always use a list of list for paths, even within the list of names)
* If you provide a single object and a list of names, the same object will be stored in all the names (useful to zero-out many variables at once when you start your program). However, that object cannot be a list otherwise each element will be stored in a separate variable. This means you can't store the same list to multiple variables.

I know it departs from the standard, but it's useful to have list processing on STO and RCL.

EDIT: Since we are already touching the topic, there's another enhancement in paths: UPDIR can be used anywhere within a path just like you would use "..":

Code:

"./mydir/myvar" ---> { { 'mydir' 'myvar' } }

"../otherdir/othervar" ---> { { UPDIR 'otherdir' 'othervar' } }

"/maindir/../mydir/myvar" ---> { { HOME 'maindir' UPDIR 'mydir' 'myvar } }

Claudio L. · 07-30-2019, 02:35 PM

I'd like to open a discussion here:

What would be a good syntax for ASSUME?

The way it used to be, the old CAS would store information in the current directory. Now that information is included within the symbolic expression (see the wiki on symbolic expressions). Now we need a "simple" syntax so the user can make all occurrences of a variable within an expression have the same assumptions (the 3-digit numeric code).
Something along the lines of:

Code:

'X^2+4' 6 'X' ASSUME

Which would make 'X' in the expression known to be a matrix (numeric code 6). But... is there a way to do it without forcing the user to remember those cryptic codes?
Right now the user can type the equation with the codes in it as subscripts, but I'd like to be able to change it after the fact.
I know, this would be ideal for the equation writer: just select the variable and a couple of check boxes appear where you select the type and be done, but that is "in planning stages" still.

What could be a good syntax instead of numeric code?

JoJo1973 · (This post was last modified: 07-30-2019 03:15 PM by JoJo1973.)

(07-30-2019 02:35 PM)Claudio L. Wrote: I'd like to open a discussion here:

What would be a good syntax for ASSUME?

The way it used to be, the old CAS would store information in the current directory. Now that information is included within the symbolic expression (see the wiki on symbolic expressions). Now we need a "simple" syntax so the user can make all occurrences of a variable within an expression have the same assumptions (the 3-digit numeric code).
Something along the lines of:

Code:

'X^2+4' 6 'X' ASSUME

Which would make 'X' in the expression known to be a matrix (numeric code 6). But... is there a way to do it without forcing the user to remember those cryptic codes?
Right now the user can type the equation with the codes in it as subscripts, but I'd like to be able to change it after the fact.
I know, this would be ideal for the equation writer: just select the variable and a couple of check boxes appear where you select the type and be done, but that is "in planning stages" still.

What could be a good syntax instead of numeric code?

First, a list of mnemonics, short and easy to memorize, e.g.

Type: "R." Real and finite; "C∞" complex and infinite, etc.
Sign: "≥0", "≠0", etc.
Parity: "OD", "EV"

plus standard codes for "Any" and "Unknown" ("*" and "?", perhaps?)

Then you can collect these mnemonics in a list or a string with separators.

Empty list or string clears assumptions.

Of course there should be a command that given a variable lists its assumptions.

Claudio L. · 07-30-2019, 05:14 PM

(07-30-2019 03:12 PM)JoJo1973 Wrote:
(07-30-2019 02:35 PM)Claudio L. Wrote: I'd like to open a discussion here:

What would be a good syntax for ASSUME?

The way it used to be, the old CAS would store information in the current directory. Now that information is included within the symbolic expression (see the wiki on symbolic expressions). Now we need a "simple" syntax so the user can make all occurrences of a variable within an expression have the same assumptions (the 3-digit numeric code).
Something along the lines of:

Code:

'X^2+4' 6 'X' ASSUME

Which would make 'X' in the expression known to be a matrix (numeric code 6). But... is there a way to do it without forcing the user to remember those cryptic codes?
Right now the user can type the equation with the codes in it as subscripts, but I'd like to be able to change it after the fact.
I know, this would be ideal for the equation writer: just select the variable and a couple of check boxes appear where you select the type and be done, but that is "in planning stages" still.

What could be a good syntax instead of numeric code?

First, a list of mnemonics, short and easy to memorize, e.g.

Type: "R." Real and finite; "C∞" complex and infinite, etc.
Sign: "≥0", "≠0", etc.
Parity: "OD", "EV"

plus standard codes for "Any" and "Unknown" ("*" and "?", perhaps?)

Then you can collect these mnemonics in a list or a string with separators.

Empty list or string clears assumptions.

Of course there should be a command that given a variable lists its assumptions.

I like it, let's say we use a string because it's easier to compare.
The string could be:
1 character for type (and parity): "R"=real, "I","O","E"=Integer/Odd/Even, "C"=complex, "M"=matrix, "?"=unknown (internal), "*"= No information given about this variable, it would remove all assumptions on the variable.
1 (optional) character for infinity bit: "∞" is present only if the variable can be infinite or NaN, otherwise is known to be finite.
2 (optional) characters related to sign: "≥0", "≠0", etc.

Noticed I merged the parity with the type information to make it more readable: A positive integer would be:
"I>0" (being an integer already implies it's a real number, so no need for a separate letter).
"C∞≠0" (a complex number that can be infinite but is known not to be zero)
"*" (clear all assumptions about this variable)
"E∞≥0" (a positive even integer, counting from zero all the way up to infinity)

I think this will work!
Commands needed:
ASSUME to apply this to a variable in an expression
ASSGET to extract the assumptions made for a certain variable within an expression.
ASSCOMB to get the combined assumptions for a whole expression
ASSCMP to compare if a given assumption meets certain other assumptions (using the fancy text string).

For example you could do 'X^2' ASSCOMB (combine assumptions) and should return "R≥0" if X has default assumptions (when complex mode is disabled, default assumption is that variables are real and finite).
Then you could also check that result against some restrictions:
"R≥0" "R" ASSCMP would return true, since a real >=0 meets the minimum requirement of being a real.

The Shadow · 07-30-2019, 06:06 PM

That system looks good to me! Very intuitive. I like it better than the subscripts, in fact.

I'm not quite clear on what ASSCOMB does? Is it the ASSUME string for the result of the expression?

JoJo1973 · (This post was last modified: 07-30-2019 07:20 PM by JoJo1973.)

(07-30-2019 05:14 PM)Claudio L. Wrote: I like it, let's say we use a string because it's easier to compare.
The string could be:
1 character for type (and parity): "R"=real, "I","O","E"=Integer/Odd/Even, "C"=complex, "M"=matrix, "?"=unknown (internal), "*"= No information given about this variable, it would remove all assumptions on the variable.
1 (optional) character for infinity bit: "∞" is present only if the variable can be infinite or NaN, otherwise is known to be finite.
2 (optional) characters related to sign: "≥0", "≠0", etc.

Noticed I merged the parity with the type information to make it more readable: A positive integer would be:
"I>0" (being an integer already implies it's a real number, so no need for a separate letter).
"C∞≠0" (a complex number that can be infinite but is known not to be zero)
"*" (clear all assumptions about this variable)
"E∞≥0" (a positive even integer, counting from zero all the way up to infinity)

I think this will work!
Commands needed:
ASSUME to apply this to a variable in an expression
ASSGET to extract the assumptions made for a certain variable within an expression.
ASSCOMB to get the combined assumptions for a whole expression
ASSCMP to compare if a given assumption meets certain other assumptions (using the fancy text string).

For example you could do 'X^2' ASSCOMB (combine assumptions) and should return "R≥0" if X has default assumptions (when complex mode is disabled, default assumption is that variables are real and finite).
Then you could also check that result against some restrictions:
"R≥0" "R" ASSCMP would return true, since a real >=0 meets the minimum requirement of being a real.

I'm doing devil's advocate:

How would ASSCOMB work for more complicated expressions? For example if X is "R>=0" and Y is "O∞<0" then 'Y^2+SIN(X)' ASSCOMB gives what? I suppose there's an algorithmic way to solve this, but this would mean that internally each built-in function must provide its domain to the rules engine, not to mention the fact that some operations between sets are possible only in a certain order (matrix divided by scalar it's possible, scalar divided by matrix isn't)

Exciting feature, anyway!

P.S.: Using "Z" rather than "I" would be classy! Wink