[HP48] Memory Management

[Bruno]

Hi RPL's gurus

Maybe this question has been answered on comp.sys.hp48, but I have not found or I use bad tokens.
So, I'll ask you

Is there any RPL objects that can be expanded directly in User Object Area ?
(ie. Without making first a copy in Temporary Object Area before going back to USEROB)

Here is why :

I wan't to handle about 52 Arrays of Binary Integers without dispatching them in many(52) variables.
(The full set of datas may be very large)
If possible, I want to store them in only one composite or 'sized' object (Library data, HXS or CHR string)
and be able to extract an array, edit/expand it, then replace it in the composite/sized object
with minimum memory requirement (MEM<BYTES(composite/sized)), and of course using built-in ROM routines

Thanks for your help
And sorry for my poor English

Bruno

[David Hayden]

I don't think you can do this in userRPL. In systemRPL it may be possible to modify the data in place. However, if the size of the array changes, then it's definitely not possible.

RPL lets you have multiple pointers to objects, but then what happens if you change the object? You don't want to change the data that something else (like a stack level, or a local variable) is pointing to. RPL handles this in (at least) 3 ways: first, most operations create a new object, so objects are mostly "read only." Second, sometimes if will check to see if an object has multiple references to it and modify it in-place if not. Third, the words that do modify an object in-place usually start with "!" and are called "bang type" operators.

[Claudio L.]

As David said, it's not directly possible, but you could hack your way around it.
A directory is not too different from a list internally, so you could (in theory) do the following:
Let's say you have variables A B and C in a directory:

In memory this looks like (more or less, there's other things especially at the beginning of a directory):
<DIR> A <value-of-A> B <value-of-B> C <value-of-C> <END>

If you want to destructively stretch A, you could create a variable with a string, libdata, or whatever, with enough extra space as needed (let's call it EXTRA), and you store it in the directory. Now you get:

<DIR> EXTRA <string> A <value-of-A> B <value-of-B> C <value-of-C> <END>

Now you could surgically change the size field of the string/libdata object to "engulf" the name A and the content of A. After that, the directory becomes:

<DIR> EXTRA <string + name A + value-of-A> B <value-of-B> C <value-of-C> <END>

Now it's up to you to properly stretch <value-of-A> within the allocated space.

If the variable to stretch is not the first of the directory, you'd have to sort the variables, which implies moving the large data. So this technique will not give you any performance advantage, since you'll end up moving the data anyway, but it does not need to make a second copy of the data to patch it, allowing your routine to work with less memory.

Do I need to tell you that if you miscalculate the size to "engulf" even by one nibble you'll corrupt your directories and likely loose all your precious and large data? You probably guessed that already.

Claudio

[Bruno]

Thanks for your very accurate answers

David: Yes, 'builtin ROM routines' mean SysRPL/ASM entry points, not User RPL.
Thank to explained the 3 ways to handle objects, this is much clear for me now !

Claudio: Wow, this is very interesting, indeed this is the 'easiest' way to expand an object in UOA, but as you say, also risky.
I want to measure the trade off, so I've studied a little bit the RRP structure this weekend, and if I'm right,
I can summarize your method in these few steps:

1. Move destination object 'A' at the end (in memory) of the current directory.
2. Create EXTRA variable of size: (object to insert in 'A') - (2.5 bytes of 'A' offset field) - ('EXTRA' Name Length) - (2*1 bytes of EXTRA name length fields)
   • Q -> Why the HP48 need the name length field to be present before AND after the name field ???
   • This is also true for Backup and Library objects. Curious...
     
3. (3 or 4?)- Now 'EXTRA' is after 'A' in memory, and is the last variable of the current directory.
   • So, we need to update the last RRP's offset field (after 'EXTRA' variable) to skip 'EXTRA' and point directly to 'A'
   • Thus, 'A' became again the last variable of the current directory. 'EXTRA' becoming a ghost variable.
     
4. (4 or 3?)- Expand 'A' to "engluf" 'EXTRA' and insert the new object in 'A'
   

That's All ?

Bruno

[Paul Dale]

Doesn't a RPL array contain the header address for its elements? If I am remembering this correctly (and it has been a very long time), it should be possible to build an array of binary integers.

Failing that, use a string and write some machine code access words.

A final idea, create a pile of unnamed local variables and access them by index.


- Pauli

[Claudio L.]

(10-27-2014 10:12 AM)Bruno Wrote:  I can summarize your method in these few steps:

1. Move destination object 'A' at the end (in memory) of the current directory.
   

Make that the "beginning" of the directory. Remember that UserOB is at the end of the memory and grows down. STO adds objects at the beginning (that's why my example added EXTRA as the first variable in the dir) to move less memory, and so should you.

(10-27-2014 10:12 AM)Bruno Wrote:  

• Create EXTRA variable of size: (object to insert in 'A') - (2.5 bytes of 'A' offset field) - ('EXTRA' Name Length) - (2*1 bytes of EXTRA name length fields)
  • Q -> Why the HP48 need the name length field to be present before AND after the name field ???
  • This is also true for Backup and Library objects. Curious...
    

Because sometimes it needs to skip the object backwards. If you know the object and want to know the name of the variable, you simply take the pointer of the object, read the length of the name which is stored 2 nibbles before, and from then you can skip the text and get a pointer to the prolog of the name, without scanning through the whole directory.

(10-27-2014 10:12 AM)Bruno Wrote:  

(3 or 4?)- Now 'EXTRA' is after 'A' in memory, and is the last variable of the current directory.

• So, we need to update the last RRP's offset field (after 'EXTRA' variable) to skip 'EXTRA' and point directly to 'A'
• Thus, 'A' became again the last variable of the current directory. 'EXTRA' becoming a ghost variable.
  

(4 or 3?)- Expand 'A' to "engluf" 'EXTRA' and insert the new object in 'A'

That's All ?

That's all. It's not difficult at all. The trick is to sort the variables properly to assure that EXTRA is after A (it seems to be your preferred way, in my example I put it before for performance reasons, so you only move memory within the object, otherwise you have to sort the variables first, moving everything in userOB which can be slow).
You can use sysRPL or even the userRPL commands to STO and sort the variables, then RCL 'A' to the stack (to get a pointer) and 'EXTRA' as well. With a little ASM, just do a SKIPOB over EXTRA to get a pointer to the end of it, add 5 to the A pointer to get the size field address, and the difference between the pointers should be stored right there as the new size. The main thing is don't mess up the new size calculation, other than that it's relatively safe as long as you sort the variables properly.
If you don't, your 'A' will "engulf" any other variables that are between 'A' and 'EXTRA'. And if EXTRA is before A, your size will become negative... causing a system crash, and almost surely a memory clear. But you can detect this condition in you ASM and refuse to do it.

Claudio

[Bruno]

Ok, thank you Claudio.
I didn't know that UserOB grows down, this revert many things lol
and I better understand your example. (EXTRA before A)

Thank Paul for trying to help me, but I think you misunderstood my question.

[Han]

You'd be much better off using a list of arrays saved inside USEROB. Since EXTRA is going to have to be a pre-determined size, you may as well reserve a fixed amount of space in USEROB but leave it in the form of a list of arrays.

Code:


DOLIST (5 nibbles)
  DOARRAY (5 nibbles) + <len> + <prologue> + <dimcount> + <dimsize> + data_nibbles
  DOARRAY ...
  DOARRAY ...
  DOHSTR ...
SEMI (5 nibbles)

You can modify each array's length, prologue, dimcount, and dimsize easily through assembly language. All those parameters are easily calculated. A call to MOVEUP or MOVEDN will easily allow moving large chunks of data within the memory space between DOLIST and SEMI. The last object should be a string which could simply be the "extra" that contains useless data.

With this technique, you don't have to bother with variables. The only limitation would be the total RAM that you pre-allocate -- which would be the case for Claudio's idea as well. As long as you pre-allocate sufficient memory, you can also have as few or as many arrays as you want. Just make sure there is enough "extra" space in the string at the end of the list. Lastly, you can access the arrays easily with regular RPL via GET. Nothing should be copied to TEMPOB since it can be referenced via a pointer into USEROB. When you store your new data, you simply contract/expand the string to adjust for the change in the modified array, move the existing arrays (if necessary) via MOVEUP/MOVEDN and then insert the modified array (all done via assembly, though).

If you don't want users to accidentally mess with the contents via UserRPL commands, you can optionally embed everything inside a library object. However, access to the contents will require a custom program to extract the data from the library object (you can simply push a pointer to the embedded array).

[Bruno]

Thank Han, yet another good idea! It increase my chances

When I read your post, I thought to Backup Objects,
they have the advantage of being sizable, and include a CRC that is a useful thing in my project!
However, they have the disadvantage of being hardly displaceable ...
Another drawback of this scheme is the waste of memory, more or less,
although I can almost predict the data volume.

So, I try other ways by decompilling some built-in commands to understand 'How they do that?'.
And I found the BANG command SMATREDIM (# 37E2Dh SX) used by xRDM and xSigma+.

SMATREDIM does a CKSREF that return FALSE against objects in USEROB,
then, don't make any FREEREFS or TOTEMPOB, and resize object directly in USEROB !

However, I've not tryed with array of Array of bint lol , maybe tomorrow