How much memory may a HPPL program use on a G2?
|
09-12-2023, 10:20 AM
Post: #41
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
Bonjour
Désolé mais c'est ce site que j'ai utilisé. Je crois pouvoir dire que j'ai résolu mes problèmes de vitesses ! J'ai réécris complétement ma routine de conversion 'avec accent' <-> 'sans accents' Je vous fournirai celle-ci lorsque vous aurais testé la vôtre et nous pourrons comparer. Voici les recherches que j'ai lancé : ??e??r??? --> 386 mots trouvés en 21s environ. ??e??r??s --> 130 mots trouvé en 21 s aussi. ??e?????s --> 4417 mots trouvé en 29 s Avant j'étais à plus de 10 mn. La variable N9 contient 50 183 mots et comme la première lettre est inconnues tous les mots doivent être testés. Hello Sorry, but I used this site. I think I can say I've solved my speed problems! I've completely rewritten my 'with accents' <-> 'without accents' conversion routine. I'll provide you with this when you've tested yours and we can compare. Here are the searches I've run: ??e??r??? --> 386 words found in about 21s. ??e??r??s --> 130 words found in 21s too. ??e?????s --> 4417 words found in 29 s Before, I was more than 10 minutes behind. Variable N9 contains 50,183 words and as the first letter is unknown all the words must be tested. Sorry for my english |
|||
09-12-2023, 12:35 PM
(This post was last modified: 09-12-2023 01:00 PM by komame.)
Post: #42
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
(09-12-2023 10:20 AM)Tyann Wrote: I think I can say I've solved my speed problems! For 9-letter words I managed to reduce the time below 12 seconds (G2) for words starting with '?' and much less for all queries where the beginning of the word is not '?'. For example: ??e??r??? --> 386 words found in 9 - 11s (G2) and 23s (G1) ??e??r??s --> 130 words found in 7.4 - 8.7s (G2) and 23s (G1) ??e?????s --> 1031 words found in 8.6 - 12s (G2) and 25s (G1) With the first letter filled in: p?e??r??? --> 93 words forund in 2.6s (G2) and 7.8s (G1) sc???r??? --> 9 words found in 1.2s (G2) and 3.8s (G1) The more initial letters are filled in, the faster the algorithm will work. For the query with the mask '??e?????s', your algorithm found 4417 words, while mine found 1031 words. We have a difference, so one of us may have an error in the algorithm. Best regards, Piotr Kowalewski |
|||
09-12-2023, 03:06 PM
Post: #43
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
Je viens de refaire 3 fois la dernière requête, je trouve effectivement 1030 mots
en 24 s. de 1) "aberrants" à 1030 "zieutâtes Le nombre de mots trouvés s'affiche pendant un cout instant sur mon application et j'avais du mal voir. Pour être tout à fait honnête, j'ai supprimé la conversion en 'MAJUSCULE' qui finalement ne servait à rien pour ne garder que la conversion des accents. Vôtre algorithme semble donc meilleur que le mien. Pour les requêtes avec la première lettre connue, j'envisage de revoir l'organisation des mots, je pense que chaque liste sera divisée en 26 sous listes, une par lettre (a..z) et ainsi le recherche se fera exclusivement sur une sous liste. J'ai testé un petit programme de conversion sur 'N5' et cela fonctionne bien. I've just repeated the last query 3 times and found 1030 words in 24 seconds. from 1) "aberrants" to 1030 "zieutâtes The number of words found is displayed for a short while on my application and I had trouble seeing. To be perfectly honest, I've removed the 'UPPERCASE' conversion, which was useless in the end, and only kept the accent conversion. So your algorithm seems better than mine. I'm thinking of dividing each list into 26 sub-lists, one per letter (a..z), so that the search will be carried out exclusively on one sub-list. I've tested a small conversion program on 'N5' and it works well. Sorry for my english |
|||
09-12-2023, 04:17 PM
Post: #44
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
(09-12-2023 03:06 PM)Tyann Wrote: I've just repeated the last query 3 times and found 1030 wordsI also double-checked, and mine finds 1031 words, from 'ABERRANTS' to 'ZIEUTÂTES' as well. If you're returning the result in the form of a list on the HP Prime stack, please save it to the global list 'L1', then perform a synchronization to your PC using the Connectivity Kit. Copy the contents of 'L1' to the clipboard (you can select the entire list by clicking the header of 'L1' or using the keyboard shortcut ctrl-A), and then paste it into a text file. Please attach it here as a file. I will compare the results to see if there is any excess data in my results or if something is missing from your results. (09-12-2023 03:06 PM)Tyann Wrote: So your algorithm seems better than mine.I'm using text strings in this algorithm instead of PPL lists. I'm not sure if it makes a difference in terms of speed. When I find some time, I'll make the modification and use built-in PPL lists to compare performance. Thank you anyway. You'll see it soon (09-12-2023 03:06 PM)Tyann Wrote: I'm thinking of dividing each list into 26 sub-lists, one per letter (a..z), so that the search will be carried out exclusively on one sub-list.I've also thought about it, but it seems to me that in the case of my algorithm, it won't help much or maybe not at all. I would like to clarify how exact search should work, that is, when all the letters are provided. If I enter "RÉESSAYES," should it find only "RÉESSAYES," or should it also include "RÉESSAYÉS"? On the other hand, if I enter "REESSAYES" (without an accent on "É"), should it show anything at all, or nothing? Piotr Kowalewski |
|||
09-12-2023, 05:10 PM
Post: #45
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
Since the speed of different algorithms is compared here, another forum thread might be useful if you are not familiar with it.
Prime G2, 15C CE |
|||
09-12-2023, 05:49 PM
Post: #46
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
Demain je reprends le travail et je vais avoir moins de temps pour faire joujou, bien dommage
En ce qui concerne la réorganisation, j'ai fais N6 et a????? renvoie les 377 mots en <2 s. Pour ce qui est de la recherche pour mots croisés, j'applique la conversion au mot qui est entré donc les accents seront ignorés, les accents servent pour la vérification orthographique qui est la première option de l'application. Une troisième option est de montrer tous les anagrammes possibles (mots existant) et en fait l'idée ensuite est proposer ces fonctionnalités en tant que fonctions de l'application pour des programmes extérieurs, puisque l'on peut maintenant savoir si une application est installée. Je vous remercie également pour ces échanges qui ont été très agréables, motivant et enrichissant. Je vais tacher de vous envoyer la liste des mots de 9 lettres trouvés, mais soyez patient svp. Tomorrow I go back to work and I'll have less time to play, which is a shame As far as the reorganisation is concerned, I've done N6 and a????? returns the 377 words in <2 s. As for the crossword search, I apply the conversion to the word entered The accents are used for spell checking, which is the first option in the application. A third option is to show all the possible anagrams (existing words) and in fact the next idea is to offer these features as functions of the application for external programs, since we can now know if an application is installed. I'd also like to thank you for these discussions, which have been very pleasant, motivating and enriching. I'll try to send you the list of 9-letter words found, but please be patient. Sorry for my english |
|||
09-12-2023, 06:27 PM
Post: #47
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
(09-12-2023 05:10 PM)chromos Wrote: Since the speed of different algorithms is compared here, another forum thread might be useful if you are not familiar with it. Yes, I'm familiar with it - I read it a long time ago. Unfortunately, those results are worthless today. These tests were conducted in 2014, so it was firmware 6975 or older. Since then, there have been 15 firmware versions for the HP Prime, and believe me, if you were to conduct these tests on each of them, the results would be different for each. Some would favor lists, while others would favor text strings, and that has changed over time. All those measurements, ceased to be relevant on the day the next firmware version was released, and that was 8 years ago... After that, the results kept changing, and unfortunately, when averaged out, with each new firmware release, they got worse and worse. The current firmware 14730 is the slowest in history, as I mentioned in this thread (a very long text, but in the 1st and 6th post, you can see charts for very similar measurements I performed in the context of different firmware releases): https://www.hpmuseum.org/forum/thread-20403.html Perhaps this 'variability' issue only applies to the PPL environment, as Python seems to be more resilient to various firmware versions when it comes to performance changes. However, some differences can still be observed (only four firmware versions with Python have been released so far, but that has allowed me to measure certain things). I really hope this negative trend will change, and someone will pay attention to performance in the next firmware release. Best regards, Piotr Kowalewski |
|||
09-12-2023, 07:23 PM
(This post was last modified: 09-12-2023 07:28 PM by komame.)
Post: #48
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
(09-12-2023 05:49 PM)Tyann Wrote: Tomorrow I go back to work and I'll have less time to play, which is a shame (09-12-2023 05:49 PM)Tyann Wrote: As far as the reorganisation is concerned, I've done N6 and a????? returns the 377 words in <2 s.Hmm, mine returns 1101 words (from "ABAQUE" to "ÂPRETÉ") in 0.42 - 0.46s (G2), generally always less than 0.5s. The difference in the number of words is surprising. Perhaps you didn't put the full 6-letter word dictionary from this website into your application? (09-12-2023 05:49 PM)Tyann Wrote: The accents are used for spell checking, which is the first option in the application.As for the first option, since you use accents for spell checking, does it mean that if you enter 'REESSAYES,' nothing will appear in the result because such a word doesn't exist? To get results for the first option, do you need to enter the word exactly with accents, right? (please, just confirm or deny) (09-12-2023 05:49 PM)Tyann Wrote: I'd also like to thank you for these discussions, which have been very pleasant, motivating and enriching.I also thank you very much. Thanks to this conversation, I was able to take a break from the standard business stuff I code every day. Here, I felt great, squeezing as much as I could out of this device, pushing certain optimization boundaries. Great training and a wonderful conversation. Again, thank you, Tyann! P.S. I will show my source code once we clarify point 1 (spell-checking) and I'm confident that I've implemented it correctly. Hopefully, tomorrow. Best regards, Piotr Kowalewski |
|||
09-12-2023, 10:47 PM
(This post was last modified: 09-12-2023 10:51 PM by jte.)
Post: #49
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
There’s certainly many algorithms for subset checking, string searching, etc. but one thing this reminded me of was my dad’s program (that he did for a comp. sci. assignment in the ’60s, on punched cards — he told me about it several times…) for handling pizza toppings. His idea was to encode each pizza topping option as a prime so that a selection of pizza toppings could be encoded as a product of those selected.
This sort of encoding could be used whilst still staying in the “lists” (or “lists of lists” etc.) approach. (Rather than introducing fancier data structures.) Since the positions are important, primes 2, 3, 5, … could encode “a”, “b”, “c”, for the first letter; 103, 107, 109 encoding “a”, “b”, “c”, for the second letter; … (I used CAS.ithprime(27) to determine 103.) With this sort of approach, one can use divisibility to check whether a pattern matches a word. For short words, this approach allows a single MOD to decide a match (rather than looping in PPL). (The length of “short” being decided by the need to keep the product of encoded letters within the 64 bits PPL provides for integers; evalf(product(ithprime(K*26),K,1,7)/2^64) produces 0.215352256925.) For long words, one could employ the longer integers of CAS or break up the longer words into pieces. Hand-tuning the encoding to the used word dictionaries might help “tweak” performance. (Rearranging the ordering of which prime corresponds to which placed letter, … perhaps using products of primes rather than individual primes for placed letters.) |
|||
09-13-2023, 05:03 AM
(This post was last modified: 09-13-2023 05:06 AM by jte.)
Post: #50
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
Another approach to handle blocks of characters when checking for matches (against a pattern of the sort mentioned above — single-character wildcards) that follows more traditional computer-y approaches would be to use 5 bits per character (assuming <= 32 different characters) for the encoding (and thus pack up to 12 characters into a 64-bit integer) and then use a mask to select which characters must match specific characters.
E.g., using Code: BITAND(#00001 00010 00011b,#11111 00000 11111b) = #00001 00000 00100b |
|||
09-13-2023, 05:08 AM
(This post was last modified: 09-13-2023 05:11 AM by Tyann.)
Post: #51
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
Bonjour koname
Erreur de ma part, 377 c'est pour N5. Bonjour Jte Merci pour ces idées, bien que là cela devient un peu compliqué pour moi, je suis un petit amateur qui programme pour ses loisirs. Si vous avez suivi un peu ce poste, j'espère que vous vu ma requête pour INSTRING et son troisième argument et vous pourrez y donner suite Hello koname My mistake, 377 is for N5 Hi Jte Thank you for these ideas, although it's getting a bit complicated for me, I'm a small amateur who programs for his hobbies. If you've been following this post a bit, I hope you've seen my request for INSTRING and its third argument and you'll be able to follow it up Sorry for my english |
|||
09-13-2023, 12:09 PM
Post: #52
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
Hi Jeff,
(09-12-2023 10:47 PM)jte Wrote: There’s certainly many algorithms for subset checking, string searching, etc. but one thing this reminded me of was my dad’s program (that he did for a comp. sci. assignment in the ’60s, on punched cards — he told me about it several times…) for handling pizza toppings. His idea was to encode each pizza topping option as a prime so that a selection of pizza toppings could be encoded as a product of those selected.My respects to your dad. In those times, with such limited possibilities, coming up with and additionally implementing something like that was certainly a great achievement. (09-13-2023 05:03 AM)jte Wrote: Another approach to handle blocks of characters when checking for matches [...] that follows more traditional computer-y approaches would be to use 5 bits per character (assuming <= 32 different characters) for the encoding [...] and then use a mask to select which characters must match specific characters.At first, I was thinking about such an approach, but in this case, we are dealing with the French language, which includes accents. So, this approach would require us to maintain two dictionaries because converting text to binary for a 9-letter dictionary with over 50,000 words would take a very long time with each run (involving a lot of bitwise AND/OR/BITSL operations). Additionally, it's not feasible to store such a dictionary directly in binary form because 5 bits per character cannot accommodate 39 combinations for letters (26 + 13 accents). This would require 6 bits per word, limiting the maximum word length to 10 letters when using 64-bit integers. Instead, I opted for lists based on text strings, which are very fast and have no restrictions on both word length and list length. However, if we assume that words have no accents and have a maximum of 12 characters, your solution could indeed be faster. Later, I realized that such iteration was also unnecessary because since I have a sorted dictionary, I could implement binary search (reducing the search range). This greatly sped up the program, and now there are a maximum of a few dozen comparisons across the entire dictionary while searching for the first word, so the comparison method has become less critical. Best regards, Piotr |
|||
09-13-2023, 12:11 PM
Post: #53
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
Hello Tyann,
(09-13-2023 05:08 AM)Tyann Wrote: My mistake, 377 is for N5Nope For N5 with 'a????', there should be 423 words => 0.14s (G2). I am waiting for clarification of the rules for point 1 (spell-checking). Piotr Kowalewski |
|||
09-13-2023, 04:42 PM
Post: #54
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
(09-13-2023 12:11 PM)komame Wrote: Hello Tyann, Bonsoir Oui le vérification prend le mot tel qu'il est entré et le cherche dans la liste, si il n' y est pas le mot est désigné inconnu. (trés est inconnu, très est connu). Pour les mots de 5 lettres, la liste des mots en 'a' contient 377 mots, donc je ne peux pas en trouvé plus. le 1="abaca", le dernier "âcres" J'ai sauvegardé la liste L1 avec les 1030 mots, j'ai voulu vous l'envoyé en message privé sur le forum mais à priori on ne peut pas joindre de pièces aux messages ou j'ai pas vu comment. Pouvez-vous m'indiquer en message privé une adresse où je peux vous envoyer cela? Good evening Yes, the check takes the word as it is entered and searches for it in the list. the word is designated unknown ("trés" is unknown, "très" is known). For 5-letter words, the list of 'a' words contains 377 words, so I can't find any more. the 1st="abaca", the last="âcres". I've saved the L1 list with the 1030 words, and I wanted to send it to you in a private message on the forum, but apparently you can't attach documents to messages, or I didn't see how. Can you give me a private message with an address where I can send it to you? Sorry for my english |
|||
09-13-2023, 05:19 PM
Post: #55
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
(09-13-2023 04:42 PM)Tyann Wrote: Good evening Good evening, Just check this website for all 5-letter words starting with A: https://www.liste-de-mots.com/mots-nombre-lettre/5/a/ If you've copied them all to your application, you should have as many words as there are on that website. Thank you for explaining point 1. I will make a small correction, and tonight I will show the source code of my solution. Best regards, Piotr |
|||
09-13-2023, 07:39 PM
(This post was last modified: 09-13-2023 07:41 PM by Tyann.)
Post: #56
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
Il se trouve que mon programme de conversion a eu un raté lors de la première utilisation
et je viens de recharger la liste des a en 5 lettres 426 mots. Il se trouve que j'ai également éliminé quelques mots qui me semblait suspects à l'époque où je programmais sur G1 qui avait une mémoire trop faible pour ce projet. Je me suis aperçu aussi que le temps d'exécutions de ma fonction de recherche est plus rapide lorsque je l'appelle depuis l'écran de calcul, alors que dans l'application elle est appelée comme sous programme et renvoie la liste des solutions comme résultats et ensuite je teste si cette liste est vide ou pas avant l'affichage. Bref quand j'aurai un peu de temps je vais refaire les recherches précédentes depuis l'écran de calcul. pour exemple je viens de refaire la recherche a???? , qui me renvoie maintenant 426 mots (idem N5(1)) en 0.6 s. It turns out that my conversion program failed on the first use and I've just reloaded the list of a in 5 letters 426 words. I've also eliminated a few words that seemed suspicious when I was programming on G1, which had too little memory for this project. I've also noticed that the execution time of my search function is faster when I call it from the calculation screen, whereas in the application it's called as a sub-program and returns the list of solutions as results and then I test whether this list is empty or not before displaying it. In short, when I have a bit of time I'm going to redo the previous searches from the calculation screen. For example, I've just redone the search a???? which now returns 426 words (idem N5(1)) in 0.6 s. Sorry for my english |
|||
09-13-2023, 10:23 PM
(This post was last modified: 09-14-2023 06:16 AM by komame.)
Post: #57
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
This is the moment when I reveal the details of my solution
Although it might seem very complex, in reality, the algorithm is straightforward, and, moreover, really fast. The program is named CROSSWORDS. It performs word matching to a pattern - crossword search, where we input only known letters, and we use '?' in place of unknown ones. Here, diacritics are ignored, meaning their Latin equivalents are considered for the search. However, if we were to abandon this by implementing an exact word matching, it would further speed up the search because conversion would not be necessary. During searches, letter case doesn't matter; the program converts everything to uppercase. Usage examples: CROSSWORDS("ROTI?") => {"ROTIN","RÔTIE","RÔTIR","RÔTIS","RÔTIT","RÔTÎT"} CROSSWORDS("?A?TS") => {"FAITS","FARTS","GANTS","HARTS","HASTS","HAUTS","KARTS","LAITS","MALTS","PARTS","RAPTS","SAUTS","TACTS","WATTS"} CROSSWORDS("P?T") => {"PAT","POT","PST","PUT","PÛT"} CROSSWORDS("P?AT") => {"PLAT","PUÂT"} If at least one letter at the beginning of the word is specified (meaning the pattern doesn't start with '?'), a binary search is activated, significantly reducing the time to obtain results. If there is a '?' at the beginning, this binary search cannot be used, and then the dictionary is scanned, which takes considerably longer. EDIT: I finally concluded that spell-checking functionality is unnecessary because by explicitly providing all the letters of the word, all permissible forms of that word will appear in the result. There will be few of them, as they may differ only in accents, if the language uses them at all. CROSSWORDS("dEs") => {"DES","DÈS","DÉS"} CROSSWORDS("Age") => {"ÂGE","ÂGÉ"} Congratulations to everyone who made it this far in this thread. Attached is the complete solution with a dictionary from 2 to 9-letter words (currently French). If you want longer words, you'll need to add them to the 'WordDict' file in the same way it's done for other word lengths. You can replace the dictionary with a different language, but it's important that the words are sorted alphabetically. If you change the language, adjust the function for converting diacritics: 'convAccents'. If anyone wants to delve deeper into how it works, I'd be happy to explain (if someone asks). Best wishes, Piotr Kowalewski |
|||
09-14-2023, 04:59 AM
Post: #58
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
Bonjour Komane
Je vais regarder cela de près, cependant la méthode itérative n' a pas dit son dernier mot J'ai une nouvelle idée d'optimisation que je vais tester aujourd'hui et vous donnerez les résultats ensuite ainsi que mon code si ça peut intéressé . Encore merci pour tout ceci. Hello Komane I'm going to take a close look at this, but the iterative method hasn't said its last word I've got a new optimisation idea that I'm going to test today and I'll give you the results afterwards as well as my code if you're interested. Thanks again for all this. Sorry for my english |
|||
09-14-2023, 06:05 PM
Post: #59
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
Hello Tyann,
(09-14-2023 04:59 AM)Tyann Wrote: I'm going to take a close look at this, but the iterative method hasn't said its last wordOf course! And good luck! (09-14-2023 04:59 AM)Tyann Wrote: I've got a new optimisation idea that I'm going to test today and I'll give you the results afterwards as well as my code if you're interested.I also have some ideas, but in my case, when the first letters of the word are provided, the algorithm is already so fast that there's not much to gain (maybe a few dozen milliseconds). It's definitely worse when there's a "?" at the beginning. Perhaps in such a case, there's still room for optimization. Thanks! Piotr Kowalewski |
|||
09-14-2023, 06:28 PM
Post: #60
|
|||
|
|||
RE: How much memory may a HPPL program use on a G2?
Hi Stefan,
Are you still reading this thread? After all, you started it, and Tyann and I turned it into something completely different, which didn't really help you. So that you can still extract something interesting from this thread, I'm attaching your QUEENS solution (under the name QUEENS2), slightly modified by me to make it resistant to the list length limitations by using the text strings. For larger board sizes like 10x10, 11x11, and so on, it definitely works faster than your original solution and can handle 12x12, 13x13, 14x14 and maybe more... if you like to wait Code: LOCAL Piotr Kowalewski |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 3 Guest(s)