Longest computation that your calculator performed for you?

03302017, 08:36 PM
(This post was last modified: 11122017 08:55 AM by Garth Wilson.)
Post: #4




RE: Longest computation that your calculator performed for you?
(03302017 07:37 PM)pier4r Wrote:(03302017 07:30 PM)Garth Wilson Wrote: A few years ago I used my HP71, in BASIC, to generate large lookup tables (most being 65,536 cells each, and 16bit, meaning 128KB per table) for hyperfast, accurate, 16bit scaledinteger math, to use as a virtual coprocessor for hardware that didn't have a normal coprocessor. It took my HP71 a few weeks to generate the tables and convert to Intel Hex files. I posted them, along with a discussion of the value of scaledinteger math and how to use it effectively, and information on how each table was calculated, on my website, at http://wilsonminesco.com/16bitMathTables/ . Fixedpoint is a limited subset of the broader, moreflexible scaledinteger math. "Fixed" moves the point right or left one in increments of one or more digits at a time, while "scaled" can, in essence, put it within a digit, basically infinitely variable rather than in digit steps. For example, a 16bit integer representing 0360° in fixedpoint can go up to 36,000 (for .01° resolution) using base 10, or perhaps even 46,080 (for 1°/128); but then you have the problem of carry/borrow/rollover. But if you scale it so the whole circle is 65,536, it's like the decimal point is, in essence, somewhere in a binary digit, and you get a resolution of 1°/182.044, and the rollover works as it should, so 359°+2°=1°, or 1°2°=359°. Starting at zero degrees/radians and adding 3π/2 Radians (ie, 270°) puts you in the same place as subtracting π/2 Radians (ie, 90°). The high bit always tells which half of the circle you're in. The odd scale factor remains throughout the calculations until it's time for humanreadable output, then you multiply or divide by the appropriate constant. If the output is in the form of graphics rather than printed digits, the number never may never need to exist in any format where the point is tucked neatly between digits in any base. Depending on the size of image, you might multiply by yet another odd scale factor which will change as you change the size of the window. Or suppose you have a 07V range you want to measure with as much precision as possible with a 16bit A/D converter and represent in memory with 16 bits. You might set the gain of the amplifier ahead of it so 7V gives an output of 65,535, meaning you get 9,362.3 counts per volt, 9.3623 counts per mV, or 106.81µV per count. The odd scale factor remains throughout the calculations until it's time for humanreadable output, then you multiply or divide by the appropriate constant. Actually, my workbench computer has a fast 8bit A/D converter that's 05V, or actually 0255/256*(5.08V) since the regulator is wee bit off; so one count is 19.84mV. A common reference for the converters is 4.096V so that the user can be lazy and used fixedpoint rather than scaledinteger, so you get 1mV per count if it's a 12bit converter, 4mV per count if it's a 10bit converter, 16mV per count if it's an 8bit converter, etc., on even boundaries of binary digits. http://WilsonMinesCo.com (Lots of HP41 links at the bottom of the links page, http://wilsonminesco.com/links.html ) 

« Next Oldest  Next Newest »

User(s) browsing this thread: 1 Guest(s)