Touchscreen in calculators

[nsg]

Notes on design of touchscreen-based handheld calculator
History

There are two features that define a classic hand held calculator. One is a small display capable of displaying one (very rarely more than one) number. Second is a more or less rectangular panel of buttons which lets user enter numbers and execute functions. That was a hardware available at the time and the user interface and operating system desing was driven by this available hardware.

Modern Android calculator apps follow this same pattern: they dedicate fixed portion of a screen to numeric display and the rest to fixed on-screen keyboard.

Following this tradition grossly underuses UI abilities granted by touch screen and results in products that are not as efficient problem solvers as they could have been.

Guidelines

The following is a set of statements I believe are true. To save space, I am not writing caveat "in my opinion" before every such statement, but it is strongly implied. I would like to design a hand held calculating device based on the outlined principles and see if it actually makes it better then classic.
Spreadsheet

The notion of spreadsheet is among the most important inventions of computer industry. It put the ability to perform calculations of unfathomable complexity in the hands of normal people (that is, people who are not programmers). The hand-held calculating device should use a spreadsheet as its main mechanism of presenting data.

More structured form of a spreadsheet -- fixed width table with named columns -- can be used as an element of relational model, or simply as an easily lookuppable reference data storage.

Keyboard vs menus

Some sort of virtual keyboard seems to be inevitable. There is simply no better way of entering numbers then touching well defined and labeled spots on a screen with a finger. However, there is no reason the virtual keyboard has to remain fixed all the time and there is no reason it should always be a rectangular-ish array of rectangular buttons.

Classic calculators usually have one or more "shift" keys -- pressing such keys temporary redefines the rest of the keyboard to have different meaning. This alternative meaning is eather spelled on each button with alternative color, or sometimes is implied and it is up to a user to memorize this meaning.

In this situations the shift key acts as a menu invocator and the rest of the keyboard as menu items. Touchscreen keyboard should simply present menu elements as such. Furthermore, the menu items do not have to be simple rectangles in a grid. They may be arranged in a way that is mnemonic to a menu function and accompanied by notes and diagrams to assist with recollection.

Programming

Programming should not be required to successfully use a calculator to solve many meaningful problems. But if user chooses to program, programming environment should be adapted to the realities of a handheld device.

If a terminal (fullsize keyboard+large user facing screen) is available, then the best way to program is to use a programming language and compose a program as a set of text files. This conjecture is supported by the fact that numerous systems of so called "graphic programming" are not doing so well and occupy only marginal niches in programming industry, despite decades of hype and aggressive marketing by their vendors.

Handheld device presents different challenges. Typing long text is easy on a full size keyboard, but hard on a handheld virtual keyboard. Different approaches involving elements of "graphic programming" may be tried to reduce typing. This includes drag-and-drop, selecting blocks from menus, representing control flow or data flow as a directed graph, etc.

I believe that ability to program on-device (as opposed to programming in a desktop environment and then uploading complete "applet" to a caclulator) significantly improves its capacity as a problem solving tool.

Implementation

Cells

Spreadsheet cells may hold number, text of formula. Selecting a cell activates numeric keyboard, text keyboard or formula editor respectively.

Numeric cells and numeric keyboard

When activated, numeric cell and its value becomes rX of an RPN calculator. The rest of the stack is intrinsic to OS and follows the keyboard to whatever cell it is attached to. The keyboard provides facilites for number entry, basic arithmetic, stack manipulation and one or more menu buttons to access advanced functions and extensions. RCL and STO commands allow data exchange with other cells or with named variables.

When user leaves the cell, the current value of rX becomes the new value of the cell. At this point the cell is considered "updated" which triggers dependent recalculation if any.

[EugeneNine]

Are you planning on writing calculator software, wanting to build a new calculator?

Why a virtual keyboard, why not a slider such as: http://us.blackberry.com/smartphones/pri...ifications

[nsg]

Anyway, here is my firstest proptotype.

RPN Spreadsheet

This link is best opened by smartphone, this should simulate the experience that I have in mind. You have a device in your hand and it is pretty much a handheld calculator, but it utilizes powers of touchscreen in a way that fixed keyboard/indicator cannot provide.

Initially all you see is a spreadsheet. When you tap a cell, the keyboard pops up and that cell becomes rpn calculator. You can forget about all other cells and operate it as a regular scientific rpn calculator (this is a prototype, so only thing available is number entry, 4 basic arithmetic ops and sto/rcl and there is no text, formulas or programming). Tap cell again to deactivate keyboard and navigate through spreadsheet.


When you select another cell, the stack is preserved, but the regX is now the contents of that new cell (the cell is also highlighted to show which one acts as an "indicator")

You can also use all the cells you see as memory registers: tap STO and then cell to save current indicator to that cell (unlike regular tapping, this does not switch current cell). Same thing with RCL. RCL and cell copies contents of that cell to a current indicator and pushes old contents up.

I used chrome on my Android phone, so, I guess I am interested to hear if it is not working (or working) on other platforms.

(03-20-2017 01:38 AM)EugeneNine Wrote:  Are you planning on writing calculator software, wanting to build a new calculator?

Eugene,

I definitely want an actual device. I am pretty sure I can make an html-based prototype/emulator for a smartphone, designing and building an actual piece is a bit outside of my comfort zone.

Quote:Why a virtual keyboard, why not a slider such as: http://us.blackberry.com/smartphones/pri...ifications

Slider is still a fixed keyboard. I want fluid mnemonic context sensitive keyboard that does not strain the memory as much as overloaded fixed keyboard. And is out of the way when it is not needed.

My prototype does not feature anything of this kind yet, but I hope by next weekend I will have something to illustrate my point.

So, what do you guys think? Am i up to something, or should've just done some yardwork this weekend instead?

[grsbanks]

For what it's worth, the HP Prime does go some way towards this.

It still has a conventional keyboard but many operations are possible using tap, hold, drag and pinch/zoom on the touch-sensitive screen.

One of the "applications" on the Prime happens to be a spreadsheet, as it is on some other graphing calculators such as the TI Nspire CX and CX CAS so it is clearly seen by the industry as a requirement.

[nsg]

Thanks for your comments.

I am aware of these calculators, and their spreadsheet applications, I just think that they do not take this idea nearly far enough. And none of those calculators is something I would like to have and use (which has nothing to do with spreadsheet app per se). I understand that the designers have their reasons for that and I am not their target demographic.

[EugeneNine]

How would this be an improvement over a spreadsheet today. For example I have several spreadsheets on my phone now and use opendocumentviewer or AndrOperOffice.

[nsg]

Improvement would be integration of calculator operation logic with presentation style of a spreadsheet.

For example in a typical spreadsheet if you need to multiply a contents of a cell by 2.54, you need go to
a) select temporary free cell
b) enter formula there =2.54*[ref to cell]
c) copy that cell
d) paste-value to original cell
e) clear temporary cell

In my scenario you select a cell and tap [2][.][5][4][*]

[mfleming]

(03-20-2017 03:14 AM)nsg Wrote:  Anyway, here is my firstest proptotype.

RPN Spreadsheet

.... I want fluid mnemonic context sensitive keyboard that does not strain the memory as much as overloaded fixed keyboard. And is out of the way when it is not needed.

The prototype works fine in Chrome, but not Firefox Seriously though, interesting idea. You want a keyboard customized to a particular context, not one invoked by extra keystrokes like shift or menu keys, right? To do that, I would think you'd need the format of an individual cell, column or row to convey the information about which keyboard to use. Currency format invokes a financial-oriented keyboard, text format an alphanumeric keyboard, scientific format a specialized keyboard, etc. Maybe colorize cells to indicate format?

Any thought on that, and reducing the user's burden so you're not just transferring shift-key effort to format-cell action?

~ Mark

[EugeneNine]

seems to be webkit only.

Where is the space so we can enter a formula within a cell without needing to reference another?

[nsg]

(03-20-2017 09:50 PM)mfleming Wrote:  The prototype works fine in Chrome, but not Firefox Seriously though, interesting idea. You want a keyboard customized to a particular context, not one invoked by extra keystrokes like shift or menu keys, right?

Kind of. You start with a keyboard suited for context and go from there. I did not think of using formats to provide different menus, but this may be interesting idea.

In my vision you still have to follow through the sequence of shift keys to reach deep down functions. The improvement comes in a form that as you descend, each next menu (keyboard) presents in a clear way what choices are available. There is nothing special in this idea -- all menus do just that, however here I am blending menu and virtual keyboard for (hopefully) better ui.

(03-20-2017 11:34 PM)EugeneNine Wrote:  seems to be webkit only.

Where is the space so we can enter a formula within a cell without needing to reference another?

Havent got to that part yet. I want to make a "calculator" part first. So far I replaced indicator and memory registers with a spreadsheet, next I am trying to do dynamic keyboard/menu to implement standard scientific functions to see how and if that idea of keyboard-as-menu even works.

[mfleming]

(03-21-2017 02:07 AM)nsg Wrote:  Kind of. You start with a keyboard suited for context and go from there. I did not think of using formats to provide different menus, but this may be interesting idea.

I think I see your direction. The Acron calculator (from Play Store or iTunes) has a three line menu where the top line is typical of what you'd see on a 42S or 48. The bottom two lines are like the menu buttons that pick a particular menu item list at the top line. Have a look at the app pictures or try out the free version (paid version is quite reasonable!) to see if it sparks some ideas.

(03-21-2017 02:07 AM)nsg Wrote:  Havent got to that part yet. I want to make a "calculator" part first. So far I replaced indicator and memory registers with a spreadsheet, next I am trying to do dynamic keyboard/menu to implement standard scientific functions to see how and if that idea of keyboard-as-menu even works.

I really like the idea of spreadsheet cells as calculator registers. Referencing another calculator implementation, the go41cxt (tablet version) can show the calculator on the left, printer output on the right and all registers in the middle panel. Very neat for running something like the 41Z module and seeing the contents of registers. If your cells could hold text, matrices, etc. then a column of cells could be like files in the HP-41's extended memory.

A last thought. The pop-up keyboard can easily cover the cell you select and hide what you're typing. Perhaps you could also have a pop-up display, either separate from or attached to the keyboard?

[EugeneNine]

Now that I think about this looks a lot like the pictures of xpander http://www.hpmuseum.org/xpand.htm

[nsg]

(03-21-2017 03:43 PM)mfleming Wrote:  I think I see your direction. The Acron calculator (from Play Store or iTunes) has a three line menu where the top line is typical of what you'd see on a 42S or 48. The

I've looked at Acron calc. It does have a nice menu, but like with other similar things, while having have the right idea it does not take it far enough. In it keyboard is keyboard and menu is menu, they are separate. In my project keyboard is menu and menu is keyboard.

Quote:A last thought. The pop-up keyboard can easily cover the cell you select and hide what you're typing. Perhaps you could also have a pop-up display, either separate from or attached to the keyboard?

I already noticed that. My original intention was to scroll the obstructed cell into a viewing area when keyboard pops up, but then I noticed that it is nice to have a full view of a number. The number in a cell may be tightly formatted to fit, so there may be a benefit of having attached indicator. Maybe even with full stack display, while we are at it.

(03-22-2017 03:03 PM)EugeneNine Wrote:  Now that I think about this looks a lot like the pictures of xpander http://www.hpmuseum.org/xpand.htm

Nice. I never heard of this project, thanks. It does look a little bit like it, with spreadsheet-like property sheets. But again, see, how xpand wastes screen space? Menu on top, menu on bottom, toolbar -- my idea is to turn it all into a keyboard/menu hybrid. Scrollbars are a waste too -- swipe-scroll is the way to do things these days. But, I guess, that was not apparent at the time.

[EugeneNine]

The wasted space is because it is on Windows CE. The first couple versions of windows CE looked just like desktop windows with the start menu at the bottom left. Then there were the clueless magazine reviews which said "its not like palm" or "its not like iphone" so then CE tried to be both palm and iphone and flopped.

The first pic though with the table, make something like that but make it rpn entry and be able to enter complex formulas in each cell.

I still prefer the physical keyboard though, give me a screen that slides open to reveal an hp48 keyboard.

[mfleming]

(03-23-2017 01:35 AM)nsg Wrote:  I've looked at Acron calc. It does have a nice menu, but like with other similar things, while having have the right idea it does not take it far enough. In it keyboard is keyboard and menu is menu, they are separate. In my project keyboard is menu and menu is keyboard.

Well, you're definitely going where no calculator (that I know of) has gone before. Looking forward to your next update.