The Simple On-screen Keyboard (SOK)

[frafu]

Hello,

I have looked at the new specs of sok and it seems to be good from my standpoint as an m2 user, except for 2 points (This time I will be more explicit in order to avoid misunderstandings like above with the interaction between onscreen and physical keyboard):


1) The first point about iconizing the onscreen keyboard comes from my reallife experience:

I really miss the option to iconize the onscreen keyboard. The onscreen keyboard that I am using now can be resized to any size that the user wants. But it also has a key that reduces the keyboard to the size of an icon. That icon also floates and is resizable. When I click on that icon, the onscreen keyboard reappears with the size that it had before iconzing it.

In fact, I keep my onscreen keyboard nearly all the time in its iconized state because I want it to cover the desktop the least possible. When I have to type something, I click on the icon; the onscreen keyboard appears and I type what I want to type with the onscreen keyboard; as soon as I am finished with typing, I put the onscreen keyboard back to its iconized state.

Further, I want to explain why the fact of being able to resize the onscreen keyboard cannot totally replace the fact of being able to iconize it: I am using the onscreen keyboard with a determined size which I am accustomed to. By clicking on the icon, I get the onscreen keyboard back at exactly that size. It would be a pain if I had to resize it every time by dragging the resizing corner of the onscreen keyboard window to that determined size.

Of course, if Ubuntu (that I don't really know yet) already has the ability to make an application disappear by a single click and reappear by another click somewhere on the desktop, the iconizing feature will be useless. Otherwise, I hope that you will be able to add it already to the first release of sok.

(To be complete, when the onscreen keyboard that I use is in its "iconized" state, I think it is not really an icon, but it still is a small window as there is a small empty titlebar on top of the icon.)

2) The second point is about the scanning feature and it does not come from my reallife experience, but something I thought about, while reading the new specs:

A user that relies on scanning will not be able to click on the button to activate scanning. Or do I miss anything? Maybe that you already considered this, and the specs are not explicit enough!?


frafu



PS: Could you please tell me what you mean with the 3rd point in the design section about the ability to easily configure the layout and the localisation? Do you mean that the user can choose where he wants to place the keys on the main and secondary window? Or do you mean that the user can also choose what keys he wants on the main and secondary window?

Can I for example imagine the layout configuration window this way: there are
- the 2 empty keyboard windows
- different shape-templates for keys
- all the letters and functions provided by the onscreen keyboard
The user can associate a letter or a function to a shape and choose where to put shape on the keyboard?

[Henrik]

1. Resizing: At the moment we haven't actually included a button for resizing/minimizing, but you easily could (see point 3). At the moment we only use the window decoration button to minimize. This brings the keyboard down to a minimised state at the taskbar, as any other window. Clicking on that taskbar entry will bring the window back to its original size and location (ie. it remembers the changes you made to it). We will also have the option of having a taskbar applet instead of the normal taskbar entry. I think the floating icon idea is a Mac-ish thing, which I guess doesn't have a taskbar. I don't think we need that, but if we do, it should be fairly easy to make.

2. Scanning: You are right that a user who needs scanning won't be able to click that button with a normal mouse. You could say that that is a proof-of concept implementations, which allos us to implement all the other aspects of scanning, such as how the scan pattern goes through the keys, setting speed, etc. We would then want to provide several ways in which that input signal could be provided. The mouse button is a fair place to start because some people will make a swith using parts from a mouse. Others will use a spacebar, while others again will use special switching equipment connected to the serial or USB port.

3. Key placement: I think you will find that the layout system for this keyboard will be more flexible than anything you have seen so far We talk of two keyboards, but that's just the default layout we will implement first. Others can implement other variants, with 3-4 keyboards including macros and scripts or all the keys on a single keyboard. The keyboard layouts are generated in any drawing program that supports SVG such as http://www.inkscape.org/ The keys can be any shape (well, just rectangles for now) or size and have any position. A keypad can have 3 or 100 keys and be in any language.

So if you want to contribute to the development at this point, the best thing to do might be to learn how to use inkscape and draw some useful keyboard layouts. There even seems to be a Mac OS X version available

[frafu]

Hello Henrik,

First of all, thanks for your reply.

1. Resizing: You guessed it right: Macs don't have a taskbar. The solution of minimizing to the taskbar is probably good in most cases. However, there is at least one case where another solution has to be considered: Applications that hide the taskbar: For example a movieplayer running at fullscreen. Could you please tell me how you planned for a m2-user to make the onscreen keyboard reappear when no taskbar is available?

By the way, what is a taskbar applet?

2. Scanning: I thought that the specs would refer to the final application. After reading your explanation, I assume the specs will evolve with the different milestones you defined in the accessibility-list.


3. Key placement: So, I assume that the onscreen keyboard will parse the svg-file and "learn" the layout. Keys with letters will be no problem; but keys like modifiers, return-key, pageup-key,... will probably have to be coded in a determined way!? I don't expect an answer to this question now, because I suppose that a precise answer will only be possible after the implementation.

There is indeed a MacOSX-version of inkscape available; thanks for pointing me to it.


frafu

[t0rtois3]

1. Resizing: That's a fair point. Though all applications I have tried let you exit fullscreen with the mouse. An example of a panel applet would be the clock on the bar at the top of your screen, and the workspace switcher. Ubuntu ships with a whole host of these, you can add more by right clicking on the panels at the top and bottom of your screen.

A panel applet would unfortunately also suffer the problem you mention.

3. Key placement: Inkscape assigns every object, be it a box or a drawing made with lines and curves a unique identifier. This can be seen by right clicking on an object and selecting object properties. I then have a seperate file which describes the actions of each key in turn.

Chris

[frafu]

1. Resizing: That's a fair point. Though all applications I have tried let you exit fullscreen with the mouse.

Though I am not interested in games, I wonder whether you considered them?

By the way, thanks for the explanation.

frafu

[frafu]

Hello,

In the thread titled "GOK alternative" of this "Accessibility Talk" Forum I read about another onscreen keyboard called xvkbd. I went to the homepage of xvkbd and learned about a feature called "Quick-modifiers" that seems quite interesting: if I understood it correctly, when the button-up of a key-click does not occur on the clicked key (in other words the pointer is moved away from the key before releasing the mousebutton) it performs the same action as a "modifierclick+keyclick".

Maybe it could be a good idea to also implement a similar feature in sok.

Have a nice day.

frafu

[Henrik]

Interesting. And by modifier you typically mean Shift I guess.

I was thinking of a similar function for those who have two mouse buttons available (left and right). A right click on a button would be the same as a Shift+click.

Alternatively one of these input techniques could give you a space after the letter, thus forming a word (saving you one click per word).

[frafu]

Interesting. And by modifier you typically mean Shift I guess.

I was thinking of a similar function for those who have two mouse buttons available (left and right). A right click on a button would be the same as a Shift+click.

Alternatively one of these input techniques could give you a space after the letter, thus forming a word (saving you one click per word).

In fact, I thought at first to leave the user the choice of what modifier it should replace: user customizable in the settings.

Now I wonder, why not expand the idea: for example moveup before releasing = shift, movedown=alt, moveleft=ctrl, moveright=space.


However I think it should be fully customizable in the settings; maybe a gui like this:
A main checkbox to activate/deactivate the "quickmodifiers"; below a secondary checkbox to differentiate between "any move has the same action" and "each move has a different action"; finally in the third level 4 checkboxes (moveup, movedown, moveleft, moveright) that are only active when "each move has a different action" is active; and next to each of the 4 checkboxes a popup to set the wanted action. (If more actions are available: up+left, up+rught, right+up, right+down, down+left,...)

Maybe an alternative could be an empty list with an add, edit and remove button; by clicking on the add-button, the user gets a dialogue where he can define a new mousegesture by using 2 popups: the first popup containing a list of mousegestures and the second popup with the available actions...


Finally, I wonder now whether the ubuntu desktop offers the mousegestures more generally to replace different right-button-clicks as an accessibility option. It would probably be appreciated when the desktop would provide at least a mousegesture for the right click to get to the contextual menu: in fact, the main reason why I use Firefox instead of the browser shipped with my OS is because a "click and hold" in Firefox (I am talking about MacOSX; I don't know for other OSes) makes the contextual menu appear; the other browsers instead require a right click. Are general mousegestures already available under ubuntu?


frafu

[Henrik]

Now I wonder, why not expand the idea: for example moveup before releasing = shift, movedown=alt, moveleft=ctrl, moveright=space.

OK, this is getting a bit complex now

I'm not sure having this range of options is actually more efficient. First of all you use Ctrl and Alt quite rarely compared with the typing letters. Shift gets used 3-4 perhaps in each sentene (first word and some names). Space gets used once for every word though.

Second, click-and-hold + move-up is a fairly complex opperation that will take extra consentration and slow down the process. If the next letter you wanted in a different direction you then have to stop and change direction to get to it.

Of course points 1 and 2 do cancel each other out to some extent Since you normally don't use the Ctrl and Alt keys during normal typing it is less jarring to configure them to drag like this.

I guess only real-life trials will decide if it works.

Finally, I wonder now whether the ubuntu desktop offers the mousegestures more generally to replace different right-button-clicks as an accessibility option. It would probably be appreciated when the desktop would provide at least a mousegesture for the right click to get to the contextual menu: in fact, the main reason why I use Firefox instead of the browser shipped with my OS is because a "click and hold" in Firefox (I am talking about MacOSX; I don't know for other OSes) makes the contextual menu appear; the other browsers instead require a right click. Are general mousegestures already available under ubuntu?

I don't think so. This would be a good feature to have as part of the underlying X framework. There is some support in individual applications AFAIK, such as GDM and Firefox via extensions.

What we could do quite easily (I think) is to add an (optional) button to SOK that was a sticky mouse button reverser. If you click on it once it changes the active mouse button until you click the mouse somewhere else. Clicking it twice makes the change stick.

[frafu]

OK, this is getting a bit complex now

I'm not sure having this range of options is actually more efficient. First of all you use Ctrl and Alt quite rarely compared with the typing letters. Shift gets used 3-4 perhaps in each sentene (first word and some names). Space gets used once for every word though.

I guess only real-life trials will decide if it works.

You are probably making a good point when saying that the shift-modifier is more often used than the alt- and ctrl-modifier. So only using a general action for the shift could be really more efficient than some complex gestures requiring concentration.

Further, I want to point out, that your suggestion about the space is only useful as long as the deferred feature "word prediction and completion" is not implemented. Afterwards, won't an automatic space be an option of the word prediction?

By the way: the Keyboard.py source file contains this line:

 
modifiers = {"shift":1,"caps":2, "control":4, "mod1":8, "mod2":16, "mod3":32, "mod4":64, "mod5":128}

Why isn't the alt-modifier explicitely named?

Click and hold to make the contextual menu appear would be a good feature to have as part of the underlying X framework.

Does this mean that it would still be possible to implement it, so that it would work with all the applications that already exist? (I mean adding it to the X framework without having to change caracteristques that already exist in order to avoid braking existing applications.)

What we could do quite easily (I think) is to add an (optional) button to SOK that was a sticky mouse button reverser. If you click on it once it changes the active mouse button until you click the mouse somewhere else. Clicking it twice makes the change stick.

Click and hold to make the contextual menu appear in every application would be very nice for people only using one mousebutton. But as it is not the case, your idea about making the mousebutton reverser (I suppose you are talking about the feature of switching the left and right mousebutton of the specs) only stick when you doubleclic it is probably a good way to go.

(In fact, as far as I know, a similar behaviour is also planned for the modifiers.)


frafu