Announcing easystroke, a gesture recognition app [Archive]

Tom Jaeger

June 22nd, 2008, 08:53 AM

Chokkan

June 22nd, 2008, 09:23 AM

Looks very cool. I'm currently using Gestikk (https://launchpad.net/gestikk) which, I think shares some similarities to your project. I'll give easystroke a try. Could you give me a few pointers on what makes this different (better?) than Gestikk?

Tom Jaeger

June 22nd, 2008, 09:56 AM

Hmm, I haven't spent too much time evaluating gestikk, so I might not be the best person to comment on this, but at least from the point of view of a tablet pc user, easystroke has a few clear advantages: It is more flexible in what strokes it allows (I haven't been able to define an S-shaped stroke in gestikk, for example) and there are actions such as ignore and scroll that are specifically designed to make keyboardless operation easier. Another obvious difference is that gestikk will pass strokes to the applications, producing spurious right-clicks. I'd also like to think that easystroke's UI is more polished, but of course, this is a matter of opinion.

Luffield

June 22nd, 2008, 10:27 AM

I use Gestikk too and I'm pretty happy with it, but I think I'll give easystroke a try because I'm curious about the statistics :)

bash

June 22nd, 2008, 11:30 AM

I'm sitting in front of a windows box atm, so I can't try it. But personally what I'm looking for is basically a version of the firefox plugin all-in-one-gestures for the desktop. Does your programm come close to that?

Chokkan

June 22nd, 2008, 01:18 PM

I think that is pretty much what it is. Check out the sourceforge page. Pretty good documentation there.

barbedsaber

June 22nd, 2008, 01:52 PM

its bad enough going onto a windows computer, pressing super key+space, and getting annoyed when gnome-do doesn't come up.

Mazza558

June 22nd, 2008, 01:56 PM

How will this work with Firefox and a mouse gestures addon?

Luffield

June 22nd, 2008, 02:05 PM

its bad enough going onto a windows computer, pressing super key+space, and getting annoyed when gnome-do doesn't come up.
You can install Launchy. It's not as good as Gnome-Do but it's better than nothing.

barbedsaber

June 22nd, 2008, 02:08 PM

You can install Launchy. It's not as good as Gnome-Do but it's better than nothing.

school computer, but thanks anyway.

Tycho Quad

June 22nd, 2008, 02:17 PM

Thank you so much Tom! This is exactly what I have been looking for since I moved to Ubuntu and lost my precious Strokeit.

Small feature request, I would like the ability to set mouse buttons as gesture triggers. For example, If I hold down the gesture button and flick the scroll wheel up 3 times, I would like easystroke to skip through 3 tabs. This is how I had it set up in Strokeit, and I really liked it :)

Mazza558

June 22nd, 2008, 02:28 PM

It installs in Gutsy too, just tested.

EDIT:

It doesn't run though.

easystroke
easystroke: error while loading shared libraries: libboost_serialization-gcc42-1_34_1.so.1.34.1: cannot open shared object file: No such file or directory

barbedsaber

June 22nd, 2008, 02:33 PM

It installs in Gutsy too, just tested.

EDIT:

It doesn't run though.

well, that would make it very usefull

zmjjmz

June 22nd, 2008, 03:03 PM

What arguments would I use to set it to unminimize a window?

Luffield

June 22nd, 2008, 06:34 PM

I must be missing something in the configuration procedure, because my gestures don't do anything and the statistics tab always shows -100%.
The documentation is a bit vague there I think. After I set the name, the type and the argument I click "record stroke" and a dialog box pops up, asking if I want to delete current or cancel. I click "delete current" (makes more sense to me but "cancel" doesn't work either), I make the gesture and, well, nothing happens. Nothing shows up in the Stoke column.
Bug? User error?
Other than this problem Easystroke looks very good indeed.

Tom Jaeger

June 22nd, 2008, 07:02 PM

Small feature request, I would like the ability to set mouse buttons as gesture triggers. For example, If I hold down the gesture button and flick the scroll wheel up 3 times, I would like easystroke to skip through 3 tabs. This is how I had it set up in Strokeit, and I really liked it :)

Interesting idea. I've used strokeit and it actually was part of my motivation for writing easystroke (although its gui servered more as a negative example), but I've never noticed this feature.

This isn't very hard to implement and easystroke is actually doing something quite similar right now in its 'click during stroke' feature (which is actually a lot more involved). Of course, 'click during stoke' is the exact opposite of what you are looking for.

I don't think anyone would want to use this feature and the current 'click during stroke' at the same time, so it would make sense to enable the feature in the same combo box. You would then set up individual actions using "Record Stroke", so this won't clutter up the UI, which is very important to me.

The only remaining question is whether this should ignore the gesture that was drawn prior to flicking the scroll wheel or if the action should depend on it, so that you could set up gesture button + scroll wheel to switch tabs in firefox and an 'up' gesture + scroll wheel to do alt+tab. I don't have a preference here, so I'm inclined to give the user a choice, it'd be just one additional combo box entry.

Tom

Tom Jaeger

June 22nd, 2008, 07:13 PM

It installs in Gutsy too, just tested.

EDIT:

It doesn't run though.

My bad, I got the dependencies wrong. I'll see if I can fix that, you might need to install the hardy version of libboost-serialization. Can you paste the output of 'apt-cache search libboost-serial' on your system?

In any case, the binary (easystroke-0.1.2.gz) has boost-serialization statically linked in, so it should run on gutsy.

When I started developing easystroke, I had a few problems with bugs in the version of the gtk library that comes with gutsy. Hopefully the workarounds I put in place are still working, but I haven't tested this recently, so you might experience some UI glitches.

Tom Jaeger

June 22nd, 2008, 07:17 PM

What arguments would I use to set it to unminimize a window?
That would be a feature that would have to be implemented in the window manager. The only thing I can think of right now is Alt+Tab, but at least under compiz, that only works if there are at most two windows on the desktop. Changing the alt+tab order might help, but I couldn't find an option for that.

Tom

Tom Jaeger

June 22nd, 2008, 07:20 PM

Luffield

June 22nd, 2008, 08:37 PM

Thanks Tom. It works great now. Excellent job, I love this.

Luffield

June 22nd, 2008, 08:54 PM

Tom Jaeger

June 22nd, 2008, 11:23 PM

I think I found a real problem now :)
I added an exception for Firefox and it worked well (although Easystroke called it Navigator for some reason). Then I added Easystroke to my session, logged out and logged in. It started automatically and worked well, but the exception for Firefox didn't work. I quit Easystroke and ran it from the terminal, and then the exception worked again.

Thanks, I understand the issue now: It occurs whenever firefox is started while easystroke is already running. Easystroke currently asks for the WM_CLASS property only when the window is created and uses a cached value later. The problem is that firefox sets WM_CLASS later than expected. This should be easy to fix.

Tom Jaeger

June 22nd, 2008, 11:39 PM

I think I found a real problem now :)

Fixed in the darcs repository and the attached binary.

Luffield

June 23rd, 2008, 05:18 AM

Wonderful! Thank you!

Tom Jaeger

June 23rd, 2008, 06:21 AM

Tom Jaeger

June 23rd, 2008, 07:22 AM

In particular, I'm pretty sure it won't work correctly if your trigger button is > 5.
Yup, this is a problem. The X server does not report events correctly in this case. I can think of an evil workaround, but I'm not sure I want to go there.

Tycho Quad

June 23rd, 2008, 08:21 AM

Works fabulously thank you very much!

I'm getting some weirdities in regard to certain mouse buttons but you already told me of that, besides, i think it's partly because linux is not identifying all my mouse buttons correctly anyway. I have a Logitech MX1000 (non bluetooth) if anyone knows how I could fix that, please let me know.

EDIT: Another small feature request... Would it be possible to perform the gesture on the window under the point where the gesture started, rather than the window in focus?

Tom Jaeger

June 23rd, 2008, 09:45 AM

EDIT: Another small feature request... Would it be possible to perform the gesture on the window under the point where the gesture started, rather than the window in focus?
Good catch, this was a bug. I hadn't noticed it since I use focus on enter.

The attached version might be a little unstable since I've just done some minor refactoring, but it does fix this specific issue :)

Tycho Quad

June 23rd, 2008, 10:19 AM

Tom, your my hero. Not a single update to strokeit in over 3 years, and here you are fulfilling mine hours after I make them!

Speaking of bugs, I think I've found another one. I can't get an alt-tab gesture to work with mouse buttons. Ultimately I'd like to have it work like my tab switching gesture, but that might be difficult as you have to hold down the alt and press the tab key many times before getting to the window you want

This would go above and beyond what strokeit did for me. I used another program called TaskSwitchXP to perform this action, but compiz is a much better looking solution.

Luffield

June 23rd, 2008, 11:16 AM

Tom, your my hero. Not a single update to strokeit in over 3 years, and here you are fulfilling mine hours after I make them!

Yeah, but easystroke is becoming bloatware lately. It's over 220KB now but it used to be under 135KB, it just ROCKED back then. I remember these times as if they were yesterday...

Seriously, though: easystroke is just brilliant and I think it's not too early to try to get it into the Ubuntu repositories. What's the procedure for this? Should the developers do it or can anyone do it?

geoken

June 23rd, 2008, 10:50 PM

Fantastic app.

reacocard

June 23rd, 2008, 11:37 PM

Sweet app! I compiled it under arch linux and it's working great. Only have a few actions bound right now but I'm sure I'll add more. :D

Tom Jaeger

June 23rd, 2008, 11:54 PM

Speaking of bugs, I think I've found another one. I can't get an alt-tab gesture to work with mouse buttons. Ultimately I'd like to have it work like my tab switching gesture, but that might be difficult as you have to hold down the alt and press the tab key many times before getting to the window you want

This one is harder than it sounds: Keeping Alt pressed between clicks wouldn't be a too difficult (although I'm not doing it atm), the problem is somewhere else: In order to show the alt-tab dialog compiz needs to grab the server, which fails since easystroke's button grab is still active (because there are still buttons pressed down).

I still have a trick up my sleeve, but it's a little risky: Using Xtest to trick the server into thinking the buttons are actually up (which really isn't as big a deal as it seems) and then relying on Xinput to report to us the actual button up event. If that fails (I'm mainly worried about hot-plugging here, which Xinput doesn't handle well) the user might be stuck with easystroke forever in alt-tab mode.
That said, I have a few ideas how to remedy the situation, but it'll take me a little bit longer to implement it this time.

Tom Jaeger

June 24th, 2008, 01:36 AM

If that fails (I'm mainly worried about hot-plugging here, which Xinput doesn't handle well) the user might be stuck with easystroke forever in alt-tab mode.
That said, I have a few ideas how to remedy the situation, but it'll take me a little bit longer to implement it this time.

Okay, this went faster than I expected: The solution is to check if xinput is working correctly as opposed to checking if it is available (and just to be safe, I'll probably still make it possible to get out of there by pressing Esc).

I've attached the latest version, which enables alt-tab switching in compiz. As always, it's highly experimental. (And I'm back to ~135kB again, forgot to strip the symbols last time...)

Tycho Quad

June 24th, 2008, 08:52 AM

I've already made you my hero, haven't I? How about God then? My own personal Deity?

EDIT: Sorry, I think I've found another bug in relation to the new features. Whenever I perform a tab or window switching gesture, it performs it twice for the first one, however, every subsequent press in that same gesture only triggers it once like it should.

Tom Jaeger

June 26th, 2008, 08:11 AM

EDIT: Sorry, I think I've found another bug in relation to the new features. Whenever I perform a tab or window switching gesture, it performs it twice for the first one, however, every subsequent press in that same gesture only triggers it once like it should.

I've reworked much of the event handling code and the problem you were describing should be fixed now. There's also been a few user-visible changes, most notably the introduction of a 'button' action. I have no idea how to call the checkbox that controls whether easystroke will care about the stroke leading up to advanced gestures, so it's called '???' for now. Feature-wise, I think this is it for 0.1.3, but it needs a little bit more testing first since the implementation of the new features is quite complex. The plan for 0.2.0 is to enable application-dependent behavior in an unobtrusive way via tags.

Oh, and one last thing: The configuration file format has changed in the development tree, which means that once you've switched to the attached version, you can't go back. It's probably safest to backup the configuration directory first.

Luffield

June 29th, 2008, 05:47 AM

Tom Jaeger

June 30th, 2008, 09:23 PM

Tom, on my machine the latest version is causing right-click to stop working altogether after a few minutes of use. I have to end the easystroke process to get the right button active again.
Can someone confirm this?

Sorry, I haven't seen this problem before. There've been some more changes to the event handling code lately, so I've attached the latest version, which might fix this, but it's a long shot. I still need to make some changes to the code in order to be able to effectively debug issues like this. This is going to have to wait a little bit since I'm severely jetlagged at the moment.

Edit: Attachment added

Luffield

June 30th, 2008, 11:56 PM

No worries Tom.
I can tell that your jetlag is really severe by the fact that you forgot to attach the file :D
Anyway, take your time. I know it will be worth the wait.

Tom Jaeger

July 5th, 2008, 03:16 AM

Luffield, sorry to keep you waiting so long, I've been busy with other things this week. Anyway, I've reworked the event handling code once again and I'm attaching the lasted development version. Since it should be more or less functionally equivalent to the last version I posted here, I'd expect your bug to still be present. I'd greatly appreciate it if you could give this version a try and if you're still having the same issues, I'd be very interested in the last 10-20 lines of the output of "./easystroke -vv" before you start experiencing the problem. It also might be helpful to know if the issue is present if xinput is disabled, that is if the program is invoked as "./easystroke -x".

Thank you,
Tom

pibach

July 5th, 2008, 11:00 AM

Tom, thanks a lot for this little app! I have been looking for long to find a proper replacement for StrokeIt. Investigated wavy, xstroke, chickenscratch and gestikk. The latter one seemed to be most suitable and it is nice but suffers from the problems you mentioned as it is not meant to be for tablet pen input but for regular mouse usage.

I had 3 minor issue I ran into using easystroke. First one is that I don't see easystroke as a menu entry (I am using Xubuntu on a ThinkPad x61t). So I cannot start it from the GUI but only from a terminal.
Secondly, I could not store gestures as the gesture file belongs to root, so I had to change file permissions.
Thirdly, when I right click the easystroke taskbar icon, the context menu shows up at a little weird offset position.

Of course these issues are minor and easystroke works great here. I have defined a couple of gestures including:

* minimize and close window
* cut, copy, paste
* delete, backspace, space, return
* forward, backward, new tab, close tab for Firefox
* launchers for cellwriter and some other tools

works great so far, accurate and lag free, and makes a whole different tablet experience

Now I am wondering whether it is possible to define a full blown xstroke/graffiti-like alphabet to input text? Anybody did that?

Other questions include:
* is it possible to share the gesture definition with other users? Or have a default gestures file included. This might be very helpful for all first-time users of gesture software and easystroke.
* is it possible to issue a right-click, Ctrl-Click, Shift click, Alt-click to the gesture starting position? This would be great (never succeeded in doing this with StrokeIt).

best regards,

Peter

Tom Jaeger

July 5th, 2008, 07:26 PM

Thanks for you comments, Peter.

I had 3 minor issue I ran into using easystroke. First one is that I don't see easystroke as a menu entry (I am using Xubuntu on a ThinkPad x61t). So I cannot start it from the GUI but only from a terminal.

That seems easy enough, I'll be sure to include a proper .desktop file in the next release.

Secondly, I could not store gestures as the gesture file belongs to root, so I had to change file permissions.

I have no idea what's going on here. I'm using the standard C++ functions to open and write the config files, so they should get the default permissions. Is this reproducible?

Thirdly, when I right click the easystroke taskbar icon, the context menu shows up at a little weird offset position.

Thanks, I hadn't even noticed that. Turned out to be trivial to fix. Now it would be nice if GTK could be configured to always open menus a few pixels away from the mouse to avoid accidentally activating the first menu item.
[/QUOTE]

Other questions include:
* is it possible to share the gesture definition with other users? Or have a default gestures file included. This might be very helpful for all first-time users of gesture software and easystroke.

I've thought about that but it's not quite clear to me what the right user interface would be. One idea is to somehow use tags (which I will implement soon anyway to allow application-dependent behavior) in order not to clutter up the UI too much. I'll get to it eventually, but it's not the highest priority right now.

* is it possible to issue a right-click, Ctrl-Click, Shift click, Alt-click to the gesture starting position? This would be great (never succeeded in doing this with StrokeIt).

Emulating clicks is already possible in the development version, however, the click will originate at the position where the stroke ended. This seems to be a limitation of XTest, which can't move the pointer and issue the click atomically, so 4 times out of 5, the tablet pen causes the mouse to move back before we get a chance to send the click. It doesn't even seem to work well with a mouse. I've tried to send the clicks directly to the active application via XSendEvent, but that didn't work either, presumably because gtk ignores fake button events.

Luffield

July 5th, 2008, 07:41 PM

Thanks for the update Tom. I'm afraid it will be a few days before I'm able to test it - I have some problems with my ISP and I can't connect to the internet from Ubuntu at the moment. I'm not sure I'll be able to test the current version before you release the next one :)

pibach

July 5th, 2008, 09:01 PM

Thanks, Tom again for the detailed answers.

I think I got the purpose of the "Matrix" feature: As long as there are no >70% similarities, you can define as many gestures as you like, with high correct recognition probability. So I'll going to try this for a full alphabet...

Still I have some issues:

* I am starting easystroke via autostart entry. But then x11 crashes when I do a gesture, everything freezes. To fix this I go to console (CTRL-ALT-F1) and do a killall easystroke.
But when I start easystroke from terminal it does not crash. Weird. But there I get the error : "XError: BadWindow (invalid Window parameter): request_code=20". Any hint?

pibach

July 6th, 2008, 12:21 AM

Tom Jaeger

July 6th, 2008, 02:29 AM

Gotta make this quick, I don't have much time.

Thanks, Tom again for the detailed answers.
* I am starting easystroke via autostart entry. But then x11 crashes when I do a gesture, everything freezes. To fix this I go to console (CTRL-ALT-F1) and do a killall easystroke.

I can't reproduce this one. If you're using the latest development version, the output of "easystroke -vv" would be very helpful. In the future you will just be able to press Esc to get everything back to normal, but that's not implemented yet. In any case, this is a bug that should be fixed.

But when I start easystroke from terminal it does not crash. Weird. But there I get the error : "XError: BadWindow (invalid Window parameter): request_code=20". Any hint?
Probably not a big deal, this usually means that a window disappeared right after you moved the mouse into it, so we get an error when easystroke queries the window.
[/QUOTE]

Tom Jaeger

July 6th, 2008, 02:50 AM

I have defined a full alphabet. I am using it to write this post. Works well so far. My observations:

* works fine and convenient on left click. If I use it on right click it becomes cumbersome after a while. Solution could be either a) a gesture to switch the click options b) on/off toggle c) ingnore if gestures start on GUI elements (e.g. scrollbars). ritePen from evernote does these things pretty well on Windows.

I'll think about it. Scrollbar detection will probably require a good amount of guess-work, I'm not sure how reliable that can be. You're best bet right now are "Ignore" actions which will simply ignore the next stroke.

* I cannot have a shift gesture, right? This might be helpful. Otherwise one needs to define a second set of upper case letters.

It should be pretty easy to rig something up with shell scripts and xte. I'll elaborate on that later.

* when I write in the URL bar of Firefox, then the popup listbox steals focus. I have the same issue with xvkbd, so every key must be enterd twice, quite annoying. GOK (Gnome Onscreen Keyboard) somehow has solved this problem.

cellwriter has the same issue (and I actually have a patch that solves it that I need to send to the author sometime). These programs have a much easier time working around this problem since their windows select on Button events and they will still receive the ButtonUp event. easystroke uses a button grab, which won't get any events if it misses the Down event. I need to explore whether I can use an Xinput workaround here, but that might just be too hacky - even for my taste.

* the visual feedback in the taskbar icon does not help much. Would be better to visualize the actual recognized prototype there. And make it red if first and second recognition choices are too close or first choice is below threshold. I still get some wrong classifications.

I know, the recognition algorithm could definitely benefit from a complete redesign. It's one of those things that will happen eventually, but probably not too soon.

* still it has some slight performance issues when you get going to write fast. Also the windows and panel get flickering.
I think what you're seeing here are the delays I put in place to reduce flickering, which I believe is mostly a compiz issue. It'd probably be a good idea to make these delays a little shorter - especially since they don't eliminate flickering completely anyway. Eventually, I'd like to draw the strokes using a more modern extension like XRender (or whatever it is that compiz' annotate plugin is using, but once again, this is kind of low priority.

pibach

July 6th, 2008, 03:17 PM

Tom, I highly appreciate that you take this much time to answer everything in that detail.

I agree that the above issues are low priority as easystroke works fine. It makes my tablet a lot more usable. This little tool has the potential to become a killer app for all tablet users. What is required most is only a bit of promotion and some more specific use-case advices. BTW: here (http://think-wiki.org/index.php?title=TabletBuntu) I am collection everything for a "TabletBuntu" Edition together with some friends from the ThinkPad Community (in German).

Some more thoughts:

* is there a good way to define macros (as ritePen 3.0 does)? For example "wkr" Gesture Stroke should expand to "with kind regards". Would you do this by a sendevent command?

Edit: I did define some macros successfully using xte command plus str option. In the above example just place in the command field: xte "str with kind regards". Works great!

Edit2: Regarding flickering: I am not using compiz but XFCE without compositing. When I switch on the built in composition, flickering is gone! (Unfortunately, I get the old scroll performance problems on complex Web pages again then)

Edit3:
* Regarding the crash I could pinpoint it: it occurs if I do a gesture associated to launch xvkbd. This allway crashes. Another gesture launching cellwriter only crashes from time to time. All other gestures I have defined work fine.

* regarding the activation button: what about when placing the cursor into an input field, then around this cursor position all strokes activate on left click? This would be similar to MacOS inking.

Tom Jaeger

July 7th, 2008, 10:48 PM

BTW: here (http://think-wiki.org/index.php?title=TabletBuntu) I am collection everything for a "TabletBuntu" Edition together with some friends from the ThinkPad Community (in German).

This is great. I have an x61t, too.

Edit: I did define some macros successfully using xte command plus str option. In the above example just place in the command field: xte "str with kind regards". Works great!

Seems like good way of accomplishing this sort of thing. I'll mention it in the documentation. As for defining actions consisting of multiple strokes, I might introduce a simple state-based model (since we will need a concept of state for application-dependent gestures anyway), but I somehow resist the idea of introducing full-fleged scripting capabilities.

Edit2: Regarding flickering: I am not using compiz but XFCE without compositing. When I switch on the built in composition, flickering is gone! (Unfortunately, I get the old scroll performance problems on complex Web pages again then)

That's odd, I'm don't see flickering in metacity.
The 'Standard' setting in Preferences/Appearance/Method... might work better for you, but it will prevent screen updates until the gesture is over.

Edit3:
* Regarding the crash I could pinpoint it: it occurs if I do a gesture associated to launch xvkbd. This allway crashes. Another gesture launching cellwriter only crashes from time to time. All other gestures I have defined work fine.

Fixed. These weren't really crashes, I previously used the system() call, which would wait for the application to finish before continuing. Of course you can easily walk around this by appending '&' to your command if you don't want to switch to the development version.

* regarding the activation button: what about when placing the cursor into an input field, then around this cursor position all strokes activate on left click? This would be similar to MacOS inking.
This is a great idea and I shouldn't give up on it too early, but it might not be possible: The X server is not aware of what widgets windows consist of, so there's no way to tell where the text input fields are.

EDIT: It looks like I might be able to abuse the XDND drag and drop protocol, which is based around the idea that the source application must grab the server in order track the pointer outside its window and in order to change the pointer, so the target application can't get any mouse events and has to rely on the source to provide it with the necessary information. X is a mess.

As for defining a "shift gesture" that causes the next character to be upper case, I would associate the command "touch /tmp/shift" with the shift gesture and then use the command "/path/to/key ..." to press the keys where key is the following shell script (untested):

#!/bin/bash
if [ -f /tmp/shift ]
then
xte "keydown Shift_L" "key $1" "keyup Shift_L"
rm -f /tmp/shift
else
xte "key $1"
fi

when I write in the URL bar of Firefox, then the popup listbox steals focus.
Okay, I've taken the plunge and allowed gestures to be initiated as long as we are receiving XInput events. This seems to work better than I expected, but the fundamental problem remains: If the server is grabbed (usually because an app is showing a menu), the app will still receive the click event.

I've attached another snapshot. Again, there've been lots of changes under the hood. If easystoke is acting weird, you should be able to fix things by pressing escape now (but unfortunately, that means that gestures sending an Escape key will not work for the time being).

pibach

July 8th, 2008, 01:47 AM

Great! I can confirm that with new snapshot:

* Focus issue: I can enter gestures. But as you explained, app receives the click. But when I write on an area that does not do anything on the click it works fine. In my case I use easystroke on right click, and Firefox URL list do not react on right click. So as long as I start gesturing within the popup list, there is no interference. Working. if you use left click for easystroke it conflicts with grab&drag scrolling. If you disable that, it steals focus...

* "crash": fixed, I can invoke xvkbd. Working.

* flickering: still get flickering on "standard" mode, but that seems to be normal (?). When I choose Xshape it works fine without flickering. Still there seems to be sligth performance issues, as when I do gestures very fast, first part sometimes is not correctly drawn and recognised.

As easystroke works so nicely I experiment using it on left click and did define a "pause" gesture (should be explained in the Documentation as this is very important!). But for more convenient working on left click without interference I have the following idea:

* If you start gesturing and do not move the pen for a threshold (e.g. 500milliseconds, 5 pixels), then ignore gesture mode and transmit input to the underlaying window. This way when you do a scrollbar slide, a drag&grad scroll slide, or some text selection slide, for example, you only get some short delay but can go ahead afterwards.

* nevertheless there should be an easy way to change button option. Then you could switch to right button by a gesture (or by window context in future versions, e.g., no left click gestures over cellwriter). Also there should be a "deactivate" checkbox that can be automatically chosen (by a script) if changing back to laptop mode. To reactivate, one could tab the taskbar icon (and have options on right click menu).

Tom, what do you think about these ideas?

* another nice thing would be to place cellwriter, which I invoke by a geture, nearby the current cursor position (similar as Vista does with the TIP). Is there a way to do this?

Tom Jaeger

July 8th, 2008, 03:42 AM

Thanks, Peter, your feedback is very helpful and your ideas are great.

Great! I can confirm that with new snapshot:
* flickering: still get flickering on "standard" mode, but that seems to be normal (?). When I choose Xshape it works fine without flickering. Still there seems to be sligth performance issues, as when I do gestures very fast, first part sometimes is not correctly drawn and recognised.

Yes, the first part of strokes is not drawn correctly... Totally forgot about that. It should be easy enough to fix, after all, all the necessary information is already there. Both methods have severe drawbacks (XShape's high CPU consumption, for example) and ultimately need to be replaced by something more modern.

* If you start gesturing and do not move the pen for a threshold (e.g. 500milliseconds, 5 pixels), then ignore gesture mode and transmit input to the underlaying window. This way when you do a scrollbar slide, a drag&grad scroll slide, or some text selection slide, for example, you only get some short delay but can go ahead afterwards.

Great idea, especially for devices that only have one "button" such as passive digitizers/touch screens. It shouldn't be too hard since we can have the X server do most of the work for us here. It requires just a little bit more bookkeeping...
This is definitely 0.1.3 material. Most of the rest will have to wait for 0.2.x.

For a tablet pen, it'll probably be more convenient to assign a left click to the left-click - middle-click sequence -- another hidden feature. To set this up, just define a gesture where you first do a left click and then press the side switch while the pen is still on the tablet and then assign the action Button/Button1 to it.

* nevertheless there should be an easy way to change button option. Then you could switch to right button by a gesture (or by window context in future versions, e.g., no left click gestures over cellwriter).

I agree, the hard part here is getting the UI right. I'll need to think about it a little more.

Also there should be a "deactivate" checkbox that can be automatically chosen (by a script) if changing back to laptop mode. To reactivate, one could tab the taskbar icon (and have options on right click menu).

Deaktivate is something I've been meaning to implement for a while now. But I'll put it in the menu -- apps that violate the unspoken rule that a left-click on a tray icon should open a window always drive me nuts.

* another nice thing would be to place cellwriter, which I invoke by a geture, nearby the current cursor position (similar as Vista does with the TIP). Is there a way to do this?

I investigated this at some point. Of course it must be done in cellwriter -- if I remember correctly, cellwriter already has a fifo that it is listening to - so it shouldn't be too intrusive a patch.

The plan is to implement the things that are relatively simple now and then get some more testing done before the release of 0.1.3. And target all the features that require state in one form or another for 0.2.x

pibach

July 8th, 2008, 06:13 PM

For a tablet pen, it'll probably be more convenient to assign a left click to the left-click - middle-click sequence -- another hidden feature.
Sounds great. But how do you use this feature exactly? Did you redefine the normal left click to be always a right click (invoking easystroke directly), and then get the left click by doing the above?
I think we should record a video to explain the best usage scenarios as it get quite abstract/complex here...

But I'll put it in the menu -- apps that violate the unspoken rule that a left-click on a tray icon should open a window always drive me nuts.
Hm. This might take too long. A very quick activation is important! Deactivation then can be done by a gesture.

I investigated this at some point. Of course it must be done in cellwriter -- if I remember correctly, cellwriter already has a fifo that it is listening to - so it shouldn't be too intrusive a patch.
Isn't there something simpler and more generic, e.g., send a move window x,y event to the window manager?

Tom Jaeger

July 9th, 2008, 08:17 PM

Sounds great. But how do you use this feature exactly? Did you redefine the normal left click to be always a right click (invoking easystroke directly), and then get the left click by doing the above?

No, the normal left click is still a left click, it's only when you press two buttons at the same time that advanced gestures kick in.

Hm. This might take too long. A very quick activation is important! Deactivation then can be done by a gesture.

I can't say that I understand the use case, but I'd suggest using a tablet button then. Easystroke being activated and deactivated seemingly at random because someone clicked on the tray icon might easily confuse users.

Isn't there something simpler and more generic, e.g., send a move window x,y event to the window manager?
Moving the window isn't a big deal at all if the window is already there, but I don't see how I would make the cellwriter window appear in the first place.

A new snapshot is attached. I've implemented the abort-the-stroke-if-the-mouse-isn't-moved feature, which it turns out requires Xinput. I also added a workaround for clicks registering twice when the server is grabbed, but I've decided that the whole accepting gestures when a menu is shown thing is too quirky and confusing, so it has to explicitely be enabled in the preferences now. In order to solve the problem that clicks on the taskbar steal focus if the gesture button is Button1, I'm now using NET_WM to transfer focus to the app that the user clicked on (EDIT: I'm using yet another method now and I've removed the option again). The problem is that some window managers do stupid things if they are told to activate a window (compiz, but not metacity raises the window in addition to giving it focus even if there is no Raise-On-Click policy), so I've added an option to disable giving the focus to the application where a stroke originates.

pibach

July 9th, 2008, 11:52 PM

Tom, I am going to test new version asap.

In the mean time I stumbled over some things:

* Ubuntu MID: this seems to have "Finger Gestures". Any information on how this works?

* Tilt detection: on Wacom digitizers you can query the pen tilt. Might be an option for gestures. See this Nokia Paper (http://whitepapers.silicon.com/0,39024759,60317849p,00.htm).

* I also have a Nokia n81O. Do you think easystroke will be suitable for that device?

Tom Jaeger

July 10th, 2008, 02:10 AM

* Ubuntu MID: this seems to have "Finger Gestures". Any information on how this works?

From looking at youtube videos, it appears to be similar to Vista's flick gestures. But their apps have custom UIs anyway, so this is not directly comparable.

* Tilt detection: on Wacom digitizers you can query the pen tilt. Might be an option for gestures. See this .
Direct link to the Paper (http://research.nokia.com/files/Tilt%20MenuCHI2008.pdf)
Unfortunately, we don't get tilt information on Tablet PCs (though they might have it on the more expensive wacom tablets). The only additional information that we can get through xinput is pressure and proximity (i.e. whether the pen or eraser is "in range"). I've thought about whether we can make use of this information, but I haven't come up with anything good. Maybe a "proximity click", i.e. moving the pen quickly in and out of the range, but that's not very impressive.

Tom Jaeger

July 10th, 2008, 06:45 PM

Here's an idea for pressure: Abort the gesture and pass the click to the application when high pressure is exerted on the pen.
The attached version is identical to the one above, except that strokes are aborted if pressure exceeds 192 (out of 255). Works pretty well, I think, though it might need a configuration option, since people's pens and writing habits are different.

pibach

July 11th, 2008, 12:06 AM

pressure sensitive stroke abortion works great! Good idea. Alsoabort-the-stroke-if-the-mouse-isn't-moved feature works nicely!

Now I prefers easystrke on left click. Then, biggest problem ist that over cellwriter, xournal and gimp easystroke should change to work on right click.

Tom> "so I've added an option to disable giving the focus to the application where a stroke originates"

I cannot reproduce this here. Does not work on left click.

with these improvements easystroke became my most most important tabet tool. I'll test it on my n810 soon...

pibach

July 11th, 2008, 12:13 AM

pressure sensitive stroke abortion works great! Good idea. Also abort-the-stroke-if-the-mouse-isn't-moved feature works nicely!

Now I would prefers easystroke on left click. Then, biggest problem is that over cellwriter, xournal and gimp easystroke should change to work on right click.
Edit: is the 'add exeption' option in the experimental mode intended for this?

Tom> "so I've added an option to disable giving the focus to the application where a stroke originates"

I cannot reproduce this here. Does not work on left click. Firefox continues to steal focus (when I am on left click in easystroke).

Edit: Accuracy issues at gesture begning seem to be fixed. Also xshape visualisation works fine for me, no flickering.

With these improvements easystroke became my most most important tabet tool. I'll test it on my n810 soon...

Tom Jaeger

July 11th, 2008, 09:04 PM

Now I would prefers easystroke on left click. Then, biggest problem is that over cellwriter, xournal and gimp easystroke should change to work on right click.

I'll start working on that once 0.1.3 is released (or maybe I'll call it 0.2.0). Until then, as you said, adding those applications to the exception list (not experimental, btw) will at least give you your left button back, but gestures won't work in those windows.

I cannot reproduce this here. Does not work on left click. Firefox continues to steal focus (when I am on left click in easystroke).

Hmm, are you sure you set the "Accept gestures when menus are shown" option?

I had another idea why the advanced gestures feature is not working for you. It only works if the wacom driver's TPCButton (http://easystroke.wiki.sourceforge.net/TipsAndTricks) option is enabled. Speaking of advanced gestures, I've just implemented a (pretty insane) workaround for the problem with the ordering of two button events coming in at the same time, so it won't be necessary to patch the wacom driver anymore to take full advantage of advanced gestures.

pibach

July 11th, 2008, 11:25 PM

I'll start working on that once 0.1.3 is released (or maybe I'll call it 0.2.0). Until then, as you said, adding those applications to the exception list (not experimental, btw) will at least give you your left button back, but gestures won't work in those windows.

How do I add ecxeptions (I did not succeed doing that). I did not find anything in the documentation.

Hmm, are you sure you set the "Accept gestures when menus are shown" option?

yes. I did enable this. Still when I am on left click in easystroke, Firefox kicks in and I cannot continue writing with easystroke in the URL bar.

I had another idea why the advanced gestures feature is not working for you. It only works if the wacom driver's TPCButton (http://easystroke.wiki.sourceforge.net/TipsAndTricks) option is enabled.
I have been playing with TPCButton option. This hoovering is nice if you place gestures on left click.

Regarding advanced gestures I have to admit that I did not understand the documentation about using that for alt-tab-sitching with a pen.

Tom Jaeger

July 12th, 2008, 02:09 AM

How do I add ecxeptions (I did not succeed doing that). I did not find anything in the documentation.

Just click on "Add Exception" and then on a window belonging to the application that you want to exclude.

yes. I did enable this. Still when I am on left click in easystroke, Firefox kicks in and I cannot continue writing with easystroke in the URL bar.

Weird. This works here. Maybe there's some WM interaction. What' the name of the window manager that are you using? The output of ./easystroke -vv might also be helpful.

Regarding advanced gestures I have to admit that I did not understand the documentation about using that for alt-tab-sitching with a pen.
That was an example of what you can do with the scroll wheel of a mouse. But you can do the same thing with the pen: Draw the gesture and then hold down "the other" button.

Attaching the latest snapshot. We're getting closer to a release now, I hope.

Tycho Quad

July 12th, 2008, 08:29 AM

You have don excellent work with easystroke Tom, I've never before met a developer who is so willing to listen to what people want out of a program, as well as being so quick to implement said features. You have already nailed what I want out of this program, and I look forward to seeing what comes in the future, features that I never would have thought of.

pibach

July 12th, 2008, 10:07 AM

Just click on "Add Exception" and then on a window belonging to the application that you want to exclude.

Ok, this works in Gnome. But I am using XFCE (Windowmanager ist Xfwm4).

The main reason why I choose XFCE is its superior GUI performance. Might result from my SXGA+ resolution. I get unbearable slow scrolling, e.g., in Firefox, when I am running Gnome. I tested with or without Compiz. Without it is a lot faster, but still Xfwm4 apparently renders a lot faster than Metacity or the KDE window managers do. Don't know why, but apparently here are still severe performance bugs in xorg 7.2 plus Intel drivers and font rendering. I also investigated some options (Exa versus XAA acceleration, Intel batch). No solution yet. Do you know of any? See also http://www.cworth.org/talks/lca_2008/

Tom Jaeger

July 12th, 2008, 11:02 AM

I get unbearable slow scrolling, e.g., in Firefox, when I am running Gnome. I tested with or without Compiz. Without it is a lot faster, but still Xfwm4 apparently renders a lot faster than Metacity or the KDE window managers do.

There's something fishy here. I'm not seeing this kind of slowness in compiz. Sure, it's a little less smooth than on vista, and xfwm might be slightly better, but it's nowhere near unbearable. I'll attach my xorg.conf

As for the exception problem, I can reproduce this with xfwm, so expect a solution soon. The weird part is that xprop is working fine with xfwm and I thought I ripped the window picking code from the same codebase.

pibach

July 12th, 2008, 11:37 AM

Tom, do you have the SXGA+ resolution? Do you use antialiasing (subpixel hinting)? Do you use Firefox smooth scrolling (pixel wise scrolling)? Then you should see the same sluggish scrolling performance on complex web pages (all other GUI things perform well, it is only a font rendering issue in combination with high resolution and antialiazing). Whether to call it "unbearable" or "tolerable" might be a matter of taste though. BTW: XFCE without compositing is surely more responsive than Windows.

Tom Jaeger

July 12th, 2008, 12:24 PM

I do have the high-resolution display and I am using subpixel smoothing. There is a slight slowdown in compiz compared with, say xfwm, but it's barely noticeable in practice. It's not like in the hardy betas where they enabled EXA without enabling greedy migration heuristics.

I've fixed exceptions by moving over to the latest dsimple.c code, apparently I was using an older version.

Tom Jaeger

July 12th, 2008, 12:25 PM

Tycho Quad, thanks, this would never be possible without the high-quality feedback I'm getting in this forum.

pibach

July 12th, 2008, 01:28 PM

I've fixed exceptions by moving over to the latest dsimple.c code, apparently I was using an older version.

Working here on XFCE!

Is is incredible how fast you are implementing these changes and fixes. Thanks a lot!

Regarding sluggish scroll: Yes, this "greedy migration heuristics" improved my performance a 40%. Still there seem to be font rendering issues (see the link I posted above). It might also be related to Java Script/Flash implementation in Firefox as I do measure higher CPU load on some Web pages in Firefox compared to Opera for example. There are pages where scrolling almost freezes if I enable compositing.

Edit: BTW, why is you post count showing "0 posts"??

pibach

July 12th, 2008, 03:07 PM

Tom, I did some usability test with newest snapshot. And it mostly works fine for me, when I keep easystroke on right click. Biggest issue so far is that I have to enter strokes over list boxes such that the window does not accidentially act on the right click. Only for writing longer with my Graffiti-like strokes holding the right button gets a bit cumbersome. Not a big thing as I usually activate cellwriter for that by a quick easystroke gesture. But cellwriter has some other usability issues and I am wondering how one could best combine the benefits of both tools.

When I change to have easystroke on left click, I loose the grab&drag scroll feature in Firefox. Also I used to open links in new tabs by engaging Quickdrag extension. As you usually do this in a lets say 10:3:1 ratio (scroll:quickdrag link:gesture) you would like to keep scroll&quickdrag on left button in this use case. Also it is a little bit counter intuitive that you cannot slide GUI elements, such as the scrollbar oder the window titlebar.

Moreover it would be nice if easystroke scroll option could serve as a general scroll method similar to g&d scrolling in Firefox but application independent. Currently the scroll option is nice, but too cumbersome (one issue is that it requires re-activation for every scroll). Maybe you could add a scroll mode to easystroke that is activated on a specific button.

Threfore, the GUI could have an additional "scroll button" to be selected beneath the "gesture button". Ideally the button assignment is application specific so you would need two button parameters for the "exceptions" list that indicate the different assignment. Thus, for each item in that list you'll have an individual gesture button and scroll button different to the default setting. Setting both buttons to "none" would completely disable easystroke then for this application.

And finally a specific gesture then could switch between the two button assignments that are currently active. This could be solved in the easystroke GUI by an additional "switch buttons" action type. These modes should be visualised by the task bar icon.

What do you think about that?

pibach

July 12th, 2008, 07:01 PM

Sluggish scroll in Firefox:
I again tested Firefox versus Opera and it seems to be a more Firefox/xorg related issue. Try for example scrolling this long Webpage: http://forum.notebookreview.com/showthread.php?t=144783
which lets my Firefox almost freeze. Opera does scroll it without problems. If you look at process TOP, you can see that most load goes to xorg, indicating some issue between FF, xorg and Intel drivers, probably. Here is also some related info in this bug report (https://bugs.launchpad.net/ubuntu/+source/xserver-xorg-video-intel/+bug/177492).

Melodic_BM

July 12th, 2008, 10:45 PM

I cannot make easystroke. It's telling me that there aren't enough parameters for "XFreeCursor".

Islington

July 13th, 2008, 02:32 AM

Tom: I Love You.

This tool is amazing( written on a tablet comp)

pibach

July 14th, 2008, 01:02 AM

Sluggish scroll in Firefox
This is off topic but I wanted to tell that I found out the problem of the sluggish scrolling after deeper investigation: If you set mousewheel.withnokey.numlines to 1 (for smother scrolling), engage full page zoom and also enable smooth scrolling, then there are more TrackPoint events generated than Firefox/xorg can handle on time. This results in buffer overload and permanent pre-emption of processes which freezes the scrolling. According to my testing this also happens on Windows and other programs including Opera. Anyway, there additionally seem to be Intel driver issues as load is too high for this kind of 2D operation. But seemingly Windows drivers suffer from exactly the same problem.

Tom Jaeger

July 14th, 2008, 08:02 PM

Moreover it would be nice if easystroke scroll option could serve as a general scroll method similar to g&d scrolling in Firefox but application independent. Currently the scroll option is nice, but too cumbersome (one issue is that it requires re-activation for every scroll). Maybe you could add a scroll mode to easystroke that is activated on a specific button.

This is not a bad idea. Would be a good use for the eraser. The way I see it, this requires two things: (1) support for multiple gesture buttons and (2) allowing some of them to fire on button press instead of button release. (2) is essentially trivial, but (1) is more complicated than it might seem. The backend code has recently been moving in the direction of (1) for unrelated reasons, so I should eventually get around to implementing this.

Until then, you can get very similar functionality by assigning 'scroll' to the Button1 advanced gesture (without motion). But be sure to use the attached snapshot, advanced-gesture scroll was pretty unreliable with the last one.

Ideally the button assignment is application specific so you would need two button parameters for the "exceptions" list that indicate the different assignment.

The plan is to make easystroke's behavior dependent on some sort of state that incorporates the current application and a user-definable element, so all of this will be possible. This idea is appealing because we'll be able to use essentially the same user interface for actions and preferences. I haven't figured out the details yet, but it's going to be sweet.

If you set mousewheel.withnokey.numlines to 1 (for smother scrolling), engage full page zoom and also enable smooth scrolling, then there are more TrackPoint events generated than Firefox/xorg can handle on time.

Ah, that explains it. I have disabled smooth scrolling and I don't usually use the track point to scroll in firefox. But you're right, scrolling under those circumstances is annoying.

Tom Jaeger

July 14th, 2008, 08:08 PM

I cannot make easystroke. It's telling me that there aren't enough parameters for "XFreeCursor".

Sorry about that. I don't always compile before checking stuff in. It should work now. If this should happens again, you should be able to fix it by 'darcs unpull'ing the last patch.

pibach

July 14th, 2008, 08:23 PM

This is not a bad idea. Would be a good use for the eraser.

The eraser button is hardly usable on our ThinkPads. Therefore best might be to activate scroll mode on right click (i.e., hovering, without left click) and then moving up/down/left/right. Also the scroll behavior ideally could be a bit different to current scroll, by having a speed up the further the pen is moved away from originating position.

The plan is to make easystroke's behavior dependent on some sort of state that incorporates the current application and a user-definable element, so all of this will be possible. This idea is appealing because we'll be able to use essentially the same user interface for actions and preferences. I haven't figured out the details yet, but it's going to be sweet.

sounds great! Looking forward for this.

Any news about (auto-) disabling easystroke on Laptop Mode?

Islington

July 14th, 2008, 10:02 PM

is there a way to launch cellwriter & another program at the same time? :confused:

Alt+F2 dialog is pretty useless without cell writer also being present...

Tom Jaeger

July 14th, 2008, 10:53 PM

is there a way to launch cellwriter & another program at the same time? :confused:
Commands are passed to the shell, so "app1; app2" (without quotes) will execute the two apps sequentially and "app1 & app2" will execute them at the same time.

So this is the command that you'd use to start cellwriter and open the Alt+F2 dialog:

cellwriter --show-window & xte 'keydown Alt_L' 'key F2' 'keyup Alt_L'

Tom Jaeger

July 15th, 2008, 12:11 AM

The eraser button is hardly usable on our ThinkPads. Therefore best might be to activate scroll mode on right click (without left click) and then moving up/down/left/right.

How is this better than what is currently possible: have gestures be bound to right click and activate scroll mode by an additional left click? (Try it out, it works really well now)

Also the sroll behavior ideally could be a bit different to current scroll, by having a speed up the further the pen is moved away from originating position.

Yup, the scrolling algorithm could use some improvements. I was thinking of something along the lines of

scroll speed = const * mouse speed ^ c

where c is some suitable exponent between 1 and 2.

Is horizontal scrolling something we need? Obviously, it wouldn't be very hard to implement, but I would imagine that it could be annoying to accidentally trigger it. At the very least, I think horizontal scrolling should be less sensitive than vertical scrolling.

Any news about (auto-) disabling easystroke on Laptop Mode?
The natural thing to do would be to have a fifo that you can write to in order to change easystroke's "state", then you could disable it via a script or a gesture and enable it using a launcher.

Islington

July 15th, 2008, 12:18 AM

Commands are passed to the shell, so "app1; app2" (without quotes) will execute the two apps sequentially and "app1 & app2" will execute them at the same time.

So this is the command that you'd use to start cellwriter and open the Alt+F2 dialog:

cellwriter --show-window & xte 'keydown Alt_L' 'key F2' 'keyup Alt_L'

terrific. I never knew about xte.

pibach

July 15th, 2008, 01:23 AM

How is this better than what is currently possible: have gestures be bound to right click and activate scroll mode by an additional left click? (Try it out, it works really well now)

Yes, you're right, works quite good with new snapshot. Right click+left click to activate scrolling is fine (but don't you think a hovering right click wouldn't be better?). And the new formula with some "accelerating factor" as you suggest should help to better hit targets (while background is scrolling). Would be nice if therefore scrolling could be as smooth as possible, i.e., pixel-wise scrolling (as the scrollbar or G&D scroll extension does). But I guess this might no be easily possible (?). Might also be reasonable to send the deactivation click already to the application?

Is horizontal scrolling something we need?

I don't see this to be on high priority. But there are definitely apps that will benefit from horizontal scrolling. So ideally this option (+sensitivity) would be context specific

BTW: did you contact the accessibility developers? They might be interested to include easystroke into the upsteam.

Islington

July 15th, 2008, 02:20 AM

On the note of (auto-) disabling easystroke. The program crashes whenever I rotate my screen. I am using a Tablet X41 and the rotation scripts that I am using were found here link. (http://liken.otsoa.net/blog/index.php?entry=entry080617-120522)

Running it through terminal I get:

islington@Mu:~$ easystroke
XError: BadWindow (invalid Window parameter): request_code=20
XError: BadWindow (invalid Window parameter): request_code=20
XError: BadWindow (invalid Window parameter): request_code=20
islington@Mu:~$

Melodic_BM

July 15th, 2008, 01:17 PM

Damn: undefined reference to `gui_buffer'
Now - there are packages... I could've seen this earlier. -.-

Tom Jaeger

July 15th, 2008, 02:19 PM

Damn: undefined reference to `gui_buffer'

Fixed.

Tom Jaeger

July 15th, 2008, 02:31 PM

On the note of (auto-) disabling easystroke. The program crashes whenever I rotate my screen. [...]

Running it through terminal I get:

islington@Mu:~$ easystroke
XError: BadWindow (invalid Window parameter): request_code=20
XError: BadWindow (invalid Window parameter): request_code=20
XError: BadWindow (invalid Window parameter): request_code=20
islington@Mu:~$

Really strange. This does not look like a crash, it seems easystroke exited cleanly. (The XError things don't mean anything, I filter most of them out in the development version now). I can't reproduce this issue, but I once had a possibly related problem where easystroke would receive spurious WM_DELETE messages and then dutifully shut down. Rather than try and find the cause of this (it's a really, really strange problem, and could be related to easystroke opening two connections to the X server), I just decided to ignore WM_DELETE.

So the issue might be fixed in the development builds that are attached to some of my posts, if you want to try that out. But be sure to back up your .easystroke directory first: Your actions and prefs will migrate to the new version just fine, but they will be overwritten and you won't be able to go back to the earlier version (most likely it'll just crash). This is a limitation of the boost::serialization library that I use for storage.

Tom Jaeger

July 15th, 2008, 04:15 PM

Yes, you're right, works quite good with new snapshot. Right click+left click to activate scrolling is fine (but don't you think a hovering right click wouldn't be better?).

Personally, I don't, but if we enable actions to be triggered by a single button press, then we might as well do something useful if an additional button is held down. It seems kind of complex, though, so I need to be sure that it's worth the trouble.

Would be nice if therefore scrolling could be as smooth as possible, i.e., pixel-wise scrolling (as the scrollbar or G&D scroll extension does). But I guess this might no be easily possible (?).

I'm afraid not. Scroll wheel events are dumb button presses and apps translate them into a fixed scroll amount. This amount might be configurable for gtk apps, but that's about it.

Might also be reasonable to send the deactivation click already to the application?

I don't understand.

BTW: did you contact the accessibility developers? They might be interested to include easystroke into the upsteam.
The thing about accessibility is that enabling it breaks a few applications and easystroke is one of them: Accessibility is causing deadlocks, even though it uses gtk's own locking operations. This looks like a problem with the atk implementation. [Note to self: Mention this in the docs]

EDIT: I have fixed this now. It turns out that gtk+accessibility doesn't like it at all if its main loop is not run in the main thread. Weird. They don't mention any of this in the GTK+ FAQ (http://library.gnome.org/devel/gtk-faq/stable/x500.html)

So I've changed the scrolling algorithm as previously indicated (with c = 4/3), and it basically works fine, except for the problems with fast scroll speeds that you mentioned earlier, which become really prominent here. Interestingly enough, this fails in different ways depending on whether compositing is enabled or not: In compiz, motion events that the driver can't handle are simply dropped and in metacity/xfwm4, they are queued and the computer becomes unresponsive after scrolling. But I blame the toolkits, they should be able to cope with slow drivers.

pibach

July 18th, 2008, 07:55 PM

Personally, I don't, but if we enable actions to be triggered by a single button press, then we might as well do something useful if an additional button is held down. It seems kind of complex, though, so I need to be sure that it's worth the trouble.
Currently I run easystroke on left click. And enter scroll mode on a simple gesture, this is the gesture I use most of course. It works fine, but still, requires to bring down the pen, do the gesture, lift the pen again, do the scrolling, then deactivate scrolling by bringing the pen back down. Then you need to lift it up again before you can select, for example, a link in the browser. This works, but it is not particularly convenient. The combination of Grab&Drag Scroll plus FireGestures is more convenient, in comparison, but of course does not work OS-wide and application independent. Anyway, a simple and quick way to distinguish gestures from scrolling strokes is very important. The Vista solution to this is to check for 'flick' gestures, where the pen is been lifted after th egesture. But this is resource consuming as it requires permanent tracking of pen movement and it induces some lag. I don't know the best way, but to distinguish scroll from gesture mode would be a possible solution. And from user perspective, a hoovering right click might be the fastest and easiest to do mode switching. For ressistive touch a quick tab on a non-GUI element might be another good option.

I'm afraid not. Scroll wheel events are dumb button presses and apps translate them into a fixed scroll amount. This amount might be configurable for gtk apps, but that's about it.
That's what I've expected. Maybe one could lower the system wide scroll parameter in tablet mode? When I set numlines parmeter in about:config to 1 (instead of the default 3 lines scrolling) it is much smoother - but might lead to the mentioned event-overload problem.

I don't understand.

The problem I see is that scrolling in hoovering mode is too sensitive to directly hit a target (e.g., click-open a link). Probably you could add to your formula an offset parameter. Then, in some range the movement of the pen would not result in scroll, but only if you exceed certain range (e.g. 15mm might be reasonable, best range will need some testing). Within this reach you can precisely hit targets or even do some gesturing (without resulting in a scroll). Then, scroll mode could probably remain permanently active, if these clicks are trasmitted to the app. This might eliminate the need for frequently activating/deactivating scroll mode.

Edit: I gues "offset parameter" is not possible as you cannot remember originating koordinates of a gesture, right? Then probably exponential acceleration in the scroll formula instead of polinomial might be the way to go? Or could you incorporate the speed the pen is moving? And when you end at the border of some scroll area, is there a way to do some continious scroll? Because otherwise you would need to lift pen up an down multiple times to reissue scrolling.

Youst some thoughts, some might be useful. What do you think?

BTW: I switched from Firefox to Epiphany as this browser is more lightweight an does not suffer from the flush()/SQLlight bug that all the time spins up HDD. This way I can surf the web for hours without any HDD activity. And easystroke allows me to conveniently scroll or open links in new tabs by just a gesture. Works great.

So I've changed the scrolling algorithm as previously indicated (with c = 4/3), and it basically works fine.

yes, works fine, huge improvement.

This is a great feature except for the problems with fast scroll speeds that you mentioned earlier, which become really prominent here. Interestingly enough, this fails in different ways depending on whether compositing is enabled or not: In compiz, motion events that the driver can't handle are simply dropped and in metacity/xfwm4, they are queued and the computer becomes unresponsive after scrolling. But I blame the toolkits, they should be able to cope with slow drivers.
Interesting. I think EXA acceleration is part of the problem. As far as I know it obtains only 60% the performance regarding font rendering compared to XAA acceleration. There might be some settings to get this faster. Then these buffer overflows would not appear. But I agree, the window managers/toolkits should be able to handle that. Isn't there a bug report filed?

pibach

July 20th, 2008, 01:01 PM

There a some issues when having easystroke set for left click:

1) easystroke gives unintended focus to apps like cellwriter
2) when I set "accept gestures when menues are shown" then exceptions do not work, e.g., using cellwriter

Edit: with the "Yet another Smooth Scroll" Extension for Firefox you get a smoother scrolling and it accellerates nicely if you want to.

Tom Jaeger

July 21st, 2008, 12:42 AM

I'm a little short on time at the moment, I'll post a more detailed response later this week.

There a some issues when having easystroke set for left click:
1) easystroke gives unintended focus to apps like cellwriter

I'll look into that, I haven't found a way yet to delegate to the window manager the decision of whether to give focus to a specific window. I'll probably end up duplicating more wm functionality.

2) when I set "accept gestures when menues are shown" then exceptions do not work, e.g., using cellwriter

Thanks, should be fixed in the attached snapshot.

pibach

July 21st, 2008, 01:16 AM

Thanks Tom for the updated shnapshot.

Another small issue:

The ignore gesture after some timeout does not work if I stop moving the pen completely (which might be irritating).

Tom Jaeger

July 23rd, 2008, 10:29 PM

I'd like to release version 0.2.0 soon; no new features will go into that version, only bugfixes and minor UI changes. Consider the attached file "easystroke-0.2.0-rc1" as the release candidate. As always, comments are welcome.

The plan for 0.3.x is to introduce some form of state - so that actions can be chosen based on the current application and yes, the name and the type of the currently selected GUI element -- provided accessibility is enabled and the app supports it. My ideas on how to represent this in the GUI are still too half-baked to be discussed here -- but I think I'm going to change the actions list into a tree. The most important requirement is that the user interface not become more complicated to use for users that don't care about the new features.

Speaking of accessibility, if you want to execute more elaborate actions than are possible via simple keystrokes, GUI-testing frameworks such as dogtail (http://people.redhat.com/zcerza/dogtail/) and LDTP (http://ldtp.freedesktop.org/wiki/) might be worth investigating. [Note to self: mention this on the wiki]

The ignore gesture after some timeout does not work if I stop moving the pen completely (which might be irritating).

I've added a timeout to work around this problem. I also needed to generate an additional motion event in order for the app to see the updated cursor position. I also thought about replaying the whole motion history, but XTest is acting weird here - and I can't send out atomic sequences of XTest commands anyway.

1) easystroke gives unintended focus to apps like cellwriter

Fixed - by inspecting the WMHints of the window.

Regarding scrolling, I've decided that I'd eventually like to implement the thing you proposed (albeit in a more general setting) - but this will require major changes GUI-wise and in the backend. For now, I have a different idea: proximity, as implemented in the attached file easystroke-dev. What this does is that it'll stay in scroll or ignore mode until the pen is lifted far enough to be out of range. Also, it'll only scroll if there's a button pressed.

Scroll wheel events are dumb button presses and apps translate them into a fixed scroll amount. This amount might be configurable for gtk apps

I actually don't think anymore that it's configurable. There's no gconf setting for it and I think ~/.gtkrc-2.0 only controls style settings (and it would only be effective after restarting the apps anyway).

But I agree, the window managers/toolkits should be able to handle that. Isn't there a bug report filed?

I'm not aware of any bug reports. An easy fix would be for GtkAdjustment to only send "value-changed" events when idle. But I guess there's always a risk of breaking applications that depend on things being done in a certain order. The much harder part would be setting up a gtk build environment to actually test this. And even then, there are apps like firefox that have a (probably modified) version of gtk statically linked in.

EDIT: Shoot, I had already uploaded the attachments, but firefox crashed. Luckily, it had saved the message, but the attachments were gone...
Anyway, since rc1 had issues with modifiers, here's rc2.

EDIT: New RC (I've merged the changes from easystroke-dev)

pibach

July 25th, 2008, 10:57 AM

I think for a typical usage on right click it is fine to be released. However, for elaborate usage scenarios with graffiti or left click, I have issues:

* when I enter letters in some input area in Firefox, then the list of terms to choose from pops up. The input box looses fokus then, when I gesture somewhere, i.e., option "accept gestures when menus are shown" does not work reliably.
* I have a couple of performance issues: sometimes the gesture is recognized as a click, apparently when I do it too fast. Edit: works if I change pixel settings in "Tread as click if..."
* the start of any gesture line does show up only after some delay. But for efficient usage it would be very important to have this lag free.
* when set to right click, gestures do also let the context menu pop up in some situations

Then there still are usability things:
* the new proximity scroll lock is an interesting idea. However lifting the pen that far away from the screen is one of the most inefficient "gestures"
* then in scroll mode you cannot click links, cannot gesture etc. Wouldn't it be possible, to scroll in hoover mode but still accept clicks/gestures for the application on left click?

Luffield

July 25th, 2008, 11:03 AM

Back on Ubuntu, at last.
RC2 seems to work well on my computer, without any issues. I only have a mouse, no tablet.
Looking forward to try the next version.

Tom Jaeger

July 25th, 2008, 11:35 PM

Thanks for the feedback, guys, it's very much appreciated.

* when I enter letters in some input area in Firefox, then the list of terms to choose from pops up. The input box looses fokus then, when I gesture somewhere, i.e., option "accept gestures when menus are shown" does not work reliably.

The problem is that by the time easystroke receives the click event, it already been transmitted to the application, so it seems that there's nothing I can do about this -- short of what gok is doing: detach all input devices from the X server and then send them via XTest, which I don't think is a good idea. A less evil idea would be to remap the tip to, say button 9 and then fake corresponding button 1 events if we receive button 9 events. I'll have think about this a little more, but I don't expect it to be easy.

* I have a couple of performance issues: sometimes the gesture is recognized as a click, apparently when I do it too fast. Edit: works if I change pixel settings in "Tread as click if..."
* the start of any gesture line does show up only after some delay. But for efficient usage it would be very important to have this lag free.

This is intentional: Everything that stays within r pixels of the original position is interpreted as a click and as long we're not sure that it's a gesture, no trail is shown. I could decrease the default value of r, but I'm not convinced any changes beyond that would make sense.

It would probably a good idea for the wacom driver to release clicks early if pressure is rapidly decreasing in order to prevent motion events from being generated after the pen has been lifted from the surface.

EDIT: You seem to draw your gestures much smaller than I do. Do you think that you would benefit from subpixel accuracy (as in xournal)?

* when set to right click, gestures do also let the context menu pop up in some situations

This one is the only show-stopper in the list. What are the situations? Does this only happen if the menu workaround is enabled? Can you send me the output of "./easystroke -vvv" when it happens?

* then in scroll mode you cannot click links, cannot gesture etc. Wouldn't it be possible, to scroll in hoover mode but still accept clicks/gestures for the application on left click?
Is what the attached easystroke-dev does what you had in mind. Maybe scrolling should only be enabled here if the pen speed is at least a certain value.

Tom Jaeger

July 26th, 2008, 02:14 AM

The little tool in the attachment keeps track of what windows have been minimized most recently and undoes activates those windows whenever the user presses Alt+Super+F9.

Also, I've just tried easystroke on intrepid and it appears that there's a bug in recent xorg versions which causes xinput events to be reported in device coordinates rather than screen coordinates. Hopefully, I can get them to fix the problem, but it wouldn't be impossible to work around this either. As a temporary workaround, you can disable xinput with the -x command line switch. On the bright side, XShape performance seems to have improved, meaning that it is possible to draw longer strokes on the screen.

Luffield

July 26th, 2008, 06:51 AM

I found a problem using esaystroke but I'm not sure it's an easystroke issue.

I'm using Hardy, Firefox 3.01 and Mouse Gestures Redox. I use the right mouse button for gestures in both Firefox's extension and easystroke. Easystroke is disabled on the Firefox window.
I minimized the Firefox window with a mouse gesture and then tried to open my home folder using an easystroke gesture on the desktop but the context menu popped up and the gesture didn't work. I went back to Firefox and minimized it again by clicking the minimize thingy on the top of the window, and then easystroke worked normally on the desktop.
A bit more messing around, and I found that minimizing Firefox using the window list on the panel doesn't cause this problem, but going to the desktop with Alt-Ctrl-D or minimizing Firefox with Alt-F9 do.

I guess it could also be a window manager issue. I'm using compiz.

Tom Jaeger

July 26th, 2008, 09:09 AM

I minimized the Firefox window with a mouse gesture and then tried to open my home folder using an easystroke gesture on the desktop but the context menu popped up and the gesture didn't work.

Okay, I can reproduce the issue. It only happens if easystroke is started on login. The problem is that easystroke some misses the creation of the desktop background window. I'll look into this tomorrow.

EDIT: The problem was that in order to recognize top-level windows, easystroke has to search for windows that are tagged by the window manager, so if no wm is running yet, it won't find any windows. The solution is to bypass this test if there's no wm. The problem is fixed in rc4.

pibach

July 26th, 2008, 10:38 PM

EDIT: You seem to draw your gestures much smaller than I do. Do you think that you would benefit from subpixel accuracy (as in xournal)?

the Graffiti gestures I do are quite small, yes. And the recognizer sometimes does produce weird results. Subpixels would be nice user interface improvement, but more important is high recognition accuracy and responsiveness. Another way could be closer integration with cellwriter (e.g., auto invocation, insert and close).

This one is the only show-stopper in the list. What are the situations? Does this only happen if the menu workaround is enabled? Can you send me the output of "./easystroke -vvv" when it happens?

happens when I gesture Graffiti in a text box in Firefox. Then a list pops up to chose from. Gestures outside this list will result in a click event to Firefox. This is regardless of option "Accept gestures when menus are shown" on or off. But I would not rank this a show stopper as you can only produce this by a letter gesture (which most users won't ever do).

Is what the attached easystroke-dev does what you had in mind. Maybe scrolling should only be enabled here if the pen speed is at least a certain value.

Or, if you can figure out scroll area, the other way round: Only if you hover slowly to the edges of the scroll area (e.g., a Firefox Web page) the scroll is activated. The further you go to the edges, the faster is the scroll. If you move faster certain treshold then scroll is not activated (important to reach some item outside scroll area). Also there is some inner area that does not act on hovering scroll such that you can do all gestures and clicks without causing unwanted scrolling. Don't know if this works but might be worth a try. But this of course is stuff for future releases...

Luffield

July 27th, 2008, 07:26 AM

The problem was that in order to recognize top-level windows, easystroke has to search for windows that are tagged by the window manager, so if no wm is running yet, it won't find any windows. The solution is to bypass this test if there's no wm. The problem is fixed in rc4.
Thanks for the new version, Tom. I sorry to report that the problem still happens on my computer.

Tom Jaeger

July 27th, 2008, 06:00 PM

Thanks for the new version, Tom. I sorry to report that the problem still happens on my computer.
I'm not giving up yet. Does this also happen when easystroke is started when the window manager is already running?

Luffield

July 27th, 2008, 06:09 PM

I'm not giving up yet.
I didn't think you would :D

Does this also happen when easystroke is started when the window manager is already running?
It works fine when I start it from the terminal.

Tom Jaeger

July 27th, 2008, 08:17 PM

It works fine when I start it from the terminal.

Can you check this version? It simply waits for a little bit if there's no wm running.

Luffield

July 28th, 2008, 04:23 AM

No change, I'm afraid.

Tom Jaeger

July 29th, 2008, 11:06 PM

No change, I'm afraid.

This one has to work. It simply re-scans 15 seconds after startup.

I'll release the new version as soon as I find some time to update the documentation.

Tom Jaeger

July 29th, 2008, 11:42 PM

the Graffiti gestures I do are quite small, yes. And the recognizer sometimes does produce weird results. Subpixels would be nice user interface improvement, but more important is high recognition accuracy and responsiveness.

Yeah, that's what I meant. The hope is that using the device resolution will improve recognition results, but ultimately, a new algorithm is needed. The higher resolution will have to wait before the xorg device event reporting issue is settled, though. As for responsiveness, I'm not sure what you mean. Is the 200ms minimum delay between strokes the problem, or is the actual recognition slow (that is, does producing the matrix take a long time)?

Do you think we should draw the stroke before it is even clear that the user is actually doing a gesture?

happens when I gesture Graffiti in a text box in Firefox. Then a list pops up to chose from. Gestures outside this list will result in a click event to Firefox. This is regardless of option "Accept gestures when menus are shown" on or off. But I would not rank this a show stopper as you can only produce this by a letter gesture (which most users won't ever do).

Sorry, I have no clue what's going on here. Unless the app has grabbed the server (in which case nothing should happen if the option is turned off), no clicks should go through to the application.

Or, if you can figure out scroll area, the other way round: Only if you hover slowly to the edges of the scroll area (e.g., a Firefox Web page) the scroll is activated. The further you go to the edges, the faster is the scroll. If you move faster certain treshold then scroll is not activated (important to reach some item outside scroll area). Also there is some inner area that does not act on hovering scroll such that you can do all gestures and clicks without causing unwanted scrolling. Don't know if this works but might be worth a try. But this of course is stuff for future releases...

Like firefox's autoscrolling? I've always hated that feature. Anyway, I think figuring out the extents of scroll areas is possible using accessibility tools, but it might not be very stable (as in apps might randomly hang). I'd rather stick to what can be accomplished by listening to at-spi events. See the attached track-focus binary (it's in the access subdirectory in darcs) for an example that tracks focus and this link (http://accessibility.freestandards.org/a11yspecs/atspi/adoc/atspi-events.html) for what events are available. Note that this will only work if accessibility is enabled in the gnome settings.

pibach

July 30th, 2008, 01:44 AM

Do you think we should draw the stroke before it is even clear that the user is actually doing a gesture?

Yes, definitely. Also I do consecutive gestures faster than 200ms. There shouldn't be any delay in between.

To get more confidence in what gestures are doing, there should be additionally some feedback, either visual or audio. Could you draw the recognized/executed gesture pattern&command? Visual feedback should be next to where you do gesturing.

Regarding autoscroll: yes, I agree. I also do not like that implementation. But I think the problem is that it is only based on cursor position, not on its movement. The trick might be, to find a suitable formula that balances the influence of cursor movement and position.

There's also another simple option:
Enter scrolling as soon as you move the gesture for more than x pixels (all non-scrolling gestures I do are rather small).
Or as soon as you fall below certain pen speed (all non-scrolling gestures I do quite fast).

Luffield

July 30th, 2008, 06:34 AM

Tom, I'm posting this before installing RC5 - everything works well with RC4. After I installed it I logged out and logged in again and the problem still occured, so I guess I had to restart my computer (or X?) for the changes to kick in.
Sorry for making you work a bit harder than needed...

Tom Jaeger

July 30th, 2008, 08:45 AM

Yes, definitely. Also I do consecutive gestures faster than 200ms. There shouldn't be any delay in between.

The delay is there to avoid flickering. The new behavior (implemented in the attached binary) is to still have the delay but fast-track the process if another stroke is started. This version will also start drawing the gesture once the mouse has moved by 4 pixels, though I'm still on the fence about whether to include the change in the stable version.

To get more confidence in what gestures are doing, there should be additionally some feedback, either visual or audio. Could you draw the recognized/executed gesture pattern&command? Visual feedback should be next to where you do gesturing.

Like what vista does after flick gestures?

There's also another simple option:
Enter scrolling as soon as you move the gesture for more than x pixels (all non-scrolling gestures I do are rather small).
Or as soon as you fall below certain pen speed (all non-scrolling gestures I do quite fast).
One problem with the long gestures is that it might easily lead the pointer out of the window where scrolling can't work. For low speeds we already have aborting the stroke. Are you suggesting the the timeout action be configurable?

Luffield

July 30th, 2008, 09:32 AM

Regarding the geture feedback: I agree, it would be great to have this. Mouse Gestures Redox pops up a tooltip one the corner of the Firefox window and writes the gesture there (I think by default this option is off).
I'm not familiar with the vista flick gestures.

Eudinaesis

July 31st, 2008, 03:20 AM

Hi there - thanks so much for making this program!

This is probably just some really silly mistake on my part (I'm a linux newbie) but -- am I supposed to need to run this as root? When I don't, I get the following message:

easystroke
XError: BadAccess (attempt to access private resource denied): X_GrabKey
Error: Couldn't save preferences.
Segmentation fault

And even when I run it with sudo, I still get: "XError: BadAccess (attempt to access private resource denied): X_GrabKey"

What exactly do I need to do to have this run at startup?

Thanks,
Peter

Tom Jaeger

July 31st, 2008, 06:09 AM

am I supposed to need to run this as root? When I don't, I get the following message:

No, the program should be run as a regular user. What probably happened here is that you were root the first time you ran the program, so the configuration directory is now owned by root and you don't have write access to it unless you're root. You can correct this by typing

sudo chown -R $UID ~/.easystroke

in the command line.

Error: Couldn't save preferences.
Segmentation fault

Now I actually had code in there to test for this case and display a hopefully somewhat helpful error message, but unfortunately (as always is the case with code that gets little testing), this did the exact opposite and just crashed the program. This will be fixed in the upcoming 0.2.0 release.

And even when I run it with sudo, I still get: "XError: BadAccess (attempt to access private resource denied): X_GrabKey"

This means that easystroke was unable to grab the Escape key, which could be used to abort gestures. Not a big deal.

What exactly do I need to do to have this run at startup?

System -> Preferences -> Sessions -> Add

I should probably look into whether there is a way to do enable autostart in easystroke's preferences dialog (EDIT: I looks like this is just a matter of creating an appropriate .desktop file in the ~/.config/autostart/ directory. Won't make the cut for 0.2.0, though)

Tom Jaeger

July 31st, 2008, 07:55 PM

Peter, I've noticed some weird behavior related to focus. This might be a similar issue. Can you check if the issue is present in the attached version, which completely disables messing with the input focus? Thanks.

* when set to right click, gestures do also let the context menu pop up in some situations

pibach

August 1st, 2008, 02:12 AM

Eudinaesis

August 1st, 2008, 05:09 AM

Thanks!

No, the program should be run as a regular user. What probably happened here is that you were root the first time you ran the program, so the configuration directory is now owned by root and you don't have write access to it unless you're root. You can correct this by typing

sudo chown -R $UID ~/.easystroke

in the command line.

Now I actually had code in there to test for this case and display a hopefully somewhat helpful error message, but unfortunately (as always is the case with code that gets little testing), this did the exact opposite and just crashed the program. This will be fixed in the upcoming 0.2.0 release.

This means that easystroke was unable to grab the Escape key, which could be used to abort gestures. Not a big deal.

System -> Preferences -> Sessions -> Add

I should probably look into whether there is a way to do enable autostart in easystroke's preferences dialog (EDIT: I looks like this is just a matter of creating an appropriate .desktop file in the ~/.config/autostart/ directory. Won't make the cut for 0.2.0, though)

Tom Jaeger

August 3rd, 2008, 10:01 AM

same focus-stealing here.

Let's give this another try. This version replays the button press instead of faking it, which seems to solve a few annoying left-click bugs here.

NB. This is a development snapshot, which re-introduces a few bugs and contains some unrelated changes (among them taking advantage of the full tablet resolution, but that will break rotation for now).

tom66

August 3rd, 2008, 11:48 AM

Seems very slow with wobbly windows enabled.

Tom Jaeger

August 3rd, 2008, 07:17 PM

Seems very slow with wobbly windows enabled.

What do you mean by slow? Wobbly windows certainly doesn't have any effect on performance.

Tom Jaeger

August 4th, 2008, 05:40 AM

I've finally taken the plunge and released 0.2.0, a little too soon, as it turns out, since a little mistake in the Makefile rendered the debian packages unusable. I corrected this in version 0.2.1 a few hours later.

The new features have been discussed extensively in this thread, therefore I'll be short -- highlights are the so-called Advanced Gestures (http://easystroke.wiki.sourceforge.net/FeatureAdvancedGestures#content) and much improved tablet support.

Of course, development doesn't stop at this point. I plan to release version 0.2.2 before the Intrepid feature freeze (Aug 28); this version will contain only minor improvements such as better visual feedback; the bigger changes (application-dependent behavior) will have to wait until version 0.3.0.

I've also created a launchpad PPA (https://launchpad.net/~thjaeger/+archive). By adding the line

deb http://ppa.launchpad.net/thjaeger/ubuntu hardy main
to you /etc/apt/sources.list file, you'll be able to install the program using synaptics or 'apt-get install' and upgrade it via update-manager or 'apt-get update'. This also means that in addition to i386, precompiled amd64 and lpia packages are now available.

I'm hoping to get the package accepted into intrepid; as a first step, I've uploaded it to REVU for review.

Tom Jaeger

August 5th, 2008, 07:36 AM

To get more confidence in what gestures are doing, there should be additionally some feedback, either visual or audio. Could you draw the recognized/executed gesture pattern&command? Visual feedback should be next to where you do gesturing.

Preliminary support for visual feedback is included in the attached development snapshot.

pibach

August 9th, 2008, 11:10 AM

Preliminary support for visual feedback is included in the attached development snapshot.
Thanks, this works just fine. Only it has some delay to pop up (I like it ultimately responsive)

BTW, in Compiz (which I do not use due to the Firefox scroll problem) the visual feedback gets animated (window open/close effect). To disable this one would need to add the easystroke window to the exceptions.

Tom Jaeger

August 10th, 2008, 09:31 PM

Thanks, this works just fine. Only it has some delay to pop up (I like it ultimately responsive)

There's no artificial delay, and I don't notice one either. As soon as the gesture is recognized, the popup will be created.

BTW, in Compiz (which I do not use due to the Firefox scroll problem) the visual feedback gets animated (window open/close effect). To disable this one would need to add the easystroke window to the exceptions.
It's just a regular popup, so its open animation can be configured in the "Open Animation" tab of the Animation plugin. The default is a 150ms fade-in effect. If that bothers you, chances the fade-in time for menus and such will bother you, too, so you'll probably just want to adjust this setting.

In other news, development is continuing at a steady pace. I've basically rewritten the window-tracking code, which will hopefully eliminate a few bugs. As a user-visible consequence, window manager frames are now their own window class, so they can be listed as an exception, which should make it easier to drag windows if easystroke is set to left-click.
The parameters of the gesture timeout are now configurable if the program is invoked as 'easystroke -e'.
There finally is a decent gesture trail under compiz. It works by communicating with the 'annotate' plugin via dbus and requires compiz's 'annotate' and 'dbus' plugins to be enabled. It'll give you smooth, antialiased gestures, reduced CPU-usage compared with the XShape method and configurable color, alpha-transparency and thickness.

I'm attaching a current snapshot. Note that the internal file format changed again, so I'd suggest a backup of the ~/.easystroke directory if you want to go back to an earlier version.

My efforts to get the program accepted into the Ubuntu repositories turned out to be a tremendous waste of time and energy, as apparently no MOTU is willing to review my package. It has been suggested to me that I try to get the package into debian first, but I'm not sure about that yet because that too would be a huge time commitment and the outcome would be just as uncertain -- and besides, it would almost certainly mean that I would miss my intrepid goal, so what's the point?

Tycho Quad

August 11th, 2008, 12:41 PM

Very awesome!

Would it be possible to get it to use the "Paint fire on the screen" plugin instead?

Tom Jaeger

August 11th, 2008, 08:43 PM

Very awesome!

Would it be possible to get it to use the "Paint fire on the screen" plugin instead?

Yes, but...

First of all, the firepaint plugin doesn't respond to dbus events in the same way the annotate plugin does. So we need to patch it first (instructions below).
The next issue is that compiz stops receiving dbus messages after a while, for reasons I don't understand. This only comes up if you're drawing really long strokes and isn't a problem in practice, but it's something to be aware of. If this should happen, minimizing or maximizing a window will usually restore things back to normal.

EDIT: Now I know what's going on here: The compiz main loop will only read from other file descriptors if there are no events waiting on the X connection, so if an iteration of painting the fire takes too long, compiz won't receive any dbus messages. It's not obvious if or how that should be fixed.

Finally, because of these issues I can't possibly include the fire option by default, so you'll have to invoke the program as "./easystroke -e" to even see the option.

Instructions for setting up the dbus-enabled fire plugin:

sudo apt-get install compiz-dev compiz-fusion-bcop git-core
git clone git://anongit.compiz-fusion.org/fusion/plugins/firepaint
cd firepaint
make install # you don't need to be root, this goes into the user's HOME directory
compiz --replace &

(EDIT: patching is not necessary anymore)

pibach

August 11th, 2008, 10:56 PM

It's just a regular popup, so its open animation can be configured in the "Open Animation" tab of the Animation plugin. The default is a 150ms fade-in effect. If that bothers you, chances the fade-in time for menus and such will bother you, too, so you'll probably just want to adjust this setting.

There finally is a decent gesture trail under compiz. It works by communicating with the 'annotate' plugin via dbus and requires compiz's 'annotate' and 'dbus' plugins to be enabled. It'll give you smooth, antialiased gestures, reduced CPU-usage compared with the XShape method and configurable color, alpha-transparency and thickness.

Interestng. But currently I do not use Compiz (just tested it) as it does slow down scrolling in Firefox unacceptably. And then there is the xrandr problem for rotation. But maybe both problems get solved soon? This would make a very attractive package for a "Tabuntu Editon". BTW: what is the name of that window to put into Compiz as exception?

Tom Jaeger

August 11th, 2008, 11:28 PM

And then there is the xrandr problem for rotation.

Solved in intrepid and there's a good workaround for hardy. However, we need this patch (http://gitweb.freedesktop.org/?p=xorg/xserver.git;a=commit;h=37927b8bfa78670b263311ae1f0 6d2aae973601d) in xserver before easystroke is usable on intrepid (EDIT: package in my PPA).

BTW: what is the name of that window to put into Compiz as exception?
This works for me: (class=Easystroke) & (override_redirect=1)

Tom Jaeger

August 14th, 2008, 07:26 AM

This time around, I'd like to try to avoid dragging out the release so long, so here's 0.2.2-rc1. It features visual feedback, a new window tracking logic and "timeout profiles", which control how long it takes for gestures to timeout and how fast you have to move your pen/mouse. You can see the details if you start easystroke with the -e option. I chose the parameters more or less at random, so I'm very open to suggestions as to what reasonable profiles might be.

The dbus fire patch will hopefully be in compiz fusion git soon. [EDIT: here (http://gitweb.compiz-fusion.org/?p=fusion/plugins/firepaint;a=commit;h=3d8ef8eeb0b5f0138f02f941111ac b73790452c5) it is]

By the way, I've now switched to intrepid's xserver/driver/mesa combination for good, scrolling has gotten a lot smoother, but it seems like all the 3d stuff is noticeably slower now. Also, negative and reflection work now. Blur, too, but not with reasonable performance.

damagedspline

August 14th, 2008, 07:18 PM

pibach, in the beginning of the thread you mentioned testing it on n810.

do you still have the pre-built binaries? (or maybe more updated ones?)

pibach

August 16th, 2008, 05:02 PM

pibach, in the beginning of the thread you mentioned testing it on n810.

do you still have the pre-built binaries? (or maybe more updated ones?)
Yet I did not run easystroke on my n810 but I would like to. So let's get ahead, probably best way would be to post this in some n810 forum?

pibach

August 16th, 2008, 05:25 PM

By the way, I've now switched to intrepid's xserver/driver/mesa combination for good, scrolling has gotten a lot smoother, but it seems like all the 3d stuff is noticeably slower now.
Could you point me to some howto?
I also would be happy with just XFCE compositing for shadows and transparency. This also might be less battery hungry compared to compiz and gives the option to switch of compositing, triggered by laptop mode tools. As far as I know XFCE compositor does not require OpenGL (?). Would you recommend intrepid Mesa 3D GL driver for that purpose? Also the predominant performance issue I get under Compiz concerns scrolling, all 3D effects are reasonably fast.

Tom Jaeger

August 17th, 2008, 03:12 AM

Could you point me to some howto?
The process is just a matter of adding intrepid lines to sources.list and then upgrading the xserver and libgl1-mesa-dri packages. If you add 'APT::Default-Release "8.04";' to /etc/apt/apt.conf update-manager won't bother you with updates from intrepid. You might also need to edit your xorg.conf.

That said, I wouldn't really recommend the upgrade at this point, it turns out there are quite a few issues: Many 3d operations are noticeably slower (I had to disable the animation plugin and cube reflection), there seem to be problems with textures in 3d apps and videos crash the x server. I could live with all that, but the biggest problem are random lockups when activating alt+tab switching or the shift switcher. They don't happen very often, but if they do, the only thing you can do is ssh into your machine and reboot. Not even the SysRq combinations help anymore. [EDIT: This apparently went away after I disabled the reflection plugin.]

Another mesa 7.1 rc has just been released a few hours ago, let's hope that it fixes most of the issues.
UPDATE: It doesn't.

I also would be happy with just XFCE compositing for shadows and transparency. This also might be less battery hungry compared to compiz

One should hope that the relevant XRender operations are hardware-accelerated, but if they're not, it'll probably use more power than compiz.
Do the additional 60 interrupts per second that compiz causes really make a such big difference, though? I find that a little hard to believe.

Tom Jaeger

August 17th, 2008, 10:54 PM

I've released 0.2.2. Debs are available through my PPA:

i386 (http://ppa.launchpad.net/thjaeger/ubuntu/pool/main/e/easystroke/easystroke_0.2.2-0ubuntu1_i386.deb)
amd64 (http://ppa.launchpad.net/thjaeger/ubuntu/pool/main/e/easystroke/easystroke_0.2.2-0ubuntu1_amd64.deb)

Update: 0.2.2-0ubuntu1 is in the New queue (https://launchpad.net/ubuntu/intrepid/+queue) for intrepid.

pibach

August 19th, 2008, 12:46 AM

So this means it is evaluated for Intrepid release? Would be great.
I Like the new compiz annotation draw method. I now have configured my desktop with Gnome+Compiz+Emerald+Cairo Dock, very pen friendly (although a bit slow).
However I sometimes get slight corruption, i.e., the red line of the annotation if fine, but the line stored when I define actions shows some hickups. So I need to redraw. Otherwise it works great.
Some slight suggestion: keep the red line visualisation some time (e.g., 500ms) visible.

Tom Jaeger

August 20th, 2008, 08:06 AM

It's already in intrepid. So I'll have to support version 0.2.2 in some form until at least 8 months from now. Kind of scary if you think about it.

However I sometimes get slight corruption, i.e., the red line of the annotation if fine, but the line stored when I define actions shows some hickups. So I need to redraw. Otherwise it works great.

This only happens with annotate? Either way, I don't have much of a clue what's going on here.

Some slight suggestion: keep the red line visualisation some time (e.g., 500ms) visible.

I'm already doing something similar for fire, so this should be easy to do. I think I'm going to go with 250ms, though.

pibach

August 20th, 2008, 01:58 PM

ok, I'm back to Xfwm4+Compositing, it is a lot more responsive.

Here are some further questions:

* is it possible to query the cursor or mouse position? I would like to start cellwriter close to the input area.
* if activating easystroke on left click you can put the scroll area in the list of exceptions ("Navigator"). Unfortunately it as well disables gestures for the rest of the window. If I add the window frame (for dragging a window) I cannot gesture on the desktop (why is that?). Anyway the "medium" gesture abort timing seems to work fine for me.

Tom Jaeger

August 20th, 2008, 06:35 PM

* is it possible to query the cursor or mouse position? I would like to start cellwriter close to the input area.

The position before or after the stroke? Both are easy to do, though one of them obviously has to be done inside easystroke.

* if activating easystroke on left click you can put the scroll area in the list of exceptions ("Navigator").

"Navigator" is just the name that firefox gives itself, so this behavior is expected.

If I add the window frame (for dragging a window) I cannot gesture on the desktop (why is that?).
I can't test this since even when I start xfwm4, nautilus handles the background window. What is the output of

xwininfo | grep 'Window id'
when you click on the desktop?

pibach

August 20th, 2008, 09:28 PM

The position before or after the stroke?

Doesn't matter much as I was just looking for some simple solution outside easystroke. Basicaly I want a behavior similar to Vista's TIP (ultimately better of course), i.e., launch cellwriter and bring it close to the cursor position whenever I hit some text input area with the pen. Currently I use a workaround: I defined a gesture to invoke cellwriter. It allows to have x,y coordinates where to brint it up. These coordinates have to be set according to some query for the mouse (or cursor) position.

I can't test this since even when I start xfwm4, nautilus handles the background window.

I am using the exact same setup, xfwm4 on top of Gnome. Finally this will require to retrieve the GTK elements under the pen's position and only invoke a gesture if it is not a "strokeable" one.

But biggest issue still for left click usage is to distinguish mouse or trackpoint versus pen. Best solution herefore would probably be to deactive easystroke if pen is not in proximity.

linuxguymarshall

August 20th, 2008, 09:30 PM

Very nice app. I would like to see webcam integration soon so that maybe we can get a Nintendo wii-like thing going. I would love to hold up X fingers and open up Firefox

Tom Jaeger

August 21st, 2008, 04:38 AM

Basicaly I want a behavior similar to Vista's TIP (ultimately better of course), i.e., launch cellwriter and bring it close to the cursor position whenever I hit some text input area with the pen.

This isn't actually that hard to do: At-spi can notify us when the focus changes to a text field, and then we could place cellwriter as close to the cursor as possible without overlapping the input field. The hardest part would actually be to control cellwriter, which would have to be patched.

Currently I use a workaround: I defined a gesture to invoke cellwriter. It allows to have x,y coordinates where to brint it up. These coordinates have to be set according to some query for the mouse (or cursor) position.

Querying the mouse position is trivial. I've attached an example (compile via 'gcc -lX11 query-pointer.c -o query-pointer').

I am using the exact same setup, xfwm4 on top of Gnome. Finally this will require to retrieve the GTK elements under the pen's position and only invoke a gesture if it is not a "strokeable" one.

GTK widgets only exist as data structures inside client applications. We have to rely on with what information they export through ATK, which isn't good enough to, say, decide if there is a scroll bar under the cursor.

But biggest issue still for left click usage is to distinguish mouse or trackpoint versus pen. Best solution herefore would probably be to deactive easystroke if pen is not in proximity.
The information what device an event came from is available through Xi. So this is possible, but in practice it will require making the assumption that Xi events are always generated. This is tantamount to dropping xserver 1.3 support (which is what gutsy uses), but since I can't do the necessary testing, it is a gamble to use easystroke with old xserver versions anyway (if you manage to compile it, that is...).

Tom Jaeger

August 21st, 2008, 04:59 AM

Very nice app. I would like to see webcam integration soon so that maybe we can get a Nintendo wii-like thing going. I would love to hold up X fingers and open up Firefox
It's an interesting idea, but it requires a completely different set of algorithms. Since I don't know anything about image/video processing, I'm really the wrong person to ask, but I'd accept patches.

Taking advantage of the additional axes of a wiimote would probably be feasible, but don't have one to test this on (is there even a way to hook it up to your computer?).

savantelite

August 21st, 2008, 05:06 AM

once its in the repos I will have it in a flash. Even a deb file I can click install now. I am just to simple for command line.

reacocard

August 22nd, 2008, 12:19 AM

Taking advantage of the additional axes of a wiimote would probably be feasible, but don't have one to test this on (is there even a way to hook it up to your computer?).

Yes there is, all you need is a bluetooth adapter for the computer. I've used a wiimote as a presentation remote before under ubuntu, using cwiid to turn the wiimote buttons into normal buttons. I haven't investigated the axis abilities at all, but wmgui shows it can get the signals quite readily.

pibach

August 27th, 2008, 02:48 PM

The hardest part would actually be to control cellwriter, which would have to be patched.

according to man cellwriter it has the options --windows-x and --windows-y to specify where it should pop up. But I cannot get this to work.

Another issue with cellwriter is that cellwriter --show-window does sometimes not bring up the window when running compiz (works fine on Xfwm4 though and also clicking the tray icon always works). I can get this fixed by restarting cellwriter, weired.

Another downer is that execution of easystroke commands takes a bit long, for cellwriter to come up it takes for example ~2s, whereas cellwriter pops up immediately if I just hit its tray icon.

Tom Jaeger

August 27th, 2008, 07:01 PM

The only solution here is to patch cellwriter and create, say, a dbus interface. I'd send the author an email, but I'm kind of swamped at the moment.

Another downer is that execution of easystroke commands takes a bit long, for cellwriter to come up it takes for example ~2s, whereas cellwriter pops up immediately if I just hit its tray icon.
This is weird. Is it only cellwriter that has this problem? In any case, restarting cellwriter every time you want to use it is not an ideal solution anyway.

Risujin

August 27th, 2008, 07:56 PM

Hi guys! I'm the author of CellWriter. Peter sent me an email about this thread, maybe I can be of help.

The only solution here is to patch cellwriter and create, say, a dbus interface. I'd send the author an email, but I'm kind of swamped at the moment.

CellWriter uses something much hackier, it just has an open fifo in its settings directory that accepts input to toggle the window.

This is weird. Is it only cellwriter that has this problem? In any case, restarting cellwriter every time you want to use it is not an ideal solution anyway.

Running CellWriter when it is already running does not start a second instance or "restart". The second instance just toggles the main window and quits. It doesn't communicate the --window-x/y parameters this way though, it only checks those when it is started for real. This is fixable of course, if this is really useful for you guys.

all2ez

August 27th, 2008, 10:11 PM

pibach

August 31st, 2008, 03:50 PM

It doesn't communicate the --window-x/y parameters this way though, it only checks those when it is started for real. This is fixable of course, if this is really useful for you guys.
Hi Risujin, great to have you here!

I think the challenging goal is a "Tabuntu" edition with comprehensive pen integration, (at least) at the level Vista does it. Cellwriter and easystroke are the core apps in this respect. So lets discuss how to get them work flawlessly together.

Current workflow for pen-based text input is to activate Cellwriter by clicking its tray icon and moving its window to the desired position. To get this more efficient I suggested to open cellwriter on an easystroke gesture at the position of the gesture. This gesture activation might also be better (and easier to implement) than the auto-activation (mimicking the Vista TIP behavior) we discussed above.

I don't know if fixing the --window-x/y in conjunction with --show-window would be best, as it seems to introduce some delay (Risujin, could you explain this delay?). But anyway this seams to be the most straight forward approach.

Another option might be a direct activation/window placement for common applications from within easystroke, for example, to activate xvkbd at the gesture position. Probably this is quicker and also more general? What do you guys think?

Another important problem is that most tablet users just don't know that there is such a good pen support when using Linux on a tablet PC. The question here is how to package and deliver these tablet goodies to get some more visibility. This should be coordinated with the activities in Ubuntu Netbook Remix and MID Edition. Risujin, what is your suggestion in this respect?

BTW, I installed the Window-Picker-Applet from Netbook Remix, which does a good job at maximizing screen real estate, perfect for devices with low resolutions/small displays.

--Peter

pibach

September 8th, 2008, 12:46 AM

No follow up post?

Probably wmctrl (http://www.sweb.cz/tripie/utils/wmctrl/) or devilspie (http://www.burtonini.com/blog/computers/devilspie) could serve placing windows close to the pen's position?

Tom Jaeger

September 9th, 2008, 01:38 AM

Sorry I haven't replied earlier, I've been very busy with other things lately. I had already worked on the automated cellwriter invocation, so I'm just going to dump the code on you that I have so far. It's proof-of-concept code, it'll just kill cellwriter and restart it later instead of hiding and showing it. It doesn't do any overlap detection and will happily move a window partially off the screen. Also, trying to quit cellwriter the regular way will make the X server hang.

To do placement properly, we'll need to somehow get the coordinates of the cellwriter window. I originally suggested dbus for this, but I've now realized that this won't work because the size of the window might change over time, and i don't think dbus makes callbacks particularly easy. So we're probably going to have to use X11 anyway to get a hold of the cellwriter window and watch it for size changes. Moving the window is also possible, but it might take some trickery to wait for the right moment. So it seems like we're good as far as cellwriter is concerned, but it might still be a good idea to make cellwriter's --window-x and --window-y parameters work when cellwriter is already running.

In other news, my PPA contains updated packages that fix a few minor bugs. I'll make a proper release in time for the intrepid beta freeze.

EDIT: Just looked at cellwriter's source, right now there is a 1-Hz timer responsible for checking the fifo, which causes the delays. I'll send risujin a patch.

EDIT2: Updated cellwriter package in my PPA.

pibach

September 9th, 2008, 11:57 AM

Thanks for the update, I'll give this a try.
But isn't the window placement possible by simply engaging wmctrl (http://www.sweb.cz/tripie/utils/wmctrl/)? There is the following placement option:
"<MVARG> Specifies a change to the position and size
of the window. The format of the argument is:
<G>,<X>,<Y>,<W>,<H>"
Regarding cellwriter I hope that Risujin will do that window-show at x/y fix so there's no workaround needed (in conjunction with your patch this should be fine).

I still see the Grab&Drag scroll option of easystroke being a killer feature, if it works for all apps such as OpenOffice or Epiphany (which do not provide G&D scrolling natively).

What about this simple approach: There's already a gesture timeout implemented. Now check after the timeout if the gesture was a straight vertical movement (up or down). If yes, switch scrolling mode on.

Another demanding feature are right click and middle click (plus Ctrl-click) gestures that act on the originating position.

Tom Jaeger

September 9th, 2008, 10:01 PM

Thanks for the update, I'll give this a try.
But isn't the window placement possible by simply engaging wmctrl (http://www.sweb.cz/tripie/utils/wmctrl/)?

Moving the window is easy, that's just a XMoveWindow() call. But it's important to move the window after it has been mapped and moved by cellwriter, so there's some additional work to do.

I still see the Grab&Drag scroll option of easystroke being a killer feature, if it works for all apps such as OpenOffice or Epiphany (which do not provide G&D scrolling natively).

I actually like the side button idea better, but in any event, this'll have to wait for a little bit. Next on my list is application-dependent actions and preferences. I already worked out roughly how the GUI is supposed to look like, I just need to find the time to implement it.

Another demanding feature are right click and middle click (plus Ctrl-click) gestures that act on the originating position.
Unfortunately, this is impossible as there is no way to execute XTest requests in an atomic way.

pibach

September 9th, 2008, 10:20 PM

I actually like the side button idea better, but in any event, this'll have to wait for a little bit.

Ok, I am eagerly awaiting your next improvements. Application specific gestures sounds good to me. Nevertheless convenient scrolling is a must.

Currently it works just fine with G&D scrolling in Firefox. I have easystroke on the left click. So I have to wait for the timeout to get G&D kick in. This works very well, as I keep the pen down while reading which triggers the timeout and thus I can scroll immediately on pen movement. But of course this does not work in apps other than firefox.

Implementation shouldn't be too difficult. Isn't it just an additional If-Then line to check the movement and in case it has been less than X pixels (say 10) horizontal (e.g., to mark text) then trigger the scrolling mode?

Unfortunately, this is impossible as there is no way to execute XTest requests in an atomic way.
Strange. Isn't it possible to store the starting point of the gesture line? That's a bummer.

SunnyRabbiera

September 9th, 2008, 11:23 PM

I personally never liked mouse gestures, as I am always dropping my mouse.

Gourgi

September 12th, 2008, 01:08 PM

bumping this 'cause this app really ROCKS ! :guitar:

Tom Jaeger

September 12th, 2008, 10:39 PM

I'm attaching a new development snapshot, which contains an experimental version of the feature you requested. Enable 'Timeout Gestures' in the preferences dialog, then you should be able to record gestures involving timeouts. Timeout gestures require a greater accuracy for a match.

Standard disclaimer: Keep backups if you want to test development versions, you won't be able to go back to the stable version otherwise.

Strange. Isn't it possible to store the starting point of the gesture line? That's a bummer.
The difference is that in this case the event is just delayed and replayed at a later time, so it will use the original coordinates. But we don't know upfront which button needs to be pressed, so we can't use the same method.

EDIT: Bugfixes

pibach

September 14th, 2008, 01:10 AM

I'm attaching a new development snapshot, which contains an experimental version of the feature you requested. Enable 'Timeout Gestures' in the preferences dialog, then you should be able to record gestures involving timeouts. Timeout gestures require a greater accuracy for a match.
Tom, it's gread at what pace you do enhance easystroke and manage to incorporate feature request.
However, you have to help me as I did not see how to define timeout gestures. I did enable the checkbox under preferences. And then added a new gestures. But then, where can I assign this gesture to be a timeout one?

Tom Jaeger

September 14th, 2008, 04:43 AM

Tom, it's gread at what pace you do enhance easystroke and manage to incorporate feature request.
However, you have to help me as I did not see how to define timeout gestures. I did enable the checkbox under preferences. And then added a new gestures. But then, where can I assign this gesture to be a timeout one?
Just let the gesture time out when you're recording :)

pibach

September 14th, 2008, 10:58 PM

Just let the gesture time out when you're recording :)
LoL
Ok, that way it works to record timeout gestures. They are marked by a red cross. But they are not recognised. Sometimes some wrong (non-timeout) gesture is recognised.

Tom Jaeger

September 15th, 2008, 03:29 AM

LoL
Ok, that way it works to record timeout gestures. They are marked by a red cross. But they are not recognised. Sometimes some wrong (non-timeout) gesture is recognised.

You're probably using the version I posted at first, which had a bug in it. The updated version should fix this.

Gan Quan

September 15th, 2008, 05:55 AM

Thank you! Great app, I've been looking for a easy-to-use replacement of strokeit since I switched from windows to linux, and now I found my answer, thank you thank you thank you!

Just two suggestions:
1. Application dependent gesture settings, allow gestures to behave differently in different applications.
2. Hide the tray icon.

pibach

September 15th, 2008, 12:51 PM

You're probably using the version I posted at first, which had a bug in it. The updated version should fix this.
Yes, it does. Works wonderfully now. You have to bring this into the repo release, it improves functionality a lot. Also the scroll acceleration works fine. Only suggestion: the cursor should change to indicate scroll mode (and further visualisation of the scroll gesture is not needed then). Have to play with timeout gestures a bit more, e.g., to define Middle-Click and Ctrl-Click as timeout gestures. And whether it is better to define scroll for up/down gestures or even for a click (I would prefere the latter, but here timeout seems to be a tad too aggressive - I am using medium profile).
BTW, I get segmentation fault when returning to old version, seemingly because format of gesture file has changed.

Edit: still it is a bit difficult to scroll list boxes etc. Thinking for some intuitive solution. Would it, for example, be possible to scroll further if pen exceeds the active scrolling area (currently the scroll then goes to the new area under pen)?

modmadmike

September 15th, 2008, 12:56 PM

Will this work with a wacom intous3 tablet?

Tom Jaeger

September 15th, 2008, 08:37 PM

the cursor should change to indicate scroll mode (and further visualisation of the scroll gesture is not needed then).

Well, X11 cursor handling leaves a lot to be desired, the best I could do is a flickering cursor (like in the original scroll mode), but even that requires a nasty hack that I'm actually kind of happy to have gotten rid of.

I would prefere the latter, but here timeout seems to be a tad too aggressive - I am using medium profile).
./easystroke -e gives you more fine-grained control over timeouts.

BTW, I get segmentation fault when returning to old version, seemingly because format of gesture file has changed.

Yup, this is how things stand, unfortunately. I might roll my own serialization code someday that fixes the issue, but until then, backups are necessary before every upgrade.

Edit: still it is a bit difficult to scroll list boxes etc. Thinking for some intuitive solution. Would it, for example, be possible to scroll further if pen exceeds the active scrolling area (currently the scroll then goes to the new area under pen)?
This has been bothering me, too, but I don't see a way around the current behavior. The problem is that X11 treats scroll wheel events as (button 4-7) clicks, which are always sent to the application under the cursor. And with absolute devices, I can't move the cursor before emitting a click.

Tom Jaeger

September 16th, 2008, 08:03 PM

Will this work with a wacom intous3 tablet?

I don't see why it wouldn't.

pibach

September 17th, 2008, 04:45 PM

./easystroke -e gives you more fine-grained control over timeouts.

Thanks for the hint, this was helpful. I configured scroll to trigger on a simple left click and changed timeout setting a bit. But this timeout of course can conflict with other intended slides. Still not sure which configuration might be best. How do others use it?

The problem is that X11 treats scroll wheel events as (button 4-7) clicks, ..
Ok, I see. So what about a "circular scroll" approach? Synaptics is calling it "Chiral Motion".

Another problem still remains and might be related: the middleclick (or mousewheel or pen) scrolling is not as responsive as the grab & drag (or scrollbar) scrolling is. Seems to be two different scrolling mechanisms.

Btw, I get the following errors:

** (easystroke:21626): WARNING **: Invalid borders specified for theme pixmap:
/usr/share/themes/OSX-theme-mod/gtk-2.0/Range/null.png,
borders don't fit within the image
XError: BadWindow (invalid Window parameter): X_QueryTree
Info: Device Presence not implemented

seemingly my Gtk-Theme is corrupted?

GrouchoMarx

September 17th, 2008, 05:32 PM

Thanks for this great app! Super useful. I second these suggestions:

Just two suggestions:
1. Application dependent gesture settings, allow gestures to behave differently in different applications.
2. Hide the tray icon.

Is there currently no way to remove the tray icon?

Tom Jaeger

September 17th, 2008, 09:46 PM

Ok, I see. So what about a "circular scroll" approach? Synaptics is calling it "Chiral Motion".
Interesting idea. I'll prototype it to see if it is a viable solution.

Another problem still remains and might be related: the middleclick (or mousewheel or pen) scrolling is not as responsive as the grab & drag (or scrollbar) scrolling is.

Scroll wheel events are discrete, so there's nothing I can do here. Maybe in a few years if the scroll wheel as valuator thing (http://thread.gmane.org/gmane.comp.freedesktop.xorg/31201/focus=31202) actually happens.

Btw, I get the following errors:

** (easystroke:21626): WARNING **: Invalid borders specified for theme pixmap:
/usr/share/themes/OSX-theme-mod/gtk-2.0/Range/null.png,
borders don't fit within the image
XError: BadWindow (invalid Window parameter): X_QueryTree
Info: Device Presence not implemented

They're all harmless.

Tom Jaeger

September 17th, 2008, 09:53 PM

1. Application dependent gesture settings, allow gestures to behave differently in different applications.
I'm working on this right now, but it will take a while as I have a lot of other things going on at the moment.

2. Hide the tray icon.
I suppose it would be nice if this were possible, but it's not a very high priority.

pibach

September 17th, 2008, 10:10 PM

Scroll wheel events are discrete, so there's nothing I can do here.
Yes sure. And I am using "Yet another Scrolling Extension" to smooth this out a bit. But somehow scrolling with grab & drag extension is slightly better regarding responsiveness. And ultimately only the best solution will succeed. The scrolling issue is very important, as in tablet mode the main activity typically is reading text (Web, PDF, or Word) while using the pen for navigating and commenting. So we might need some further brainstorming and discussion how to make this as smooth and intuitive as possible. E.g., one option might be to use easystroke scroll in all apps but grab & drag scroll in Firefox as an exception. Also an open question is what to do about scrolling in inking apps such as xournal (as this requires to be an exception to enable the inking).

Just a remark on todays compiz update (I already moved to Intrepid Ibex): this seems to improve scrolling performance. Still not as responsive as native Metacity, but it comes close now (before it was a factor of 2 difference).

ericesque

September 17th, 2008, 10:16 PM

Fantastic work, Tom. I saw this covered on Lifehacker a few days ago and HAD to boot into Ubuntu for the first time in a while (been gaming a lot lately) just to try it.

I particularly like to use it to issue commands for Compiz Fusion since all the keystroke combinations are so similar and difficult for me to remember.

I'm not using gestures for launching applications, but rather as a method of interacting with the GUI. So rotating the cube, max, min, close, show desktop etc. are all handled by gestures that make sense to me. It makes the OS feel so interactive!! The next best thing to touch commands!

Thanks again!

lovinglinux

September 21st, 2008, 08:04 PM

Thank you. Very useful application.

hongleong

September 23rd, 2008, 05:26 AM

Hi Tom,

Thanks for sharing your useful program with the rest of us.

Btw, I noticed that if I were to disable the popup feedback by unchecking Preferences > Show Popups, it'll get automatically checked again after I restart the program. Any idea how to fix this?

Thank you.
hongleong

Tom Jaeger

September 23rd, 2008, 08:55 AM

Btw, I noticed that if I were to disable the popup feedback by unchecking Preferences > Show Popups, it'll get automatically checked again after I restart the program. Any idea how to fix this?

Thanks, this is indeed a bug (I'll provide updated packages in my PPA tomorrow). As a workaround you can change any other option and then change it right back, this will save all the options.

hongleong

September 23rd, 2008, 10:25 AM

Yep, that works. Thanks. :)

russo.mic

September 30th, 2008, 04:53 AM

Totally beats gestix. Keep up the great work! Thanks!

Russo

Luffield

September 30th, 2008, 11:13 PM

I just upgraded to Intrepid beta and easystroke lost some functionality - key combinations work, but commands don't. Anyone else experiencing this problem? Is there a way to fix it?

Tom Jaeger

October 1st, 2008, 01:55 AM

I just upgraded to Intrepid beta and easystroke lost some functionality - key combinations work, but commands don't. Anyone else experiencing this problem? Is there a way to fix it?

I can't reproduce this issue on the latest intrepid with easystroke from intrepid. What is the command that is not working?

Luffield

October 1st, 2008, 08:24 AM

I have a gesture to launch nautilus and it does nothing, although it is recognized. However, today the gestures to launch gedit and firefox do work - but last night they didn't.

Tom Jaeger

October 3rd, 2008, 12:58 AM

I have a gesture to launch nautilus and it does nothing, although it is recognized. However, today the gestures to launch gedit and firefox do work - but last night they didn't.

The problem is that nautilus won't open a window if the DESKTOP_AUTOSTART_ID environment variable is set (which is the case if easystroke is autostarted). The problem disappears after easystroke is restarted.

I've fixed this issue in git by unsetting the environment variable at startup, but the fix probably won't end up in intrepid. I will eventually make it available through my PPA, though.

As a workaround, you can manually unset the environment variable before starting nautilus, i.e. use

unset DESKTOP_AUTOSTART_ID; nautilus
as the command.

Luffield

October 3rd, 2008, 09:15 AM

Thanks Tom!

olejorgen

October 5th, 2008, 07:06 AM

Tom Jaeger

October 5th, 2008, 05:48 PM

How do I open the config gui?

ole:~$ easystroke
Error: Couldn't read action database.
Error: Couldn't read preferences.

PS: I'm running a somewhat non-standard system. Debian unstable with ion3 (WM)

The configuration dialog is opened via the tray icon, which you don't see because ion doesn't support the tray specification for ideological reasons (http://modeemi.fi/~tuomov/ion/faq/features.html). I don't know what the external programs are that he is talking about, but I suppose gnome-panel will work.

olejorgen

October 5th, 2008, 07:28 PM

Yes, I suspected something like that. Would you consider adding a simple command switch(*) for starting with the configuration window? Obviously not a big deal, but I think it's good principle to force as little as possible on people

(*) or accepting a non-intrusive patch

gnome-panel is a pain beacuse it totally ignores the WM. lxpanel works

Tom Jaeger

October 5th, 2008, 10:04 PM

Yes, I suspected something like that. Would you consider adding a simple command switch(*) for starting with the configuration window?

I've committed a patch to master and the stable branch. Eventually, you'll be able to show/hide the main window using a gesture, but that will require some unpleasant cellrenderer hackery first.

Tom Jaeger

October 10th, 2008, 07:45 AM

Eventually, you'll be able to show/hide the main window using a gesture, but that will require some unpleasant cellrenderer hackery first.

I've implemented this in master now (type = Misc, argument = Show/Hide).

realn

October 13th, 2008, 09:30 AM

Thank you. That's one of my favourite apps. And I'm using it for only 1 hour.

Tom Jaeger

October 22nd, 2008, 10:57 PM

Here's a little preview of what 0.3.0 is going to bring. The biggest new feature is certainly application-dependent behavior.
To use the feature, simply open the "Applications" expander add the application that you want to give special treatment to to the list. As long as this application is selected, all changes you make to the actions list will be restricted to that application. So for example if you "delete" an action, it will still be recognized in all other applications, and you can actually restore it by enabling the "show deleted rows" option and then resetting the action.

Less notable features include

It is now possible to assign a different button for a particular application (e.g. xournal) in the exceptions list
Timeout gestures have already been mentioned in this thread
Easystroke can be disabled for specific devices
Actions can now be executed from the command line using easystroke send "foo" if the action is named foo. This is equivalent to dbus-send --type=method_call --dest=org.easystroke /org/easystroke org.easystroke.send string:"foo"
You can disable easystroke using the tray menu or by a gesture
There are 3 new actions if you select "Misc" as the action type: "unminimize" undos the last minimize actions, "Show/Hide" brings up or hides the configuration dialog and "Disable (Enable)" disables (or enables if you invoke the action from the command line) the program.
I've given up on changing the pointer in scroll mode. There's a blue popup in the upper right corner of the screen now instead.
I've dropped support for some of the advanced features when xinput is not available. This was a nightmare to maintain and xinput shouldn't be a issue anymore on recent x servers.
Due to popular demand, I've introduced an option to change the stroke color
I've probably forgotten a few things, I'll add them to the list as I think of them.

As always, since this is a pre-release version, I strongly recommend to make a backup of your ~/.easystroke directory before testing it. Also, since I've made a few changes to the fragile ecosystem that is event handling, so look out for regressions.

EDIT: Removed the binary, see the latest snapshot at the end of this thread instead.

sarah.fauzia

October 25th, 2008, 11:06 PM

Hi,

I'd like to announce easystroke (http://easystroke.sourceforge.net), a new gesture recognition application for linux. Gesture recognition means that you can draw arbitrary curves on the screen by holding down a specific mouse button, and if the program recognizes their shape, it will perform certain actions. For example, you can configure easystroke to maximize the current window if you draw a straight line in North-East direction.

My primary motivation for writing easystroke was to allow easy operation of a Tablet PCs even without a keyboard present, but of course it will work just as well with a mouse. It is not meant to replace an onscreen keyboard/input panel such as cellwriter, but rather supplement it.

Here's a short list of the program's main selling points:

It aims to be easy to set up and to configure. There are no configuration files that need to be edited and no cryptic commands that have to be entered somewhere.
It tries to give the user easy access to the most commonly used features: Setting up a new gesture requires just a few clicks and will show only one small popup dialog (to actually define the stroke)
It allows you to use strokes of arbitrary shape. There is no requirement that gestures have to be composed of line segments, and curvy shapes such as an 'S' or a 'G' work just fine.
Some of the features make life without a keyboard a lot easier: You can emulate a scrollwheel, ignore the next stroke and pass the next mouse action to the application (possibly with a modifier held down, so that you can Alt-move or Alt-resize a window without a keyboard) and emulate an additional button that your tablet pen didn't even have in the first place.
The project is still young, so there's much more to come.

EDIT (Aug 3): The latest version is 0.2.1. See this post (http://ubuntuforums.org/showpost.php?p=5519620&postcount=118) for details.
EDIT (Aug 17): Released 0.2.2.

The program is available as a .deb package tested on Ubuntu Hardy, a source tar.gz, and a standalone binary. It is also available through my launchpad PPA (https://launchpad.net/~thjaeger/+archive). There are a few screenshots on the project's documentation page (http://easystroke.wiki.sourceforge.net/Documentation).

Thank you,
Tom

I must say that your program is absolutely amazing! I'm running it in Intrepid and it's working perfectly; I was always annoyed that I could not use my computer well in tablet mode because I couldn't use my on-screen keyboard (from CellWriter) to prompt Compiz commands (such as Alt-Super, then Button 1) to toggle Annotate. My only concerns are two--one of which is a limitation of the Annotate plugin. After activating it, it only stays activated for as long as my pen is on the screen--as soon as I lift it, it's untoggled. The compiz documentation for Annotate says to hold down on the Alt-Super keys in order to be able to lift the pen and continue writing... is there a way easystroke could simulate a repeat keypress? If it's already a feature, I would appreciate if you could clarify it for me--I'm new to this, and I enjoy this software tremendously! It's far, far superior to Vista pen flicks.

The other quirk is that the EasyStroke icon always shows the last used gesture... but it defaults to the blue X which doesn't look so nice (and I must say, overall, the GUI is fantastic). Is there a possibility to let it default to the original easystroke icon, and not the last used stroke?

Tom Jaeger

October 26th, 2008, 09:32 PM

My only concerns are two--one of which is a limitation of the Annotate plugin. After activating it, it only stays activated for as long as my pen is on the screen--as soon as I lift it, it's untoggled. The compiz documentation for Annotate says to hold down on the Alt-Super keys in order to be able to lift the pen and continue writing... is there a way easystroke could simulate a repeat keypress? If it's already a feature, I would appreciate if you could clarify it for me--I'm new to this, and I enjoy this software tremendously! It's far, far superior to Vista pen flicks.

This is intentional. Easystroke uses the annotate plugin to draw the current gesture on the screen to give the user instant feedback and then immediately clears the screen as soon as the gesture is over. If you want to use easystroke to actually use the annotate plugin to draw something, you should use another method to draw the gesture line and then set up an ignore gesture with Alt+Super as the argument. This works best if the "Stay in scroll or ignore mode..." option is enabled. Alternatively, an Alt+Super+Button1 advanced gesture will also work.

The other quirk is that the EasyStroke icon always shows the last used gesture... but it defaults to the blue X which doesn't look so nice (and I must say, overall, the GUI is fantastic). Is there a possibility to let it default to the original easystroke icon, and not the last used stroke?
There's really not much of a point in the tray icon changing anymore now that we have popup notification, so I've removed this behavior in master. In case someone complains, I will revert that change and make this an option.

Tom Jaeger

October 26th, 2008, 10:13 PM

Here's another beta. The stroke previews look much better now and it is finally possible to reorder the list of actions using drag and drop, which turns out to be more painful than I anticipated, so I will probably move to a tree-based concept in the future that would make it possible to group actions. This version might still be a little unstable.

sarah.fauzia

October 27th, 2008, 01:07 AM

This is intentional. Easystroke uses the annotate plugin to draw the current gesture on the screen to give the user instant feedback and then immediately clears the screen as soon as the gesture is over. If you want to use easystroke to actually use the annotate plugin to draw something, you should use another method to draw the gesture line and then set up an ignore gesture with Alt+Super as the argument. This works best if the "Stay in scroll or ignore mode..." option is enabled. Alternatively, an Alt+Super+Button1 advanced gesture will also work.

There's really not much of a point in the tray icon changing anymore now that we have popup notification, so I've removed this behavior in master. In case someone complains, I will revert that change and make this an option.

I think didn't clarify what I meant about the Annotate plugin. I used Easystroke to activate the Annotate plugin (in addition to using it to draw the current gesture). However, the Annotate plugin only stays active, after I've drawn the gesture I keyed to Annotate, for as long as my pen is on the screen. As soon as I lift the pen, then it deactivates (like the documentation says it would, but "holding" on Super-Alt would enable it to stay toggled). I do have it in ignore mode (that enabled me to key the gesture to just Super-Alt). If I misunderstood you, then I'd appreciate the clarification :)

I'll definitely appreciate the change to the tray icon--I think I'll download that new beta!

GrouchoMarx

October 29th, 2008, 03:06 PM

It might improve usability if the gesture icons would in some way indicate a failed gesture. On a slow system (compiling) or with certain commands (firefox), it often takes some time to realize that the gesture didn't work. Conversely, sometimes you think it failed when it didn't, and you accidentally start another process. Some feedback for failed gestures, like a small red 'x' in the corner of the icon, might help.

Anyways, really appreciate your work on this great project. Easystroke has been very useful! Thanks!

Luffield

October 29th, 2008, 03:08 PM

I like your idea a lot, GrouchoMarx.

Tom Jaeger

October 31st, 2008, 11:18 PM

Sorry for the late reply, I've been busy with other things.

I think didn't clarify what I meant about the Annotate plugin. I used Easystroke to activate the Annotate plugin (in addition to using it to draw the current gesture). However, the Annotate plugin only stays active, after I've drawn the gesture I keyed to Annotate, for as long as my pen is on the screen. As soon as I lift the pen, then it deactivates (like the documentation says it would, but "holding" on Super-Alt would enable it to stay toggled). I do have it in ignore mode (that enabled me to key the gesture to just Super-Alt). If I misunderstood you, then I'd appreciate the clarification :)
I think I see what you're saying now. You want easystroke to stay in ignore mode until a particular mouse button is pressed. That's a reasonable suggestion, but I'm not sure yet in how much generality the idea should be exposed or how the best UI for it would look like. It don't think it will happen before 0.3.0, but I'll keep the idea in mind.

Tom Jaeger

October 31st, 2008, 11:23 PM

It might improve usability if the gesture icons would in some way indicate a failed gesture. On a slow system (compiling) or with certain commands (firefox), it often takes some time to realize that the gesture didn't work. Conversely, sometimes you think it failed when it didn't, and you accidentally start another process. Some feedback for failed gestures, like a small red 'x' in the corner of the icon, might help.

Right now, we have notification of successful gestures by popups and of failed gestures by the "bell". I've implemented the suggestion in git for now (see the attached snapshot, and remember to keep backups), but I can't say I like the behavior very much.

sarah.fauzia

November 1st, 2008, 12:19 AM

I know I mentioned before not wanting to see the last stroke in the notification icon, but now I request that you keep it that way, because I've disabled Compiz as it's interfering with screen rotation (see my post on this (https://bugs.launchpad.net/ubuntu/+source/mesa/+bug/132065) launchpad bug). Unless that's fixed, I don't think I'll be using any compositing manager in the future, not even metacity's, for that gave me trouble as well (and I'm worrying about it ruining my computer...I've been getting really nasty artifacts). Not meaning to go off tangent on this thread's discussion, but I just wanted to give my input on this (as Tom said he'd take out the feature unless anyone specially requested it :) ).

I apologize for the fickleness, but I actually appreciate the new version more (without the show gesture feature in the notification icon, despite not using compiz). It looks much more tidier, and I'm all-around impressed by the new features--the touch support is wonderful! I'm so glad I can map a gesture to right-click and right-click with my finger now :). If this was a feature before, I had not noticed it... I definitely noticed the new feature of rearranging gestures, and this was also a big bonus.

Thanks for the great update, Tom! :)

sarah.fauzia

November 8th, 2008, 09:16 PM

pibach

November 9th, 2008, 04:53 PM

I've been really enjoying making gestures for specific applications, especially Xournal.

How do you do that? I had to put xournal on the list of exceptions, to make easystroke not interfere with xournal's inking.

In firefox, easystroke tends to interfere with browsing...
I use easystroke most in Firefox. Works perfectly, no interference. I have it on left click, so I do not need to press the button. G&D scrolling works fine too since Tom has incorporated the gesture timeout. To open links in new tabs I use Quickdrag extension.

Also the scroll gesture is very useful, e.g., to scroll in PDF documents. Still here is some room left for improvements (e.g., "circular scroll").

Another small feature request: I would like to be able to verify the installed version to figure out whether updates did succeed - as there was the update yesterday.

I still have to play with the new features. Any experiences how to use the new app-based gestures? And where can I configure the "bell" that informs me of an unrecognised gesture?

sarah.fauzia

November 9th, 2008, 10:27 PM

If you go to Preferences under Easystroke, there are two columns under Exceptions--Applications, and Button. If you change the button modifier to Button3 (right-click), rather than leaving it as the default (<App disabled>), then you can use all your regular gestures, plus the gestures you configure specially for the specific application (they appear in bold, even the gestures themselves, so you can tell them apart that way), except, while holding down the right-click. For me that isn't hard with my tablet pen, and perhaps it'd be similarly easy for you.

As for the problem I have with firefox, I agree, it's very useful to use gestures in firefox. And if I change the timeout, then it's very convenient (I don't then experience problems such as, when selecting text, it thinks it's my gesture for the Space key). But the con to changing the timeout is that then some of my more advanced gestures (advanced as in the complexity of the shape, without lifting the pen), don't work. I have to repeat the same gesture a few times for it to then finally register. So that's why I prefer temporarily disabling easystroke.

I just checked, I've already set the gesture button for gestures in firefox to Button3, like I was just thinking to do after typing this post!

Tom Jaeger

November 10th, 2008, 11:09 PM

I've been really enjoying making gestures for specific applications, especially Xournal. It now has me worried though; I have so many gestures, what would I do if my computer crashed? Or what if I wanted to use the same gestures on another tablet? I think it'd be really good if easystroke had its own special backup utility, just to backup preferences and gestures (to a user-specified backup location). It'd give me a lot less to worry about then! :)
easystroe saves all its information in the ~/.easystroke directory, so it should be easy to back up. Gesture import/export is planned for a later version (probably not anytime soon, though, because I have to work out issues with boost::serialization, which causes segfaults on invalide data).

Another small feature request: I would like to be able to verify the installed version to figure out whether updates did succeed - as there was the update yesterday.
easystroke --version will show the version string.

And where can I configure the "bell" that informs me of an unrecognised gesture?
Supposedly the bell can be configured in gnome under System->Preferences->Sound. But I disabled it and I'm using a compiz effect instead.

imnotashinobi

November 12th, 2008, 11:11 AM

Amazing application! I love using mouse gestures and would probably love using gestures on a tablet [when i can afford one]. I used to miss autohotkey from my days in xp, until I found easystroke. The only feature I miss now is universal text replacement; I didn't bother much with autohotkey's other abilities. Anyways, the new features in easystroke are great, but I have one request:

Confirmation on "Remove Application/Group".

I don't know if this was asked before, but it would be nice to have. I found out there was no confirmation dialogue the hard way :P. Can't wait to see what is planned next! I even subscribed to SF.

Cheers!

realthor

November 12th, 2008, 11:10 PM

Hello,

I have just started with your application as it's one of my last things to improve on desktop usability, together with gnome-do and different plugins/settings in compiz. I find your app great and would like to report a little on it:

1st: I guess this is a bug...I open nautilus, then go to any picture, open it, it opens with eog and then I perform a close app gesture (the famous DR ) and instead of closing the eog nautilus is closed as if it would be in focus. I first suspected that my setting in compiz regarding "Click to focus" which isn't checked could be to blame but I have tried moving eog to be sure it has the focus and the same result.

Can this be replicated?

2nd: This is a suggestion: In firefox the gestures are very simple, like U,D,L,R and any combination of these ones. This is great for working with a mouse and there is little space for missinterpretations. Can you PLEASE add a checkbox to make each gesture an approximative shape of the real thing (in Adobe Flash if i remember well there is a function to "round" shapes drawn with freehand)

It would be great if each gesture could be approximated to segments of straight lines like up-down, left-right and oblique.

3rd: This is a question :P : I could be wrong here but i guess when I define the same gesture for a specific app -that is defined also globally- the first one takes precedence?! This would be very helpful in a close tab/close app scenario where on applications that have tabs the gesture would close the current tab and in apps that have no tabs it would close the app.

Also here the U,D,L,R, and all oblique variations will be quite helpful as an Down+Right gesture like in firefox wouldn't be ignored if the Right part is too long drawn or mixed up with the Down-only gesture (imagine minimize) if the Right part is to short. At least that's what happened to me.

Best regards and can't wait for an answer to these issues.

Tom Jaeger

November 13th, 2008, 05:41 PM

The only feature I miss now is universal text replacement; I didn't bother much with autohotkey's other abilities.
autokey (http://autokey.sourceforge.net/) might be what you're looking for (never tried it myself).

Confirmation on "Remove Application/Group".
Thanks, I was going to add that, but then somehow forgot. This is fixed in git now and will be part of the next release.

Tom Jaeger

November 13th, 2008, 06:22 PM

1st: I guess this is a bug...I open nautilus, then go to any picture, open it, it opens with eog and then I perform a close app gesture (the famous DR ) and instead of closing the eog nautilus is closed as if it would be in focus. I first suspected that my setting in compiz regarding "Click to focus" which isn't checked could be to blame but I have tried moving eog to be sure it has the focus and the same result.

This is actually intentional, and I go to considerable lengths to track which window the cursor is in and then give it focus when a gesture is performed. It never occurred to me that someone might not want that feature, but if this is important to people, I might consider making it an option.

2nd: This is a suggestion: In firefox the gestures are very simple, like U,D,L,R and any combination of these ones. This is great for working with a mouse and there is little space for missinterpretations. Can you PLEASE add a checkbox to make each gesture an approximative shape of the real thing (in Adobe Flash if i remember well there is a function to "round" shapes drawn with freehand)

It would be great if each gesture could be approximated to segments of straight lines like up-down, left-right and oblique.

Easystroke doesn't store gestures as sequences of straight lines because this is impractical for many use cases (especially on tablets). This allows for a much wider range of shapes that can be used as gestures.

3rd: This is a question :P : I could be wrong here but i guess when I define the same gesture for a specific app -that is defined also globally- the first one takes precedence?! This would be very helpful in a close tab/close app scenario where on applications that have tabs the gesture would close the current tab and in apps that have no tabs it would close the app.

There is no notion of gestures being "the same". easystroke computes a score that reflects how closely two gestures match, which never is 100% since it's practically impossible to draw two identically gestures. So in your situation it would appear more or less random which of the two gestures easystroke chooses. You have two options here:
(1) While the application is selected, just change edit the action for that gesture, it will become bold to indicate that it is different than the default action for that gesture.
(2) Delete the gesture for the application (it can be restored be enabling the option 'show deleted rows' and then resetting the deactivated row) and then create a new one.
Editing a particular application will never affect the default gestures in any way.

realthor

November 13th, 2008, 08:38 PM

This is actually intentional, and I go to considerable lengths to track which window the cursor is in and then give it focus when a gesture is performed. It never occurred to me that someone might not want that feature, but if this is important to people, I might consider making it an option.

As long as I believe that the highest window in the stack is the one the user is focusing his attention at it would be grat such an option as it's easier to close a window by a mouse gesture than trying to reach the X button every time...considering that different sizes winows have different positions for the button.

Easystroke doesn't store gestures as sequences of straight lines because this is impractical for many use cases (especially on tablets). This allows for a much wider range of shapes that can be used as gestures.

Yas it's obvious EasyStroke focuses more on the tablet market but I guess unless all laptops at least will get a touch-screen 95% of the users of your software are mouse-users and from a mouse-gesture point of view it's impractical to create two similar strokes because you'll never make the exact shape of a gesture (most often mouse-gestures exist for speed and you draw an "L" shape to close an app it's always an "L" but sometimes you make it longer on the H axis , other times on the Y axis).
This is why as a mouse user focused on speed an option like "fidelity of capture" to be set to high and low (low would be to aproximate to a few line segments) would be great but it's your choice to implement it of course.

Best Regards and keep it up.

imnotashinobi

November 14th, 2008, 09:42 AM

Thank you very much for the link, Tom. autokey doesn't seem to do anything[?], but I appreciate the heads up. It does seem to be what I'm looking for though. I'll have to go bother the author about it later.

Oh well... on to easystroke matters...
These are pretty much cosmetic matters: I believe the "Timeout Gestures" checkbox should be moved from the "Advanced" tab to the "Preferences" tab. It just makes more sense. The "Timeout profile" drop-down menu is right there. Or, perhaps, switch them? Just my little bit of user interface criticism. Take it or leave it, it's a small matter.

Now for a new "problem" [as in minor, unoccasional annoyance] I found. Would it be possible for the click NOT to go through on timeouts? I prefer to use Button 3 for gestures. However, this produces right clicks at inopportune times.

How do we get this app into the official Ubuntu repositories? It's already so polished and reliable. I can't wait to see what's planned for in the next release. My guess: a popup menu for gestures that are equal to or less than a user-defined percent from each other in response to a gesture that shows enough potential to be another one, thus allowing for two [or more] commands to have nearly identical gestures while still allowing for the user to easily select between them. I've obviously been staring at the "History" tab for too long; but I think that'd be neat!

GrouchoMarx

November 14th, 2008, 05:42 PM

Hi Tom,

Thanks for the update. When using version

easystroke 0.3.0-0ubuntu1~ppa1~hardy

I receive the following error on startup:

XError: BadAccess (attempt to access private resource denied): X_GrabKey

Also, if I hide the tray icon in the preferences, easystroke segfaults on the next attempted gesture.

Let me know if you need any system info.

Besides that, you said that you weren't happy with the check mark and red circle in the system tray icon. An alternative solution would be to show the previous gesture in blue and green if successful (like before), and have failed gestures show in red. No additional emblems would be necessary.

Cheers

realthor

November 15th, 2008, 12:48 AM

Man, I am really getting to realize the power of easystroke. The more gestures I define the less chances are easystroke will mix up gestures.
As long as there are unique shaped gestures for each action it is ok.

I would like to make a couple of suggestion and hear from wou what you think of them:

- the first one i still believe is a bug, as the starting point of my gesture is inside of the eog window, not over the below nautilus window. I agree with closing the app below the starting point of the mouse gesture. Anyway this isn't crucial as I think I can track the active window by xprop's _NET_ACTIVE_WINDOW(WINDOW).

-second it would be nice to make the argument somehow recursive and have a "Add Argument" button for the situations you want a few actions to be played one after another. Nothing that ca't be made with a command type and a script as an argument but we're for user friendly stuff right?

- the most important would be to move the contextual menu a few pixels away for users that use right click for "gesture button" because I get too many clicks on the first option in the contextual menu.

Thanks a lot for your work and hope our suggestion can help the app become better.

Tom Jaeger

November 15th, 2008, 08:10 PM

Yas it's obvious EasyStroke focuses more on the tablet market but I guess unless all laptops at least will get a touch-screen 95% of the users of your software are mouse-users and from a mouse-gesture point of view it's impractical to create two similar strokes because you'll never make the exact shape of a gesture (most often mouse-gestures exist for speed and you draw an "L" shape to close an app it's always an "L" but sometimes you make it longer on the H axis , other times on the Y axis).
This is why as a mouse user focused on speed an option like "fidelity of capture" to be set to high and low (low would be to aproximate to a few line segments) would be great but it's your choice to implement it of course.
There is no question that the recognition algorithm could be improved. I have a few thoughts on how this can be done, but the problem is highly non-trivial and the need for an algorithm that is not too computationally intensive makes it even more challenging. I don't think restricting my attention to a particular use-case is the solution.

Oh well... on to easystroke matters...
These are pretty much cosmetic matters: I believe the "Timeout Gestures" checkbox should be moved from the "Advanced" tab to the "Preferences" tab. It just makes more sense. The "Timeout profile" drop-down menu is right there. Or, perhaps, switch them? Just my little bit of user interface criticism. Take it or leave it, it's a small matter.
The idea here was most users are probably not going to use timeout gestures, which is why I relegated them to the advanced tab.

Now for a new "problem" [as in minor, unoccasional annoyance] I found. Would it be possible for the click NOT to go through on timeouts? I prefer to use Button 3 for gestures. However, this produces right clicks at inopportune times.
Where do these spurious right-clicks come from?

How do we get this app into the official Ubuntu repositories? It's already so polished and reliable.
Version 0.2.1.1 is already in intrepid, but I haven't taken the time to request an update to 0.3.x for jaunty yet. I'll probably wait for at least 0.3.1 for that, but I'll be updating my PPA regularly.

I can't wait to see what's planned for in the next release. My guess: a popup menu for gestures that are equal to or less than a user-defined percent from each other in response to a gesture that shows enough potential to be another one, thus allowing for two [or more] commands to have nearly identical gestures while still allowing for the user to easily select between them. I've obviously been staring at the "History" tab for too long; but I think that'd be neat!
Thanks, this is a good idea and probably not too hard to implement.

Hi Tom,
Thanks for the update. When using version

easystroke 0.3.0-0ubuntu1~ppa1~hardy
I receive the following error on startup:

XError: BadAccess (attempt to access private resource denied): X_GrabKey

This is not a big deal, it just means that you won't be able to use Esc to cancel gestures.

Also, if I hide the tray icon in the preferences, easystroke segfaults on the next attempted gesture.

Thanks, fixed in git. This is the reason why I'm always a little reluctant to introduce new options: They greatly increase the potential for regressions.

- the first one i still believe is a bug, as the starting point of my gesture is inside of the eog window, not over the below nautilus window. I agree with closing the app below the starting point of the mouse gesture. Anyway this isn't crucial as I think I can track the active window by xprop's _NET_ACTIVE_WINDOW(WINDOW).

The intended behavior is for the application where the gesture started to get focus, so yes, this looks like a bug. What window manager are you using? Also, can you confirm that the bug doesn't happen if you move the cursor out of the eog window first and then back in?

-second it would be nice to make the argument somehow recursive and have a "Add Argument" button for the situations you want a few actions to be played one after another. Nothing that ca't be made with a command type and a script as an argument but we're for user friendly stuff right?

If you just need a sequence of commands, you can use 'foo; bar; ...; baz' since commands are executed in a shell; I don't think there's a need for a fancy gui here. If you're trying to mix commands with key presses, I'd be interested in a use case.

- the most important would be to move the contextual menu a few pixels away for users that use right click for "gesture button" because I get too many clicks on the first option in the contextual menu.

I can't control the position of context menus. How do these spurious clicks come about, though? Unintended timeouts?

Tom Jaeger

November 23rd, 2008, 11:01 PM

- the first one i still believe is a bug, as the starting point of my gesture is inside of the eog window, not over the below nautilus window. I agree with closing the app below the starting point of the mouse gesture. Anyway this isn't crucial as I think I can track the active window by xprop's _NET_ACTIVE_WINDOW(WINDOW).

Could you check if this bug is fixed in the attached snapshot?

realthor

December 2nd, 2008, 02:17 PM

Yes!!!This snapshot fixed the issue. Thanks.

realthor

December 2nd, 2008, 08:39 PM

Risking to get really annoying I have noticed that mouse detection is better without compiz that when special effects are activated. If I deactivate visual effects I seem to double the amount of pixels drawn on the screen by easystroke. This can be noticed when you draw very quickly a gesture, with compiz on sometime i get only the final one or two strokes. Can this be improved? Mouse gestures equals speed and making them in a speedy way makes the work more efficient... .

- pushing my luck: what about on-screen strokes with anti-aliasing, opacity, glow effects? Are they difficult to implement? You could use compiz effects to do this?

Thanks.

Tom Jaeger

December 2nd, 2008, 11:24 PM

- pushing my luck: what about on-screen strokes with anti-aliasing, opacity, glow effects? Are they difficult to implement? You could use compiz effects to do this?
The XShape extension and compiz don't play together very well. I would therefore recommend to draw strokes via the 'annotate' plugin when running compiz (needs the 'dbus' plugin to be enabled). This will also give you anti-aliasing and control over width and opacity (no glow effects, though). This should probably be documented better, but I'm already behind on the documentation of the new features...

sarah.fauzia

December 6th, 2008, 03:23 PM

Tom Jaeger

December 6th, 2008, 07:08 PM

Application specific gestures are not working in Kubuntu Intrepid. I thought it was my gesture button that was perhaps off (the right click button Button3), but even when I try in other applications (where I use Button1), easystroke will show the stroke being made (I have XShape enabled), but the gesture doesn't seem to register. In xournal, if I leave the gesture button as Button3, the gesture is not even drawn on the screen (unless I use Button1), but even then the gesture doesn't do anything. All other gestures that are not application-specific work perfectly! I'm not sure if you're planning to support KDE4, but I just tried it out (and like it), and would hope this bug is possible to fix. Thanks!

Okay, so the kde window manager nests application windows more deeply than necessary (and gratuitously so; the parent of the application window has exactly the same geometry and only one child), which seems like a bad idea because it probably confuses more applications than just easystroke. Anyway, I've implemented a breadth-first search when looking the application window, which should fix your kde issues and will hopefully have minimal impact on the behavior under other window managers.

sarah.fauzia

December 7th, 2008, 04:26 AM

Tom Jaeger

December 7th, 2008, 04:48 AM

Could you possibly incorporate this as an update? It might be because I'm new to Kubuntu, but I'm having trouble executing the program from file. Again, thanks, I truly appreciate your quick response :)
This needs more testing before I can release it. I've repackaged the snapshot as a tar.gz, which preserves permissions and should therefore be easy to launch from the GUI.

sarah.fauzia

December 8th, 2008, 02:53 AM

This needs more testing before I can release it. I've repackaged the snapshot as a tar.gz, which preserves permissions and should therefore be easy to launch from the GUI.

Hmm... it's still not opening Easystroke, after extracting the archive. Is this only for 32-bit Kubuntu? (just looking at the package name). I use 64-bit.

It did occur to me later that it would be unwise to immediately upgrade it in the PPA (in case of regressions), but could you possibly include the whole source file in the archive, so I could install it from source? I hope it wouldn't be inconvenient.

Tom Jaeger

December 8th, 2008, 03:34 AM

Hmm... it's still not opening Easystroke, after extracting the archive. Is this only for 32-bit Kubuntu? (just looking at the package name). I use 64-bit.
Yes, sorry, the binary will only work in a 32-bit environment and I don't have a 64-bit installation that I could build this on. See the wiki (http://easystroke.wiki.sourceforge.net/BuildInstructions) for instructions on how to build from source. The following should work:

sudo apt-get install g++ libboost-serialization-dev libgtkmm-2.4-dev libxtst-dev libdbus-glib-1-dev
git clone git://github.com/thjaeger/easystroke.git
cd easystroke
make -j2
./easystroke

sarah.fauzia

December 8th, 2008, 02:52 PM

Thanks Tom! Now Easystroke is working perfectly--but I had to disable KDE's desktop effects first (it was making the strokes render too slowly). No other quirks I can see other than Easystroke not showing up in my programs list in KDE; is there a way to specify the correct installation directory? I'm assuming that's why it isn't showing up, as I had to create some directories in /usr/share for it to install; specifically, /usr/local/share/applications (where I had to create the applications folder), and /usr/local/share/icons/hicolor/scalable/apps for the easystroke icon (where I had to create the scalable and apps folders). If I didn't create those directories, the installation would abort. I hope my input helps, and, again, thank you for responding so quickly! If you hadn't updated it for me Kubuntu would have been utterly useless without Easystroke!

I'm attaching the DEB package checkinstall created when I installed Easystroke, both for reference and if anyone needs it.

Actually, the default launcher for Easystroke is now appearing in the Kubuntu menu. I don't know if it's because I created a custom launcher and it did something, or if it took time... I do know that when I went and made the launcher, there wasn't yet a default launcher for Easystroke.

imnotashinobi

December 10th, 2008, 06:43 AM

is it not meant to work with caps lock on?

Tom Jaeger

December 12th, 2008, 02:42 AM

Thanks Tom! Now Easystroke is working perfectly--but I had to disable KDE's desktop effects first (it was making the strokes render too slowly).

Same comment as above, XShape and a composited desktop don't play along very well. In compiz, I can work around this by using the annotate plugin to draw on the screen, not sure if there's something similar for the kde window manager. I need to look into making 32-bit visuals work (I tried this once without any luck).

No other quirks I can see other than Easystroke not showing up in my programs list in KDE; is there a way to specify the correct installation directory?

make PREFIX=/usr install

If I didn't create those directories, the installation would abort.
That's odd, 'install -D' should create those directories automatically.

Tom Jaeger

December 12th, 2008, 03:03 AM

is it not meant to work with caps lock on?

Apparently X treats Caps Lock as a modifier again (last time I tested this, this wasn't the case). Fixed in git and the attached snapshot.

imnotashinobi

December 12th, 2008, 08:22 AM

Thanks! I was panicking the first time I disabled it via caps lock and wondered what happened to easystroke. ^_^;

zefew

December 15th, 2008, 05:17 AM

Neither easystroke 3.0 nor git HEAD don't compile on gutsy. It's using
format function which is not defined in the Glib::ustring header. Has anyone
installed easystroke on gutsy successfully? If so how did you get around the error?

$ sudo apt-get install g++ libboost-serialization-dev libgtkmm-2.4-dev libxtst-dev libdbus-glib-1-dev
$ make -j2
...
stats.cc:116: error: ‘format’ is not a member of ‘Glib::ustring’
stats.cc:136: error: ‘format’ is not a member of ‘Glib::ustring’
stats.cc: In member function ‘void Stats::on_pdf()’:
stats.cc:226: error: ‘format’ is not a member of ‘Glib::ustring’
g++ -Wall `pkg-config gtkmm-2.4 gthread-2.0 dbus-glib-1 --cflags` -Os -MT util.o -MMD -MP -MF util.Po -o util.o -c util.cc
make: *** [stats.o] Error 1
make: *** Waiting for unfinished jobs....

Tom Jaeger

December 15th, 2008, 09:33 AM

Neither easystroke 3.0 nor git HEAD don't compile on gutsy.

The attached patch should do it. Notice that easystroke isn't tested with the version of xserver that is in gutsy, and there's likely to be breakage. You might need to start easystroke as "easystroke -x" to get gestures at all.

Tom Jaeger

December 15th, 2008, 10:05 AM

I'd like to get a new version 0.3.1 out soon. The biggest change is a new method to show gestures on a composited desktop. Initial testing on my hardware looks promising, but I'd like to get some more testing done on different setups before I release it. If I don't get any complaints, I'll update a new package to my PPA tomorrow.
Apart from that, I've fixed a few bugs since 0.3.0 and added an option to configure autostart.

zefew

December 15th, 2008, 10:30 AM

The attached patch should do it. Notice that easystroke isn't tested with the version of xserver that is in gutsy, and there's likely to be breakage. You might need to start easystroke as "easystroke -x" to get gestures at all.

Thanks for the patch, Tom.

Apart from `format' thing, which your patch resolves, I found there was another
errror:

win.cc:197: error: ‘class Glib::RefPtr<Gtk::StatusIcon>’ has no member named ‘reset’

So i temporarily comment #197 out, after which the built went okay. But, as
soon as I execute easystroke (either with -x or not), I get the following
segmentation fault.

| $ ./easystroke
| XError: BadClass, invalid event class: request_code=147, minor_code=6
|
| (easystroke:29334): gtkmm-CRITICAL **: gtkmm: object `treeview_actions' not found in GtkBuilder file.
| segmentation fault (core dumped)
| $ ./easystroke -x
|
| (easystroke:29343): gtkmm-CRITICAL **: gtkmm: object `treeview_actions' not found in GtkBuilder file.
| segmentation fault (core dumped)

Do you think I need a different version of libgtkmm? my system currently has:

libgtkmm-2.4-1c2a install
libgtkmm-2.4-dev install
libgtkmm-dev install
libgtkmm1.2-0c2a install
libgtkmm2.0-1c2a install
libgtkmm2.0-dev install

Tom Jaeger

December 15th, 2008, 05:21 PM

win.cc:197: error: ‘class Glib::RefPtr<Gtk::StatusIcon>’ has no member named ‘reset’

reset used to be called clear in earlier versions of gtkmm.

| (easystroke:29334): gtkmm-CRITICAL **: gtkmm: object `treeview_actions' not found in GtkBuilder file.

Make sure that libxml2-utils is installed and rebuild. You might need a newer version of the gtk-builder-convert script.

sarah.fauzia

December 23rd, 2008, 08:18 AM

I just installed openSUSE 11.1 on my system (the same x61 tablet), and again, this time, with KDE 4. I interestingly still needed to create the same folders in order for checkinstall to proceed (easystroke isn't in openSUSE's repos :(), and again I had to create a menu entry before a menu entry would appear. But, otherwise, easystroke seems to run without a hitch in openSUSE :)

Spoke slightly too soon. There appears to be an issue in portrait mode. I don't remember this occurring in Ubuntu, but it did happen in Kubuntu: When in portrait mode, and I make a gesture towards the bottom of the screen, only the top half of the stroke shows (if at all). It can get to the point where nothing shows. In Kubuntu (I haven't checked in openSUSE yet), disabling gestures showing on the screen... did not disable gestures from showing on the screen. But the stroke, despite it not showing properly, was properly recognized by Easystroke. Any thoughts?

Checked in openSUSE, disabling gestures showing worked. It could be that I was using the Easystroke that you had just fixed for me (in Kubuntu), and not the latest 3.1 that was just released (as I am using now). Maybe. But the issue in portrait mode still stands.

Tom Jaeger

December 23rd, 2008, 03:39 PM

Spoke slightly too soon. There appears to be an issue in portrait mode. I don't remember this occurring in Ubuntu, but it did happen in Kubuntu: When in portrait mode, and I make a gesture towards the bottom of the screen, only the top half of the stroke shows (if at all). It can get to the point where nothing shows. In Kubuntu (I haven't checked in openSUSE yet), disabling gestures showing on the screen... did not disable gestures from showing on the screen. But the stroke, despite it not showing properly, was properly recognized by Easystroke. Any thoughts? [/EDIT]
In 0.3.1, I've handed screen size change notification off to gdk, which listens for ConfigureNotify events on the root window, a method I thought was at least as reliable as the randr notification events I used previously. I'm assuming that restarting the app while in portrait mode fixes the problem.

I can't reproduce this, but when testing kwin, I found a more serious problem, which is hopefully restricted to xserver-1.6: The X server sneaks in an EnterNotify between reporting the core and Xi button press, which throws easystroke off. The timestamps are still identical, so this is fixable.

Speaking of xserver-1.6, there've been a few changes that broke the way easystroke implements some of its advanced features. It looks like I can fix this by using an entirely different method, but it's quite a bit of work (this work is happening mostly on the 'next' branch in git, in case someone's running jaunty).

sarah.fauzia

December 23rd, 2008, 11:55 PM

You're correct, restarting Easystroke in portrait mode does fix the problem. Good to know!

momonari

December 30th, 2008, 04:41 PM

Hi,
I've just started using easystroke.
I've found that sometimes, easystroke "locks up" my input devices. I think it only happens when I define a gesture for a particular application. E.g., I defined a gesture to hit the Escape key for aMSN message windows, easystroke just calls the application "container_0". Sometimes it works, but other times, it will lock up, my computer doesn't respond to any mouse or keyboard input but my OS is still running.
What could be the cause of this? And is there something I can do when it happens to get easystroke to unlock my mouse and keyboard?

Thanks.

Momonari

Islington

December 30th, 2008, 05:12 PM

Tom Jaeger

December 30th, 2008, 09:15 PM

What could be the cause of this? And is there something I can do when it happens to get easystroke to unlock my mouse and keyboard?

I can't reproduce the issue on a 1.5.3 X server, but I get a simliar behavior (100% reproducible) on the 1.6.0 beta servers. The cause might be the same or it might not. In my case, this looks like a bug in the X server, which seems to have problems handling overlapping pointer and keyboard grabs. Another cause could be misbehaving device drivers.

Does the problem only occur with the Escape key? What window manager are you using? What version of the X server? Can you check if the attached binary (which removes the call to XGrabKey) fixes the problem? If not, could you send me the output of "easystroke -vvv" when this problem occurs?

Normally, you should be able to reset easystroke by pressing Escape, but it looks like this is actually part of of the problem. The other option is to switch to a console (or use ssh) to kill easystroke. That should release the grab that is responsible for the lockup.

momonari

December 31st, 2008, 02:07 AM

Does the problem only occur with the Escape key? What window manager are you using? What version of the X server? Can you check if the attached binary (which removes the call to XGrabKey) fixes the problem? If not, could you send me the output of "easystroke -vvv" when this problem occurs?

I'm kinda new to Linux so I don't know how to find most of that. I'm using Ubuntu 8.10 with Compiz. How do I find out what window manager I'm using and what version of the X server?

It only happens with the Escape key, whether I define it for a particular application or define it as a global gesture.
The attached binary fixes the problem, however, after I close the message window with the gesture, I cannot type in aMSN anymore, i.e., if I open another message window, I can't type. I must restart aMSN. However, I believe this is a problem with SCIM and not easystroke.
Here is the output when I close the message window with the gesture (with the attached binary), my gesture button is the right mouse button, the gesture looks like the letter u and is of type "Key", namely, the Escape key:

Excecuting Action Gesture 15...
New event handling stack: Idle
Motion: (804, 660)
Release: 3 (804, 660)
Entered window 0x3000323 -> 0x3000323
XError: BadWindow (invalid Window parameter): X_GetProperty
Entered window 0x40000cf -> 0x40000cf

Thanks.

Momonari

Tom Jaeger

December 31st, 2008, 02:09 PM

I'm kinda new to Linux so I don't know how to find most of that. I'm using Ubuntu 8.10 with Compiz. How do I find out what window manager I'm using and what version of the X server?

Thanks, that's all the information I need. Compiz is the window manager.

It only happens with the Escape key, whether I define it for a particular application or define it as a global gesture.
The attached binary fixes the problem,

Okay, so the lesson we learn from this is that synchronous pointer and keyboard grabs can't be safely combined. I'll most likely just remove the ability to abort gestures in the next versions.

however, after I close the message window with the gesture, I cannot type in aMSN anymore, i.e., if I open another message window, I can't type. I must restart aMSN. However, I believe this is a problem with SCIM and not easystroke.

They're lots of reports out there of SCIM causing problems like this, but bug #66104 is closed for some reason.

Excecuting Action Gesture 15...
New event handling stack: Idle
Motion: (804, 660)
Release: 3 (804, 660)
Entered window 0x3000323 -> 0x3000323
XError: BadWindow (invalid Window parameter): X_GetProperty
Entered window 0x40000cf -> 0x40000cf

Thanks. Nothing unusual going on here.

momonari

December 31st, 2008, 04:22 PM

Okay, so the lesson we learn from this is that synchronous pointer and keyboard grabs can't be safely combined. I'll most likely just remove the ability to abort gestures in the next versions.

So I should be able to gesture the Escape key in a future version? Please update us when you fix this. Thanks.

They're lots of reports out there of SCIM causing problems like this, but bug #66104 is closed for some reason.

I never found a way to get SCIM working properly with aMSN. Whenever something happens in the message window, I have to click away and click back to input more text. This is one of the times I've had to actually restart aMSN to fix it. But that's off-topic.

Tom Jaeger

December 31st, 2008, 05:02 PM

So I should be able to gesture the Escape key in a future version? Please update us when you fix this. Thanks.

Yes, definitely. I'm planning on releasing the new versions (0.3.2 for xservers <= 1.5 and 0.4.0 for xserver 1.6) when the new xserver comes out.

sarah.fauzia

December 31st, 2008, 07:57 PM

Is this possibly related to a problem I have with Nautilus? I can't say it happens every time, but at least almost every time I try to do a file transfer, while easystroke is open, long lines (in the color that I make gestures in), appear on the screen, and go crazy, splitting off into many directions. This does not allow me to see what's going on with the file transfer, or do anything to stop the lines from continuing to appear. Closing easystroke before the file transfer prevents this problem. Since I only use Nautilus, I'm not sure if the problem is keyed to Nautilus (no longer using KDE, so I'm not sure if it happens with Dolphin as well). I thought that just temporarily disabling easystroke would do the trick--but it didn't.

This isn't a big inconvenience, but I thought I should mention it in case anyone else is experiencing similar problems.

momonari

January 1st, 2009, 02:52 AM

sarah.fauzia

January 1st, 2009, 08:01 PM

I'm on Nautilus and it's not happening here.

It might also be computer specific because it happens on my Motion tablet, and my sister's (both of the same model), but I haven't (yet) seen it happen on my X61t now that I think about it. I'm using almost the same exact install on both my Thinkpad and my Motion (Ubuntu 8.10 64-bit, Openbox, same software running, overall same configurations as I don't like using two computers on a daily basis with different interfaces). And my sister, running an almost pristinely fresh install of Ubuntu Intrepid (64-bit) has the same exact problem (so it can't be Openbox-specific, or specific to other interfering software). Hope there's a solution :)

I'll try to take a screenshot the next time I see it happen to give a better idea.

Tom Jaeger

January 5th, 2009, 06:20 PM

It might also be computer specific because it happens on my Motion tablet, and my sister's (both of the same model), but I haven't (yet) seen it happen on my X61t now that I think about it. I'm using almost the same exact install on both my Thinkpad and my Motion (Ubuntu 8.10 64-bit, Openbox, same software running, overall same configurations as I don't like using two computers on a daily basis with different interfaces). And my sister, running an almost pristinely fresh install of Ubuntu Intrepid (64-bit) has the same exact problem (so it can't be Openbox-specific, or specific to other interfering software). Hope there's a solution :)

I'll try to take a screenshot the next time I see it happen to give a better idea.

Hmm, could be a bug in the intel driver that is specific to GMA950. Do you have a compositor running (i.e. are strokes anti-aliased or not)? I'm assuming you're using the "Default" method.

aikiwolfie

January 5th, 2009, 07:35 PM

Islington

January 5th, 2009, 07:38 PM

Lightstar

January 5th, 2009, 09:54 PM

Just dropping by to say that I totally love this thing!

Way better than Gestikk.. at least for me!
Gestikk was way too strict with my moves.

In my short linux experience (been on linux 100% for 2 years) this program is probably the one I'm the most happy about.

momonari

January 6th, 2009, 03:11 AM

The only place I've ever seen a genuine need for gestures was on Bump Top. And even then Bump top users could probably manage with normal controls. Sorry I just don't get the whole fascination with gestures. Mostly they seem to duplicate stuff that can be done faster with a simple point and click. Like maximize a Window.

Maybe maximizing a window is a bad example if you want to sell this to people.

It's more convenient to be able to do everything with the mouse than having to let go of my mouse and press keys on my keyboard. E.g., to go to my desktop on the right, I would have to press Ctrl+Alt+Right, with a mouse gesture, I just draw a line to the right.

Most of the time, I'm sitting back in my chair or holding my chin with my left hand, so if I can do everything with the mouse with my right hand without having to change my posture, it's a lot easier.

And as others have pointed out, you don't have to move your mouse to whatever button (e.g., the maximise button) and perform the action, you just draw the gesture wherever your mouse is.

momonari

January 6th, 2009, 03:18 AM

Double post, please delete.

The only place I've ever seen a genuine need for gestures was on Bump Top. And even then Bump top users could probably manage with normal controls. Sorry I just don't get the whole fascination with gestures. Mostly they seem to duplicate stuff that can be done faster with a simple point and click. Like maximize a Window.

Maybe maximizing a window is a bad example if you want to sell this to people.

It's more convenient to be able to do everything with the mouse than having to let go of my mouse and press keys on my keyboard. E.g., to go to my desktop on the right, I would have to press Ctrl+Alt+Right, with a mouse gesture, I just draw a line to the right.

Most of the time, I'm sitting back in my chair or holding my chin with my left hand, so if I can do everything with the mouse with my right hand without having to change my posture, it's a lot easier.

And as others have pointed out, you don't have to move your mouse to whatever button (e.g., the maximise button) and perform the action, you just draw the gesture wherever your mouse is.