Compiz, Head Tracking, and the 3d Interface

[klange]

Watch the video on YouTube.

What it is: The Compiz `headtracking` plugin uses OpenCV to track the three-dimensional location of your head relative to your screen and puts that data into some OpenGL transformations in Compiz to create the illusion that your display has depth.

What you need: You will need a reasonable webcam that is V4L compatible (or in some other way works with OpenCV). You will also need to obtain OpenCV from CVS, as the Ubuntu package (even in Intrepid) lacks many important features the plugin makes use of (note: the libtool script was broken for me, its $echo wasn't set properly). Then you can get the plugin itself. You'll need a recent version of Compiz, preferably from git, but it should compile with Intrepid (I think).

Who made this: The original distortion code is mine and was made for use with a Wiimote. The OpenCV tracking code was added by pingouinfarceur, who it would appear joined the C-F forums solely to post the patches.

How it works: OpenCV is a video processing library that can track objects based on a definition file (which you'll probably want to make for your own face, unless you look like pingouinfarceur). This data is then used with some fancy trig and the OpenGL glFrustrum() function to distort the viewport in which Compiz renders. In order to make the effect visible, windows are also offset based on their z-depth (stacking order).

Looking forward: If we work out the minor glitches, we may get `headtracking` merged into plugins-extra, and it may come standard with 9.04 - we'll see. Personally, I don't even have a camera that works with OpenCV, so working bugs out and extended testing isn't something I can do myself. At the very least, I'll try and get it packaged for Intrepid and put up a PPA later on.

Discuss.

[cardinals_fan]

I think that's very cool, but how would you actually use it in your day-to-day life?

[klange]

I think that's very cool, but how would you actually use it in your day-to-day life?

To bring something over from the window-style GUI thread, you can use head tracking to literally look behind windows (though not full screen windows in the current set up). The other possibility has already been demonstrated outside of the realm of Compiz (and a long time ago): viewing 3d models (which we can do in Compiz with the `cubemodel` plugin).

[smoker]

anything new is a bonus, look forward to it's further development

[cardinals_fan]

To bring something over from the window-style GUI thread, you can use head tracking to literally look behind windows (though not full screen windows in the current set up). The other possibility has already been demonstrated outside of the realm of Compiz (and a long time ago): viewing 3d models (which we can do in Compiz with the `cubemodel` plugin).

I see. I prefer the tiling approach, but that could be quite useful with a floating window manager.

[dracule]

i think i am going to pick up a webcam for this. i been meaning to get one.

[razerbug]

This just makes you sea sick - faux 3D on a 2D screen looks like a bad "hologram" you used to get in 90's comics.

[dracule]

what happens if two people are sitting side by side? or if someone walks in behind you?

[klange]

what happens if two people are sitting side by side? or if someone walks in behind you?

I'm not sure how much leeway OpenCV will give, but assuming they look nothing like you it shouldn't cause problems for you - they'll see a slightly distorted screen, though. If your twin walks into view, that may screw with things.

[dracule]

I'm not sure how much leeway OpenCV will give, but assuming they look nothing like you it shouldn't cause problems for you - they'll see a slightly distorted screen, though. If your twin walks into view, that may screw with things.

Sounds pretty cool. is there a way to see a list of supported cams?