Update for Lucid/Maverick:
Lucid and Maverick already have everything you need. Furthermore, Maverick has va-api integrated into VLC. When you install VLC, it will pull in everything you want except vdpau-va-driver (for nvidia chips) or i965-va-driver (for Intel gma4500 or newer chips) -- install them manually. If you have a newer ATI/AMD chip (Radeon HD xxxx) you can go here to install the xvba-video package for your arch:
Note: you have to be using the fglrx graphics driver we all love so much for this to work, and even then there's no guarantee.
So, Maverick has to be one of the first Linux distros that includes support for hardware-accelerated video decoding on almost any graphics chip made in the past couple of years. How freakin' cool is that?
BTW, future nvidia/amd driver updates can be found in the X-Updates PPA here:
Update for Karmic:
This PPA contains updated mplayer/smplayer builds, xine-vdpau, all the latest Nvidia drivers, a kaffeine build for DVB-S2 users, the latest libvdpau library and vdpauinfo packaged for Ubuntu. A lot of it is packaged for multiple distros. You still must configure smplayer as described below.
If you know you want this, skip the preamble. If not, indulge me. Why should you want this? The h.264/x.264 decoding. This codec is extremely CPU intensive. My system, which is a Quad core with an 8800GT, can't play 1080 x264 files if the bitrate is ~10mbps. Not with a guarantee of smoothness the whole way through. So if that rig can't do it, yours probably can't either. With the setup described below, my system plays those files smoothly as an android's bottom. And mplayer's CPU usage is 1% every time, no exceptions. This is what it's like when your video card's manufacturer cares about their customers. Before you proceed: VDPAU does not work with Xinerama, but does work with Twinview.
With these instructions, I'm assuming some knowledge of the terminal and package installation, Synaptic and PPAs. These instructions are not basic enough for a total newbie. That's intentional on my part. You should not be trying to bugger around with this stuff unless you know what you're doing to start with.
If you want to use the Restricted Drivers Manager to enable the driver, make sure you install the modaliases package. If not, make sure the nvidia driver is in the /etc/X11/xorg.conf file. Not much else needs to be there. Something like this would be sufficient:
There's something about Nvidia's numbering system that should be understood. The 8800GTX won't work with VDPAU. Why? Because it was actually released before some of the other 8k series cards that have the newer PureVideo 2 hardware. The 8800GTS 512 will work though. Confusing enough? All 9k cards and GTX cards will work, as well as all other 8k cards (that I know of). No 7k or older cards will work, nor will any AGP card that I know of.Section "Screen"
Identifier "Default Screen"
Identifier "Default Device"
Option "NoLogo" "True"
I would avoid installing the regular mplayer package, because it contains the long-obfuscated gmplayer gui. Use another gui. After you install mplayer-nogui run:
Test your x264-encoded MKV files.$mplayer -vo vdpau -vc ffh264vdpau path/to/file
Now to Smplayer. This, for those that don't know, is a QT frontend to mplayer with lots of neat options. The best frontend for mplayer in my view. Gnome-Mplayer is also very good, and is a new gtk gui. The Nvidia VDPAU Team PPA has Smplayer and so does Karmic.
If you don't want to use it, fine. If you do, read on. Once you've got it installed, select Options>Preferences. Click on the Video tab. Change the output driver to VDPAU. You should now be able to use Smplayer with the mplayer backend and play all x264 flicks and standard DVDs with little or no CPU usage. For all other files, Smplayer will automatically select the appropriate codec.
VDPAU support has been added to Xine and the aforementioned PPA has xine-vdpau packages.
Mythbuntu builds with vpdau: http://mythbuntu.org/auto-builds
XBMC builds with vdpau: https://launchpad.net/~team-xbmc/+archive/ppa
And so we arrive, as you knew we must, at VLC. VLC is available with vdpau by way of VA-API, in this PPA: https://launchpad.net/~nvidia-vdpau/...dge-multimedia
However, to install it you'll have to upgrade your precious and very touchy ffmpeg packages. There have been a couple of SONAME upgrades since Karmic's ffmpeg was released, and this will inevitably result in breakage of anything that A) uses dynamic ffmpeg, and B) is not rebuilt against the new version, ie. every package that's on your system that didn't come from that PPA. The gstreamer-ffmpeg package will ensure that your gstreamer system will still work, and there's a build of xine-1.2 (currently unstable) for KDE users. It's possible that in the near future the intel 965 va-api driver might appear in that PPA as well.
Feed me Seymour, Feed me!
Stephen Warren talking about level 4.1 limits in the previous drivers
Line 94 and 704 hacks
Wikipedia page on PureVideo
List of supported Cards/Chips