I think you should try also Lubuntu (because it has an ultra-light foot-print, while xubuntu has a light foot-print).
- from a live desktop iso file (the clean way) or
- installing LXDE
sudo apt-get install lxde
- installing the whole lubuntu-desktop
After trying, you can select your sweet point in the trade-off between speed and eye-candy. And of course, you can run a mixture of the two flavours. I run such a mixture in one computer (Lubuntu with xubuntu-desktop).
sudo apt-get install lubuntu-desktop
If you need extra speed and efficient use of RAM, there is plain Openbox window manager as a log in option in Lubuntu (or lubuntu-desktop). Right-click on the grey background to get a terminal window or browser.