I've seen posts about getting down to XFCE4 after install but what I wanted was just XFCE4 from the start. Nothing else. I found a sight in French that had the info I needed so I thought I'd redo it for you all here.
This is geared towards an x86 install. If you are installing on a PPC system, the acpi stuff may not work for you. Actually, it may not work on an x868 system either. It all depends on your hardware.
This is short and sweet as I figure if this is what you're looking for, you probably don't need too much hand-holding.
(I should add that after completeing this, I had what I wanted but it turned out that I actually had no idea how much I'd miss Gnome. So, I'm back to the normal install.)
Boot with the CD inserted. At the initial prompt, type "server" and presss enter. This will get you your bare (or bare-ish) install. Once the install is complete and you are logged in, change your sources.list to include the universe, multiverse and, optionally, MPlayer repositories. Instructions are here.
(NOTE: In those instructions you will see a line that reads
Since you did the "server" version of the install, you won't have gedit available to you. You can use either vi or nano. So replace that line with either
sudo gedit /etc/apt/sources.list
sudo nano /etc/apt/sources.list
If you're new to Linux, you'll probably find nano a lot easier to use than vi.)
sudo vi /etc/apt/sources.list
Once that is done, do
sudo apt-get update
Firefox is not at all necessary but most people seem to want it so I threw it in there. Take it out if you want.
sudo apt-get install x-window-system-core xfce4 synaptic gnome-sudo gdm acpi acpid powermanagement-interface mozilla-firefox
Reboot and you will have a very bare install using XFCE4 without Gnome. From there just apt-get your way to your own little custom Xubuntu. (I said it!)