Following the steps here
I have successfully installed Ubuntu 8.04 on a 2GB USB disk.
Now I can save all my settings and boot from USB disk.
However, I found that booting time from live CD and USB disk are almost same!
Although applications like Open Office loads faster from USB disk compared to live CD.
I love Ubuntu