There is an easy way, limited to 4GB Persistence, (see above).
and there is a hard way, (unlimited Persistence).
Boot Live CD or Live USB.
Plug in flash drive.
Create 2 GB FAT32 partition, (on the left side of the bar). (size is optional, extra space can be used for file storage and transfer to Windows machines).
Create a 4 GB ext2 partition to the right of this, labeled it "casper-rw". (ext3 and ext4 also work).
Create a partition in the remaining space and label it "home-rw". (optional, creates a separate home partition).
Un-mount and re-mount flash drive.
Start "Create a live usb startup disk", (usb-creator).
Select "Discard on shutdown".
Press "Make Startup Disk.
When usb-creator finishes, Go to the root folder of your Live USB
Enter the syslinux directory, (or for UNetbootin the root directory).
Make the syslinux.cfg file writeable
Replace the contents of the file syslinux.cfg with:
Shutdown, remove CD, reboot.
say Booting a persistent Ubuntu session...
append file=/cdrom/preseed/ubuntu.seed boot=casper persistent initrd=/casper/initrd.lz quiet splash noprompt --
First time booting go to users and groups and create an account with yourself up as an Administrator, with password if desired.
The above code will bypass the Try/Install and Language screens.
For 64 bit change vmlinuz to vmlinuz.efi