It appears I can't boot into Ubuntu from either LiveCD or LiveUSB with any version of Ubuntu that I tried (11.04 and 11.10).
I can boot into a previously installed 11.04 version, but I wish to scrap that and start again.

The problem I get is during the first part of Ubuntu's boot (where it shows the Kernel status at time-marked intervals), it freezes at about 6 seconds, just after showing something along the lines of 'USB Root Hub' and '4 Ports detected'.

I can't exactly remember what, I'll give more details if you need them. Basically, I started getting this error since I upgraded my PC. I tried removing all USB devices, but I have a feeling it's an unsupported South Bridge or something.

Core i7 2600K Processor
Gigabyte GA-Z68X-UD5 (B3) Motherboard
4x 2GB Corsair PC3-12800 RAM

Only USB device I left in:
Logitech G510 Keyboard