walter554
July 4th, 2009, 11:50 AM
This is not general instructions for installing an Ubuntu server – I assume you know at least something about that, but it is instructions for how to get a server that supports virtualisation – and KVM virtualisation with vmbuilder in particular. I don't assume you have a graphical enviroment, I used the CLI to do this. Nor are these instructions relevant for a workstation. A server is usually trying to solve different problems.
This example shows how to set up virtualisation and install a ubuntu 9.04 guest
If you already know why you want a virtual environment, then you can skip this section. If you’re new to the subject, virtualisation allows you to run multiple ‘virtual servers’ or virtual machines (VMs) on a single physical box. You would want to do this because:
Instead of installing every bit of software on one server you can install each bit on its own virtual server. Then if that bit doesn’t work or is replaced by something else, you can simply re-configure that one virtual server.
It is really easy to create, copy and delete virtual servers, so you can stand up a new environment in minutes, and throw it away in even less time. This is great for testing and development.
If you’re starting out small, you can run each part of your business on a virtual server and as your business grows you can migrate the virtual servers to physical servers as required.
Other source references
I found some docs quite confusing – one of the problems is, I think, that the processes have changed quite rapidly and the web help is taking a while to catch up. Also, I didn’t have a linux box running x-windows to act as a graphical virtual console (which may have made things very much easlier), and I think that maybe some other 'how tos' are assuming a graphical environment.
Anyway, I’m sure there are better ways (and feel free to add your 2 cents), but this worked for me.
The ‘manual’ referred to is the Ubuntu Server Guide here (https://help.ubuntu.com/9.04/serverguide/C/serverguide.pdf)
Hardware
I started with a small new-ish machine with an AMD quad-core processor that had the AMD-V virtualisation extensions. Modern Intel processors have similar extensions. While virtualisation works on older machines, you may be better off with the free VMware Server on these machines, IMHO.
Step 1. Install Ubuntu
This is a quick overview.
Download the .iso image and burn it to CD. I found the .iso here (http://www.ubuntu.com/getubuntu/download-server)
Make sure your new machine is plugged into a working network with access to the outside world. Without this, the updating and downloading of package will be more difficult.
Boot the CD in your new machine. The install process is quite intuitive (if you know your network settings!). The important choice is near the end of the install where it asks for the additional packages required. I Chose Virtual and SSH (but not print server, or LAMP - these will be virtual servers as needed). As part of the install I created a user called walter.
After the install, I logged into the console, and gave root a password. This is frowned upon. If you don’t want to do this, you need to use ‘sudo’ in front of all the commands below, because they typically need to run as root.
Step 2 Updating Ubuntu
At login there was a message that said:
28 packages can be updated.
17 updates are security updates.
I updated ubuntu using these commands.
Note: The $ sign is to show it's a command, you don't type the $
$ apt-get update
$ apt-get upgrade
$ apt-get upgrade security
Step 3 Virtualisation
3a – pre-requisites
The manual goes on and on about how to get virtualisation up and running,
but I think most of it was already done by choosing the install option for virtualisation.
Before you go any further, check your CPU supports virtualisation. The manual uses this command to do this:
$ egrep '(vmx|svm)' /proc/cpuinfo
If nothing prints out then you don’t have the right kind of processor and you probably will be better off with some other product. (VMware worked well for me on older machines).
It seemed the virbr0 ethernet bridge was installed (but I added br0 as described below). I never figured out what to use virbr0 for.
In seemed libvirtd group existed, but no users belonged to it, so I added myself to the group. SinceI was using root, maybe I didn’t need this step.
$ adduser walter libvirtd
I never figured out if I needed this next step either - you could try without it, and if you get a complaint about virt-install being missing you could run this.
$ apt-get install python-virtinst
Ethernet Bridging
An ethernet bridge allows several IP address to share a single network card. Unless you know otherwise, you’ll need to set up a bridge.
Before you do this, make sure your network can see your new server (try and SSH into it). If the network doesn’t work before you install the bridge – it won’t work afterwards either.
First you need the packages. I found these already installed, but you might not have them:
$ apt-get install bridge-utils
Next you need to edit /etc/network/interfaces. I suggest you backup the old one first.
$ cd /etc/network
$ vi interfaces
From the eth0 section, copy these sections (your IP addresses will be the ones you specified when you installed ubuntu and you’ll want your values rather than mine which are shown here)
address 192.168.23.1
network 192.168.23.0
netmask 255.255.255.0
broadcast 192.168.23.255
gateway 192.168.23.254
dns-nameservers 31.11.10.4 31.11.10.44
dns-search wonderba.com
then delete the eth0 section (from ‘auto eth0’ to the end of the indented bit.)
I then added this section. You will need to use your own IP addresses and dns stuff (the copied ones from before are exactly right.)
auto br0
iface br0 inet static
address 192.168.23.1
network 192.168.23.0
netmask 255.255.255.0
broadcast 192.168.23.255
gateway 192.168.23.254
dns-nameservers 31.11.10.4 31.11.10.44
dns-search wonderba.com
bridge_ports eth0
bridge_fd 9
bridge_hello 2
bridge_maxage 12
bridge_stp off
Note: the bridge_ports clause has the name of the ethernet port you removed (eth0). Now theoretically, you can restart the network to use the new values, but I found that sometimes this didn’t work, so a quick reboot is needed instead:
$ init 6
After the machine boots you should still have a working network – give it a quick test. I did have some problems with a older network card – maybe not all cards support bridging.
3b – Install vmbuilder
I used vmbuilder, which I think is more recent that virt-install.
The manual (page 233) shows how to get vmbuilder:
$ apt-get install python-vm-builder
3c – Create a folder to build your VMs
I create a folder in root’s home directory (/root) called vmbuilder. This is where my virtual machines will be built.
$ cd /root
$ mkdir vmbuilder
3d – Create a partition file for the virtual machines disks
The disks are just data files stored somewhere on the physical server. The partition file describes which disk partitions are wanted and how big each one is.
The file (in the manual it’s called vmbuilder.partition), needs to be in the current directory when you build your VM, so I created it in /root /vmbuilder. The values below are in mb, so 2000 means a 2gb partition.
To give you an idea of sizes required, after I installed a LAMP server on this virtual machine I had used 400mb of the root partition and 250mb of /var. The actual files are only as large as the space used, so you can make your partitions larger if you need to.
I placed the following lines in this file:
root 2000
swap 1000
---
/var 2000
(The --- is required and shows the start of the 2nd disk file)
3e – A word about SSH
The manual describes connecting to your new VM using some sort of virtual console. I never figured this out. Since all my clients are windows, I didn’t have a graphical linux machine to install the virtual console on. So I had to make sure the new VM had SSH on it.
The manual does say that SSH should be installed as part of the initial boot of the VM instead of as part of the VM build. This is a security requirement to ensure the SSH has a unique fingerprint (encryption key). However, for your first VM, it is very much easier to include SSH as a package in the build, and after a few bad starts, this is what I did.
3f – Building the VM
To build the VM you need to be in the directory with the vmbuilder.partition file (I used /root/vmbuilder). You also need to know the network settings for your new VM. These will be the same as for the physical machine, but with a different IP address for the new VM
$ cd /root/vmbuilder
Edit the following command line to include your network settings, your dns server and your username / password. Note, at this stage you don’t name your server – it will be called ubuntu. Run the command and watch the output. I got a few warnings but the command ended cleanly in about 12 minutes. (edit: Option: see patryk77's note just below about the -d switch which you can use to place the vm disk files your choice of directory. If you don't use -d, the disks are created in the current directory)
$ vmbuilder kvm ubuntu --suite jaunty --flavour virtual --arch i386 -o --libvirt qemu:///system \
--ip 192.168.23.10 --mask 255.255.255.0 \
--gw 192.168.23.254 --dns 31.11.10.4 \
--user walter --pass walter \
--part vmbuilder.partition \
--addpkg openssh-server
3g – Looking at the new VM
To see the created VM, use the virsh command. Without a virtual console, virsh is your only friend:
$ virsh -c qemu:///system list --all
(Your VM is called ubuntu, and the disks are in a folder below the current folder i.e. /root/vmbuilder/ubuntu-kvm)
Don’t start your VM yet – I suggest you move the disks and edit the settings first.
3h – Moving the VM disks
(edit: If you used the -d switch in step 3f to place your VM disks then you can skip this step)
I moved the virtual disks to /home/vdisk-<name> where <name> is going to be my new server’s name e.g. gandalf:
$ cd /root /vmbuilder/
$ mv ubuntu-kvm /home/vdisk-<name>
3I – Editing the VM settings
There is an XML file that describes the VM’s settings. I think you can’t edit these directly, but I copied it to /tmp and edited it there. Before starting the edit I ran uuidgen to get a new unique uuid:
$ cp /etc/libvirt/qemu/ubuntu.xml /tmp/<name>.xml
$ uuidgen [copy the output to the clipboard]
$ vi /tmp/<name>.xml
These are the sections I changed: See the notes below for each section.
<name>galdalf</name>
<uuid>d4df2338-f564-4b26-8a4c-5e395339d583</uuid>
<source file='/home/vdisk-<name>/disk0.qcow2'/>
<source file='/home/vdisk-<name>/disk1.qcow2'/>
<interface type='bridge'>
<source bridge='br0'/>
name – this is the name of your new VM e.g. gandalf.
uuid – It’s important to change the uuid if you rename the VM. If you don’t change the uuid and you rename the VM, then your settings may be ignored. (virsh dumpxml <name> will show you what the OS thinks your settings are. If these are revertying to the old values after you redefine the VM, then try changing the uuid).
source file: These are the filenames of the virtual disks. If you moved them as suggested earlier, then you need to put the new pathnames here. We created 2 disks, so there are 2 similar entries (disk0 and disk1).
interface – this says we want to use the bridge we set up earlier. Change the value from ‘network’ to ‘bridge’.
source bridge – this is the name of the bridge we created. NOTE: Your original file will have ‘source NETWORK’ and you must change this to ‘source BRIDGE’ and change network=‘default’ to bridge=‘br0’ .
3j – Redefine your changed VM
Whenever you edit the VM settings, you need to re-import them using virsh:
$ virsh -c qemu:///system define /tmp/<name>.xml
Now when you list the VMs you should see the original (ubuntu) and the new one (gandalf)
$ virsh -c qemu:///system list --all
3h – Start the VM
Now, you’re ready to start the new VM:
$ virsh -c qemu:///system start <name>
This should take a few seconds. If you get a long wait or errors then check the settings of the VM using: virsh dumpxml <name>. Don’t forget if the settings aren’t ‘sticking’, try generating a new uuid and placing it in the settings file as described above.
If the VM starts, you should be able to ping its IP address. (Note: 192.168.23.1 was my physical machine’s IP address and 192.168.23.10 was the new VM’s IP address, specified when I built the VM).
$ ping 192.168.23.10
If the ping fails, use virsh dumpxml <name> and check the VM’s settings for the interface type and source bridge. I found these settings sometimes didn’t ‘stick’. Your settings should look like this:
<interface type='bridge'>
<source bridge='br0'/>
If the ping works, then try and SSH into the new VM
$ ssh 192.168.23.10
You will need to log in as the user you specified when you built the VM (e.g. walter/walter)
3k – The End
And that’s it. I found the following virsh commands very useful:
# List VMs:
$ virsh -c qemu:///system list --all
# Start VM
$ virsh -c qemu:///system start <name>
# Nice Shutdown of VM
$virsh -c qemu:///system shutdown <name e.g. ns1>
# Force shutdown (dirty)
$ virsh -c qemu:///system destroy <name>
# Delete VM
$ virsh -c qemu:///system undefine <name>
# Dump VM settings
$ virsh -c qemu:///system dumpxml <name>
# Dump VM net settings
$ virsh -c qemu:///system net-dumpxml <name>
This example shows how to set up virtualisation and install a ubuntu 9.04 guest
If you already know why you want a virtual environment, then you can skip this section. If you’re new to the subject, virtualisation allows you to run multiple ‘virtual servers’ or virtual machines (VMs) on a single physical box. You would want to do this because:
Instead of installing every bit of software on one server you can install each bit on its own virtual server. Then if that bit doesn’t work or is replaced by something else, you can simply re-configure that one virtual server.
It is really easy to create, copy and delete virtual servers, so you can stand up a new environment in minutes, and throw it away in even less time. This is great for testing and development.
If you’re starting out small, you can run each part of your business on a virtual server and as your business grows you can migrate the virtual servers to physical servers as required.
Other source references
I found some docs quite confusing – one of the problems is, I think, that the processes have changed quite rapidly and the web help is taking a while to catch up. Also, I didn’t have a linux box running x-windows to act as a graphical virtual console (which may have made things very much easlier), and I think that maybe some other 'how tos' are assuming a graphical environment.
Anyway, I’m sure there are better ways (and feel free to add your 2 cents), but this worked for me.
The ‘manual’ referred to is the Ubuntu Server Guide here (https://help.ubuntu.com/9.04/serverguide/C/serverguide.pdf)
Hardware
I started with a small new-ish machine with an AMD quad-core processor that had the AMD-V virtualisation extensions. Modern Intel processors have similar extensions. While virtualisation works on older machines, you may be better off with the free VMware Server on these machines, IMHO.
Step 1. Install Ubuntu
This is a quick overview.
Download the .iso image and burn it to CD. I found the .iso here (http://www.ubuntu.com/getubuntu/download-server)
Make sure your new machine is plugged into a working network with access to the outside world. Without this, the updating and downloading of package will be more difficult.
Boot the CD in your new machine. The install process is quite intuitive (if you know your network settings!). The important choice is near the end of the install where it asks for the additional packages required. I Chose Virtual and SSH (but not print server, or LAMP - these will be virtual servers as needed). As part of the install I created a user called walter.
After the install, I logged into the console, and gave root a password. This is frowned upon. If you don’t want to do this, you need to use ‘sudo’ in front of all the commands below, because they typically need to run as root.
Step 2 Updating Ubuntu
At login there was a message that said:
28 packages can be updated.
17 updates are security updates.
I updated ubuntu using these commands.
Note: The $ sign is to show it's a command, you don't type the $
$ apt-get update
$ apt-get upgrade
$ apt-get upgrade security
Step 3 Virtualisation
3a – pre-requisites
The manual goes on and on about how to get virtualisation up and running,
but I think most of it was already done by choosing the install option for virtualisation.
Before you go any further, check your CPU supports virtualisation. The manual uses this command to do this:
$ egrep '(vmx|svm)' /proc/cpuinfo
If nothing prints out then you don’t have the right kind of processor and you probably will be better off with some other product. (VMware worked well for me on older machines).
It seemed the virbr0 ethernet bridge was installed (but I added br0 as described below). I never figured out what to use virbr0 for.
In seemed libvirtd group existed, but no users belonged to it, so I added myself to the group. SinceI was using root, maybe I didn’t need this step.
$ adduser walter libvirtd
I never figured out if I needed this next step either - you could try without it, and if you get a complaint about virt-install being missing you could run this.
$ apt-get install python-virtinst
Ethernet Bridging
An ethernet bridge allows several IP address to share a single network card. Unless you know otherwise, you’ll need to set up a bridge.
Before you do this, make sure your network can see your new server (try and SSH into it). If the network doesn’t work before you install the bridge – it won’t work afterwards either.
First you need the packages. I found these already installed, but you might not have them:
$ apt-get install bridge-utils
Next you need to edit /etc/network/interfaces. I suggest you backup the old one first.
$ cd /etc/network
$ vi interfaces
From the eth0 section, copy these sections (your IP addresses will be the ones you specified when you installed ubuntu and you’ll want your values rather than mine which are shown here)
address 192.168.23.1
network 192.168.23.0
netmask 255.255.255.0
broadcast 192.168.23.255
gateway 192.168.23.254
dns-nameservers 31.11.10.4 31.11.10.44
dns-search wonderba.com
then delete the eth0 section (from ‘auto eth0’ to the end of the indented bit.)
I then added this section. You will need to use your own IP addresses and dns stuff (the copied ones from before are exactly right.)
auto br0
iface br0 inet static
address 192.168.23.1
network 192.168.23.0
netmask 255.255.255.0
broadcast 192.168.23.255
gateway 192.168.23.254
dns-nameservers 31.11.10.4 31.11.10.44
dns-search wonderba.com
bridge_ports eth0
bridge_fd 9
bridge_hello 2
bridge_maxage 12
bridge_stp off
Note: the bridge_ports clause has the name of the ethernet port you removed (eth0). Now theoretically, you can restart the network to use the new values, but I found that sometimes this didn’t work, so a quick reboot is needed instead:
$ init 6
After the machine boots you should still have a working network – give it a quick test. I did have some problems with a older network card – maybe not all cards support bridging.
3b – Install vmbuilder
I used vmbuilder, which I think is more recent that virt-install.
The manual (page 233) shows how to get vmbuilder:
$ apt-get install python-vm-builder
3c – Create a folder to build your VMs
I create a folder in root’s home directory (/root) called vmbuilder. This is where my virtual machines will be built.
$ cd /root
$ mkdir vmbuilder
3d – Create a partition file for the virtual machines disks
The disks are just data files stored somewhere on the physical server. The partition file describes which disk partitions are wanted and how big each one is.
The file (in the manual it’s called vmbuilder.partition), needs to be in the current directory when you build your VM, so I created it in /root /vmbuilder. The values below are in mb, so 2000 means a 2gb partition.
To give you an idea of sizes required, after I installed a LAMP server on this virtual machine I had used 400mb of the root partition and 250mb of /var. The actual files are only as large as the space used, so you can make your partitions larger if you need to.
I placed the following lines in this file:
root 2000
swap 1000
---
/var 2000
(The --- is required and shows the start of the 2nd disk file)
3e – A word about SSH
The manual describes connecting to your new VM using some sort of virtual console. I never figured this out. Since all my clients are windows, I didn’t have a graphical linux machine to install the virtual console on. So I had to make sure the new VM had SSH on it.
The manual does say that SSH should be installed as part of the initial boot of the VM instead of as part of the VM build. This is a security requirement to ensure the SSH has a unique fingerprint (encryption key). However, for your first VM, it is very much easier to include SSH as a package in the build, and after a few bad starts, this is what I did.
3f – Building the VM
To build the VM you need to be in the directory with the vmbuilder.partition file (I used /root/vmbuilder). You also need to know the network settings for your new VM. These will be the same as for the physical machine, but with a different IP address for the new VM
$ cd /root/vmbuilder
Edit the following command line to include your network settings, your dns server and your username / password. Note, at this stage you don’t name your server – it will be called ubuntu. Run the command and watch the output. I got a few warnings but the command ended cleanly in about 12 minutes. (edit: Option: see patryk77's note just below about the -d switch which you can use to place the vm disk files your choice of directory. If you don't use -d, the disks are created in the current directory)
$ vmbuilder kvm ubuntu --suite jaunty --flavour virtual --arch i386 -o --libvirt qemu:///system \
--ip 192.168.23.10 --mask 255.255.255.0 \
--gw 192.168.23.254 --dns 31.11.10.4 \
--user walter --pass walter \
--part vmbuilder.partition \
--addpkg openssh-server
3g – Looking at the new VM
To see the created VM, use the virsh command. Without a virtual console, virsh is your only friend:
$ virsh -c qemu:///system list --all
(Your VM is called ubuntu, and the disks are in a folder below the current folder i.e. /root/vmbuilder/ubuntu-kvm)
Don’t start your VM yet – I suggest you move the disks and edit the settings first.
3h – Moving the VM disks
(edit: If you used the -d switch in step 3f to place your VM disks then you can skip this step)
I moved the virtual disks to /home/vdisk-<name> where <name> is going to be my new server’s name e.g. gandalf:
$ cd /root /vmbuilder/
$ mv ubuntu-kvm /home/vdisk-<name>
3I – Editing the VM settings
There is an XML file that describes the VM’s settings. I think you can’t edit these directly, but I copied it to /tmp and edited it there. Before starting the edit I ran uuidgen to get a new unique uuid:
$ cp /etc/libvirt/qemu/ubuntu.xml /tmp/<name>.xml
$ uuidgen [copy the output to the clipboard]
$ vi /tmp/<name>.xml
These are the sections I changed: See the notes below for each section.
<name>galdalf</name>
<uuid>d4df2338-f564-4b26-8a4c-5e395339d583</uuid>
<source file='/home/vdisk-<name>/disk0.qcow2'/>
<source file='/home/vdisk-<name>/disk1.qcow2'/>
<interface type='bridge'>
<source bridge='br0'/>
name – this is the name of your new VM e.g. gandalf.
uuid – It’s important to change the uuid if you rename the VM. If you don’t change the uuid and you rename the VM, then your settings may be ignored. (virsh dumpxml <name> will show you what the OS thinks your settings are. If these are revertying to the old values after you redefine the VM, then try changing the uuid).
source file: These are the filenames of the virtual disks. If you moved them as suggested earlier, then you need to put the new pathnames here. We created 2 disks, so there are 2 similar entries (disk0 and disk1).
interface – this says we want to use the bridge we set up earlier. Change the value from ‘network’ to ‘bridge’.
source bridge – this is the name of the bridge we created. NOTE: Your original file will have ‘source NETWORK’ and you must change this to ‘source BRIDGE’ and change network=‘default’ to bridge=‘br0’ .
3j – Redefine your changed VM
Whenever you edit the VM settings, you need to re-import them using virsh:
$ virsh -c qemu:///system define /tmp/<name>.xml
Now when you list the VMs you should see the original (ubuntu) and the new one (gandalf)
$ virsh -c qemu:///system list --all
3h – Start the VM
Now, you’re ready to start the new VM:
$ virsh -c qemu:///system start <name>
This should take a few seconds. If you get a long wait or errors then check the settings of the VM using: virsh dumpxml <name>. Don’t forget if the settings aren’t ‘sticking’, try generating a new uuid and placing it in the settings file as described above.
If the VM starts, you should be able to ping its IP address. (Note: 192.168.23.1 was my physical machine’s IP address and 192.168.23.10 was the new VM’s IP address, specified when I built the VM).
$ ping 192.168.23.10
If the ping fails, use virsh dumpxml <name> and check the VM’s settings for the interface type and source bridge. I found these settings sometimes didn’t ‘stick’. Your settings should look like this:
<interface type='bridge'>
<source bridge='br0'/>
If the ping works, then try and SSH into the new VM
$ ssh 192.168.23.10
You will need to log in as the user you specified when you built the VM (e.g. walter/walter)
3k – The End
And that’s it. I found the following virsh commands very useful:
# List VMs:
$ virsh -c qemu:///system list --all
# Start VM
$ virsh -c qemu:///system start <name>
# Nice Shutdown of VM
$virsh -c qemu:///system shutdown <name e.g. ns1>
# Force shutdown (dirty)
$ virsh -c qemu:///system destroy <name>
# Delete VM
$ virsh -c qemu:///system undefine <name>
# Dump VM settings
$ virsh -c qemu:///system dumpxml <name>
# Dump VM net settings
$ virsh -c qemu:///system net-dumpxml <name>