Page 1 of 2 12 LastLast
Results 1 to 10 of 19

Thread: Qemu-KVM virtual machine storage.

  1. #1
    Join Date
    Jul 2005
    Location
    I think I'm here! Maybe?
    Beans
    Hidden!
    Distro
    Xubuntu 20.04 Focal Fossa

    Qemu-KVM virtual machine storage.

    I have just installed qemu-kvm as a trial so that I can compare it with VBox which I use quite a lot to test other distros and new *ubuntu versions.

    All was OK until I created a VM of Linux-Mint-20 using virt-manager, then noticed that the VMs it creates are placed in /var/lib/libvirt meaning that my root partition was immediately filled to 98%, so that's a non-starter for me.

    I managed to remove the VM to release space in the root partition, though I had to do it manually after uninstalling the whole shooting match of qemu-kvm and rebooting.

    Do the VMs have to be in the root partition or can they be placed in a folder in my home, which in my case is a separate partition with masses of space?
    Last edited by ajgreeny; September 7th, 2020 at 09:56 PM. Reason: typos

  2. #2
    Join Date
    Jul 2008
    Location
    The Left Coast of the USA
    Beans
    Hidden!
    Distro
    Kubuntu

    Re: Qemu-KVM virtual machine storage.

    I put mine either on different devices or even on servers. You can choose where to put them.

    Give me an hour or two to finish what I am doing out in the blazing sun (not fun, just came in for a cold sparking water and a break) and I'll see if I can find some nice instructions.
    Please read The Forum Rules and The Forum Posting Guidelines

    A thing discovered and kept to oneself must be discovered time and again by others. A thing discovered and shared with others need be discovered only the once.
    This universe is crazy. I'm going back to my own.

  3. #3
    Join Date
    Jun 2007
    Location
    Arizona U.S.A.
    Beans
    4,778

    Re: Qemu-KVM virtual machine storage.

    You can put the VM virtual disks where you want. In step 4 of 5, choose "select or create custom storage" then the Manage button. In the next window "Choose Storage Volume" you can add a location for virtual disks with the buttons in lower left. See 2nd screen shot. The contents of that location (if any) appear in the right window. Create a new storage disk for the new VM with the + button above that window. The name of the VM disk file should end with .qcow2. Then choose it and continue.
    Attached Images Attached Images
    Last edited by Dennis N; September 7th, 2020 at 11:18 PM.

  4. #4
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    21,322
    Distro
    Ubuntu Mate 16.04 Xenial Xerus

    Re: Qemu-KVM virtual machine storage.

    libvirt allows about 20 different "backend storage" options. These can range from using any file system you like for storage, to using NFS, to LVM LVs, or ZFS zpools, to using many distributed file systems like CEPH or Sheepdog spread across many other systems. You have the choice.

    So, each has slightly different instructions, but always remember, this is a server tool, not an end user tool. Storage for VMs, if you just want to use file-based storage on a file system can be located anywhere, but really a HOME directory is a poor choice. If you have a separate file system for /home/, then you can create a new directory, perhaps /home/VMs/ to hold the VM storage stuff. Then, inside virt-manager, just add that area as a file-system/directory storage pool. Or you might want to mount added storage directly to /var/lib/libvirt . I've done that a few times, just to make trivial play VMs have a place to use.
    Code:
    $ df -Th
    Filesystem                        Type  Size  Used Avail Use% Mounted on
    /dev/mapper/hadar--vg-root        ext4   22G   12G  9.4G  56% /
    /dev/sdb1                         ext2  720M  207M  477M  31% /boot
    /dev/mapper/hadar--vg-libvirt--lv ext4  197G  104G   84G  56% /var/lib/libvirt
    /dev/mapper/vg--hadar-lv--backups ext4  459G  424G   36G  93% /Backups
    /dev/mapper/hadar--vg-lxd--lv     ext4   59G   14G   42G  25% /var/lib/lxd
    lxd/containers/pihole             zfs    13G  657M   12G   6% /var/lib/lxd/storage-pools/lxd/containers/pihole
    That is all the real mounted storage on the system.

    For the best performance on a single machine, using a block storage provider like LVM or ZFS will integrate into the libvirt tools. These are both supported, LVs get created and destroyed as controlled by the virt-manager, once you setup an LVM VG to be used in the GUI.
    Code:
    $ sudo lvs
      LV                VG       Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
      libvirt-lv        hadar-vg -wi-ao---- 200.00g                                                    
      lv-blog44-1604    hadar-vg -wi-ao----  16.20g                    /dev/mapper/hadar--vg-libvirt--lv ext4  197G  104G   84G  56% /var/lib/libvirt                                
      lv-lubuntu11-1604 hadar-vg -wi-a-----  31.00g                                                    
      lv-opnsense       hadar-vg -wi-a-----   5.00g                                                    
      lv-regulus        hadar-vg -wi-ao----  30.00g                                                    
      lv-regulus-2      hadar-vg -wi-ao----  10.00g                                                    
      lv-spam3          hadar-vg -wi-ao----  10.00g                                                    
      lv-tp-lxd         hadar-vg twi-a-tz--  32.23g             0.00   10.06                           
      lv-u20.04-srv     hadar-vg -wi-a-----  10.00g                                                    
      lv-vpn09-1604     hadar-vg -wi-ao----   7.50g                                                    
      lv-xen41-1604     hadar-vg -wi-ao----  12.50g                                                    
      lv-zcs45-1604     hadar-vg -wi-ao----  25.00g                                                    
      lxd-lv            hadar-vg -wi-ao----  60.00g                                                    
      root              hadar-vg -wi-ao----  22.33g                                                    
      swap_1            hadar-vg -wi-ao----   4.25g
    Most of those are LVs provided to VMs, controlled by libvirt using KVM as the hypervisor. Plus, we can create a snapshot of a running VM, mount it and take 100% quiesced backups from the host machine. If you use file-based storage for VMs, that isn't possible, though qcow2 does support a "snapshot-like" capability, it isn't for backups. Warning - don't use qcow2 on BTRFS file systems that have CoW enabled.

    The LVs used directly for VM storage above don't get mounted on the hostOS, only inside each VM. You can see I'm not 100% consistent with my naming. Oh well.

    You can also see that regulus ran out of storage, so I added some more. That was accomplished without rebooting either the host or the VM. I posted the commands in these forums 1-3 months ago. LVM is pretty amazing.

    You can always use symlinks to redirect storage to another file system as well. I've done that a few times. On a different machine,
    Code:
    root@romulus:/raid/media/VMs/VMs# ls
    blog.qcow2                        ubudesk.img
    vpn-1604.qcow2
    nextcloud.qcow2                   vpn-1604.qcow2-2018.01.27.bak
    ntp.qcow2                         xen41.img
    redmine.img                       xen51-README-Backup_Method.txt
    spam2.img                         zcs43.img
    spam2.qcow2                       zcs43-u10.04.img
    Inside each xml file which holds the VM definition for that specific machine, the full path or connection method are stored. If you need to move stuff from /var/lib/libvirt to somewhere else, you can use virsh edit {name of VM}, and that will use your default EDITOR to modify the settings. No need to run as root, just need for the userid to be in the libvirtd gid. Further, since remote management is built into libvirt, this works from other systems that have ssh access. Just setup an qemu+ssh://thefu@romulus/system connection. I have 4 VM hosts at home, but can create, install, manage, shutdown each from any other workstation that has ssh connectivity to the host.

    One last thing. The default networking created by libvirt is only NAT. If you want bridged networking, then you'll need to set that up. For servers, that's pretty much mandatory.

    Clear as mud?

  5. #5
    Join Date
    Jul 2008
    Location
    The Left Coast of the USA
    Beans
    Hidden!
    Distro
    Kubuntu

    Re: Qemu-KVM virtual machine storage.

    Thanks for helping ajgreeny out, folks! I had to leave him hanging to run out and melt in the sun this afternoon.
    Please read The Forum Rules and The Forum Posting Guidelines

    A thing discovered and kept to oneself must be discovered time and again by others. A thing discovered and shared with others need be discovered only the once.
    This universe is crazy. I'm going back to my own.

  6. #6
    Join Date
    Jul 2005
    Location
    I think I'm here! Maybe?
    Beans
    Hidden!
    Distro
    Xubuntu 20.04 Focal Fossa

    Re: Qemu-KVM virtual machine storage.

    QIII, I hope you are now out of the blazing sun and happy again but thanks to all of you for the guiding comments..

    I found a Web at https://www.linux.com/audience/devop...es-kvm-part-1/ which seems to say what you've all said, as far as I can make out anyway, so I'll be starting again this evening when time alĺows, but even though using LVM may be a worthwhile way to go eventually, I will avoid adding that extra complication at the moment; LVM is more of a mystery to me at present than KVM.

    Wish me luck; I'll keep in touch and tell all the details later.

  7. #7
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    21,322
    Distro
    Ubuntu Mate 16.04 Xenial Xerus

    Re: Qemu-KVM virtual machine storage.

    Quote Originally Posted by ajgreeny View Post
    I found a Web at https://www.linux.com/audience/devop...es-kvm-part-1/ which seems to say what you've all said, as far as I can make out anyway, so I'll be starting again this evening when time alĺows, but even though using LVM may be a worthwhile way to go eventually, I will avoid adding that extra complication at the moment; LVM is more of a mystery to me at present than KVM.

    Wish me luck; I'll keep in touch and tell all the details later.
    I used raw images for years because the new-fangled vdi/qcow files where significantly slower. Fixing storage performance later is one of the harder things to correct, though not really THAT hard if you have downtime allowed. If SSD storage is used, the worse performance just doesn't matter.

    The whole type-1 vs type-2 is marketing. Because VMware puts that you cannot post unapproved performance results into their license agreements, nobody does. Real world testing on the same hardware between Xen, KVM and ESXi ... let's just say, KVM is fastest and most convenient. VMware ESXi is very picky about hardware. 3/4 devices here weren't supported by it due to the SATA controllers used. Also, there really is no host-based way to perform backups without spending $1000 on a bad backup tool. If you treat each VM like a physical machine and do backups that way, then it doesn't matter.

    Big performance tips:
    * always choose virtio for storage controllers
    * always choose virtio for networking (NICs)

    ** For Windows VMs, you may want to choose an Intel PRO/1000 NIC and SATA or SCSI controller to avoid driver hassles, so you don't have to load redhat virtio drivers during emergency boots. The performance for the disks isn't THAT different, though for networking 1 Gbps limitation between VMs and the host on the say system is 20-40x slower than what virtio provides.

  8. #8
    Join Date
    Jul 2005
    Location
    I think I'm here! Maybe?
    Beans
    Hidden!
    Distro
    Xubuntu 20.04 Focal Fossa

    Re: Qemu-KVM virtual machine storage.

    Thanks to all of you.

    I managed to get a VM running having now changed the storage folder to my home and have been playing around with it this evening with few problems other than sharing host folders (if it is possible) or using USB disks, eg thumb drives. It took a while to figure out how to exit from fullscreen but then I noticed the small icons that appear when the cursor is moved to the middle of the top screen edge; dead simple when you know. Speed and smoothness of the graphics is certainly very good, and faster than VBox, I think. Touchpad scrolling works perfectly which is certainly not the case with VBox where I have to scroll manually by pulling down the scrollbar with the cursor.

    I will keep searching for answers to the USB, shared folders, and any other problems I come across, but thanks again for the help.

  9. #9
    Join Date
    Jun 2007
    Location
    Arizona U.S.A.
    Beans
    4,778

    Re: Qemu-KVM virtual machine storage.

    ... few problems other than sharing host folders (if it is possible) or using USB disks, eg thumb drives.
    I takes a while to figure things out. I work from a VM much of the time (as I am right now), not just for testing. These two items I can give you an answer.

    USB: With Ubuntu host, to make the USB accessible from the VM you have to "redirect" the USB device: exit VM full screen mode, then from the Virt-Manager Menu > Virtual Machine > Redirect USB Device.

    Access Host: The only method I know to gain access to the Host is to connect over the local network by SSH, just as you would connect to any another computer on the network.
    Last edited by Dennis N; September 8th, 2020 at 11:24 PM.

  10. #10
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    21,322
    Distro
    Ubuntu Mate 16.04 Xenial Xerus

    Re: Qemu-KVM virtual machine storage.

    Use NFS for shared folders. Treat the host and the guest like physical machines.

    For USB, there are some tricks. Usually, I just scp/sftp the files from the host into the VM, if there isn't an NFS connection. With ssh-keys setup, this stuff is trivial.

    As for graphics, if you use SPICE and QXL, don't forget that all this works over the network too. You don't have to be in the same room.
    Code:
    virt-viewer --connect qemu+ssh://hadar/system regulus
    where
    * regulus is a VM running somewhere with QXL video drivers
    * hadar is a DNS name on the network known to the current host for the VM host. An IP should work. The current userid, if it matches between the VM host and current host will be used or add "thefu@hadar" where hadar is now.

    On the same machine, use:
    Code:
    $ virt-viewer regulus
    Over a trusted LAN, we can force not to use SSH tunneling by adding a --direct option, but leave everything else there.
    Thanks to SPICE and QXL, we can watch hidef video through this connection, though it would be better to access the files directly (via NFS or DLNA) and let the local hardware play the video.

    Oh ... and virt-viewer has a menu where connecting a USB device is pretty simple. A checkbox for the device and "ok" button.

    This will work over just the ssh port, over the internet too, but audio and video probably won't work. For low bandwidth connections, consider loading up x2go and using that. x2go also works over ssh using ssh credentials and honors all ~/.ssh/config settings for different hostnames, IPs, userids, and ports. Those are the most important parts, but everything is documented in the ssh_config manpage, so if you want a keepalive or a specific pki file pair to be used, that's all possible.

    A quick little connection script that just needs the KVM/Spice hostname changed:
    Code:
    #!/bin/bash
    if [[ $HOSTNAME == "hadar" ]] ; then
        # For local connection on hadar
        /usr/bin/virt-viewer -a -d  $1 &
    else
        # For remote connections on the LAN
        /usr/bin/virt-viewer --connect qemu+ssh://hadar/system $1 &
    fi
    So... if this script is in your PATH, perhaps ~/bin/vview, run it with vview regulus and it will do the correct connection for local or remote and we don't have to remember these details anymore regardless of our client. To connect to my winult box, vview winult is the command. The $1 argument is just the VM name as far as libvirt knows. It is case-sensitive. If no argument is provided - I don't know what happens. I use aliases to call the script from keyboard accel shortcuts.

    Any guest with QXL video drivers works with spice and will have a fast GUI experience. My local and remote setups both use openGL for graphics drawing. Fast. Like running on a physical machine, but without knowing it. My hardware is pretty medium-cheap for 2 yrs old. Lets the local GPU get used. There is a Windows virt-viewer too, but I don't have ssh-keys setup on that, so it isn't nearly as clean for connections.

    I need to load up an old WinXP VM and see how some old games play. Can't wait to try a flight sim out now that I've seen how fast it is. Really can't imagine any of my 10-25 yr old games performing badly with the snappiness I just saw.
    Last edited by TheFu; September 9th, 2020 at 02:51 AM. Reason: fixed which systems need to be in DNS or use IPs.

Page 1 of 2 12 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •