hooray! you can grab my sources from here:
Code:
git clone git://devel.extof.me/vps
i can't guarantee that will always be up (im doing alot of work with these containers and on these scripts), or that the git repo will be a proper one (when i update it i might break history 
, for a little while at least), but i should be up most the time, and the url wont change. that url is running in an LXC container created using the tools it's hosting.
DISCLAIMER as is, these tools are meant for an archlinux host and WILL NOT work for an ubuntu/debian host. this is something i intend to change. in addition, the scripts expect a BTRFS based root filesystem, others (ext3/4/etc) will need to replace btrfsctl commands with "cp -R" and "rm -rf" (i intend to do this in a slick/hacky way... by making a bash function with the same name as btrfsctl when its not available. when its called it will cp/rm instead) right now i am providing these to help others understand how the lxc-* tools work, and present a setup that works for me, and may for you. all the scripts together are less than 1000 lines of bash, so a light read 
FEATURES
- uses btrfs to create efficient "forks" of templates into writable domains (vps)
- complete, drop in solution (only add two lines to rc.local)
- when you enter a vps from the host, the PS1 will reflect that:
Code:
cr@ph1 ~ $ vps-enter guest-personal-tony
Type <Ctrl+a q> to exit the console
VPS[guest-personal-tony] cr@extof /srv/git $
ONLY if you're entering from the host, if you enter via ssh, the VPS[XXX] part is not there (host is on a real tty, everything else is pty) - by default, a dhcp bridge is created so all guests see each other, and have dns/dhcp/internet access
- there is more i swear...
KNOWN CAVEATS/BUGS
- at least with arch guests, the dhcp never works/times out the first time a vps is booted; this could have something to do with the bridge. fix by manually restarting network a second time doesnt affect interfaces that are bridged directly to the LAN, i.e. no dhcp
- TERM=linux in a container. "export TERM=xterm" to get stuff working nicer in vim/maybe others (home/end keys dont work without)
- vps-enter will only let one person in a vps at a time, even if the container is configured with more than one tty. i will update this soon (compare config file to used sockets)
- i run misc/check-dev-pts.sh as a cron also. this makes sure that /dev/ptmx never breaks in the event of a rouge udev process (this only happened once, but had i not caught it i would have been locked out of my server. only affects those using the newinstance devpts mount flag on the actual host) i dont know how udev did this from inside a container... im not sure what happened. i dont use udev in containers, they already have the devices they need
- i use special entries in the /etc/inittab of containers to handle rebooting/powerdown of containers:
Code:
p6::ctrlaltdel:/sbin/init 6
p0::powerfail:/sbin/init 0
this lets me send, from the host, a SIGINT to reboot, and a SIGPWR to "power down" (with the help of vps-monitor, more on that below). this trick probably doesnt work with upstart - i havent made an elegant way to edit the configs of an already running domain.
- and probably plenty of other things...
DEVELOPMENT FUTURE
- support other hosts than archlinux
- add bash completion scripts
- support creating containers that are not archlinux based
- harden/extend all scripts by using getopts
- colorize everything
- other things i can't think of ATM
HOW TO USE
place the vps folder at /vps
i create a file, /etc/profile.d/vps.sh, chmod +x it, (i think this will work for ubuntu) and put this in it:
Code:
# export VPS_ENV and add vps-* scripts to PATH
export EDITOR=vim
export VPS_ENV=/vps/usr/lib/common/env
export PATH="${PATH}:/vps/usr/bin"
# gen the motd
#/vps/usr/cron/motd
you dont need the any of it really, except the PATH part if you want to run the commands without their full path. i included the motd script (pretty) i use on my host server if anyone is interested.
i put this in my /etc/rc.local:
Code:
#!/bin/bash
#
# /etc/rc.local: Local multi-user startup script.
#
export VPS_ENV=/vps/usr/lib/common/env
/vps/usr/misc/vps-init.sh
vps-init.sh is the bootstrap file for the whole process. start there to understand how everything works, its really not too bad. archlinux has a rc.local.shutdown file where i place the vps-shutdown.sh script... not sure the equiv for ubuntu.
lastly, i create a way to exec this stuff as a regular user. create a group called vps (this is hard coded for the time being), add the users to it that you want to make/control/stop/see containers, and add this to your sudoers file:
Code:
%vps ALL=(root) NOPASSWD: /vps/usr/bin/vps-*
when you run the file as a normal user, the script does this:
Code:
# exec as VPS_USER if need be
if [ -n "${VPS_SUDO}" ] && [ -n "${VPS_USER#$(whoami)}" ]; then
exec ${VPS_SUDO} ${0} $@
fi
this may not be the best way, and might have an infinite loop possibility. i tried to drop privileges completely by making a vps user instead of root... but there are several things that need root in the scripts and it was too difficult for the time being. if you dont do this step simply run as root, it will Just Work. any suggestions welcome.
BREAKDOWN OF FILES
all the vps-* commands will dispay a usage when invoked with no arguments. i eventually want to use getopts. here is the file hierarchy and what everything does:
Code:
vps
├── def
│ ├── exec
│ │ ├── proc
│ │ └── sys
│ └── mnt
├── dom
├── log
├── tpl
├── usr
│ ├── bin
│ │ ├── vps-create
│ │ ├── vps-enter
│ │ ├── vps-ls
│ │ ├── vps-mkdom
│ │ ├── vps-mktpl
│ │ ├── vps-reboot
│ │ ├── vps-rmdom
│ │ ├── vps-rmtpl
│ │ ├── vps-start
│ │ ├── vps-stop
│ │ └── vps-tree
│ ├── conf
│ │ └── dnsmasq-dhcpbr0.conf
│ ├── cron
│ │ └── motd
│ ├── doc
│ │ ├── BUGS
│ │ ├── INSTALL
│ │ └── TODO
│ ├── lib
│ │ ├── common
│ │ │ ├── color
│ │ │ ├── env
│ │ │ └── function
│ │ ├── exec
│ │ │ └── vps-monitor
│ │ └── static
│ │ ├── autologin
│ │ ├── bash.bashrc.local
│ │ ├── rc.conf
│ │ ├── rc.shutdown
│ │ ├── rc.single
│ │ └── rc.sysinit
│ └── misc
│ ├── check-dev-pts.sh
│ ├── start-brctl-dhcpbr0.sh
│ ├── start-dnsmasq-dhcpbr0.sh
│ ├── start-iptables-dhcpbr0.sh
│ ├── start-mount-cgroup.sh
│ ├── stop-brctl-dhcpbr0.sh
│ ├── stop-dnsmasq-dhcpbr0.sh
│ ├── stop-iptables-dhcpbr0.sh
│ ├── stop-mount-cgroup.sh
│ ├── vps-init.sh
│ └── vps-shutdown.sh
└── var
└── run
/vps/def/*
definition (conf) files for each process (proc) container, or system (sys) container. also mount (mnt) definition files
/vps/dom
domains. where all your system containers will be
/vps/log
logs. when domains are started, output from the init process is logged here. i will probably extend this to log events from the vps-* scripts too
/vps/tpl
templates. system templates that will be forked (BTRFS) or copied (others) to usable domains
/vps/usr/bin
- vps-create: define a domain that has been made with vps-mkdom
- vps-enter: enter a running domain that has a free tty slot
- vps-ls: show the status of defined domains and/or templates
- vps-mkdom: create a dom from an existing template
- vps-mktpl: create a template from a list of predefined packs
- vps-reboot: send a SIGINT to the init process of a running domain
- vps-rmdom: delete a dom that isnt running
- vps-rmtpl: remove a template
- vps-start: start a dom that has been defined with vps-create, or has been stopped by vps-stop
- vps-stop: stop a running domain
- vps-tree: use the tree command to get a view of the folders (this will fail horribly without BTRFS... it uses the -x flag to avoid traversing into dom/tpl directories... useless without that)
/vps/usr/conf
config files. right now only for dnsmasq
/vps/usr/cron
not really crons, used to be. right now just the motd file i use
/vps/usr/doc
self explanatory?
/vps/usr/lib
- common: files included by other scripts (function isnt used)
- exec: executables from other scripts. vps-monitor is the only one right now, its job is to monitor the utmp file in a container and determine if the container should be killed or rebooted based off the runlevel. should still work for upstart. see here for more details: http://www.mail-archive.com/lxc-user.../msg00040.html
- static: files copied verbatim to containers
/vps/usr/misc
i dont like this folder or its name. right now it has all the bootstrap files
/vps/var/cgroup
you dont see this in the tree view, but it will be created, and the cgroup filesystem mounted here (i dont make it because git cant track empty folders without at least a .gitignore file... but then when it's mounted git complains that the .gitignore file is no longer there AND wants to place the entire cgroup under revision control... annoying)
/vps/var/run
has pidfiles (dnsmasq) and whatnot
some of this structure is definately going to change as i break things off, esp. once i start adding support for other templates than archlinux.
as always, comments welcome.
Adv Reply
Originally Posted by narcisgarcia
Bookmarks