XEN allows you to run multiple virtual servers on one physical machine.

What are virtual servers good for? You can run different operating systems on your machine, e.g. Debian Woody and Debian Sarge. This allows you to migrate very slowly from one OS to another.

And of course you can install additional test domains (this is how virtual servers are called in XEN) where you can try out your new software without disturbing production. This saves you a test machine!

Besides, XEN allows you to limit the CPU-usage for domains. If one of your domains is overloaded, your other domains will still receive enough CPU to run normally. With UML, your whole server would be slow.

Finally, you can set up domains for different applications (for example one for mail and SpamAssassin and one for Apache). If one application goes mad and eats all your memory, only that domain will be affected!

The main difference to UML ("User Mode Linux") is that XEN does the virtualisation outside of Linux in a small XEN kernel, while in UML each virtual server is a process of a master Linux.

According to comparison results, virtual servers with XEN are only a little bit slower than plain Linux, while VMWare and UML are a lot slower.

Below you can find a manual showing how I set up my server with XEN. The server can be at some Linux web hoster, you don't need physical access to it (an emergency reboot should be available though). It is assumed that the server has only one public ip address, not one for each domain. Small shell scripts are provided that help you install, backup and remove domains. The install script bootstraps Debian Linux to a new LVM-partition and sets up the networking so that the domain is reachable through Masquerade. NAT is set up so domains can reach the internet.

Backup is possible with LVM-snapshots, although this is not stable yet in Kernel 2.6 (at the time of writing).

You need to have at least one free partition which you can later use for LVM. If you don't have any and you want to shrink your root partition to get space for another partition, have a look here.

Domain 0 is the privileged domain that can access the hardware directly and can configure the other domains.

First of all, you need to have a recent Linux 2.6.X kernel that is running on your machine. Make sure you configured this kernel so that the performance is ok. Common traps are disabled chipsets, so DMA is not possible and read/write to the disk is slow.

You need a kernel with XEN-patch, LVM2 and reiserfs support.

We compile it ourselves:

1. Download the kernel:

$ cd /usr/src

$ wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.11.12.tar.bz2

2. Install kernel-package, bzip2, and grep-dctrl:

$ apt-get install bzip2 kernel-package grep-dctrl

3. Unpack the kernel with

$ tar xjf tar linux-2.6.11.12.tar.bz2

4. Download and install the xen kernel patch from http://packages.debian.org/unstable/misc/kernel-patch-xen

$ dpkg -i kernel-patch-xen_2.0.6-1_i386.deb

5. Important: Copy the .config of your current kernel into /usr/src/kernel-source-2.6.11.12/ .If your current kernel comes from a debian packet, you can find your .config at /boot/config.YOURKERNELVERSION.:

$ cp /boot/config-2.6.xyz /usr/src/kernel-source-2.6.11.12/.config

Insert the following lines into your .config:

#

CONFIG_XEN=y

CONFIG_ARCH_XEN=y

CONFIG_NO_IDLE_HZ=y

#

# XEN

#

CONFIG_XEN_PRIVILEGED_GUEST=y

CONFIG_XEN_PHYSDEV_ACCESS=y

CONFIG_XEN_BLKDEV_BACKEND=y

CONFIG_XEN_NETDEV_BACKEND=y

# CONFIG_XEN_BLKDEV_FRONTEND is not set

# CONFIG_XEN_NETDEV_FRONTEND is not set

CONFIG_XEN_WRITABLE_PAGETABLES=y

CONFIG_XEN_SCRUB_PAGES=y

CONFIG_X86=y

# CONFIG_X86_64 is not set

CONFIG_HAVE_ARCH_DEV_ALLOC_SKB=y

I don't know why this is necessairy, but without it, a lot of stuff is automatically removed when starting menuconfig.

6. Configure and compile your kernel with

$ cd /usr/src/kernel-source-2.6.11.12/

$ PATCH_THE_KERNEL=YES make-kpkg --config=menuconfig --arch=xen --revision=1 --append-to-version=myxen0 kernel_image

(--config=xconfig is ok, too, of course, as long as you have connection to your X server)

Make sure that you select:

• Privileged Guest (XEN)
• Kernel Debugging and Magic SysRq key (->X86 Processor Configuration->Kernel Hacking)
• Check that support for your chipset is compiled in (Device Drivers->ATA/ATAPI/MFM/RLL support)
• Multiple devices driver support and Device Mapper Support (Device Drivers->Multi-Device-Support) (this is for LVM)
• If you want to use LVM-snapshotting for backup, select Snapshot target, too (currently EXPERIMENTAL).
• Network packet filtering and 802.1d Ethernet Bridging (Device Driver->Networking support->Networking options)
• Connection tracking, IP tables support, Full NAT and MASQUERADE target support (Device Driver->Networking support->Networking options->Network packet filtering->Netfilter Configuration)
• Network device support (Device Driver->Networking support). Check that your ethernet card is selected!
• Reiserfs support (File systems)
• /proc file system support (File systems->Pseudo filesystems)

AGP might cause problems. Therefore disable /dev/agpgart (Device Drivers->Character devices). (See lists.xensource.com/archives/html/xen-devel/2005-03/msg00375.html and

lists.xensource.com/archives/html/xen-devel/2005-03/msg00593.html for details.)

7. After compiling, you will get the new kernel package kernel-xen0-2.6.11.12myxen0_1.deb

8. Backup your .config file in case you want to compile another domain 0 kernel:

$ cp .config ../myxen0.config

If you want to compile a new kernel run first:

$ PATCH_THE_KERNEL=YES make-kpkg --arch=xen clean

If you should encounter strange error messages when you compile the kernel for the second or the third time, try deleting /usr/src/kernel-source-2.6.11.12 and untar the kernel again. Of course you should back up your .config before doing that.

Build another Kernel in the same directory (/usr/src/kernel-source-2.6.11.12/):

1. Clean up, so that make-kpkg can run again

$ PATCH_THE_KERNEL=YES make-kpkg --arch=xen clean

2. Configure and compile your kernel with

$ PATCH_THE_KERNEL=YES make-kpkg --config=menuconfig --arch=xen --revision=1 --append-to-version=myxenu kernel_image

This time you need to change the following options:

• disable Privileged Guest and Physical device access (XEN)
• enable Network-device frontend driver (XEN)
• enable Block-device frontend driver (XEN)
• disable Loadable module support (Loadable module support)
• enable Kernel Debugging and Magic SysRq key (X86 Processor Configuration->Kernel Hacking)
• disable /dev file system support (File systems->Pseudo filesystems)
• You don't need Multiple devices driver support (Device Drivers->Multi-Device-Support)
• You can also remove Network packet filtering and 802.1d Ethernet Bridging (Device Driver->Networking support->Networking options)

If you changed module support (from enabled to disabled) make-kpkg will probably fail (see man page of make-kpkg for details). You can either wait or cancel the compile with Ctrl-C and build the kernel again:

$ PATCH_THE_KERNEL=YES make-kpkg --arch=xen clean

$ PATCH_THE_KERNEL=YES make-kpkg --config=menuconfig --arch=xen --revision=1 --append-to-version=myxenu kernel_image

3. After compiling, you will get the new kernel package kernel-xen0-2.6.11.12myxenu_1.deb

4. Backup your .config file in case you want to compile another user domain kernel:

$ cp .config ../myxenu.config

Download Adam Heath's debian packages:xen-docs,libxen2.0, xen and libxen-python.

Install these, a couple more packages and your domain 0 kernel:

$ apt-get install iproute libatm1 python2.3-twisted python2.3-twisted-bin libcurl3 bridge-utils libsysfs1

$ dpkg -i xen_2.0.6-1_i386.deb libxen2.0_2.0.6-3_i386.deb libxen-python_2.0.6-1_i386.deb xen-docs_2.0.6-3_all.deb

$ dpkg -i dpkg -i /usr/src/kernel-xen0-2.6.10myxen0_1_i386.deb

Disable Thread Local Storage (TLS):

$ mv /lib/tls /lib/tls.disabled

TLS will slow down Xen a lot. Whenever you update glibc you have to redo this. If you forget, Xen will warn you at boot time.

This howto assumes that you only own one IP address for your server. (This is the common case if you rent a server at some web hoster.) Therefore we have to change the Xen configuration so that we have a an internal bridged network and routed network that connects the bridged network with the internet.

Edit /etc/network/interfaces and add the following lines in order to set up a bridged network for the user domains:

# Internal Bridged Network.

# 192.168.3.254 is the ip address for bridge

auto xen-intbr

iface xen-intbr inet static

pre-up brctl addbr xen-intbr

post-down brctl delbr xen-intbr

address 192.168.3.254

netmask 255.255.255.0

network 192.168.3.0

broadcast 192.168.3.255

bridge_fd 0

bridge_hello 0

bridge_stp off

Edit /etc/xen/xend-config.sxp and change the networking script to network-route and the vif-bridge to xen-intbr. These two sections should like this (with xen 2.0.6-1):

## Use the following if VIF traffic is routed.

# The script used to start/stop networking for xend.

(network-script network-route)

# The default script used to control virtual interfaces.

#(vif-script vif-route)

## Use the following if VIF traffic is bridged.

# The script used to start/stop networking for xend.

#(network-script network)

# The default bridge that virtual interfaces should be connected to.

(vif-bridge xen-intbr)

# The default script used to control virtual interfaces.

(vif-script vif-bridge)

In xend-config.sxp you should also set

(xend-address 'localhost')

this makes sure that nobody from the internet changes your Xen domains.

You need Grub to run xen. If you haven't installed it yet do:

$ apt-get install grub

If /boot/ is not a separate partiton run:

$ grub-install /dev/hda

else (if /boot/ is a separate partition) run:

$ grub-install --root-directory=/boot /dev/hda

Now install your current kernels:

$ update-grub

Check that your current kernel is int /boot/grub/menu.lst

Reboot to check that your machine is still booting:

$ shutdown -r now; exit

Add xen to Grub:

Edit /boot/grub/menu.lst and append an entry for Xen:

title Xen 2.0 / XenLinux 2.6.11.12myxen0

kernel /boot/xen.gz dom0_mem=131072

root (hd0,Y)

module /boot/xen-linux-2.6.10myxen0 root=/dev/hdaX ro console=tty0

where hdaX is your root partition and Y = X - 1.

(If boot is a separate partition, all paths except the one after root= must be relative to /boot/.)

The param dom0_mem=131072 will give 128M RAM to domain 0. You can change this, of course.

Change the default to Xen. For example, change the default to 2 if Xen is the third entry in /boot/grub/menu.lst.

You can add a line

fallback=0

This will boot the first entry if booting Xen fails, but only if it fails. If it boots and you just didn't configure your ethernet card, then it doesn't help.

Ok, finished with everything? Then take a deep breath and reboot:

$ shutdown -r now; exit

Once you are back on your machine, check with

$ dmesg

that you booted the Xen kernel.

We need to set up an LVM volume group which will be used for the xen domains. First we need to install lvm2 reiserfsprogs and debootstrap:

$ apt-get install lvm2 reiserfsprogs debootstrap

Add a volume group on your box using pvcreate and vgcreate:

$ pvcreate /dev/hdaX

$ vgcreate vgxen /dev/hdaX

/dev/hdaX is your free partiton (other disks beside /dev/hda are also OK)

Install the kernel for the user domains:

$ dpkg -i /usr/src/kernel-xen0-2.6.10myxenu_1_i386.deb

I wrote a few scripts that make it easier to set up a Xen user domain. They are in this package. Download and install it:

$ dpkg -i xen-server-tools-0.8-2.deb

Check that the path to your kernel is set correctly in /etc/xen-server-tools/xen-server-tools.conf

If you want to use a different volume group than vgxen, change VOLUME there, too.

Run

$ create_xen_lvm_domain test1 1 64 3G 256M

You will have to answer a few questions (just press the default) and type in the root password. This will create a new Xen domain child with name test1, IP address 192.168.3.1, 64 MB RAM, 4 GB disk space and 256 MB swap.

Note that after the Memory (64), there is no M, while after the disk and swap space a G or M is neccessary.

On next startup the domain will be automatically started. To start it right away, you have to type

$ xm create -f /etc/xen/auto/test1.conf

$ /etc/xen-server-tools/auto-iptables/test1 start

The second command will set up NAT for the domain. The port 11022 will be forwarded to port 22 of the created domain. This allows you to access the machine directly from the internet with ssh -p 11022 or scp -P 11022.

You can add more forwarding rules in the script /etc/xen-server-tools/auto-iptables/test1. For example

iptables -t nat -A PREROUTING -p tcp -d $MY_IP --dport 80 -j DNAT --to-destination $DOMAINIP:80

will forward port 80 (http) to test1.

The script just bootstrapped the base system. You can log on your machine with ssh root@192.168.3.1 (or from outside ssh -p 11022 your-server.xyz) and set up the rest with dselect or apt-get.

One word of caution about TLS:

Thread Local Storage is implemented in a way that causes problems with Xen. Therefore, the script create_xen_lvm_domain will deactivate TLS by renaming /lib/tls to /lib/tls.disabled. But if you install a new version of glibc on your guest instance, it will reappear. Xen will warn you if it finds /lib/tls at boot time. Then you should disable it again (shut down your domain before):

$ mount /dev/vgxen/test1 /mnt/

$ mv /mnt/lib/tls /mnt/lib/tls.disabled2

$ umount /mnt/

Execute:

$ backup_xen_lvm_domain test1

This will pause your domain, sync it, make a backup and unpause it.

(Syncing might cause an internal server error. This will be fixed in one of the next xen releases. See this thread for details.)

Afterwards you will have a new logical volume /dev/vgxen/test1-backup.

You can mount the newly created logical volume test1-backup and do a backup now on this snapshot. When you are done with your backup, just delete the logical volume with lvremove.

The xen domain is halted only for taking the snapshot. This could be done in a second with LVM-snapshotting. Unfortunately LVM2-snapshotting is not stable yet in 2.6 Kernels. It crashed on my machine once and left a process which I could not kill. The machine didn't reboot cleanly either. So I disabled lvm snapshot from my script by default and use a normal copy of the full volume instead. This takes minutes.

If you like to experiment and try out LVM snapshotting you can /etc/xen-server-tools.conf and set USE_SNAPSHOTTING to Y

I found two messages on linux-kernel about this problem, the second one from February 2005:

www.ussg.iu.edu/hypermail/linux/kernel/0410.1/0861.html

www.ussg.iu.edu/hypermail/linux/kernel/0502.0/0413.html

Just type:

$ delete_xen_lvm_domain test1

This will delete the root and swap partition, the config file and the script for the network (so everything that was created).

The package mentioned above, xen-server-tools, can be found here along with the source:

This document is free; you can redistribute it and/or modify it under

the terms of the GNU General Public License as published by the Free

Software Foundation; either version 2 of the License, or (at your

option) any later version.

This document is distributed in the hope that it will be useful, but

WITHOUT ANY WARRANTY; without even the implied warranty of

MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU

General Public License for more details.

You can get a copy of the GNU GPL at at

www.gnu.org/copyleft/gpl.html

--- Fight spam with alias addresses from safersignup.com! ---