Xen is a very powerfull virtualization program, which also allows using techniques like AMD Pacifica or Intel Vanderpool to run the virtualized OS directly on the hardware of the machine without any emulation. This makes it very powerfull and fast compared to an emulated OS. But it actually is a pain in the neck if you want to install it and are a newbie concerning virtualization.

This tutorial now shows you how to compile the necessary Xen-Kernels and how to set up some example VMs running Linux itself again and Windows. It also is strongly adviced not to use the Xen-Version available in the update-manager of your OS (e.g. apt-get) because it is in most cases outdated.

First of all: How is Xen managed?

• Xen runs different OS which are called Domains
• Dom0 is the Host-OS which directly runs on the hardware and manages access to the hardware for other OS
• DomU’s are virtualized OS which run upon the Dom0.
• Dom0 needs a special kernel for hypervising the hardware. The compiling-steps are showed below.
• DomU’s (if Linux) need a special kernel to run (para-virtualization). The compiling-steps are showed below.
• If the DomU is Windows, your Hardware needs to support AMD Pacifica or Intel Vanderpool. Check this with xm dmesg | egrep ‘(VMX|SVM)’

Compiling Xen Kernels

Assuming you have Debian Lenny installed, make sure you have all necessary tools for compiling and running Xen

apt-get update
apt-get install bcc bin86 gawk bridge-utils iproute libcurl3 libcurl4-openssl-dev bzip2 module-init-tools transfig tgif texinfo pciutils-dev mercurial build-essential make gcc libc6-dev zlib1g-dev python python-dev python-twisted libncurses5-dev patch libvncserver-dev libsdl-dev libjpeg62-dev linux-headers-`uname -r`

If you have a 64bit OS, also run

apt-get install gcc-multilib

Now download Xen from Xen.org and unpack it.

wget http://bits.xensource.com/oss-xen/release/3.4.2/xen-3.4.2.tar.gz
tar -xzf xen-3.4.2.tar.gz
cd xen-3.4.2

Now start compiling the Dom0 and DomU kernel. This will take several minutes. Better get a drink – or two.

make -j4 world KERNELS="linux-2.6-xen0"
make -j4 world KERNELS="linux-2.6-xenU"

Once both is done without any errors, run

make -j4 linux-2.6-xen0-config CONFIGMODE=menuconfig
make -j4 linux-2.6-xenU-config CONFIGMODE=menuconfig

You may get asked for setting options manually by a prompt for the Dom0 Kernel. I personally prefer the following options, but you have to decide what you want and what not.
show

# y: yes, n: no, M: compile as kernel-module
### Memory settings and PCI-Passthrough options.
PRECACHE: y
PRESWAP: y
X86_MCE: y
PCI_GUESTDEV: y
PCI_IOMULTI: y
PCI_RESERVE: y
PCI_IOV: y
 
### IMQ: This made Xen not boot for me, when all set to y
IP_NF_TARGET_IMQ: n
IP6_NF_TARGET_IMQ: n
IMQ: n
 
### Networkdriver support
IGB: y
E1000E: y
IXGBE: y
 
### Xen-Kernel crashed with this option enabled for me
SENSORS_CORETEMP: n
 
IPATH_CORE: n
 
RTC_DRV_CMOS: M
 
### USB Passthrough options
XEN_USB_BACKEND: M
XEN_USB_FRONTEND: M
XEN_USB_FRONTEND_HCD_STATS: y
XEN_USB_FRONTEND_HCD_PM: n

Once both is done, create the DomU Kernel which is used in paravirtualized Linux-Guests later:

make -j4 install KERNELS="linux-2.6-xenU"

Now prepare your Host OS’ (Dom0) Kernel and install Xen

make -j4 dist
./install.sh

Register startscripts for the Xen-Daemon

update-rc.d xend defaults 20 21
update-rc.d xendomains defaults 21 20

Build dependencies

depmod 2.6.18.8-xen0

Update kernels and the bootmanager grub.

update-initramfs -c -k 2.6.18.8-xen0
update-grub

Now, the Xen-Dom0-Kernel is installed to your Host-OS. It will be available after a restart.
Attention: We experienced a problem, where after a restart our network devices got renamed from eth0 to eth1. This rendered our server’s network config useless and we weren’t able to connect to the server anymore. So make sure, this does not happen to you. If so, modify your network script

nano /etc/network/interfaces

and rename eth0 to eth1 to make networking run again.

If everything is prepared, restart your system with

shutdown -r now

and check, if the Xen-Kernel is loaded with

uname -r

It should return stuff similar to

2.6.18.8-xen0

Preparing Guests/DomU’s

Linux Guests
If your Guest is a Linux-System, you need to make it use Xen’s specialized DomU Kernel, which you have compiled above already. Assuming, you have created a space where your Guest-OS writes it’s data to (either to an image  – slow – or directly to a partition using e.g. LVM – fast), you can create a start-script for it in the Xen-Folder. The below example uses a partition for writing the data.

cd /etc/xen/
nano linux

Add the following lines

show

kernel = "/boot/vmlinuz-2.6.18.8-xenU" #Use the DomU Kernel!
ramdisk = "/boot/initrd.img-2.6.18.8-xen0"
memory = 512
name = "linux"
cpus = ["0", "1"] # Use the CPU's Core 0 and 1
vcpus = 2 #How much "Virtual" CPUs do you want to "emulate" for that Guest?
 
#vif is the virtual network interface, which is present on the HOST (Dom0). Tell it the IP to use to connect to the internet. The mac-setting is not necessary, but keeps it constant once set here (and not randomly assigned which can cause havoc)
vif = ['ip=123.123.123.123,mac=00:16:3e:00:00:01,vifname=linux-vif']
 
#Where should be the partition mounted to (/home/xen/linux) and what is it's physical address (/dev/hda1)
#Also specifies path to the installer-CD/DVD image in order to install that OS.
disk = ['phy:/home/xen/linux,/dev/hda1,w','file:/home/linux_install_image.iso,hdc:cdrom,r']
root = "/dev/hda1"
 
extra = 'console=xvc0 xencons=tty' #allow the VM to be accessed by the Xen-Console
 
#What to do on shutdown etc
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'

Try starting the VM with

xm create linux

You can now install your preferred Linux OS from the ISO CD-Image. But my suggestion is (assuming you use Debian as Dom0) using debootstrap.
After the successful boot, you should be able to access the console now with

xm console linux

You can shutdown that VM using

xm shutdown linux

or kill it with

xm destroy linux

Windows Guests

Also make sure, you have a place where the files are written to (either an image  – slow – or directly to a partition using e.g. LVM – fast). The below’s example uses a partition.

Then create a start-script

cd /etc/xen/
nano windows

Here, it should look like
show

import os, re
arch = os.uname()[4]
if re.search('64', arch):
    arch_libdir = 'lib64'
else:
    arch_libdir = 'lib'
 
kernel = "/usr/lib/xen/boot/hvmloader"
builder='hvm'
memory = 512
 
cpus = ["0","1"] # Use the CPU's Core 0 and 1
vcpus = 3 #How much "Virtual" CPUs do you want to "emulate" for that Guest?
 
shadow_memory = 16 # Should be at least 2KB per MB of domain memory, plus a few MB per vcpu.
name = "windows"
vif = ['ip=123.123.123.123,mac=00:16:3e:00:02:01,vifname=win-vif']
 
acpi = 1
apic = 1 #Necessary for MULTI-CORE setup
disk = ['phy:/home/xen/win,hda,w','file:/home/windows.iso,hdc:cdrom,r'] 
 
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'
 
#Boot-order. d is the cdrom (hdc)
boot="cd"
sdl=0
 
#Optimizes Xen for running Windows
guest_os_type='windows'
 
serial='pty'
usbdevice='tablet'
keymap="de"
 
#What to do on shutdown etc
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'
 
#VNC: Probably required for you for installation
vnc=1 #TURN THIS OFF AFTER INSTALLATION,OR FEAR SECURITY ISSUES
vncconsole=1
vncpasswd='yoursecretvncpassword'
vnclisten='0.0.0.0'

Once you started the VM for the first time, you need to connect to the machine via VNC on <Dom0.IP> :9001 in order to make the OS install. VNC is enabled in the config-file above by default.
Note: Make sure to disable VNC once the installation is done or you risk a security threat!

Making the Network work

This can be really a pain in the ass. If you run Xen at home or if you are allowed to use network-bridging, you only have to change a few things in Xen’s config.

First make sure, what your network-device name on Dom0 is (remember: for us, Xen renamed it from eth0 to eth1). Find this out with

ifconfig -a

Now edit the Xen-Config…

Dom0 Network-Bridging setup (does not work with hetzner.de)

nano /etc/xen/xend-config.sxp

and modify the network-lines by adding the netdev settings.

(network-script 'network-bridge netdev=eth1')
(vif-script     'vif-bridge netdev=eth1')

Restart the Xen-Daemon with

/etc/init.d/xend restart

Note: Our hoster (hetzner.de) does not allow the bridging. So instead we need the route-scripts which are now described below.

Dom0 Network-Routing setup (only option which works on hetzner.de servers)
Comment out

#(network-script 'network-bridge netdev=eth1')
#(vif-script     'vif-bridge netdev=eth1')

and add the lines

(network-script 'network-route netdev=eth1')
(vif-script     'vif-route netdev=eth1')

Now you also have to edit /etc/xen/scripts/vif-common.sh
Find the ip_of(){…} block and replace it by

ip_of()
{
#  ip addr show "$1" | awk "/^.*inet.*$1\$/{print \$2}" | sed -n '1 s,/.*,,p' #Old code!
  ip -4 -o addr show primary dev $1 | awk '$3 == "inet" {print $4; exit}' | sed 's!/.*!!'
}

Restart the Xen-Daemon with

/etc/init.d/xend restart

Now edit /etc/sysctl.conf and make it look like

### Hetzner Online AG installimage
# sysctl config
#Default Hetzner.cfg
##net.ipv4.ip_forward=1
#net.ipv4.conf.all.rp_filter=1
#net.ipv4.icmp_echo_ignore_broadcasts=1
 
net.ipv4.conf.all.rp_filter=1
net.ipv4.icmp_echo_ignore_broadcasts=1
 
net.ipv4.ip_forward=1
net.ipv6.conf.all.forwarding=1
#net.ipv4.conf.all.proxy_arp=1
net.ipv4.conf.default.proxy_arp=1

Make this changes available with

sysctl -p

Configure the network-settings in /etc/network/interfaces to look like

auto lo
iface lo inet loopback
 
# device: eth1
auto  eth1
iface eth1 inet static
	address   111.111.111.111 #Ip of this server
	netmask   255.255.255.192
	gateway   100.100.100.100 #Ip of the gateway
	pointopoint 100.100.100.100 #This is important to make routing work (gatway IP)

Make sure, there are no additional up route lines anywhere. The above settings are totally sufficient!

That’s quite easy. Just edit /etc/network/interfaces and add

auto lo
iface lo inet loopback
 
# The primary network interface
auto eth0
iface eth0 inet static
	address 122.122.122.122
	netmask 255.255.255.192 #Same as Dom0
	gateway 111.111.111.111 #Dom0 IP
	pointopoint 111.111.111.111 #Dom0 IP
	post-up /sbin/ip route add default dev eth0

DomU network setup – Windows

That’s not that easy. First of all, make sure you allow the installation of unsigned drivers if you run a x64 OS. Login to the OS (if installed) using eg VNC – look at the windows-installation-steps concerning VNC (network and therefore the remote-desktop shouldn’t work yet).
Run this in a command-prompt (if x64 only!)

bcdedit /set testsigning on
bcdedit /set loadoptions DDISABLE_INTEGRITY_CHECKS

Reboot. Now Windows should boot up with “Testmode” in the right lower corner.

Now install the GPLPV driver for Windows and reboot if asked. They allow Windows to use PCI-Passthrough and you will gain a noticeable performance boost from this.

Once rebooted one more time, you can configure the network. Set the IP of your server. The netmask has to be the one of the Dom0 and the gateway should be the IP of Dom0. As DNS, use the DNS servers of your hoster (for hetzner.de, you find the IPs here).

Now networking should work.

Note: I experience a low download-rate (around 12 kb/s) from my server while the upload to it was fast (maximum speed). This was caused by a network-driver-option of the GPLPV drivers. Go to Controlpanel/System/Hardware/Networkadapters/Xen Net Device Driver and click properties. At Extended, go to Large Send Offload and disable this option.