Configuring dom0

As of: 2009-10-28

dom0 configuration for admins

Currently, there are a few configuration steps that are strongly recommended for all Solaris dom0s. We're working on making this stuff happen by default, or not be needed.

ZFS ARC limitation

ZFS interacts very poorly with dom0 and will easily steal all the available memory. It's essential to limit the ARC via adding to /etc/system:

    set zfs:zfs_arc_max = 0x10000000

and rebooting.

Use a dump file system

xend will save domain crash dumps in /var/xen/dump. To prevent it from filling up the root file system, you can use a ZFS file system:

    # zfs create -o mountpoint=/var/xen/dump,quota=4G rpool/xendumps

Older OpenSolaris releases

That's all you really need if you're running build 126 or later, and you've used svcadm enable milestone/xvm. If you're on an older release, however, you also need the following.

dom0_mem

Limit dom0 to a specific amount of memory via this Xen option. For example, if you have 4Gb of memory, give dom0 1Gb like so (in /rpool/boot/grub/menu.lst):

    ...
    kernel$ /boot/$ISADIR/xen.gz dom0_mem=1024M
    ...

To prevent ballooning below this limit, you should also tell xend:

    svccfg -s svc:/system/xvm/xend setprop config/dom0-min-mem = 2334
    svcadm refresh svc:/system/xvm/xend:default
    svcadm restart svc:/system/xvm/xend:default

Configuration for developers

There's also a bunch of very useful settings recommended if you're doing Xen or Solaris development, trying bleeding-edge builds, etc.

Enable libvirt debugging

You'll need to install the debug xVM packages. You can debug libvirt clients such as virsh like this:

    # LIBVIRT_DEBUG=1 virsh list

which will output on the console. Debugging libvirtd is a little harder. First, ensure that you have coreadm set to enable global setuid cores:

    # coreadm -e global -e global-setid -e log -g /var/cores/core.%f.%p
    # coreadm
         global core file pattern: /var/cores/core.%f.%p
         global core file content: default
           init core file pattern: core
           init core file content: default
                global core dumps: enabled
           per-process core dumps: enabled
          global setid core dumps: enabled
     per-process setid core dumps: disabled
         global core dump logging: enabled

libvirtd will only coredump if you enable suid cores! Next, enable libvirtd logging:

    # svccfg -s virtd
    svc:/system/xvm/virtd> setenv LIBVIRT~_DEBUG 1
    svc:/system/xvm/virtd> end
    # svcadm refresh virtd
    # echo -e "*.debug\t/var/adm/debuglog" >>/etc/syslog.conf
    # touch /var/adm/debuglog
    # svcadm restart system-log
    # svcadm restart virtd

Serial console

Setting up a serial console is strongly recommended to allow you to catch any panic messages that might come from Xen itself. If Solaris is set up to use ttya (say), then Xen should have.

    ...
    kernel$ /boot/$ISADIR/xen.gz console=com1 com1=auto dom0_mem=1024M
    ...

More here.

Enable VNC connections from any host

Be warned: this will allow ANY host to access your VNC sessions! It's very convenient for dev labs though. Make sure no domains are running, then:

    # svccfg -s xend <<EOF
    setprop config/vnc-listen = astring: "0.0.0.0"
    EOF
    # svcadm refresh xend
    # svcadm restart xend

Enable migration from any host

Be warned: this will allow ANY host to migrate domains to your machine!

    # node=`uname -n`
    # ip=`getent hosts $node | awk '{print $1}'`
    # svccfg -s xend <<EOF
    setprop config/xend-relocation-address = astring: "$ip"
    setprop config/xend-relocation-hosts-allow = astring: ".*"
    EOF

    # svcadm refresh xend
    # svcadm restart xend