xVM Dependencies on Installer/LiveCD layouts

Install dependencies

As of: 2009-09-29

virt-install is a Python utility that is used for installing Xen/xVM
domUs. It has a number of dependencies upon Solaris install behaviour,
and this document is an attempt at enumerating them.

Detailed docs can be found via the man pages xVM(5) and
virt-install(1m).

One important point: most of these dependencies also apply to Linux
dom0s: they use the same code!

HVM installs

"HVM" installs provide a simulated x86 machine for the guest to run in.
In particular, there's an emulated BIOS that directly runs grub. In this
sense, the install path is "as usual".

For example, we might run:

    virt-install -n domu-222 -r 1024 --mac `~johnlev/bin/maca domu-222`
    -f /dev/zvol/dsk/export/zvol/domu-222-root --hvm --os-type=solaris
    --os-variant=solaris10 --vnc
    -c /net/heaped/export/netimage/s10u6/07/solarisdvd.iso

to install from an ISO.

As we run via the system's grub, we do not have the ability to pass
command line options in, so this case is much less dependent upon the
Solaris installer.

PV installs

PV installs, however, have a much higher-level platform abstraction. In
particular, instead of booting grub, we grab the kernel+ramdisk directly
from the media (typically an ISO, but also NFS), and run that directly
along with a set of command line options.

For example:

    virt-install -n domu-220 -r 1024 --mac `~johnlev/bin/maca domu-220`
    -f /dev/zvol/dsk/export/dom/domu-220-root --paravirt --os-type=solaris
    --nographics -l /net/heaped/export/netimage/opensolaris/osol-0906-109.iso

will do a PV install of OpenSolaris from the given ISO path.

Because we need to find the kernel and ramdisk binaries, we have a hard
dependency on their locations. If the guest has at least 1024Mb RAM allocated, and the host is 64-bit capable, we boot in 64-bit mode. Otherwise, we boot in 32-bit mode.

We look in the following paths for 32-bit:

    OpenSolaris kernel: /platform/i86xpv/kernel/unix
    OpenSolaris ramdisk (2009.06 and earlier): /boot/x86.microroot
    OpenSolaris ramdisk (post-2009.06, b128 or so): /platform/i86pc/boot_archive

    Nevada kernel: /boot/platform/i86xpv/kernel/unix
    Nevada ramdisk: /boot/x86.miniroot

And these paths for 64-bit:

    OpenSolaris kernel: /platform/i86xpv/kernel/amd64/unix
    OpenSolaris ramdisk (pre-2009.06): /boot/x86.microroot
    OpenSolaris ramdisk (pre-2009.06): /boot/amd64/x86.microroot
    OpenSolaris ramdisk (post-2009.06, b128 or so): /platform/i86pc/amd64/boot_archive

    Nevada kernel: /boot/platform/i86xpv/kernel/amd64/unix
    Nevada ramdisk: /boot/amd64/x86.miniroot

We also depend on being able to read the filesystem. Installation from
unusual file systems will thus be a problem (we also have to track ZFS
versions, sadly).

PV command line

As well as the actual kernel+ramdisk, we have to pass in a synthetic
command line similar to that set in the grub menus for hardware case.
This can/will include:

 - any options set by the use with virt-install -x
 - any networking setup (see below)
 - ZFS root pool information

We have some processing code that understands the difference between
(say) -kdv and -B options. We process, but ignore, -m (SMF) arguments at
the moment.

For ISO installs, we pass -B install_media=cdrom. We expect the
installer to correctly find the (emulated!) CD.

We have special ZFS handling. When reading the kernel/ramdisk from the
ZFS file system,  (so this is post-install), we grab the current root
pool objset, and pass that in to mount, for example:

    [2009-03-16 18:18:14 100341] DEBUG (image:634) cmdline        =
     /platform/i86xpv/kernel/amd64/unix -B
     zfs-bootfs=rpool/57,bootpath='/xpvd/xdf@51712:a'

Netinstalls

In order to support NFS installs in PV mode (we don't have PXE!), we
pass in a bunch of config options that the installer understands. In
particular:

1. if the user specifies subnet-mask in the -x argument, we grab and use it.
   2. If we can't resolve the given guest name on the dom0, we pass in 'dhcp'
   3. we construct the following:

     -B install_media=x.x.x.x:/path/to/media/dir/,host-ip=y.y.y.y,
     subnet-mask=...,router-ip=z.z.z.z,boot-mac=MAC, where
     x.x.x.x is the NFS server's IP address, y.y.y.y is the guest's IP
     address, z.z.z.z is dom0's default route, and MAC is the --mac option

This is generally enough to drop the Nevada installer into  NFS install
mode.

We don't do any of this currently if it's OpenSolaris media (see AI
below).

PXE

PV mode doesn't support PXE. HVM mode, however, has an emulated PXE ROM,
and works (AFAIK) the same as the real hardware case, via the --pxe
option.

JumpStart

if --autocf is specified, it points to a jumpstart config. We add the
option " install " to indicate this.

The config must be on an NFS host. We add to the above netinstall
cmdline:

     sysid_config=x.x.x.x:/path/to/jumpstart,
     install_config=x.x.x.x:/path/to/jumpstart

where x.x.x.x is the IP address of the NFS server with the JumpStart
config.

For HVM guests, --autocf points to a floppy image that contains the
jumpstart config, just like real hardware (and somewhat crazily).

AI

For HVM guests, we leave it to --pxe and the HVM-emulated PXE boot to just
do the right thing.  I'm not aware of anything in AI that allows us to
pass the equivalent of the sysidcfg file contents from jumpstart via floppy,
and we're unable to change grub entries for guests from dom0 with HVM
unfortunately.

For PV guests, virt-install downloads the kernel and boot_archive from
a path specied by the "-l" flag in order to boot the domU, the same as
Jumpstart.

This implies that the user has to know the AI is serving on port 5555.

AI guest installs differ from Jumpstart installs, in that --autocf gets
passed a comma-separated list of name=value pairs, each of which then
get passed as a -B option to the kernel.

If the -l flag above is a http url, this is automatically passed as the
'install_media' AI boot option assuming it's not overridden by an
'install_media' autocf option.  If -l isn't a http address, we're not
automatically passing it, assuming some sort of physical AI bootable media.

So, for example we could have:

    virt-install -l http://host/path [std. domU guest options] \
    --autocf install~_service=x86_install,install_media=http://otherhost/path

the virt-install code then automatically adds some default arguments to
the kernel it's about to boot:

    "-B network-interface=xnf0,xpv-hcp=dhcp,livemode=text"

and with the fix for bug 6850, this is
enough to boot the guest, configure networking using DHCP and start the
AI install.

Additional AI keywords of interest to installing xVM guests include:

The client will usually require DNS configured before /usr is mounted. As of this writing, this is broken (see bug 11606).

'install_debug=enable'
if provided, AI runs in debug mode - good for evaluating encountered issues

'livessh=enable'
if provided, AI client enables remote access via ssh 

Bootable AI

When the bootable AI project integrates, you can install entirely from an ISO. virtadm is planning on using this functionality. If an opensolaris guest is installed, and the install source is an ISO, then /by default/ the following options are set:

    media_boot=true,auto-shutdown=enable,aimanifest=default

These are set even for the live ISO. The user can over-ride any of these parameters.

Whilst it's not currently possible to specify a manifest file available via HTTP or NFS, we would find great use in such a facility, as it means users can modify their install without having to set up an install server.

Consoles

HVM Solaris can be installed either via the emulated serial console, or
via X (with vncviewer). However, to enable the serial console install,
the user has to fire up vncviewer and interact with grub to choose the
serial mode install.

PV installs do not support graphical installs directly, as there is no
framebuffer support. However, VNC from the guest can be used.
OpenSolaris has special code for this, described in detail here:

http://blogs.sun.com/levon/entry/opensolaris_2008_11_as_a

For Nevada installer, if --nographics is passed, we add 'nowin' boot
option.

Reboots

virt-install can/will wait for installation to finish. In this case, it
assumes that the user wants to reboot after the install, and will start
up the domain again.