| Solaris |
|
|
Version 3, updated 19 February 2010
The Automated Installation (AI) system for Solaris Next is required to provide a scalable and easy-to-use automatic deployment capability for organizations that wish to deploy multiple instances of Solaris. At a high level, the Automated Installation process is implemented by booting a Solaris installation environment on the (virtual) machine to be installed, locating an installation manifest, and executing the installation based on the manifest contents, which specify storage configuration, software content, and system configuration attributes. The installation environment may be booted from either local media, such as a CD (perhaps emulated by the system controller using an ISO image) or USB flash memory device, or more commonly over an attached network. The installation environment for Solaris Next is built using the Distribution Constructor (DC) and is currently published in ISO and USB image formats. These images are made available for download from either internal Sun Release Engineering servers (for Sun-internal users) or from Sun's download center and external mirrors for OpenSolaris releases.
In order to use network booting for AI, it's necessary to first provision services on a server that support the network booting architecture for the client platform (x86 or SPARC). The installadm(1M) utility is provided as part of AI to facilitate this service provisioning. As of OpenSolaris 2009.06, this procedure minimally consists of the steps:
Once the above steps have been completed, systems on the server's local network may be booted using DHCP-based network booting (PXE on x86) and will install a standard package set (currently equivalent to the OpenSolaris live CD) from the pkg.opensolaris.org repository onto the boot disk. Customizing this behavior requires generating an alternate AI manifest and publishing it for either the entire service or a specific client using installadm.
When deploying a large-scale Solaris infrastructure such as that used by Solaris engineering and many of our larger customers, it's usual to deploy AI service instances that are local to a site or a specific network in order to achieve desired levels of performance and/or to conform to administrative boundaries (the design of the software does not require this, however). Further, in many instances it's desirable to automatically update those local service instances as new releases are made available for deployment. In Solaris engineering, we've historically used an internal tool called "getimage" that automated deployment of each Solaris build from release engineering to local Jumpstart servers; this functionality has not been made available to customers, who presumably either write their own similar scripts or use Sun products such as Ops Center or third-party deployment management solutions. Providing basic deployment management functionality as part of Solaris Next would be helpful in managing its deployment both internally and in customer environments, and is desired as soon as possible.
Primary requirements to be considered in the solution include:
Secondary factors relevant to the solution include:
A solution to this problem includes several components:
In defining a solution, we note that controlling the pkg(5) repository contents effectively controls the software made available for installation by the AI clients. As such, customers of any scale can be expected to deploy their own local mirrors of Sun's release and support repositories and select the versions published to aid in change management. We also note that installation performance of AI will typically be dominated by performance of pkg(5), as the installation mode used with AI installs pkg(5) packages; this further encourages provisioning of local mirrors. Thus, to ease alignment of administrative boundariea, limit incompatibility problems, and take advantage of the infrastructure that will be deployed around pkg(5), we should align the deployment methodology for AI boot images with that used to release the corresponding packages.
While undertaking this alignment, issues encountered so far in deploying OpenSolaris AI services have been considered and, where solutions align with the basic changes required for image management, they are included in this proposal.
It is important to note that this proposal does not include any changes in how AI manifests are managed. It is expected that parallel work on the manifest and web server design issues may lead to changes in manifest management that are sympathetic with this proposal.
We propose to publish the AI boot image content into the pkg(5) repository as the package image/autoinstall. The existing ISO and USB images will continue to be made available via other download mechanisms in order to support burning bootable AI media on systems not managing software with pkg(5). The Distribution Constructor will be extended to provide automatic publication of the images to a pkg repository if specified in the construction manifest.
The packages in the product repositories will be, like other packages, "fat", containing both the x86 and SPARC image content, each element tagged with its appropriate variant.
Note that this proposal does not address publishing interactive installation images into the pkg repository.
Provisioning an AI server using IPS changes from the process described earlier, both visibly and under the covers.
The visible change is that the separate download of the ISO image is no longer required. The installadm invocation to provision a service from pkg(5) becomes:
installadm create-service [-s <package-spec>] [-a <architecture>] [-n <svcname>] [-y] [<targetdir>]
Note that -s is optional; in its absence, the image/autoinstall package will be installed from the first publisher in the system's publisher preference list providing an instance of that package. The package-spec can also be a full FMRI to an alternate package instance or a URI to a p5i file. Note that if a publisher that requires authenticated access (such as pkg.sun.com/opensolaris/support) is used then appropriate authentication must be set up for the publisher prior to execution of this command. The existing -s option to select a source ISO file will continue to be supported to allow servicing of earlier boot images that were not published into pkg(5), or for convenience when an ISO has already been downloaded. It is not possible to convert a service provisioned using the ISO to be provisioned from the pkg repository, however.
The -a option allows selection of a specific variant architecture; legal values at this writing are i386 or sparc. If not specified, the variant corresponding to the server's architecture is installed.
The -n option specifies the service name to be used. It is optional, and if not specified a service name will be generated automatically. To improve the quality of generated names (see bug 6812), the image/autoinstall package published will include a metadata tag defining a default name that can be consumed by installadm to generate an appropriate name for the service; this tag name is proposed as "org.opensolaris.autoinstall.svc_name". Typical values might be names such as "os-dev-131" for OpenSolaris development builds, "opensolaris2010.03" for OpenSolaris release builds. Generated names will append the architecture of the service in order to allow all architectures to be automatically named and served from a single server.
The <targetdir> is configured as a pkg(5) user image, and the package installed to it. A separate user image is used in order to meet the requirement to support multiple architectures and releases, and to account for the requirement that the server's release level be allowed to evolve independently of the AI services exported from it. If <targetdir> is not specified, the default system path /export/auto_install/<svcname> is used, though the command will prompt to confirm the operation; the prompt may be suppressed by supplying the -y option.
An alternative design that was considered was to publish the ISO file itself as the package contents, and to have the AI service mount that image using lofi(7D). However, this alternative is not compatible with the requirement (noted in bug 10797) to serve the images from a non-global zone, as lofi is not usable within a non-global zone (see CR 6354954).
A related change that will be made is to remove the -f option to "installadm create-service", as there are insufficient use cases to support its presence in the interface with the reorganization of the data proposed in this design.
Internally, the provisioning design will include changes to simplify the implementation:
- For both SPARC and x86 services, the service image will be mounted using lofs(7FS) at the path /etc/netboot/<service-name>. In the case of SPARC services, the associated wanboot.conf file will be found in the root of the service image. The lofs mounts will not be placed in /etc/vfstab, but will be performed as necessary by the install/server service based on the state of the services. This change requires modifying the tftp/udp6 SMF service to invoke in.tftpd rooted at /etc/netboot. If needed for compatibility purposes, /tftpboot can be created as a symlink to /etc/netboot.
(An alternate design possibility is to force a service directory to always be a separate ZFS dataset, and manipulate the mountpoint properties; however, we'd need to automatically create ZFS datasets as part of the provisioning; there doesn't seem to be a compelling reason to do this.)
- For x86 services, the pxegrub 2nd-stage loader will be referenced from the service image, rather than being copied into the /tftpboot (now /etc/netboot) directory; this requires modification to the DHCP configuration. Similarly, we will no longer generate the menu in the /tftpboot directory, but rather in the service image. Further, we propose to extend pxegrub's automatic search for the menu file (see CR 6816519 for details on how this works) to search within the path provided for pxegrub itself in the BootFile DHCP option; specifically, pxegrub will traverse up to the base directory path specified for pxegrub itself. For example, if the path to pxegrub is specified as /foo/boot/grub/pxegrub then pxegrub will search for its menu at /foo/boot/grub/menu.lst, /foo/boot/menu.lst and /foo/menu.lst in turn. This change allows us to dispense with setting the GrubMenu site option in the DHCP configuration (users can still define and use this option to override any of the automatic search behavior in pxegrub, however). Together, these changes address a number of existing issues in the implementation, including bugs 11537, 11064, 11052 and 11008. They also allow the removal of the -f option to installadm create-service; its use case was primarily around helping users to manage the contents of /tftpboot, which is accomplished more directly by this design.
- For both SPARC and x86 services, the symlinks currently used to link the images into the HTTP server's document hierarchy will be eliminated. The Apache configuration generated by installadm will instead reference /etc/netboot as the document root.
- The property group names used in SMF for recording each service's configuration will no longer be based on the service name, which is instead (and already) recorded as a property of the service; see bug 13061. Property group names will be generated to be unique for the SMF service, but are otherwise not meaningful; they are an implementation detail and classified as a Project Private interface in the ARC taxonomy.
- The install.conf file that's currently created for SPARC services will be removed from the implementation. It is currently downloaded separately during boot and used to communicate to the client the name of the AI service it should contact to locate its manifest. The wanboot infrastructure already provides a system configuration file (system.conf) for this purpose that is automatically returned, if configured in wanboot.conf, as part of the wanbootfs payload. The items passed in install.conf will be relocated to system.conf and the wanboot.conf that is generated will include the path to the system.conf file.
XXX To be investigated is whether we can use system.conf for x86 as well and eliminate the extensive use of boot arguments in the menu.lst file. Alok notes that this would help with restartability of automated installation as well.
The generated DHCP configuration will be changed substantially in comparison to the existing implementation:
- The DHCP configuration will be modified to remove the current implementation's manipulation of the per-network DHCP address assignment tables to assign clients to services. The basic idea here is to take advantage of two characteristics of the DHCP protocol specification:
The Solaris DHCP server provides auto-matching of the Client Class and Client Identifier against the names of any configured macros, returning to clients any options defined in the macros that match the Client Class and Client Identifier. ISC DHCP and other DHCP servers can be configured to provide similar capabilities. It also provides a simple inclusion-by-reference directive that allows construction of simple chains of macros. By leveraging these capabilities, we are able to provide straightforward configuration of default behavior per-architecture, while allowing for client-specific overrides, without need for manipulation of the per-network address assignment tables.
- The DHCP macro names generated for each AI service will use a prefix of "AI_" rather than the "dhcp_macro_" prefix that is currently prepended.
- The DHCP macros generated for SPARC services will no longer contain the unnecessary BootSrvA option (bug 9768).
In order to limit the need for manipulation of the DHCP configuration to support client-specific installations in environments where updates to DHCP configuration are inconvenient, we propose that:
- When x86 services are configured, the install/server service will automatically assemble a global GRUB menu incorporating the entries from all configured services. The default item selected in the global menu will be the "default-i386" service (see the "Update" section, below). The global menu will be associated with an "any-i386" service that is reserved for this purpose and is automatically created when the first x86 AI service is created. This configuration allows the user to configure all clients to use the any-i386 service in DHCP and then select interactively which specific service will be used. If this capability is not desired the user can merely delete the any-i386 service using "installadm delete-service". As there are implementation limitations on the size of the menu that pxegrub can load, a SMF property (global_menu) is created on each service, with its value set to "true". If a service should be excluded from the any-i386 menu, the administrator can set this property to "false" using the "installadm set" command, which was included in the original AI design but has not yet been implemented; this project implements that command. This property may be present on SPARC services, but has no effect since there is no analogous SPARC service to any-i386 (see following paragraph).
XXX Consider generalizing the global_menu concept into an aggregating capability that allows a service to be created as an aggregate service (which doesn't have its own image). Change the global_menu property to aggregate_menu and make it a list of aggregate service names.
- SPARC systems are capable of network booting without the assistance of DHCP using the OBP's wanboot capability. In this case, the user supplies the IP address and associated minimal network configuration, as well as the URL from which the boot program will be downloaded. This URL is the wanboot CGI program (wanboot-cgi), which walks the /etc/netboot hierarchy to locate the appropriate configuration based on the IP network of the client and its client ID. At present, wanboot-cgi requires that a client-specific configuration must be associated with a specific IP network; we propose to enhance wanboot-cgi to allow for client-specific configuration to be associated only with the client ID, without a containing network. Providing an interactive selection such as that described above for x86 is beyond the scope of this project, but may be a useful enhancement to the wanboot loader. In its absence, this project does not propose to create an "any-sparc" service analogous to the "any-i386" service described in the previous section.
- In order to configure an individual client to use a specific service, installadm create-client will appropriately link client-specific configuration files (wanboot.conf for SPARC, menu.lst for i386) to the files for the specified service, and will not update the DHCP configuration, even if DHCP is being managed locally on the installation server. See the examples at the end of this document. However, as noted above, the administrator can also use appropriate client macros on the DHCP server to assign clients to services.
As an aside, a future provisioning enhancement that seems attractive is to support the provisioning of a combined pkg depot + AI boot server from the repo-on-DVD images produced beginning with OpenSolaris 2009.06.
Once an AI server has been deployed, the administrative focus shifts to maintaining services, defining the service that is the default for the environment as services are added and removed, and configuring specific clients to access services other than the default.
The typical usage model in Solaris engineering is to maintain installation servers that serve the most recent bi-weekly build of Solaris as the default, as this is what developers are most likely to require in configuring new systems. Older builds retained for varying lengths of time; the retention policies for older builds are usually focused on living within the server's available disk space. To install the older builds, users must perform some administrative action on the server to select the desired non-default build for installation.
Specialized environments such as those managed by a given consolidation for testing of daily builds, or developers of the installation software, might generate new boot images one or more times per day, and may not wish to retain older versions. Typical customer environments seem to change somewhat less frequently, with new releases deployed monthly, quarterly, semi-annually, or even less frequently; typically they will deploy a new service for testing to a small set of systems, and then transition to it as the default once it is validated for service in that role.
Regardless of the update frequency, typical usages for updating installation boot images include:
An underlying requirement to the above usages is to be able to rename an existing service, an operation that is not yet supported. An additional requirement is to be able to run any of the update scenarios automatically at a scheduled time; for this purpose, use of cron(1) by an appropriately privileged user is asserted to be sufficient.
We propose new subcommands to installadm to support the above usages:
installadm update-service [-s <srcimage>] <svcname>
The service to be updated must be named. Additionally, as noted in the Provisioning section, update-service will not act on a service image provisioned from an ISO image. If -s is specified, the srcimage must be a package specification in the form of a pkg FMRI or URI to a .p5i package info file.
installadm alias-service <basesvcname> <aliassvcname>
An aliased service is implemented by the SMF property service_alias, which references the name of the real service underlying the aliased service.
In support of the update use cases described above, the first service of each architecture that is created on a server will be automatically aliased to the default-<architecture> name, i.e. default-sparc, default-i386. Where DHCP configuration is performed by installadm, DHCP macros are generated for aliased services; these merely include by reference the macro for the base service. The default-<architecture> service names are only a convention and are neither required nor reserved; should they not be desired, they can be deleted.
In addition, we find that it will be useful to include service rename support in installadm, as:
installadm rename-service <svcname> <newsvcname>
Any clients assigned, or services aliased, to the existing service name will be updated to use the new service name.
To satisfy each of the above use cases, the resulting command invocations would be:
One issue that is not addressed in updating of services is the effect on existing, published manifests for that service, as an updated service is potentially incompatible with the existing manifests. Manifest versioning is being examined as part of manifest design efforts and once that direction has been established the service management design will be updated if warranted.
The command for removal of an AI service is modified to be:
installadm delete-service [-r] <svcname>
The existing -x option is removed, and its behavior (delete the pkg image used to instantiate the service) is made the default.
If the service to be removed is currently aliased by another service, the delete-service will fail, noting the services that are aliased to it so that the user can either delete or re-alias them to another service. This has to be handled somewhat carefully when manipulating the DHCP configuration, as removal of a macro that is included by another macro (as in the case of the default-sparc service, see example below) creates an invalid DHCP configuration that will cause poor DHCP server behavior.
Removal must also attempt to determine whether any clients are specifically configured to use the service; if so, the delete-service will fail and note the clients that are configured to use it.
The -r option is provided to allow the user to request automatic removal of any dependent alias services and clients.
The provisioning of an AI service named os-dev-131-i386 on the server at 172.0.0.1 might be performed by the following command:
installadm create-service -n os-dev-131-i386 -a i386 -i 172.0.0.10 -c 10 /export/ai/os-dev-131-i386
We assume that this is the first AI service on the system. This would result in the following operations (details of configuring the criteria server are omitted below); the actual sequence of operations may vary somewhat from the sequence shown here.
At this point, the service is operational and a client booted via PXE will boot and install from the default-i386 service. If the administrator decides to make the any-i386 service the default used by the environment, dhtadm or DHCP Manager can be used to change the Include reference on the PXE macro to the AI_any-i386 service.
Provisioning a corresponding SPARC service to that shown in example 1 (assumes example 1 has already been done, so some global setup steps are omitted):
installadm create-service -n os-dev-131-sparc -a sparc /export/ai/os-dev-131-sparc
The resulting operations are below.
Deploying a Solaris engineering-style rolling default service with automatic naming is accomplished by initially configuring the service with:
installadm create-service -a i386 -y
This creates an x86 service based on the default name under the base directory /export/auto_install, with the default-i386 service aliased to it; an alternate base directory can be specified as an additional argument on the command line. Example 1 describes the processing that occurs.
The service can be automatically updated by:
installadm update-service default-i386
This operation checks whether an update is available for the default-i386 service. If an update is available, the fact that it is an aliased service causes update-service to internally convert to a create-service using the default name provided for the update, then re-alias the default-i386 service to the new service after the create-service has successfully completed. This command can be placed in a cron(1M) job to run at convenient times.
Configuring an x86 client with MAC address 0:14:4f:4a:99:1a to install using a specific service requires the command:
installadm create-client -e 0:14:4f:4a:99:1a -n os-dev-131-i386
The resulting operations are:
A critical assumption here is that the boot loader referenced by the default PXE client class macro is compatible with the service. At GRUB 2 transition this may require some additional work to detect incompatibilities and behave correctly.
Configuring a SPARC client with MAC address 8:0:20:a:b:c to install using a specific service is done by issuing the command:
installadm create-client -e 8:0:20:a:b:c -n os-dev-131-sparc
The resulting operations are:
Base design documents for Automated Installation can be found at the documentation page
An updated installadm(1m) man page and diffs against the existing man page as of this document's writing are available.
Terms of Use
|
Privacy
|
Trademarks
|
Copyright Policy
|
Site Guidelines
|
Site Map
|
Help
Your use of this web site or any of its content or software indicates your agreement to be bound by these Terms of Use.
© 2012, Oracle Corporation and/or its affiliates.