| Solaris |
|
|
Version 2.0, 2007-Feb-16
| John Beck | Jim Carlson | Renee Danson | Michael Hunter |
| Anay Panvalkar | Kacheong Poon | Garima Tripathi | Jan Xie |
There are six focus areas described below:
There are also two appendices:
This Architecture document was completed in April 2006, then used as the basis for the Design Document which was completed in February 2007. Most of the high-level plans laid out here can be found in a similar lower-level form there, though as one might expect, some changes were made as part of the design process:
Keep these changes in mind when reading this document.
Network Profiles, the primary component of the Network Auto-Magic project, are a way to simplify network configuration management. They work by allowing users to specify various properties which determine how things work in different circumstances. The properties include, but are not limited to:
Note that this dual-layer model was chosen to support "overlay" profiles, as discussed in our Story Boards and Requirements. Examples are provided below to illustrate this.
Let us begin with an architectural overview. The primary components are:
How they interact is roughly as follows:
One of our focus areas is "State Machine", which needs to cover both the abstract set of states for the profile daemon, and the set of possible transitions between those states. For now, we will focus on the transitions, with the idea that sufficiently specifying the transitions may suggest what the states themselves should be.
Since the new network/profile service will be replacing the existing network/physical and network/initial services (see §5.2), all of the start-up functionality of those two services will need to be accounted for. Though the state machine is designed to make bringing up an interface at boot time as much as possible like bringing up an interface on a running system, there will be a few initialization tasks required:
A transition is made up of a series of reactions to a given event. These events come in pairs: some new thing is available, some old thing is no longer available. There are three event pairs which frame the "life-cycle" of a link:
Note that booting and resuming from suspend are really just special cases where one or more of the ++ cases appear to happen at once, as the daemon will attempt to "de-queue" all pending events whenever it starts or resumes (i.e., it will attempt to examine all pending events before handling any of them). This will be part of the daemon's "damping" to maximize stability (more on this below).
Likewise, shutdown and suspend are really just special cases where the -- events happen on all links at the same time.
Generally, the normal sequence for an interface being added to the system would be link++, followed by network++, followed by one or more ip++. When an interface is removed, there would be a network~-- followed by a link~-- event; it is quite likely that there would not be explicit ip~-- events initiating the process.
An interesting case would be when a network~-- is not followed by a link~--. It could be a transient failure, or it could be a move from one network to another (in which case a network++ would follow at some point), or it could simply be that that link is "gone," whether by admin choice or because of a longer-term failure. Since the network is gone, we can do nothing with respect to it per se. But we can start a timer, then once that timer "pops" (per the profile), we might either reset all connections (if the number of networks is now 0) or try to get all services using the "dead" network to transition to one of the other networks (if the number of networks is now ≥ 1). Also note that when the timer pops, we set the state so that a subsequent "network++" event follows the "there was no previous network" path rather than the "there was a previous network" path. But note that if we get a "network++" event before the timer pops, and determine that the "new network" is the same as the "old network", then we will attempt to "damp" the events out and act as if neither event had occurred.
There are also user-driven transitions: whenever a user modifies the active profile (of either layer), or activates a different profile (at either layer), then the new active profile(s) may result in a transition. Depending on the change(s), there may be nothing to do, or there may be minor reconfigurations to make, or it may be that the user did the equivalent of pressing a giant red "reset" button.
A note on "punchin" (the IPsec-based VPN which many of us use to access the SWAN remotely): although tunnels coming and going should be detected by the event handler and thus be handled by the profile daemon as an event-driven transition, it would probably be better for us to work with the punchin team to integrate our stuff together so that punching in and out would involve using our interfaces, and thus be user-driven transitions, with the profile daemon doing the heavy lifting instead of the punchin script.
It is not clear if these transitions suggest any sort of traditional simple state model. E.g., the Zones model whose primary states are Configured, Installed and Running seems impossibly simplistic for what we are trying to achieve. Instead, it seems that we ought to come up with an abstract representation of the network configuration, and that abstraction will become the "state". Then whenever the users modifies the active profile or activates a different profile, the network configuration will be changed accordingly, as will our abstract "state". Likewise, whenever an event forces a reconfiguration, the new configuration will be reflected in our abstract "state".
The event handler must interface with the kernel, but will probably mostly run in user-land.
The event handler will monitor several sources of information: hald, routing socket, sysevents (and, longer term, link FMRIs); current thinking is that this monitoring will take place within the "profile daemon" entity.
In addition to the monitoring component, work may be required in the kernel to ensure that information is reported in a consistent manner; hald back-end support may also need to be added (this will benefit other projects as well as this one).
What information needs to be delivered? What events are we concerned with?
How will the information be delivered to consumers?
This is probably a more detailed design question, as the current thinking is that the event handler will be part of the profile daemon entity, our primary consumer.
Will the information be stored anywhere? If so, when should snapshots be taken? How many should be stored?
It is not clear that we need a repository at this point.
Roaming can be broken up into two different layers: L2 and L3. L2 is the case where there are multiple APs, on the same LAN; the IP address does not change when migrating from one AP to another. This is something that hardware implementations (either in the hardware itself or in the driver) seem to take care of (based on casual Googling and toting OS X and Solaris laptops around the building).
L3 is harder. MobileIP tries to make it transparent; that is something that we might consider doing at some point under the NWAM umbrella, but will not be part of phase 1 of the project. It has also been suggested that there are some cases where simply making the switch, without worrying about keeping existing connections alive, might be preferred behavior. This sort of gets back to the question of intent we have discussed elsewhere: is the user just surfing and checking mail, so switching to a cheaper network when it pops up is painless; or does s/he have long-term ssh sessions that must stay up at all costs?
Specification of network configuration is divided into two types of profiles. The first is applied at the link++ and network++ stages of a link's life. It is made up of the conditions and attributes required to determine which links should be used and how those links should be configured. This will be referred to as the Link-Layer Profile.
The second is applied at the ip++ stage of a link's life, though not every ip++ or ip~-- transition will result in a configuration change. It is made up of the conditions and attributes required to configure higher-layer aspects of a link: things like name services, firewall rules, or proxies. This part of the configuration is not dependent on how IP connectivity is achieved, just that it exists on some set of links. This will be referred to as the Upper-Layer Profile.
Links are configured individually, as link++ and network++ transitions occur (see §2.2). A Link-Layer profile contains attributes used to configure a link. A rule-set will determine link/profile mappings; some of the types of mappings that should be allowed are:
The following attributes make up a Link-Layer profile:
A set of these properties and resources is applied to the system once IP services are available (i.e. IP addresses are configured on running interfaces). Unlike Link-Layer profiles, only one Upper-Layer profile may be active at any time. Upper-Layer profile selection criteria include:
The following attributes make up an Upper-Layer profile:
An issue which has come up repeatedly during design discussions has been that of "user intent". For example, laptops are used very differently than servers and test servers may be used very differently than production servers. So a knob to indicate this intent seems like a good idea. The form this knob may take needs to be worked out.
This section might also be called "how we interact with SMF".
There will be two system network states, each of which is represented by an SMF milestone service:
milestone/name-services should depend on milestone/network.
The profile daemon (profiled) will manipulate these two milestones. When network events happen or the active profile changes (via any of the mechanisms described in §2.2 and §2.3), profiled will decide the state to which each of these milestones should be set. When conditions change sufficiently (again, see §2.2 and §2.3), milestone/network will likely need to be refreshed. Network services should either depend on milestone/network or milestone/name-services. We need to identify what the complete lists are, and update their dependencies accordingly.
A new SMF service, network/profile, will start profiled. It should only depend on network/pfil[1] and should be started very early. The current network configuration services, network/physical, network/initial and network/service should be removed and most of their tasks[2] taken over by the profile daemon.
Based on the information from the active profile, the profile daemon may also enable or disable some name services. For example, when the system migrates from a profile which specifies using DNS to a profile which specifies using NIS, profiled should disable dns/client and enable nis/client.
| [1] | network/pfil is an existing SMF network service. It autopushes pfil onto the filtering interface and restricts network traffic during startup. It does not depend on anything. |
| [2] | Some of the pieces, such as ipqos and IPsec, need to be split out into separate services. More details of this will be worked out as we migrate from architecture to design. The IPsec part is covered by Sun bug 6185380; Sun bug 5105194 covers the rest. |
The Sparks project has just gone live. We are in contact with that project team and in fact several of their members are subscribed to our nwam-discuss list, so as their work progresses, our plans will evolve to match it, and this section will be updated accordingly.
Although improving diagnosability of network problems is not one of our requirements, everyone agrees that we should at least not make this problem worse. Our plans are not yet more defined than using the LOG_DAEMON facility and syslogging various messages at various severities so that curious users can understand what profiled does and why, but these plans should become more refined as we progress from architecture to design.
A great deal of discussion has occurred regarding whether or not profiled should be a delegated network restarter [see smf(5), smf_restarter(5), smf_method(5), svc.startd(1M) and inetd(1M) for what this means]. At present, our intention is that profiled will not be such a restarter, as we do not see sufficient upside to match the complexity downside. But this may change as we move into design and prototyping.
In order to tie the NWAM architecture into the overall system we need to specify what other subsystems it interfaces with and what requirements it drives on those systems.
Bonjour is a zero configuration technology developed and made popular by Apple. Gnome has support via Avahi. Overall this set of features is called Service Discovery. Its main intersection with profiles will be in the assignment of a hostname. The system will keep a set of aliases including a user provided hostname, a service discovery provided hostname, and a DHCP provided hostname allowing any to refer to the current node. NWAM will have to be able to detect when a name changes in order to be able to allow the user to propagate that to interested applications.
Another aspect of configuration is how we deal with the plethora of virtualization technologies being developed. Virtualizations technologies include:
Currently the administrative model of zones is restrictive. Anything that involves configuration has to be done in the global zone. Additionally neither sysevents nor routing sockets (information sources) work in zones. That drives NWAM to be mostly in the global zone. Future plans might change zones to be able to administer resources given to them and to receive various events. In this case a profile daemon could exist in a zone providing independent management of its configuration.
HAL is a schema and API for accessing information about the system. Initially it was envisioned as the layer Gnome needed to abstract the hardware. Applications of HAL are as diverse as the battery status meter and a network manager.
There is an effort (to be documented soon on the OpenSolaris web site) to get HAL into Solaris for use in volume management. HAL uses DBUS as its IPC mechanism. There are system specific back-ends and an upper layer which also just work.
HAL's schema provides support for link-layer devices. While we could add entries to the HAL schema in order to meet our needs, it is not clear that this would be efficient. We have decided not to rely on HAL as a source of information for NWAM.
Visual Panels are a new project aimed at creating a better way to configure Solaris. The current demo includes a hook for Network Profiles. Our UI design will have to reflect the need to integrate with Visual Panels.
Sun's Predictive Self-Healing features in Solaris are represented by Fault Management (FMA) and the Service Manager (SMF; section 5 of this document). These form an important part of Solaris' diagnosability story.
With SMF we investigated various levels of interface. These broke down to a choice between a fairly coarse / high-level manipulation of SMF objects or a finer / lower-level modeling of the system in SMF objects and subsequently using SMF to drive many of the state changes. With a more detailed model of the system in SMF we could use the diagnosability features of SMF (svcs -x) to help users determine what is wrong with their system. Initially we thought modeling an interface per SMF instance would not be possible so we discussed using properties to store interface information. But upon discussion with the SMF team we discovered that such modeling had been considered when SMF was designed.
The next question was to see what value we could give to our users by implementing a finer SMF model. There were 3 classes of errors we discussed:
The first case is already covered by much of what we have. Most services are managed by SMF. Even in our coarser model those services would still be managed by SMF.
The second class could be modeled in SMF. We could not come up with any examples where modeling in SMF helped diagnose problems any better than the current mechanisms.
The third class is the interesting one. There are many tough problems in this DNS example alone, from trying to figure out any of the problems which might have come from disruptions in network connectivity outside of the user's network (e.g. making the server unreachable), to remote DNS server problems (e.g. making response slow or non-existent for some subset of the namespace). Due to the way that most modern networking is designed and specifically how IP works, these kinds of problems do not trace back to an interface in any useful manner. From having a single interface onto a backbone of many paths to having redundant and/or independent interfaces onto the network, the interface that sends out the request is not related to most of the problems those packets can encounter. Instead the model would really need to contain objects which are external to the box (routers, servers, services). This problem does not seem to be tractable.
Given that the current model suffices for NWAM and we could not find interesting diagnosibility problems that a finer model would help the user solve, we decided to use the coarser model with SMF.
FMA models system components so that faults can be centrally managed. The control mechanism for FMA is provided by fmd(1M). One way to do this for the networking subsystem would be putting the MAC layers in a hardware class (hc:/...) and having administrator named data-link objects to which they map. Tools would be provided to set up relationships and control these devices. Clearview provides much of this functionality other than the FMA objects.
In attempting to find uses for MAC level FMA objects we run into the same issues we did with SMF. The interesting problem are not caused by local objects (e.g. interfaces) but instead by things on the network. At this time we do not think building MAC level FMA functionality would add to this project.
NWAM will interact with install by providing tools that install can use to modify individual system parameters (e.g. hostname) and to configure basic networking. A document discussing the install strategy is available here.
| Revision | Date | Changes |
|---|---|---|
| 0.1 | 2006-Feb-13 | initial draft |
| 0.1.1 | 2006-Feb-15 | minor clarifications |
| 0.1.2 | 2006-Feb-15 | minor clarifications |
| 0.2 | 2006-Feb-17 | Section 1 organized & clarified, conditions introduced |
| 0.2.1 | 2006-Feb-23 | Section 2 organized & clarified, other minor clarifications |
| 0.2.2 | 2006-Feb-24 | Section 3 organized & clarified |
| 0.2.3 | 2006-Feb-27 | Appendix A added, ¶ added to §3.1, §3.5 moved to §6.5 |
| 0.2.4 | 2006-Mar-06 | §3.1 and §3.2 updated, § 3.5 added |
| 0.2.5 | 2006-Mar-09 | §6.1 and §6.2 updated |
| 0.3 | 2006-Mar-20 | Section 5 organized & clarified |
| 0.3.1 | 2006-Mar-28 | Modified §3.2 & §3.5; added §3.6 |
| 0.3.2 | 2006-Mar-28 | Added Section 7 |
| 0.3.3 | 2006-Mar-28 | Modified §6.1 |
| 0.3.4 | 2006-Mar-28 | Modified §6.2 |
| 0.3.5 | 2006-Mar-29 | Modified §6.3 |
| 0.3.6 | 2006-Mar-29 | Added §6.6 |
| 0.3.7 | 2006-Mar-29 | Modified §6.4 |
| 0.4 | 2006-Mar-29 | Significantly modified §2.2 |
| 0.4.1 | 2006-Mar-30 | Modified §6.2, §6.6 and Section 7 |
| 0.4.2 | 2006-Mar-31 | Glossary moved from Section 7 to Appendix A Revision History moved from Appendix A to B |
| 0.5 | 2006-Mar-31 | Significantly modified §1.1, §1.3, §1.4, §2.1, §2.3 & §4 Link-Layer and Upper-Layer profiles introduced |
| 0.5.1 | 2006-Apr-03 | Significantly modified §6.5 Changed § name from FMA to Predictive Self-Healing |
| 1.0 | 2006-Apr-03 | Architecture complete |
| 1.0.1 | 2006-Apr-04 | Revised §5.3 per Sparks project going live |
| 1.0.2 | 2006-Apr-05 | Revised §6.5 Dave B's comments on Predictive Self Healing |
| 1.0.3 | 2006-Apr-05 | Added introduction to §4.1 |
| 1.0.4 | 2006-Apr-18 | Changed Section 4 name from Configuration to Profiles |
| 1.0.5 | 2006-Jul-27 | Added Renee's blog URL to team list |
| 1.0.6 | 2006-Aug-30 | Moved Glossary to its own page |
| 2.0 | 2007-Feb-16 | Section 0 added to point out differences between Architecture & Design |
Terms of Use
|
Privacy
|
Trademarks
|
Copyright Policy
|
Site Guidelines
|
Site Map
|
Help
Your use of this web site or any of its content or software indicates your agreement to be bound by these Terms of Use.
© 2012, Oracle Corporation and/or its affiliates.