Wiki code for Development Process

Hide Line numbers
1:  Last updated 29 October 2008.
2:  Earlier 2005 draft also available as [[PDF>>attach:d-devproc-alpha.pdf]].
3: 
4: = OpenSolaris Development Process
5: 
6: == Version 1 [DRAFT]
7: 
8: // John Beck, Rich Teer, Al Hopper, David Comay, Stephen Hahn, Ed Hunter, Joe Kowalski, Keith Wesolowski, Casper Dik, and Bill Sommerfeld //
9: 
10: **Abstract.** This document outlines the OpenSolaris development process: its intent, its general attributes, and its high-level process. The discussion identifies specific changes to effect the extension of existing processes to collaborative development. These recommendations are intended to give all contributors an equitable opportunity to participate in OpenSolaris development without regard for familiarity with or access to existing Sun tools and processes, and to preserve and strengthen existing technical standards. The discussion intentionally avoids addressing implementation issues; the process is independent of the procedures used to select individuals for specific roles and the tools and infrastructure implementing each process step.
11: 
12: = About this Document
13: 
14: We have organized the materials in this document from general to specific. If you are seeking a broad understanding of OpenSolaris and its development processes, you need only read the introductory materials. Step-by-step process overviews follow in subsequent sections, with appendices containing deeper details relevant to particular steps in the process. The specific, detailed steps—and the tools involved—are covered in the [[OpenSolaris Developer’s Guide>>Community Group on.devref_toc]]. In all cases, the processes are described from the implementer’s point of view; however, responsibilities of individuals acting in other roles are also described.
15: 
16: The [[glossary>>#glossary]] contains a list of definitions for commonly-used terms. These terms have a wide array of inconsistent and even conflicting uses throughout the software engineering industry and in various Open Source communities. The definitions are intended to provide an anchor for consistency throughout the rest of the document.
17: 
18: The first chapter, [["Fundamentals">>#fundamentals]], provides background information, and the rationale for a rigorous and methodical development process. We then describe at a high level the scope and purpose of the OpenSolaris co-development effort, and the relationship between the Solaris Operating System and that effort. Finally, we present a set of high-level technical design and process criteria which have been established in the OpenSolaris code base over a period of years. Design and implementation of software consistent with these principles is the key purpose of OpenSolaris development.
19: 
20: Following the background information, we describe the processes involved in managing OpenSolaris code bases. This chapter, [["Release Management">>#release-mgmt]], includes release naming and numbering and change management rules. Additionally, we describe branch management strategy and release criteria. This discussion is intended to be independent of any particular source code management regime.
21: 
22: In the third chapter, [["Development Process">>#development-process]], we present process flows and narratives from the perspective of an implementer. This process closely mirrors the Software Development Framework (SDF) as currently implemented within the Solaris organization at Sun, modified to incorporate contributions by individuals and teams working without continuous and direct access to one another. It is therefore intended to be appropriate to development dispersed both geographically and organizationally. Additional detail is provided in the areas of testing and quality assurance for contributed implementations.
23: 
24: The [[appendices>>#backmatter]] offer detailed information about each of the steps in the development process, and are referenced liberally throughout the rest of the document.
25: 
26: = Fundamentals
27: 
28: This chapter provides an overview of what OpenSolaris is, its relationship to the Solaris product from Sun Microsystems, Inc and a description of the design principles to be used by contributors who wish to provide changes back into a release branch of an OpenSolaris consolidation. Finally, it describes a set of principles to be applied when making changes to the development process itself.
29: 
30: == The OpenSolaris Project
31: 
32: //Note: the following section indicates our original thinking when drafting all these ideas in the summer-fall of 2005. The subsequent section indicates our evolved plans as of October 2008.//
33: 
34: Original 2005 Thoughts|
35: What is the OpenSolaris project?
36: >The OpenSolaris project is an open source project which is initially based on the source code for the Solaris Operating System. It is a nexus for a community development effort where contributors from Sun and elsewhere can collaborate on developing and improving operating system technology. The OpenSolaris source code will find a variety of uses, including being the basis for future versions of the Solaris OS product, other operating system projects, third-party products and distributions of interest to the community. The OpenSolaris project is currently sponsored by Sun Microsystems, Inc.
37: Initially, the OpenSolaris project will provide the core kernel, libraries and commands that are currently distributed with the Solaris OS. Over time, additional parts of the Solaris OS will be made available through the project.
38: How does the OpenSolaris project differ from the Solaris Operating System?
39: >The OpenSolaris project does not provide an end-user product or complete distribution but rather an open source code base, as well as build tools necessary for developing with the code and a infrastructure for communicating and sharing related information. Any support for the code will be provided by the community; Sun offers no formal support for the OpenSolaris product in either source or binary form.
40: The Solaris OS is Sun’s operating system distribution and is branded, maintained and supported as a Sun product. Future releases of the Solaris operating system will be built from the OpenSolaris source code, but will still be supported in the same manner as current versions of the Solaris OS. At any given time, there may be some software in either the OpenSolaris project or the Solaris OS product which is not present in the other. However the intent is to release, over time, as much of the existing source code as possible through the OpenSolaris project and for future development to take place in the OpenSolaris community.
41: Evolved 2008 Plan|
42: What is OpenSolaris?
43:  OpenSolaris is an operating system (OS), a source base, and a community. The project’s goals are innovation, collaboration, and the extension of OpenSolaris technology. Below are key OpenSolaris-related technologies:
44: ; OpenSolaris Source Code
45: : This is the source base for open development. It consists of several components called consolidations. See the [[downloads page>>Main.downloads]] for the technologies released and the [[roadmap>>Main.roadmap]] for future releases. At present, the OpenSolaris source base is not enough to bootstrap an entire system, so developers start by downloading an OpenSolaris distribution and installing the OpenSolaris bits on top.
46: ; OpenSolaris 2008.05
47: : This is a community-developed [[binary distribution>>http://www.opensolaris.com/]] of an operating system based on the OpenSolaris source code, and it’s the first release from [[Project Indiana>> /os/project/indiana/]] It runs on Intel and AMD processors on the server and desktop. It is free to use, modify, and redistribute. Support is available from Sun Microsystems.
48: ; Solaris Express Community Edition
49: : This is Sun’s unsupported binary release of OpenSolaris plus additional technology not released as source. The release is also known as Nevada and it is updated every two weeks, available as a free download.
50: ; Solaris OS
51: : This is Sun’s fully supported and tested enterprise operating system, and future versions of Solaris will be based on technology from the OpenSolaris project. Solaris is available as a free binary download, and Sun offers service packages and regular updates.
52: 
53: == Design Principles
54: 
55: The source code which forms the basis of OpenSolaris has been developed for many years using a set of design principles and core values which have been applied earnestly in order to provide users with a rich, coherent and stable platform for developing and running applications. The attributes of this platform and the technologies that make it up are a consequence of these design principles.
56: 
57: Software is inherently something which evolves over time and improvements are almost always possible. Indeed, there are parts of OpenSolaris today which do not embody these principles to the extent that they should. Nevertheless, these principles are essential to the character of OpenSolaris and contributors are expected to operate with them in mind when making even the smallest changes.
58: 
59: These principles are not unique to OpenSolaris, of course. But they do serve as a cornerstone of first-quality software design. Software is only as usable as how it fits with the rest of the operating system and with the user’s expectations of that operating system. For example, new functionality that is delivered rapidly but with no thought on how it will be maintained is likely to be software that does not evolve and eventually falls into disrepair. Another example is some new feature that is novel in its own right, but which cannot be observed or debugged. Such software presents a larger maintenance burden, not only for its original author but also for every other contributor or user who comes in contact with it at some point.
60: 
61: Reliability
62: 
63: >Whether run on a single-processor laptop or a mainframe-class, multi-processor system, the operating system must be reliable above all else. This means that OpenSolaris must perform correctly, providing accurate results with no data loss or corruption. In the event of a failure to perform correctly, an infrastructure must be in place to determine the failure’s root cause the first time it is encountered. This is so that both the problem in the software can be remedied and also so the first failure does not cause additional ones to take place.
64: 
65: Availability
66: 
67: >OpenSolaris must provide a highly available system to the fullest extent possible without compromising its reliability. As such, the operating system must be robust in dealing with software failures, and if possible, hardware failures. Services must be designed to be restartable in the event of an application failure and OpenSolaris itself must be able to recover from non-fatal hardware failures. Whenever possible, OpenSolaris must not require a reboot in order to bring a new service online, to take one offline or to repair a software or hardware component.
68: 
69: Serviceability
70: 
71: >Since problems may exist in every level of the software stack as well at various hardware levels, it is essential that the operating system provides the ability to diagnose problems. New functionality must provide a way to observe what is taking place in both a running system and in a post-mortem crash dump, preferably leveraging existing observability mechanisms. OpenSolaris must not require reproducing the problem somewhere else or require instrumented software to diagnose a problem but rather one must be able to diagnose arbitrary problems in a production environment. It must be possible to diagnose both fatal and transient issues and wherever possible, automate the diagnosis. In the cases where diagnosis cannot be automated, then the methodologies to be used must be clearly documented so that others can determine the nature of the issue.
72: 
73: Security
74: 
75: >In an increasingly hostile computing world, OpenSolaris security must be designed into the operating system, with a critical eye cast on every component and subsystem. The operating system must be secure, out of the box, with mechanisms in place in order to audit changes done to the system and by whom. In the event of a security breach, OpenSolaris must be designed to minimize the exposure as much as possible.
76: 
77: Performance
78: 
79: >While getting the answer correct is a constraint for an operating system, getting it fast is a goal. That said, the performance of OpenSolaris must be second to none when compared to other operating systems running on identical environments. The operating system must not degrade when faced with additional load and it must scale with the hardware resources available on the system. OpenSolaris must allow for deterministic latency in order to support real-time applications
80: 
81: Manageability
82: 
83: >As operating systems and the hardware that run on them become more complex, managing them efficiently becomes a high priority. OpenSolaris must provide powerful abstractions which simplify the management of the system, and not add to its complexity. It must allow for the management of individual components, software or hardware, in a consistent and straightforward manner.
84: 
85: Compatibility
86: 
87: >At each level of the software stack, compatibility over time must be considered a constraint and not just a goal. OpenSolaris interfaces must be designed with a documented commitment level and changes to those interfaces must be designed to maintain compatibility as committed to. New subsystems and interfaces must be extensible and versioned in order to allow for future enhancements and changes without sacrificing compatibility. When there is a change in compatibility of an interface (including in the case when the interface is removed), there must be formal communication of this change to the community.
88: 
89: Maintainability
90: 
91: >Software is rarely a collection of static algorithms, unchanging over time. Rather, software is something which continues to evolve as additional needs and requirements are discovered. OpenSolaris frameworks must be designed to be maintainable over time, particularly by a large development community. OpenSolaris must be architected so that common subroutines are combined into libraries or kernel modules that can be used by an arbitrary number of consumers. OpenSolaris must be well documented, through the effective use of comments within the source code as well as in other places such as design documents.
92: 
93: Platform Neutrality
94: 
95: >OpenSolaris is built from a single source base and in almost all cases, capabilities and feature are equivalent across all platforms. OpenSolaris must continue to be platform neutral and lower level abstractions must be designed with multiple and future platforms in mind.
96: 
97: == Process Principles
98: 
99: Contributing to OpenSolaris is more than just understanding the design principles of the project. There are also development processes in place to steer, encourage and ultimately enforce future development to adhere to the stated design principles. These development processes must be understood by contributors to the project.
100: 
101: The development processes themselves were derived with a number of values in mind including being able to scale from a project spanning many different layers and subsystems down to a simple fix that addresses a very localized defect. Each of these values or process principles is meant to guide the creation of any new processes, with the overriding goal of the continuing evolution of OpenSolaris, developed as efficiently as possible, but with the highest quality in mind.
102: 
103: Open
104: 
105: >OpenSolaris clearly benefits from having many contributors using, learning and eventually having the opportunity to change it. Therefore, OpenSolaris must be developed out in the open using processes that are transparent and which allow participation at all levels.
106: 
107: Shrink To Fit
108: 
109: >Although each step of the OpenSolaris development process has an important role to play in general, certain steps may not be applicable for certain types of proposed changes or the scope of a step may be excessive. Processes must be designed in such a way that the time and materials required to complete a step can be reduced, optimized or even eliminated when the full step is not required, as long as the quality of a consolidation’s release branch is not compromised or lowered.
110: 
111: Fair
112: 
113: >The development process must be structured such that it can be administered equitably across the whole contributor section of community. Variations on the scope of a particular step in the process and what is being asked of a contributor must be based not on the contributor but on what is being contributed.
114: 
115: Deliberate Intent
116: 
117: >Each step in the OpenSolaris development process is meant to address a very specific and real requirement for contributing changes into a consolidation. As such, steps in the process must be developed with specific intent in mind. Process steps must not be added or changed in a arbitrary fashion but must be justifiable based on real data that has been obtained and analyzed to determine how best to address a specific issue with the current process.
118: 
119: == Policies
120: 
121: All OpenSolaris consolidations must conform to the policies below unless a specific exception has been granted by the Governing Board or its designate and documented in the meeting minutes or an approved fast track. These policies each distill the positions of many stakeholders into a minimal requirement for project teams and consolidations in each of the technical areas. It is expected that consolidations, individually or in concert, will reiterate these policies in greater detail for the benefit of contributors.
122: 
123: With respect to external code bases, such as open source software generated by third parties, consolidations may choose to relax certain of these policies. For instance, the ON consolidation does not expect integrated open source software to meet the internationalization policy requirements.
124: 
125: Accessibility
126: 
127: >All OpenSolaris software shall conform to the U.S. Government’s section 508 accessibility requirements.
128: 
129: Architecture Review Committee (ARC)
130: 
131: >All software integrated into OpenSolaris shall conform to the OpenSolaris interface taxonomy.
132: 
133: Architecture Neutral
134: 
135: >All generic (i.e., hardware independent) code shall be tested and run on all supported OpenSolaris target platforms. Prior to integration, there must be a commitment to keep the code functional and the resources to do so.
136: 
137: End of Feature (EOF)
138: 
139: >User-visible functionality can be removed from OpenSolaris minor releases subject to (1) Governing Board (or its designate) and ARC approval of public notification for a defined period of time prior to actual removal, and (2) Governing Board (or its designate) and ARC approval of actual removal. User-visible functionality usually has a commitment level of Unstable or higher. Functionality visible only to specific groups (e.g. contracted or partner private) can be removed subject to ARC approval and the time frame may be decided with the specific group.
140: 
141: Internationalization (I18N)/Localization (L10N)
142: 
143: >All OpenSolaris software shall be internationalized, meaning that the software will call supporting interfaces to allow the replacement of messages, numeric and date formats, and similar data representations with those consistent for the user’s locale. The provision of localized data representations is welcomed, but not expected to be contributed to the consolidation by the original code contributor.
144: 
145: IPv6
146: 
147: >All OpenSolaris software supporting IPv4 shall also support IPv6 or have a committed and resourced plan for supporting IPv6 in a reasonable time frame.
148: 
149: Consolidation Bug Policy
150: 
151: >Contributors are expected to resolve all bugs in new features prior to publishing their changes through the release branch of a consolidation. Resolution is typically defined by the consolidation. A consolidation may state that no projects may integrate with high priority bugs unfixed, for example. Exceptions may be granted by the team managing the consolidation, acting on behalf of the Governing Board. Such exceptions are expected to be documented, such that consolidations establish precedents and become predictable bodies.
152: 
153: Licensing of contributions
154: 
155: >All contributed software must be freely modifiable and redistributable, licensed under the Common Development and Distribution License (CDDL) license or a license that is compatible with CDDL.
156: 
157: = Release Management
158: 
159: == Release Types
160: 
161: The core OpenSolaris value of compatibility and the processes designed to achieve that goal may seem daunting and a hindrance to innovation. This section is intended to provide a very high level overview of the processes and what can be done within them. Pointers to reference documents will be provided for those interested in the details.
162: 
163: One should first note that Solaris 10 is a very different beast than Solaris 2.0 was. All the innovation which appeared in the interim was accomplished within the same compatibility constraints in place today.
164: 
165: Indeed, these compatibility constraints can serve as a facilitator for innovation, rather than a hindrance. This is referred to as "the Freedom of Constraints": If the interfaces upon which your project depends are constrained, you are free to concentrate on developing your project rather than constantly adapting it to a changing environment.
166: 
167: To understand and work within the processes designed to facilitate compatibility, one needs to understand a few basic concepts.
168: 
169: It is common throughout the industry to classify product releases as Major, Minor or Micro and reflect this in the "dot" notation we are all familiar with. It is also common throughout the industry that these terms imply little more than a value judgment by someone as to whether a release is small, medium or large. It is just marketing splash.
170: 
171: In Solaris, and now OpenSolaris, additional constraints are applied as to which interfaces can change incompatibly in a given type of release. In a Major release, any interface might change incompatibly. In a Minor release, most interfaces will not change incompatibly and in a Micro release even fewer can change incompatibly. Which interfaces are which is made clear by assigning an Interface Taxonomy (reference) level to each interface. More on Interface Taxonomy levels later.
172: 
173: Note that in the second paragraph, the references are "Solaris 2.0" and "Solaris 10". "Solaris 10" would actually be "Solaris 2.10" if one expects the common Major.Minor notation. However, since Sun made the decision to not do another Major release of Solaris for the foreseeable future, the ’2’ became silent. Note further, that the output of {{code}}uname -r{{/code}} yields the full Major.Minor[.Micro] release number of the SunOS component (consolidation). The dropping of the ’2’ at the top level reflects the commitment to the core value of compatibility and it is expected that OpenSolaris will have the same commitment. A Minor release of Solaris and OpenSolaris has the meaning of "Big, cool and compatible".
174: 
175: This brings us to the topic of interface stability and the Interface Taxonomy. //The Interface Taxonomy document is in the process of being modified.// The intent is to simplify it, partly in response to the demands of working in an open environment. Although that document is not yet available in its final form, the substantive changes are known and the following overview reflects those changes.
176: 
177: The Interface Taxonomy divides interfaces into two broad categories; Public and Private. Public interfaces are available for anybody to use while the supported use of Private interfaces is limited to a subset of possible consumers. There are several subclasses of both Public and Private interfaces. For Public interface these subclasses reflect in what type of release the interface may change incompatibly. For Private interfaces the subclasses reflect the domain of supported users.
178: 
179: It’s important to emphasize that Private does not mean secret and conversely visible does not imply Public. Indeed, many Private interfaces are easily discovered by simply examining the header files.
180: 
181: The following are the (new) relevant Public taxonomy levels:
182: 
183: >//Stable:// The interface may only change incompatibly in a Major release. Interfaces intended for usage by ISVs and system integrators require this level of support to be useful to these customers.
184: //Uncommitted:// May only change incompatibly in a Major or Minor release. This is appropriate for some system management facilities and new, untested interfaces.
185: //Volatile:// May change in any release vehicle. This usually only useful when the interface definition is controlled by a body other than the OpenSolaris community and it is viewed that tracking that community is more important than interface compatibility.
186: //Not an Interface:// Just a convenience term to label what may appear to a supported interface as not being supported. This is the default classification and this term is only explicitly applied when there is likely to be confusion.
187: 
188: //The precise terms for the public taxonomy levels is still under discussion, as part of the update to the Interface Taxonomy document.//
189: 
190: Note that the ability to make an incompatible change in a given release vehicle does not make that a requirement. For example, most interfaces controlled by someone other than the OpenSolaris community are currently classified as Volatile, but synchronization with major incompatibilities introduced by those communities is often deferred until a Minor release is available.
191: 
192: Exceptions are made to these rules, but they require an exceptionally good reason. The three generally accepted reasons are:
193: 
194: 1. A supported standard (which is branded) has been reinterpreted and the existing implementation no longer conforms.
195: 1. Security hole inherent in the interface definition.
196: 1. Data corruption inherent in the interface definition.
197: 
198: Other reasons may be accepted, on a case by case basis. For example, we broke compatibility in {{code}}pcfs{{/code}} (with respect to case sensitivity) when Microsoft abandoned the 8.3 DOS convention. The world would have thought us daft if we had not.
199: 
200: It should be noted, that OpenSolaris (and particularly the core ON/SunOS consolidation) make little use of the Uncommitted taxonomy level; it is used primarily by other Sun products which work in a less compatibility constrained market. However, its use is expected to increase for two reasons:
201: 
202: * Those "other Sun products" are tending to become part of OpenSolaris. (This doesn’t affect the core ON/SunOS consolidation.)
203: * Interfaces currently classified as Volatile are being promoted to Uncommitted. As use of these interfaces increases, compatibility across Micro releases now trumps tracking the external communities at a rapid pace. (This does affect the core ON/SunOS consolidation.)
204: 
205: The following are the relevant Private taxonomy levels:
206: 
207: >//Project Private:// May only be consumed by the project which provides them. This is the default level for Private interfaces and formal review is not required. However, if the intent is to make them more available later, review can be useful. (OpenSolaris may regard "Project" as roughly equivalent to "Community".)
208: //Consolidation Private:// May only be consumed within the consolidation which contains the project which provides them.
209: //OpenSolaris Private:// (Internal to Sun, this is known as Sun Private.) May be used by any component of OpenSolaris. It should be noted that this is often a poor choice due to the coordination efforts required should an incompatible change be produced.
210: 
211: How does this affect the OpenSolaris developer?
212: 
213: On the very positive side, you can make informed decisions as to what interfaces you can consume (or even if you are allowed to consume them). This gives you tremendous control over the impact other projects can have on your project. The benefit of this over time can’t be understated even though it might seem to be a short term inconvenience.
214: 
215: On the negative side, you may have to jump through a few hoops when needing to modify an interface provided by your project to maintain compatibility. The classic UNIX example is the {{code}}wait{{/code}}/{{code}}wait3{{/code}}/{{code}}wait4{{/code}}/{{code}}waitpid{{/code}}/... set of routines; as the required semantics evolved, the older routines could not be evolved to meet the requirements, so new routines were invented while the old routines remain available, compatible and supported. One should note that such extreme examples tend to be rare. A more common consequence is that you need to evolve interfaces in ways that are slightly less elegant than might be if starting with a completely blank piece of paper. The word "slightly" is key in the above.
216: 
217: == Branch Management
218: 
219: OpenSolaris governing policies presume the existence of multiple concurrent lines of development. A given line of development is associated with some codebase, typically but not always a consolidation. Different consolidations may have related lines of development but need not complete their releases at the same time. These assumptions hold for the existing Solaris development model and many open source development efforts.
220: 
221: The terminology used to describe parallel development often depends on the particular source code control model in use; however, underlying work flows tend to be similar. To avoid confusion, see the [[glossary>>#glossary]] for definitions of terms.
222: 
223: Before a consolidation makes its current release final (the last official snapshot of that release is delivered), the trunk will split into two release branches: the trunk, on which development will be integrated for the next release, and the current release’s branch, on which the current release will be completed and made final. At the time of splitting, any version information in the source base will be modified in the trunk to reflect the next release and left unchanged in the previous release branch. A release team is associated with a versioned release; that is, this step may be seen as the creation of a new trunk with a new release team. The existing release team remains responsible for the previous release branch until it is made final.
224: 
225: All release branches are subject to the [[quality criteria outlined later in this document>>#quality-criteria]]. Despite this, the release team does not bless arbitrary snapshots selected at random. While these snapshots can be used by distributions, only the final snapshot within a release is considered the official publication of the consolidation’s contents. Often, given human fallibility, there will be some number of defects in any incomplete release branch which should not be included in an official release. Therefore, when a release is being finalized it is necessary to reduce the rate of change so those defects can be squeezed out.
226: 
227: On any branch, in order to manage risk, the release team has the the responsibility and the authority to require that projects integrate only when both the project is ready to integrate and the branch is ready to receive the project. This may mean that a project targeting integration late in a release may be deferred until the next release opens.
228: 
229: Exactly when to "split" in the development cycle is a matter of balancing conflicting concerns; splitting near the beginning of the new release increases the period of time during which two Minor release branches are in active development, which divides resources and requires more merging. Splitting near the time when the release is to be completed usually implies that the next release does not open until the previous one is completed, increasing delays in project integration.
230: 
231:  [[image:p-branches.png||alt="p-branches.png"]]
232: 
233: //Active branches and equivalent releases.//
234: 
235: The trunks of all consolidations officially comprising OpenSolaris will normally operate under Minor release binding rules at all times, and under the control of their respective release teams.
236: 
237: After the split, the pending release branch continues under Minor release binding rules until the release is completed, but integrations should occur at a reduced rate. New development should be targeted at the next release, and the release team for the pending release may refuse integrations for risk management reasons.
238: 
239: Optionally, branches may be created from a final release and opened to continuing development. If this occurs, the children are update releases. Each update release has its own release team and operates under Micro release binding rules. Although each update release team may establish its own integration criteria, in general all changes targeting an update release must first integrate into the trunk, and must be backported (rather than merged) to the update release branch. Each of these steps requires independent testing. Most update release teams will require some amount of "soak time" in the trunk; the exact amount of soak time required may depend on the type or scope of change. The release team always has the authority to impose any additional requirements it deems appropriate.
240: 
241: Note that in the common case most of the materials generated for a change’s integration into the trunk can be reused during the process of integrating into an update. A central concern of the update integration process is ensuring that the change is appropriate for the update release.
242: 
243: == Release Bindings
244: 
245: The release team defines the appropriate release binding for changes to its branch. This is typically Micro for an update release branch and Minor for all other release branches. A project branch normally operates under the same release binding rules as the release branch to which it is targeted; project teams not targeting integration may apply any release binding they wish to their branch(es).
246: 
247: As the definition of release implies, compatibility rules are not in effect within a release, only between a completed release and its completed parent release. These rules do not apply from one snapshot of a release to the next. It’s worth noting, however, that some defects are most easily fixed with an incompatible change. If a new interface is integrated into a release branch and then backported to another release branch before a defect of this type is discovered, getting the same incompatible change made to both branches before either one is completed will require substantial coordination.
248: 
249: == Quality Criteria
250: 
251: OpenSolaris is expected to encompass a number of open source projects which will operate under the OpenSolaris governance model. It is expected that each project will have some freedom in its respective charter which defines how it will operate including in the area of quality assessments and defect management. However, at which time that a project within OpenSolaris wishes to publish its changes via a release branch of a consolidation, including a trunk, the quality of the proposed changes must meet certain quality criteria.
252: 
253: == Regression Commit Criteria
254: 
255: A regression is an unintended change from correct or often, documented or expected behavior. One type of regression is when the operating system behaves less in accordance with its documented and stated specification. These regressions may be as trivial as a new spurious error message bring printed or as serious as the operating system suffering a catastrophic failure. No change made to a release branch should cause a functional regression of the branch from its current state. Such regressions cause all projects based on OpenSolaris to spend time tracking down potential issues with their work, which wastes the time of a larger part of the community than just the contributor who introduced the regression. This in turn also slows down progress on future development. Functional regressions can be prevented in most cases through the execution of comprehensive and rigorous test plans prior to the commitment of a proposed change.
256: 
257: Another type of regression is one in the performance of the system as measured through one or more micro and/or macro benchmarks. Such regressions must be avoided if at all possible and can be prevented through the same testing methodology used to prevent functional regressions. As each release branch will have a set of performance criteria associated with it (which will be part of larger quality criteria for the release), a performance regression caused by a proposed change may be allowed by the release team depending on the criteria of the specific branch and the current state of the release with respect to those criteria.
258: 
259: == Completeness Commit Criteria
260: 
261: Even if a newly introduced change into a release branch does not cause a regression, it is essential that change does what it is expected to do. New functionality must meet the key requirements that were set out by the project team. This can be established by executing against a rigorous test plan which has been reviewed to be comprehensive and complete. Projects which are unable to show they have met their key requirements will not be permitted to publish their changes via the release branch of a consolidation.
262: 
263: In addition, whenever any type of change is made to a release branch, a corresponding change to the existing regression test suites may also be required. This is particularly true when new functionality is added so that future changes, most likely not by the original project, can be tested and shown not to regress the quality of OpenSolaris. Even in the case of fixing existing defects, it is often useful and sometimes will be required to add additional test suites or assertions to existing suites. The key is provide a robust test suite which is available to all OpenSolaris contributors in order to assure to the greatest extent possible that a proposed change is functionally complete and will not cause a regression.
264: 
265: == Summary
266: 
267: The quality requirement of OpenSolaris is perhaps best stated as "Production Ready All The Time". The idea is that at any time, the release branch of a consolidation must be of sufficient quality so that it can be used as-is as the basis of a distribution or product. By using rigorous quality criteria, the community can enforce that OpenSolaris avoids as much as possible the Quality Death Spiral. This occurs when contributors and users alike hear that a branch is broken and as a result, they avoid incorporating the most recent changes into their own project area or even stop using the branch itself. This causes less real-world testing to take place, additional bugs are not discovered and the quality of the branch declines even further. When the Quality Death Spiral occurs, it can be difficult to reverse and the time it takes to recover can be very long.
268: 
269: = Development Process
270: 
271:  The development process flows are too large to fit on a single flow chart so we split it up into four phases: Idea, Design, Implementation and Integration. As mentioned earlier, one process development criterion for OpenSolaris has been "shrink to fit", meaning that various steps in the process are only applied as needed and skipped if not needed. Note also that developers are allowed to "skip ahead" and do steps out of order, but that would significantly increase the risk of rework. Finally, nothing can replace good judgment in deciding what is and is not needed.
272: 
273: [[image:overview-0.1.gif||alt="overview-0.1.gif"]]
274: 
275: Notes on the colors on the charts below, which indicate scale:
276: 
277: * The light gray boxes apply to everything.
278: * The medium blue boxes apply to medium and large changes.
279: * The brighter blue boxes apply to large changes only.
280: 
281: == Idea
282: 
283: [[image:idea-0.3-crop.gif||alt="idea-0.3-crop.gif"]]
284: 
285: First, someone has an idea for an enhancement or has a gripe about a defect. The first thing to be done is to see if a bug/RFE is filed and file one if not. The next thing is to announce it somewhere to precipitate discussion, which should help determine the complexity of the proposed change(s), gauge community interest, and identify potential team members. This is where developers find out if they have the support they will need to move forward. If the set of changes is sufficiently large, a team may need to be formed, a project may need to be defined, and design reviewers may need to be identified. Once these are all done to satisfaction, things progress to the Design phase.
286: 
287: == Design
288: 
289: [[image:design-0.3-crop.gif||alt="design-0.3-crop.gif"]]
290: 
291: The Design phase has several things going on in parallel. The first is whether or not a formal design review is even needed; the general answer is //No// for small RFEs and bug fixes and //Yes// for medium and large RFEs and projects, but as with much of the process, this is a judgment call. If a formal review is needed, reviewers will need to be identified, a design to review created, etc. If needed, architectural review should occur as well. If there are dependencies, convincing the appropriate stake-holders that it is appropriate is needed. A test plan, however minimal or complex, must be created and approved also. And if needed, a schedule detailing resources etc. should be produced.
292: 
293: == Implementation
294: 
295: [[image:implementation-0.3-crop.gif||alt="implementation-0.3-crop.gif"]]
296: 
297: The Implementation phase also has several things going on in parallel. The first and often foremost among these is the writing of the actual code, along with making sure that code adheres to applicable policies, and passes various unit and pre-integration tests. Along with this are writing the documentation and writing the test suites. And in preparation for the Integration phase, code reviewers should be identified.
298: 
299: == Integration
300: 
301: [[image:integration-0.3-crop.gif||alt="integration-0.3-crop.gif"]]
302: 
303: The Integration phase is to make sure everything that was supposed to be done has in fact been done, which means a lot of review, including code, documentation and completeness. Note that the "Review for Completeness" step is conducted by the //Final Approvers//; in the Solaris model, these are the //C-team// and/or the //CRT//, who exactly they are in the OpenSolaris model is not specified, although expected to be acting in an equivalent role for their consolidation. Once all reviews have been completed, and permission to integrate has been granted, integration can occur. Finally, the change needs to be communicated as needed: heads-up and/or flag-day messages to appropriate communities, and possibly a transfer of information to a support organization.
304: 
305: = Glossary
306: 
307: Application Binary Interface (ABI)
308: 
309: >An ABI defines a system interface for compiled programs. This is usually specified as an API coupled with source to binary translation rules.
310: 
311: Application Programing Interface (API)
312: 
313: >An API defines a system interface for source level programs.
314: 
315: Branch
316: 
317: >A line of development within a code base, created from a snapshot of another branch, usually the trunk. Development proceeds independently on separate branches, but merges may occur in either direction in accordance with the branches’ integration rules. There are two general types of branch: release and project.
318: 
319: C-Team
320: 
321: >Consolidation Team: the release team managing a specific Consolidation.
322: 
323: Change Review Team (CRT)
324: 
325: >A change review team is the group of technical contributors that a particular consolidation team may charter to manage the day-to-day business of evaluating the technical risk and merit of proposed changes". Typical evaluations handled by the CRT are the integration requests for fixes for groups of bugs; project integrations are usually managed by the consoldation’s release team.
326: 
327: Command Line Interface (CLI)
328: 
329: >A CLI defines the invocation syntax for command line based utilities.
330: 
331: Community
332: 
333: >A community is a group of people ("members") sharing one or more common interests that is carrying out a conversation; the conversation takes place in one or more known public forums. A community is not expected to have a source repository. It is expected to be interested in one or more of the ongoing projects related to its area of interest and, optionally, the consolidations that publish those projects.
334: 
335: Consolidation
336: 
337: >A consolidation is a technical effort involving one or more people working in an open manner; the specific outcome of a consolidation is to assemble the technical artifacts produced by one or more projects into a coherent collection convenient for assembly into distributions. The consolidation may also include contributions of its own generation, or from other sources. The consolidation has one or more source repositories, which group the source products of the contributing projects and other selected artifacts into a coherent whole. The consolidation’s focus is to be a publication mechanism of an artifact useful to a distribution. Utility to a User is a secondary consideration. The consolidation membership, like a project, is typically composed of contributors.
338: The output of a consolidation is minimally expected to be a source tree, associated tools, and a "proto area", which is the in-file-system binary delivery associated with the compiled output of that source tree. Depending on the expectations of the distributions utilizing the consolidation, it may elect to publish the binaries in a package format.
339: 
340: Contributor (a person)
341: 
342: >A contributor is able to perform work of use to a project or consolidation. The work performed include examples such as the creation or modification of code, the analysis of failures, the management of tools related to a project, the creation of architectural or specification documentation, and even the act of coordinating other contributor actions. The specific notion of a piece of work being contributory or not is left to the project or consolidation to decide. Projects concerned with communications, like an Advocacy Project, might consider an individual’s active presence at a conference or other physical meeting as a contribution.
343: The label of "Core Contributor" is associated with a Contributor who gives substantial direction to a project or consolidation. A Core Contributor would be expected to participate in all voting and dispute mechanisms.
344: 
345: Distribution
346: 
347: >A distribution is a technical effort involving one or more people potentially working in either an open or closed manner; the specific outcome of a distribution is to assemble the technical artifacts produced by one or more consolidations and, optionally, one or more projects into a coherent collection capable of performing a meaningful subset of the set of features typically associated with an application of the given class. The distribution may also include contributions of its own generation, or from other sources. The distribution’s focus is to be a publication mechanism of an artifact intended for a User. The distribution membership is typically composed of contributors, although distributions using a non-open process need not include any actual contributors.
348: The output of a distribution producing a software artifact is minimally expected to be an executable form of that artifact.
349: 
350: Documentation
351: 
352: >Descriptions of interface syntax and semantics intended for human consumption. This may include manual pages, info files, "Guides", README files and other forms. It explicitly does not include header or source files contents.
353: 
354: Fast Track
355: 
356: >A lightweight process for reviewing a proposal by the community. Proposals that are suitable for such a review are typically those that are obvious and non-controversial. Fast track reviews will typically take place over email over a fixed period of time. If the time allotted to the proposal is used up and significant objections have not been raised, the proposal is automatically approved.
357: 
358: Flag day
359: 
360: >A flag day refers to a code integration that affects a large subset of the contributors using the development release (or current version) of a consolidation. For instance, the modification of build tools that requires the deletion of existing object files from a build source tree generally is considered a flag day. Flag days are accompanied by a clarifying message from the integrating contributor, or the C-team on their behalf. The message describes aspects of the change, and the steps required to continue working across the flag day.
361: 
362: Interface
363: 
364: >Merriam-Webster defines Interface as:
365: 2 a : the place at which independent and often unrelated systems meet and act on or communicate with each other. b : the means by which interaction or communication is achieved at an interface.
366: In the context of this document, an interface is limited to the documented syntax and semantics one software object exports or imports from another. This may include (but is not limited to):
367: * Library ABIs (implicitly APIs).
368: * Utility CLIs.
369: * File formats
370: * File system layout (including file locations)
371: An implementation of an interface also has behavioral artifacts, such as performance under certain conditions, which are not specified, and are therefore not part of the "interface". The developers of the implementation do not assume anyone is depending on these undocumented features; a client should not depend on these artifacts (or should get them made an explicit part of the specification). However, the capacity of an interface or its gross performance characteristics could be deemed an implicit part of the interface semantics.
372: 
373: Interface Taxonomy
374: 
375: >A document which defines the terms applied to interface commitment levels and provides the semantic definitions and the rules associating with those levels with release types. These commitment levels are exposed to customers in the Attributes sections of manual pages under the type "Interface Stability",
376: 
377: Member (a person)
378: 
379: >A member participates, potentially only by observing, in the conversation held by the community of which s/he is a member.
380: 
381: Parent/Child Branches
382: 
383: >When a new branch is created, the preexisting branch serving as the initial content source is the parent and the newly created branch is the child. Immediately after branch creation, the contents of the parent and child are identical.
384: 
385: Product
386: 
387: >A Product is a specific publication of a distribution, which can be distinguished at least by the time of publication.
388: 
389: Project
390: 
391: >A project is a technical effort involving one or more people that is working in an open manner; the specific outcome of a project is to produce a small number of coordinated technical artifacts. A project has one or more source repositories, each of which is intended to in some way be published. A project may choose that its primary form of publication is through one or more consolidations, although this is not necessary. A project has at least one distinction between those actively working on the project (contributors) as opposed to people merely interested in the project’s progress (members).
392: 
393: Project Branch
394: 
395: >A line of development used by a project team to coordinate their work. A project branch typically is created from a consolidation’s trunk, receives regular merges from its parent, and is merged back into its parent and deactivated when the project is completed. A project not intended to be published in a consolidation may occupy a separate code base rather than a branch of an existing consolidation. A project may also have a branch in each of several code bases.
396: 
397: Project Team
398: 
399: >All contributors to a project. Project teams are responsible for establishing their own internal governance and development processes, and for deciding what changes are appropriate for their project branch(es) or code base(s).
400: 
401: Proto Area
402: 
403: >A hierarchical file system containing the binary delivery associated with the compiled output of a given source tree. For example, a source tree with a root of {{code}}/path/foo{{/code}} might have a {{code}}proto{{/code}} directory {{code}}/path/foo/proto{{/code}}; everything under this would be referred to as the proto //area//, and if {{code}}/usr/bin/x{{/code}} and {{code}}/etc/x.conf{{/code}} were both part of the binary delivery, then they would be in the proto area at {{code}}/path/foo/proto/usr/bin/x{{/code}} and {{code}}/path/foo/proto/etc/x.conf{{/code}} respectively.
404: 
405: Release Branch
406: 
407: >A branch of a consolidation’s code base in which projects and/or bug fixes are integrated for the purpose of publication. A consolidation may have multiple active release branches at any one time, each employing a different release team and enforcing different integration requirements.
408: 
409: Release
410: 
411: >A snapshot or series of snapshots of a single release branch. Architectural consistency requirements are applied to releases.
412: 
413: Stable
414: 
415: >The most common interface commitment level in OpenSolaris (see Interface Taxonomy). Incompatible change to a Stable interface can only happen in a Major release of the product.
416: Note that this usage is not the same as the common community usage of referring to Stable branches of a development tree.
417: The Solaris {{code}}attributes(5){{/code}} manual page has information on all interface stability levels.
418: 
419: Trunk
420: 
421: >A consolidation’s most current release branch. The trunk is the release branch opened most recently and the one with the highest version number if the consolidation employs versioning. All projects and all applicable bug fixes integrate into the trunk before any other release branch. The specific name of the trunk as seen in the source code management system and whether that name is fixed or changes over time are implementation details, as is any distinction in representation between the trunk and other branches.
422: 
423: User (a person)
424: 
425: >A person who uses one or more distributions, consolidations, or projects in some output form, but does not participate in any of the conversations associated with those efforts. The user may be viewed as a pure consumer of the technology produced by the collection of technical efforts.
426: 
427: = Appendix A. Interface Taxonomy
428: 
429: (Incorporate PSARC/2005/220 once complete and opinion available.)
OpenSolaris

Top Menu

Wiki code for Development Process

Search

Collectives

Community Group

Project

User Group

Subsites

Community Group on Pages
