OpenSolaris

Adding a component into the Web Stack Project Repository

The Web Stack Project Repository is here: http://pkg.opensolaris.org/webstack

If you are interested in adding a new web tier component (and becoming its maintainer), read on!

Summary of Steps

1. Sun legal approvals
2. Create a buildable workspace which can hook into Web Stack builds
3. Create a proposal, discuss with webstack-discuss
4. Request a Web Stack source repository and check it in
5. Iterate between community discussion and working on the build

High-level Detail

1. Sun legal approvals

Unfortunately the first stage of the process can only be done inside Sun as it involves getting various internal approvals for publishing the component. Contact webstack-discuss to request a Sun employee to help process these approvals.

For external contributors, a contributor agreement is also needed.

2. Create a Buildable Workspace

You'll probably start by creating a first cut of the build workspace privately just to explore what it takes to build the component and make it work since you will need some of this information in order to write a first draft proposal.

Strictly speaking the only requirements in order to hook up into the Web Stack build system are as follows:

• There must be a Makefile at the top level of the source tree.
• The Web Stack RE process will set PKG_REPO to point to the relevant IPS repository and call 'make publish'. Thus, your Makefile must have a 'publish' target which will publish the built package into IPS.
• The Web Stack RE process will set PKG_REPO to point to the relevant IPS repository and call 'make publish-src'. Thus, your Makefile must have a 'publish-src' target. Source packages are optional so this can be an empty target or a target which publishes a -source package to IPS.

In practice, while not required, you should most likely take advantage of the webstack-dev-tools package which helps you accomplish this integration with much less manual work. Refer to the Build Details section later in this document for more information.

3. Proposal

Next, create a document describing the details of the package. Important details to include are:

• An introduction/summary of the component being proposed. What is it, what does it do, who will typically use it, some common use cases. This section should be short, but do provide enough information to those who are unfamiliar with the component to have a sense of what is it about.
• File layout - what directories and files will the package deliver where? Which files can the user edit? What files can the component create during runtime? And any other file details...
• Package name(s)
• Configuration details - many components have a number of compile-time configuration options. Which ones will be enabled and/or disabled and why?
• Dependencies - does the new component depend on any interfaces (APIs, file locations, CLIs and such) from any other components? Which ones?
• Interfaces - what interfaces does this new component make available for consumers? How stable do you expect these interfaces to be over time? Read the interface taxonomy for some background on this.
• In short, the proposal should serve as a draft for a possible future ARC case. While we don't require an approved ARC case for Web Stack project repository inclusion you'll want to write it with this purpose in mind because it'll help collect all the important information and if a decision is made to formalize the component later then you'll have the materials ready to go without duplicating effort. Check out some past Web Stack ARC cases for inspiration: Ruby, Apache 2.2.6 update.

The proposal and implementation can be done as an iterative process so don't worry about making a perfect proposal in the first draft. Start with just enough information so others on webstack-discuss can start giving feedback.

4. Repository

Request a source repository to be created for this project. If you have a contact already from step#1, ask the same person. Once it is ready, populate it with your build tree.

5. Iterate...

After the build is stable, we'll add it to the nightly build system which in turn will cause it to show up (a bit later) on the Web Stack project repository. Ask everyone to install the package and try it out and give feedback. Continue improving the integration details until everyone is happy ;-)

Build Details

Note we may have (at least) two kinds of source repositories under Web Stack. This document covers components which only support building for (and on) OpenSolaris. There will be some component repositories which support multi-platform builds because they are also used for the Sun Web Stack product. This will be documented elsewhere.

As mentioned in the previous section, technically the only build requirement is the component must support a 'publish' target and a 'publish-src' target (which may be empty).

But in practice you will want to both conform to the layout and style of other Web Stack components and you'll want to leverage the webstack-dev-tools package if at all possible.

A build system is best described by example, so use existing Web Stack components such as varnish, nginx or others as a template when creating a new one.

Some key points to follow:

• Store the upstream source archive (tarball) as-is at the top level of the source tree.
• Add target to 'extract' the tarball into ./build/$COMPONENTNAME/ dir.
• Add target to 'configure' (if applicable) it and 'build' it. If you must change the source (try to avoid it), create a 'patches' directory and store your patch files (one patch per bug) in there and apply them before building. Never modify the upstream tarball.
• Avoid generating any build-time content outside of the ./build/ directory. This way we can just add 'build' to .hgignore.
• Add a target to 'install' the component into ./build/proto/ directory. The ./build/proto/ dir contains all the filesystem entries delivered by this component. For example if the package will deliver /usr/bin/foo, you'll have ./build/proto/usr/bin/foo. There are two approaches for doing the 'install' target:
  • (Preferred!) Use wsproto tool by creating a template file (./etc/wsproto.data) which specifies the layout and files and package details. The just call wsproto from the 'install' target to do the work. Again, examples help here more than text so take a look at existing Web Stack components. Also, after installing the webstack-dev-tools package ('pkg install webstack-dev-tools') read the comments on /usr/bin/wsproto for the latest info on how to use it.
  • Alternatively, use the 'make install' target provided by most upstream components to install it into ./build/proto/ area. While this may work well most of the time, you'll want to avoid it if at all possible due to the next step.
• Add a target to 'publish' the component into IPS. If you used wsproto above, your work is done already. Simply call the ./build/proto/publish script which wsproto created and it takes care of publishing the package to IPS. If not, you'll have to manually handle defining and publishing the package to IPS. So try to use wsproto (if it doesn't work let us know why and hopefully it can be enhanced instead).
• If you want to support a -source package add a publish-src target to publish it. If not, just create an empty publish-src target. webstack-dev-tools includes 'wsmksrc', a tool to help set up the source packages. At this point it is fairly rudimentary though and source packages are themselves just an experiment, so they're not required.
• Create a METADATA toplevel file containing the OSR number and BUILDDEPS packages.

Please note the Web Stack build infrastructure (layout, tools and invocation) are all relatively new and subject to continuous improvement. We'll almost certainly change it a number of times in ways that break the builds. We'll either fix your build or work with you to fix it, but do be prepared for some growing pains.