OpenSolaris

Info for developers

Common issues

• Using localhost as a repository doesn't mix well with http_proxy environment settings. Your proxy will probably get confused about who you meant by localhost. There's code to handle this case on a few paths, but not all paths. For now, turn off your proxy setting before invoking the pkg commands.
• I don't know Python--how best to learn? The current developers are all relatively new to Python, but are old hands at C/Make/shell and so on. For programmers already familiar with Unix programming, Stephen recommends David Beazley's Python Essential Reference (3rd Edition). Danek has had as much success just using online resources, such as docs.python.org. (The bundled Python we're using is presently 2.4.4.)
• The elf module won't compile. The current targets assume that you have either a DevPro C compiler or GCC in your path. If you don't, you'll see something like

$ make
...
ERROR: no C compiler not found; update your PATH or set the CC env variable
error: command '/usr/lib/python2.4/pycc' failed with exit status 1
*** Error code 1
make: Fatal error: Command failed for target `all'

 You'll need to install a compiler: you can use either "pfexec pkg install ss-dev", to install SunStudio Express, or "pfexec pkg install gcc-dev", to install the GCC toolchain.

• What does "ImportError: No module named setuptools" mean? You don't have the SUNWpython-setuptools package installed. From build 122 onwards, there is a package called developer/opensolaris/pkg5 which you can install to get all the dependencies required to build the pkg(5) workspace, except for the compilers.

Running a development server, smf(5)-style

 One convenient way to run the server is, after pulling down a child, is to use the link Makefile target to connect your repository's bits into the system. That is,

$ cd path/to/child
$ cd src
$ su
Password:
# make link
# svccfg import pkg-server.xml

 Then, anytime you want to reload the server (because you edited module source, say), you only have to issue

# ./bump-server.ksh

 (You can do this as non-root, if you've given yourself the "Service Management" profile in /etc/user_attr.)

Running the client and server from the workspace proto area

 Developers may find it easier to run the client and server from their location in the workspace proto area.

Server

$ WS=<path to root of workspace>
$ PYTHONPATH=$WS/proto/root_`uname -p`/usr/lib/python2.4/vendor-packages/; export PYTHONPATH
$ PKG_REPO=<path to repo> $WS/proto/root_`uname -p`/usr/lib/pkg.depotd

Client

$ WS=<path to workspace root>
$ PYTHONPATH=$WS/proto/root_`uname -p`/usr/lib/python2.4/vendor-packages/; export PYTHONPATH
$ PATH=$WS/proto/root_`uname -p`/usr/bin:$PATH; export PATH
$ PKG_IMAGE=<path to image root>; export PKG_IMAGE
$ pkg image-create -F -a authname=http://<server address>:10000 $PKG_IMAGE
$ pkg refresh
$ pkg install <package>

Bug Management

Bugs should be filed on defect.opensolaris.org, under classification "Development", product "pkg".

When you grab a bug to work on it, please assign yourself to the bug. If you decide to stop working on it, reset the assignee to the default. When assigning a priority to a bug, please follow the guidelines linked to from the Priority field label.

Although bugzilla should constrain the state transition to some extent, if we constrain it too much, you have to go through a number of state transitions to get to where you want to go. If there are problems with the state transitions (like they're proving to be super inconvenient), raise the issue on pkg-discuss.

Generally, when you figure out why the buggy behavior is as it is, explain it and change the state to CAUSEKNOWN. When you figure out how to fix it, explain the fix and change the state to FIXUNDERSTOOD. When you actually start fixing the bug, change the state to FIXINPROGRESS. Once you've gotten all your testing done, reviews completed, etc, and pushed a changeset containing the fix to the main repository, add a comment like "Fixed in changeset <changesetid>." (where <changesetid> is either the short or long hex changeset id), and change the state to FIXINSOURCE.

Unless you're a project leader, don't change the state to anything more resolved than that, unless you're closing it off as a duplicate, saying you won't fix it, or can't reproduce, etc. Don't mark it as RESOLVED/FIXED or CLOSED/FIXED or FIXINBUILD. These are all "gatekeeper" states, at least for now.

If you want a simple bug to work on to get started, look for the keyword "bite-size". Conversely, if you find a bug that looks like it should be pretty easy to solve, then set the keyword "bite-size". Another good query for starter bugs is on the whiteboard keyword "needstestcase".

Pushing to the (Source) Repository

You'll need to affiliate to the pkg project and ask for commit rights before you'll be able to push changesets yourself. Before you'll be allowed to do that, you'll need to have produced three successful, non-trivial changesets. Until you have commit rights, you'll want to make your changeset available either as the output of "hg export --git" or (if there are binary files or copies / additions / deletions) "hg bundle". Ask on pkg-discuss for someone to pick up your change, and send your patch or bundle to the person who takes it, or point them at it.

Changeset comments should be of the form

<bugid> <synopsis>

 Multiple lines are allowed for multiple bugs. Note that each bug should be all on one line, and there should be exactly one space between the bugid and the synopsis -- no extraneous punctuation is allowed. The synopsis should be exactly as it is in the bug tracking system. Correct spelling of the synopsis is highly encouraged.

DTrace, profiling, and other means of measuring pkg(5) performance

• Dan blogged about using DTrace with pkg(5).

Extracting self-signed certificates from remote servers

In some cases, remote servers have SSL certificates that aren't trusted by the client. If this is because the server has a self-signed certificate, it's easiest to download the cert from the remote host. (In other cases, the client may be missing one or more CA certificates if a server fails to verify). The following script uses openssl's s_client to connect to a remote host and extract its certificate.

Code

#!/usr/bin/python
#
# CDDL HEADER START
#
# The contents of this file are subject to the terms of the Common Development
# and Distribution License (the "License").  You can obtain a copy of the
# license at usr/src/OPENSOLARIS.LICENSE or
# http://www.opensolaris.org/os/licensing.  When distributing Covered Code,
# include this comment block in each file and include the License file at
# usr/src/OPENSOLARIS.LICENSE.
#
# CDDL HEADER END
#

#
# Copyright 2009 Sun Microsystems, Inc.  All rights reserved.
# Use is subject to license terms.
#

import getopt
import subprocess
import sys

def main():

        dir = None
        openssl = "/usr/bin/openssl"
        outfile = None
        server = None

        opts, pargs = getopt.getopt(sys.argv[1:], "o:")

        for opt, arg in opts:
                if opt == "-o":
                        outfile = arg

        if not outfile:
                print >> sys.stderr, "Usage: ss_certs -o  "
                sys.exit(1)

        server = pargs[0]

        try:
                outf = file(outfile, "wb")
        except EnvironmentError, e:
                print >> sys.stderr, "Unable to open %s: %s" % (outfile, e)
                sys.exit(1)

        cmd = [openssl, "s_client", "-connect", server]
        p = subprocess.Popen(cmd, stdin=subprocess.PIPE,
            stdout=subprocess.PIPE, stderr=subprocess.PIPE)

        stdout, stderr = p.communicate("QUIT\n")

        writecert = False

        for l in stdout.splitlines():
                if l.startswith("~-----BEGIN CERTIFICATE"):
                        writecert = True
                        outf.write(l + "\n")
                elif l.startswith("~-----END CERTIFICATE") and writecert:
                        writecert = False
                        outf.write(l + "\n")
                        break
                elif writecert:
                        outf.write(l + "\n")

        outf.close()

if __name__ == '__main__':
        main()

Usage

Although unnecessary, one could extract the certificate from pkg.sun.com by using the following example.

$ ss_cert.py -o test.pem pkg.sun.com:443/opensolaris/extra

The s_client command to openssl prints out all kinds of details about the certificate and the connection to the client. The script scrapes out the certificate portion of the transaction and saves that to the file specified in the -o option. For this particular tool, the host is specified without a URI and must include the correct ssl port. (443 for https)