Developer information

Info for developers

Common issues

• Using localhost as a repository doesn't mix well with http_proxy environment settings. Your proxy will probably get confused about who you meant by localhost. There's code to handle this case on a few paths, but not all paths. For now, turn off your proxy setting before invoking the pkg commands.
• I don't know Python--how best to learn? The current developers are all relatively new to Python, but are old hands at C/Make/shell and so on. For programmers already familiar with Unix programming, Stephen recommends David Beazley's Python Essential Reference (3rd Edition). Danek has had as much success just using online resources, such as docs.python.org. (The bundled Python we're using is presently 2.4.4.)
• The elf module won't compile. The current targets assume that you have either a DevPro C compiler or GCC in your path. If you don't, you'll see something like

$ make
...
ERROR: no C compiler not found; update your PATH or set the CC env variable
error: command '/usr/lib/python2.4/pycc' failed with exit status 1
*** Error code 1
make: Fatal error: Command failed for target `all'

 You'll need to install a compiler: you can use either "pfexec pkg install ss-dev", to install SunStudio Express, or "pfexec pkg install gcc-dev", to install the GCC toolchain.

• What does "ImportError: No module named setuptools" mean? You don't have the SUNWpython-setuptools package installed. From build 122 onwards, there is a package called developer/opensolaris/pkg5 which you can install to get all the dependencies required to build the pkg(5) workspace, except for the compilers.

Running a development server, smf(5)-style

 One convenient way to run the server is, after pulling down a child, is to use the link Makefile target to connect your repository's bits into the system. That is,

$ cd path/to/child
$ cd src
$ su
Password:
# make link
# svccfg import pkg-server.xml

 Then, anytime you want to reload the server (because you edited module source, say), you only have to issue

# ./bump-server.ksh

 (You can do this as non-root, if you've given yourself the "Service Management" profile in /etc/user_attr.)

Running the client and server from the workspace proto area

 Developers may find it easier to run the client and server from their location in the workspace proto area.

Server

$ WS=<path to root of workspace>
$ PYTHONPATH=$WS/proto/root_`uname -p`/usr/lib/python2.4/vendor-packages/; export PYTHONPATH
$ PKG_REPO=<path to repo> $WS/proto/root_`uname -p`/usr/lib/pkg.depotd

Client

$ WS=<path to workspace root>
$ PYTHONPATH=$WS/proto/root_`uname -p`/usr/lib/python2.4/vendor-packages/; export PYTHONPATH
$ PATH=$WS/proto/root_`uname -p`/usr/bin:$PATH; export PATH
$ PKG_IMAGE=<path to image root>; export PKG_IMAGE
$ pkg image-create -F -a authname=http://<server address>:10000 $PKG_IMAGE
$ pkg refresh
$ pkg install <package>

Bug Management

We're transitioning the bugs in the "pkg" product in the bugzilla instance on defect.opensolaris.org/bz to the Oracle-internal Bugster database.  In order to do this sanely, write-access to the product for defect.opensolaris.org is disabled, though bugs can still be seen as before.

For anyone inside Oracle who needs to file a bug, please do so directly in Bugster using product "solaris", category "pkg".  The subcategories are same as the components of product "pkg" in bugzilla (e.g. solaris/pkg/cli, etc.).  For anyone outside Oracle who would like to do so, please report the bug to the pkg-discuss mailing list, and we'll do it for you.  We hope at some point in the future there will be some mechanism for external folks to file and update bugs, but there is none now.

Pushing to the (Source) Repository

You'll need to affiliate to the pkg project and ask for commit rights before you'll be able to push changesets yourself. Before you'll be allowed to do that, you'll need to have produced three successful, non-trivial changesets. Until you have commit rights, you'll want to make your changeset available either as the output of "hg export --git" or (if there are binary files or copies / additions / deletions) "hg bundle". Ask on pkg-discuss for someone to pick up your change, and send your patch or bundle to the person who takes it, or point them at it.

Changeset comments should be of the form

<bugid> <synopsis>

 Multiple lines are allowed for multiple bugs. Note that each bug should be all on one line, and there should be exactly one space between the bugid and the synopsis -- no extraneous punctuation is allowed. The synopsis should be exactly as it is in the bug tracking system. Correct spelling of the synopsis is highly encouraged.

DTrace, profiling, and other means of measuring pkg(5) performance

• Dan blogged about using DTrace with pkg(5).

Extracting self-signed certificates from remote servers

In some cases, remote servers have SSL certificates that aren't trusted by the client. If this is because the server has a self-signed certificate, it's easiest to download the cert from the remote host. (In other cases, the client may be missing one or more CA certificates if a server fails to verify). The following script uses openssl's s_client to connect to a remote host and extract its certificate.

Code

#!/usr/bin/python
#
# CDDL HEADER START
#
# The contents of this file are subject to the terms of the Common Development
# and Distribution License (the "License").  You can obtain a copy of the
# license at usr/src/OPENSOLARIS.LICENSE or
# http://www.opensolaris.org/os/licensing.  When distributing Covered Code,
# include this comment block in each file and include the License file at
# usr/src/OPENSOLARIS.LICENSE.
#
# CDDL HEADER END
#

#
# Copyright 2009 Sun Microsystems, Inc.  All rights reserved.
# Use is subject to license terms.
#

import getopt
import subprocess
import sys

def main():

        dir = None
        openssl = "/usr/bin/openssl"
        outfile = None
        server = None

        opts, pargs = getopt.getopt(sys.argv[1:], "o:")

        for opt, arg in opts:
                if opt == "-o":
                        outfile = arg

        if not outfile:
                print >> sys.stderr, "Usage: ss_certs -o  "
                sys.exit(1)

        server = pargs[0]

        try:
                outf = file(outfile, "wb")
        except EnvironmentError, e:
                print >> sys.stderr, "Unable to open %s: %s" % (outfile, e)
                sys.exit(1)

        cmd = [openssl, "s_client", "-connect", server]
        p = subprocess.Popen(cmd, stdin=subprocess.PIPE,
            stdout=subprocess.PIPE, stderr=subprocess.PIPE)

        stdout, stderr = p.communicate("QUIT\n")

        writecert = False

        for l in stdout.splitlines():
                if l.startswith("~-----BEGIN CERTIFICATE"):
                        writecert = True
                        outf.write(l + "\n")
                elif l.startswith("~-----END CERTIFICATE") and writecert:
                        writecert = False
                        outf.write(l + "\n")
                        break
                elif writecert:
                        outf.write(l + "\n")

        outf.close()

if __name__ == '__main__':
        main()

Usage

Although unnecessary, one could extract the certificate from pkg.sun.com by using the following example.

$ ss_cert.py -o test.pem pkg.sun.com:443/opensolaris/extra

The s_client command to openssl prints out all kinds of details about the certificate and the connection to the client. The script scrapes out the certificate portion of the transaction and saves that to the file specified in the -o option. For this particular tool, the host is specified without a URI and must include the correct ssl port. (443 for https)