Running a content mirror of pkg.opensolaris.org/dev

The image packaging client, pkg(1), can pull content from mirror servers. Using a mirror close to you can speed up system updates, distribution construction, zone creation, and other packaging-intensive operations. You can mirror everything from another package server, or create a smaller mirror of only the packages you install on your mirror server. The following instructions give a basic procedure for both.

1. If you're running OpenSolaris 2008.05 or later, continue on to step 2.  If you're running something other than OpenSolaris 2008.05 or a successor, you'll need to install the image packaging system. You can use the following packages on recent builds of SXCE:
   • SUNWipkg-omni.i386.pkg.gz
   • SUNWipkg-omni.sparc.pkg.gz
     You'll need SUNWipkg, SUNWpython-cherrypy, SUNWpython-mako, and SUNWpython-pyopenssl at a minimum; these are all present in the "omni" packages above. You'll then need to ensure that the SMF service manifest is imported:
     # svccfg import /var/svc/manifest/application/pkg-server.xml
     
2. Prepare a location for your mirror server. We'll use /export/pkg in these examples.  If you're running OpenSolaris 2009.06 or later, simply instruct pkgsend(1) to create the repository.
   # mkdir /export/pkg
   # pkgsend -s file:///export/pkg create-repository
   
   
   For earlier versions of the pkg(5) software, you'll need to briefly run a packaging depot to create the appropriate directory structures.
   # mkdir /export/pkg
   # /usr/lib/pkg.depotd -d /export/pkg -p 8009
   [wait for startup messages to conclude]
   ^C
   
3. The file contents get updated with every build. You can either run a full mirror with rsync, or a partial mirror of only the packages you already have on your mirror system.
   • To run a full mirror of http://pkg.opensolaris.org/dev using rsync, it's probably easiest to use a cron job like the following to stay current:
     0 2 * * * /usr/bin/rsync -a pkg.opensolaris.org::osol_dev_files /export/pkg/file
     
     
     The rsync server on pkg.opensolaris.org is currently configured to allow eight simultaneous sessions; if you are running a public mirror, please pull at 0200 your time. If you are running a mirror for private use, please update weekly or biweekly. (We can increase the number of sessions, but we'll need to set up a group of mirrors if usage becomes excessive.) You can manually run the rsync in the background to get a copy of the current files.  Finally, if you want to speed up the rsync process, and believe your previous runs to be correct, you can add the --ignore-existing option since older package content is never updated in the repository and only new files are added when new builds are published.
   • To simply mirror the packages you've already downloaded, so that they are easily available to other machines on your network, link your client's download directory to the server.
     # rm /export/pkg/file
     # ln -s /var/pkg/download /export/pkg/file
     
4. Deploy a second instance of the packaging server to run as a read-only mirror. You will want to set pkg/port to a different value if you do not want your mirror to run on port 80.
   # svccfg -s pkg/server
   > add mirror
   > select mirror
   > addpg pkg application
   > addpg start method
   > setprop pkg/mirror = boolean: true
   > setprop pkg/inst_root = astring: "/export/pkg"
   > setprop pkg/threads = count: 50
   > exit
   # svcadm refresh pkg/server:mirror
   # svcadm enable pkg/server:mirror
   
   
   The default logging configuration will send errors to standard error, and discard access requests. You can change these settings; see pkg.depotd(1M).
5. For system performance, you will probably want to mount the filesystem that contains your repository with "noatime", so that file accesses do not cause directory metadata updates. If using ZFS, the command is
   # zfs set atime=no <fs-name>
   
6. Instruct your local packaging clients to add a mirror to their configuration for the "opensolaris.org" authority:
   $ pfexec pkg set-authority -m http://my.hostname.example.com opensolaris.org
   
7. Note that a content mirror serves only the file/0/ and filelist/0/ operations to clients: the mirror doesn't know anything about the packages (or repository) it is serving. The client still relies on the origin server for metadata and remote search.