Documentation

Squashfs Project Documentation

Useful links

 Developing file systems isn't trivial. If you want to get started with VFS, here's some interesting resources:
Genunix Wiki:  Writing Filesystems
Frank Hofmann's Conference: Here
Solaris Internals Chapter 14: File System Framework

Squashfs for OpenSolaris README

 Here is a basic documentation explaining how to work with the Squashfs stuff, and how we integrate in an OpenSolaris sources tree. It is also available in the SVN Repository (README).

1. Description of the Squashfs development environment
~------------------------------------------------------

  This documents explains the typical Squashfs build environment. ZFS have been
used a lot to allow easier reverts in case of an error.

  1.1. Tree explanation
  ~---------------------

  I use a ZFS pool, named `data'. Its size is 15G, but only 10G is really
needed. It has two datasets, `on' and `squashfs'.

        -+- /               <~--- Root filesystem
         |
         `~-- data/          <~--- ZFS pool
         |   |
         |   `~-- on/        <~--- ON sources tree
         |   `~-- squashfs/  <~--- SquashFS sources tree
         :
         .

    1.1.1. The Squashfs dataset
    ~---------------------------

  The Squashfs dataset holds a copy of the SVN repository hosted on
OpenSolaris.org. Inside it, you will find a shell script used to setup a
Squashfs environment, starting with an ON sources tree. This script is called
``setup_env.sh''. It takes the path to your ON sources tree as an argument.

  In the ``scripts'' subdirectory, you will find a bunch of "useful" scripts,
mainly used to perform usual tasks.

    - gen_patch.sh: Generates a patch to be applied against a vanilla ON sources
                    tree in order to integrate the Squashfs sourcecode.

    - apply_patch.sh: Applies a Squashfs patch on an ON sources tree.

    - make_cmds.sh: Builds the Squashfs commands (`mount', etc.)

    - make_module.sh: Builds the Squashfs loadable file system kernel module.

    - make_clean.sh: Cleans Squashfs objects and binaries.

    - make_all.sh: Calls make_clean.sh, then make_cmds.sh, and make_module.sh.

    - patch.sh: Calls gen_patch.sh, then apply_patch.sh.

  The usr/ subdirectory contains files to be modified in order to enable the
Squashfs module compilation when building OpenSolaris.

  The Squashfs kernel module source code can be found in
usr/src/uts/common/fs/squashfs.

    1.1.2. The ON dataset
    ~---------------------

  The ON dataset holds a vanilla copy of the ON tree, downloaded from
OpenSolaris.org. Please follow the OpenSolaris.org tutorials in order to setup
it correctly.

  1.2. Setting up the environment, generating and applying patches
  ~----------------------------------------------------------------

    1.2.1. Prerequisites
    ~--------------------

  From now, we assume that you have two directories: one for the ON tree, one
for the Squashfs files, checked out from the SVN repository. Example:

cath>ls -l                           ### Root directory
total 42
drwxr-xr-x   6 deather  root           8 Dec 10 10:55 on
drwxr-xr-x   4 root     root           3 Dec  7 13:49 squashfs

cath>ls -l squashfs/squashfs         ### Squashfs SVN repository 
total 23
-rw-r~--r~--   1 deather  other       6321 Dec 10 13:55 README
drwxr-xr-x   3 deather  other          7 Dec 10 13:51 scripts
-rwxr-xr-x   1 deather  other        586 Dec 10 13:37 setup_env.sh
drwxr-xr-x   4 deather  other          4 Dec 10 13:40 usr

cath>ls -l on                        ### Vanilla ON tree
total 50
-rw-r~--r~--   1 deather  other      10420 Nov 12 11:59 README.opensolaris
drwxr-xr-x   3 deather  other          6 Nov 12 11:43 closed
drwxr-xr-x   3 deather  other          3 Nov 29 19:18 log
-rw-r~--r~--   1 deather  other       7383 Nov 29 14:45 opensolaris.sh
drwxr-xr-x   3 deather  other          3 Nov 29 14:53 proto
drwxr-xr-x   3 deather  other          3 Nov 12 11:52 usr

    1.2.2. Install Squashfs scripts
    ~-------------------------------

  Well, now, we can setup the environment. If using ZFS, i recommand taking a
snapshot of the ON sources before starting.

cath>zfs snapshot data/on@b77
cath>cd /data/squashfs/squashfs 
cath>./setup_env.sh /data/on
./squashfs/setup_env.sh: Importing scripts...
./squashfs/setup_env.sh: Linking the Squashfs `usr' directory...
./squashfs/setup_env.sh: Done. cd to /data/on to start working.
cath>

  Easy, isn't it? 

    1.2.3. Generating a patch
    ~-------------------------

  The setup_env.sh script added a link to the Squashfs' usr directory, named
usr_squashfs. A script called `gen_patch.sh' has been copied, you can use it to
generate a patch.

cath>cd /data/on
cath>./squashfs/gen_patch.sh 
./squashfs/gen_patch.sh: Writing patch to squashfs-opensolaris.diff
./squashfs/gen_patch.sh: Generating patch...
./squashfs/gen_patch.sh: Done
cath>ls -l *.diff
-rw-r~--r~--   1 deather  other      23034 Dec 10 11:16 squashfs-opensolaris.diff
cath>

  You may want to specify the filename of the patch, for example to add the
version against which the diff have been generated:

cath>./squashfs/gen_patch.sh squashfs-opensolaris-b77.diff
./squashfs/gen_patch.sh: Generating patch...
./squashfs/gen_patch.sh: Done
cath>

    1.2.4. Applying a patch on the ON tree
    ~--------------------------------------

  Now, using the diff generated in 1.2.3., you can patch the ON sources, either:

    - automatically:

cath>pwd
/data/on
cath>./squashfs/apply_patch.sh squashfs-opensolaris-b77.diff
patching file src/uts/common/Makefile.files
patching file src/uts/common/Makefile.rules
patching file src/uts/common/fs/squashfs/squashfs_vfsops.c
patching file src/uts/common/fs/squashfs/squashfs_vnops.c
patching file src/uts/intel/Makefile.intel.shared
patching file src/uts/intel/squashfs/Makefile
patching file src/uts/sparc/Makefile.sparc.shared
patching file src/uts/sparc/squashfs/Makefile
./squashfs/apply_patch.sh: Patch applied.
cath>

    - manually:

cath>patch -d usr -p1 < squashfs-opensolaris-b77.diff
patching file src/uts/common/Makefile.files
patching file src/uts/common/Makefile.rules
patching file src/uts/common/fs/squashfs/squashfs_vfsops.c
patching file src/uts/common/fs/squashfs/squashfs_vnops.c
patching file src/uts/intel/Makefile.intel.shared
patching file src/uts/intel/squashfs/Makefile
patching file src/uts/sparc/Makefile.sparc.shared
patching file src/uts/sparc/squashfs/Makefile
cath>

  /!\ Warning: you need the GNU patch, not the Sun's one. Check it out on
  sunfreeware.com.

  1.2. Compiling the Squashfs kernel module
  ~-----------------------------------------

  To build the Squashfs kernel module, just type:

cath>cd /data/on
cath>ls opensolaris.sh
opensolaris.sh                  ### This is the nightly script.
cath>./squashfs/make_module.sh opensolaris.sh 
Build type   is  non-DEBUG
RELEASE      is  
VERSION      is  tlecomte-onx
RELEASE_DATE is  

The top-level 'setup' target is available to build headers and tools.

Using /bin/zsh as shell.
dmake: defaulting to parallel mode.
See the man page dmake(1) for more information on setting up the .dmakerc file.

*** Some stuff compiling ***

  You can check the result:

cath>file proto/root_sparc/kernel/fs/sparcv9/squashfs  
proto/root_sparc/kernel/fs/sparcv9/squashfs:    ELF 64-bit MSB relocatable
                                                SPARCV9 Version 1

  1.3. Compiling the Squashfs commands
  ~------------------------------------

  It's as easy as building the kernel module:

cath>./squashfs/make_cmds.sh 
./squashfs/make_cmds.sh: I need the nightly shell script.
cath>./squashfs/make_cmds.sh opensolaris.sh 
Build type   is  non-DEBUG
RELEASE      is  
VERSION      is  tlecomte-onx
RELEASE_DATE is  

The top-level 'setup' target is available to build headers and tools.

Using /bin/zsh as shell.
dmake: defaulting to parallel mode.
See the man page dmake(1) for more information on setting up the .dmakerc file.
cath ~--> 1 job
cath ~--> Job output
/data/on/usr/src/cmd/fs.d/squashfs/mount

  You can check the result for the `mount' command:

cath>file proto/root_sparc/usr/lib/fs/squashfs/mount 
proto/root_sparc/usr/lib/fs/squashfs/mount:   ELF 32-bit MSB executable SPARC
Version 1, dynamically linked, not stripped, no debugging information available
cath>

*** TO BE CONTINUED... ***