Quick Start Guide to FileBench

 Temporary Note: we are about to give FileBench a much needed facelift. This guide does *NOT* represent the current bits hanging out on sourceforge. This guide is for bits that will update sourceforge and appear in ON. This guide is made temporary to help us get PSARC approval to integrate FileBench into Solaris, and will be made permanent once the PSARC case gets approved.

FileBench Command

 The 'filebench' command can be found in /usr/benchmarks/filebench:

diskmonster# /usr/benchmarks/filebench/bin/filebench 
Usage:
        filebench <profile name>
        filebench -c <stat_dir> ...
diskmonster# 

Profiles

 In order to do a run, you first need a "profile". A profile specifies what workload(s) you'd like to run, where you would like your results to be saved, what parameters you'd like to have for each workload, what directory you'd like to test against, your filesystem type, a description of your run, and how long you'd like to run for. A nice feature of having a profile is that you don't have to re-enter your specifics every time you want to do a run.

 A profile is described by a file ending in '.prof'. Example profiles can be found here:

diskmonster# ls /usr/benchmarks/filebench/config                                
fileio.prof     filemacro.prof  filemicro.prof  generic.func    seqread.prof
diskmonster# 

 Let's take a look at one profile:

diskmonster# cat /usr/benchmarks/filebench/config/seqread.prof 

DEFAULTS {
        runtime = 120;
        dir = /tmp;
        stats = /tmp;
        filesystem = tmpfs;
        description = "fileio tmpfs";
}

CONFIG seqread1m {
        function = generic;
        personality = multistreamread;
        filesize = 40g;
        iosize = 1m;
}
diskmonster# 

 For the "DEFAULTS" part, what this tells us is that filebench will run for 120 seconds, will run (create files, do I/O, etc) on /tmp, save the results in /tmp, treats the filesystem as tmpfs, and has the "fileio tmpfs" description (this is used for comparing results).

 For the "CONFIG" part, what this tells us is that filebench is going to run the 'multistreamread' workload with files of size 40GB and I/O sizes of 1MB. The 'function' definition can be safely ignored for now as its for future use. The parameters that you specify here (filesize, iosize) override the workload's default. The defaults can be found in the workload description (a '.f' file). If you want to know more about a workload's description, go here.

Doing an Actual Run

 The first thing to do is to create your own profile. Let's leverage an example profile:

diskmonster# pwd
/var/tmp/ekstarz/fb
diskmonster# cp /usr/benchmarks/filebench/config/seqread.prof my_seqread.prof

 Now edit the parts that you wish to change. In my case, i have a ZFS filesystem mounted at '/bigIO'. Here's an example of what you could change it to (i've changed dir, stats, filesystem, description, and filesize):

diskmonster# cat my_seqread.prof 

DEFAULTS {
        runtime = 120;
        dir = /bigIO;
        stats = /var/tmp/ekstarz/fb/stats;
        filesystem = zfs;
        description = "zfs ON 8/1/2007";
}

CONFIG seqread1m {
        function = generic;
        personality = multistreamread;
        filesize = 40g;
        iosize = 1m;
}
diskmonster# 

 Ok, now that we have our own profile, let's do a run (note that we pass in the profile file, 'my_seqread.prof', with the ending suffix of '.prof' chopped off):

diskmonster# /usr/benchmarks/filebench/bin/filebench my_seqread      
parsing profile for config: seqread1m
Running /tmp/diskmonster-tmpfs-my_seqread-May_7_2008-14h_31m_48s/seqread1m/thisrun.f
FileBench Version 1.3.0
102330: 0.020: Multi Stream Read Version 2.0 personality successfully loaded
102330: 0.020: Creating/pre-allocating files and filesets
102330: 0.021: File largefile4: mbytes=40
102330: 0.021: Re-using file largefile4.
102330: 0.021: Creating file largefile4...
102330: 0.021: Re-using file /tmp/largefile4/00000001/00000001
102330: 0.035: Preallocated 1 of 1 of file largefile4 in 1 seconds
102330: 0.035: File largefile3: mbytes=40
102330: 0.035: Re-using file largefile3.
102330: 0.035: Creating file largefile3...
102330: 0.035: Re-using file /tmp/largefile3/00000001/00000001
102330: 0.050: Preallocated 1 of 1 of file largefile3 in 1 seconds
102330: 0.050: File largefile2: mbytes=40
102330: 0.050: Re-using file largefile2.
102330: 0.050: Creating file largefile2...
102330: 0.050: Re-using file /tmp/largefile2/00000001/00000001
102330: 0.066: Preallocated 1 of 1 of file largefile2 in 1 seconds
102330: 0.066: File largefile1: mbytes=40
102330: 0.066: Re-using file largefile1.
102330: 0.066: Creating file largefile1...
102330: 0.066: Re-using file /tmp/largefile1/00000001/00000001
102330: 0.080: Preallocated 1 of 1 of file largefile1 in 1 seconds
102330: 0.080: waiting for fileset pre-allocation to finish
102330: 0.080: Running '/usr/benchmarks/filebench/scripts/fs_flush tmpfs /tmp'
filesystem type is: tmpfs, no action required, so exiting
102330: 0.089: Change dir to /tmp/diskmonster-tmpfs-my_seqread-May_7_2008-14h_31m_48s/seqread1m
102330: 0.089: Starting 1 seqread instances
102333: 1.091: Starting 1 seqread4 threads
102333: 1.092: Starting 1 seqread3 threads
102333: 1.092: Starting 1 seqread2 threads
102333: 1.092: Starting 1 seqread1 threads
102330: 4.101: Running...
102330: 125.301: Run took 120 seconds...
102330: 125.302: Per-Operation Breakdown
limit                       0ops/s   0.0mb/s      0.0ms/op        0us/op-cpu
seqread4                  633ops/s 618.0mb/s      1.6ms/op     1471us/op-cpu
limit                       0ops/s   0.0mb/s      0.0ms/op        0us/op-cpu
seqread3                  596ops/s 581.5mb/s      1.7ms/op     1559us/op-cpu
limit                       0ops/s   0.0mb/s      0.0ms/op        0us/op-cpu
seqread2                  615ops/s 600.4mb/s      1.6ms/op     1516us/op-cpu
limit                       0ops/s   0.0mb/s      0.0ms/op        0us/op-cpu
seqread1                  638ops/s 622.2mb/s      1.6ms/op     1454us/op-cpu

102330: 125.302: 
IO Summary:      300905 ops 2482.7 ops/s, (2483/0 r/w) 2422.1mb/s,   1509us cpu/op,   1.6ms latency
102330: 125.302: Stats dump to file 'stats.seqread1m.out'
102330: 125.302: in statsdump stats.seqread1m.out
102330: 125.303: Shutting down processes
Generating html for /tmp/diskmonster-tmpfs-my_seqread-May_7_2008-14h_31m_48s
file = /tmp/diskmonster-tmpfs-my_seqread-May_7_2008-14h_31m_48s/seqread1m/stats.seqread1m.out

diskmonster# 

 In order to compare results, use the '-c' option:

diskmonster# /usr/benchmarks/filebench/bin/filebench -c stats/diskmonster-zfs-my_sqread-Aug_1_2007-13h_04m_05s stats/diskmonster-zfs-my_sqread-Aug_1_2007-13h_18m_00s 
Generating html for stats/diskmonster-zfs-my_sqread-Aug_1_2007-13h_04m_05s
file = stats/diskmonster-zfs-my_sqread-Aug_1_2007-13h_04m_05s/seqread1m/stats.seqread1m.out
Generating html for stats/diskmonster-zfs-my_sqread-Aug_1_2007-13h_18m_00s
file = stats/diskmonster-zfs-my_sqread-Aug_1_2007-13h_18m_00s/seqread1m/stats.seqread1m.out
diskmonster#

 This generates an 'index.html' file that you can proudly show your management of how you've increased performance by 2x. An example of what the output looks like is here.

Workloads

 You can find the available workloads at:

diskmonster# ls /usr/benchmarks/filebench/workloads 
bringover.f                     fileserver.f
compflow_demo.f                 mongo.f
copyfiles.f                     multistreamread.f
createfiles.f                   multistreamreaddirect.f
deletefiles.f                   multistreamwrite.f
filemicro_create.f              multistreamwritedirect.f
filemicro_createfiles.f         oltp.f
filemicro_createrand.f          randomread.f
filemicro_delete.f              randomrw.f
filemicro_rread.f               randomwrite.f
filemicro_rwrite.f              singlestreamread.f
filemicro_rwritedsync.f         singlestreamreaddirect.f
filemicro_rwritefsync.f         singlestreamwrite.f
filemicro_seqread.f             singlestreamwritedirect.f
filemicro_seqwrite.f            tpcso.f
filemicro_seqwriterand.f        varmail.f
filemicro_seqwriterandvargam.f  webproxy.f
filemicro_seqwriterandvartab.f  webserver.f
filemicro_writefsync.f
diskmonster# 

 So what is a workload, and what does it do? Let's take a look at the 'multistreamread.f' workload. We can find its description here:

diskmonster# cat /usr/benchmarks/filebench/workloads/multistreamread.f 

set $dir=/tmp
set $filesize=1g
set $nthreads=1
set $iosize=1m
set $directio=0

define file name=largefile1,path=$dir,size=$filesize,prealloc,reuse
define file name=largefile2,path=$dir,size=$filesize,prealloc,reuse
define file name=largefile3,path=$dir,size=$filesize,prealloc,reuse
define file name=largefile4,path=$dir,size=$filesize,prealloc,reuse

define process name=seqread,instances=1
{
  thread name=seqread1,memsize=10m,instances=$nthreads
  {
    flowop read name=seqread1,filename=largefile1,iosize=$iosize,directio=$directio
    flowop bwlimit name=limit
  }
  thread name=seqread2,memsize=10m,instances=$nthreads
  {
    flowop read name=seqread2,filename=largefile2,iosize=$iosize,directio=$directio
    flowop bwlimit name=limit
  }
  thread name=seqread3,memsize=10m,instances=$nthreads
  {
    flowop read name=seqread3,filename=largefile3,iosize=$iosize,directio=$directio
    flowop bwlimit name=limit
  }
  thread name=seqread4,memsize=10m,instances=$nthreads
  {
    flowop read name=seqread4,filename=largefile4,iosize=$iosize,directio=$directio
    flowop bwlimit name=limit
  }
}

echo  "Multi Stream Read Version 1.9 2005/06/21 21:18:52 personality successfully loaded"
usage "Usage: set \$dir="
usage "       set \$filesize=    defaults to $filesize"
usage "       set \$nthreads=   defaults to $nthreads"
usage "       set \$iosize= defaults to $iosize"
usage "       set \$directio= defaults to $directio"
usage " "
usage "       run runtime (e.g. run 60)"

diskmonster# 

 We can see that the workload has pre-defined defaults:

set $dir=/tmp
set $filesize=1g
set $nthreads=1
set $iosize=1m
set $directio=0

 Since our 'my_seqread.prof' profile specified filesize and iosize, those parameters will override the defaults.

 We can also see that this workload creates four files (of name 'largefile1', 'largefile2', 'largefile3', and 'largefile4'):

define file name=largefile1,path=$dir,size=$filesize,prealloc,reuse
define file name=largefile2,path=$dir,size=$filesize,prealloc,reuse
define file name=largefile3,path=$dir,size=$filesize,prealloc,reuse
define file name=largefile4,path=$dir,size=$filesize,prealloc,reuse

 This workload then creates four threads (each thread operates on its own file). Each of these main threads then creates 'nthreads' to do the processing. In our case (the default), 'nthreads' is set to 1.

thread name=seqread1,memsize=10m,instances=$nthreads
  {
    flowop read name=seqread1,filename=largefile1,iosize=$iosize,directio=$directio
    flowop bwlimit name=limit
  }

 Without being a '.f' expert, one can easily see what this workload is doing: four parallel threads, each thread reading in 1MB sequential I/Os on a 40GB file.

 More detailed information can be found on the FileBench wiki.

 Happy FileBenching!