Wiki code for findunref and unreferenced files

Hide Line numbers
1: This is the original email announcing the findunref tool. It needs a bit of updating, but still explains the rationale behind the tool.
2: 
3: {{{
4: 
5: Date: Wed, 21 Mar 2001 22:25:19 -0500 (EST) (19:25:19 PST)
6: From: [[Peter Memishian>>/viewProfile.jspa?username=meem]] <meem at sun dot com>
7: Subject: heads-up: findunref and unreferenced files in ON
8: 
9: Gatelings,
10: 
11: My recent putback adds a new tool to ON called `findunref’.  Simply put,
12: findunref allows both developers and gatekeepers to track down files that are
13: never used during the nightly build process.  I hope that each of you will
14: take a minute to get familiar with it and make it part of your development
15: environment.
16: 
17: You may wonder why we care about unreferenced files.  Simply put, unused
18: files in the build are like unused variables in a program: they are signs of
19: brokenness.  While there are some legitimate reasons why files are not used
20: during the build process (see "Permitted Unreferenced Files", below), the
21: vast majority of these files are just crud taking up space in our tree.
22: Others are less benign: unused packaging scripts are signs that something is
23: amiss in a package; unused .xcl files are evidence of programs that have not
24: been correctly internationalized.
25: 
26: There are currently a staggering ~1100 accidentally unreferenced files in ON
27: ~-- that is, roughly 1 out of every 28 files is accidentally unused.  This is
28: an unacceptably high number which we will be working to reduce in the coming
29: months (and years).  That said, we understand that most of you are hard at
30: work on your own projects and do not have time to help fight a war against
31: the unreferenced file backlog.
32: 
33: However, starting as of now, the gatekeepers will be tracking differences
34: in the unused file list as part of the nightly build.  Additionally, any
35: putbacks which introduce new unreferenced files into ON will be required to
36: remedy the problem or be subject to backout.  Since you will be required to
37: fix up any unreferenced files that result from your putbacks, you may want
38: to run findunref yourself prior to putback.
39: 
40: Using Findunref
41: ~---------------
42: 
43: The findunref tool is quite simple: it merely walks through the files in a
44: source tree and compares the access times of each file against a timestamp
45: file.  All files under SCCS control which have not been accessed more
46: recently than the timestamp and are not in a special exception list are
47: considered unreferenced.
48: 
49: At the start of each nightly build, nightly touches $SRC/.build.tstamp and
50: then proceeds with the build.  Thus, once the build is finished, all files
51: which have not been accessed more recently than $SRC/.build.tstamp were never
52: used by the build.  Since findunref’s default timestamp file is .build.tstamp
53: at the root of the specified source tree, running findunref on the $SRC of a
54: completed nightly build will list the files that were unused by the build.
55: 
56: Unfortunately, a considerable number of files are only used on SPARC or Intel
57: nightly builds (but not both).  As a result, findunref must be run on both
58: SPARC and Intel nightly workspaces and the results must be merged to get the
59: correct list of unreferenced files.  For instance, if $INTELSRC and $SPARCSRC
60: are set to the usr/src directories of your Intel and SPARC nightly
61: workspaces, then you must merge the results like:
62: 
63: 	$ findunref $INTELSRC exception_list | sort > ~/unref-i386.out
64: 	$ findunref $SPARCSRC exception_list | sort > ~/unref-sparc.out
65: 	$ comm -12 ~/unref-i386.out ~/unref-sparc.out > ~/unref.out
66: 
67: In the example above, exception_list can be any file (or /dev/null if there
68: are no exceptions), but should be the same exception list that the gate uses:
69: $SRC/tools/findunref/exception_list.  You can then diff your results against
70: the gate’s current list at /ws/on81-gate/usr/src/unrefmaster.out to see any
71: changes that are due to your work (as an aside, if you get odd results, make
72: sure that the `noatime’ option (see mount_ufs(1M)) is turned off on the
73: filesystem that your workspace is on).
74: 
75: Since that process is admittedly cumbersome, another approach is to include
76: the new `-f’ flag in NIGHTLY_OPTIONS.  This option, which is ignored for
77: incremental builds, tracks any differences in the unreferenced file list from
78: the previous nightly and lists those differences in the mail message.  Note
79: of course that this may produce false positives: a new file which is only
80: used on Intel will show up in as a new unreferenced file when a SPARC nightly
81: is next run on the workspace.  This option also builds $SRC/unref-$(MACH).out
82: in your nightly workspace, which may be compared at any time against the
83: unref-$(MACH).out files in /ws/on81-gate/usr/src to see changes that are due
84: to your work.
85: 
86: Project gatekeepers and enterprising developers may also wish to make use of
87: the enhanced `-u’ option, which when used with `-f’ not only copies the
88: unref-$(MACH).out files to the parent workspace, but also merges all of the
89: parent’s unref-$(MACH).out files into a master file (usr/src/unrefmaster.out
90: in the parent) which may be compared against the gate’s unrefmaster.out to
91: see any changes that are due to your project’s workspace.
92: 
93: Permitted Unreferenced Files
94: ~----------------------------
95: 
96: As mentioned previously, we understand that not all unreferenced files
97: indicate problems.  For instance, projects are encouraged to include READMEs
98: and other textfiles that aid understanding along with their putback.  Other
99: examples of permitted unreferenced files include the source for test
100: programs, goop for internal packages, and files which are part of programs
101: which are maintained externally (such as perl and apache).
102: 
103: To handle these cases without continually adding exceptions to the exception
104: list, we are asking developers to follow some common naming conventions for
105: these files.  For instance, the current exception list causes findunref to
106: ignore any file ending in ".txt" or starting with "README", so as long as you
107: use these naming conventions with your new textfiles, they will automatically
108: be ignored.  There are similar conventions for the source for test programs
109: and other files; please see $(SRC)/tools/findunref/exception_list for the
110: current list of permitted exception globs.
111: 
112: Please let me know if you have questions, comments, or issues...
113: ~-- 
114: meem
115: 
116: }}}
OpenSolaris

Top Menu

Wiki code for findunref and unreferenced files

Search

Collectives

Community Group

Project

User Group

Subsites

Community Group on Pages
