new documentation files.

2001-12-31 20:02:05 +00:00 · 2001-12-31 20:02:05 +00:00 · e9e256356f
--- a/webtools/tinderbox2/Overview
+++ b/webtools/tinderbox2/Overview
@ -0,0 +1,153 @@
 Overview of the Tinderbox System
 --------------------------------
 Tinderbox is an information display system. It runs on a machine with
 a webserver and will periodically write static HTML files to the disk
 so that the webserver can serve these documents. Tinderbox is run out
 of cron every five minutes.  It gathers up information from various
 databases including: CVS Logs, Bonsai, and Perforce.  It will also
 process mail which is sent to it. Mail is sent from Bug Ticketing
 software and Build/Test Machines.  All this information is combined to
 produce the HTML pages.
 Since no two companies will structure their development processes the
 same way, the tinderbox code has to be highly configurable to account
 for most possible uses.  There is a main configuration file which
 allows most of the major user configurable variables to be set.
 Novice users can expect to edit only this file and get a working
 tinderbox system.  Additionally each library has been broken into two
 parts.  One part is the library specific configurations.  This file is
 expected to need modifications in some installations.  I have put all
 the library configurations into one directory to make it easy to find
 the parts of tinderbox which are easy to modify.  Each configuration
 library can be thought of as a table which might need to be edited or
 extended for use at your company.  I have provided a working system
 but the defaults may not suit your needs.  These tables can be easily
 changed in small ways by simply looking at the file and making obvious
 changes.  I have also allowed for the possibility of making complex
 changes that only a competent perl programmer could define.  Changes
 are not made to the files which I have provided.  Rather the changes
 are made to copies of the files which are stored in a local
 configuration directory.  This ensures that you can easily version the
 Tinderbox code as it is provided to you from the official distribution
 and you can separately version the local configurations which you
 make.  It is also easy to see the local configurations since you have
 both the original and the modified code on the same server and can
 difference the two.  As an example you might need to change the
 BuildStatus I assume that you have the following possible build
 outcomes (Build in progress, Build failed, Build succeded but tests
 failed, Build and all tests were successful) You may have additional
 outcomes to specifiy which kind of tests failed (unit test failed, not
 enough unit test coverage, performance tests failed).  Similarly you may have unusual requirements for how the filesystem should be laied out.  I provide a 
 I suggest that you read through the files to see
 how they are laid out and what types of changes are possible.
 The build machines are not considered part of the tinderbox server.
 They are clients just like Bug Ticketing systems and Version control
 systems are clients.  Build machines mail their build logs to the
 server in a special format.  This format specifies that name/value
 pairs must appear at the top of the mail message followed by the
 complete build log.  Scripts for setting up a tinderbox build client
 can be found in the clientbin directory but you may have other build
 needs and may use any build methods you choose.
 The central concept of the Tinderbox system is the notion of a 'Tree'.
 When several different groups are working out of the same version
 control system often the files are partitioned into separate modules
 with each group working on one or more disjoint modules.  Over time
 the developers need to branch their code because several different
 versions of the files are under development at the same time.  A tree
 is a module/branch pair.  This corresponds to a set of files which can
 be checked out and built.  Tinderbox makes one page for each tree and
 displays what work is being done on that tree.  CVS has a notion of
 branches and of modules but not of trees.  It is not possible to give
 a branch/module pair a name.  The tinderbox TreeData provides the
 mappings between treenames and branch/module pairs.  Tinderbox
 displays the updates to bug tickets on the appropriate tree page.
 This requires an easy mapping between bug tickets and trees. One
 example of a complex function to determine tree name would be if each
 of the product product types listed in the bug tracking data base
 refers to one development project, except for a particular
 feature/platform of one particular project which is being developed by
 a separate group of developers.  So the version control notion of
 trees (a set of modules on a branch) may not have a direct map into
 the bug tracking database at all times.  In large projects it is
 sometimes convenient to have a tree called 'ALL' which is used to
 display all checkins performed on any trees and all bug tickets worked
 on by any programmers.  It is not possible to build or test the 'ALL'
 tree and neither the version control nor bug ticketing system knows of
 its existence.
 The Bug Tracking code was intended to be as general as possible.
 Most bug ticketing systems send mail when tickets change state.  The
 mail is often of the same form.  It is a name/value pair which the
 separator being the string ": ".  Tinderbox will parse mail of this
 form and display the interesting fields on the appropriate tree page.
 The configuration of this module involves specifying which bug ticket
 names are interesting and should be displayed.  Also you will need to
 specify how to map a bug ticket into a a tree.  This could be very
 simple if each bug ticket has a field which represents the tree it is
 applicable to (in this case tree could equal project) or can be very
 complex if the tree must be computed by the values of a set of fields.
 Also tinderbox keeps track of which bugs are "reopened" and displays
 them in a different column.  The idea is that some bugs are moving
 backwards and creating duplicate work.  These tickets are particularly
 troublesome and should be watched specially.  So possible all ticket
 status are partitioned into "progress" or "slippage" categories.  You
 will need to specify what status values are possible for your ticket
 system and you will also need to specify the set of columns which you
 would like to see on the status page.
 The heart of the tinderbox system is the 'status table'.  This is an
 HTML table which graphically shows how the changes made to the
 development databases.  It will show what is going on in the version
 control system, the bug tracking system, the build system, automatic
 regression tests and provide a notice board for developers to inform
 each other of current news.  By placing all this information in the
 same table it is possible to correlate and cross check how different
 types of changes effected each other and what was going on with the
 whole project at different times in the day. The rows of the table
 represent time with the most current events at the top of the page.
 There are different sets of columns for each database which needs to
 be displayed.  The sets of columns are managed by independent modules.
 There is one module for each version control system and each bug
 tracking system which tinderbox knows how to interface with.  It is
 easy to port the system to new databases by just adding a new module
 using the same style as the existing modules.  Modules never share or
 peek at each others data all combining of data is done by the humans
 who stare at the table and interpret what is going on.  The main
 tinderbox system does not know how many columns the final table will
 have. It only knows about a list of table modules.  Each module in the
 list is called in turn to generate the complete row then the entire
 row is displayed. The user must configure tinderbox with the list of
 modules which are of important to their own environment.  There is no
 restriction on the number of modules which may be configured, though
 due to implementation details each module can only appear once in the
 table.  There are many pop up windows embedded in the status table
 these will provide extra level of detail when a mouse is placed over
 the link.  By moving your mouse around the page you may effectively
 drill down into an item of interest and learn more about it without
 leaving the page.  Most of the links will click through to the
 appropriate database.  Thus if you need more data about an item you
 can click on the link and query the database directly.
 Besides the status table there is one other feature of the status
 page.  The page displays some information which is not correlated
 through time and with other data.  This information is called status
 table headers.  The main headers are the message of the day (MOTD),
 and the Tree State though there are a few others headers of mainly
 historical interest.  The important issue with the headers is that
 they are not optional.  Tinderbox can render a table with as little or
 as many columns in the status table as you wish but each of the
 headers has a particular place on the status page and needs to be
 rendered in a particular way (font size, font type, etc) thus the
 tinderbox server must know where each header must go and how to
 specify the appropriate html context for this header.  Users may set
 null defaults for headers that they do not need but it is much harder
 for a user to add new headers to the code in a modular fashion.
--- a/webtools/tinderbox2/Policies
+++ b/webtools/tinderbox2/Policies
@ -0,0 +1,277 @@
 Preparations you will need to make and 
 policies you will need to set:
 -----------------------------------
 To install tinderbox you will need some information about your
 existing computer systems and some idea about what your goals are.
 Here is a list of questions to help get you started, some of these
 ideas may not be apropriate for your environment.
 The webserver will serve the tinderbox pages.  
 Webserver configuration is a bit of an art and you will need to
 understand the policies which are used to administer your webserver.
 *) You will need to decide the directory where tinderbox should write
 the static HTML pages.  This will depend on how your webserver is
 configured. The default location is based on the RedHat 7.1
 (apache-1.3.19-5) installation and is: /var/www/html/tinderbox2.  You
 will also need to know what the URL browsers will need to use to find
 this directory. Since tinderbox generates static web pages, it is
 possible to run tinderbox and not run a web server.  One way this
 could be done is if you have a network file system and all users have
 browsers which can read from the HTML directories.  In this case all
 URL's should begin with "file:/" instead of the usual "http://".
 *) Project level administration is done via cgi scripts.  These
 scripts allow administrators to set the message of the day, and the
 state of the tree (open, closed, restricted).  Also all users can post
 notices to the web pages via a cgi script.  CGI programs are often
 restricted to a portion of the file system which is disjoint from the
 HTML files. You will need to figure out where the CGI programs will
 go.  Tinderbox takes its defaults from RedHat 7.1 and uses:
 /var/www/cgi-bin/tinderbox2. You will also need to know what the URL
 browsers will need to use to find this directory.
 *) CGI scripts will run as an unauthenticated user on your system.
 You will need to decide which user will run the tinderbox CGI scripts.
 The same user id must be used for running the scripts as for tinderbox
 mail delivery.  The Tinderbox Configuration files will define this
 user id and as a security precaution check that it is running as the
 required id.  It is suggested that this id not be a privileged id
 (higher ids are better, please make this number be grater then 10 and
 bigger then 100 is recommended).  Smaller ids are often assumed to
 have more privileges on a Unix box then larger ids.  It is not a good
 idea for an unauthenticated user to have any privileges so a large id
 is recommended. It is also recommended that you not use the id 'nobody'
 as this id is over used and it would be better to partition the
 unauthenticated user into separate ids in case of security problems.
 RedHat runs all its CGI scripts as the user 'apache', this is an
 acceptable user.  I would prefer to have a separate user to run the
 tinderbox CGI scripts but this would require recompiling apache to
 enable suEXEC, and it is more effort then most groups can afford.
 *) Tinderbox Files. There are other tinderbox files which need to be
 placed on the webserver.  These include libraries and non-cgi
 programs. You will need to decide where to place these files.  Most
 users put them in /home/tinderbox2.
 *) Tinderbox Data. Tinderbox stores its data in the file system.  For
 security it is often a good idea to keep this data out of the HTML and
 CGI directories so that malicious users can not directly access this
 data.  The compressed build logs can grow quite large, so it is
 recommended to put the data on a file system with room.  The default
 is to put them in the directory /home/tinderbox2/data.
 Mail
 ----
 *) Many of the tinderbox modules (Bug Ticket, Build, CVS) receive
 their data via mail.  The mail system on you web server machine must
 be configured to deliver the mail into the tinderbox mail processing
 programs.  You should spend some time understanding how your mail
 delivery system can be configured to allow user mail to be delivered
 into a program and how to set the user id under which this delivery
 occurs.  If you do not wish to configure your mail delivery program
 then you can use fetchmail to pull the mail out of a mail box and push
 it into the programs on a periodic basis.  See the install page for
 details on what I have learned about mailing systems.
 Production Version Control
 -------------------------
 One of the biggest responsibilities which a "buildmaster" has is the
 requirement that all code should be reproducible.  That is that at
 any point in the future, even more than one year later, the current
 binaries should be able to be rebuilt byte for byte from sources.
 This requirement can be broken down as follows:
 1) The build machine must be reproducible.  
 We must be able to get back the same build machine we had at any point
 in the past.  This means that all OS libraries, all header files, all
 compilers, all build tools (make, grep, sed) must have some mechanism
 to roll back.  It is common to use a backup of the build machine to
 reconstruct it.  Most OS will give you a list of the software packages
 which are installed on the machine and their version numbers.  I like
 to keep the list of software packages which are installed on the
 machine checked into version control.  This allows me to compare the
 state of the build machine at any two points in time.  I have tools to
 recreate the build-machine from just a list of packages with version
 numbers. It is considered a best practice to limit the amount of
 software which is available on the build machine.  A build machine
 with too much installed will only make it difficult to reproduce older
 builds should the need arise.  I recommend not installing any
 web servers or graphical window managers on your build machine.  It
 should be clear that the build machine should not be the same machine
 where the tinderbox server runs.
 2) The build process must be reproducible.  That is all the steps
 which are used to create the application must be reproducible.
 *) Build Interface: We must be able to run exactly the same build
 process in the future including: all commands with command line
 arguments, all environmental variables.  I recommend that the entire
 build process be viewed as something outside of the build master
 control.  Developers are responsible for ensuring that there is a
 simple build master interface to construct all the software products
 which go into a build.  Typically there is a makefile in a standard
 place where the buildmaster can run something like "make all; make
 install;" and be guaranteed that this will build the product.  The
 build interface should be viewed as something which never changes and
 are part of the build machine, like the OS and are changed only
 rarely.  It is hard enough to track all the parts of the build process
 which we expect to change, we should not need to track complex build
 procedures.  The build procedures should have a standard interface.
 By keeping the build instructions in one makefile which is checked
 into the same version control system as the sources it is easy to
 recreate any previous build even if the commands used to build the
 software fluctuate rapidly between releases.  There must be a simple
 interface to construct the software which will hide all the complexity
 of the actual construction.  
 *) Build Environment: The makefile will code all the build commands
 and all the environmental variables (PATH, UMASK, LD_LIBRARY_PATH,
 CLASSPATH) needed to build the software though it may rely on some
 well defined command line arguments (PREFIX, CCFLAGS, JAVA_LIBS) to
 make these prematurely.  These command line arguments should not
 change between versions of the software but should be a fixed set of
 build parameters.  The parameters may be needed to specify where some
 files are found on the build machine (Ideally the build machine is set
 up the same as developers machines so these directories can be
 hard-coded into the makefiles but often there is a need for some
 directories to be specified at build time) or where files are to be
 created/installed on the build machine (typically a subdirectory of
 /var/tmp but there may be several builds running at once and each will
 need a different directory) or what kind of build is being created.
 Each part of the build which needs a particular environmental variable
 set or a special header file in some path should have tests which
 ensure that the build environment is valid.  I keep my build scripts
 installed on the build machine and they are always started by running
 /etc/rc.d/init.d/build start this ensures that I am not relying on any
 build environmental variables which are set by logging into the build
 account and are thus not tracked and versioned.
 *) Environmental safety issues: 
 If the build environment can not be used to build the software then a
 human readable error message should be generated.  My makefiles often
 run various checks on the environmental variables before they
 construct the code.  They check that all required environmental
 variables are set, that the required libraries are found, that
 directories which must be disjoint (build and install directories) do
 not overlap. This test suite becomes a build regression test and as I
 discover additional possible build problems I add new tests to the
 makefile.  I make it a habit to explicit set all environmental
 variables so that there is no doubt as to their expected values.  It
 is important for the QA group to only use Builds which were created by
 an automated process so that we are sure that there are no
 undocumented steps in either the test builds or the released build.
 3) Track the Build numbers.  Given a clean install of your product you
 should have all the information necessary to reproduce the executable
 from sources.  If a customer shows you the application binaries you
 must be able to get the source code which build the application,
 reconstruct the build machine which created the application and
 possibly rerun the build exactly the same way as the application was
 created before, this may include making some minor source code changes
 before the build is run. I like to keep a file which contains:
 	The product release name
 	The sources 'as of date'. (I always checkout my sources using
 		cvs -D 'date time' so that exactly the same sources
 		can be recovered knowing only the 'data time' which
 		was used to check them out. I am sure a similar trick
 		could be used with a perforce 'change set number'.)
 	The branch name.
 	The module name.
 This can be stored as a file in the product (encrypted if necessary)
 or may be stored in some secure build master database where the data
 can be looked up by release name.  My preference is to keep all data
 necessary to reproduce a build in the build output and delivered as
 part of the product.  This means that I can generate as many builds as
 I want automatically and not need to keep track of any of them.  When
 the QA team deems that a certain build is 'important', by making a
 particular build the official released copy then I can take a look at
 its contents and tag/branch the code at the sources which I used to
 build it.
 4) Build Prefix: It is a good idea to familiarize yourself with the
 makefile conventions regarding the make variable PREFIX.  It is
 easiest to understand if you think about what RedHat does when they
 build their distribution of RPM's but this will apply in many
 different systems including the Andrew File System (AFS) and most
 packaging systems. This variable is used during the build process
 "make all PREFIX=/home/apache" to tell the package where it will be
 installed (examples include /usr, /usr/local, /home/apache).  I
 suggest reading a few RedHat Spec files to see how this works in
 practice.  The application may need to hard-code this value into its
 object code.  When the application is installed it must not be
 installed into its proper place on the build machine.  The package we
 are constructing could cause the build machine to stop working
 correctly if it is a buggy version of a system library or major OS
 application.  Instead the makefile will install "make install
 PREFIX=/var/tmp/build-root/home/apache" the package into some other
 directory with a similar tree structure to its final destination.  The
 packaging system will then move the files into the correct place
 during an installation step on the target machine.  The installation
 step only moves files and sets permissions.  The makefile is not
 supposed to use the installation directories to hard code values into
 the application since the application will never be run from this
 installation directory.  The hard part of the build including any
 PREFIX magic is in the build section.  Notice the clear separation
 between build machine / target machine and installation on the build
 machine and installation on the target machine and construction of the
 application binaries and installation of the application binaries.
 This is one of the reasons why building an application on a build
 machine is different from the way in which developers build their code
 on their personal development machines.  This PREFIX issue will arise
 when you try and build the Tinderbox system and also when you
 construct the makefiles for your own application.  Since the build
 machine is not the target machine it can not be assumed that files
 will always be in the same places on both (for example perl).
 5) Application Architecture: 
 *) The build process should mimic the architecture of the code.  It should
 be a final test that the code was coded to the same specifications
 that it was designed.  It is a common problem for code to turn into
 spaghetti with each piece of code using functions and creating
 dependencies on every other piece of code.  For example it is probably
 a mistake for code in the database abstraction layer to be implemented
 in terms of code in the HTML generation layer.  These two libraries
 should probably be independent of each other, though they both might
 depend on a common string library.  The code architecture should limit
 the dependency graph between code modules.  The BuildMaster must
 enforce the restrictions on information flow between components. Thus
 no libraries should be in the path unless the architecture allows this
 module to depend on those libraries.
 *) The architecture must not have circular dependencies.  Circular
 dependencies not only make upgrading individual libraries difficult
 but also make testing components nearly impossible.  That is it should
 be possible to build some set of libraries L0 which depend on no
 libraries and then build some other set of libraries L1 which depend
 only on L0 libraries then build L2 which depend only on the L0 and L1
 libraries.  This "build chain" will prevent circular dependencies and
 help keep your code testable and the dependencies understandable.
 More information about why this is a good practice is available in
 "Large-Scale C++ Software Design" (Addison-Wesley Professional
 Computing Series) by John Lakos
 *) I enforce the convention that developers are not allowed to overload
 standard system libraries.  I always put standard libraries in the
 path before any library our company develops.  I build the application
 in stages to ensure that parts of the application which are not
 intended to depend on other code will not have other header files on
 the build machine at the time that they are constructed.  Build
 dependencies between modules which are expected are explicitly
 controlled with build scripts and version numbers.