pjs/webtools/tinderbox
slamm%netscape.com 08ab1fc548 Generate warnings 1999-07-22 17:48:29 +00:00
..
examples adding the smoke test build script 1999-01-19 18:54:41 +00:00
1afi003r.gif Bonsai and Tinderbox have been freed. 1998-06-16 21:43:24 +00:00
Backwards.pm Add 'express_fast' which is a much faster than 'express'. Uses Backwards.pm. This will probably replace 'express'. 1999-03-04 21:46:07 +00:00
Empty.html Bonsai and Tinderbox have been freed. 1998-06-16 21:43:24 +00:00
Makefile Fix up NPL and comments that cls%seawood.org horked. 1998-09-01 01:41:07 +00:00
README Update with changes from Ken Estes <kestes@staff.mail.com>. 1999-06-09 17:06:56 +00:00
addimage.cgi make the offensive photo link point to terry 1999-05-21 01:25:34 +00:00
addnote.cgi Find builds with <tree>/<logfile> to shorten urls 1999-07-06 22:35:00 +00:00
admintree.cgi Only show checkboxes for trees that have had activity in the last two days. 1999-03-10 03:03:27 +00:00
buildwho.pl When we do "showall", or ask to rebuild checkin history, then we'll do 1999-02-23 20:31:02 +00:00
bustagestats.cgi Simple script to show some build bustage stats. 1999-01-20 19:38:13 +00:00
channelflames.gif Add (rather lame) images to the tinderbox channel. 1999-04-01 18:58:06 +00:00
channelok.gif Add (rather lame) images to the tinderbox channel. 1999-04-01 18:58:06 +00:00
clean.pl Don't be chatty. 1998-06-19 15:36:21 +00:00
cont.gif Add continue line to bottom. 1999-04-15 19:28:40 +00:00
copydata.pl Bonsai and Tinderbox have been freed. 1998-06-16 21:43:24 +00:00
doadmin.cgi When creating a new tinderbox tree, give a chance to specify the 1998-12-31 21:33:19 +00:00
ep_mac.pl back out attempted changes 1998-10-05 17:20:01 +00:00
ep_unix.pl Bonsai and Tinderbox have been freed. 1998-06-16 21:43:24 +00:00
ep_windows.pl Bonsai and Tinderbox have been freed. 1998-06-16 21:43:24 +00:00
faq.html Bonsai and Tinderbox have been freed. 1998-06-16 21:43:24 +00:00
fixupimages.pl Bonsai and Tinderbox have been freed. 1998-06-16 21:43:24 +00:00
globals.pl rounding messes up comments 1999-07-21 23:01:09 +00:00
handlemail.pl Can now optionally put the tinderbox directory on the command line. 1998-12-22 02:23:13 +00:00
imagelog.pl Bonsai and Tinderbox have been freed. 1998-06-16 21:43:24 +00:00
index.html Bonsai and Tinderbox have been freed. 1998-06-16 21:43:24 +00:00
processbuild.pl Generate warnings 1999-07-22 17:48:29 +00:00
reledanim.gif Bonsai and Tinderbox have been freed. 1998-06-16 21:43:24 +00:00
showbuilds.cgi Include warning summary at top after message of the day. 1999-07-21 21:08:40 +00:00
showimages.cgi Bonsai and Tinderbox have been freed. 1998-06-16 21:43:24 +00:00
showlog.cgi allow 'fulltext' with short url (e.g. log=<tree>/logfile&fulltext=1) 1999-07-06 22:37:17 +00:00
star.gif Bonsai and Tinderbox have been freed. 1998-06-16 21:43:24 +00:00
tinderbox.spec Added MPL header and changed version to SNAP. 1998-11-13 01:25:48 +00:00
viewallnotes.cgi based the query on note time, rather than build time 1999-07-20 23:44:32 +00:00
warnings.pl Avoid regenerating the same warnings file twice. 1999-07-22 01:09:50 +00:00

README

This is Tinderbox.  See <http://www.mozilla.org/tinderbox.html>.


	==========
	DISCLAIMER
	==========

This is not very well packaged code.  It's not packaged at all.  Don't
come here expecting something you plop in a directory, twiddle a few
things, and you're off and using it.  Much work has to be done to get
there.  We'd like to get there, but it wasn't clear when that would be,
and so we decided to let people see it first.

Don't believe for a minute that you can use this stuff without first
understanding most of the code.


	============
	DEPENDENCIES
	============

To use tinderbox, you must first have bonsai up and running.
See <http://www.mozilla.org/bonsai.html>.

Be warned now that bonsai is not easily installed.


	====================================
	What's What in the Tinderbox sources:
	====================================

This is a rough first pass at cataloging and documenting the Tinderbox
sources.  Many hands have been in this code over the years, and it has
accreted wildly.  There is probably quite a lot of dead code in here.


PROGRAMS
========

handlemail.pl   This is the mail deliverty agent (MDA) for tinderbox,
                the local message transfer agent (MTA) calls this
                program to deliver the tinderbox mail into the system.
                It is a wrapper for processbuild.pl.

processbuild.pl Update the "$tbx{tree}/build.dat" database as new
                mail comes in then run "./buildwho.pl $tree" and
                "./showbuilds.cgi" (to build static versions of 
                tinderbox data.

buildwho.pl     Update the who.dat file with the list of authors who
                checked in in the last two days.  This is run from
                processbuild.pl, always, and from showbuild.cgi when
                'rebuildguilty' is clicked


Conceptually, the three programs, handlemail.pl, processbuild.pl, and
buildwho.pl, make up the MDA for tinderbox.


showbuilds.cgi  Create the Tinderbox web page.  

showlog.cgi     Show a build log (brief and full) update the brief_log if
                necessary. Requires the ep_$form{errorparser}.pl error
                parser.

ep_unix.pl      Knows how to parse Unix build error logs.  Used by
                processbuild.pl There needs to be one ep_* file for
                each distinct parsing algorithm used, typically this
                is per platform.  This file defines the regular
                expressions which locate error lines: has_error()
                has_warning() and the function has_errorline() which
                parses the line into the global variables:
                $error_file, $error_file_ref, $error_line, $error_guess,

admintree.cgi   Displays the admin cgi which depends on:
                $message_of_day, $ignore_builds

doadmin.cgi     Actually do the work to admin a tinderbox tree 
                (change message of the day, turn off displays for a
                channel)

clean.pl        Run `find . -name \"*.gz\" -mtime +7 -print ` and 
                unlink those files.  Does not appear to be run from
                other tools.  It is a good candidate for a cron job.



OPTIONS to showbuilds.cgi
=========================

Options to showbuilds are specified in the URL and are undocumented
elsewhere.

If called with no 'tree' option display the possible build trees to
pick from.  The 'tree' picks the build to display.  An additional tree
can be specified with 'tree2'.

Interesting visual params are:

current state monitoring mode:
          express=1
        or
          panel=1
text mode state monitoring:
        quickparse=1

These modes do not show on my browser:
        flash=1
        rdf=1
        static=1 

These are self explanatory:
        nocrap=1;  
        hours=n; 


EMAIL FORMAT
============

Each tinderbox client mails status updates to the tinderbox server.
These mails contain special tinderbox data lines describing the
progress of the build it may also contain the log of the build process
and could contain a uuencoded binary.


The email to the tinderbox server looks like:

tinderbox: tree: Mozilla
tinderbox: builddate: 900002087
tinderbox: status: building
tinderbox: build: IRIX 6.3 Depend
tinderbox: errorparser: unix

If binaryname is set then the mail message is run through uudecode to
create a file called "$binaryname" on the tinderbox server.

# NOT USED tinderbox: buildfamily: unix


DATA STRUCTURES IN showbuilds.cgi 
=================================

load_buildlog() creates build_list a list of hash refernces of this
type
      $buildrec = {    
         mailtime => $mailtime,
         buildtime => $buildtime,
         buildname => ($tree2 ne '' ? $t->{name} . ' ' : '' ) . $buildname,
         errorparser => $errorparser,
         buildstatus => $buildstatus,
         logfile => $logfile,
         binaryname => $binaryname,
         td => $t
      };

the $buildrec->{rowspan} variable holds the number of rows that the build
should occupy on the table and is not stored in the build.dat file.

These are other add ons to the data structure
$buildrec->{hasnote}=1;
$buildrec->{noteid} = (0+@note_array);

commonly  buildrec's are accessed through:
$build_table->[$build_time][$build_name] = $buildrec;

The list of users who updated this build is stored as:
$who_list->[$checkin_time]->{$author} = 1;

There are numerous duplicate data structures which hold partial
information:

        hashes which hold all indices:
            $build_name_index->{$br->{buildname}} = 1;
            $build_time_index->{$br->{buildtime}} = 1;
        other access into $build_table:
            $build_name_names->[$i] = $n;
            $build_time_times->[$i] = $n;


loadquickparseinfo creates these references
      $build->{$buildname} = $buildstatus;
      $times->{$buildname} = $buildtime;

    

DATA FILES
==========

These files are used to store data structures.  They are a persistent
store of data between executions of the same program and they pass the
data between cooperating program.  This data is often databases with
rows separated by "\n" and columns by '|'.


$tree/${logfile}
$tree/${logfile}.brief.html
$tree/ignorebuilds.pl
$tree/mod.pl
$tree/notes.txt
$tree/treedata.pl
$tree/who.dat
$treename/build.dat

The logfile that the tinderbox client sent stored in gziped format.
The filename is ${tree}/$builddate.$$.gz so its quite random and does
not depend on the clients$build string and multiple logs are kept for
each build, one for each mail message sent.

The brief.html file is a cache of the error log summary for this log
file and is recreated when the logfile gets updated.

ignorebuilds.pl is a file which specifies builds that should not be
performed.  It is valid perl code which sets the hash reference
$ignore_builds, each key is a tree name.

mod.pl stores the tree specific message of the day.  This is not to be
       confused with mod perl the CGI library. Its contents looks like:
        $message_of_day = 'message';

notes.txt store the database of notes:
$buildtime|$buildname||$author|$time_now|$note 


treedata.pl stores the cvs information pertaining to this tree and
                looks like this:
        $cvs_module='$modulename';
        $cvs_branch='$branchname';
        $cvs_root='$repository';

who.dat file has lines like:
            $checkin_time|$author
        This gets stored in the data structure, where checkin_time
        gets fudged up so that is is a time already in the
        build_table:
                      $who_list->[$checkin_time]->{$author} = 1;
 
build.dat stores the build results table.  It is a flat file
                representation of $build_table

build.dat is a database file each row is a build and has pipe
separated columns: 

1) the time stamp of the tinderbox server 
2) time stamp of the build machine 
3) the official build name (should include build machine name)
   ( note: that 2 & 3 together uniquely identify the build 
        and all relevant build data)
4) the architecture dependent error parser to use on the log files
5) status of the build (success|busted|building|testfailed)
6) The log file for this build (if completed)
7) the name of the binary (if any) that came from the build


Other Files
====================
1afi003r.gif	The "flames" animation used by showbuilds.cgi

Empty.html	Document used for an empty frame by ???

addimage.cgi	The form that lets you add a new image to the list of
		images that Tinderbox picks from randomly.

addnote.cgi	Add a note to a build log.

admintree.cgi	Lets you perform various admin tasks on a Tinderbox tree.
		This just prompts for a password and posts to doadmin.cgi.

clean.pl	???
copydata.pl	???

doadmin.cgi	Actually do the work to admin a tinderbox tree 

ep_mac.pl	Knows how to parse Mac build error logs.  Used by ???
ep_unix.pl	Knows how to parse Unix build error logs.  Used by ???
ep_windows.pl	Knows how to parse Windows build error logs.  Used by ???

faq.html	Wildly out of date.

fixupimages.pl	???
globals.pl	???
imagelog.pl	???
index.html	???
reledanim.gif	???

showimages.cgi	Show all the images in the Tinderbox list.  Password-protected.

star.gif	The "star" image used to annotate builds by ???