# -*- mode: indented-text -*- # # Author: Artem Belevich # # (Changes have been made to Artem's original doc, as things evolve.) # # ********************************************************************** As it's said in README "This is not very well packaged code. It's not packaged at all. Don't come here expecting something you plop in a directory, twiddle a few things, and you're off and using it. Much work has to be done to get there." This file is intended to make some things *easier* but not easy. You are still required to make some changes on your own. There is no guaranteed solution yet and it's unlikely that there will be one in the nearest future. ********************************************************************** 0. OVERVIEW Some time ago I've seen Linux Source Navigator (LSN) at http://sunsite.unc.edu/linux-source. I was impressed. It was and is a wanderful tool to explore Linux kernel source code. Then Mozilla.org came up with a more elaborate tool that includes source browser with crossreferencing (LXR http://lxr.linux.no) and CVS tree control (Bonsai - http://www.mozilla.org/bonsai.html). While LXR formatting is not as pretty as LSN's one, it has a huge advantage - it lets you see where the identifier is defined and used. And Bonsai brings nice and easy (though sometimes incompatible with browsers other but Netscape's own) intarface to the CVS history. This includes getting list of changes, giffs between revisions, etc. All in all LXR+Bonsai+other stuff beneath is a useful tool capable of handling huge projects. It's not that easy to make it work with other source tree but Mozilla's own but it's possible. And there are a lot of things to improve. Now I'm going to concentrate on the first goal - to make it work. 1. GETTING IT UP First of all you have to get all the tools in mozilla's mozilla/webtools CVS repository. This includes lxr,bonsai,registry and tinderbox. You're likely will not need neither tinderbox nor registry but get them just in case. To get the sources you have to follow instructions on http://www.mozilla.org/bonsai.html. OK, now you've got the sources but don't rush to try it right away. It's likely that you will not be able to even start most of the scripts. There are more things you will have to get and install. The short list of the things you will need: 1) MySQL database server. 2) Tcl 7.6 3) TclX 7.6 4) mysqltcl program (hmm.. This was tricky.. Read on) 5) Perl 5.004+ with Mysql module (included with MySQL). 6) Date::Parse module for Perl 7) Some kind of HTTP server so you could use CGI scripts 1.1 Getting and setting up MySQL database Visit MySQL homepage at http://www.tcx.se and grab the latest stable binary release of the server. Sure, you can get sources and compile them yourself, but binaries are the easiest and the fastest way to get it up and running. Follow instructions found in manual. There is a section about installing binary-only distributions. You should create database bonsai. It may be a good idea to make it writable by all users on your machine and change access level later. This would save you a lot of time trying to guess whether it's permissions or a mistake in the script that make things fail. 1.2-3 Getting and building Tcl & TclX 7.6 Tcl homepage is at http://www.scriptics.com. You may get sources for UNIX from ftp://ftp.scriptics.com/pub/tcl/tcl7_6/tcl7.6p2.tar.gz. TclX is an extension for Tcl that adds a lot of useful functions that are heavily used in Bonsai tool. TclX page is http://www.neosoft.com/tclx. Download sources from ftp://ftp.neosoft.com/pub/tcl/TclX/tclX7.6.0.tar.gz. Watch out for the case of the letters in URL. These guys are going to bring some fun into your life by spelling their program name in various ways. Now you've probably got both Tcl and TclX 7.6. You may try to use version 8.X but I'm not sure about results. Unfortunately I'm not an expert in "Tcl&Co.". Build and install Tcl first. Then build and install TclX. This should go without serious problems 1.4 mysqltcl - the tricky part Grab msqltcl 1.50 (yes, "msqltcl" without 'y'. That's not a typo) from MySQL site's contributed software area (http://www.tcx.se/Contrib/) or from mSQL site (www.hughes.com.au). I've used version 1.50 and it works for me, though you may try more recent version at your own risk. You're risking anyway. Then grab mysqltcl.c-patch from MySQL's contrib area and apply this patch to msqltcl.c file from msqltcl-1.50 distribution. Try to make msqltcl binary which is in fact mYsqltcl already. Very likely that you will not be able to compile it without modifications. You can use the patch in APPENDIX 1 to see what changes I had to make to compile mysqltcl. Your mileage may vary. 1.5 Perl + Mysql Besides mysqltcl you will need Perl 5.004 with DB and Mysql extensions. DB is required to use LXR browser and crossreferencer for storing its database. Mysql is used by Bonsai. If you have Perl already installed, try to run genxref program from LXR suite. If it complains that it misses DB terribly then you're probably will have to get and install DB 1.86 distribution from one of the CPAN (www.cpan.org) mirrors in src/misc directory. I personally got it from http://www.cpan.org/src/misc/db.1.86.tar.gz. Having DB compiled and installed you will also have to rebuild and reinstall Perl itself so It would recognize and compile DB module in. This can be tricky if you have DB installed in some strange place as I did. I've got an error during linking phase - there was a function missing in hash/ndbm.c file, so I just commented it out. It may potentially cause troubles, but I think it does not matter in our case as this was intended only for DBM compatibility - the feature we don't really use. Now you hopefully have Perl + DB compiled installed and working. Time to set up Mysql module. This one is easy. Just follow instructions in MySQL manual. You have to read manuals sometimes.. I think I'm getting older.. 8-) Next step is to get TimeDate module from one of the CPAN mirrors. Go to CPAN search page (http://theory.uwinnipeg.ca/search/cpan-search.html) and search for the "TimeDate" module. Then get it and install. 1.6 HTTP server You have a freedom of choice here - Apache, Netscape or any other server on UNIX would do. The only thing - to make configuration easier you'd better run HTTP daemon on the same machine that you run MySQL server on. Make sure that you can access 'bonsai' database with user id you're running the daemon with. 2. TWEAKING THE TOOLS Now you should have all necessary tools to be able to run LXR and Bonsai scripts and see why the wouldn't work for you right now. First of all you have to change "#!/usr/bonsaitools" to wherever you've installed your binaries in all executable scripts in LXR and Bonsai directories. There is now a Makefile with a "make install" command that can make this easier. See the Makefile for details. 2.1 LXR The first thing to set up is LXR tool. All it needs is the source tree (not CVS tree). It's relatively easy and works almost right of the box. Follow instructions in LXR README file. Having set LXR you will see that regardless what your source tree contains you will see that everything refers to it as Mozilla. Mozilla is a great thing and this tool was primarily tailored to mozilla tree but you'd like to control your own tree. First step is to edit your Here is the short list of changes I had to make file: ident 1) change "&root=/cvsroot" to tour CVSROOT path 2) change "file=/mozilla/" to the directory under CVSROOT where your sources are. In my case it is just "/" file: index.html Nothing vital here but probably worth changing to reflect your own environment file: lxr.conf Changes to this file are described in LXR README file and are quite simple. file: source You may find it useful to uncomment "$img = "/icons/..." lines if you yse Explorer as it does not have internal-gopher-* images built in. Actually Bonsai contains a lot of netscapism that will make your IE4 unhappy anyway. You'd better stick with Netscape if you are going to use LXR/Bonsai file: template-* Here you will probably want to watch closely at the places whare you see the word 'mozilla' near '.cgi'. There are a lot of mozilla-specific paths hardcoded cgange/get rid of banner that loads straight from mozilla.org that may be very dangerous if you're working for micro$oft and your boss comes by.. 8-) 2.2 Bonsai This stuff sometimes gets very specific about your CVS repository setup. You have to make a lot of changes until more portable configuration mechanism is introduced. First of all, create data/ directory in your Bonsai dir. Then edit data/configdata file as described in README file. Create appropriate data/XXX directory for each tree XXX you've configured in 'configdata'. Compile trapdoor program and place binary in the data/ dir ('make install' will do this for you.). run 'trapdoor >data/passwd' it will set up admin's password. At least it should. It worked for me with empty password, but I still am unable to change it from the web. file: cvsblame.cgi @line 120: you may add your own translation to help find appropriate RCS file for the given source file. TODO: We'd better get this information from CVS/Repository file. It's more generic than hardcoded tweaks. file: cvslog.cgi @line 90: the same tweaks to find RCS file for given source file @ line 463: add translations to $lxr_path to help find source file for given RCS file. TODO: This should be done using information in $CVSROOT/CVSROOT/modules file. Typical example is the following line: 'mymodule -d mydir repositorydir' which causes files for module mymodule from repositorydir to be placed in mydir. This contains enough information about backward translation mydir/file->repositorydir/file,v file: cvsquery.cgi @line 23: There are some cases when cvsrot is not passed as parameter for cvsquery.cgi. You should better check for empty CVS_ROOT and set it to your CVSROOT path. @line 71: Here we should try to make backward translation for directory name similar to one @line 463 in cvslog.cgi. @line 151: you will probably want to send email somewhere else but people @netscape.com 8-) file: dolog.pl @line 42: change path to rlog program file: toplevel.cgi @line 240: remove links that are not accessible to you anyway That's basically it. With some luck and persistence you will have 90% working system at this point. A lot of these things are just asking to be fixed in near feature. And I hope they will be. 3. Setting up database This is quite simple but time consuming operation. First create database structure using maketable.sh csript. Then go to Bonsai administration page and press "Rebuild CVS history" button. Then you may go to the theater and watch a movie or two. It will talke a lot of time. It takes several seconds to process one file. The more revisions in file the more time it will take. My SUN workstation with 2x200Mhz UltraSPARC processors run about an hour to process about 4K files with 20K+ revisions. Your mileage may vary. Read the comments at the top of dolog.pl to see how to keep your database updated when people make checkins. 4. Things to do a) There should be better way to track CVS tree changes. Now it's done by making CVS send e-mail about each checkin. (See the comments at the top of dolog.pl for some clues.) One alternative theory would be to take advantage of the CVS history command, which provides all necessary information to get list of files recently commited, so there is no need to send/process email. Just set up a cron job that will periodically look for CVS tree changes and update database. On the other hand, it's not at all clear how efficient the cvs history command is for large, active repositories. b) Better configuration. One should not hardcode CVS tree <-> Source tree translations. Another thing to configure - banners. c) LXR could be improved in a number of ways. Using MySQL database instead of DB would probably be a good idea. It's unclear what impact it will have on performance though. Incremental database updates would be nice. It might be also nice to borrow syntax highliting from LSN. 5. Conclusion. OK. This may or may not work for you. But I hope you had a great time trying. Or just reading. Any suggestions/additions are welcome. ******************************************************* APPENDIXES ******************************************************* APPENDIX 1. Patch to build mysqltcl. -------------------------------------- diff -u -r msqltcl-1.50/Makefile mysqltcl-1.50/Makefile --- msqltcl-1.50/Makefile Tue Jun 6 07:25:39 1995 +++ mysqltcl-1.50/Makefile Tue Jun 23 18:20:07 1998 @@ -38,11 +38,11 @@ # #----- Tcl/Tk libraries & such # Path for Tcl include files. -TCLINCL = -I/usr/local/include +TCLINCL = -I../include # Path for Tk include files, if different from the above. TKINCL = # Libraries required to link plain Tcl. -TCLLIBS = -L/usr/local/lib -ltcl -lm +TCLLIBS = -L../lib -ltclx7.6.0 -ltcl7.6 -lm -lnsl -lsocket # Libraries required to link plain Tk. TKLIBS = -L/usr/local/lib -ltk -ltcl -lX11 -lm @@ -66,11 +66,11 @@ # #----- Tcl/Tk libraries & such # Path for Tcl include files. -NEWTCLINCL = -I/usr/local/new/include +NEWTCLINCL = -I../include # Path for Tk include files, if different from the above. NEWTKINCL = # Libraries required to link plain Tcl. -NEWTCLLIBS = -L/usr/local/new/lib -ltcl -lm +NEWTCLLIBS = -L../lib -ltclx7.6.0 -ltcl7.6 -lm -lnsl -lsocket # Libraries required to link plain Tk. NEWTKLIBS = -L/usr/local/new/lib -ltk -ltcl -lX11 -lm @@ -82,7 +82,7 @@ # Path for TclX/TkX include files, if different from plain Tcl. NEWTCLXINCL = # Extra libraries required to link TclX. -NEWTCLXLIBS = -L/usr/local/new/lib -ltclx +NEWTCLXLIBS = -L../mysql/lib -ltclx # Extra libraries required to link TkX. NEWTKXLIBS = -L/usr/local/new/lib -ltkx -ltclx # TclX/TkX 'AppInit' files (base names). @@ -94,16 +94,16 @@ # #----- mSQL libraries & such # Path for mSQL include files. -MSQLINCL = -I/usr/local2/src/Minerva/include +MSQLINCL = -I../mysql/include # Libraries required to link an mSQL application. -MSQLLIB = -L/usr/local2/src/Minerva/lib -lmsql +MSQLLIB = -L../mysql/lib -lmysqlclient #===== END OF CONFIGURATION DEFINITIONS ===== INCL = -I. ${MSQLINCL} ${TCLINCL} ${TKINCL} CFLAGS = ${OPTIM} -c LDFLAGS = ${OPTIM} -PROGS = msqltcl msqlwish +PROGS = msqltcl # msqlwish TCLLINK = ${MSQLLIB} ${TCLLIBS} TKLINK = ${MSQLLIB} ${TKLIBS} diff -u -r msqltcl-1.50/new-tclAppInit.c mysqltcl-1.50/new-tclAppInit.c --- msqltcl-1.50/new-tclAppInit.c Tue Jun 6 07:25:38 1995 +++ mysqltcl-1.50/new-tclAppInit.c Tue Jun 23 18:28:14 1998 @@ -14,7 +14,7 @@ static char sccsid[] = "@(#) tclAppInit.c 1.11 94/12/17 16:14:03"; #endif /* not lint */ -#include "tcl.h" +#include "tclExtend.h" /* * The following variable is a special hack that is needed in order for @@ -48,7 +48,7 @@ int argc; /* Number of command-line arguments. */ char **argv; /* Values of command-line arguments. */ { - Tcl_Main(argc, argv); + TclX_Main(argc, argv, Tcl_AppInit); return 0; /* Needed only to prevent compiler warning. */ } @@ -79,6 +79,10 @@ return TCL_ERROR; } + if (Tclx_Init(interp) == TCL_ERROR) { + return TCL_ERROR; + } + /* * Call the init procedures for included packages. Each call should * look like this: @@ -90,7 +94,7 @@ * where "Mod" is the name of the module. */ - if (Msqltcl_Init(interp) == TCL_ERROR) { + if (Mysqltcl_Init(interp) == TCL_ERROR) { return TCL_ERROR; } @@ -106,6 +110,6 @@ * then no user-specific startup file will be run under any conditions. */ - tcl_RcFileName = "~/.tclshrc"; +/* tcl_RcFileName = "~/.tclshrc"; */ return TCL_OK; }