Mstone TODO List Updated: 3-20-2000 Dan Christian ====================================================================== Minor improvements IncludeOnce Variant on the current include that will include the specified file only once. This will allow test workloads to be used either as a stand alone test (which requires including general.wld) or as part of another test (which has allready included general.wld). IMAP delete without reading Just get the message list and delete everything. This could be the fastest way to drain out the store. Display MIN/MAX or standard deviation on graphs Gnuplot can do data points with error bars. You could either use MIN/MAX or the standard deviation for the error bars. There are issues with calculating the standard deviation numbers throughout the graph that need to be addressed. Statistics reset At least the MIN and MAX statistics could be easily reset during the test (after ramp up). This keeps the transients during startup and shutdown from dominating these numbers. The standard deviation statistics are much trickier to reset during the run. It may be better to isolate sections in post processing. Perl web server Sometimes it would be nice to include our own web server to provide the results (instead of using file: URLs). This would also be a gateway to form based test configuration editing and allow results to be interactively updated during tests. Perl with a socket library could handle this without too much trouble. Dynamic test loading Finalize an API for dynamically loading tests. This would allow tests to be added or updated separately from the core functionality. This may be needed for some types of security testing. Link graphs to the results tables There are already tags by each graph. You should be able to link the appropriate results table entry to each graph. This sort of tricky since graphs often combine multiple entries. Show statistics for every block Statistics are actually kept for every block in every thread. There should be a way to view information at this detail. Man pages The online docs are nice, but good 'ol man pages would be great in a different way. Reduce namespace polution Scripts names like setup and cleanup are too general. They should be part of the main 'mstone' script (e.g. mstone setup). Examples of script series Include example scripts to run entire series of tests in a sane way. Reconnect timer Protocols based on HTTP (e.g. WMAP) may re-connect during a session. Currently this is logged as a connect error and then another connect in order to get the current connection graph right. There should be a separate reconnect timer that the connection count graph knows about. Fix FORMAT clash At the start of a test, each client process outputs the information needed to report all its protocols. When there are multiple processes on one client, these FORMAT lines can intermix and cause parsing errors. Set connection drop rate Drop some percentage of the connections without a proper shutdown. This tests how well a server can detect and recover from hard disconnects. Improve randomness The way that we generate random numbers in a range may be not generate the proper randomness. We are using lrand48()%range we should use (lrand48/RAND_RANGE)*range. There are some end conditions that need to be thought about. All of this is in the sequence code; one change should fix (or break :) everything. Also, we may be generating numbers that are never used. This may be costly, and can create holes in the proper sequence. Improve printing The color graphs are great on screen, but tend to print poorly. Either we need a better way to generate a printable version (maybe through a CGI button), or the seperate protocols need to be printed individually. Also, Communicator does a lousy job of keeping title with tables or graphs. Hopefully, Mozilla will do better. ====================================================================== Whole new protocol tests: ICQ Test high volume instant messaging. Very useful for the bridges and gateways that people are considering using. WAP WAP is the emerging standard for mobile phones. WCAP Web based calendar services LDAP Use the LDAP SDK to do basic LDAP testing. The SDK probably isn't fast enough to call this a real performance test, but you can at least test performance degredation due to load from a real application (like a mail server). DNS Mail servers use DNS a lot. You should at least be able to see if performance is degrading due to load. Disk/filesystem Test read, write, sync, link, un-link, and append performance under multiple threads. Cert servers Test certificate authenticity checking performance ====================================================================== Possible dummy servers: SMTP Receive mail via SMTP and ignore it. Usefull for SMTP relay testing. DNS Simulate slow DNS server lookups. Usefull for SMTP relay testing. ====================================================================== Major changes Throttling Monitor and control transaction rates so that specific load levels can be easily specified. The rates should be able to vary to simulate peek hour usage and disconnect-restore. NSPR threading Use NSPR for threading and sockets. This may allow other OSes to be used as clients. This might be easy, since mstone does not need any locking, just simple thread creation and harvesting. NSPR argument parsing and hashes may also be useful. SSL Support SSL on the protocols that allow it. May require NSPR. Line speed emulation Simulate the variable delays and limited throughput of dial up connections. Scripting Allow more detailed control of protocol tests. It looks difficult to make this scalable and fast. Combined tests Deliver mail over SMTP and then see when it arrives using IMAP.