Overview
Get a build
-
download.py - This downloads a browser build for the given configuration and repository. Supported is Mozilla, Chrome and Safari.
-
build.py - This command will pull the resources of the repository and build a JS shell. Supported is Spidermonkey (Mozilla), V8 (Chrome), Webkit (Safari).
-
edge.py - This will generate the files needed to run Edge (currently installed version).
Run benchmarks
- execute.py - After running download.py, edge.py, execute.py the executor can run the fetched / compiled resources to run a benchmark. A lot of different benchmarks are supported. Supported is Sunspider, Octane, Speedometer, Dromeao, Massive, Sixspeed ... Run "execute.py -h" to get a full list.
Benchmark locally
DNS Config
When you want to run browser benchmarks you need to add the lines in slave/hosts. to your /etc/hosts (or windows equivalent) file and flush DNS cache.
Prerequisites
Note: this includes the prerequisites to download and execute browser builds, but not the ones needed to compile a shell build. For those follow the instructions at Mozilla prerequisites
In order to benchmark locally you need to install some prerequisites.
Linux
- sudo apt-get install python2.7 python-requests git mercurial svn
Windows
- cygwin64: https://cygwin.com/install.html
- using cygwin64 install python2.7, python2.7-requests, git, mercurial, svn, unzip
Mac OSX
- port: https://www.macports.org/
- using port install python2.7, py2.7-requests, git, mercurial, svn
Execute
- Using download.py or build.py get a build
- Using execute.py execute a benchmark
Examples
Run octane on a Spidermonkey build (mozilla-inbound).
python build.py -s mozilla
python execute.p -b shell.octane
Run octane and sunspider on a Spidermonkey try build.
python build.py -s mozilla-try --rev {revision}
python execute.p -b shell.octane -b shell.sunspider
Run Speedometer on a Mozilla browser build.
python download.py http://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-inbound-linux/1484772086/
python execute.p -b remote.speedometer
Run kraken on a V8 and Spidermonkey build.
python build.py -s mozilla -c android -o ~/repos/mozilla-inbound
python build.py -s chrome -c android -o ~/repos/chrome
python execute.p -b shell.kraken -e ~/repos/mozilla-inbound -e ~/repos/chrome
Non-default configuration
It is possible for the slave to run non-default configurations. That means it can provide specific argument flags or set environment flags. It can even adopt the content of the profile file for Mozilla.
To run a non-defaut configuration the "-c" flag needs to get added to "execute.py". That name corresponds to a class in slave/configs.py.
Creating a non-default configuration is as simple as adding a class that extends Default and appending flags to "args_", "env_" and "profile_". An extra possibility is to set "omit_" if that particular configuration has no meaning for other engines.
Examples
Adding an argument flag
class NonWritableJitcode(Default):
def __init__(self, engine, shell):
super(NonWritableJitcode, self).__init__(engine, shell)
if engine != "firefox" or not shell:
self.omit_ = True
return
self.args_.append("--non-writable-jitcode");
Adding content to the profile
class NoE10S(Default):
def __init__(self, engine, shell):
super(NoE10S, self).__init__(engine, shell)
if engine != "firefox" or shell:
self.omit_ = True
return
self.profile_ += "user_pref(\"browser.tabs.remote.autostart\", false);\n"
self.profile_ += "user_pref(\"browser.tabs.remote.autostart.1\", false);\n"
self.profile_ += "user_pref(\"browser.tabs.remote.autostart.2\", false);\n"
self.profile_ += "user_pref(\"browser.tabs.remote.autostart.3\", false);\n"
self.profile_ += "user_pref(\"browser.tabs.remote.autostart.4\", false);\n"