зеркало из https://github.com/mozilla/gecko-dev.git
Bug 1147029 - Land luciddream in-tree, r=ted
This commit is contained in:
Jonathan Griffin
Родитель
f9df352463
Коммит
8b1839a458
|
|
@ -0,0 +1,59 @@
|
|||
Luciddream is a test harness for running tests between a Firefox browser and another device, such as a Firefox OS emulator.
|
||||
|
||||
The primary goal of the project is to be able to test the Firefox Developer Tools' [remote debugging feature](https://developer.mozilla.org/en-US/docs/Tools/Remote_Debugging), where the developer tools can be used to debug web content running in another browser or device. Mozilla currently doesn't have any automated testing of this feature, so getting some automated tests running is a high priority.
|
||||
|
||||
The first planned milestone (for Q4 2014) is to stand up a prototype of a harness, in the process figuring out what the harness will look like and what tests will look like. This work is tracked in [bug 1064253](https://bugzilla.mozilla.org/show_bug.cgi?id=1064253) and is nearing completion. The current harness is based on the [Marionette Test Runner](https://developer.mozilla.org/en-US/docs/Marionette_Test_Runner), and tests right now are a subclass of [Marionette Python tests](https://developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette/Marionette_Python_Tests).
|
||||
|
||||
The second planned milestone (for Q1 2015) is to get the harness running in Mozilla's continuous integration environment. Whether this will be in the legacy Buildbot environment or the new TaskCluster environment is yet to be determined. The bare minimum for this milestone will be having the tests run per-checkin against a Firefox Linux desktop build and a Firefox OS emulator both built from the same changeset. A stretch goal for this milestone will be to get tests running against a Firefox Linux desktop build per-checkin paired with stable release builds of the Firefox OS emulator, to be able to test backwards-compatibility of remote debugging. It's likely that as part of this work the repository of record will move from GitHub to mozilla-central, and the harness may be renamed from the current codename to a more descriptive (but less fun) name.
|
||||
|
||||
Future directions will likely include testing Firefox desktop remote debugging against other platforms, such as Firefox for Android, desktop Chrome, Chrome for Android, and Safari on iOS.
|
||||
|
||||
Points of Contact
|
||||
=================
|
||||
|
||||
The primary developer of this project is Ted Mielczarek (@luser), ted on irc.mozilla.org, :ted in bugzilla.mozilla.org.
|
||||
|
||||
The primary DevTools point of contact is Alexandre Poirot (@ochameau), ochameau on irc.mozilla.org, :ochameau in bugzilla.mozilla.org.
|
||||
|
||||
Installation and Configuration
|
||||
==============================
|
||||
|
||||
Currently running Luciddream is only supported on Linux, as the Firefox OS emulator is only well-supported there.
|
||||
|
||||
Install this module and its Python prerequisites in a virtualenv:
|
||||
```
|
||||
virtualenv ./ve
|
||||
. ./ve/bin/activate
|
||||
python setup.py develop
|
||||
```
|
||||
|
||||
[Download a Firefox build](http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central/) (if you don't already have one).
|
||||
|
||||
|
||||
Download one of:
|
||||
* [A Firefox OS emulator build](http://pvtbuilds.pvt.build.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds/mozilla-central-emulator/) (if you don't already have one, this link requires Mozilla VPN access).
|
||||
* [A B2G desktop build](http://ftp.mozilla.org/pub/mozilla.org/b2g/nightly/latest-mozilla-central/)
|
||||
|
||||
|
||||
Unzip both your Firefox and your Firefox OS emulator/B2G desktop somewhere.
|
||||
|
||||
If you're on a 64-bit Ubuntu, you may need to do some fiddling to ensure you have the 32-bit OpenGL libraries available. See the "Solution : have both 32bit and 64bit OpenGL libs installed, with the right symlinks" section [in this blog post](http://rishav006.wordpress.com/2014/05/19/how-to-build-b2g-emulator-in-linux-environment/).
|
||||
|
||||
|
||||
Running Tests
|
||||
=============
|
||||
|
||||
To run with a Firefox OS emulator:
|
||||
```
|
||||
runluciddream --b2gpath /path/to/b2g-distro/ --browser-path /path/to/firefox/firefox example-tests/luciddream.ini
|
||||
```
|
||||
|
||||
To run with B2G desktop:
|
||||
```
|
||||
runluciddream --b2g-desktop-path /path/to/b2g/b2g --browser-path /path/to/firefox/firefox example-tests/luciddream.ini
|
||||
```
|
||||
|
||||
If you're using a locally-built B2G desktop build which doesn't have a Gaia profile included you should get Gaia and build a profile, and then pass that in with the `--gaia-profile` option:
|
||||
```
|
||||
runluciddream --b2g-desktop-path /path/to/obj-b2g/dist/bin/b2g --gaia-profile /path/to/gaia/profile --browser-path /path/to/firefox/firefox example-tests/luciddream.ini
|
||||
```
|
||||
|
|
@ -0,0 +1 @@
|
|||
[test_sample.py]
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
ok(true, "Assertion from a JS script!");
|
||||
//ok(false, "test failure");
|
||||
setTimeout(function() {
|
||||
ok(true, "Assertion from setTimeout!");
|
||||
finish();
|
||||
}, 15);
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
# Any copyright is dedicated to the Public Domain.
|
||||
# http://creativecommons.org/publicdomain/zero/1.0/
|
||||
|
||||
from luciddream import LucidDreamTestCase
|
||||
|
||||
class TestSample(LucidDreamTestCase):
|
||||
def test_sample(self):
|
||||
#TODO: make this better
|
||||
self.assertIsNotNone(self.marionette.session)
|
||||
self.assertIsNotNone(self.browser.session)
|
||||
|
||||
def test_js(self):
|
||||
'Test that we can run a JavaScript test in both Marionette instances'
|
||||
self.run_js_test('test.js', self.marionette)
|
||||
self.run_js_test('test.js', self.browser)
|
||||
|
|
@ -0,0 +1,110 @@
|
|||
#
|
||||
# This Source Code Form is subject to the terms of the Mozilla Public
|
||||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
#
|
||||
|
||||
import os
|
||||
import sys
|
||||
from marionette.marionette_test import MarionetteTestCase, MarionetteJSTestCase
|
||||
from marionette_driver.errors import ScriptTimeoutException
|
||||
|
||||
class LucidDreamTestCase(MarionetteTestCase):
|
||||
def __init__(self, marionette_weakref, browser=None, **kwargs):
|
||||
self.browser = browser
|
||||
MarionetteTestCase.__init__(self, marionette_weakref, **kwargs)
|
||||
|
||||
def run_js_test(self, filename, marionette):
|
||||
'''
|
||||
Run a JavaScript test file and collect its set of assertions
|
||||
into the current test's results.
|
||||
|
||||
:param filename: The path to the JavaScript test file to execute.
|
||||
May be relative to the current script.
|
||||
:param marionette: The Marionette object in which to execute the test.
|
||||
'''
|
||||
caller_file = sys._getframe(1).f_globals.get('__file__', '')
|
||||
caller_file = os.path.abspath(caller_file)
|
||||
script = os.path.join(os.path.dirname(caller_file), filename)
|
||||
self.assert_(os.path.exists(script), 'Script "%s" must exist' % script)
|
||||
if hasattr(MarionetteTestCase, 'run_js_test'):
|
||||
return MarionetteTestCase.run_js_test(self, script, marionette)
|
||||
#XXX: copy/pasted from marionette_test.py, refactor this!
|
||||
f = open(script, 'r')
|
||||
js = f.read()
|
||||
args = []
|
||||
|
||||
head_js = MarionetteJSTestCase.head_js_re.search(js);
|
||||
if head_js:
|
||||
head_js = head_js.group(3)
|
||||
head = open(os.path.join(os.path.dirname(script), head_js), 'r')
|
||||
js = head.read() + js;
|
||||
|
||||
context = MarionetteJSTestCase.context_re.search(js)
|
||||
if context:
|
||||
context = context.group(3)
|
||||
else:
|
||||
context = 'content'
|
||||
marionette.set_context(context)
|
||||
|
||||
if context != "chrome":
|
||||
marionette.navigate('data:text/html,<html>test page</html>')
|
||||
|
||||
timeout = MarionetteJSTestCase.timeout_re.search(js)
|
||||
if timeout:
|
||||
timeout = timeout.group(3)
|
||||
marionette.set_script_timeout(timeout)
|
||||
|
||||
inactivity_timeout = MarionetteJSTestCase.inactivity_timeout_re.search(js)
|
||||
if inactivity_timeout:
|
||||
inactivity_timeout = inactivity_timeout.group(3)
|
||||
|
||||
try:
|
||||
results = marionette.execute_js_script(js,
|
||||
args,
|
||||
special_powers=True,
|
||||
inactivity_timeout=inactivity_timeout,
|
||||
filename=os.path.basename(script))
|
||||
|
||||
self.assertTrue(not 'timeout' in script,
|
||||
'expected timeout not triggered')
|
||||
|
||||
if 'fail' in script:
|
||||
self.assertTrue(len(results['failures']) > 0,
|
||||
"expected test failures didn't occur")
|
||||
else:
|
||||
for failure in results['failures']:
|
||||
diag = "" if failure.get('diag') is None else failure['diag']
|
||||
name = "got false, expected true" if failure.get('name') is None else failure['name']
|
||||
self.logger.test_status(self.test_name, name, 'FAIL',
|
||||
message=diag)
|
||||
for failure in results['expectedFailures']:
|
||||
diag = "" if failure.get('diag') is None else failure['diag']
|
||||
name = "got false, expected false" if failure.get('name') is None else failure['name']
|
||||
self.logger.test_status(self.test_name, name, 'FAIL',
|
||||
expected='FAIL', message=diag)
|
||||
for failure in results['unexpectedSuccesses']:
|
||||
diag = "" if failure.get('diag') is None else failure['diag']
|
||||
name = "got true, expected false" if failure.get('name') is None else failure['name']
|
||||
self.logger.test_status(self.test_name, name, 'PASS',
|
||||
expected='FAIL', message=diag)
|
||||
self.assertEqual(0, len(results['failures']),
|
||||
'%d tests failed' % len(results['failures']))
|
||||
if len(results['unexpectedSuccesses']) > 0:
|
||||
raise _UnexpectedSuccess('')
|
||||
if len(results['expectedFailures']) > 0:
|
||||
raise _ExpectedFailure((AssertionError, AssertionError(''), None))
|
||||
|
||||
self.assertTrue(results['passed']
|
||||
+ len(results['failures'])
|
||||
+ len(results['expectedFailures'])
|
||||
+ len(results['unexpectedSuccesses']) > 0,
|
||||
'no tests run')
|
||||
|
||||
except ScriptTimeoutException:
|
||||
if 'timeout' in script:
|
||||
# expected exception
|
||||
pass
|
||||
else:
|
||||
self.loglines = marionette.get_logs()
|
||||
raise
|
||||
|
|
@ -0,0 +1,151 @@
|
|||
#!/usr/bin/env python
|
||||
#
|
||||
# This Source Code Form is subject to the terms of the Mozilla Public
|
||||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
#
|
||||
|
||||
from __future__ import print_function
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
|
||||
from luciddream import LucidDreamTestCase
|
||||
from marionette import Marionette
|
||||
from marionette.runner import BaseMarionetteTestRunner
|
||||
import marionette
|
||||
from mozlog import structured
|
||||
|
||||
|
||||
class CommandLineError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def validate_options(options):
|
||||
if not (options.b2gPath or options.b2gDesktopPath):
|
||||
raise CommandLineError('You must specify --b2gpath or ' +
|
||||
'--b2g-desktop-path')
|
||||
if options.b2gPath and options.b2gDesktopPath:
|
||||
raise CommandLineError('You may only use one of --b2gpath or ' +
|
||||
'--b2g-desktop-path')
|
||||
if options.gaiaProfile and options.b2gPath:
|
||||
raise CommandLineError('You may not use --gaia-profile with ' +
|
||||
'--b2gpath')
|
||||
if not options.browserPath:
|
||||
raise CommandLineError('You must specify --browser-path')
|
||||
if not os.path.isfile(options.manifest):
|
||||
raise CommandLineError('The manifest at "%s" does not exist!'
|
||||
% options.manifest)
|
||||
|
||||
|
||||
# BaseMarionetteOptions has a lot of stuff we don't care about, and
|
||||
# it seems hard to apply directly to this problem. We can revisit this
|
||||
# decision later if necessary.
|
||||
def parse_args(in_args):
|
||||
parser = argparse.ArgumentParser(description='Run Luciddream tests.')
|
||||
parser.add_argument('--emulator', dest='emulator', action='store',
|
||||
default='arm',
|
||||
help='Architecture of emulator to use: x86 or arm')
|
||||
parser.add_argument('--b2gpath', dest='b2gPath', action='store',
|
||||
help='path to B2G repo or qemu dir')
|
||||
parser.add_argument('--b2g-desktop-path', dest='b2gDesktopPath',
|
||||
action='store',
|
||||
help='path to B2G desktop binary')
|
||||
parser.add_argument('--browser-path', dest='browserPath', action='store',
|
||||
help='path to Firefox binary')
|
||||
parser.add_argument('--gaia-profile', dest='gaiaProfile', action='store',
|
||||
help='path to Gaia profile')
|
||||
parser.add_argument('manifest', metavar='MANIFEST', action='store',
|
||||
help='path to manifest of tests to run')
|
||||
structured.commandline.add_logging_group(parser)
|
||||
|
||||
args = parser.parse_args(in_args)
|
||||
try:
|
||||
validate_options(args)
|
||||
return args
|
||||
except CommandLineError as e:
|
||||
print('Error: ', e.args[0], file=sys.stderr)
|
||||
parser.print_help()
|
||||
raise
|
||||
|
||||
|
||||
class LucidDreamTestRunner(BaseMarionetteTestRunner):
|
||||
def __init__(self, **kwargs):
|
||||
BaseMarionetteTestRunner.__init__(self, **kwargs)
|
||||
#TODO: handle something like MarionetteJSTestCase
|
||||
self.test_handlers = [LucidDreamTestCase]
|
||||
|
||||
|
||||
def start_browser(browserPath):
|
||||
'''
|
||||
Start a Firefox browser and return a Marionette instance that
|
||||
can talk to it.
|
||||
'''
|
||||
marionette = Marionette(
|
||||
bin=browserPath,
|
||||
# Need to avoid the browser and emulator's ports stepping
|
||||
# on each others' toes.
|
||||
port=2929,
|
||||
)
|
||||
runner = marionette.runner
|
||||
if runner:
|
||||
runner.start()
|
||||
marionette.wait_for_port()
|
||||
marionette.start_session()
|
||||
marionette.set_context(marionette.CONTEXT_CHROME)
|
||||
return marionette
|
||||
|
||||
|
||||
#TODO: make marionette/client/marionette/runtests.py importable so we can
|
||||
# just use cli from there. A lot of this is copy/paste from that function.
|
||||
def main():
|
||||
try:
|
||||
args = parse_args(sys.argv[1:])
|
||||
except CommandLineError as e:
|
||||
return 1
|
||||
|
||||
logger = structured.commandline.setup_logging(
|
||||
'luciddream', args, {"tbpl": sys.stdout})
|
||||
|
||||
# It's sort of debatable here whether the marionette instance managed
|
||||
# by the test runner should be the browser or the emulator. Right now
|
||||
# it's the emulator because it feels like there's more fiddly setup around
|
||||
# that, but longer-term if we want to run tests against different
|
||||
# (non-B2G) targets this won't match up very well, so maybe it ought to
|
||||
# be the browser?
|
||||
browser = start_browser(args.browserPath)
|
||||
kwargs = {
|
||||
'browser': browser,
|
||||
'logger': logger,
|
||||
}
|
||||
if args.b2gPath:
|
||||
kwargs['homedir'] = args.b2gPath
|
||||
kwargs['emulator'] = args.emulator
|
||||
elif args.b2gDesktopPath:
|
||||
# Work around bug 859952
|
||||
if '-bin' not in args.b2gDesktopPath:
|
||||
if args.b2gDesktopPath.endswith('.exe'):
|
||||
newpath = args.b2gDesktopPath[:-4] + '-bin.exe'
|
||||
else:
|
||||
newpath = args.b2gDesktopPath + '-bin'
|
||||
if os.path.exists(newpath):
|
||||
args.b2gDesktopPath = newpath
|
||||
kwargs['binary'] = args.b2gDesktopPath
|
||||
kwargs['app'] = 'b2gdesktop'
|
||||
if args.gaiaProfile:
|
||||
kwargs['profile'] = args.gaiaProfile
|
||||
else:
|
||||
kwargs['profile'] = os.path.join(
|
||||
os.path.dirname(args.b2gDesktopPath),
|
||||
'gaia',
|
||||
'profile'
|
||||
)
|
||||
runner = LucidDreamTestRunner(**kwargs)
|
||||
runner.run_tests([args.manifest])
|
||||
if runner.failed > 0:
|
||||
sys.exit(10)
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
marionette-client>=0.8.5
|
||||
mozlog
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
#!/usr/bin/env python
|
||||
#
|
||||
# This Source Code Form is subject to the terms of the Mozilla Public
|
||||
# License, v. 2.0. If a copy of the MPL was not distributed with this file,
|
||||
# You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
#
|
||||
|
||||
import os
|
||||
from setuptools import setup, find_packages
|
||||
|
||||
try:
|
||||
here = os.path.dirname(os.path.abspath(__file__))
|
||||
description = file(os.path.join(here, 'README.md')).read()
|
||||
except IOError:
|
||||
description = ''
|
||||
|
||||
version = '0.1'
|
||||
|
||||
dependencies = open('requirements.txt', 'r').read().splitlines()
|
||||
|
||||
setup(
|
||||
name='luciddream',
|
||||
version=version,
|
||||
description='''
|
||||
Luciddream is a test harness for running tests between
|
||||
a Firefox browser and another device, such as a Firefox OS
|
||||
emulator.
|
||||
''',
|
||||
long_description=description,
|
||||
classifiers=[], # Get strings from http://www.python.org/pypi?%3Aaction=list_classifiers
|
||||
author='Ted Mielczarek',
|
||||
author_email='ted@mielczarek.org',
|
||||
url='',
|
||||
license='MPL 2.0',
|
||||
packages=find_packages(exclude=['ez_setup', 'examples', 'tests']),
|
||||
include_package_data=True,
|
||||
package_data={'': ['*.js', '*.css', '*.html', '*.txt', '*.xpi', '*.rdf', '*.xul', '*.jsm', '*.xml'],},
|
||||
zip_safe=False,
|
||||
install_requires=dependencies,
|
||||
entry_points = {
|
||||
'console_scripts': ['runluciddream=luciddream.runluciddream:main'],
|
||||
}
|
||||
)
|
||||
|
|
@ -407,6 +407,7 @@ package-tests: \
|
|||
stage-cppunittests \
|
||||
stage-jittest \
|
||||
stage-web-platform-tests \
|
||||
stage-luciddream \
|
||||
$(NULL)
|
||||
ifdef MOZ_WEBRTC
|
||||
package-tests: stage-steeplechase
|
||||
|
|
@ -534,6 +535,11 @@ stage-steeplechase: make-stage-dir
|
|||
cp -RL $(DIST)/xpi-stage/specialpowers $(PKG_STAGE)/steeplechase
|
||||
cp -RL $(topsrcdir)/testing/profiles/prefs_general.js $(PKG_STAGE)/steeplechase
|
||||
|
||||
LUCIDDREAM_DIR=$(PKG_STAGE)/luciddream
|
||||
stage-luciddream: make-stage-dir
|
||||
$(NSINSTALL) -D $(LUCIDDREAM_DIR)
|
||||
@(cd $(topsrcdir)/testing/luciddream && tar $(TAR_CREATE_FLAGS) - *) | (cd $(LUCIDDREAM_DIR)/ && tar -xf -)
|
||||
|
||||
MARIONETTE_DIR=$(PKG_STAGE)/marionette
|
||||
stage-marionette: make-stage-dir
|
||||
$(NSINSTALL) -D $(MARIONETTE_DIR)/tests
|
||||
|
|
@ -590,5 +596,6 @@ stage-instrumentation-tests: make-stage-dir
|
|||
stage-steeplechase \
|
||||
stage-web-platform-tests \
|
||||
stage-instrumentation-tests \
|
||||
stage-luciddream \
|
||||
$(NULL)
|
||||
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче