f579e95fd9
* updated setup.py and explitly specifying dependencies versions + removed user_scm_version flag * Tweaked CI configuration to comment out any potential publishing steps * trying to fix CI build * fixing linting error * Tweaked CI configuration to comment out any potential publishing steps |
||
---|---|---|
.circleci | ||
bin | ||
docs | ||
moztelemetry | ||
tests | ||
.dockerignore | ||
.gitignore | ||
Dockerfile | ||
LICENSE | ||
README.md | ||
setup.cfg | ||
setup.py | ||
tox.ini |
README.md
python_moztelemetry
PySpark bindings for Mozilla Telemetry
The underlying data no longer exists, so this repository is archived.
See the fx-data-dev
mailing list posting for more information.
Documentation
Online documentation is available at https://mozilla.github.io/python_moztelemetry/
Deploying a code change
After having your PR reviewed and merged create a new release on github. A new pypi release will be automatically triggered by Travis.
Installing from pypi
- To install this package from pypi run:
pip install python_moztelemetry
Updating parse_histograms.py
moztelemetry/parse_histograms.py is a mirror of its counterpart from mozilla-central. To update it to the latest version you can run
bin/update_parser_scripts
Note that this file was formerly called histogram_tools.py and was renamed in Bug 1419761.
Updating message_pb2.py
moztelemetry/heka/message_pb2.py
is generated from
mozilla-services/heka.
To regenerate it, you'll need to install a protobuf
package for your system.
To avoid installing go-specific extensions, remove the gogo.proto
import
and the gogoproto
options and then run protoc
:
git clone https://github.com/mozilla-services/heka
mkdir pythonfiles/
protoc -I heka/message --python_out pythonfiles/ heka/message/message.proto
Testing locally
To test/debug this package locally, you can run exactly the job that CircleCI runs for continuous integration by installing the CircleCI local CLI and invoking:
circleci build --job py36
See [.circleci/config.yml] for the other configured job names (for running tests on different python versions).
The above process takes a few minutes to run every time, so there
is also a bin/test
script that builds a docker image and
python environment (both of which are cached locally) and allows
you to run a subset of tests. Here are some sample invocations:
./bin/test tests/ -k test_unpack # runs only tests with key "test_unpack"
./bin/test tests/heka/ # runs only tests in tests/heka
PYTHON_VERSION=2.7 ./bin/test # specify a python version
It's also possible to run the tests locally outside of docker
by invoking tox
directly, but the details of doing so depend
on your local development environment and are outside the scope
of these docs. Be aware that you will need to have a working
installation of Java and libsnappy, likely via your OS's package
manager (i.e. brew install snappy
on MacOS).
If you're receiving mysterious errors, try removing cached files via:
./bin/clean