A jar containing all the spark packages we want available on Mozilla analysis clusters

Перейти к файлу

Jeff Klukas eb94c9cf33 Merge pull request #3 from Mozilla-GitHub-Standards/master Add Mozilla Code of Conduct		2021-02-03 15:24:36 -05:00
.circleci	Remove unused REPO var	2018-07-11 16:50:23 -04:00
project	Initial version	2018-07-11 16:07:44 -04:00
.gitignore	Initial version	2018-07-11 16:07:44 -04:00
CODE_OF_CONDUCT.md	Add Mozilla Code of Conduct file	2019-03-29 23:58:13 -07:00
LICENSE	Initial commit	2018-07-11 14:12:15 -04:00
README.md	Initial version	2018-07-11 16:07:44 -04:00
build.sbt	Initial version	2018-07-11 16:07:44 -04:00

README.md

telemetry-spark-packages-assembly

A jar containing all the spark packages we want available on Mozilla analysis clusters.

Deploying to S3

CircleCI will handle building the assembly jar and uploading to S3 whenever a tag is applied on the master branch.

Deploying to ATMO and Airflow

Once a new tag has been applied and the jar is present in S3, you'll need to update telemetry.sh via a PR to emr-bootstrap-spark updating the path to match the new tag. Once your PR is merged and ops deploys emr-bootstrap-spark, new clusters spun up by ATMO or Airflow will reflect the updated assembly jar.