Перейти к файлу
Victor Ng c265915362
Merge pull request #22 from mozilla/features/merge-notebook
Preliminary rework of the ETL job
2020-01-10 16:31:02 -05:00
.circleci Add additional test deps 2019-12-19 11:51:24 -05:00
bin Added dataproc and gcloud integration 2019-12-13 18:36:25 -05:00
cfretl reworked the weight computation 2020-01-10 01:02:06 -05:00
docs Adde sentry integration 2019-12-19 11:18:31 -05:00
schema Added dataproc and gcloud integration 2019-12-13 18:36:25 -05:00
tests flake8 fixes 2019-12-19 12:05:14 -05:00
.gitignore Added test fixtures for CFR buckets 2019-11-29 23:30:20 -05:00
CODE_OF_CONDUCT.md Added dataproc and gcloud integration 2019-12-13 18:36:25 -05:00
Dockerfile Changed ENTRYPOINT to CMD in Dockerfile 2019-12-18 23:05:35 -05:00
LICENSE init checkin 2019-11-07 15:40:14 -05:00
MANIFEST.in cfr_weights.json schema added and validation added 2019-11-27 17:08:59 -05:00
Makefile reworked the weight computation 2020-01-10 01:02:06 -05:00
README.md Added dataproc and gcloud integration 2019-12-13 18:36:25 -05:00
envfile added GCP project env for container startup 2019-12-18 23:52:18 -05:00
requirements.txt Adde sentry integration 2019-12-19 11:18:31 -05:00
setup.cfg init checkin 2019-11-07 15:40:14 -05:00
setup.py cleanup for docker build 2019-12-13 22:15:14 -05:00

README.md

CFR Personalization ETL Server

Setting up your dev enviroment:

To run any of the test cases that query GCP directly, you'll need to login to GCP using your own credentials.

From the command line - issue: gcloud auth application-default login

To build and startup the container - the simplest thing to do is to run

make all

Building the container

A docker file is provided to build the container. You can issue make build to create a local image.

Kinto authentication

The container is setup to use a default user with a username/password pair of : (devuser, devpass) against the kinto dev server.

Building the container

A standard Dockerfile is provided to build the container - the simplest thing to build the container is to issue: make build

Install the devuser and setup cfr-control, cfr-experiments and cfr-models

Use make run to spin up a testing container.

This will install initial data into the dev instance of Remote Settings at https://kinto.dev.mozaws.net/v1 and start writing out weight updates. Updates are currently set as a constant of 1 second updates to ease testing.

The run target will automatically mount your GCloud authentication credentials into the /app home directory in the container.