bugbug/infra/dockerfile.spawn_pipeline

FROM python:3.7.3-slim

# Setup dependencies in a cacheable step
ADD spawn_data_pipeline_requirements.txt /code/

RUN pip install --disable-pip-version-check --no-cache-dir -r /code/spawn_data_pipeline_requirements.txt

ADD spawn_data_pipeline.py /code

ADD data-pipeline.yml /code
ADD check-pipeline.yml /code
ADD annotate-pipeline.yml /code

CMD python /code/spawn_data_pipeline.py
Add basic check method and check script (#341) * Add basic check method and check script * Ensure the check of component will correctly use super result * Add required infra to schedule model checks * Add scheduling bits for the model checks * Remove the filtering on classification * Extract counting bugs to a new function in bugzilla.py * Also checks conflated components * Fix new hook id * Call bugzilla with the count_only param to speed up the check * Fix the new hook scope to match the hook id * Fix component model check after previous refactoring * Fix component model check method * Use a bugzilla report for even faster component model check * Clarify get_product_component_count docstring We are already filtering out full component with 0 bugs * Update conflated components mapping check A conflated component could also be part of the conflated components mapping * Distinguish between non-existing full components and empty full components * Remove the filter on resolution and unnecessary url params * Update component check method Keep checks as separate as possible for clarity, we could merge them or makes them faster later * Generate dynamically the CSV report url * Fix Docker image name the hook * Implement component check number 5 Get the meaningful components for the last 6 months * Handle reviews comments * Remove extraneous print * Removes TODO * Use a different threshold ration when checking for new meaningful components As we are only checking new bugs for 6 months, adjust the threshold ration to be less sensitive to occasional burst ob bugs for q given component. * Reduce the threshold ratio As we check on a disjoint time window, reduce the chance of false positives * Handle review nits * Fix last nits 2019-05-10 13:20:23 +03:00			`FROM python:3.7.3-slim`

Add a pre-commit hook using codespell (#411) * Add a pre-commit hook using codespell Fixes #410 * Fix some spelling mistakes 2019-05-16 18:24:18 +03:00			`# Setup dependencies in a cacheable step`
Add basic check method and check script (#341) * Add basic check method and check script * Ensure the check of component will correctly use super result * Add required infra to schedule model checks * Add scheduling bits for the model checks * Remove the filtering on classification * Extract counting bugs to a new function in bugzilla.py * Also checks conflated components * Fix new hook id * Call bugzilla with the count_only param to speed up the check * Fix the new hook scope to match the hook id * Fix component model check after previous refactoring * Fix component model check method * Use a bugzilla report for even faster component model check * Clarify get_product_component_count docstring We are already filtering out full component with 0 bugs * Update conflated components mapping check A conflated component could also be part of the conflated components mapping * Distinguish between non-existing full components and empty full components * Remove the filter on resolution and unnecessary url params * Update component check method Keep checks as separate as possible for clarity, we could merge them or makes them faster later * Generate dynamically the CSV report url * Fix Docker image name the hook * Implement component check number 5 Get the meaningful components for the last 6 months * Handle reviews comments * Remove extraneous print * Removes TODO * Use a different threshold ration when checking for new meaningful components As we are only checking new bugs for 6 months, adjust the threshold ration to be less sensitive to occasional burst ob bugs for q given component. * Reduce the threshold ratio As we check on a disjoint time window, reduce the chance of false positives * Handle review nits * Fix last nits 2019-05-10 13:20:23 +03:00			`ADD spawn_data_pipeline_requirements.txt /code/`

			`RUN pip install --disable-pip-version-check --no-cache-dir -r /code/spawn_data_pipeline_requirements.txt`

			`ADD spawn_data_pipeline.py /code`

Use a single spawn image for both hooks (#594) This removes the need to keep two different Docker images for spawning the data pipeline and the check pipeline. Fixes #365 2019-06-14 19:54:21 +03:00			`ADD data-pipeline.yml /code`
Add basic check method and check script (#341) * Add basic check method and check script * Ensure the check of component will correctly use super result * Add required infra to schedule model checks * Add scheduling bits for the model checks * Remove the filtering on classification * Extract counting bugs to a new function in bugzilla.py * Also checks conflated components * Fix new hook id * Call bugzilla with the count_only param to speed up the check * Fix the new hook scope to match the hook id * Fix component model check after previous refactoring * Fix component model check method * Use a bugzilla report for even faster component model check * Clarify get_product_component_count docstring We are already filtering out full component with 0 bugs * Update conflated components mapping check A conflated component could also be part of the conflated components mapping * Distinguish between non-existing full components and empty full components * Remove the filter on resolution and unnecessary url params * Update component check method Keep checks as separate as possible for clarity, we could merge them or makes them faster later * Generate dynamically the CSV report url * Fix Docker image name the hook * Implement component check number 5 Get the meaningful components for the last 6 months * Handle reviews comments * Remove extraneous print * Removes TODO * Use a different threshold ration when checking for new meaningful components As we are only checking new bugs for 6 months, adjust the threshold ration to be less sensitive to occasional burst ob bugs for q given component. * Reduce the threshold ratio As we check on a disjoint time window, reduce the chance of false positives * Handle review nits * Fix last nits 2019-05-10 13:20:23 +03:00			`ADD check-pipeline.yml /code`
Add annotate-pipeline.yml file to the spawn_pipeline Docker image 2019-06-26 20:45:22 +03:00			`ADD annotate-pipeline.yml /code`
Add basic check method and check script (#341) * Add basic check method and check script * Ensure the check of component will correctly use super result * Add required infra to schedule model checks * Add scheduling bits for the model checks * Remove the filtering on classification * Extract counting bugs to a new function in bugzilla.py * Also checks conflated components * Fix new hook id * Call bugzilla with the count_only param to speed up the check * Fix the new hook scope to match the hook id * Fix component model check after previous refactoring * Fix component model check method * Use a bugzilla report for even faster component model check * Clarify get_product_component_count docstring We are already filtering out full component with 0 bugs * Update conflated components mapping check A conflated component could also be part of the conflated components mapping * Distinguish between non-existing full components and empty full components * Remove the filter on resolution and unnecessary url params * Update component check method Keep checks as separate as possible for clarity, we could merge them or makes them faster later * Generate dynamically the CSV report url * Fix Docker image name the hook * Implement component check number 5 Get the meaningful components for the last 6 months * Handle reviews comments * Remove extraneous print * Removes TODO * Use a different threshold ration when checking for new meaningful components As we are only checking new bugs for 6 months, adjust the threshold ration to be less sensitive to occasional burst ob bugs for q given component. * Reduce the threshold ratio As we check on a disjoint time window, reduce the chance of false positives * Handle review nits * Fix last nits 2019-05-10 13:20:23 +03:00
Use a single spawn image for both hooks (#594) This removes the need to keep two different Docker images for spawning the data pipeline and the check pipeline. Fixes #365 2019-06-14 19:54:21 +03:00			`CMD python /code/spawn_data_pipeline.py`