Add 1.10.13 to CI, Breeze and Docs (#12652)

2020-11-27 13:35:28 +00:00 · 2020-11-27 13:35:28 +00:00 · 9a74ee5fff
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@ -340,7 +340,7 @@ jobs:
    runs-on: ubuntu-20.04
    needs: [build-info, ci-images]
    env:
-      INSTALL_AIRFLOW_VERSION: "1.10.12"
+      INSTALL_AIRFLOW_VERSION: "1.10.13"
      PYTHON_MAJOR_MINOR_VERSION: ${{needs.build-info.outputs.defaultPythonVersion}}
      BACKPORT_PACKAGES: "true"
      VERSION_SUFFIX_FOR_SVN: "rc1"
--- a/BREEZE.rst
+++ b/BREEZE.rst
@ -1206,8 +1206,8 @@ This is the current syntax for  `./breeze <./breeze>`_:
          If specified, installs Airflow directly from PIP released version. This happens at
          image building time in production image and at container entering time for CI image. One of:

-                 1.10.12 1.10.11 1.10.10 1.10.9 1.10.8 1.10.7 1.10.6 1.10.5 1.10.4 1.10.3 1.10.2
-                 wheel
+                 1.10.13 1.10.12 1.10.11 1.10.10 1.10.9 1.10.8 1.10.7 1.10.6 1.10.5 1.10.4 1.10.3
+                 1.10.2 wheel

          When 'wheel' is used, Airflow is installed from /dist/apache_airflow-*.whl file.

@ -2246,8 +2246,8 @@ This is the current syntax for  `./breeze <./breeze>`_:
          If specified, installs Airflow directly from PIP released version. This happens at
          image building time in production image and at container entering time for CI image. One of:

-                 1.10.12 1.10.11 1.10.10 1.10.9 1.10.8 1.10.7 1.10.6 1.10.5 1.10.4 1.10.3 1.10.2
-                 wheel
+                 1.10.13 1.10.12 1.10.11 1.10.10 1.10.9 1.10.8 1.10.7 1.10.6 1.10.5 1.10.4 1.10.3
+                 1.10.2 wheel

          When 'wheel' is used, Airflow is installed from /dist/apache_airflow-*.whl file.

--- a/IMAGES.rst
+++ b/IMAGES.rst
@ -39,7 +39,7 @@ The images are named as follows:

 where:

-* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``, ``v1-10-test``, ``1.10.12``
+* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``, ``v1-10-test``, ``1.10.13``
  The ``master`` and ``v1-10-test`` labels are built from branches so they change over time. The ``1.10.*`` and in
  the future ``2.*`` labels are build from git tags and they are "fixed" once built.
 * ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.5``, ``3.7``
@ -115,15 +115,15 @@ parameter to Breeze:
 .. code-block:: bash

  ./breeze build-image --python 3.7 --additional-extras=presto \
-      --production-image --install-airflow-version=1.10.12
+      --production-image --install-airflow-version=1.10.13

 This will build the image using command similar to:

 .. code-block:: bash

    pip install \
-      apache-airflow[async,aws,azure,celery,dask,elasticsearch,gcp,kubernetes,mysql,postgres,redis,slack,ssh,statsd,virtualenv,presto]==1.10.12 \
-      --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.6.txt"
+      apache-airflow[async,aws,azure,celery,dask,elasticsearch,gcp,kubernetes,mysql,postgres,redis,slack,ssh,statsd,virtualenv,presto]==1.10.13 \
+      --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.13/constraints-3.6.txt"

 You can also build production images from specific Git version via providing ``--install-airflow-reference``
 parameter to Breeze (this time constraints are taken from the ``constraints-master`` branch which is the
@ -210,8 +210,8 @@ For example:
  apache/airflow:master-python3.6                - production "latest" image from current master
  apache/airflow:master-python3.6-ci             - CI "latest" image from current master
  apache/airflow:v1-10-test-python2.7-ci         - CI "latest" image from current v1-10-test branch
-  apache/airflow:1.10.12-python3.6               - production image for 1.10.12 release
-  apache/airflow:1.10.12-1-python3.6             - production image for 1.10.12 with some patches applied
+  apache/airflow:1.10.13-python3.6               - production image for 1.10.13 release
+  apache/airflow:1.10.13-1-python3.6             - production image for 1.10.13 with some patches applied


 You can see DockerHub images at `<https://hub.docker.com/repository/docker/apache/airflow>`_
@ -292,7 +292,7 @@ additional apt dev and runtime dependencies.
    --build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
    --build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
    --build-arg AIRFLOW_INSTALL_SOURCES="apache-airflow" \
-    --build-arg AIRFLOW_INSTALL_VERSION="==1.10.12" \
+    --build-arg AIRFLOW_INSTALL_VERSION="==1.10.13" \
    --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1-10" \
    --build-arg AIRFLOW_SOURCES_FROM="empty" \
    --build-arg AIRFLOW_SOURCES_TO="/empty" \
@ -308,7 +308,7 @@ the same image can be built using ``breeze`` (it supports auto-completion of the
 .. code-block:: bash

  ./breeze build-image -f Dockerfile.ci \
-      --production-image  --python 3.7 --install-airflow-version=1.10.12 \
+      --production-image  --python 3.7 --install-airflow-version=1.10.13 \
      --additional-extras=jdbc --additional-python-deps="pandas" \
      --additional-dev-apt-deps="gcc g++" --additional-runtime-apt-deps="default-jre-headless"
 You can build the default production image with standard ``docker build`` command but they will only build
@ -326,7 +326,7 @@ based on example in `this comment <https://github.com/apache/airflow/issues/8605
    --build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
    --build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
    --build-arg AIRFLOW_INSTALL_SOURCES="apache-airflow" \
-    --build-arg AIRFLOW_INSTALL_VERSION="==1.10.12" \
+    --build-arg AIRFLOW_INSTALL_VERSION="==1.10.13" \
    --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1-10" \
    --build-arg AIRFLOW_SOURCES_FROM="empty" \
    --build-arg AIRFLOW_SOURCES_TO="/empty" \
--- a/README.md
+++ b/README.md
@ -77,7 +77,7 @@ Airflow is not a streaming solution, but it is often used to process real-time d

 Apache Airflow is tested with:

-|              | Master version (2.0.0dev) | Stable version (1.10.12) |
+|              | Master version (2.0.0dev) | Stable version (1.10.13) |
 | ------------ | ------------------------- | ------------------------ |
 | Python       | 3.6, 3.7, 3.8             | 2.7, 3.5, 3.6, 3.7, 3.8  |
 | PostgreSQL   | 9.6, 10, 11, 12, 13       | 9.6, 10, 11, 12, 13      |
@ -123,15 +123,15 @@ correct Airflow tag/version/branch and python versions in the URL.
 1. Installing just Airflow:

 ```bash
-pip install apache-airflow==1.10.12 \
- --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.7.txt"
+pip install apache-airflow==1.10.13 \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.13/constraints-3.7.txt"
 ```

 2. Installing with extras (for example postgres,google)

 ```bash
-pip install apache-airflow[postgres,google]==1.10.12 \
- --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.7.txt"
+pip install apache-airflow[postgres,google]==1.10.13 \
+ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.13/constraints-3.7.txt"
 ```

 For information on installing backport providers check [/docs/backport-providers.rst][/docs/backport-providers.rst].
--- a/1
+++ b/1
@ -49,6 +49,7 @@ _breeze_allowed_test_types="All Core Providers API CLI Integration Other WWW Hei
 }

 _breeze_allowed_install_airflow_versions=$(cat <<-EOF
+1.10.13
 1.10.12
 1.10.11
 1.10.10
--- a/docs/installation.rst
+++ b/docs/installation.rst
@ -32,7 +32,7 @@ version of libraries if needed. This means that from time to time plain ``pip in
 not work or will produce unusable Airflow installation.

 In order to have repeatable installation, however, starting from **Airflow 1.10.10** and updated in
-**Airflow 1.10.12** we also keep a set of "known-to-be-working" constraint files in the
+**Airflow 1.10.13** we also keep a set of "known-to-be-working" constraint files in the
 ``constraints-master`` and ``constraints-1-10`` orphan branches.
 Those "known-to-be-working" constraints are per major/minor python version. You can use them as constraint
 files when installing Airflow from PyPI. Note that you have to specify correct Airflow version
@ -48,22 +48,22 @@ and python versions in the URL.
      sudo apt-get install build-essential


-1. Installing just airflow
+1. Installing just Airflow

 .. code-block:: bash

-    AIRFLOW_VERSION=1.10.12
+    AIRFLOW_VERSION=1.10.13
    PYTHON_VERSION="$(python --version | cut -d " " -f 2 | cut -d "." -f 1-2)"
    # For example: 3.6
    CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
-    # For example: https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.6.txt
+    # For example: https://raw.githubusercontent.com/apache/airflow/constraints-1.10.13/constraints-3.6.txt
    pip install "apache-airflow==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"

 2. Installing with extras (for example postgres, google)

 .. code-block:: bash

-    AIRFLOW_VERSION=1.10.12
+    AIRFLOW_VERSION=1.10.13
    PYTHON_VERSION="$(python --version | cut -d " " -f 2 | cut -d "." -f 1-2)"
    CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
    pip install "apache-airflow[postgres,google]==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"
--- a/docs/production-deployment.rst
+++ b/docs/production-deployment.rst
@ -64,7 +64,7 @@ You should be aware, about a few things:

 .. code-block:: dockerfile

-  FROM apache/airflow:1.10.12
+  FROM apache/airflow:1.10.13
  USER root
  RUN apt-get update \
    && apt-get install -y --no-install-recommends \
@ -81,7 +81,7 @@ You should be aware, about a few things:

 .. code-block:: dockerfile

-  FROM apache/airflow:1.10.12
+  FROM apache/airflow:1.10.13
  RUN pip install --no-cache-dir --user my-awesome-pip-dependency-to-add


@ -92,7 +92,7 @@ You should be aware, about a few things:

 .. code-block:: dockerfile

-  FROM apache/airflow:1.10.12
+  FROM apache/airflow:1.10.13
  USER root
  RUN apt-get update \
    && apt-get install -y --no-install-recommends \
@ -125,7 +125,7 @@ in the `<#production-image-build-arguments>`_ chapter below.

 Here just a few examples are presented which should give you general understanding of what you can customize.

-This builds the production image in version 3.7 with additional airflow extras from 1.10.12 Pypi package and
+This builds the production image in version 3.7 with additional airflow extras from 1.10.13 PyPI package and
 additional apt dev and runtime dependencies.

 .. code-block:: bash
@ -134,7 +134,7 @@ additional apt dev and runtime dependencies.
    --build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
    --build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
    --build-arg AIRFLOW_INSTALL_SOURCES="apache-airflow" \
-    --build-arg AIRFLOW_INSTALL_VERSION="==1.10.12" \
+    --build-arg AIRFLOW_INSTALL_VERSION="==1.10.13" \
    --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1-10" \
    --build-arg AIRFLOW_SOURCES_FROM="empty" \
    --build-arg AIRFLOW_SOURCES_TO="/empty" \
@ -150,7 +150,7 @@ the same image can be built using ``breeze`` (it supports auto-completion of the
 .. code-block:: bash

  ./breeze build-image \
-      --production-image  --python 3.7 --install-airflow-version=1.10.12 \
+      --production-image  --python 3.7 --install-airflow-version=1.10.13 \
      --additional-extras=jdbc --additional-python-deps="pandas" \
      --additional-dev-apt-deps="gcc g++" --additional-runtime-apt-deps="default-jre-headless"

@ -166,7 +166,7 @@ based on example in `this comment <https://github.com/apache/airflow/issues/8605
    --build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
    --build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
    --build-arg AIRFLOW_INSTALL_SOURCES="apache-airflow" \
-    --build-arg AIRFLOW_INSTALL_VERSION="==1.10.12" \
+    --build-arg AIRFLOW_INSTALL_VERSION="==1.10.13" \
    --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1-10" \
    --build-arg AIRFLOW_SOURCES_FROM="empty" \
    --build-arg AIRFLOW_SOURCES_TO="/empty" \
@ -225,7 +225,7 @@ Preparing the constraint files and wheel files:

  pip download --dest docker-context-files \
    --constraint docker-context-files/constraints-1-10.txt  \
-    apache-airflow[async,aws,azure,celery,dask,elasticsearch,gcp,kubernetes,mysql,postgres,redis,slack,ssh,statsd,virtualenv]==1.10.12
+    apache-airflow[async,aws,azure,celery,dask,elasticsearch,gcp,kubernetes,mysql,postgres,redis,slack,ssh,statsd,virtualenv]==1.10.13


 Building the image (after copying the files downloaded to the "docker-context-files" directory:
@ -233,7 +233,7 @@ Building the image (after copying the files downloaded to the "docker-context-fi
 .. code-block:: bash

  ./breeze build-image \
-      --production-image --python 3.7 --install-airflow-version=1.10.12 \
+      --production-image --python 3.7 --install-airflow-version=1.10.13 \
      --disable-mysql-client-installation --disable-pip-cache --add-local-pip-wheels \
      --constraints-location="/docker-context-files/constraints-1-10.txt"

@ -245,7 +245,7 @@ or
    --build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
    --build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
    --build-arg AIRFLOW_INSTALL_SOURCES="apache-airflow" \
-    --build-arg AIRFLOW_INSTALL_VERSION="==1.10.12" \
+    --build-arg AIRFLOW_INSTALL_VERSION="==1.10.13" \
    --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1-10" \
    --build-arg AIRFLOW_SOURCES_FROM="empty" \
    --build-arg AIRFLOW_SOURCES_TO="/empty" \
@ -400,7 +400,7 @@ The following build arguments (``--build-arg`` in docker build command) can be u
 |                                          |                                          | ``constraints-master`` but can be        |
 |                                          |                                          | ``constraints-1-10`` for 1.10.* versions |
 |                                          |                                          | or it could point to specific version    |
-|                                          |                                          | for example ``constraints-1.10.12``      |
+|                                          |                                          | for example ``constraints-1.10.13``      |
 +------------------------------------------+------------------------------------------+------------------------------------------+
 | ``INSTALL_PROVIDERS_FROM_SOURCES``       | ``true``                                 | If set to false and image is built from  |
 |                                          |                                          | sources, all provider packages are not   |
@ -519,7 +519,7 @@ production image. There are three types of build:
 | ``AIRFLOW_INSTALL_VERSION``       | Optional - might be used for      |
 |                                   | package installation case to      |
 |                                   | set Airflow version for example   |
-|                                   | "==1.10.12"                       |
+|                                   | "==1.10.13"                       |
 +-----------------------------------+-----------------------------------+
 | ``AIRFLOW_CONSTRAINTS_REFERENCE`` | reference (branch or tag) from    |
 |                                   | GitHub where constraints file     |
@ -528,7 +528,7 @@ production image. There are three types of build:
 |                                   | ``constraints-1-10`` for 1.10.*   |
 |                                   | constraint or if you want to      |
 |                                   | point to specific version         |
-|                                   | might be ``constraints-1.10.12``  |
+|                                   | might be ``constraints-1.10.13``  |
 +-----------------------------------+-----------------------------------+
 | ``SLUGIFY_USES_TEXT_UNIDECODE``   | In case of of installing airflow  |
 |                                   | 1.10.2 or 1.10.1 you need to      |
@ -562,7 +562,7 @@ of 2.0 currently):

  docker build .

-This builds the production image in version 3.7 with default extras from 1.10.12 tag and
+This builds the production image in version 3.7 with default extras from 1.10.13 tag and
 constraints taken from constraints-1-10-12 branch in GitHub.

 .. code-block:: bash
@ -570,14 +570,14 @@ constraints taken from constraints-1-10-12 branch in GitHub.
  docker build . \
    --build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
    --build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
-    --build-arg AIRFLOW_INSTALL_SOURCES="https://github.com/apache/airflow/archive/1.10.12.tar.gz#egg=apache-airflow" \
+    --build-arg AIRFLOW_INSTALL_SOURCES="https://github.com/apache/airflow/archive/1.10.13.tar.gz#egg=apache-airflow" \
    --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1-10" \
    --build-arg AIRFLOW_BRANCH="v1-10-test" \
    --build-arg AIRFLOW_SOURCES_FROM="empty" \
    --build-arg AIRFLOW_SOURCES_TO="/empty"

-This builds the production image in version 3.7 with default extras from 1.10.12 Pypi package and
-constraints taken from 1.10.12 tag in GitHub and pre-installed pip dependencies from the top
+This builds the production image in version 3.7 with default extras from 1.10.13 PyPI package and
+constraints taken from 1.10.13 tag in GitHub and pre-installed pip dependencies from the top
 of v1-10-test branch.

 .. code-block:: bash
@ -586,14 +586,14 @@ of v1-10-test branch.
    --build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
    --build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
    --build-arg AIRFLOW_INSTALL_SOURCES="apache-airflow" \
-    --build-arg AIRFLOW_INSTALL_VERSION="==1.10.12" \
+    --build-arg AIRFLOW_INSTALL_VERSION="==1.10.13" \
    --build-arg AIRFLOW_BRANCH="v1-10-test" \
-    --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1.10.12" \
+    --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1.10.13" \
    --build-arg AIRFLOW_SOURCES_FROM="empty" \
    --build-arg AIRFLOW_SOURCES_TO="/empty"

-This builds the production image in version 3.7 with additional airflow extras from 1.10.12 Pypi package and
-additional python dependencies and pre-installed pip dependencies from 1.10.12 tagged constraints.
+This builds the production image in version 3.7 with additional airflow extras from 1.10.13 PyPI package and
+additional python dependencies and pre-installed pip dependencies from 1.10.13 tagged constraints.

 .. code-block:: bash

@ -601,15 +601,15 @@ additional python dependencies and pre-installed pip dependencies from 1.10.12 t
    --build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
    --build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
    --build-arg AIRFLOW_INSTALL_SOURCES="apache-airflow" \
-    --build-arg AIRFLOW_INSTALL_VERSION="==1.10.12" \
+    --build-arg AIRFLOW_INSTALL_VERSION="==1.10.13" \
    --build-arg AIRFLOW_BRANCH="v1-10-test" \
-    --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1.10.12" \
+    --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1.10.13" \
    --build-arg AIRFLOW_SOURCES_FROM="empty" \
    --build-arg AIRFLOW_SOURCES_TO="/empty" \
    --build-arg ADDITIONAL_AIRFLOW_EXTRAS="mssql,hdfs" \
    --build-arg ADDITIONAL_PYTHON_DEPS="sshtunnel oauth2client"

-This builds the production image in version 3.7 with additional airflow extras from 1.10.12 Pypi package and
+This builds the production image in version 3.7 with additional airflow extras from 1.10.13 PyPI package and
 additional apt dev and runtime dependencies.

 .. code-block:: bash
@ -618,7 +618,7 @@ additional apt dev and runtime dependencies.
    --build-arg PYTHON_BASE_IMAGE="python:3.7-slim-buster" \
    --build-arg PYTHON_MAJOR_MINOR_VERSION=3.7 \
    --build-arg AIRFLOW_INSTALL_SOURCES="apache-airflow" \
-    --build-arg AIRFLOW_INSTALL_VERSION="==1.10.12" \
+    --build-arg AIRFLOW_INSTALL_VERSION="==1.10.13" \
    --build-arg AIRFLOW_CONSTRAINTS_REFERENCE="constraints-1-10" \
    --build-arg AIRFLOW_SOURCES_FROM="empty" \
    --build-arg AIRFLOW_SOURCES_TO="/empty" \