Minor improvements to dev/README.md (#11525)

This commit is contained in:
Kaxil Naik 2020-10-14 15:23:33 +01:00 коммит произвёл GitHub
Родитель 6c8cf6aebe
Коммит 2509d13096
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
1 изменённых файлов: 86 добавлений и 50 удалений

Просмотреть файл

@ -156,9 +156,9 @@ You can add and copy the tokens here:
* [Prod PyPI](https://pypi.org/manage/account/token/) * [Prod PyPI](https://pypi.org/manage/account/token/)
Create a ~/.pypirc file: Create a `~/.pypirc` file:
```shell script ```ini
[distutils] [distutils]
index-servers = index-servers =
pypi pypi
@ -172,7 +172,6 @@ password=<API Upload Token>
repository=https://test.pypi.org/legacy/ repository=https://test.pypi.org/legacy/
username=__token__ username=__token__
password=<API Upload Token> password=<API Upload Token>
``` ```
Set proper permissions for the pypirc file: Set proper permissions for the pypirc file:
@ -193,7 +192,7 @@ pip install twine
- Set proper permissions for the pypirc file: - Set proper permissions for the pypirc file:
`$ chmod 600 ~/.pypirc` `$ chmod 600 ~/.pypirc`
- Confirm that airflow/version.py is set properly. - Confirm that `airflow/version.py` is set properly.
## Hardware used to prepare and verify the packages ## Hardware used to prepare and verify the packages
@ -213,52 +212,67 @@ prepared using such hardware. More information can be found in this
The Release Candidate artifacts we vote upon should be the exact ones we vote against, without any modification than renaming – i.e. the contents of the files must be the same between voted release canidate and final release. Because of this the version in the built artifacts that will become the official Apache releases must not include the rcN suffix. The Release Candidate artifacts we vote upon should be the exact ones we vote against, without any modification than renaming – i.e. the contents of the files must be the same between voted release canidate and final release. Because of this the version in the built artifacts that will become the official Apache releases must not include the rcN suffix.
- Set environment variables - Set environment variables
```
# Set Version ```shell script
export VERSION=1.10.2rc3 # Set Version
export VERSION=1.10.2rc3
# Set AIRFLOW_REPO_ROOT to the path of your git repo # Set AIRFLOW_REPO_ROOT to the path of your git repo
export AIRFLOW_REPO_ROOT=$(pwd) export AIRFLOW_REPO_ROOT=$(pwd)
# Example after cloning # Example after cloning
git clone https://github.com/apache/airflow.git airflow git clone https://github.com/apache/airflow.git airflow
cd airflow cd airflow
export AIRFLOW_REPO_ROOT=$(pwd) export AIRFLOW_REPO_ROOT=$(pwd)
``` ```
- set your version to 1.10.2 in airflow/version.py (without the RC tag) - Set your version to 1.10.2 in `airflow/version.py` (without the RC tag)
- Commit the version change. - Commit the version change.
- Tag your release - Tag your release
`git tag ${VERSION}` ```shell script
git tag ${VERSION}
```
- Clean the checkout: the sdist step below will - Clean the checkout: the sdist step below will
`git clean -fxd`
```shell script
git clean -fxd
```
- Tarball the repo - Tarball the repo
`git archive --format=tar.gz ${VERSION} --prefix=apache-airflow-${VERSION}/ -o apache-airflow-${VERSION}-source.tar.gz`
```shell script
git archive --format=tar.gz ${VERSION} --prefix=apache-airflow-${VERSION}/ -o apache-airflow-${VERSION}-source.tar.gz`
```
- Generate sdist - Generate sdist
NOTE: Make sure your checkout is clean at this stage - any untracked or changed files will otherwise be included in the file produced. NOTE: Make sure your checkout is clean at this stage - any untracked or changed files will otherwise be included
in the file produced.
`python setup.py compile_assets sdist bdist_wheel` ```shell script
python setup.py compile_assets sdist bdist_wheel
```
- Rename the sdist - Rename the sdist
```
mv dist/apache-airflow-${VERSION%rc?}.tar.gz apache-airflow-${VERSION}-bin.tar.gz ```shell script
mv dist/apache_airflow-${VERSION%rc?}-py2.py3-none-any.whl apache_airflow-${VERSION}-py2.py3-none-any.whl mv dist/apache-airflow-${VERSION%rc?}.tar.gz apache-airflow-${VERSION}-bin.tar.gz
``` mv dist/apache_airflow-${VERSION%rc?}-py2.py3-none-any.whl apache_airflow-${VERSION}-py2.py3-none-any.whl
```
- Generate SHA512/ASC (If you have not generated a key yet, generate it by following instructions on http://www.apache.org/dev/openpgp.html#key-gen-generate-key) - Generate SHA512/ASC (If you have not generated a key yet, generate it by following instructions on http://www.apache.org/dev/openpgp.html#key-gen-generate-key)
```
${AIRFLOW_REPO_ROOT}/dev/sign.sh apache-airflow-${VERSION}-source.tar.gz ```shell script
${AIRFLOW_REPO_ROOT}/dev/sign.sh apache-airflow-${VERSION}-bin.tar.gz ${AIRFLOW_REPO_ROOT}/dev/sign.sh apache-airflow-${VERSION}-source.tar.gz
${AIRFLOW_REPO_ROOT}/dev/sign.sh apache_airflow-${VERSION}-py2.py3-none-any.whl ${AIRFLOW_REPO_ROOT}/dev/sign.sh apache-airflow-${VERSION}-bin.tar.gz
``` ${AIRFLOW_REPO_ROOT}/dev/sign.sh apache_airflow-${VERSION}-py2.py3-none-any.whl
```
- Push Tags - Push Tags
`git push --tags` `git push --tags`
@ -284,16 +298,25 @@ svn commit -m "Add artifacts for Airflow ${VERSION}"
At this point we have the artefact that we vote on, but as a convenience to developers we also want to At this point we have the artefact that we vote on, but as a convenience to developers we also want to
publish "snapshots" of the RC builds to pypi for installing via pip. To do this we need to publish "snapshots" of the RC builds to pypi for installing via pip. To do this we need to
- Edit the airflow/version.py to include the RC suffix. - Edit the `airflow/version.py` to include the RC suffix.
- Build the package: - Build the package:
`python setup.py compile_assets sdist bdist_wheel`
```shell script
python setup.py compile_assets sdist bdist_wheel
```
- Verify the artifacts that would be uploaded: - Verify the artifacts that would be uploaded:
`twine check dist/*`
```shell script
twine check dist/*
```
- Upload the package to PyPi's test environment: - Upload the package to PyPi's test environment:
`twine upload -r pypitest dist/*`
```shell script
twine upload -r pypitest dist/*
```
- Verify that the test package looks good by downloading it and installing it into a virtual environment. The package download link is available at: - Verify that the test package looks good by downloading it and installing it into a virtual environment. The package download link is available at:
https://test.pypi.org/project/apache-airflow/#files https://test.pypi.org/project/apache-airflow/#files
@ -549,13 +572,13 @@ downloaded from the SVN).
There is also an easy way of installation with Breeze if you have the latest sources of Apache Airflow. There is also an easy way of installation with Breeze if you have the latest sources of Apache Airflow.
Running the following command will use tmux inside breeze, create `admin` user and run Webserver & Scheduler: Running the following command will use tmux inside breeze, create `admin` user and run Webserver & Scheduler:
``` ```shell script
./breeze start-airflow --install-airflow-version <VERSION>rc<X> --python 3.7 --backend postgres ./breeze start-airflow --install-airflow-version <VERSION>rc<X> --python 3.7 --backend postgres
``` ```
For 1.10 releases you can also use `--no-rbac-ui` flag disable RBAC UI of Airflow: For 1.10 releases you can also use `--no-rbac-ui` flag disable RBAC UI of Airflow:
``` ```shell script
./breeze start-airflow --install-airflow-version <VERSION>rc<X> --python 3.7 --backend postgres --no-rbac-ui ./breeze start-airflow --install-airflow-version <VERSION>rc<X> --python 3.7 --backend postgres --no-rbac-ui
``` ```
@ -642,27 +665,41 @@ Verify that the packages appear in [airflow](https://dist.apache.org/repos/dist/
At this point we release an official package: At this point we release an official package:
- Build the package: - Build the package:
`python setup.py compile_assets sdist bdist_wheel`
```shell script
python setup.py compile_assets sdist bdist_wheel`
```
- Verify the artifacts that would be uploaded: - Verify the artifacts that would be uploaded:
`twine check dist/*`
```shell script
twine check dist/*`
```
- Upload the package to PyPi's test environment: - Upload the package to PyPi's test environment:
`twine upload -r pypitest dist/*`
```shell script
twine upload -r pypitest dist/*
```
- Verify that the test package looks good by downloading it and installing it into a virtual environment. - Verify that the test package looks good by downloading it and installing it into a virtual environment.
The package download link is available at: https://test.pypi.org/project/apache-airflow/#files The package download link is available at: https://test.pypi.org/project/apache-airflow/#files
- Upload the package to PyPi's production environment: - Upload the package to PyPi's production environment:
`twine upload -r pypi dist/*`
- Again, confirm that the package is available here: ```shell script
https://pypi.python.org/pypi/apache-airflow twine upload -r pypi dist/*
```
- Again, confirm that the package is available here: https://pypi.python.org/pypi/apache-airflow
### Update CHANGELOG.md ### Update CHANGELOG.md
- Get a diff between the last version and the current version: - Get a diff between the last version and the current version:
`$ git log 1.8.0..1.9.0 --pretty=oneline`
```shell script
$ git log 1.8.0..1.9.0 --pretty=oneline
```
- Update CHANGELOG.md with the details, and commit it. - Update CHANGELOG.md with the details, and commit it.
### Notify developers of release ### Notify developers of release
@ -753,14 +790,14 @@ Details about maintaining the SEMVER version are going to be discussed and imple
Prepare release notes for all the packages you plan to release. Where YYYY.MM.DD is the CALVER Prepare release notes for all the packages you plan to release. Where YYYY.MM.DD is the CALVER
date for the packages. date for the packages.
``` ```shell script
./breeze --backports prepare-provider-readme YYYY.MM.DD [packages] ./breeze --backports prepare-provider-readme YYYY.MM.DD [packages]
``` ```
If you iterate with merges and release candidates you can update the release date without providing If you iterate with merges and release candidates you can update the release date without providing
the date (to update the existing release notes) the date (to update the existing release notes)
``` ```shell script
./breeze --backports prepare-provider-readme google ./breeze --backports prepare-provider-readme google
``` ```
@ -788,9 +825,8 @@ export AIRFLOW_REPO_ROOT=$(pwd)
* Build the source package: * Build the source package:
``` ```shell script
./provider_packages/build_source_package.sh ./provider_packages/build_source_package.sh
``` ```
It will generate `apache-airflow-backport-providers-${VERSION}-source.tar.gz` It will generate `apache-airflow-backport-providers-${VERSION}-source.tar.gz`
@ -1154,7 +1190,7 @@ backport packages. This is especially helpful when you want to test integrations
additional tools. Below is an example Dockerfile, which installs backport providers for Google and additional tools. Below is an example Dockerfile, which installs backport providers for Google and
an additional third-party tools: an additional third-party tools:
``` ```dockerfile
FROM apache/airflow:1.10.12 FROM apache/airflow:1.10.12
RUN pip install --user apache-airflow-backport-providers-google==2020.10.5.rc1 RUN pip install --user apache-airflow-backport-providers-google==2020.10.5.rc1
@ -1174,7 +1210,7 @@ USER ${AIRFLOW_UID}
To build an image build and run a shell, run: To build an image build and run a shell, run:
``` ```shell script
docker build . -t my-airflow docker build . -t my-airflow
docker run -ti \ docker run -ti \
--rm \ --rm \
@ -1415,7 +1451,7 @@ Details about maintaining the SEMVER version are going to be discussed and imple
[the related issue](https://github.com/apache/airflow/issues/11425) [the related issue](https://github.com/apache/airflow/issues/11425)
``` ```shell script
./breeze prepare-provider-readme [packages] ./breeze prepare-provider-readme [packages]
``` ```