We have recently seen a number of issues created by users who
tried to install airflow with poetry or pip-tools or who had
successes with using the latest pip 20.3.3. This change aims
to update the 'note' content and make sure installation
instructions are consistent everywhere, so that new users
are warned against using anything else than PIP and that they
are aware about potential problems with 'pip 20.3' and ways
to mitigate the problems.
This responds to the needs of confused users such as
one in https://github.com/apache/airflow/issues/13711#issuecomment-761694781
We cannot skip installing providers, but this causes
problems when installing airflow in editable mode, because providers
are in two places - in airflow sources and in provider packages.
This change removes installed provider packages when airflow
is installed in editable mode to mitigate the problem.
This way, there is no need to use INSTALL_PROVIDERS_FROM_SOURCES
variable when installing in editable mode.
We still need to keep INSTALL_PROVIDERS_FROM_SOURCES for cases when
non-editable mode is used. In this way one can easily install curent
version of provider packages locally with pip install and have
the latest sources of both airflow and providers installed.
Also INSTALL_PROVIDERS_FROM_SOURCES is particularly useful if you
develop a new provider and reinstall airflow - because otherwise
it will try to install the provider from a non-existing package.
This is why all regular CI jobs and Breeze have
INSTALL_PROVIDERS_FROM_SOURCES set by default.
So far, the production images of Airflow were using sources
when they were built on CI. This PR changes that, to build
airflow + providers packages first and install them
rather than use sources as installation mechanism.
Part of #12261
* Constraint files are now maintained automatically
* No need to generate requirements when setup.py changes
* requirements are kept in separate orphan branches not in main repo
* merges to master verify if latest requirements are working and
push tested requirements to orphaned branches
* we keep history of requirement changes and can label them
individually for each version (by constraint-1.10.n tag name)
* consistently changed all references to be 'constraints' not
'requirements'
This change introduces sub-commands in breeze tool.
It is much needed as we have many commands now
and it was difficult to separate commands from flags.
Also --help output was very long and unreadable.
With this change help it is much easier to discover
what breeze can do for you as well as navigate with it.
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Co-authored-by: Kamil Breguła <mik-laj@users.noreply.github.com>
Jarek Potiuk
Kaxil Naik
ashwinshankar77
dstandish
Xinbin Huang
Darren Weber
Kamil Breguła
Jarek Potiuk