Fixes these logs -- we only want one of them
[2020-06-15 12:19:38,673] 3613796 {{airflow.utils.db db.py:610}} INFO - Creating tables
[2020-06-15 12:19:38,673] 3613796 {{airflow.utils.db db.py:565}} INFO - Creating tables
* Resolve upstream tasks when template field is XComArg
closes: #8054
* fixup! Resolve upstream tasks when template field is XComArg
* Resolve task relations in DagRun and DagBag
* Add tests for serialized DAG
* Set dependencies only in bag_dag, refactor tests
* Traverse template_fields attribute
* Use provide_test_dag_bag in all tests
* fixup! Use provide_test_dag_bag in all tests
* Use metaclass + setattr
* Add prepare_for_execution method
* Check signature of __init__ not class
* Apply suggestions from code review
Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
* Update airflow/models/baseoperator.py
Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
* Wait for pipeline state in Data Fusion operators
fixup! Wait for pipeline state in Data Fusion operators
fixup! fixup! Wait for pipeline state in Data Fusion operators
fixup! fixup! fixup! Wait for pipeline state in Data Fusion operators
* Use quote to encode url parts
* fixup! Use quote to encode url parts
If you run DAG with `{"\"": ""}` configuration tree view will be broken:
```
tree:1 Uncaught SyntaxError: Unexpected string in JSON at position 806
at JSON.parse (<anonymous>)
at tree?dag_id=hightlight_test&num_runs=25:1190
```
JSON.parse is given incorrectly escaped json string.
In attempting to debug some other behaviour I discovered that we spawn a
"pool" of 1 process (main process, plus another) -- which makes it
hard to use a debugger against.
This changes it so that if a single process is configured that it is
just run directly, preserving the ability to step through with a
debugger
- use official isort pre-commit-hook
- use official yamllint pre-commit-hook
- run isort pre-commit-hook on all python files instead of files ending with py
* add connection schema with tests
* add endpoints for connection
* update patch
* update endpoint methods
* add readonly connection endpoints
* improve base schema and add tests
* update spec set connection id to string
* update type hint
* improve base schema
* add pre_load processing to return data to normal
* remove pre_load processing as it is not needed
* readonly endpoints
* improve code
* handle exception and improve code
* improve pagination test
* add nullable to spec and improve code
* remove base and add parameterized tests
* add pagination limit test
* Add generic CLI tool wrapper
* Pas working directory to container
* Share namespaces between all containers
* Fix permissions hack
* Unify code style
Co-authored-by: Felix Uellendall <feluelle@users.noreply.github.com>
* Detect standalone execution by checking symboli link
* User friendly error message when env var is missing
* Display error to stderr
* Display errors on stderr
* Fix permission hack
* Fix condition in if
* Fix missing env-file
* TEST: Install airflow without copying ssources
* Update scripts/ci/in_container/run_prepare_backport_readme.sh
Co-authored-by: Felix Uellendall <feluelle@users.noreply.github.com>
* Upload kind logs to Github Artifacts
* get_ci_environment needed to correctly dump kind logs
Without this the "setup cluster" step dumps the logs as
`kind_logs_2020-06-11_default_default.tar.gz`
Since this is now used outside of just building images, I have moved the
function to _initialization.sh
* Github already compresses artifacts for download.
If we upload a .tar.gz, the download is a .zip file containing that
.tar.gz, which isn't ideal
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
There is some DNS problem causing it to be unable to resolve github.com
in the container.
To unblock tests we are disabling the git-sync mode tests for the
moment.
These were passing for a while, but seemingly broke after _something_
outside of the airflow code base changed (DNS of Github Actions runners?
Phase of the moon) so adding this config map that we had when KIND
inside docker, rather than in the host.
Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
Co-authored-by: Daniel Imberman <daniel.imberman@gmail.com>
Kamil Breguła
Ash Berlin-Taylor
Mathew Wicks
Tomek Urbaszek
Igor Khrol
Kaxil Naik
Nathan Toups
James Timmins
Felix Uellendall
Chao-Han Tsai
Ephraim Anierobi
zikun