This PR adds an undocumented AWS-related operator
into the "Integration" section and fixes some
obsolete description.
Closes#3340 from sekikn/AIRFLOW-2446
MySqlTest and PostgresTest in
tests/operators/operators.py
have some unexecuted tests due to wrong naming
convention.
This PR fixes it.
Closes#3344 from sekikn/AIRFLOW-2449
[AIRFLOW-2424] Add dagrun status endpoint and
increase k8s test coverage
[AIRFLOW-2424] Added minikube fixes by @kimoonkim
[AIRFLOW-2424] modify endpoint to remove 'status'
Closes#3320 from dimberman/add-kubernetes-test
This PR fixes HiveCliHook.load_df to:
1. encode delimiter with the specified encoding
before passing it to pandas.DataFrame.to_csv
so as not to fail
2. flush output file by pandas.DataFrame.to_csv
before executing LOAD DATA statement
3. remove header and row index from output file
by pandas.DataFrame.to_csv so as to read it
as expected via Hive
Closes#3334 from sekikn/AIRFLOW-2441
Closes#3279 from feng-tao/reduce-tree-view
This introduces a new configuration variable to set the default
number of dag runs displayed in the tree view. For large DAGs, this
could cause timeouts in the webserver.
HiveCliHook.load_file doesn't actually execute
LOAD DATA statement via beeline bundled with
Hive under 2.0 due to HIVE-10541.
This PR provides a workaround for this problem.
Closes#3327 from sekikn/AIRFLOW-2412
ENH: Add the navigation bar color parameter
When operating multiple Airflow's (eg. Production,
Staging, etc.),
we cannot distinguish which Airflow is.
This feature enables us to distinguish which
Airflow you see
by the color of navigation bar.
Merge branch 'master' into add-navigation-bar-
color-parameter
Closes#2903 from Licht-T/add-navigation-bar-
color-parameter
The built-in email utils does not support
multibyte string content, for example,
Japanese or emojis. The fix is to add
mime_charset parameter to allow for other
values such as `utf-8`.
Closes#3308 from wolfier/AIRFLOW-1914
[AIRFLOW-1899] Add full deployment
- Made home directory configurable
- Documentation fix
- Add licenses
[AIRFLOW-1899] Tests for the Kubernetes Executor
Add an integration test for the Kubernetes
executor. Done by
spinning up different versions of kubernetes and
run a DAG
by invoking the REST API
Closes#3301 from Fokko/fix-kubernetes-executor
Three additional optional parameters to
DataprocClusterCreateOperator
which configure Cloud Dataproc Cluster Scheduled
Deletion features.
Closes#3217 from ebartkus/dataproc-ttl
This commit makes it possible to use jinja
templates when passing
JAR file URIs to the DataProc operators that
require JAR files,
specifically the DataProc Hive, Pig, SparkSql,
Spark, Hadoop and
PySpark operators.
Closes#3305 from mchalek/template-dataproc-jars
Alejandro Fernandez
Kevin McHale
lucianoviola
Kengo Seki
Daniel Imberman
Fokko Driesprong
Jin Hyuk Chang
Tao feng
Sergio Ballesteros
r39132
Jordan Zucker
James Meickle
Luke Bodeen
Giovanni Lanzani
Joy Gao
cclauss
Licht-T
Berislav Lopac
Kaxil Naik
Alan Ma
Allan Jiang
ebartkus
Kevin McHale
Richard Thai
Matthew Bowden