Граф коммитов

3548 Коммитов

Автор SHA1 Сообщение Дата
gtoonstra d8383038ac [AIRFLOW-591] Add datadog hook & sensor
Closes #1851 from gtoonstra/contrib_datadog
2016-11-14 07:21:08 +00:00
Jack Arenas f8f7d1af20 [AIRFLOW-561] Add RedshiftToS3Transfer operator
Add a redshift operator to unload tables from redshift into a single CSV in s3 with headers

Closes #1831 from jackar/redshift-to-s3-csv
2016-11-14 07:08:29 +00:00
Betsy Cannon 1944eee62e [AIRFLOW-570] Pass root to date form on gantt
Closes #1837 from forklady42/gantt-root
2016-11-14 06:25:11 +00:00
Bolke de Bruin 910c0ddd78 [AIRFLOW-504] Store fractional seconds in MySQL tables
Both utcnow() and now() return fractional seconds. These
are sometimes used in primary_keys (eg. in task_instance).
If MySQL is not configured to store these fractional seconds
a primary key might fail (eg. at session.merge) resulting in
a duplicate entry being added or worse.

Postgres does store fractional seconds if left unconfigured,
sqlite needs to be examined.
2016-11-13 22:43:17 +01:00
Bolke de Bruin d12ef6ffc0 [AIRFLOW-507] Use Travis' ubuntu trusty for CI
Travis' ubuntu trusty provides a more up to date
environment for CI. It allows for better testing
by integration more services like kerberos and
celery. Also it comes closer to actual production
environments (e.g. MySQL 5.6).
2016-11-13 22:43:16 +01:00
Wyndham Blanton 6adb15ca0c [AIRFLOW-623] LDAP attributes not always a list
Sometimes the search attributes that come back are
not a list always, but a string, so we need to
check for an exact match as well as if it's a list

Closes #1876 from wyndhblb/master
2016-11-12 12:59:03 -08:00
Giovanni Briggs 98f32184a6 [AIRFLOW-611] source_format in BigQueryBaseCursor
Check source_format in BigQueryBaseCursor

The edits to `bigquery_hook.py` are made to
`BigQueryBaseCursor`.

Closes #1873 from Jalepeno112/bug/AIRFLOW-611
2016-11-11 09:57:54 -08:00
Daniel Gies 868bc83137 [AIRFLOW-619] Fix exception in Gannt chart
If a TaskInstance has no end_date, use
datetime.now()

Closes #1874 from dgies/master
2016-11-11 10:08:27 +05:30
jlowin f192770038 [AIRFLOW-618] Cast DateTimes to avoid sqllite errors
For some reason, sqllite does not always match
datetimes correctly.
This causes unit test failures, in particular the
trigger_dagrun test.
Casting the datetime explicitly appears to solve
the issue.

Closes #1871 from jlowin/failing-sqlite-test
2016-11-07 19:23:24 +01:00
Alex Guziel 286c0478a6 [AIRFLOW-422] Add JSON endpoint for task info
Expose task information in an API endpoint for
usage in other tools.
This API is located in the experimental endpoints.

Dear Airflow Maintainers,

Please accept this PR that addresses the following
issues:
https://issues.apache.org/jira/browse/AIRFLOW-422

Testing Done:
- Added a simple unit test.

Expose task information in a web endpoint for
usage in other tools.

Closes #1740 from saguziel/master
2016-11-07 02:15:57 -05:00
jlowin fa977b60ad [AIRFLOW-616][AIRFLOW-617] Minor fixes to PR tool UX
[AIRFLOW-616] Store credentials for life of script

[AIRFLOW-617] Fix redundant prompt text in PR tool

Closes #1868 from jlowin/pr-tool-fixes
2016-11-04 15:43:19 -07:00
John Bodley 4fb86f920a [AIRFLOW-179] Fix DbApiHook with non-ASCII chars
String serialization fails when string contains non-ASCII characters

Closes #1553 from
johnbodley/dbapi_hook_serialization-remedy
2016-11-04 09:41:46 -04:00
Sumit Maheshwari 9ae583eb12 closes apache/incubator-airflow#1603 *PR abandonned by submitter* 2016-11-04 15:55:00 +05:30
Siddharth Anand f1bb8f33c2 closes apache/incubator-airflow#1538 *Not acceptable* 2016-11-03 23:24:38 -07:00
Sumit Maheshwari 0d3ed4d01f [AIRFLOW-566] Add timeout while fetching logs
Closes #1834 from msumit/AIRFLOW-566
2016-11-03 13:50:46 -07:00
Adam Stracener 26bab26ac6 [AIRFLOW-615] Set graph glyphicon first
Closes #1867 from NeckBeardPrince/glyphicon_resort
2016-11-03 11:35:14 -07:00
Li Xuanji 43a41b6ec1 [AIRFLOW-606] Add requirements.txt for airflow-pr
Closes #1859 from zodiac/dev_pr_requirements
2016-11-02 23:41:26 -07:00
gtoonstra ac6d42912d [AIRFLOW-609] Add application_name to PostgresHook
Closes #1861 from
gtoonstra/postgres_application_name
2016-11-02 23:30:41 -07:00
Siddharth Anand 04168f34a1 closes apache/incubator-airflow#1797 *closed for inactivity* 2016-11-02 22:42:10 -07:00
Martin Breining b8d739aaf4 [AIRFLOW-614][AIRFLOW-1] Add Madrone as Airflow user
Closes #1866 from mbreining/master
2016-11-02 22:33:41 -07:00
schnie 372e4c7a28 [AIRFLOW-613][AIRFLOW-1] Add Astronomer as Airflow user
Closes #1864 from schnie/master
2016-11-02 13:37:53 -07:00
Maxime Beauchemin e79ea3b8f0 [AIRFLOW-612] Move resources/articles links to wiki
Closes #1863 from mistercrunch/docs_links
2016-11-02 12:24:11 -07:00
Dmitriy Lee 26c2df33be [AIRFLOW-411] Add Python3 support to hipchat_operator
Closes #1721 from d-lee/hipchat_operator_python3
2016-11-02 14:36:15 -04:00
jedipi 39499e8aa8 [AIRFLOW-551] pin flask to >=0.11, <0.12
Closes #1825 from jedipi/improvement/upgrade-flask
2016-11-02 00:45:45 -07:00
jedipi 61f92b7f2b [AIRFLOW-552] upgrade funcsigs to 1.0.2
Closes #1826 from jedipi/improvements/upgrade-
funcsigs
2016-11-02 00:40:11 -07:00
gtoonstra 2daf7a8168 [AIRFLOW-590] Set module parameter in OracleHook
Closes #1848 from gtoonstra/oracle_action_module
2016-11-02 00:33:54 -07:00
Siddharth Anand 0b6ac66e43 closes apache/incubator-airflow#1692 *closed for inactivity* 2016-11-02 00:24:10 -07:00
Siddharth Anand 0448948742 closes apache/incubator-airflow#915 *closed for inactivity* 2016-11-02 00:20:28 -07:00
Siddharth Anand 730d353249 closes apache/incubator-airflow#1439 *fixed by other PR:#1760* 2016-11-02 00:18:47 -07:00
Siddharth Anand e26c2e4b1b closes apache/incubator-airflow#1497 *closed for inactivity* 2016-11-02 00:10:59 -07:00
Siddharth Anand 4f4285746c closes apache/incubator-airflow#1391 *closed for inactivity* 2016-11-02 00:09:39 -07:00
Bolke de Bruin 527e3ecdb8 [AIRFLOW-604] Revert .first() to .one()
.one() enforces the integrity of airflow as we
expect
a tuple to be returned here. If not the database
is
inconsistent and airflow should error out.

Dear Airflow Maintainers,

.one() enforces the integrity of airflow as we
expect
a tuple to be returned here. If not the database
is
inconsistent and airflow should error out.

partially reverts: https://github.com/apache
/incubator-airflow/pull/1730

Closes #1858 from bolkedebruin/AIRFLOW-604
2016-10-30 18:54:59 -04:00
jlowin 5616c029dc [AIRFLOW-370] Create AirflowConfigException in exceptions.py
AirflowConfigException should be created in
the exceptions utility file, not in
configuration.py.

All exceptions should be created in
`exceptions.py`.

https://issues.apache.org/jira/browse/AIRFLOW-370

Closes #1689 from jlowin/refactor-exception
2016-10-30 18:15:09 -04:00
Benjamin Tallman 1660aa024a [AIRFLOW-582] Fixes TI.get_dagrun filter (removes start_date)
Closes #1844 from btallman/ti_get_dagrun
2016-10-28 15:21:16 -07:00
Daniel Gies 6f4704a447 [AIRFLOW-568] Fix double task_stats count if a DagRun is active
Closes #1836 from dgies/master
2016-10-28 15:59:48 +02:00
Vijay Bhat 97934318bc [AIRFLOW-585] Fix race condition in backfill execution loop
A subtle race condition in the backfill execution
loop gives
rise to occasional deadlocks, causing Travis CI
builds to
randomly fail. The root cause is unsynchronized
access to
individual task instance states for a DAG run in
the execution
inner loop.

The fix involves atomically reading the state of
all task
instances for a DAG run once at the beginning of
every
iteration of the inner loop.

Closes #1846 from vijaysbhat/travis-ci-debugging
2016-10-28 15:50:38 +02:00
Dinesh Sharma 4a496fa928 [AIRFLOW-600] Added BandwidthX as a user of Airflow
Closes #1857 from dineshdsharma/master
2016-10-27 11:48:16 -07:00
Siddharth Anand 75c0eaeea6 closes apache/incubator-airflow#1491 *dupe of #1497* 2016-10-27 00:53:49 -07:00
robin_miller_ow 1aa241d840 [AIRFLOW-580] Prevent landscape warning on .format
Prevent landscape warning for use of .format in logging

Closes #1843 from robin-miller-ow/DisableLoggingSt
ringFormattingStyleWarningInLandscapeIO
2016-10-27 00:26:29 -07:00
George Leslie-Waksman 2f26126694 [AIRFLOW-597] Check if content is None, not false-equivalent
Closes #1856 from gwax/non_boolean_templates
2016-10-26 23:15:41 -07:00
Kengo Seki 46236fa707 [AIRFLOW-586] test_dag_v1 fails from 0 to 3 a.m.
dags/test_dag.py tries to set START_DATE to 3
hours before using
datetime.replace, but it doesn't support minus
value as argument.
So we have to use timedelta instead of simple
numeric subtraction.

Closes #1852 from sekikn/AIRFLOW-586
2016-10-26 19:47:35 -07:00
Sumit Maheshwari 2bbcdc3a99 [AIRFLOW-453] Add XCom Admin Page
Closes #1756 from msumit/AIRFLOW-453
2016-10-26 14:42:18 -07:00
Arthur Wiedmer 84cb7e8095 closes apache/incubator-airflow#1384 *obsolete* 2016-10-26 11:26:59 -07:00
Arthur Wiedmer c1747b7bb0 closes apache/incubator-airflow#1379 *obsolete* 2016-10-26 11:26:09 -07:00
Arthur Wiedmer 64d7e3fdeb closes apache/incubator-airflow#1301 *obsolete* 2016-10-26 11:23:39 -07:00
Arthur Wiedmer 1425d72468 closes apache/incubator-airflow#1274 *no movement from submitter* 2016-10-26 11:21:59 -07:00
Arthur Wiedmer f2f1b29a1b closes apache/incubator-airflow#1276 *no movement from submitter* 2016-10-26 11:21:28 -07:00
Arthur Wiedmer fb40505795 closes apache/incubator-airflow#989 *no movement from submitter* 2016-10-26 11:15:27 -07:00
Arthur Wiedmer 3c5d980822 closes apache/incubator-airflow#908 *no movement from submitter* 2016-10-26 11:14:18 -07:00
Maxime Beauchemin 8ec5f7f99b closes apache/incubator-airflow#746 *no movement from submitter* 2016-10-26 10:54:55 -07:00