Граф коммитов

40 Коммитов

Автор SHA1 Сообщение Дата
Kaxil Naik 4e8f9cc8d0
Enable Black - Python Auto Formmatter (#9550) 2020-11-03 23:51:54 +00:00
Katsunori Kanda 349b0811c3
Add D200 pydocstyle check (#11688) 2020-10-20 22:33:54 +02:00
Matt Buell 4d03e33c11
[AIRFLOW-6817] remove imports from `airflow/__init__.py`, replaced implicit imports with explicit imports, added entry to `UPDATING.MD` - squashed/rebased (#7456) 2020-02-22 08:21:19 +01:00
Kaxil Naik a311d3d82e
[AIRFLOW-6718] Fix more occurrences of utils.dates.days_ago (#7341) 2020-02-03 03:25:49 +05:30
Kamil Breguła 97a429f9d0
[AIRFLOW-6714] Remove magic comments about UTF-8 (#7338) 2020-02-02 22:18:19 +01:00
Kevin Yang d719e1fd67 [AIRFLOW-5362] Reorder imports (#5944) 2019-10-02 16:30:03 +01:00
Felix Uellendall 571ffcc97f [AIRFLOW-4686] Make dags Pylint compatible (#5753) 2019-08-13 03:26:50 +02:00
Fokko Driesprong d494d5a656 [AIRFLOW-2918] Fix Flake8 violations (#3772)
- Unused imports
- Wrong import order
- Small identation fixes
- Remove one letter variables
- Fix noqa annotations
2018-08-20 23:44:36 +01:00
Bolke de Bruin c7a472ed6b [AIRFLOW-2287] Fix incorrect ASF headers
Closes #3219 from bolkedebruin/fix_header
2018-04-14 09:13:23 +02:00
AllisonWang b49986c3b2 [Airflow 1332] Split logs based on try number
This PR splits logs based on try number and add
tabs to display different task instance tries.

**Note this PR is a temporary change for
separating task attempts. The code in this PR will
be refactored in the future. Please refer to #2422
for Airflow logging abstractions redesign.**

Testing:
1. Added unit tests.
2. Tested on localhost.
3. Tested on production environment with S3 remote
storage, MySQL database, Redis, one Airflow
scheduler and two airflow workers.

Closes #2383 from AllisonWang/allison--add-task-
attempt
2017-07-20 18:08:18 -07:00
Siddharth Anand c64e876bd5 [AIRFLOW-1054] Fix broken import in test_dag
Closes #2201 from
r39132/fix_broken_import_on_test_dag
2017-04-03 13:10:51 -07:00
Jeremiah Lowin bbfd43df46 [AIRFLOW-863] Example DAGs should have recent start dates
Avoid unnecessary backfills by having start dates
of
just a few days ago. Adds a utility function
airflow.utils.dates.days_ago().

Closes #2068 from jlowin/example-start-date
2017-02-12 15:37:56 -05:00
Kengo Seki 46236fa707 [AIRFLOW-586] test_dag_v1 fails from 0 to 3 a.m.
dags/test_dag.py tries to set START_DATE to 3
hours before using
datetime.replace, but it doesn't support minus
value as argument.
So we have to use timedelta instead of simple
numeric subtraction.

Closes #1852 from sekikn/AIRFLOW-586
2016-10-26 19:47:35 -07:00
Siddharth Anand 62768bc087 [AIRFLOW-399] - Remove dags/testdruid.py
Remove the current example in dags/testdruid.py. It requires the installation of an extraneous library (Hive) and executes a query against table that no one has. In its place, I am creating a simple DAG that only depends on DummyOperator and standard Python packages.

Dear Airflow Maintainers,

Please accept this PR that addresses the following issues:
- https://issues.apache.org/jira/browse/AIRFLOW-399

Testing Done:
Manual.

aoen criccomini artwr jlowin bolkedebruin

Closes #1709 from r39132/master
2016-08-08 09:23:49 -07:00
Ajay Yadav 3ffa656d97 [AIRFLOW-248] Add Apache license header to all files
- Added Apache license header for files with extension (.service, .in, .mako, .properties, .ini, .sh, .ldif, .coveragerc, .cfg, .yml, .conf, .sql, .css, .js, .html, .xml.
- Added/Replaced shebang on all .sh files with portable version - #!/usr/bin/env bash.
- Skipped third party css and js files. Skipped all minified js files as well.

Closes #1598 from ajayyadava/248
2016-06-21 08:15:42 -07:00
jlowin 851adc5547 [AIRFLOW-31] Use standard imports for hooks/operators 2016-06-16 14:55:07 -04:00
Qi Wang da43a94219 adding druid hook and operator 2015-07-13 19:17:13 +00:00
Maxime f991783b11 Changing configuration scheme 2015-01-16 14:31:54 -08:00
Maxime Beauchemin e8d005058b depends_on_past backfill fix 2014-12-17 08:33:41 -08:00
Maxime Beauchemin e23f9eea65 Adding email functionality 2014-12-12 08:06:39 +00:00
Maxime Beauchemin 6f91a67ada Pessimistic pool connection handling for master and run 2014-11-30 14:59:33 -08:00
Krishna Puttaswamy d093c705ea Merge pull request #19 from mistercrunch/start_date_fix
a task needs to have start_date; either from the dag or the task
2014-11-25 23:00:25 -08:00
Maxime Beauchemin 33c39b912e Debug master scheduler, added file timestamp based dagbag update 2014-11-25 12:11:03 -08:00
Krishna Puttaswamy 8b3bfe500e a task needs to have start_date; either from the dag or the task 2014-11-21 17:24:12 -08:00
Krishna Puttaswamy 0a673b4cc0 integrating config parser to read config values from a file 2014-11-07 13:58:38 -08:00
Krishna Puttaswamy 25cb7264b9 Providing a way to specify the executor to use while constructing the DAG object
adding host=0.0.0.0 to ensure the service is accesible from outside localhost

Revert "adding host=0.0.0.0 to ensure the service is accesible from outside localhost"

This reverts commit ce5436bc7dcd70dcb129c1dfa46f9fd7ce743e43.
2014-11-07 13:39:38 -08:00
Maxime Beauchemin 162acc4755 Syncing 2014-11-05 18:40:12 +00:00
Maxime Beauchemin dc9537df63 Making Jinja templates as files work 2014-11-02 05:26:36 +00:00
Maxime Beauchemin 35a5864da7 Making a generic SqlSensor, works for all db_ids 2014-11-01 16:32:31 +00:00
Maxime Beauchemin b04c6b85a0 Merge branch 'master' of github.com:mistercrunch/Airflow 2014-10-29 09:12:46 -07:00
Maxime Beauchemin 4d20e3a082 Tree view, repeated nodes are now collapsed by default 2014-10-29 08:31:33 -07:00
Maxime Beauchemin 0d56f3641a Fixed dag view to not repeat edges 2014-10-28 00:31:28 +00:00
Krishna Puttaswamy 67020e534e first version of the celery executor 2014-10-25 21:54:31 -07:00
Maxime Beauchemin eab8db7bf5 Fixed a failing example 2014-10-24 22:19:36 +00:00
Maxime Beauchemin 88c99624f3 Renaming project from Flux to Airflow 2014-10-14 18:28:29 -07:00
Maxime Beauchemin 323571ebd1 Minor touchups 2014-10-14 14:55:41 -07:00
Maxime Beauchemin 1f6f53872f Added new ExternalTaskSensor operator, other minor fixes 2014-10-08 13:51:19 -07:00
Maxime Beauchemin 1f890d0ae3 Moving folders around 2014-10-07 23:33:00 -07:00
Maxime Beauchemin 86e4c01d3c Pickling dags and transfering pickles through the database 2014-10-07 21:15:21 -07:00
Maxime Beauchemin 1047940ca4 First official commit 2014-10-06 14:29:38 -07:00