Граф коммитов

6916 Коммитов

Автор SHA1 Сообщение Дата
Jarek Potiuk 6f6112e262 [AIRFLOW-5405] Fixed Unbound variable in force build 2019-09-04 23:07:35 +02:00
Tomek 9143893c99 [AIRFLOW-5347] Add system tests for GoogleCloudStorage (#5951) 2019-09-04 22:21:15 +02:00
kaxil 6fb821713f [AIRFLOW-XXX] Add Chagelog for 1.10.5 2019-09-04 15:35:01 +01:00
Felix Uellendall 9f16c68a5f [AIRFLOW-4391] Fix tooltip for None-State Tasks in 'Recent Tasks' (#5909) 2019-09-04 15:12:23 +01:00
Zacharya c9e2d04fde [AIRFLOW-4085] FileSensor now takes glob patterns for `filepath` (#5358) 2019-09-04 11:02:28 +01:00
Igor Khrol ba9e521e71 [AIRFLOW-5344] Add --proxy-user parameter to SparkSubmitOperator (#5948)
`spark2-submit` supports `--proxy-user` parameter which should be handled by SparkSubmitOperator.

```
$ spark2-submit --help 2>&1 | grep proxy
  --proxy-user NAME           User to impersonate when submitting the application.
```
2019-09-04 10:47:09 +01:00
Tomek Urbaszek 54fa1cf75e [AIRFLOW-XXX] Remove flake8 from PR template
Since we have pre-commit and flake8 on CI there seems to be no
reason to keep this bullet point in PR template.
2019-09-04 10:48:13 +02:00
Omkar Desai 13843151fb [Airflow-4668] Make airflow/contrib/utils Pylint compatible (#5916) 2019-09-04 09:23:57 +01:00
Tomek 6e53281af6 [AIRFLOW-5399] Add invoke operator for GCP Functions (#5995) 2019-09-04 09:07:45 +02:00
Tomek fa9e91227b [AIRFLOW-5386] Move Google Dataproc to core (#5986)
For more information check AIP-21.
2019-09-03 23:00:38 +02:00
Hao Liang 4af841c925 [Airflow-XXXX] Fix a typo 2019-09-03 20:58:59 +02:00
Hao Liang f497d1d5aa [AIRFLOW-4858] Deprecate "Historical convenience functions" in airflow.configuration (#5495)
1. Issue old conf method deprecation warnings properly and remove current old conf method usages.
2. Unify the way to use conf as `from airflow.configuration import conf`
2019-09-03 17:08:55 +01:00
Kamil Breguła c5c6448ff9 [AIRFLOW-5367] Remove hook initialization in ctor from BigtableTableWaitForReplicationSensor (#5974) 2019-09-03 15:38:26 +01:00
Jarek Potiuk 9d6b58d871 [AIRFLOW-5389] better organized scripts for building CI docker deps 2019-09-03 16:13:41 +02:00
Sergio Soto 16dad6a203 [AIRFLOW-XXX] Adds BlueKiri and Logitrabel to companies using Airflow (#5991)
[ci skip]
2019-09-03 13:57:47 +01:00
Ryan Yuan a9ba915799 [AIRFLOW-5129] Add typehint to GCP DLP hook (#5980) 2019-09-03 13:39:37 +02:00
Jarek Potiuk 319b80437c [AIRFLOW-5365] No need to do image rebuild when switching master/v1-10-test (#5972) 2019-09-03 11:20:25 +01:00
TobKed f46b54a10e [AIRFLOW-5360] Type annotations for BaseSensorOperator (#5966) 2019-09-03 11:36:26 +02:00
Felipe F. Barral 6060c2aca0 [AIRFLOW-XXX] Added Bexs Bank to list of Airflow Users (#5988) 2019-09-02 20:35:24 +01:00
Tomek 2ae5256df6 [AIRFLOW-5388] Add airflow version label to newly created buckets (#5987)
To tag and track GCP resources spawned from Airflow, we have
been adding airflow specific label(s) to GCP API service calls
whenever possible and applicable.
2019-09-02 20:54:55 +02:00
Fatma Ali 7d74a13194 [AIRFLOW-XXX] Added AMPATH Kenya to list of Airflow Users (#5983) 2019-09-02 18:53:52 +01:00
João A Silva bcd8390fbb [AIRFLOW-XXX] Add Kroton as official user (#5984) 2019-09-02 18:51:53 +01:00
Tomek c35b93218e [AIRFLOW-5384] Improve dst param info in FileToGCSOperator (#5985)
This commit add more info about dst parameter to indicate that the path
must include file name.
2019-09-02 18:50:47 +01:00
Jarek Potiuk a8dc50db4b [AIRFLOW-5371] Remove yamllint as prerequisite to run pre-commit 2019-09-02 11:37:32 -04:00
Jarek Potiuk 4d7bf965cc [AIRFLOW-5372] Apache license check runs locally on LICENCE changes
Licence check for RAT runs too often (every time pre-commit is modified) and it
should only be run (locally) when any of *LICEN[S|C]E* files change. We anyhow
run full check on CI so this is local optimisation (it runs too long while you
play with .pre-commit-config.yaml - and we will probably be able to detect some
problems locally as well when new modules are added.
2019-09-02 11:08:48 -04:00
Ryan Yuan 1158f57375 [AIRFLOW-XXXX] Update BigQuery View section URL to ViewDefinition (#5981) 2019-09-02 09:17:10 +02:00
Tomek 7ee2b99229 [AIRFLOW-5314] Create test for new import paths (#5920) 2019-09-01 02:51:28 +02:00
Jarek Potiuk 593dfb9134 [AIRFLOW-5364] Fix missing port numbers for local ci scripts 2019-08-31 09:49:26 -04:00
Jarek Potiuk 572465e020 [AIRFLOW-5363] Fixed building docs in breeze 2019-08-31 07:47:10 -04:00
Tomek 7a59358ffd [AIRFLOW-5356] Fix GCP Datastore unit tests (#5961)
This commit adds additional mocking for Datastore tests which
tests fallback_to_default_project_id decorator.
2019-08-30 18:17:28 +02:00
Kaxil Naik 6138208749
[AIRFLOW-5335] Simplify GCSHook test (#5958) 2019-08-30 16:50:54 +01:00
Kaxil Naik dc0018b2f2
[AIRFLOW-5357] Fix Content-Type for exported variables.json file (#5962) 2019-08-30 15:20:48 +01:00
Tomek bc5c8f32a1 [AIRFLOW-5320] Add system tests for PubSub (#5925) 2019-08-30 15:28:12 +02:00
Kamil Breguła 1b69a0b120
[AIRFLOW-5353] Simplify GKEClusterHook (#5960) 2019-08-30 14:26:48 +02:00
Hao Liang a670e5cd2c [Airflow-XXXX] Fix a comment error in _utils.sh 2019-08-29 22:58:33 -04:00
Kamil Breguła 7c0d9b46ce
[AIRFLOW-5303] Use project_id from GCP credentials (#5907) 2019-08-30 01:11:34 +02:00
Aizhamal Nurmamat kyzy 381aac5d75 [AIRFLOW-5148] Adding GA and privacy notice to website (#5930)
Co-authored-by: kaxil <kaxilnaik@gmail.com>
Co-authored-by: mik-laj <kamil.bregula@polidea.com>
2019-08-29 23:35:55 +01:00
Kaxil Naik 890adde7bd
[AIRFLOW-5350] Fix bug in the num_retires field in BigQueryHook (#5955) 2019-08-29 23:05:10 +01:00
Kaxil Naik b420ada05b
[AIRFLOW-5351] Move all GCP Cloud SQL tests in 1 file (#5956) 2019-08-29 23:04:00 +01:00
Tomek 5fb9972672 [AIRFLOW-5330] Add project_id to Datastore hook and operators (#5935) 2019-08-29 23:04:39 +02:00
Tomek 8eeedf538c [AIRFLOW-5346] Add system tests for GKECluster (#5947) 2019-08-29 22:54:30 +02:00
Tomek 88a94cb1bb [AIRFLOW-5341] Use more precise mock of time.sleep (#5950)
Some tests were mocking 'time.sleep' instead of using more precise
'airflow.module.path.time.sleep' mock.
2019-08-29 22:52:10 +02:00
Géraud 3d14a51cdb [AIRFLOW-4833] Allow to set Jinja env options in DAG declaration (#5943) 2019-08-29 20:06:17 +01:00
Shamin Meerankutty 7219ab2915 [AIRFLOW-XXX] Add FullContact to list of companies that use airflow (#5953) 2019-08-29 10:31:16 -07:00
Kaxil Naik b1d3d55da9
[AIRFLOW-5335] Update GCSHook methods so they need min IAM perms (#5939) 2019-08-29 16:40:33 +01:00
Tomek 3c7891983e [AIRFLOW-5340] Fix GCP DLP example (#5945)
Use do_xcom_push instead of deprecated xcom_push
2019-08-29 14:21:07 +01:00
Guy Maliar a8cb800662 [AIRFLOW-5345] Allow SqlSensor's hook to be customized by subclasses (#5946)
Due to security reasons we have an alternative PostgreSQL hook that fetches credentials dynamically.
We would still like to use the `SqlSensor` functionality as is as our hook is still backed by sqlalchemy.
By extracting SqlSensor's `connection.get_hook` into a method we can override the `_get_hook` method without affecting the functionality of the hook.
2019-08-29 13:24:30 +02:00
dependabot[bot] 5bfd7f481c [AIRFLOW-XXX] Bump mixin-deep from 1.3.1 to 1.3.2 in /airflow/www (#5941)
Bumps [mixin-deep](https://github.com/jonschlinkert/mixin-deep) from 1.3.1 to 1.3.2.
- [Release notes](https://github.com/jonschlinkert/mixin-deep/releases)
- [Commits](https://github.com/jonschlinkert/mixin-deep/compare/1.3.1...1.3.2)

Signed-off-by: dependabot[bot] <support@github.com>
2019-08-28 20:04:12 -07:00
Igor Dralyuk fd544eb0cb [AIRFLOW-5118] Add ability to specify optional components in DataprocClusterCreateOperator (#5821) 2019-08-28 21:12:30 +02:00
Michal Brys fb0910e847 [AIRFLOW-5275] Add support for template parameters in DataprocWorkflowTemplateInstantiateOperator (#5877) 2019-08-28 21:06:30 +02:00