Граф коммитов

10078 Коммитов

Автор SHA1 Сообщение Дата
eladkal 4fd81f52ff
Add howto doc for Salesforce connection (#10482) 2020-08-31 19:50:10 +02:00
Ephraim Anierobi aa2db70494
Unify error messages and complete type field in response (#10333)
Co-authored-by: Kamil Breguła <mik-laj@users.noreply.github.com>
2020-08-31 15:36:52 +02:00
Marco Aguiar e6a0a5374d
Display conf as a JSON in the DagRun list view (#10644)
Co-authored-by: Marco Aguiar <marco@DESKTOP-8IVSCHM.localdomain>
2020-08-31 15:31:58 +02:00
Jarek Potiuk 596bc13379
Adds 'cncf.kubernetes' package back to backport provider packages. (#10659) 2020-08-31 14:45:58 +02:00
Jarek Potiuk 46c8d6714c
Implement Google Shell Conventions for breeze script … (#10651)
Part of #10576

First (and the biggest of the series of commits to introduce
Google Shell Conventions in our bash scripts.

This is about the biggest and the most complex breeze script
so it is rather huge but it is difficult to split it into
smaller pieces.

The rules implemented (from the conventions):

 * constants and exported variables are CAPITALIZED, where
   local/temporary variables are lowercase

 * following the shell guide, once all the variables are set to their
   final values (either from exported variables, calculation or --switches
   ) I have a single function that makes all the variables read-only. That
   helped to clean-up a lot of places where same functions was called
   several times, or where variables were defined in a few places. Now the
   behavior should be rather consistent and we should easily catch some
   duplications

 * function headers (following the guide) explaining arguments,
   variables expected, variables modified in the functions used.

 * setting the variables as read-only also helped to clean-up the "ifs"
   where we often had ":=}" in variables and != "" or == "". Those are
   replaced with `=}` and tests are replaced with `-n` and `-z` - also
   following the shell guide (readonly helped to detect and clean all
   such cases). This also should be much more robust in the future.

 * reorganized initialization of those constants and variables - simplified
   a few places where initialization was overlapping. It should be much more
   straightforward and clean now

 * a number of internal function breeze variables are "local" - this is
   helpful in accidental variables overwriting and keeping stuff localized

 * trap_add function is separated out to help in cases where we had
   several traps handling the same signals.
2020-08-31 13:24:53 +02:00
Jarek Potiuk c320d99c4b
Small fixes in Breeze/Static check/docs documentation (#10658) 2020-08-31 12:46:39 +02:00
Jarek Potiuk 0be140684b
Adds pip-wheel metadata in .gitignore (#10657) 2020-08-31 12:45:13 +02:00
QP Hou e4878e6775
fix type hints for s3 hook read_key method (#10653) 2020-08-31 09:25:14 +02:00
Diego Lopes 4476a09f44
Update INTHEWILD.md (#10649)
Added Datasprints as INTHEWILD markdown section
2020-08-30 20:00:35 +02:00
Masato Ohba 11c00bc820
Fix typos: duplicated "the" (#10647) 2020-08-30 09:57:24 +02:00
Vikram Koka 9b2efc6dcc
Improve logging & monitoring docs (#10618)
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
2020-08-30 01:23:52 +01:00
Kamil Breguła 2ca615cffe
Update Google Cloud branding (#10642) 2020-08-29 23:36:52 +02:00
João Marques 934115b728
Move roles to CONTRIBUTING.rst (#10327)
* feat: add initial roles, committers and contributors

* refactor: move roles to after contributions

* fix: remove extra spaces

* fix: fix text according to PR discussion

* refactor: move roles to 2nd point

* Update CONTRIBUTING.rst

Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
2020-08-29 21:56:21 +02:00
Xiaodong DENG 7431867ac0
Exclude CSRF tokens in Log's attribute "extra" in database (#10640) 2020-08-29 19:27:28 +02:00
Jarek Potiuk 5e36152235
Helm Chart is using 1.10.12 image by default (#10639) 2020-08-29 18:33:26 +02:00
gardnerdev e495be9fd9
Update scheduler deployment - dags volume mount (#10630) 2020-08-28 21:57:34 +02:00
Kamil Breguła 7200835d0e
Improve output of check_environment.sh (#10631) 2020-08-28 21:53:11 +02:00
Kamil Breguła 8e0d9f09d9
Add airflow cheat-sheet command (#10619) 2020-08-28 21:25:29 +02:00
Kamil Breguła 1b533f617e
Fix broken master - DLP (#10635) 2020-08-28 19:27:23 +02:00
Michał Słowikowski 8969b7185e
Removed bad characters from AWS operator (#10590) 2020-08-28 16:57:54 +02:00
flvndh 5f403a8759
Wrong key in DAGs Persistent Volume Claim (#10627)
Co-authored-by: Flavien Dereume-Hancart <flavien@LL-PC0BE1K9-1.goiba.net>
2020-08-28 16:52:05 +02:00
Tomek Urbaszek 5ae82a56da
Fix Google DLP example and improve ops idempotency (#10608) 2020-08-28 16:35:47 +02:00
Kamil Breguła 3867f76625
Update Google Cloud branding (#10615) 2020-08-28 12:19:27 +02:00
Kaxil Naik 725bf330ef
Revert Clean up DAG serializations based on last_updated (#7424) (#10613)
This PR reverts the behavior of https://github.com/apache/airflow/pull/7424
2020-08-27 20:56:41 +01:00
Tomek Urbaszek 479d6220b7
Fix downstream rendering in WebUI (#10612)
closes #10611
2020-08-27 20:14:15 +01:00
Vikram Koka 775c22091e
Added a logging and monitoring architecture diagram and page (#10609) 2020-08-27 19:56:45 +01:00
Vikram Koka 540957df96
Made use of authentication consistent (#10610)
Fixed a couple of places where authorization was used instead of authentication
2020-08-27 19:23:42 +01:00
Alex Begg 246c8f2952
Add Currency to INTHEWILD.md (#10607) 2020-08-27 20:04:47 +02:00
Jarek Potiuk e565368f2e
Nightly tag push is not skipped in scheduled builds (#10597)
With recent refactors, nightly tag was not pushed on
scheduled event because it was depending on pushing images
to github registry. Pushing images to github registry is
skipped on scheduled builds, so pushing tag was also skipped.
2020-08-27 15:40:07 +02:00
Alex Begg 900f15ab69
URL encode execution date in the Last Run link (#10595)
Fixes #10434
2020-08-27 12:07:05 +01:00
Anton Bryzgalov 2e56ee7b22
DockerOperator extra_hosts argument support added (#10546) 2020-08-27 11:36:04 +01:00
Omair Khan 91ff31ad10
Documentation for Google Cloud Data Loss Prevention (#8201) (#9651) 2020-08-27 10:56:02 +02:00
Kaxil Naik bb3e45c250
Add Airflow 1.10.12 & 'black' to breeze-complete (#10592) 2020-08-27 10:10:08 +02:00
Ephraim Anierobi 276a15a07f
Add info about update mask to API doc introduction (#10572) 2020-08-27 01:31:16 +02:00
Ryan McCormack 8624224715
Remove Outdated SQLCheckOperator Docstring (#10589)
This method is now implemented
2020-08-26 23:56:40 +01:00
Kaxil Naik 2fc961458e
Fix typo in Custom XCom backend (#10588) 2020-08-26 19:04:33 +01:00
Theodor 8349061f9c
Improve Docstring for AWS Athena Hook/Operator (#10580) 2020-08-26 19:57:32 +02:00
Vikram Koka 46ac09d5c9
Enhanced the Kubernetes Executor doc (#10433)
A simple architecture diagram to show the Airflow setup when used with the Kubernetes executor
2020-08-26 18:42:35 +01:00
Beni Ben zikry 1e5aa4465c
Spark-on-K8S sensor - add driver logs (#10023) 2020-08-26 18:14:20 +02:00
Sanjana Gupta 3b0aa169c7
updated official docker images in README (#10579) 2020-08-26 18:01:27 +02:00
Ry Walker 13a3a30490
Improve .mailmap (#10582) 2020-08-26 16:48:17 +01:00
mattiagiupponi 35efb1d554
Add Jobrapido to INTHEWILD.md (#10583)
Co-authored-by: Mattia Giupponi <mattia.giupponi@jobrapido.com>
2020-08-26 16:47:50 +01:00
Marcus Levine b81bdaaf86
Add OpenSlate to INTHEWILD.md (#10581) 2020-08-26 11:01:23 -04:00
Ping Zhang db378c09b7
[k8s] Store the raw ti key info to pod annotations (#10568)
The value of annotations can store the raw dag_id, task_id and
execution_date so that k8s executor can easily map pod event back
to the task instance
2020-08-26 07:53:37 -07:00
Jarek Potiuk 8a7c37281c
Untangle cyclic deps configuration <> secrets (#10559) 2020-08-26 16:38:57 +02:00
王清山 3a349624a2
Update CHANGELOG complete class name (#10573) 2020-08-26 09:50:19 +02:00
Kamil Breguła 855d495f1e
Move more operators/hooks/sensors to fundamentals (#10567) 2020-08-26 02:56:47 +02:00
Kaxil Naik 7c1855fb5d
Add License to INTHEWILD.md (#10570) 2020-08-26 00:11:32 +02:00
Ry Walker d84ec038d8
Add content to file (forgot to save before committing) (#10565) 2020-08-25 17:23:06 -04:00
Ry Walker 51676b75e6
Migrate companies list to INTHEWILD.md (#10563) 2020-08-25 14:12:16 -07:00