* [AIRFLOW-3752] Add/remove user from role via the CLI
Update the `users` subcommand to enable 2 new actions:
- `--add-role`: Make the user a member of the given role
- `--remove-role`: Remove the user's membership in the given role
For installations that use an external identity provider (e.g., Google
OAuth) the username is typically a long ID string. For the sake of
convenience, we allow the CLI operator to reference the target user
via either their `username` or their `email` (but not both).
* Update argparse spec
Accidentally left off this update to the argparse spec in the last
commit.
* Add unit tests
* Fix lint failures
Local users were always a superuser, this adds a column to the DB (and defaults to false,
which is going to cause a bit of an upgrade pain for people, but defaulting to not being an
admin is the only secure default.)
Started with "habe", "serever" and "certificiate" needing to be:
"have", "server", and "certificate".
Ran a check, ignoring British and US accepted spellings.
Kept jargon. EG admin, aync, auth, backend, config, dag, s3, utils, etc.
Took exception to: "num of dag run" meaning "number of dag runs",
"upness" is normally for quarks,
"url" being lower-case, and
sftp example having an excess file ending.
Python documentation writes "builtin" hyphenated, cases "PYTHONPATH".
Gave up on mixed use of "dag" and "DAG" as well as long line lengths.
This adds ASF license headers to all the .rst and .md files with the
exception of the Pull Request template (as that is included verbatim
when opening a Pull Request on Github which would be messy)
This updates the scheduler_heartbeat metric from a gauge to a counter to
better support the statsd_exporter for usage with Prometheus. A counter
allows users to track the rate of the heartbeat, and integrates with the
exporter better. A crashing or down scheduler will no longer emit the
metric, but the statsd_exporter will continue to show a 1 for the metric
value. This fixes that issue because a counter will continually change,
and the lack of change indicates an issue with the scheduler.
Add statsd change notice in UPDATING.md
artifacts in default_airflow.cfg
- fixed incorrect instructions in UPDATING.md regarding core.log_filename_template and elasticsearch.elasticsearch_log_id_template
- removed comments referencing "additional curly braces" from
default_airflow.cfg since they're irrelevant to the rendered airflow.cfg
By default one of Apache Airflow's dependencies pulls in a GPL
library. Airflow should not install (and upgrade) without an explicit choice.
This is part of the Apache requirements as we cannot depend on Category X
software.
Make sure you have checked _all_ steps below.
### JIRA
- [x] My PR addresses the following [Airflow JIRA]
(https://issues.apache.org/jira/browse/AIRFLOW/)
issues and references them in the PR title. For
example, "\[AIRFLOW-XXX\] My Airflow PR"
-
https://issues.apache.org/jira/browse/AIRFLOW-2267
- In case you are fixing a typo in the
documentation you can prepend your commit with
\[AIRFLOW-XXX\], code changes always need a JIRA
issue.
### Description
- [x] Here are some details about my PR, including
screenshots of any UI changes:
Provide DAG level access for airflow. The detail
design could be found at https://docs.google.com/d
ocument/d/1qs26lE9kAuCY0Qa0ga-80EQ7d7m4s-590lhjtMB
jmxw/edit#
### Tests
- [x] My PR adds the following unit tests __OR__
does not need testing for this extremely good
reason:
Unit tests are added.
### Commits
- [x] My commits all reference JIRA issues in
their subject lines, and I have squashed multiple
commits if they address the same issue. In
addition, my commits follow the guidelines from
"[How to write a good git commit
message](http://chris.beams.io/posts/git-
commit/)":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not
"adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"
- [x] Passes `git diff upstream/master -u --
"*.py" | flake8 --diff`
Closes#3197 from feng-tao/airflow-2267
The new names are in-line with Celery 4, but if
anyone upgrades Airflow
without following the UPDATING.md instructions
(which we probably assume
most people won't, not until something stops
working) their workers
would suddenly just start failing. That's bad.
This will issue a warning but carry on working as
expected. We can
remove the deprecation settings (but leave the
code in config) after
this release has been made.
Closes#3549 from ashb/AIRFLOW-1840-back-compat
- Improved the retries times to jobs below 60s
- Renamed property queue to job_queue to prevent
AWS Batch and CeleryExecutor queue conflict
- Added Breaking Chain note for the UPDATING.md
master
- Fixed operator infinit loop
- Added documentation warning about the Breaking
chain
- Fixed the commit parameter to keep it on Airflow
guidelines
- Fixed logging typo
- rebased with master
Changes to be committed:
modified: ../../../UPDATING.md
modified: awsbatch_operator.py
modified: ../../../tests/contrib/operators/test_
awsbatch_operator.py
Closes#3436 from hprudent/master
Closes#3279 from feng-tao/reduce-tree-view
This introduces a new configuration variable to set the default
number of dag runs displayed in the tree view. For large DAGs, this
could cause timeouts in the webserver.
Currently, data is ordered by first column in
descending order
Header row comes as first only if the first column
is integer
This fix puts header as first row regardless of
first column data type
Closes#3180 from sathyaprakashg/AIRFLOW-2254
Felix
Kamil Breguła
Tao Feng
Kaxil Naik
Andrew Stahlman
Jarek Potiuk
Verdan Mahmood
bolkedebruin
Fokko Driesprong
aoen
Camilo Aguilar
Joshua Carp
Kaxil Naik
thomasbrockmeier
Daniel Lamblin
Ash Berlin-Taylor
Kevin Yang
Greg Neiheisel
Eric Chang
Dan Davydov
Taylor D. Edmiston
Jacob Hayes
Kevin Yang
Eric Lo
Hugo Prudente
niels
Fokko Driesprong
Philippus Baalman
DerekRoy
Sathyaprakash Govindasamy
Dan Davydov