Disables provider's manager warning for source-installed prod image. (#13729)
When production image is built for development purpose, by default it installs all providers from sources, but not all dependencies are installed for all providers. Many providers require more dependencies and when you try to import those packages via provider's manager, they fail to import and print warnings. Those warnings are now turned into debug messages, in case AIRFLOW_INSTALLATION_METHOD=".", which is set when production image is built locally from sources. This is helpful especially when you use locally build production image to run K8S tests - otherwise the logs are flooded with warnings. This problem does not happe in CI, because there by default production image is built from locally prepared packages and it does not contain sources from providers that are not installed via packages.
This commit is contained in:
Родитель
1ec63123c4
Коммит
f74da5025d
|
@ -464,6 +464,11 @@ EXPOSE 8080
|
|||
|
||||
USER ${AIRFLOW_UID}
|
||||
|
||||
# Having the variable in final image allows to disable providers manager warnings when
|
||||
# production image is prepared from sources rather than from package
|
||||
ARG AIRFLOW_INSTALLATION_METHOD="apache-airflow"
|
||||
ENV AIRFLOW_INSTALLATION_METHOD=${AIRFLOW_INSTALLATION_METHOD}
|
||||
|
||||
ARG BUILD_ID
|
||||
ENV BUILD_ID=${BUILD_ID}
|
||||
ARG COMMIT_SHA
|
||||
|
|
|
@ -289,12 +289,27 @@ class ProvidersManager:
|
|||
self._add_customized_fields(provider_package, hook_class, field_behaviours)
|
||||
|
||||
except Exception as e: # noqa pylint: disable=broad-except
|
||||
log.warning(
|
||||
"Exception when importing '%s' from '%s' package: %s",
|
||||
hook_class_name,
|
||||
provider_package,
|
||||
e,
|
||||
)
|
||||
if os.environ.get("AIRFLOW_INSTALLATION_METHOD") != ".":
|
||||
# print providers manager warning when airflow is not installed from sources in the
|
||||
# production image
|
||||
log.warning(
|
||||
"Exception when importing '%s' from '%s' package: %s",
|
||||
hook_class_name,
|
||||
provider_package,
|
||||
e,
|
||||
)
|
||||
else:
|
||||
# This is special case - when airflow is installed from sources in production
|
||||
# image, AIRFLOW_INSTALLATION_METHOD is set to ".". In this case we know that there
|
||||
# Will be some warnings generated by ProviderManager, when it tries to import providers
|
||||
# With missing dependencies, therefore we are turning such warnings into debug message
|
||||
# so that we do not pollute logs.
|
||||
log.debug(
|
||||
"Exception when importing '%s' from '%s' package: %s",
|
||||
hook_class_name,
|
||||
provider_package,
|
||||
e,
|
||||
)
|
||||
return
|
||||
|
||||
conn_type: str = self._get_attr(hook_class, 'conn_type')
|
||||
|
|
Загрузка…
Ссылка в новой задаче