incubator-airflow/provider_packages
Jarek Potiuk 3dd5043fb1 Parallelize build of documentation. (#15062)
This is far more complex than it should be because of
autoapi problems with parallel execution. Unfortunately autoapi
does not cope well when several autoapis are run in parallel on
the same code - even if they are run in separate processes and
for different packages. Autoapi uses common _doctree and _api
directories generated in the source code and they override
each other if two or more of them run in parallel.

The solution in this PR is mostly applicable for CI environment.
In this case we have docker images that have been already built
using current sources so we can safely run separate docker
containers without mapping the sources and run generation
of documentation separtely and independently in each container.

This seems to work really well, speeding up docs generation
2x in public GitHub runners and 8x in self-hosted runners.

Public runners:

* 27m -> 15m

Self-hosted runners:

* 27m -> < 8m

(cherry picked from commit 741a54502f)
2021-04-15 14:00:31 +01:00
..
.flake8
.gitignore Remove Backport Providers (#14886) 2021-04-15 14:00:31 +01:00
INSTALL The .tar.gz provider packages are installable now. (#11630) 2020-10-23 16:47:47 +02:00
LICENSE The .tar.gz provider packages are installable now. (#11630) 2020-10-23 16:47:47 +02:00
NOTICE The .tar.gz provider packages are installable now. (#11630) 2020-10-23 16:47:47 +02:00
README.rst Parallelize build of documentation. (#15062) 2021-04-15 14:00:31 +01:00
dist
pyproject.toml The .tar.gz provider packages are installable now. (#11630) 2020-10-23 16:47:47 +02:00

README.rst

.. Licensed to the Apache Software Foundation (ASF) under one
   or more contributor license agreements.  See the NOTICE file
   distributed with this work for additional information
   regarding copyright ownership.  The ASF licenses this file
   to you under the Apache License, Version 2.0 (the
   "License"); you may not use this file except in compliance
   with the License.  You may obtain a copy of the License at

..   http://www.apache.org/licenses/LICENSE-2.0

.. Unless required by applicable law or agreed to in writing,
   software distributed under the License is distributed on an
   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
   KIND, either express or implied.  See the License for the
   specific language governing permissions and limitations
   under the License.


Package ``apache-airflow-providers-ssh``

Release: ``1.0.0dev``


`Secure Shell (SSH) <https://tools.ietf.org/html/rfc4251>`__


Provider package
================

This is a provider package for ``ssh`` provider. All classes for this provider package
are in ``airflow.providers.ssh`` python package.

You can find package information and changelog for the provider
in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-ssh/1.0.0/>`_.


Installation
============

You can install this package on top of an existing airflow 2.* installation via
``pip install apache-airflow-providers-ssh``

PIP requirements
================

=============  ==================
PIP package    Version required
=============  ==================
``paramiko``   ``>=2.6.0``
``pysftp``     ``>=0.2.9``
``sshtunnel``  ``>=0.1.4,<0.2``
=============  ==================