[engsys] Migrate to tox4 (#30159)
* Remove references to {envbindir}
tox prepends the {env_bin_dir} to $PATH, so commands
will default to using the virtual environment first.
See: https://tox.wiki/en/4.4.11/config.html#commands
One of tox's core maintainers also discourages its use:
https://github.com/tox-dev/tox/issues/2909#issuecomment-1409806654
* refactor: Write paths relative to git root, instead of package dir
Uses an inline plugin (toxfile.py) to make a computed config
value avaiable to tox environments.
* refactor: Rename {toxinidir} to {tox_root}
{toxinidir} is an alias for {tox_root}, but {tox_root} is:
* More clear: {tox_root} isn't necessarily where the ini file is
* Shorter
* refactor: Flatten some commands that are no longer super long
* refactor: Make a pytest section
* refactor: Remove `changedir = {tox_root}`
This is the default behavior, see:
https://tox.wiki/en/4.4.11/config.html#change_dir-external
* fix: {distdir} is deprecated
See: https://tox.wiki/en/latest/upgrading.html#removed-tox-ini-keys
* fix: Remove --ignore-installed from pip command
--ignore-installed can break a python installation if multiple
conflicting versions of a package are installed
* refactor: Add descriptions to all tox environments
* fix: Remove ineffective platform config
platform = linux: linux
macos: darwin
windows: win32
Setting the above in the base environment has no effect:
* None of the environments have {linux,macos,windows} in the
name, so the platform config is always empty.
* chore: Bump `tox` to `>4.4.11` and no longer install `tox-monorepo`
tox4 natively solves what `tox-monorepo` was written to solve:
`--root` lets you set `toxinidir` independently of the config
file in use.
4.4.11 was chosen as the cutoff since it was the first release to
include a fix to `--root` that prevented `{work_dir}` from being
changed when `{toxinidir}` was changed by `--root`.
`tox-monorepo` should no longer be needed
* docs: Update CONTRIBUTING.md to use tox 4 commands
* References to tox-monorepo were removed
* Replaced discussion of `tox -l` with tox4's `tox list`
* Updated examples to use `--root`
* docs: Make `azure-media-{videoanalyzer,analytics}-edge`'s DevTips.md point to CONTRIBUTING.md for tox
* docs: Remove remaining references to tox-monorepo
* doc: Use `--root` throughout documentation
* engsys: Make tox_harness set `{toxinidir}` when invoking tox
* engsys: Set required tox version in config
* docs: Use the new url for tox docs
* docs: Add a callout that TOX_* environment variables control defaults
* TOX_CONFIG_DIR can be used to permanently set --conf
* TOX_ROOT_DIR can be used to permanently set --root
* update packaging requirement everywhere it is used.
* fix cspell
* docs: Use `tox run` syntax throughout documentation
* engsys: Use `tox run` syntax in tox_harness.py
* docs: Replace – (U+2013 : EN DASH) with - (ASCII code 45)
* update pytest requirements + coverage.
* update virtualenv
* more conflicts resolved
* bringing in urllib3 restriction, aligning ci_tools.txt with test_tools.txt
* check_call requires a list of type string. if we append ['blah', 'blah'] we are actually inserting another array. just use regular array concatenation
* engsys: Tox uses `run` for serial execution and `run-parallel` for parallel`
* azure-core: Use `@pytest_asyncio.fixture` to mark an asyncio fixture
See https://pytest-asyncio.readthedocs.io/en/latest/concepts.html#strict-mode
* pin coverage to newer version with no conflicts
* identify issue with missing code coverage being caused by wrongly named namespace (#30344)
* remove fix of dotfile!
* apply black formatting to resolve failing analyze step
* tox installing tox. probably not a good thing :)
* engsys: Remove `tox` from `azure-media-analytics-edge` dev_requirements
* align regression version of ci/test_tools.txt
* new coverage format requires that the package sources be present when combining .coverage files. amend our tox tree cleanup to exclude the whl directory when running coverage.
* ensure that missing hidden folder can't break cleanup operation
* ensure coverage isn't generated on whl_no_aio
* ensure that coverage is generated without wonky package paths
---------
Co-authored-by: scbedd <45376673+scbedd@users.noreply.github.com>
This commit is contained in:
kdestin
GitHub
Родитель
62bbb9f333
Коммит
982234bf05
|
|
@ -416,6 +416,12 @@
|
|||
"pytestmarkparametrize"
|
||||
]
|
||||
},
|
||||
{
|
||||
"filename": "doc/dev/pylint_checking.md",
|
||||
"words": [
|
||||
"pylintrc"
|
||||
]
|
||||
},
|
||||
{
|
||||
"filename": "tools/azure-sdk-tools/devtools_testutils/proxy_startup.py",
|
||||
"words": [
|
||||
|
|
|
|||
|
|
@ -11,13 +11,13 @@ If you want to contribute to a file that is generated (header contains `Code gen
|
|||
|
||||
The Azure SDK team's Python CI leverages the tool `tox` to distribute tests to virtual environments, handle test dependency installation, and coordinate tooling reporting during PR/CI builds. This means that a dev working locally can reproduce _exactly_ what the build machine is doing.
|
||||
|
||||
[A Brief Overview of Tox](https://tox.readthedocs.io/en/latest/)
|
||||
[A Brief Overview of Tox](https://tox.wiki/en/latest/)
|
||||
|
||||
#### A Monorepo and Tox in Harmony
|
||||
|
||||
Traditionally, the `tox.ini` file for a package sits _alongside the setup.py_ in source code. The `azure-sdk-for-python` necessarily does not adhere to this policy. There are over one-hundred packages contained here-in. That's a lot of `tox.ini` files to maintain!
|
||||
|
||||
Instead, the CI system leverages an tox plugin called `tox-monorepo`. This plugin allows `tox` to act as if the `tox.ini` is located in whatever directory you executed tox in!
|
||||
Instead, the CI system leverages the `--root` argument which is new to `tox4`. The `--root` argument allows `tox` to act as if the `tox.ini` is located in whatever directory you specify!
|
||||
|
||||
#### Tox Environments
|
||||
|
||||
|
|
@ -32,26 +32,50 @@ Internally `tox` leverages `virtualenv` to create each test environment's virtua
|
|||
|
||||
This means that once the `tox` workflow is in place, all tests will be executed _within a virtual environment._
|
||||
|
||||
To see the default environments from a specific `tox.ini` file, use the command `tox -l` in the same directory as the file itself.
|
||||
You can use the command `tox list` to list all the environments provided by a `tox.ini` file. You can either use that command in the
|
||||
same directory as the file itself, or use the `--conf` argument to specify the path to it directly.
|
||||
|
||||
> sdk-for-python/eng/tox> tox -l
|
||||
|
||||
Sample output of `tox list`:
|
||||
|
||||
```
|
||||
sdk-for-python/eng/tox> tox list
|
||||
default environments:
|
||||
whl -> Builds a wheel and runs tests
|
||||
sdist -> Builds a source distribution and runs tests
|
||||
|
||||
whl
|
||||
sdist
|
||||
|
||||
additional environments:
|
||||
pylint -> Lints a package with a pinned version of pylint
|
||||
next-pylint -> Lints a package with pylint (version 2.15.8)
|
||||
mypy -> Typechecks a package with mypy (version 1.0.0)
|
||||
next-mypy -> Typechecks a package with the latest version of mypy
|
||||
pyright -> Typechecks a package with pyright (version 1.1.287)
|
||||
next-pyright -> Typechecks a package with the latest version of static type-checker pyright
|
||||
verifytypes -> Verifies the "type completeness" of a package with pyright
|
||||
whl_no_aio -> Builds a wheel without aio and runs tests
|
||||
develop -> Tests a package
|
||||
sphinx -> Builds a package's documentation with sphinx
|
||||
depends -> Ensures all modules in a target package can be successfully imported
|
||||
verifywhl -> Verify directories included in whl and contents in manifest file
|
||||
verifysdist -> Verify directories included in sdist and contents in manifest file. Also ensures that py.typed configuration is correct within the setup.py
|
||||
devtest -> Tests a package against dependencies installed from a dev index
|
||||
latestdependency -> Tests a package against the released, upper-bound versions of its azure dependencies
|
||||
mindependency -> Tests a package against the released, lower-bound versions of its azure dependencies
|
||||
apistub -> Generate an api stub of a package ( for https://apiview.dev )
|
||||
bandit -> Runs bandit, a tool to find common security issues, against a package
|
||||
samples -> Runs a package's samples
|
||||
breaking -> Runs the breaking changes checker against a package
|
||||
```
|
||||
|
||||
Unfortunately, the command `tox -l` only returns the _default_ test builds. The common `tox.ini` file also supports `pylint` and `mypy` environments.
|
||||
|
||||
### Example Usage of the common Azure SDK For Python `tox.ini`
|
||||
|
||||
Basic usage of `tox` within this monorepo is:
|
||||
|
||||
1. `pip install tox<4 tox-monorepo`
|
||||
2. `cd` to target package folder
|
||||
3. run `tox -c path/to/tox.ini`
|
||||
1. `pip install tox<5`
|
||||
2. Run `tox run -e ENV_NAME -c path/to/tox.ini --root path/to/python_package`
|
||||
* **Note**: You can use environment variables to provide defaults for tox config values
|
||||
* With `TOX_CONFIG_FILE` set to the absolute path of `tox.ini`, you can avoid needing `-c path/to/tox.ini` in your tox invocations
|
||||
* With `TOX_ROOT_DIR` set to the absolute path to your python package, you can avoid needing `--root path/to/python_package`
|
||||
|
||||
The common `tox.ini` location is `eng/tox/tox.ini` within the repository.
|
||||
|
||||
|
|
@ -59,13 +83,11 @@ If at any time you want to blow away the tox created virtual environments and st
|
|||
|
||||
#### Example `azure-core` mypy
|
||||
|
||||
1. `cd` to `sdk/core/azure-core`
|
||||
2. Run `tox -e mypy -c ../../../eng/tox/tox.ini`
|
||||
1. Run `tox run -e mypy -c ../../../eng/tox/tox.ini --root sdk/core/azure-core`
|
||||
|
||||
#### Example `azure-storage-blob` tests
|
||||
|
||||
1. `cd` to `sdk/storage/azure-storage-blob`
|
||||
2. Execute `tox -c ../../../eng/tox/tox.ini`
|
||||
2. Execute `tox run -c ../../../eng/tox/tox.ini --root sdk/storage/azure-storage-blob`
|
||||
|
||||
Note that we didn't provide an `environment` argument for this example. Reason here is that the _default_ environment selected by our common `tox.ini` file is one that runs `pytest`.
|
||||
|
||||
|
|
@ -75,7 +97,7 @@ Used for test execution across the spectrum of all the platforms we want to supp
|
|||
* Installs the wheel, runs tests using the wheel
|
||||
|
||||
```
|
||||
\> tox -e whl -c <path to tox.ini>
|
||||
\> tox run -e whl -c <path to tox.ini> --root <path to python package>
|
||||
|
||||
```
|
||||
|
||||
|
|
@ -87,7 +109,7 @@ Used for the local dev loop.
|
|||
|
||||
```
|
||||
|
||||
\> tox -e sdist -c <path to tox.ini>
|
||||
\> tox run -e sdist -c <path to tox.ini> --root <path to python package>
|
||||
|
||||
```
|
||||
|
||||
|
|
@ -95,7 +117,7 @@ Used for the local dev loop.
|
|||
Pylint install and run.
|
||||
|
||||
```
|
||||
\> tox -e pylint -c <path to tox.ini>
|
||||
\> tox run -e pylint -c <path to tox.ini> --root <path to python package>
|
||||
```
|
||||
|
||||
|
||||
|
|
@ -103,14 +125,14 @@ Pylint install and run.
|
|||
Mypy install and run.
|
||||
|
||||
```
|
||||
\> tox -e mypy -c <path to tox.ini>
|
||||
\> tox run -e mypy -c <path to tox.ini> --root <path to python package>
|
||||
```
|
||||
|
||||
#### `sphinx` environment
|
||||
Generate sphinx doc for this package.
|
||||
|
||||
```
|
||||
\> tox -e sphinx -c <path to tox.ini>
|
||||
\> tox run -e sphinx -c <path to tox.ini> --root <path to python package>
|
||||
```
|
||||
|
||||
### Custom Pytest Arguments
|
||||
|
|
@ -120,7 +142,7 @@ Generate sphinx doc for this package.
|
|||
[Tox Documentation on Positional Arguments](https://tox.wiki/en/latest/config.html#substitutions-for-positional-arguments-in-commands)
|
||||
|
||||
**Example: Invoke tox, breaking into the debugger on failure**
|
||||
`tox -e whl -c ../../../eng/tox/tox.ini -- --pdb`
|
||||
`tox run -e whl -c <path to tox.ini> --root <path to python package> -- --pdb`
|
||||
|
||||
### Performance Testing
|
||||
|
||||
|
|
@ -150,7 +172,7 @@ a. cd to package root folder
|
|||
b. run tox environment devtest
|
||||
|
||||
```
|
||||
\> tox -e devtest -c <path to tox.ini>
|
||||
\> tox run -e devtest -c <path to tox.ini> --root <path to python package>
|
||||
```
|
||||
|
||||
This tox test( devtest) will fail if installed dependent packages are not dev build version.
|
||||
|
|
|
|||
|
|
@ -33,11 +33,11 @@ or execute the various commands available in the toolbox.
|
|||
|
||||
4. Setup your development environment
|
||||
|
||||
Install the development requirements for a specific library (located in the `dev_requirements.txt` file at the root of the library), [Tox][tox], [Tox monorepo][tox_monorepo] and an editable install of your library. For example, to install requirements for `azure-ai-formrecognizer`:
|
||||
Install the development requirements for a specific library (located in the `dev_requirements.txt` file at the root of the library), [Tox][tox] and an editable install of your library. For example, to install requirements for `azure-ai-formrecognizer`:
|
||||
```
|
||||
azure-sdk-for-python> cd sdk/formrecognizer/azure-ai-formrecognizer
|
||||
azure-sdk-for-python/sdk/formrecognizer/azure-ai-formrecognizer> pip install -r dev_requirements.txt
|
||||
azure-sdk-for-python/sdk/formrecognizer/azure-ai-formrecognizer> pip install tox tox-monorepo
|
||||
azure-sdk-for-python/sdk/formrecognizer/azure-ai-formrecognizer> pip install tox<5
|
||||
azure-sdk-for-python/sdk/formrecognizer/azure-ai-formrecognizer> pip install -e .
|
||||
```
|
||||
|
||||
|
|
@ -58,6 +58,5 @@ Follow the instructions in the [Perform one-time test proxy setup][proxy_setup]
|
|||
[python_39]: https://www.microsoft.com/p/python-39/9p7qfqmjrfp7
|
||||
[proxy_setup]: https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/tests.md#perform-one-time-test-proxy-setup
|
||||
[tests]: https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/tests.md
|
||||
[tox]: https://tox.readthedocs.io/en/latest/
|
||||
[tox_monorepo]: https://pypi.org/project/tox-monorepo/
|
||||
[tox]: https://tox.wiki/en/latest/
|
||||
[virtual_environment]: https://docs.python.org/3/tutorial/venv.html
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ In the Azure SDK for Python repository, in addition to the standard pylint libra
|
|||
|
||||
One way to run pylint is to run at the package level with tox:
|
||||
|
||||
.../azure-sdk-for-python/sdk/eventgrid/azure-eventgrid>tox -e pylint -c ../../../eng/tox/tox.ini
|
||||
.../azure-sdk-for-python/sdk/eventgrid/azure-eventgrid>tox run -e pylint -c ../../../eng/tox/tox.ini --root .
|
||||
|
||||
If you don't want to use tox, you can also install and run pylint on its own:
|
||||
|
||||
|
|
@ -58,9 +58,9 @@ In addition to being a part of the CI, the custom pylint checkers are also integ
|
|||
|
||||
There is now a new step on the CI pipeline called `Run Pylint Next`. This is merely a duplicate of the `Run Pylint` step with the exception that `Run Pylint Next` uses the latest version of pylint and the latest version of the custom pylint checkers.
|
||||
|
||||
This next-pylint enviornment can also be run locally through tox:
|
||||
This next-pylint environment can also be run locally through tox:
|
||||
|
||||
tox -e next-pylint -c ../../../eng/tox/tox.ini
|
||||
tox run -e next-pylint -c ../../../eng/tox/tox.ini --root <path to python package>
|
||||
|
||||
The errors generated by the `Run Pylint Next` step will not break your weekly test pipelines, but make sure to fix the warnings so that your client library is up to date for the next pylint release.
|
||||
|
||||
|
|
|
|||
|
|
@ -52,13 +52,13 @@ The given `START`/`END` keywords can be used in a [sphinx literalinclude][sphinx
|
|||
[Literalinclude example][literalinclude]
|
||||
|
||||
The rendered code snippets are sensitive to the indentation in the sample file. Adjust the `dedent` accordingly to ensure the sample is captured accurately and not accidentally trimmed.
|
||||
You can preview how published reference documentation will look by running [tox][tox]: `tox -e sphinx -c ../../../eng/tox/tox.ini`.
|
||||
You can preview how published reference documentation will look by running [tox][tox]: `tox run -e sphinx -c ../../../eng/tox/tox.ini --root <path to python package>`.
|
||||
|
||||
## Test run samples in CI live tests
|
||||
Per the [Python guidelines][snippet_guidelines], sample code and snippets should be test run in CI to ensure they remain functional. Samples should be run in the package's live test pipeline which is scheduled to run daily.
|
||||
To ensure samples do get tested as part of regular CI runs, add these [lines][live_tests] to the package's tests.yml.
|
||||
|
||||
You can test this CI step locally first, by utilizing [tox][tox] and running `tox -e samples -c ../../../eng/tox/tox.ini` at the package-level.
|
||||
You can test this CI step locally first, by utilizing [tox][tox] and running `tox run -e samples -c ../../../eng/tox/tox.ini --root <path to python package>`.
|
||||
|
||||
The `Test Samples` step in CI will rely on the resources provisioned and environment variables used for running the package's tests.
|
||||
|
||||
|
|
|
|||
|
|
@ -175,9 +175,9 @@ version of the type checker ships. All client libraries in the Python SDK repo a
|
|||
|
||||
The easiest way to install and run the type checkers locally is
|
||||
with [tox](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/tests.md#tox). This reproduces the exact type checking
|
||||
environment run in CI and brings in the third party stub packages necessary. To begin, first install `tox` and `tox-monorepo`:
|
||||
environment run in CI and brings in the third party stub packages necessary. To begin, first install `tox`:
|
||||
|
||||
`pip install tox tox-monorepo`
|
||||
`pip install tox<5`
|
||||
|
||||
### Run mypy
|
||||
|
||||
|
|
@ -185,7 +185,7 @@ mypy is currently pinned to version [1.0.0](https://pypi.org/project/mypy/1.0.0/
|
|||
|
||||
To run mypy on your library, run the tox mypy env at the package level:
|
||||
|
||||
`.../azure-sdk-for-python/sdk/textanalytics/azure-ai-textanalytics>tox -e mypy -c ../../../eng/tox/tox.ini`
|
||||
`.../azure-sdk-for-python/sdk/textanalytics/azure-ai-textanalytics>tox run -e mypy -c ../../../eng/tox/tox.ini --root .`
|
||||
|
||||
If you don't want to use `tox` you can also install and run mypy on its own:
|
||||
|
||||
|
|
@ -217,7 +217,7 @@ Note that pyright requires that node is installed. The command-line [wrapper pac
|
|||
|
||||
To run pyright on your library, run the tox pyright env at the package level:
|
||||
|
||||
`.../azure-sdk-for-python/sdk/textanalytics/azure-ai-textanalytics>tox -e pyright -c ../../../eng/tox/tox.ini`
|
||||
`.../azure-sdk-for-python/sdk/textanalytics/azure-ai-textanalytics>tox run -e pyright -c ../../../eng/tox/tox.ini --root .`
|
||||
|
||||
If you don't want to use `tox` you can also install and run pyright on its own:
|
||||
|
||||
|
|
@ -249,7 +249,7 @@ from the code in the PR vs. the latest release on PyPi.
|
|||
|
||||
To run verifytypes on your library, run the tox verifytypes env at the package level:
|
||||
|
||||
`.../azure-sdk-for-python/sdk/textanalytics/azure-ai-textanalytics>tox -e verifytypes -c ../../../eng/tox/tox.ini`
|
||||
`.../azure-sdk-for-python/sdk/textanalytics/azure-ai-textanalytics>tox run -e verifytypes -c ../../../eng/tox/tox.ini --root .`
|
||||
|
||||
If you don't want to use `tox` you can also install and run pyright/verifytypes on its own:
|
||||
|
||||
|
|
|
|||
|
|
@ -123,20 +123,20 @@ If you have print statements in your tests for debugging you can add the `-s` fl
|
|||
|
||||
## Tox
|
||||
|
||||
The Python SDK uses the [tox project](https://tox.readthedocs.io/en/latest/) to automate releases, run tests, run linters, and build our documentation. The `tox.ini` file is located at `azure-sdk-for-python/eng/tox/tox.ini` for reference. You do not need to make any changes to the tox file for tox to work with your project. Tox will create a directory (`.tox`) in the head of your branch. The first time you run tox commands it may take several moments, but subsequent runs will be quicker. To install tox run the following command from within your virtual environment.
|
||||
`(env) > pip install tox tox-monorepo`.
|
||||
The Python SDK uses the [tox project](https://tox.wiki/en/latest/) to automate releases, run tests, run linters, and build our documentation. The `tox.ini` file is located at `azure-sdk-for-python/eng/tox/tox.ini` for reference. You do not need to make any changes to the tox file for tox to work with your project. Tox will create a directory (`.tox`) in the head of your branch. The first time you run tox commands it may take several moments, but subsequent runs will be quicker. To install tox run the following command from within your virtual environment.
|
||||
`(env) > pip install tox<5`.
|
||||
|
||||
To run a tox command from your directory use the following commands:
|
||||
```cmd
|
||||
(env) azure-sdk-for-python\sdk\my-service\my-package> tox -c ../../../eng/tox/tox.ini -e sphinx
|
||||
(env) azure-sdk-for-python\sdk\my-service\my-package> tox -c ../../../eng/tox/tox.ini -e pylint
|
||||
(env) azure-sdk-for-python\sdk\my-service\my-package> tox -c ../../../eng/tox/tox.ini -e mypy
|
||||
(env) azure-sdk-for-python\sdk\my-service\my-package> tox -c ../../../eng/tox/tox.ini -e pyright
|
||||
(env) azure-sdk-for-python\sdk\my-service\my-package> tox -c ../../../eng/tox/tox.ini -e verifytypes
|
||||
(env) azure-sdk-for-python\sdk\my-service\my-package> tox -c ../../../eng/tox/tox.ini -e whl
|
||||
(env) azure-sdk-for-python\sdk\my-service\my-package> tox -c ../../../eng/tox/tox.ini -e sdist
|
||||
(env) azure-sdk-for-python\sdk\my-service\my-package> tox -c ../../../eng/tox/tox.ini -e samples
|
||||
(env) azure-sdk-for-python\sdk\my-service\my-package> tox -c ../../../eng/tox/tox.ini -e apistub
|
||||
(env) azure-sdk-for-python\sdk\my-service\my-package> tox run -e sphinx -c ../../../eng/tox/tox.ini --root .
|
||||
(env) azure-sdk-for-python\sdk\my-service\my-package> tox run -e pylint -c ../../../eng/tox/tox.ini --root .
|
||||
(env) azure-sdk-for-python\sdk\my-service\my-package> tox run -e mypy -c ../../../eng/tox/tox.ini --root .
|
||||
(env) azure-sdk-for-python\sdk\my-service\my-package> tox run -e pyright -c ../../../eng/tox/tox.ini --root .
|
||||
(env) azure-sdk-for-python\sdk\my-service\my-package> tox run -e verifytypes -c ../../../eng/tox/tox.ini --root .
|
||||
(env) azure-sdk-for-python\sdk\my-service\my-package> tox run -e whl -c ../../../eng/tox/tox.ini --root .
|
||||
(env) azure-sdk-for-python\sdk\my-service\my-package> tox run -e sdist -c ../../../eng/tox/tox.ini --root .
|
||||
(env) azure-sdk-for-python\sdk\my-service\my-package> tox run -e samples -c ../../../eng/tox/tox.ini --root .
|
||||
(env) azure-sdk-for-python\sdk\my-service\my-package> tox run -e apistub -c ../../../eng/tox/tox.ini --root .
|
||||
```
|
||||
A quick description of the five commands above:
|
||||
* sphinx: documentation generation using the inline comments written in our code
|
||||
|
|
|
|||
|
|
@ -169,14 +169,14 @@ Analyze job in both nightly CI and pull request validation pipeline runs a set o
|
|||
[`MyPy`](https://pypi.org/project/mypy/) is a static analysis tool that runs type checking of python package. Following are the steps to run `MyPy` locally for a specific package:
|
||||
|
||||
1. Go to root of the package
|
||||
2. Execute following command: `tox -e mypy -c ../../../eng/tox/tox.ini`
|
||||
2. Execute following command: `tox run -e mypy -c ../../../eng/tox/tox.ini --root .`
|
||||
|
||||
### Pyright
|
||||
|
||||
[`Pyright`](https://github.com/microsoft/pyright/) is a static analysis tool that runs type checking of python package. Following are the steps to run `pyright` locally for a specific package:
|
||||
|
||||
1. Go to root of the package
|
||||
2. Execute following command: `tox -e pyright -c ../../../eng/tox/tox.ini`
|
||||
2. Execute following command: `tox run -e pyright -c ../../../eng/tox/tox.ini --root .`
|
||||
|
||||
|
||||
### Verifytypes
|
||||
|
|
@ -184,7 +184,7 @@ Analyze job in both nightly CI and pull request validation pipeline runs a set o
|
|||
[`Verifytypes`](https://github.com/microsoft/pyright/blob/main/docs/typed-libraries.md#verifying-type-completeness) is a feature of pyright that checks the type completeness of a python package. Following are the steps to run `verifytypes` locally for a specific package:
|
||||
|
||||
1. Go to root of the package
|
||||
2. Execute following command: `tox -e verifytypes -c ../../../eng/tox/tox.ini`
|
||||
2. Execute following command: `tox run -e verifytypes -c ../../../eng/tox/tox.ini --root .`
|
||||
|
||||
|
||||
### Pylint
|
||||
|
|
@ -192,14 +192,14 @@ Analyze job in both nightly CI and pull request validation pipeline runs a set o
|
|||
[`Pylint`](https://pypi.org/project/pylint/) is a static analysis tool to run lint checking, it is automatically run on all PRs. Following are the steps to run `pylint` locally for a specific package.
|
||||
|
||||
1. Go to root of the package.
|
||||
2. Execute following command: `tox -e pylint -c ../../../eng/tox/tox.ini`
|
||||
2. Execute following command: `tox run -e pylint -c ../../../eng/tox/tox.ini --root .`
|
||||
|
||||
### Bandit
|
||||
|
||||
`Bandit` is static security analysis tool. This check is triggered for all Azure SDK package as part of analyze job. Following are the steps to `Bandit` tool locally for a specific package.
|
||||
|
||||
1. Got to package root directory.
|
||||
2. Execute command: `tox -e bandit -c ../../../eng/tox/tox.ini`
|
||||
2. Execute command: `tox run -e bandit -c ../../../eng/tox/tox.ini --root .`
|
||||
|
||||
### ApiStubGen
|
||||
|
||||
|
|
@ -255,7 +255,7 @@ This test installs wheel of the package being tested and runs all tests cases in
|
|||
|
||||
1. Go to package root folder on a command line
|
||||
2. Run following command
|
||||
``tox -e whl -c ../../../eng/tox/tox.ini``
|
||||
`tox run -e whl -c ../../../eng/tox/tox.ini --root .`
|
||||
|
||||
#### sdist
|
||||
|
||||
|
|
@ -263,7 +263,7 @@ This test installs sdist of the package being tested and runs all tests cases in
|
|||
|
||||
1. Go to package root folder on a command line
|
||||
2. Run following command
|
||||
``tox -e sdist -c ../../../eng/tox/tox.ini``
|
||||
`tox run -e sdist -c ../../../eng/tox/tox.ini --root .`
|
||||
|
||||
#### depends
|
||||
|
||||
|
|
@ -273,7 +273,7 @@ Following is the command to run this test environment locally.
|
|||
|
||||
1. Go to package root folder on a command line
|
||||
2. Run following command
|
||||
``tox -e sdist -c ../../../eng/tox/tox.ini``
|
||||
`tox run -e sdist -c ../../../eng/tox/tox.ini --root .`
|
||||
|
||||
## Nightly CI Checks
|
||||
|
||||
|
|
@ -321,7 +321,7 @@ Note: Any dependency mentioned only in dev_requirements are not considered to id
|
|||
Tox name of this test is `latestdependency` and steps to manually run this test locally is as follows.
|
||||
|
||||
1. Go to package root. For e.g azure-storage-blob or azure-identity
|
||||
2. Run command `Tox –e latestdependency –c ../../../tox/tox.ini`
|
||||
2. Run command `tox run -e latestdependency -c ../../../tox/tox.ini --root .`
|
||||
|
||||
#### Minimum Dependency Test
|
||||
|
||||
|
|
@ -338,7 +338,7 @@ Tox name of this test is `mindependency` and steps to manually run this test loc
|
|||
|
||||
1. Go to package root. For e.g azure-storage-blob or azure-identity
|
||||
2. Run following command
|
||||
`Tox –e mindependency –c ../../../tox/tox.ini`
|
||||
`tox run -e mindependency -c ../../../tox/tox.ini --root .`
|
||||
|
||||
#### Regression Test
|
||||
|
||||
|
|
|
|||
|
|
@ -1,28 +1,20 @@
|
|||
# requirements leveraged by ci tools
|
||||
cryptography<4
|
||||
setuptools==44.1.0; python_version == '2.7'
|
||||
setuptools==67.6.0; python_version >= '3.5'
|
||||
virtualenv==20.13.2
|
||||
virtualenv==20.23.0
|
||||
wheel==0.37.0
|
||||
Jinja2==2.11.2
|
||||
packaging==20.4
|
||||
tox==3.25.1
|
||||
tox-monorepo==0.1.2
|
||||
packaging==23.1
|
||||
tox==4.5.0
|
||||
twine==3.1.1; python_version >= '3.6'
|
||||
pathlib2==2.3.5
|
||||
doc-warden==0.7.2
|
||||
# we pin coverage to 4.5.4 because there is an bug with `pytest-cov`. the generated coverage files cannot be `coverage combine`ed
|
||||
coverage==4.5.4
|
||||
beautifulsoup4==4.9.1
|
||||
pkginfo==1.5.0.1
|
||||
pip==20.3.3
|
||||
wrapt==1.12.1; python_version <= '3.10'
|
||||
wrapt==1.14.1; python_version >= '3.11'
|
||||
markupsafe==2.0.1; python_version > '2.7'
|
||||
markupsafe==1.1.1; python_version == '2.7'
|
||||
|
||||
# python 3.10 pinned packages
|
||||
cffi==1.15.0rc2; python_version >= '3.10'
|
||||
markupsafe==2.0.1
|
||||
|
||||
# locking packages defined as deps from azure-sdk-tools or azure-devtools
|
||||
pytoml==0.1.21
|
||||
|
|
@ -31,10 +23,9 @@ json-delta==2.0
|
|||
ConfigArgParse==1.2.3
|
||||
six==1.14.0
|
||||
pyyaml==5.3.1
|
||||
pytest==4.6.9; python_version == '2.7'
|
||||
pytest==5.4.2; python_version >= '3.5' and python_version <= '3.9'
|
||||
pytest==6.2.4; python_version >= '3.10'
|
||||
pytest-cov==2.8.1
|
||||
pytest==7.3.1
|
||||
pytest-cov==4.0.0
|
||||
coverage==7.2.5
|
||||
|
||||
# local dev packages
|
||||
./tools/azure-devtools
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ jobs:
|
|||
- script: |
|
||||
python -m pip freeze
|
||||
python -m pip --version
|
||||
python -m pip install setuptools==58.3.0 wheel==0.37.0 tox==3.24.3 tox-monorepo==0.1.2 packaging==21.0 requests
|
||||
python -m pip install setuptools==58.3.0 wheel==0.37.0 tox==4.5.0 packaging==23.1 requests
|
||||
python -m pip install $(Build.SourcesDirectory)/tools/azure-sdk-tools[build]
|
||||
displayName: Install Dependencies
|
||||
|
||||
|
|
@ -67,7 +67,7 @@ jobs:
|
|||
sudo apt-get install build-essential -y
|
||||
python -m pip freeze
|
||||
python -m pip --version
|
||||
python -m pip install setuptools==58.3.0 wheel==0.37.0 tox==3.24.3 tox-monorepo==0.1.2 packaging==21.0 requests
|
||||
python -m pip install setuptools==58.3.0 wheel==0.37.0 tox==4.5.0 packaging==23.1 requests
|
||||
python -m pip install $(Build.SourcesDirectory)/tools/azure-sdk-tools[build]
|
||||
displayName: Install Dependencies
|
||||
|
||||
|
|
@ -131,7 +131,7 @@ jobs:
|
|||
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
|
||||
python3 get-pip.py
|
||||
python3 -m pip install setuptools==67.6.0 wheel
|
||||
python3 -m pip install tox tox-monorepo packaging twine beautifulsoup4
|
||||
python3 -m pip install tox packaging twine beautifulsoup4
|
||||
python3 --version
|
||||
cd $(Build.SourcesDirectory)
|
||||
python3 ./scripts/devops_tasks/setup_execute_tests.py "$(BuildTargetingString)" --junitxml="junit/test_results_38.xml" --toxenv="whl" --filter-type="None"
|
||||
|
|
|
|||
|
|
@ -23,7 +23,7 @@ steps:
|
|||
python3 -m pip install -e "git+https://github.com/Azure/azure-sdk-for-python#subdirectory=tools/azure-sdk-tools&egg=azure-sdk-tools"
|
||||
displayName: 'Install Azure SDK tools'
|
||||
|
||||
- script: python3 -m packaging_tools.update_pr -v --pr-number $(System.PullRequest.PullRequestNumber) --repo $(Build.Repository.Name)
|
||||
- script: python3 -m packaging_tools.update_pr -v --pr-number $(System.PullRequest.PullRequestNumber) --repo $(Build.Repository.Name)
|
||||
displayName: 'Update packaging of PR'
|
||||
env:
|
||||
GH_TOKEN: $(azuresdk-github-pat)
|
||||
|
|
|
|||
|
|
@ -89,7 +89,7 @@ steps:
|
|||
CustomCondition: '${{ parameters.RunCoverage }}'
|
||||
|
||||
- pwsh: |
|
||||
python -m pip install coverage==4.5.4
|
||||
python -m pip install coverage==7.2.5
|
||||
python scripts/devops_tasks/create_coverage.py
|
||||
displayName: Create Coverage Report
|
||||
condition: and(succeeded(), ${{ parameters.RunCoverage }})
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ steps:
|
|||
versionSpec: 3.8
|
||||
|
||||
- pwsh: |
|
||||
python -m pip install packaging==20.4
|
||||
python -m pip install packaging==23.1
|
||||
displayName: Prep Environment
|
||||
|
||||
# select the appropriate version from manifest if present
|
||||
|
|
|
|||
|
|
@ -1,15 +1,17 @@
|
|||
pip==20.2
|
||||
cryptography<4
|
||||
|
||||
# requirements leveraged by ci for testing
|
||||
pytest==4.6.9; python_version == '2.7'
|
||||
pytest==5.4.2; python_version >= '3.5'
|
||||
pytest-asyncio==0.12.0; python_version >= '3.5'
|
||||
pytest-cov==2.8.1
|
||||
pytest==7.3.1
|
||||
pytest-asyncio==0.21.0
|
||||
pytest-cov==4.0.0
|
||||
pytest-custom-exit-code==0.3.0
|
||||
pytest-xdist==1.32.0
|
||||
# we pin coverage to 4.5.4 because there is an bug with `pytest-cov`. the generated coverage files cannot be `coverage combine`ed
|
||||
coverage==4.5.4
|
||||
pytest-xdist==3.2.1
|
||||
coverage==7.2.5
|
||||
bandit==1.6.2
|
||||
protobuf==3.17.3; python_version == '2.7'
|
||||
chardet>=2.0,<5.0
|
||||
cmarkgfm<0.7.0
|
||||
|
||||
# locking packages defined as deps from azure-sdk-tools or azure-devtools
|
||||
pytoml==0.1.21
|
||||
|
|
@ -19,15 +21,17 @@ json-delta==2.0
|
|||
ConfigArgParse==1.2.3
|
||||
six==1.14.0
|
||||
pyyaml==5.3.1
|
||||
packaging==20.4
|
||||
wheel==0.34.2
|
||||
packaging==23.1
|
||||
Jinja2==2.11.2
|
||||
markupsafe==2.0.1; python_version > '2.7'
|
||||
markupsafe==1.1.1; python_version == '2.7'
|
||||
wrapt==1.12.1; python_version <= '3.10'
|
||||
wrapt==1.14.1; python_version >= '3.11'
|
||||
|
||||
# Locking pylint and required packages
|
||||
pylint==1.8.4; python_version < '3.4'
|
||||
pylint==2.5.2; python_version >= '3.4'
|
||||
pylint==2.9.3; python_version >= '3.6' and python_version <= '3.10'
|
||||
pylint==2.14.5; python_version >= '3.11'
|
||||
|
||||
# python-dotenv
|
||||
python-dotenv==0.15.0
|
||||
|
|
@ -1,26 +1,20 @@
|
|||
# requirements leveraged by ci tools
|
||||
cryptography==3.3.2
|
||||
setuptools==44.1.0; python_version == '2.7'
|
||||
setuptools==67.6.0; python_version >= '3.5'
|
||||
virtualenv==20.0.23
|
||||
wheel==0.34.2
|
||||
virtualenv==20.23.0
|
||||
wheel==0.37.0
|
||||
Jinja2==2.11.2
|
||||
packaging==20.4
|
||||
tox==3.25.1
|
||||
tox-monorepo==0.1.2
|
||||
twine==1.15.0; python_version == '2.7' or python_version == '3.5'
|
||||
packaging==23.1
|
||||
tox==4.5.0
|
||||
twine==3.1.1; python_version >= '3.6'
|
||||
pathlib2==2.3.5
|
||||
readme-renderer[md]==25.0
|
||||
doc-warden==0.7.1
|
||||
# we pin coverage to 4.5.4 because there is an bug with `pytest-cov`. the generated coverage files cannot be `coverage combine`ed
|
||||
coverage==4.5.4
|
||||
doc-warden==0.7.2
|
||||
beautifulsoup4==4.9.1
|
||||
pkginfo==1.5.0.1
|
||||
pip==20.2
|
||||
black==21.6b0; python_version >= '3.6'
|
||||
markupsafe==2.0.1; python_version > '2.7'
|
||||
markupsafe==1.1.1; python_version == '2.7'
|
||||
pip==20.3.3
|
||||
wrapt==1.12.1; python_version <= '3.10'
|
||||
wrapt==1.14.1; python_version >= '3.11'
|
||||
markupsafe==2.0.1
|
||||
|
||||
# locking packages defined as deps from azure-sdk-tools or azure-devtools
|
||||
pytoml==0.1.21
|
||||
|
|
@ -29,10 +23,10 @@ json-delta==2.0
|
|||
ConfigArgParse==1.2.3
|
||||
six==1.14.0
|
||||
pyyaml==5.3.1
|
||||
pytest==5.4.2; python_version >= '3.5'
|
||||
pytest==4.6.9; python_version == '2.7'
|
||||
pytest-cov==2.8.1
|
||||
pytest==7.3.1
|
||||
pytest-cov==4.0.0
|
||||
coverage==7.2.5
|
||||
|
||||
# local dev packages
|
||||
./tools/azure-devtools
|
||||
./tools/azure-sdk-tools
|
||||
./tools/azure-sdk-tools[build]
|
||||
|
|
|
|||
|
|
@ -1,15 +1,12 @@
|
|||
cryptography<4
|
||||
|
||||
# requirements leveraged by ci for testing
|
||||
pytest==4.6.9; python_version == '2.7'
|
||||
pytest==5.4.2; python_version >= '3.5' and python_version <= '3.9'
|
||||
pytest==6.2.4; python_version >= '3.10'
|
||||
pytest-asyncio==0.12.0; python_version >= '3.5'
|
||||
pytest-cov==2.8.1
|
||||
pytest==7.3.1
|
||||
pytest-asyncio==0.21.0
|
||||
pytest-cov==4.0.0
|
||||
pytest-custom-exit-code==0.3.0
|
||||
pytest-xdist==1.32.0
|
||||
# we pin coverage to 4.5.4 because there is an bug with `pytest-cov`. the generated coverage files cannot be `coverage combine`ed
|
||||
coverage==4.5.4
|
||||
pytest-xdist==3.2.1
|
||||
coverage==7.2.5
|
||||
bandit==1.6.2
|
||||
protobuf==3.17.3; python_version == '2.7'
|
||||
chardet>=2.0,<5.0
|
||||
|
|
@ -23,7 +20,7 @@ json-delta==2.0
|
|||
ConfigArgParse==1.2.3
|
||||
six==1.14.0
|
||||
pyyaml==5.3.1
|
||||
packaging==20.4
|
||||
packaging==23.1
|
||||
Jinja2==2.11.2
|
||||
markupsafe==2.0.1; python_version > '2.7'
|
||||
markupsafe==1.1.1; python_version == '2.7'
|
||||
|
|
|
|||
285
eng/tox/tox.ini
285
eng/tox/tox.ini
|
|
@ -6,13 +6,18 @@
|
|||
# In all cases, whenever we install an azure-* package from a requirement (read: not a specific file), the only source will be from the dev feed.
|
||||
# once we've downloaded these dependencies, only then do we install other packages from pypi.
|
||||
[tox]
|
||||
requires=
|
||||
# Ensure that we're running a version of tox compatible with this config
|
||||
# 4.4.10 is the lower bound because it incorporates a fix for a bug with `--root`:
|
||||
# https://github.com/tox-dev/tox/pull/2962
|
||||
tox>=4.4.10
|
||||
# note that this envlist is the default set of environments that will run if a target environment is not selected.
|
||||
envlist = whl,sdist
|
||||
|
||||
|
||||
[tools]
|
||||
deps =
|
||||
-r ../../../eng/test_tools.txt
|
||||
-r {repository_root}/eng/test_tools.txt
|
||||
|
||||
|
||||
[coverage:paths]
|
||||
|
|
@ -31,27 +36,25 @@ deps =
|
|||
|
||||
[dependencytools]
|
||||
deps =
|
||||
-r ../../../eng/dependency_tools.txt
|
||||
-r {repository_root}/eng/dependency_tools.txt
|
||||
|
||||
|
||||
[packaging]
|
||||
pkgs =
|
||||
wheel==0.37.0
|
||||
packaging==20.4
|
||||
urllib3==1.26.12
|
||||
wheel==0.40.0
|
||||
packaging==23.1
|
||||
urllib3==1.26.15
|
||||
tomli==2.0.1
|
||||
|
||||
[pytest]
|
||||
ignore_args=--ignore=.tox --ignore=build --ignore=.eggs
|
||||
default_args = --junitxml={tox_root}/test-junit-{envname}.xml --verbose --durations=10 --ignore=azure {[pytest]ignore_args}
|
||||
|
||||
[testenv]
|
||||
ignore_args=--ignore=.tox --ignore=build --ignore=.eggs
|
||||
default_pytest_params = --junitxml={toxinidir}/test-junit-{envname}.xml --verbose --durations=10 --ignore=azure {[testenv]ignore_args}
|
||||
parallel_show_output =True
|
||||
skip_install = true
|
||||
skipsdist = true
|
||||
usedevelop = false
|
||||
platform = linux: linux
|
||||
macos: darwin
|
||||
windows: win32
|
||||
passenv = *
|
||||
download=true
|
||||
requires=
|
||||
|
|
@ -64,85 +67,83 @@ setenv =
|
|||
VIRTUALENV_PIP=20.3.3
|
||||
VIRTUALENV_SETUPTOOLS=67.6.0
|
||||
deps = {[base]deps}
|
||||
changedir = {toxinidir}
|
||||
install_command = python -m pip install {opts} {packages} --cache-dir {toxinidir}/../.tox_pip_cache_{envname} --ignore-installed
|
||||
install_command = python -m pip install {opts} {packages} --cache-dir {tox_root}/../.tox_pip_cache_{envname}
|
||||
commands =
|
||||
{envbindir}/python -m pip --version
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_package_and_install.py -d {envtmpdir} -p {toxinidir} -w {envtmpdir}
|
||||
{envbindir}/python -m pip freeze
|
||||
pytest \
|
||||
{[testenv]default_pytest_params} \
|
||||
{posargs} \
|
||||
{toxinidir}
|
||||
python -m pip --version
|
||||
python {repository_root}/eng/tox/create_package_and_install.py -d {envtmpdir} -p {tox_root} -w {envtmpdir}
|
||||
python -m pip freeze
|
||||
pytest {[pytest]default_args} {posargs} {tox_root}
|
||||
|
||||
|
||||
[testenv:pylint]
|
||||
description=Lints a package with a pinned version of pylint
|
||||
skipsdist = true
|
||||
skip_install = true
|
||||
usedevelop = false
|
||||
changedir = {toxinidir}
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
PROXY_URL=http://localhost:5002
|
||||
deps =
|
||||
{[base]deps}
|
||||
commands =
|
||||
{envbindir}/python -m pip install azure-pylint-guidelines-checker==0.0.5 --index-url="https://pkgs.dev.azure.com/azure-sdk/public/_packaging/azure-sdk-for-python/pypi/simple/"
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_package_and_install.py \
|
||||
-d {distdir} \
|
||||
-p {toxinidir} \
|
||||
python -m pip install azure-pylint-guidelines-checker==0.0.5 --index-url="https://pkgs.dev.azure.com/azure-sdk/public/_packaging/azure-sdk-for-python/pypi/simple/"
|
||||
python {repository_root}/eng/tox/create_package_and_install.py \
|
||||
-d {envtmpdir}/dist \
|
||||
-p {tox_root} \
|
||||
-w {envtmpdir} \
|
||||
--package-type sdist
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/run_pylint.py -t {toxinidir}
|
||||
python {repository_root}/eng/tox/run_pylint.py -t {tox_root}
|
||||
|
||||
[testenv:next-pylint]
|
||||
description=Lints a package with pylint (version {[testenv:next-pylint]pylint_version})
|
||||
pylint_version=2.15.8
|
||||
skipsdist = true
|
||||
skip_install = true
|
||||
usedevelop = false
|
||||
changedir = {toxinidir}
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
PROXY_URL=http://localhost:5002
|
||||
deps =
|
||||
{[base]deps}
|
||||
commands =
|
||||
{envbindir}/python -m pip install pylint==2.15.8
|
||||
{envbindir}/python -m pip install azure-pylint-guidelines-checker==0.0.8 --index-url="https://pkgs.dev.azure.com/azure-sdk/public/_packaging/azure-sdk-for-python/pypi/simple/"
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_package_and_install.py \
|
||||
-d {distdir} \
|
||||
-p {toxinidir} \
|
||||
python -m pip install {[testenv:next-pylint]pylint_version}
|
||||
python -m pip install azure-pylint-guidelines-checker==0.0.8 --index-url="https://pkgs.dev.azure.com/azure-sdk/public/_packaging/azure-sdk-for-python/pypi/simple/"
|
||||
python {repository_root}/eng/tox/create_package_and_install.py \
|
||||
-d {envtmpdir}/dist \
|
||||
-p {tox_root} \
|
||||
-w {envtmpdir} \
|
||||
--package-type sdist
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/run_pylint.py -t {toxinidir} --next=True
|
||||
python {repository_root}/eng/tox/run_pylint.py -t {tox_root} --next=True
|
||||
|
||||
[testenv:mypy]
|
||||
description=Typechecks a package with mypy (version {[testenv:mypy]mypy_version})
|
||||
mypy_version=1.0.0
|
||||
skipsdist = true
|
||||
skip_install = true
|
||||
usedevelop = true
|
||||
changedir = {toxinidir}
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
PROXY_URL=http://localhost:5003
|
||||
deps =
|
||||
{[base]deps}
|
||||
mypy==1.0.0
|
||||
mypy=={[testenv:mypy]mypy_version}
|
||||
types-chardet==4.0.3
|
||||
types-requests==2.27.9
|
||||
types-six==1.16.10
|
||||
types-redis==4.3.21.2
|
||||
commands =
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_package_and_install.py \
|
||||
-d {distdir} \
|
||||
-p {toxinidir} \
|
||||
python {repository_root}/eng/tox/create_package_and_install.py \
|
||||
-d {envtmpdir}/dist \
|
||||
-p {tox_root} \
|
||||
-w {envtmpdir} \
|
||||
--package-type sdist
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/run_mypy.py -t {toxinidir}
|
||||
python {repository_root}/eng/tox/run_mypy.py -t {tox_root}
|
||||
|
||||
[testenv:next-mypy]
|
||||
description=Typechecks a package with the latest version of mypy
|
||||
skipsdist = true
|
||||
skip_install = true
|
||||
usedevelop = true
|
||||
changedir = {toxinidir}
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
PROXY_URL=http://localhost:5020
|
||||
|
|
@ -154,41 +155,41 @@ deps =
|
|||
types-six
|
||||
types-redis
|
||||
commands =
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_package_and_install.py \
|
||||
-d {distdir} \
|
||||
-p {toxinidir} \
|
||||
python {repository_root}/eng/tox/create_package_and_install.py \
|
||||
-d {envtmpdir}/dist \
|
||||
-p {tox_root} \
|
||||
-w {envtmpdir} \
|
||||
--package-type sdist
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/run_mypy.py -t {toxinidir} --next=True
|
||||
python {repository_root}/eng/tox/run_mypy.py -t {tox_root} --next=True
|
||||
|
||||
|
||||
[testenv:pyright]
|
||||
description=Typechecks a package with pyright (version {[testenv:pyright]pyright_version})
|
||||
pyright_version=1.1.287
|
||||
skipsdist = true
|
||||
skip_install = true
|
||||
usedevelop = true
|
||||
changedir = {toxinidir}
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
PROXY_URL=http://localhost:5018
|
||||
deps =
|
||||
{[base]deps}
|
||||
pyright==1.1.287
|
||||
pyright=={[testenv:pyright]pyright_version}
|
||||
commands =
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_package_and_install.py \
|
||||
-d {distdir} \
|
||||
-p {toxinidir} \
|
||||
python {repository_root}/eng/tox/create_package_and_install.py \
|
||||
-d {envtmpdir}/dist \
|
||||
-p {tox_root} \
|
||||
-w {envtmpdir} \
|
||||
--package-type sdist
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_dependencies_and_install.py \
|
||||
-p {toxinidir}
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/run_pyright.py -t {toxinidir}
|
||||
python {repository_root}/eng/tox/create_dependencies_and_install.py -p {tox_root}
|
||||
python {repository_root}/eng/tox/run_pyright.py -t {tox_root}
|
||||
|
||||
|
||||
[testenv:next-pyright]
|
||||
description=Typechecks a package with the latest version of static type-checker pyright
|
||||
skipsdist = true
|
||||
skip_install = true
|
||||
usedevelop = true
|
||||
changedir = {toxinidir}
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
PROXY_URL=http://localhost:5021
|
||||
|
|
@ -196,21 +197,20 @@ deps =
|
|||
{[base]deps}
|
||||
pyright
|
||||
commands =
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_package_and_install.py \
|
||||
-d {distdir} \
|
||||
-p {toxinidir} \
|
||||
python {repository_root}/eng/tox/create_package_and_install.py \
|
||||
-d {envtmpdir}/dist \
|
||||
-p {tox_root} \
|
||||
-w {envtmpdir} \
|
||||
--package-type sdist
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_dependencies_and_install.py \
|
||||
-p {toxinidir}
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/run_pyright.py -t {toxinidir} --next=True
|
||||
python {repository_root}/eng/tox/create_dependencies_and_install.py -p {tox_root}
|
||||
python {repository_root}/eng/tox/run_pyright.py -t {tox_root} --next=True
|
||||
|
||||
|
||||
[testenv:verifytypes]
|
||||
description=Verifies the "type completeness" of a package with pyright
|
||||
skipsdist = true
|
||||
skip_install = true
|
||||
usedevelop = true
|
||||
changedir = {toxinidir}
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
PROXY_URL=http://localhost:5019
|
||||
|
|
@ -218,81 +218,72 @@ deps =
|
|||
{[base]deps}
|
||||
pyright==1.1.287
|
||||
commands =
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_package_and_install.py \
|
||||
-d {distdir} \
|
||||
-p {toxinidir} \
|
||||
python {repository_root}/eng/tox/create_package_and_install.py \
|
||||
-d {envtmpdir}/dist \
|
||||
-p {tox_root} \
|
||||
-w {envtmpdir} \
|
||||
--package-type sdist
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_dependencies_and_install.py \
|
||||
-p {toxinidir}
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/run_verifytypes.py -t {toxinidir}
|
||||
python {repository_root}/eng/tox/create_dependencies_and_install.py -p {tox_root}
|
||||
python {repository_root}/eng/tox/run_verifytypes.py -t {tox_root}
|
||||
|
||||
|
||||
[testenv:whl_no_aio]
|
||||
description=Builds a wheel without aio and runs tests
|
||||
skipsdist = true
|
||||
skip_install = true
|
||||
changedir = {toxinidir}
|
||||
deps =
|
||||
{[base]deps}
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
PROXY_URL=http://localhost:5004
|
||||
commands =
|
||||
- {envbindir}/python -m pip uninstall aiohttp --yes
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_package_and_install.py -d {envtmpdir} -p {toxinidir} -w {envtmpdir}
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/try_import.py aiohttp -p {toxinidir}
|
||||
{envbindir}/python -m pip freeze
|
||||
pytest \
|
||||
{[testenv]default_pytest_params} \
|
||||
--ignore-glob='*async*.py' \
|
||||
{posargs} \
|
||||
{toxinidir}
|
||||
- python -m pip uninstall aiohttp --yes
|
||||
python {repository_root}/eng/tox/create_package_and_install.py -d {envtmpdir} -p {tox_root} -w {envtmpdir}
|
||||
python {repository_root}/eng/tox/try_import.py aiohttp -p {tox_root}
|
||||
python -m pip freeze
|
||||
pytest {[pytest]default_args} --ignore-glob='*async*.py' {posargs} --no-cov {tox_root}
|
||||
|
||||
|
||||
[testenv:whl]
|
||||
description=Builds a wheel and runs tests
|
||||
|
||||
[testenv:sdist]
|
||||
description=Builds a source distribution and runs tests
|
||||
skipsdist = true
|
||||
skip_install = true
|
||||
changedir = {toxinidir}
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
PROXY_URL=http://localhost:5005
|
||||
deps =
|
||||
{[base]deps}
|
||||
commands =
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_package_and_install.py \
|
||||
python {repository_root}/eng/tox/create_package_and_install.py \
|
||||
-d {envtmpdir} \
|
||||
-p {toxinidir} \
|
||||
-p {tox_root} \
|
||||
-w {envtmpdir} \
|
||||
--package-type sdist
|
||||
{envbindir}/python -m pip freeze
|
||||
pytest \
|
||||
{posargs} \
|
||||
--no-cov \
|
||||
{[testenv]ignore_args} \
|
||||
{toxinidir}
|
||||
python -m pip freeze
|
||||
pytest {posargs} --no-cov {[pytest]ignore_args} {tox_root}
|
||||
|
||||
|
||||
[testenv:develop]
|
||||
description=Tests a package
|
||||
skipsdist = false
|
||||
skip_install = false
|
||||
usedevelop = true
|
||||
changedir = {toxinidir}
|
||||
deps =
|
||||
{[base]deps}
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
PROXY_URL=http://localhost:5006
|
||||
commands =
|
||||
pytest \
|
||||
{posargs} \
|
||||
--ignore=.tox \
|
||||
{toxinidir}
|
||||
pytest {posargs} --ignore=.tox {tox_root}
|
||||
|
||||
|
||||
[testenv:sphinx]
|
||||
description=Builds a package's documentation with sphinx
|
||||
skipsdist = true
|
||||
skip_install = true
|
||||
changedir = {toxinidir}
|
||||
passenv = *
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
|
|
@ -305,27 +296,21 @@ deps =
|
|||
mistune<2.0.0
|
||||
m2r==0.2.1
|
||||
commands =
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_package_and_install.py \
|
||||
-d {distdir} \
|
||||
-p {toxinidir} \
|
||||
python {repository_root}/eng/tox/create_package_and_install.py \
|
||||
-d {envtmpdir}/dist \
|
||||
-p {tox_root} \
|
||||
-w {envtmpdir} \
|
||||
--package-type sdist
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/prep_sphinx_env.py -d {distdir} -t {toxinidir}
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/run_sphinx_apidoc.py \
|
||||
-w {distdir} \
|
||||
-r {toxinidir}
|
||||
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/run_sphinx_build.py \
|
||||
-w {distdir}/unzipped/docgen \
|
||||
-o {distdir}/site \
|
||||
-r {toxinidir}
|
||||
python {repository_root}/eng/tox/prep_sphinx_env.py -d {envtmpdir}/dist -t {tox_root}
|
||||
python {repository_root}/eng/tox/run_sphinx_apidoc.py -w {envtmpdir}/dist -r {tox_root}
|
||||
python {repository_root}/eng/tox/run_sphinx_build.py \
|
||||
-w {envtmpdir}/dist/unzipped/docgen \
|
||||
-o {envtmpdir}/dist/site \
|
||||
-r {tox_root}
|
||||
|
||||
|
||||
[testenv:depends]
|
||||
platform = linux: linux
|
||||
macos: darwin
|
||||
windows: win32
|
||||
changedir = {toxinidir}
|
||||
description = Ensures all modules in a target package can be successfully imported
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
PROXY_URL=http://localhost:5008
|
||||
|
|
@ -333,78 +318,73 @@ deps =
|
|||
{[packaging]pkgs}
|
||||
cryptography<4
|
||||
commands =
|
||||
{envbindir}/python -m pip install {toxinidir}/../../../tools/azure-sdk-tools --no-deps
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/sanitize_setup.py -t {toxinidir}
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_package_and_install.py \
|
||||
python -m pip install {repository_root}/tools/azure-sdk-tools --no-deps
|
||||
python {repository_root}/eng/tox/sanitize_setup.py -t {tox_root}
|
||||
python {repository_root}/eng/tox/create_package_and_install.py \
|
||||
-d {envtmpdir} \
|
||||
-p {toxinidir} \
|
||||
-p {tox_root} \
|
||||
-w {envtmpdir} \
|
||||
--force-create true
|
||||
{envbindir}/python -m pip freeze
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/import_all.py -t {toxinidir}
|
||||
python -m pip freeze
|
||||
python {repository_root}/eng/tox/import_all.py -t {tox_root}
|
||||
|
||||
|
||||
[testenv:verifywhl]
|
||||
description=Verify directories included in whl and contents in manifest file
|
||||
skipsdist = true
|
||||
skip_install = true
|
||||
changedir = {toxinidir}
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
PROXY_URL=http://localhost:5009
|
||||
deps =
|
||||
{[packaging]pkgs}
|
||||
commands =
|
||||
{envbindir}/python -m pip install {toxinidir}/../../../tools/azure-sdk-tools --no-deps
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_package_and_install.py -d {envtmpdir} -p {toxinidir} --skip-install True
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/verify_whl.py -d {envtmpdir} -t {toxinidir}
|
||||
python -m pip install {repository_root}/tools/azure-sdk-tools --no-deps
|
||||
python {repository_root}/eng/tox/create_package_and_install.py -d {envtmpdir} -p {tox_root} --skip-install True
|
||||
python {repository_root}/eng/tox/verify_whl.py -d {envtmpdir} -t {tox_root}
|
||||
|
||||
|
||||
[testenv:verifysdist]
|
||||
description=Verify directories included in sdist and contents in manifest file. Also ensures that py.typed configuration is correct within the setup.py
|
||||
skipsdist = true
|
||||
skip_install = true
|
||||
changedir = {toxinidir}
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
PROXY_URL=http://localhost:5010
|
||||
deps =
|
||||
{[packaging]pkgs}
|
||||
commands =
|
||||
{envbindir}/python -m pip install {toxinidir}/../../../tools/azure-sdk-tools --no-deps
|
||||
{envbindir}/python {toxinidir}/setup.py --q sdist --format zip -d {envtmpdir}
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/verify_sdist.py -d {envtmpdir} -t {toxinidir}
|
||||
python -m pip install {repository_root}/tools/azure-sdk-tools --no-deps
|
||||
python {tox_root}/setup.py --q sdist --format zip -d {envtmpdir}
|
||||
python {repository_root}/eng/tox/verify_sdist.py -d {envtmpdir} -t {tox_root}
|
||||
|
||||
|
||||
[testenv:devtest]
|
||||
description=Tests a package against dependencies installed from a dev index
|
||||
deps = {[base]deps}
|
||||
changedir = {toxinidir}
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
PROXY_URL=http://localhost:5011
|
||||
commands =
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_package_and_install.py -d {envtmpdir} -p {toxinidir}
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/install_dev_build_dependency.py -t {toxinidir}
|
||||
pytest \
|
||||
{[testenv]default_pytest_params} \
|
||||
--ignore=.tox \
|
||||
{posargs} \
|
||||
{toxinidir}
|
||||
python {repository_root}/eng/tox/create_package_and_install.py -d {envtmpdir} -p {tox_root}
|
||||
python {repository_root}/eng/tox/install_dev_build_dependency.py -t {tox_root}
|
||||
pytest {[pytest]default_args} --ignore=.tox {posargs} {tox_root}
|
||||
|
||||
|
||||
[deptestcommands]
|
||||
commands =
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/install_depend_packages.py -t {toxinidir} -d {env:DEPENDENCY_TYPE:} -w {envtmpdir}
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_package_and_install.py -d {envtmpdir} -p {toxinidir} -w {envtmpdir} --pre-download-disabled
|
||||
{envbindir}/python -m pip freeze
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/verify_installed_packages.py --packages-file {envtmpdir}/packages.txt
|
||||
pytest {[testenv]default_pytest_params} {posargs} --no-cov {toxinidir}
|
||||
python {repository_root}/eng/tox/install_depend_packages.py -t {tox_root} -d {env:DEPENDENCY_TYPE:} -w {envtmpdir}
|
||||
python {repository_root}/eng/tox/create_package_and_install.py -d {envtmpdir} -p {tox_root} -w {envtmpdir} --pre-download-disabled
|
||||
python -m pip freeze
|
||||
python {repository_root}/eng/tox/verify_installed_packages.py --packages-file {envtmpdir}/packages.txt
|
||||
pytest {[pytest]default_args} {posargs} --no-cov {tox_root}
|
||||
|
||||
|
||||
[testenv:latestdependency]
|
||||
description=Tests a package against the released, upper-bound versions of its azure dependencies
|
||||
deps =
|
||||
{[dependencytools]deps}
|
||||
{[tools]deps}
|
||||
changedir =
|
||||
{toxinidir}
|
||||
passenv = *
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
|
|
@ -415,13 +395,13 @@ commands =
|
|||
|
||||
|
||||
[testenv:mindependency]
|
||||
description=Tests a package against the released, lower-bound versions of its azure dependencies
|
||||
deps =
|
||||
azure-mgmt-keyvault<7.0.0
|
||||
azure-mgmt-resource<15.0.0
|
||||
azure-mgmt-storage<15.0.0
|
||||
{[dependencytools]deps}
|
||||
{[tools]deps}
|
||||
changedir = {toxinidir}
|
||||
passenv = *
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
|
|
@ -432,6 +412,7 @@ commands =
|
|||
|
||||
|
||||
[testenv:apistub]
|
||||
description=Generate an api stub of a package ( for https://apiview.dev )
|
||||
skipsdist = true
|
||||
skip_install = true
|
||||
usedevelop = false
|
||||
|
|
@ -443,12 +424,13 @@ deps =
|
|||
{[base]deps}
|
||||
commands =
|
||||
# install API stub generator
|
||||
{envbindir}/python -m pip install -r {toxinidir}/../../../eng/apiview_reqs.txt --index-url="https://pkgs.dev.azure.com/azure-sdk/public/_packaging/azure-sdk-for-python/pypi/simple/"
|
||||
{envbindir}/python -m pip freeze
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/run_apistubgen.py -t {toxinidir} -w {envtmpdir} {posargs}
|
||||
python -m pip install -r {repository_root}/eng/apiview_reqs.txt --index-url="https://pkgs.dev.azure.com/azure-sdk/public/_packaging/azure-sdk-for-python/pypi/simple/"
|
||||
python -m pip freeze
|
||||
python {repository_root}/eng/tox/run_apistubgen.py -t {tox_root} -w {envtmpdir} {posargs}
|
||||
|
||||
|
||||
[testenv:bandit]
|
||||
description=Runs bandit, a tool to find common security issues, against a package
|
||||
skipsdist = true
|
||||
skip_install = true
|
||||
usedevelop = false
|
||||
|
|
@ -460,16 +442,17 @@ deps =
|
|||
{[base]deps}
|
||||
importlib-metadata<5.0
|
||||
commands =
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_package_and_install.py \
|
||||
python {repository_root}/eng/tox/create_package_and_install.py \
|
||||
-d {envtmpdir} \
|
||||
-p {toxinidir} \
|
||||
-p {tox_root} \
|
||||
-w {envtmpdir} \
|
||||
--package-type sdist
|
||||
{envbindir}/python -m pip freeze
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/run_bandit.py -t {toxinidir}
|
||||
python -m pip freeze
|
||||
python {repository_root}/eng/tox/run_bandit.py -t {tox_root}
|
||||
|
||||
|
||||
[testenv:samples]
|
||||
description=Runs a package's samples
|
||||
skipsdist = false
|
||||
skip_install = false
|
||||
usedevelop = false
|
||||
|
|
@ -481,26 +464,26 @@ deps =
|
|||
{[base]deps}
|
||||
subprocess32; python_version < '3.5'
|
||||
commands =
|
||||
{envbindir}/python -m pip freeze
|
||||
{envbindir}/python {toxinidir}/../../../scripts/devops_tasks/test_run_samples.py -t {toxinidir}
|
||||
python -m pip freeze
|
||||
python {repository_root}/scripts/devops_tasks/test_run_samples.py -t {tox_root}
|
||||
|
||||
|
||||
[testenv:breaking]
|
||||
description=Runs the breaking changes checker against a package
|
||||
skipsdist = true
|
||||
skip_install = true
|
||||
usedevelop = true
|
||||
changedir = {toxinidir}
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
PROXY_URL=http://localhost:5017
|
||||
deps =
|
||||
{[base]deps}
|
||||
jsondiff==1.2.0
|
||||
-e {toxinidir}/../../scripts/breaking_changes_checker
|
||||
-e {repository_root}/scripts/breaking_changes_checker
|
||||
commands =
|
||||
{envbindir}/python {toxinidir}/../../../eng/tox/create_package_and_install.py \
|
||||
-d {distdir} \
|
||||
-p {toxinidir} \
|
||||
python {repository_root}/eng/tox/create_package_and_install.py \
|
||||
-d {envtmpdir}/dist \
|
||||
-p {tox_root} \
|
||||
-w {envtmpdir} \
|
||||
--package-type sdist
|
||||
{envbindir}/python {toxinidir}/../../../scripts/breaking_changes_checker/detect_breaking_changes.py -t {toxinidir}
|
||||
python {repository_root}/scripts/breaking_changes_checker/detect_breaking_changes.py -t {tox_root}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,19 @@
|
|||
from logging import getLogger
|
||||
from pathlib import Path
|
||||
|
||||
from tox import plugin
|
||||
from tox.config.sets import CoreConfigSet
|
||||
from tox.session.state import State
|
||||
|
||||
logger = getLogger(__file__)
|
||||
|
||||
NAME = "azure-sdk-for-python tox plugin"
|
||||
|
||||
@plugin.impl
|
||||
def tox_add_core_config(core_conf: CoreConfigSet, state: State):
|
||||
core_conf.add_constant(
|
||||
"repository_root",
|
||||
"The root of this git repository",
|
||||
next(p for p in Path(core_conf["config_file_path"]).resolve().parents if (p / ".git").exists()),
|
||||
)
|
||||
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
azure-identity
|
||||
python-dotenv==0.15.0
|
||||
ghapi==0.1.19
|
||||
packaging==21.3
|
||||
packaging==23.1
|
||||
pytest==6.2.5
|
||||
azure-storage-blob==12.9.0
|
||||
fastcore==1.3.25
|
||||
|
|
@ -9,15 +9,15 @@ Add your package name to the `RUN_BREAKING_CHANGES_PACKAGES` found [here](https:
|
|||
|
||||
## Run locally with tox
|
||||
|
||||
**1) Install tox and tox-monorepo:**
|
||||
**1) Install tox:**
|
||||
|
||||
`pip install tox tox-monorepo`
|
||||
`pip install tox<5`
|
||||
|
||||
**2) Run the `breaking` environment.**
|
||||
|
||||
Here we run the breaking changes tool against azure-storage-blob, for example:
|
||||
|
||||
`C:\azure-sdk-for-python\sdk\storage\azure-storage-blob>tox -c ../../../eng/tox/tox.ini -e breaking`
|
||||
`C:\azure-sdk-for-python\sdk\storage\azure-storage-blob>tox run -c ../../../eng/tox/tox.ini --root . -e breaking`
|
||||
|
||||
|
||||
## Ignore a reported breaking change
|
||||
|
|
|
|||
|
|
@ -32,7 +32,6 @@ def collect_tox_coverage_files():
|
|||
for f in files:
|
||||
if re.match(".coverage_*", f):
|
||||
coverage_files.append(os.path.join(root, f))
|
||||
fix_dot_coverage_file(os.path.join(root, f))
|
||||
|
||||
logging.info(".coverage files: {}".format(coverage_files))
|
||||
|
||||
|
|
@ -59,20 +58,25 @@ def generate_coverage_xml():
|
|||
logging.error("Coverage file is not available in {} to generate coverage XML".format(coverage_dir))
|
||||
|
||||
|
||||
def fix_dot_coverage_file(coverage_file):
|
||||
def fix_coverage_xml(coverage_file):
|
||||
print("running 'fix_dot_coverage_file' on {}".format(coverage_file))
|
||||
|
||||
out = None
|
||||
with open(coverage_file) as cov_file:
|
||||
with open(coverage_file, encoding="utf-8") as cov_file:
|
||||
line = cov_file.read()
|
||||
out = re.sub("\/\.tox\/[\s\S]*?\/site-packages", "/", line)
|
||||
|
||||
# replace relative paths in folder structure pattern
|
||||
out = re.sub("\/\.tox\/[\s\S_]*?\/site-packages", "", line)
|
||||
|
||||
# replace relative paths in python import pattern
|
||||
out = re.sub("\.?\.tox[\s\S\.\d]*?\.site-packages", "", out)
|
||||
|
||||
if out:
|
||||
with open(coverage_file, 'w') as cov_file:
|
||||
cov_file.write(out)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
collect_tox_coverage_files()
|
||||
generate_coverage_xml()
|
||||
create_coverage_report()
|
||||
create_coverage_report()
|
||||
fix_coverage_xml(os.path.join(root_dir, 'coverage.xml'))
|
||||
|
|
@ -198,6 +198,25 @@ def collect_log_files(working_dir):
|
|||
for f in glob.glob(os.path.join(root_dir, "_tox_logs", "*")):
|
||||
logging.info("Log file: {}".format(f))
|
||||
|
||||
def cleanup_tox_environments(tox_dir: str, command_array: str) -> None:
|
||||
"""The new .coverage formats are no longer readily amended in place. Because we can't amend them in place,
|
||||
we can't amend the source location to remove the path ".tox/<envname>/site-packages/". Because of this, we will
|
||||
need the source where it was generated to stick around. We can do that by being a bit more circumspect about which
|
||||
files we actually delete/clean up!
|
||||
"""
|
||||
if "--cov-append" in command_array:
|
||||
folders = [folder for folder in os.listdir(tox_dir) if "whl" != folder]
|
||||
for folder in folders:
|
||||
try:
|
||||
shutil.rmtree(folder)
|
||||
except Exception as e:
|
||||
# git has a permissions problem. one of the files it drops
|
||||
# cannot be removed as no one has the permission to do so.
|
||||
# lets log just in case, but this should really only affect windows machines.
|
||||
logging.info(e)
|
||||
pass
|
||||
else:
|
||||
shutil.rmtree(tox_dir)
|
||||
|
||||
def execute_tox_serial(tox_command_tuples):
|
||||
return_code = 0
|
||||
|
|
@ -218,7 +237,8 @@ def execute_tox_serial(tox_command_tuples):
|
|||
|
||||
if in_ci():
|
||||
collect_log_files(cmd_tuple[1])
|
||||
shutil.rmtree(tox_dir)
|
||||
|
||||
cleanup_tox_environments(tox_dir, cmd_tuple[0])
|
||||
|
||||
if os.path.exists(clone_dir):
|
||||
try:
|
||||
|
|
@ -241,7 +261,7 @@ def prep_and_run_tox(targeted_packages: List[str], parsed_args: Namespace, optio
|
|||
:param parsed_args: An argparse namespace object from setup_execute_tests.py. Not including it will effectively disable "customizations"
|
||||
of the tox invocation.
|
||||
:param options_array: When invoking tox, these additional options will be passed to the underlying tox invocations as arguments.
|
||||
When invoking of "tox -e whl -c ../../../eng/tox/tox.ini -- --suppress-no-test-exit-code", "--suppress-no-test-exit-code" the "--" will be
|
||||
When invoking of "tox run -e whl -c ../../../eng/tox/tox.ini -- --suppress-no-test-exit-code", "--suppress-no-test-exit-code" the "--" will be
|
||||
passed directly to the pytest invocation.
|
||||
"""
|
||||
if parsed_args.wheel_dir:
|
||||
|
|
@ -260,6 +280,13 @@ def prep_and_run_tox(targeted_packages: List[str], parsed_args: Namespace, optio
|
|||
|
||||
tox_execution_array = [sys.executable, "-m", "tox"]
|
||||
|
||||
if parsed_args.tenvparallel:
|
||||
tox_execution_array.extend(["run-parallel", "-p", "all"])
|
||||
else:
|
||||
tox_execution_array.append("run")
|
||||
|
||||
# Tox command is run in package root, make tox set package root as {toxinidir}
|
||||
tox_execution_array += ["--root", "."]
|
||||
local_options_array = options_array[:]
|
||||
|
||||
# Get code coverage params for current package
|
||||
|
|
@ -324,8 +351,6 @@ def prep_and_run_tox(targeted_packages: List[str], parsed_args: Namespace, optio
|
|||
|
||||
tox_execution_array.extend(["-e", filtered_tox_environment_set])
|
||||
|
||||
if parsed_args.tenvparallel:
|
||||
tox_execution_array.extend(["-p", "all"])
|
||||
|
||||
if parsed_args.tox_env == "apistub":
|
||||
local_options_array = []
|
||||
|
|
|
|||
|
|
@ -2,5 +2,5 @@ PyGithub==1.55
|
|||
requests==2.28.0
|
||||
azure-devops==6.0.0b4
|
||||
msrest>=0.6.21
|
||||
packaging==23.0
|
||||
packaging==23.1
|
||||
urllib3==1.26.15
|
||||
|
|
@ -7,4 +7,4 @@ urllib3==1.26.6
|
|||
pytest==6.2.5
|
||||
pytest-cov==3.0.0
|
||||
PyGithub==1.55
|
||||
packaging==21.3
|
||||
packaging==23.1
|
||||
|
|
@ -9,10 +9,11 @@ from azure.core.rest._requests_asyncio import RestAsyncioRequestsTransportRespon
|
|||
from rest_client_async import AsyncTestRestClient
|
||||
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
from utils import readonly_checks
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
@pytest_asyncio.fixture
|
||||
async def client(port):
|
||||
async with AsyncioRequestsTransport() as transport:
|
||||
async with AsyncTestRestClient(port, transport=transport) as client:
|
||||
|
|
|
|||
|
|
@ -5,6 +5,7 @@
|
|||
# license information.
|
||||
# -------------------------------------------------------------------------
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
from azure.core.pipeline.transport import HttpRequest as PipelineTransportHttpRequest
|
||||
from azure.core.rest import HttpRequest as RestHttpRequest
|
||||
from azure.core.pipeline import Pipeline
|
||||
|
|
@ -19,8 +20,7 @@ def old_request(port):
|
|||
return PipelineTransportHttpRequest("GET", "http://localhost:{}/streams/basic".format(port))
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
@pytest.mark.asyncio
|
||||
@pytest_asyncio.fixture
|
||||
async def get_old_response(old_request):
|
||||
async def _callback(transport, **kwargs):
|
||||
async with transport() as sender:
|
||||
|
|
@ -44,8 +44,7 @@ def new_request(port):
|
|||
return RestHttpRequest("GET", "http://localhost:{}/streams/basic".format(port))
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
@pytest.mark.asyncio
|
||||
@pytest_asyncio.fixture
|
||||
async def get_new_response(new_request):
|
||||
async def _callback(transport, **kwargs):
|
||||
async with transport() as sender:
|
||||
|
|
|
|||
|
|
@ -4,5 +4,3 @@
|
|||
-e ../../identity/azure-identity
|
||||
aiohttp>=3.0; python_version >= '3.5'
|
||||
aiodns>=2.0; python_version >= '3.5'
|
||||
tox>=3.20.0
|
||||
tox-monorepo>=0.1.2
|
||||
|
|
@ -11,26 +11,8 @@
|
|||
|
||||
Tox is the testing and virtual environment management tool that is used to verify our sdk will be installed correctly with different Python versions and interpreters. To run tox follow these instructions
|
||||
|
||||
```
|
||||
pip install tox tox-monorepo
|
||||
cd path/to/target/folder
|
||||
tox -c eng/tox/tox.ini
|
||||
```
|
||||
To run a specific tox command from your directory use the following commands:
|
||||
```bash
|
||||
> tox -c ../../../eng/tox/tox.ini -e sphinx
|
||||
> tox -c ../../../eng/tox/tox.ini -e pylint
|
||||
> tox -c ../../../eng/tox/tox.ini -e mypy
|
||||
> tox -c ../../../eng/tox/tox.ini -e whl
|
||||
> tox -c ../../../eng/tox/tox.ini -e sdist
|
||||
```
|
||||
A quick description of the five commands above:
|
||||
* sphinx: documentation generation using the inline comments written in our code
|
||||
* lint: runs pylint to make sure our code adheres to the style guidance
|
||||
* mypy: runs the mypy static type checker for Python to make sure that our types are valid
|
||||
* whl: creates a whl package for installing our package
|
||||
* sdist: creates a zipped distribution of our files that the end user could install with pip
|
||||
|
||||
Please see the repo's [contributing guide](https://github.com/Azure/azure-sdk-for-python/blob/main/CONTRIBUTING.md#building-and-testing) for instructions
|
||||
on how to install and run `tox`.
|
||||
|
||||
### Troubleshooting tox errors
|
||||
|
||||
|
|
|
|||
|
|
@ -4,5 +4,3 @@
|
|||
-e ../../identity/azure-identity
|
||||
aiohttp>=3.0
|
||||
aiodns>=2.0
|
||||
tox>=3.20.0
|
||||
tox-monorepo>=0.1.2
|
||||
|
|
@ -11,26 +11,8 @@
|
|||
|
||||
Tox is the testing and virtual environment management tool that is used to verify our sdk will be installed correctly with different Python versions and interpreters. To run tox follow these instructions
|
||||
|
||||
```
|
||||
pip install tox tox-monorepo
|
||||
cd path/to/target/folder
|
||||
tox -c eng/tox/tox.ini
|
||||
```
|
||||
To run a specific tox command from your directory use the following commands:
|
||||
```bash
|
||||
> tox -c ../../../eng/tox/tox.ini -e sphinx
|
||||
> tox -c ../../../eng/tox/tox.ini -e pylint
|
||||
> tox -c ../../../eng/tox/tox.ini -e mypy
|
||||
> tox -c ../../../eng/tox/tox.ini -e whl
|
||||
> tox -c ../../../eng/tox/tox.ini -e sdist
|
||||
```
|
||||
A quick description of the five commands above:
|
||||
* sphinx: documentation generation using the inline comments written in our code
|
||||
* lint: runs pylint to make sure our code adheres to the style guidance
|
||||
* mypy: runs the mypy static type checker for Python to make sure that our types are valid
|
||||
* whl: creates a whl package for installing our package
|
||||
* sdist: creates a zipped distribution of our files that the end user could install with pip
|
||||
|
||||
Please see the repo's [contributing guide](https://github.com/Azure/azure-sdk-for-python/blob/main/CONTRIBUTING.md#building-and-testing) for instructions
|
||||
on how to install and run `tox`.
|
||||
|
||||
### Troubleshooting tox errors
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
packaging==19.2
|
||||
packaging==23.1
|
||||
pyparsing==2.4.5
|
||||
six==1.13.0
|
||||
setuptools==67.6.0
|
||||
Загрузка…
Ссылка в новой задаче