* trim markupsafe, wrapt, and pyopenssl from azure-sdk-tools install set, add pyopenssl to keyvault certificates dev reqs
* update utilized versions of setuptools and wheel
This PR updates our Python testing framework to adopt the new TestProxy
which enables numerous common sanitizers.
Signed-off-by: Paul Van Eck <paulvaneck@microsoft.com>
* allow an override FILE to be honored by test-proxy-tool
* update local test-framework to honor proxy override version by default
---------
Co-authored-by: Scott Beddall (from Dev Box) <scbedd@microsoft.com>
With the common sanitizers added to test-proxy, several tests among our
SDKs are failing. This attempts to address a number of these failures.
Signed-off-by: Paul Van Eck <paulvaneck@microsoft.com>
* Add sanitizer enum and batch sanitizing method
* Add shared, fake credentials
* Add central set of sanitizers
* Remove KV custom matcher
* Add extra batch of sanitizers
* add async iterator random stream for async httpx
* update perf tests with same args for easier comparison of perf
* pauls comments
* update asynciterator comment to iterable
* use optional env for azure openai tests and remove v0 tests
* update param ToxTestEnv in live test template
* enable optional
* debug
* fix config options
* fixes to path for audio file and install aiohttp dev req
* remove debug logging line
* debugging test failure
* more debug
* undo debug testing, use pathlib for paths, target sys.executable for cli tests
* cspell ignores
* no logprobs
* probably need to ensure that sanitize_setup and create_build_and_install both submit True as well, but let's see if this works
* version_set_dev implies that we will always be working in a dev context, so should pass along dev=True
* add close_vnext_issue function
* add close_vnext_issue function to ci checks that create issues
* add strict sphinx
* log GH issue number we're closing
* remove 3.7
* require pygithub (and from there pynacl) from the default requirements set. this should allow testing on windows pypy39 to successfully install requirements
* remove python_version spec
* rebalance to force windows onto py310. there iwn't a wheel for pywin32 that works with pypy
* Update tools/azure-sdk-tools/setup.py
Encountered some issues with the generation.py script
when comments are in the dev_requirements.txt file. This
change correctly parses them.
Signed-off-by: Paul Van Eck <paulvaneck@microsoft.com>
* add strict-sphinx for issue creation
* add vnext_issue creator to run_sphinx_build
* add GH_TOKEN to generate docs to step
* update template
* update date to april deadline
* don't need args
* only open issues if running in analyze-weekly
* add in rohit's feedback
* try ci_enabled flag
* try using a gh env var
* fix import
* set env var, add filter in discover_targeted_packages
* add Public param to ci.yml test
* retry setting env var
* pass through template
* missing paren
* don't look for Public in ci.yml
* try again
* remove unrecognized arg
* print vars
* try checking build.reason
* few fixes + prints
* try testing skip
* try more prints
* try more prints
* try more prints
* try nonempty packages list
* ignore discover_targeted_packages for now
* try again
* try again
* try no parameter
* missed import
* accidentally deleted other var
* try again
* fix
* test set PULLREQUEST
* missed var
* set var correctly
* don't fail docs step
* remove condition
* comment
* fake pull request run
* with discover targeted packages filter run
* remove print debugs
* try only publish artifacts if they exist for docs
* remove filter for now
* try again
* set in right tempalte
* switch to a service we actually want to disable
* formatting
* feedback
* block on release if ci_enabled=false
* test disabling ci for template, should block on release
* remove test setting from azure-template
* remove unused import and add small doc
* updates to eng/ci_tools and eng/test_tools.txt to allow necessary additions for compatibility
* added new namespace to azure-sdk-tools: ci_tools.scenario.
* ci_tools.scenario is used to emplace any code that actually does the action of installing a dependency, replacing requirements, building on the fly, etc. This means that create_package_and_install and replace_dev_reqs both are moved into this module
* created tox environment 'optional'. This tox environment utilizes array items of pyproject.toml config item 'tool.azure-sdk-build.optional' to install requirements for specific test scenarios before invoking tests against them. These tests are run within a custom venv for each optional configuration item in pyproject.toml, location .tox/optional/<envname>/
* the new 'optional' environment is enhanced to allow users to pass in the target environment EG: 'tox -e optional --root . -c ../../../eng/tox/tox.ini -- --optional no_requests'
Co-authored-by: Laurent Mazuel <laurent.mazuel@gmail.com>
Co-authored-by: McCoy Patiño <39780829+mccoyp@users.noreply.github.com>
* update docs and debugging info
* Update doc/dev/code_snippets.md
Co-authored-by: Krista Pratico <krpratic@microsoft.com>
---------
Co-authored-by: Krista Pratico <krpratic@microsoft.com>
* Temporarily Scaffold new azure-storage-extensions package
* Update parse_functions.py to pick up extension modules and package names
* Add additional code testing parse_function for the new functionality
* Update build.py to invoke cibuildwheel to produce platform-specific wheels where necessary
Co-authored-by: Jacob Lauzon <jalauzon@microsoft.com>
Co-authored-by: Azure SDK Bot <adxpysdk@microsoft.com>
* updated replace_dev_reqs to handle all cases of relative installs to handle local relative deeps _deeper_ than the target package. Previously only handled relative deps starting with ../
* added unit tests for replace_dev_reqs to ensure that packages are replaced (or left alone) as expected
* apply black
* update scenario to include 'here'. update test to use automatic tempdir cleanup per Kevin's feedback
---------
Co-authored-by: kdestin <101366538+kdestin@users.noreply.github.com>
* cleaning up setup_execute_tests. removing code paths that have gone completely unused since being implemented (to mirror previous methodology if necessary). cleaning up arguments that aren't used. cleaning up imports. generally just tightening up of the script and clarifying it's purpose.
* adding the ability to update a pyproject.toml easily
* adding tests for new update functionality
Co-authored-by: McCoy Patiño <39780829+mccoyp@users.noreply.github.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
Co-authored-by: Scott Beddall (from Dev Box) <scbedd@microsoft.com>
* Add python 3.12 to platform matrix for tests
* Various updates to test_tools.txt and ci_tools.txt to enable compatibility with python 3.12 at test run time
* Adjust build matrix for metricsadvisor and tables to route python 3.11 from windows to ubuntu machine, bypassing issue #32409
* Add manually built cffi and aiohttp to test_tools.txt to enable async recorded tests on python 3.12
* Various updates to minimum range in install_depend_packages.py. All due to various levels of support for py 3.12
* Add the concept of platform specific override for minimum version, allowing the invoking platform to further refine the selection from pypi
* Updates and extensions to tools/azure-sdk-tools to handle cleanup of git idx files with wonky permissions
* Disable latestdependency and mindependency in azure-eventhub-checkpointstoreblob-aio. Follow-up issue #32422
* Various other small updates to enable further compatibility
Co-authored-by: Krista Pratico <krpratic@microsoft.com>
* updates to test sever, removing a couple of the pins from test_tools and ci_tools.txt
* remove bad readme_renderer requirement
* simplifying requirements. fixing requirement file for invoking analyze
* repair issues with pypy39 due to twine installation
* exclude azure-core-tracing-opencensus from sphinx build
* eliminate pytoml dependency
* Update Sphinx to v6.x
* safe import of tomli on <py311. import tomllib on >=py311. always pull in toml-w
Co-authored-by: Laurent Mazuel <laurent.mazuel@gmail.com>
* parse 'keywords' from setup.py in ParsedSetup class
* take advantage of parsed keywords to add new tox environment that verifies that 'azure sdk' is present in the keywords for our packages
* add check to 'analyze' step to invoke newly added keyword check
- Restructure per-service conda builds to a single, unified conda build located within `conda-sdk-client`
- This newly refactored pipeline takes the form of two stages. the first builds multiplatform conda packages against py38,39,310,311. the second builds all our universal (noarch) conda packages.
- Move all conda releaselogs under `conda/conda-releaselogs`
- Move all conda recipes from Xiang's personal build store into the official repo
- Refactor conda build pipeline from a script within `devops_tasks` directory to an entrypoint within `azure-sdk-tools`
- Update conda build within azure-sdk-tools to _also retrieve package data_ prior to assembling a conda package. This totally eliminates the need for a difference in CI and local invocation, and massively simplifies the CI pipeline steps
- Update conda version to 2023.09.01
* remove all references to *.zip files throughout CI
* source distribution and documentation artifact are now all tar.gz files
* update all references to artifact .zip -> .tar.gz
Co-authored-by: McCoy Patiño <39780829+mccoyp@users.noreply.github.com>
* try getting build id
* try more env vars in ci
* try print env var
* try print env var
* wip
* add vnext script
* update label for creating issue
* move dep
* add envar to vnext steps
* add gh token to pipeline
* fix package name and lines too long
* move vnext script to azure-sdk-tools/gh_tools per feedback
* cspell
* fix more cspell
Scott Beddall
Yuchao Yan
Paul Van Eck
Xiang Yan
Krista Pratico
Libba Lawrence
McCoy Patiño
Jacob Lauzon
Azure SDK Bot
swathipil
mccoyp
catalinaperalta
Patrick Hallisey
Scott Beddall
Laurent Mazuel
Chenjie Shi
ChenxiJiang333