* filter saved package properties by an optional runtime variable BuildTargetingString
* if this is set, the package set for the service will be further filtered to matching the targetingstring. globbing is supported EG azure-storage*
---------
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* transition validate_formatting.py to run_black.py called from within the tox env
* update filter_tox_env_string to handle check defaults that are nonTrue
* default black to opt-in, not opt-out
* change our verifyChangelogs over to utilizing the artifact details. if we don't have an artifact details at all, don't verify changelog. if we do, but skipverifychangelog is present, don't verify changelog. if we have artifact details, but no disable, verify the changelog
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* First version of client with endpoint operations
* Make Endpoints sample work
* Create EndpointProperties class. Implement client.endpoints.get_default
* Support AzureAIClient.from_connection_string()
* Make default input argument for .endpoints.get
* Re-emit from latest TypeSpec. First attempt to support 3 different endpoints
* Fix construction of the three endpoints
* Re-emit from TypeSpec. Rename host_name to endpoint
* Bug fix
* Add methods ai_client.inference.get_chat_completions_client() and ai_client.inference.get_azure_openai_client()
* Adding sample and changing client name (#37724)
* Add .inference.get_embeddings_client() method
* Jhakulin/azure ai client agents (#37748)
* generated agents
* merge latest
* update
* add patches
* update
* new version generated
* fixes
* fix merge error
* update
* use assistant_id for now
* update
* Create seperate inference samples
* Move SASTokenCredential into .models
* Add more agents samples (#37777)
* add more samples
* adjust sleep
* review updates
* Update _patch.py (#37760)
* Online Evaluation changes (#37794)
* Patch constructor for async AzureAIClient (#37804)
* Users/singankit/evaluation add data mapping (#37824)
* Update sample_evaluations.py
* Update sample_evaluations.py
* Fix .inference.get_azure_openai_client() for the async AzureAIClient (#37828)
* migrate async, poll functions, and samples from assistant to agent (#37831)
* migrate async, poll functions, and samples from assistant to agent
* migrate async, poll functions, and samples from assistant to agent
* resolved comments
* resolve comments
* using print instead of logging
* moved ai_client to main function
* Resolved comment
* Fixed samples
* [AI Client] adapt Sophia's assistant tests to client repo (#37819)
* [AI Client] adapt Sophia's assistant tests to client repo
* get tests running and passing with pytest
* recordings pass and pushed
* review feedback
* Took out streaming from create_run, create_and_process_run, and submit_tool_outputs_to_run and declare separate functions for streaming (#37851)
* Took out streaming from create_run, create_and_process_run, and submit_tool_outputs_to_run and declare separate functions for streaming
* Took out create_and_process_stream and update samples accordingly
* fixed samples
* Miscellaneous updates (#37852)
* Jhakulin/azure ai client pylance (#37887)
* Fixes to Pylance error
* fix sample
* update
* use file_upload_and_poll
* updates to async samples
* remove comment as we are using poll methods
* more fixes
* Use List instead of Sequence and fixed samples (#37898)
* fixed warning for aio and get call function tools for stream within t… (#37913)
* fixed warning for aio and get call function tools for stream within the SDK
* Fixed another warning
* Fixed return type
* Evaluation API Review Feedback (#37915)
* Evaluation API Review Feedback
* add sample for evaluation schedule
---------
Co-authored-by: Sai Kothinti <saikothinti@microsoft.com>
* Fixed poll while loop and sample (#37922)
* Jhakulin/azure ai client agents updates2 (#37937)
* minor fixes and updates
* update
* update
* create_vector_store_file_batch_and_poll + samples and update aio using the latest code from syn (#37952)
* create_vector_store_file_batch_and_poll + samples and update aio using the latest code from sync
* Resolved comments
* Clean up on the Optional (#37971)
* Switch from .endpoints to .connection (re-emit from latest TypeSpec)
* updates and minor fixes (#37984)
* Clean up (#37985)
* Rename connection string environment variables
Rename AI_CLIENT_CONNECTION_STRING to PROJECT_CONNECTION_STRING, to match how it's shown in Azure AI Studio.
Also rename AZURE_AI_CLIENT_AGENTS_CONNECTION_STRING (used in Agents tests) to PROJECT_CONNECTION_STRING_AGENTS_TESTS
* updated codeinterpreter attachment sample (#37990)
* Filter parameters, returned by SSE event (#37991)
* nirovins/filter_parameters
* Fixes
* Move utility functions out of class
* Evaluation data set id change revert (#38007)
* Updates to .connection operators (#38024)
- Re-emit from latest TypeSpec
- Updates to .connections operators to support the new preview version of REST API
- Add "id" on ConnectionProperties
* Users/singankit/ai project utils (#38033)
* Sample upload file method on AI Client
* Updating sample evaluation
* Improving Evaluation user experience
* Adding pip install link for evaluation
* Update sample_evaluations.py
* Reverting connection changes
* Update _patch.py
* First emit of azure-ai-project (_patch.py, samples and test not yet copied)
* Copy _patch.py, samples, test, update names
* Fix wrong import
* M hietala/agent tools updates (#38031)
* no need for function name separately in function definitions
* more changes related to not needing to specify function names separately
* adding capability to remove vector stores from file search tool
* adding remove file to code interpreter
* review comment related change
---------
Co-authored-by: Marko Hietala <markhiet@microsoft.com>
* add changes to latest typespec and ai-project changes (#38051)
* Move agents test to agents folder (#38055)
* Move agents test to agents folder
* Copy changes to project
* Prep for connection tests. (#38060)
* Create azure-ai-projects folder
* Add Inference tests (#38069)
* re-doing the agent toolu updates after package changes (#38078)
Co-authored-by: Marko Hietala <markhiet@microsoft.com>
* Add .diagnostics.enable() method to fetch App Insights connection strings (#38109)
* Users/singankit/evaluation async samples (#38104)
* Adding async sample for evaluation
* Update sample_evaluations_async.py
* Fix sphinx docstring and cSpell (#38125)
* Fix sphinx
* Fix spelling
* Fix
* Fix II
* Generate new code
* Fix test
* Fix
* Disable evaluation schedule and remove sampling rate (#38161)
* sdk changes for online eval disable schedule and remove sampling rate
* change sample
* Add tests for Connection operations (#38156)
* update to get file content (#38165)
* Jhakulin/parse messages (#38167)
* parse messages
* update
* add test file
* update
* use stream for get file content
* add get_file_content_stream
* update
* fix merge error
* review feedback
* more review comments
* more review comments
* Update README.md (#38173)
* update to get file content (#38192)
* Enable tracing (#38185)
* added validation to tools and tools_resources, added update agent (#38195)
* Rename `.diagnostics` to `.telemetry` (#38209)
* Jhakulin/agents get file content async (#38201)
* update to get file content
* agents async updates
* update
* fix await/pylance issue
* fix pylance error on AsyncToolSet
* return empty set if no tool resources for code interpreter
* added async toolset sample
* Fixed test (#38222)
* Support the new unified models deployed to the AIServices connection
* Fixed validation to put AyncFunctionTool into ToolSet and FunctionTool into AsyncToolSet. Introduce BaseToolset. (#38244)
* adding ai agents instrumentor (#38200)
* adding ai agents instrumentor
* streaming tracing sample and fixes to streaming tracing
* moved tracing to agents tracing folder
* adding support for tracing list_messages and async tracing sample
* removing some commented out code
* adding separate console and azure monitor tracing samples
* removing unused code
* removing more unused code
* renamed tracing to telemetry. removed throwing from instrument and uninstrument
* Update sdk/ai/azure-ai-projects/azure/ai/projects/telemetry/agents/_utils.py
Co-authored-by: Liudmila Molkova <limolkova@microsoft.com>
* Update sdk/ai/azure-ai-projects/azure/ai/projects/telemetry/agents/_ai_agents_instrumentor.py
Co-authored-by: Liudmila Molkova <limolkova@microsoft.com>
* Update sdk/ai/azure-ai-projects/azure/ai/projects/telemetry/agents/_ai_agents_instrumentor.py
Co-authored-by: Liudmila Molkova <limolkova@microsoft.com>
* fix for streaming with eventhandler and toolset
* adding previous fix for async version also
* adding missing function
---------
Co-authored-by: Marko Hietala <markhiet@microsoft.com>
Co-authored-by: Liudmila Molkova <limolkova@microsoft.com>
* Jhakulin/user functions updates (#38248)
* update to get file content
* agents async updates
* update
* fix await/pylance issue
* fix pylance error on AsyncToolSet
* return empty set if no tool resources for code interpreter
* added async toolset sample
* updates to user functions
* update
* fix the array type for parameters
* Connections & Inference test updates to support new unified models
* Improve tracing samples and add docs on telemetry.enable (#38257)
* Improve tracig samples and add docs on telemetry.enable
* fix typo
* Jhakulin/user functions optional (#38261)
* update to get file content
* agents async updates
* update
* fix await/pylance issue
* fix pylance error on AsyncToolSet
* return empty set if no tool resources for code interpreter
* added async toolset sample
* updates to user functions
* update
* fix the array type for parameters
* support for Optional
* update
* one more example with list of dicts
* adding content recording control to agents instrumentor api (#38268)
Co-authored-by: Marko Hietala <markhiet@microsoft.com>
* Howie/save file test (#38306)
* Added test for save_file
* resolved comment
* Mypy fixes for instrumentor (#38302)
* Fixes
* Regenerate code
* Merge again
* Linter fixes
* Regenerate the code
* Fix mypy
* More mypy fixes
* Fix
* Fix handling of SAS token and fix mypy (#38312)
* Fix handling of SAS token and fix mypy
* typo
* Fix also pylance issue
* Glharper/ai client new tools (#38202)
* [AI Client] Add new search tools
* add hooks to update connection lists for bing search
* use updated typespec
* use ConnectionType and ConnectionsOperations
* endpointOperations -> ConnectionsOperations
* add AzureAISearch tool def
* adapt project with new tool changes
* Add new tool defs for bing and azure ai search to projects.models
* use service schema for BingGroundingTool
* add samples for new tools
* add sharepoint tool
* update prompt to use bing browsing
* add sharepoint_grounding discriminator
* code gen from merged typespec
* update to latest typespec
* review feedback
* add preview header to bing grounding and azure AI serach samples
* Use 'Union' insted of '|' to make it run on Pyton 3.8 and up (#38344)
* Fix mypy issues and re generate the code. (#38340)
* Add more mypy fixes
* Fix imports
* Fix
* Add safe conversion method
* Fix mypy
* Merge to main
* Fix mypy
* Fix unit test
* Sphinx fix
* Fix cspell
* Use DefaultAzureCredential from azure.identity.aio for async samples (#38362)
* Rename AAD to ENTRA_ID (#38363)
* added agents example section in readme and added tags to samples (#38351)
* added agents example section in readme and added tags to samples
* Resolved comments
* Added more detail to readme
* Fix mypy and do not scan examples (#38375)
* Changes in a package
* Fix
* Fix mypy
* fix mypy
* Fix
* Connections APIs throw instead of returning None if a connection name/type is not found (#38383)
* Make some APIs throw instead of returning None. Re-try to get connection string
* Update README.md. Work in progress
* Fixes
* Add code interpreter file attachment sample and link to readme. Add alot more detail to readme also. (#38384)
* Fix broken async .telemetry.enable() method, when you don't pass in "destination" (#38399)
* Fix README. Fix Optional[str]
* Fix async telemetry
* Sample for thread with resource and document it in readme (#38393)
* Users/singankit/evaluation test (#38402)
* Test file
* Adding fixture
* Adding evaluation e2e test
* Update azure_ai_projects_tests.env
* Update how the inference URL is constructed (#38409)
* First
* Revert back this change
* Fix some cspell errors, and link errors (#38420)
* cspell lint --config .vscode/cspell.json --no-summary "sdk/ai/azure-ai-projects/**/*"
* Resolved comments from Krista (#38405)
* Address comments
* Run black
* Add support of Azure asset IDs for the vector store. (#38066)
* Generate code
* Generate new code
* Expose file API and add the async test.
* Remove unnecessary change
* Add async samples
* Do not create ML Client in tests
* Better file retrieval
* Regenerate code
* Re generate the code
* Re generate code
* Fix samle name and make iterface more clear
* Add samples for attachment enterprise search
* Add unit tests for message attachment
* Finzlize sync tests.
* Add unit tests and use constants as a URI types
* Regenerate code
* Fix
* Generate new code
* Draft commit
* Add another unit test
* Fix and add test data
* Regenerate files
* Add more unit tests
* Re generate code
* Regenerate files
* Rename classes and regenerate the code
* Fixes
* Fix parameter name
* Fix file names
* Script to check sample names inside file comments.
* Fix PR comment by Krista on async credential in ConnectionProperties class (#38424)
* Black
* User/singankit/evaluation spellcheck (#38427)
* Spell check errors evaluation
* Removing test environment from evaluation schedule test
* Fix doc string errors
* Fix typing in samples (#38354)
* Fix samples
* Re enable type checks and remove extra code
* Remove non needed code
* Remove non needed changes
* Remove non needed changes
* Fixes
* Return non optional client
* Remove extra line
* Fix file names
* Fixes
* Fix async file
* Fix mypy issues and suppress the ones related to typespec
* Fix cSpell
* Re generate code and fix unit tests
* Delete sdk/ai/azure-ai-projects/samples/config.json
* Fix typo
* Fix unit test
* Run "black" (again)
* Fix a few pyright errors, and address a comment in the Python PR (#38435)
* Address some of Krista's PR comments (#38444)
* sdk changes for name change for provisioning state and isEnabled online evaluation (#38298)
* sdk changes for name change for provisioning state and isEnabled online evaluation
* add correct tsp-location.yaml
* azure-ai-projects: configure otel logging in telemetry.enable() helper (#38434)
* Configure otel events in projects.telemetry.enable()
* [ai] fix(pylint): Resolve `W1309`, `W0107`, `W0707`, `R1705`, `W0401`, `R1714`, `W0611`, `W0404`, `W0201`, `W1201` and `W1203` (#38447)
* style: Run isort
* fix(pylint): Resolve W1309(f-string-without-interpolation)
From sdk/ai/azure-ai-projects, run:
ruff check --select=F541 azure --fix
* fix(pylint): Resolve W0107(unnecessary-pass)
From sdk/ai/azure-ai-projects, run
ruff check --select=PIE790 --fix azure
* fix(pylint): Resolve W0707(raise-missing-from)
* fix(pylint): Resolve R1705(no-else-return)
From azure/ai/azure-ai-projects, run
ruff check --select=RET505 --fix azure
* fix(pylint): Resolve W0401(wildcard-import)
* fix(pylint): Resolve R1714(consider-using-in)
* fix(pylint): Resolve W0611(unused-import)
* fix(pylint): Resolve W0404(reimported)
* fix(pylint): Resolve W0201(attribute-defined-outside-init)
* fix(pylint): Resolve W1201(logging-not-lazy) and W1203(logging-fstring-interpolation)
* Add readme for get_message, save_file, get_file_content, tracing, Bing grounding, enterprise (#38426)
* git Add readme for get_message, save_file, get_file_content, tracing
* Readme for enterprise search, code interpreter, attachment
* More documentation for tools
* Fixed create_thread documentation
* resolved comments
* Resolved comments
* Resolved comments from Krista
* More fixes to mypy (#38446)
* Fix unit tests and some type issues.
* Fixes
* Fix mypy
* agents telemetry pyright fixes (#38457)
Co-authored-by: Marko Hietala <markhiet@microsoft.com>
* Fix pyright for agents. (#38464)
* Fix as if python version is 3.8
* Fixed
* Add custom config
* Move line, ignoring import
* fix pip install instruction (#38480)
* [ai] fix(pylint): Resolve `C0301`, `C4739`, `C4740`, `C4741`, `C4742`, `C4743` for `azure-ai-projects` (#38459)
* fix(pylint): Resolve C0301(line-too-long)
* fix(pylint): Resolve C4743(docstring-should-be-keyword)
* fix(pylint): Resolve C4739(docstring-missing-param)
* fix(pylint): Resolve C4740(docstring-missing-type)
* fix(pylint): Resolve C4741(docstring-missing-return)
* fix(pylint): Resolve C4742(docstring-missing-rtype)
* style: Run black
* Enable test recordings for tests\inference, tests\telemetry and tests\connection (#38475)
* Update cspell.json
* Removed enterprise and add installation for tracing (#38483)
* Removed enterprise and add installation for tracing
* clean up
* resolved comments
* Fix mypy errors (#38487)
* Run black --config ../../../eng/black-pyproject.toml
* agents tracing pylint fixes (#38496)
Co-authored-by: Marko Hietala <markhiet@microsoft.com>
* Remove premature adding of asserts.json file
* [ai] fix(pylint): Resolving remaining pylint errors (#38500)
* fix(pylint): Resolve R1705(no-else-return)
* fix(pylint): Resolve C0207(use-maxsplit-arg)
* fix(pylint): Resolve C0412(ungrouped-imports)
* fix(pylint): Resolve W1401(anomalous-backslash-in-string)
* fix(pylint): Resolve W0105(pointless-string-statement)
* fix(pylint): Ignore W0221(arguments-differ)
pylint has an open issue tracking false positives for
arguments-differ with overloads in subclasses
* fix(pylint): Ignore W0718(broad-exception-caught)
* fix(pylint): Ignore E0401(import-error) and E0611(no-name-in-module)
* fix(pylint): Ignore C4748(client-accepts-api-verison-keyword)
Currently, neither client classes supports overriding the api version
* fix(pylint): Ignore W0212(protected-access)
* fix(pylint): resolve W0707(raise-missing-from)
* fix(pylint): Ignore E1102(not-callable)
pylint appears to fail to correctly infer that
settings.tracing_implementation actually is callable
* fix(pylint): Ignore W0231(super-init-not-called)
* fix(pylint): Ignore W0613(unused-argument)
* fix(pylint): Ignore W0236(invalid-overriden-method)
* fix(pylint): Ignore R0914(too-many-locals)
* fix(pylint): Ignore R0902(too-many-instance-attributes)
* fix(pylint): Ignore R0915(too-many-statements)
* fix(pylint): Ignore R0911(too-many-return-statements)
* Disable all agents tests. Push recordings for all other tests to the assents folder (#38481)
* run "black --config ../../../eng/black-pyproject.toml ."
* Fix pylint and new mypy issues (#38503)
* Ignore most of pylint errors
* Add generated paths to pylint.rc
* Use backslash for consistency in pylintrc files
---------
Co-authored-by: Darren Cohen <39422044+dargilco@users.noreply.github.com>
* Remove async from .telemetry.enable (#38514)
* Fix pylint
* Rename `with_credentials` to `include_credentials` per apiview.dev review comment (#38517)
* Try to fix test issues on Python 3.13. Run black
* Try again to fix Python 3.13 test failure
* Attempt to fix azure-ai-projects not showing up in the release pipeline
* Fix release date in CHANGELOG.md
* Fix CHANGELOG.md
---------
Co-authored-by: Ankit Singhal <30610298+singankit@users.noreply.github.com>
Co-authored-by: Jarno Hakulinen <jhakulin@microsoft.com>
Co-authored-by: howieleung <howieleung@microsoft.com>
Co-authored-by: Glenn Harper <64209257+glharper@users.noreply.github.com>
Co-authored-by: Sai Kothinti <saikothinti@microsoft.com>
Co-authored-by: nick863 <30440255+nick863@users.noreply.github.com>
Co-authored-by: M-Hietala <78813398+M-Hietala@users.noreply.github.com>
Co-authored-by: Marko Hietala <markhiet@microsoft.com>
Co-authored-by: Liudmila Molkova <limolkova@microsoft.com>
Co-authored-by: kdestin <101366538+kdestin@users.noreply.github.com>
* ensure that mac runs on py311+ by shifting it to a deliberate include for py311
* rework to instead use MatrixFilters in ml/ci.yml and storage/ci.yml instead of fully new matrices
* enable bandit
* update
* test
* trigger tests
* update
* update
* update
* these integration tests need to be updated not to need these fixes, but I'll do that in a separate PR
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* ensure that ChangedServices value within the generated pr diff is always properly an array of strings
* upload the changed services as an attachment named AdditionalTags in save-package-properties.yml, to allow PipelineWitness to tag public builds during log processing
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
Co-authored-by: Patrick Hallisey <hallipr@gmail.com>
* Add step for emitting rate limits for a token
* add step to eng/common sync template
* Fix resource label output
* Use propery name instead of nested 'resource' property
* Add percent metric too
* Add divide by zero safety
---------
Co-authored-by: Patrick Hallisey <pahallis@microsoft.com>
* Common cleanup with the docs docker images no longer being used
* Remove docker-pull-image.yml
---------
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
* Retry container deletion
* Do not try to purge keyvaults with purge protection
* Delete all blobs when container has immutability
* Skip missing blob container properties
* Fix null container
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Enable py2docfx docs gen tool, remove the dockerimage docs validation
* Remove the --no-venv-required flag from the command line which was a leftover copy/paste from the original PR
* Change the version to a pre-release for testing as the released version doesn't appear to have these changes
* Update py2docfx version since previous version had problems on Linux
* Change the python command to add -u so the sdtout and stderr streams are unbuffered and we see the full output
* Updates for feedback and to see what's going on with the output
* Remove the Get-Content line which should have been removed in the last commit
* Print the command output on actual different lines
* Finish the sentence in the comment
* identify and resolve missing function GenerateMatrixForConfig
* identify and resolve miss-used null-coalesce operator
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* package-properties are now populated with matrix configurations from their ci.yml if present
* create new code path for generate-job-matrix.yml which combines Create-JobMatrix and the "distribute-packages-to-matrix" action to generate dynamic matrices for PRs
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* Run perf tests under federated auth
Run the tests under the same federated auth used to deploy the tests and setup the variables needed to configure AzurePipelineCredential.
* Fix indention for working directory
---------
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Refactor PackageProps to make the Yaml loading and processing methods common
* Updates for feedback
---------
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
* Use federated auth and TME for smoke testsing
- Move to using federated auth
- Move to using TME sub
- Remove other clouds as the public should be enough for smoke testing
* Set default cloud
* Remove testApplicationSecret and run tests under AzPS
* Remove unsupported python 2.7
* add support for parsing and building pure pyproject.toml packages within azure-sdk-tools
* various typing updates across azure-sdk-tools, cleaning up what code paths I'm touching with the new build and parse support
* add tests for various pyproject.toml parse scenarios
* bump setuptools to 74.1.3 to support pure-pyproject.toml extension packages (eg without a setup.py)
---------
Co-authored-by: McCoy Patiño <39780829+mccoyp@users.noreply.github.com>
Azure SDK Bot
Scott Beddall
Krista Pratico
iscai-msft
Ben Broderick Phillips
Yuchao Yan
Darren Cohen
Xiang Yan
James Suplizio
Paul Van Eck
Wes Haggard