1e212cda6a
Revert "Add readme validation step" ( #2951 )
2024-01-12 15:13:46 -08:00
a8431e1b9c
Add readme validation step ( #2748 )
...
* Add script to check README.md and README template
* Refactor check-readme.sh and add messages
* Add step to cli-assets-component-pipeline workflow to test
* Update workflow
* Update workflow
* Fix syntax error
* Update action
* Fix typo
* Add exclusion logic
* Try adding readme validation to readme.py
* Update workflows with readme.py
* Update workflows with readme.py
* Update workflows with readme.py
* Update workflows with readme.py
* Fix github workspace variable
* Fix github workspace variables for exclusion check
* Fix exclusion logic
* Update readme exclusions
* Revert accidental change
* Update readme_exclusions.txt
* Revert accidental change to working directory
* Update check-readme.sh
* Update exclusion logic and file name
* Add debug message
* Update exclusion logic and file name
* FIx working directory file names
* Update exclusion logic and file name
* Add debugging statements
* Remove debugging statements
* Update readme.py and regenerate cli workflows
* Update working-directory
* Update readme to add validate readme to sdk workflows
* Move templates folder inside infra
* Add validate readme check to tutorials and sdk/python files
* Fix readme.py and revert unintended changes
* Revert unnecessary changes and update readme_validation_exclusions.txt
* Update exclusions list
* Update template
* Update readme_validation_exclusions.txt
* Update exclusions
* Replace check-readme.sh with check-readme.py
* Update readme template
* Update check-readme.py with Kevin's suggestions
* Remove 2nd CLI argument
* Add debugging messages
* Add debugging messages
* Add debugging messages
* Update debugging messages
* Update debugging messages
* Strip whitespace
* Convert sample path to string
* Fix exclusions
* Remove debugging messages
* re-format with black
* Manually update one sample with logging logic to test
* Regenerate resolved notebooks
* Remove appinsights logging step
* Exclude /home/runner/work/azureml-examples/azureml-examples/cli/jobs/automl-standalone-jobs/cli-automl-forecasting-task-github-dau
2024-01-12 15:36:15 -05:00
276eb40f28
Add random numbers at the end of endpoint name in workflows ( #2558 )
...
* Add random numbers at the end of endpoint name
* Fix bootstrapping directory
2023-08-15 14:24:58 -07:00
2c8e615d87
Update resources name ( #2521 )
...
* Update keyvault name
* Update attached compute name
* Fix if condition
* Update compute name
2023-08-09 11:14:35 -07:00
0b829b9277
Automation test for spark CLI samples ( #2377 )
...
* Enable test for submit_spark_standalone_jobs
* Generate workflow yaml
* update spark job files for automation test
* Add workflow for serverless spark with user identity job
* Add scripts to upload input data
* Update workflow to refer the script
* Update source file path
* Update workflow with correct file path
* Update working directory
* Update workflow
* Update the path
* Update the script to upload data
* Update the overwrite mode
* Update destination blob name
* Use blob upload batch
* Add spark pipeline tests
* Update spark component extension
* Add script to attache uai
* Update property name in workflow
* Update script parameters
* Update assign uai script
* Format the script
* Update setup identities script
* Update path to infra bootstraping
* Enable automation test for attached spark job
* Update resource path
* Update setup attached resource script
* Update script of setup resources
* Update setup attached resource script2
* Add logic to assign identity role
* Format the empty check
* Check if identity is empty
* Update to get compute properties
* update readme
* Reformat the script
* Update schema location and revert sdk notebook changes
* Attach pool first
* Rename resources and merge main
* Update format in yml
* Add role assigment to uid
2023-07-06 13:36:07 -07:00
f87d6591d5
refactor: Reorganize infra folder ( #2328 )
...
* refactor: Move infra scripts to subdir
* refactor: Rewrite paths in sdk_helpers.sh
* refactor: Update ROOT_DIR
* refactor: Update paths in workflow generators
* refactor: Update path in doc comment
* refactor: Update workflows
* fix: Fix incorrect script path
2023-05-24 15:56:58 -04:00
044805f4b2
updating python version ( #2196 ) ( #2197 )
...
* updating python version (#2196 )
Co-authored-by: Raghu Ramaswamy <raghura@microsoft.com>
* Added endpoint paths to trigger create deployment workflows to test Python ver change
* Removed unrelated workflows that are failing
* Removed unrelated workflows
* Ignore kubernetes green deployment
* Fixed typo in pip and joblib versions
---------
Co-authored-by: Raghu Ramaswamy <13340619+raghutillu@users.noreply.github.com>
Co-authored-by: Raghu Ramaswamy <raghura@microsoft.com>
2023-04-12 23:20:01 -07:00
d826128126
Add workflows for all endpoints ( #2203 )
...
* Add workflows for all endpoints
* Fixed online vs batch endpoint workflow command bug
* Testing workflows catting endpoint .yml files
* Removed all newly generated non-endpoint related workflows that were under endpoint folder
* Testing workflows deleting endpoints before creation if exists
* Add continue on failure for delete endpoint
* Testing replacing endpoint names during action and deleting
* Testing extra delete section
* Testing -n over -f
* Testing new endpoint names
* Reduced endpoint names to 32 chars max
* Replaced lowercase endpoint_name with uppercase, added quotes
* Removed uai create endpoint workflow
* Added create deployment sections to workflows if exist
* Standardized all usages of conda*.yml to use .yaml ending in cli/endpoints
* Standardized all usages of conda*.yml to use .yaml ending in cli/endpoints
* Used same modified endpoint name for deployments in workflow
* Temporarily removed deployments and endpoints which were failing due to malformed .yml and other transient reasons
* Removed extra print statement and comment
* Added missing name for deployment
* Added 1-sai-create-endpoint to ignore list
2023-04-12 18:54:26 -07:00
b4ace52a7e
Update DeepSpeed Training example launcher ( #2103 )
...
* change training example
* fix workflow
* fix syntax
* update readmes
* edit readme
* remove env variables
2023-03-09 11:40:32 -08:00
94a0f1f485
Prs examples ( #2064 )
...
* Update PRS examples
* Update PRS examples
* Update prs examples
* Update PRS examples
* Update format
* Update by comments
* fix sdk default datastore issue.
* Update wording
2023-02-08 09:59:59 +08:00
52a50eb93a
Changed schedule for cli samples to run every 12 hours at random times ( #2054 )
2023-01-31 17:01:04 -08:00
a066a80455
resolve log_param conflict issue ( #2045 )
...
* resolve log_param conflict issue
* reslove black format
---------
Co-authored-by: Clement Wang <clwan@microsoft.com>
2023-01-30 16:10:02 +08:00
53ca2137a1
Add deepspeed autotuning and training examples ( #2028 )
...
* add deepspeed example
* fix mlflow logging in train.py
* move examples to cli folder
* move to jobs, add job.yaml files
* generate workflow file
* add comment in train.py
* add latest tag to env
* try different way of starting run
* set up job as pipeline
* move to pipelines folder
* add temp yaml file to pass validation
* modify readme.py to support deepspeed
* move generate-yml script
* recreate workflow files.
* move generated key location
* change environment
* change env for both examples
* fix env
* try v100 computes
* create nd40 cluster
* add compute create line
* change max number of nodes
* change data directory
* change command to block
* move data files into src
* remove unused data thing
* change data file type
* fix mlflow get run fail
* change command style in job
* change location of job
* change generate key path
* change code: path
* rename results folder
* change back key path
* change max number of nd40 nodes
* try moving autotuning folder into src
* change output file name
* try overwriting output
* add dockerfile for custom env
* change env in yaml
* update README and move autotune example
* move dockerfile to parent dir
2023-01-26 09:10:33 -08:00
b62735fea9
Update the readme.py script in cli folder to work on Windows as well as Linux ( #2035 )
...
* Updated the cli readme.py to support Windows
2023-01-23 22:04:32 -08:00
c7adadc045
delay registry component workflows by 59 min ( #2033 )
...
* delay registry component workflows by 59 min
* fix cleanup.sh
Co-authored-by: Ubuntu <fvm@azureml.hdto2glhnacenmzoxwwt5zmpza.phxx.internal.cloudapp.net>
Co-authored-by: Jun Qi <junqi@microsoft.com>
2023-01-23 10:47:08 -08:00
1d5025990e
Format with black ( #2021 )
2023-01-11 13:39:41 -08:00
967d7b799c
Create workflows to run pipeline job using registry components ( #1900 )
...
Co-authored-by: Ubuntu <fvm@azureml.hdto2glhnacenmzoxwwt5zmpza.phxx.internal.cloudapp.net>
2023-01-11 12:57:50 -08:00
27c3a943a2
SDK and CLI code samples for AML Spark Public Preview ( #1814 )
...
* SDK and CLI code samples for AML Spark Public Preview -- first pass.
* Fixing build errors.
* Fixed code format errors.
* Adding missing cli/jobs/pipelines-with-components/pipeline_with_hyperparameter_sweep/.
* Converted samples to notebooks and updated sdk/readme.py and cli/readme.py to add automation exclusions.
* Modified notebook sample and removed GitHub workflow for pipeline with Spark nodes.
* Deleted GitHub workflow file.
* Fixing build errors.
2022-11-03 20:31:59 -07:00
78cc5e7e01
Add yaml examples for RAI ( #1834 )
...
* [WIP] Add yaml examples for RAI notebooks
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
* Fix nb
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
* Fix ml registry
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
* Default to azureml registry
Signed-off-by: Gaurav Gupta <gaugup@microsoft.com>
* Revert "Default to azureml registry"
This reverts commit 2f8710514eabcb33fe419e8179d780
2022-11-03 11:41:12 -07:00
65cf226959
Users/harnvir/tempresources ( #1841 )
...
* Adding checks for k8 extension and check for attaching the compute to workspace
* Adding a note on the top of all the workflows about regenerating using python script
* Update header in the workflows
2022-11-02 16:26:05 -07:00
2531497d78
Adding reusable bootstrapping script to configure infra resources ( #1602 )
...
* Adding reusable bootstrapping script to configure infra resources
* Fixing tput error
* Disable color coding for the output
* Removing the tput call
* Adjusting the directory structure of infra scripts
* Adjusting the path of the bootstrapping script
* Rerunning 2 workflows only
* Addressing the indendation issue
* Fixing the path of the init script
* loggign the path
* Adding workspace path
* adding helper script call
* Removing typo from the file name
* Adding source call
* Refactoring the Compute creation script
* Updating single-step workflows
* Updating CLI script to use bootstrapping script
* Updating all of the cli workflows
* Updating all of the CLI and SDK workflows
* Allowing endpoint workflows to continue ot use old subscription
* Updating all of the CLI and SDK workflows for cli configuration
* Refactoring logging
* Refactoring some of the existing scipts
* Updating the dataset call
* Updating pipeline workflows for extracting the correct config
* Updating the azure config
* Adding helper scripts for resource provisioning
* Refactoring samples to use the new subscription
* fix permissions for running sh files
* fix permissions for running sh files
* Ensuring the tool are installed
* Add copy data script
* Updating the config generation in SDK workflows
* Updating the config generation directory
* Updating the config generation directory one subdirectory level
* Extracting the folder name from the path for creating config
* Adding logging
* Adding logging for pwd
* Adding config.json
* Adding generic scripts for granting permissions on rg
* Updating the workflows by adding the cluster info
* Adding the new dataset
* Adding concurrency and updating the schedules of the jobs
* Updating the workflows to use concurrency
* cancel a currently running workflow from the same PR, branch or tag
* Updating workflows for concurrency
* Logging the information about workflow names
* Changign the group name to variable
* Printing the notebooks content
* Print the content of the notebooks
* Merging changes from main
* Adding pre-commit-hook
* Updating the task names
* Reverting the set-up repo changes
* Addressing the formatting of the files
* Addressing the formatting of the files
* Updating the moe scripts to fetch the registry name
* Updating the new workflows
* Adding new workflows and fixing the formatting of the files
* Updating the replacements
* Updating the ACR details
* Updating bootstrapping script
* Updating the ACR
* Fixing the typo in notebook
* Adding the script to create vnet/subnet
* Updating the workflows
* Updating the identity creation with appropriate roles
* Archieve the model while running endpoint workflows and create vnets
* Adding bootstrapping workflow
* Removing managed online endpoints VS workflow
* Continue on error for bootstrap workflow
* Adding the path in the workflow
* Debug the bootstrapping workflow
* Adding call to bootstrap mltable
* Adding call az ml commands
* Adding call az ml commands by removing data calls
* Adding the replacements for InteractiveBrowserCredential
* Reformatting the notebook
* Updating the bootstrap workflow
* Updating the bootstrap workflow with less logging
* Updating the bootstrap workflow with less logging
* Updating the bootstrap workflows
* Adding more bootstrapping steps
* Removing newly added workflows
* Install tools as part of bootstrap step
* Updating the script to grant permissions on identity
* Updating the ACR Details
* Importing data for automl jobs
* Reformatting the files and regenrating all files
* Updating the SDK package version
* Updating bootstrapping script name
* fix pipeline sample 4b
* fix pipeline sample 4b
* Reformating and updating the folder path in bootstrapping workflow
* Update sdk path in bootstrapping workflow
* Regenerate the workflows
* Merging changes from main and regenerating the workflows
* Adding owner for the tags
* update image cli to run prepare_data.py first (#1699 )
Co-authored-by: Riti Sharma <risha@microsoft.com>
* Regenerating the workflows after adding data prep step
* add automl in the check for prepare_data
* update ymls
* Updating the script to reinstall ML extension
* Regenerating the workflows for automl to address identity installation issue
* Regenerating the workflows for automl to address identity installation issue and reformatting
* Installing SDK required for running prepare dataset
* Updating the SDK call for automl workflows
* Fix failed CLI pipeline(add restriction on azureml-mlflow) (#1703 )
* add restriction
* fix
* update
* directly use expected version
* update
* update
* remove deleted wf
* Updating ARC script
* Updating the readme file
* Regenerate the workflows
* Updating arc scripts
* Adding macro for account name and container
* Calling update dataset script
* Regenerating workflows and addressing formatting
* Merging changes from main after sdk release
* Address the vulnerabilities in the VMSS agents pool
* Merging changes from main for new workflows
* Update SDK v2 in setup file
* update cli cifar distributed
* init (#1786 )
* Updating registry env variable
* Automated resource provisioning - registries (#1812 )
* init
* enable registry check
* init REGISTRY_NAME
* changed env var names and import
* try to fix checks
* ran readme.py; changed registry yml location; modified helper
* fixing syntax
* [mldesigner] Modify sdk/1b to adopt newest mldesigner 0.1.0b7 (#1794 )
* update
* update
* update
* update
* update
* update
* change registry name for testing
* fixing syntax
* fixing syntax
* change registry name for testing
* change registry name for testing
* Dont run samples for any checkin (#1799 )
* Dont run samples for any checkin
* Reformat with black
* change registry name for testing
* change registry name back
* replace version
* Batch Endpoints scenarios (#1777 )
* batch scenarios
remove model
* black
* comments
* typo
* typos
* formatting
* build
* batch.sh fix
* modified model deployment
* fixing workflow
* Fix model path in Local Debug in VSCode (#1679 )
* Resolve model path issue and add to readme ignore
* Update readme
* Revert readme
* Update readme
* Fix environment
* Remove stray line
* Change to default azure credential
* remove workflow
* [mldesigner] Fix mldesigner version in sample (#1797 )
* update
* update
* update
* update
* update
* update
* ran readme.py, edited readme.py, edited workflow
* added log for debugging
* fixing env var
* added enviroment to deploy.yml
* revert deploy.yml
* resolved conflicts
* revert changes in v1
Co-authored-by: Korin <0mza987@gmail.com>
Co-authored-by: Bala P V <33712765+balapv@users.noreply.github.com>
Co-authored-by: Facundo Santiago <fasantia@microsoft.com>
Co-authored-by: Alex Wallace <80542152+xanwal@users.noreply.github.com>
* Updating the bootstrapping script
* Update sdk_helpers.sh
fixing the registry creating bug
* modified ensure_registry
* Updating the resource names to use November month resources
Co-authored-by: lochen <cloga0216@gmail.com>
Co-authored-by: sharma-riti <52715641+sharma-riti@users.noreply.github.com>
Co-authored-by: Riti Sharma <risha@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: Ayush Mishra <61145377+novaturient95@users.noreply.github.com>
Co-authored-by: Douglas Xiao <xiake@microsoft.com>
Co-authored-by: Komnus丶Q <40655746+quchuyuan@users.noreply.github.com>
Co-authored-by: Korin <0mza987@gmail.com>
Co-authored-by: Bala P V <33712765+balapv@users.noreply.github.com>
Co-authored-by: Facundo Santiago <fasantia@microsoft.com>
Co-authored-by: Alex Wallace <80542152+xanwal@users.noreply.github.com>
Co-authored-by: Chuyuan Qu <chuyuanqu@microsoft.com>
2022-10-28 19:23:32 -07:00
5dc65ced96
add schedule sample of CLI/SDK ( #1524 )
...
* add schedule sample of CLI/SDK
* add ci workflows
* temp update setup.sh to a tested wheel to test the workflow, need to remove this before merge back to main
* update CLI setup.sh
* update to a less frequent schedule
* update samples to support doc
* update sdk example for doc
* update schedule sample
* refine code to set import at begining and output list/get result
* remove file prefix, add no-wait for async
* clean up outputs
* remove timezone in start time
* update sdk version
* update meta for doc
* update cron expression
* update yaml example
* remove file prefix
* update coments on start_time
* remove file prefix
* update for doc
* update readme.py to auto generate schedule workflow and readme
* add auto generated workflow and README for schedule
* update script and workflow
* minor readme table file name update
* update readme file
* remove the 2 workflows written manually
* Fix format
Signed-off-by: Brynn Yin <biyi@microsoft.com>
* Add update part
Signed-off-by: Brynn Yin <biyi@microsoft.com>
* Remove unused import
Signed-off-by: Brynn Yin <biyi@microsoft.com>
* Refine enable/disable
Signed-off-by: Brynn Yin <biyi@microsoft.com>
* update schedule file names
* update readme and workflow for cli
* split runsettings
Signed-off-by: Brynn Yin <biyi@microsoft.com>
Co-authored-by: lochen <cloga0216@gmail.com>
Co-authored-by: Brynn Yin <biyi@microsoft.com>
2022-08-12 15:20:51 +08:00
32ea7aab53
Remove CLI sample compute-cluster-user-identity from automation ( #1560 )
...
* Remove CLI sample compute-cluster-user-identity from automation
* lint
Co-authored-by: Neehar Duvvuri <neduvvur@microsoft.com>
2022-08-10 17:37:58 -04:00
d75447d655
deploy-arm-templates api version update ( #1458 )
...
* updating api versions
* addressing comments
* black formatting
Co-authored-by: Raghu Ramaswamy <raghura@microsoft.com>
2022-07-05 15:36:27 -07:00
8d1290bd5f
run formatting, fix smoke test ( #1352 )
2022-05-24 08:04:55 -07:00
5285f9c7e8
Merge Build 2022 changes to main ( #1182 )
...
* Standalone Jobs and Pipelines with Core SDK
* Dataset Examples
* Adding environment examples
* Examples for models
* remove unwanted files
* Create workspace.ipynb
* Create compute.ipynb
* Update readme.md
* Removing credentials from notebook
* Lochen/pipeline dsl example (#913 )
* add DSL example
* update readme
* Formatting (#914 )
* add batch sdk sample (#918 )
* Remove unwanted files from command-job-in-pipeline
* removing unwanted files from pipelines with components
* Lochen/pipeline dsl example (#920 )
* add DSL example
* update readme
* change component name
* update folder name
* remove sample add more notebook
* Adding example for online inferencing
* Add files via upload
* Add files via upload
* Create datastore.ipynb
* Update MLClient to the latest interface and making use of promoted properties (#921 )
* Updated command job examples to align with the latest release
* ignore notebook cache
* remove cache
* Online inferencing example
* Update samples
* Pin to a specific private preview SDK version (#924 )
* Pin to a specific private preview API version
* typo
* readme consistency edits (#926 )
* readme consistency edits
* rename?
* fix errant backticks (#927 )
* Adding sweep job to samples (#931 )
* Adding sweep job to samples
* Update lightgbm-iris-sweep.ipynb
* Update .gitignore
* Update README.md
* How to use datasets in a job
* How to use datastore
* Adding Documentation the PyTorch notebook.
* Create notebook-template.ipynb
* Accidents Example for R
* Update accident-prediction.ipynb
* Adding R sample
* Updates to documentation
* Updating Documentation
* Updating documentation
* Updating documentation
* Updating documentation for single step jobs
* Update model.ipynb
* Update sklearn-mnist.ipynb
* Updating documentation for endpoints
* Paulsh/sdk dashboard (#985 )
* First pass
* Add trigger
* Fix sed commands
* Fix path to workbook
* Updates
* Updates
* Updates
* Skip some workspace creation commands and use a random workspace name
* Shorten workspace name to fit ARM requirements
* Test (#988 )
* Test
* Update
* Add config file
* Comment out config file lines
* Update (#989 )
* Workflows for testing sample notebooks (#997 )
* Adding workflows for single step jobs
* python file for workflow creation
* Adding versions to environments
* Adding workflows for assets
* Add files via upload
* Update tensorflow-mnist-distributed.ipynb
* Adding workflows for assets
* Adding workflows for endpoints
* Update tensorflow-mnist-distributed.ipynb
* Updating pipeline samples
* Adding workflows for pipelines with components
* Revert "Updating pipeline samples"
This reverts commit 468438ec4f#1004 )
* added e2e data scientist sample part I
Co-authored-by: amah <ma.mahmoudzadeh@gmail.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
* Janhavi13/model nb (#1008 )
* Fix model_uri in model notebook
* Update model upload from cloud path and run
* Update model upload from cloud path and remove run
* Move Tutorials to dedicated folder (#1034 )
* moved e2e ds tutorial to a dedicated folder
* removed from old location
Co-authored-by: amah <ma.mahmoudzadeh@gmail.com>
* Update dev-requirements.txt
* Hancwang/batch fix (#1011 )
* Batch deployment fix
* typo
* few more changes to invoke and deployment init
Co-authored-by: Banibrata De <banibrata@outlook.com>
* Distributed Deep Learning image end to end tutorial (#1023 )
* write e2e object dl notebook
* implement distributed pytorch
* implement mlflow model logging
* implement profiling
* March sdk preview (#1021 )
* Address CLI breaking changes
* Update nyc_taxi_data_regression.ipynb
* Updating pipelines
* Inline documentation updates
* Incorporating Sam's PR into this branch
* Installing new build
* Adding temp workflows for automated testing of notebooks
* Update model.ipynb
* Update model.ipynb
* Reverting model to sdk-preview version
* Update setup.sh
* Update setup.sh
* Adding a new build
* Removing all workflows from this branch
* Create sdk-assets-model-model.yml
* Update dev-requirements.txt
* Removing local path, adding @latest
* Updating to latest version of SDK/CLI and adding ipython_genutils
* Adding workflows for assets
* Update dev-requirements.txt
* Updating to Build # 58490278
* Adding tests for single step jobs
* Adding tests for endpoints
* Adding test workflows for pipelines
* Adding all test workflows
* Removing dataset notebook since we now have data
* Automated README.md
* Workflows for this branch
* Updating to latest build
* Update mnist-nonmlflow.ipynb
* Updating to latest build
* Adding description metadata to notebooks
* Improving README and Workflow generation
* Lochen/sdk march (#1041 )
* innit
* init
* Lochen/sdk march (#1042 )
* innit
* init
* merge folder
* fix pipeline sample: cifar-10, tf_mnist
* update workflows for pipeline
* Lochen/sdk march (#1044 )
* innit
* init
* merge folder
* change e2e registered, basic, nyc
* basic pipeline nyc
* add instructions.
* update nyc
* fix basic pipeline
* Upgrade to latest build
* Adjusting workflows and README for new pipeline samples
* Dummy commit for testing
* Update README.md
* Adding descriptions for pipeline notebooks
* Updating to latest build
* Removing datastores from testing and compute from scheduled runs
* Update lightgbm-iris-sweep.ipynb
* Updating to latest build
* Update README.md
* Fixing source directory for data and updating CLI/SDK build
* Updating to latest build
* Update README.md
* Adding clean up for endpoints and workspaces
* Update workspace.ipynb
* Updating to new build
* Update README.md
* Fixing issues with workspace notebook
* Update setup.sh
* Update setup.sh
* Updating to latest build
* Update README.md
* Updating to latest build and testing in eastus
* Fixing workflows
* Update readme.py
* Online endpoints sdk (#1059 )
* online endpoints sdk samples
* online endpoints sdk samples
* minor update
* updating readme.md
* adding back orginal sample
* minor
* update to sample folder
* minor changes
* Adding workflows and README for the new online endpoint examples
* Updating to the RC build
* Update README.md
* Fixing issues with MIR samples
* Standardizing kernelspec for all notebooks
* Formatting Cleaning python files with black
* Fixing readme.py
* Moving workflows from march-sdk-preview to sdk-preview
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
* Update README.md
* April Changes to SDK and CLI v2 (#1114 )
* Address CLI breaking changes
* Update nyc_taxi_data_regression.ipynb
* Updating pipelines
* Inline documentation updates
* Incorporating Sam's PR into this branch
* Installing new build
* Adding temp workflows for automated testing of notebooks
* Update model.ipynb
* Update model.ipynb
* Reverting model to sdk-preview version
* Update setup.sh
* Update setup.sh
* Adding a new build
* Removing all workflows from this branch
* Create sdk-assets-model-model.yml
* Update dev-requirements.txt
* Removing local path, adding @latest
* Updating to latest version of SDK/CLI and adding ipython_genutils
* Adding workflows for assets
* Update dev-requirements.txt
* Updating to Build # 58490278
* Adding tests for single step jobs
* Adding tests for endpoints
* Adding test workflows for pipelines
* Adding all test workflows
* Removing dataset notebook since we now have data
* Automated README.md
* Workflows for this branch
* Updating to latest build
* Update mnist-nonmlflow.ipynb
* Updating to latest build
* Adding description metadata to notebooks
* Improving README and Workflow generation
* Lochen/sdk march (#1041 )
* innit
* init
* Lochen/sdk march (#1042 )
* innit
* init
* merge folder
* fix pipeline sample: cifar-10, tf_mnist
* update workflows for pipeline
* Lochen/sdk march (#1044 )
* innit
* init
* merge folder
* change e2e registered, basic, nyc
* basic pipeline nyc
* add instructions.
* update nyc
* fix basic pipeline
* Upgrade to latest build
* Adjusting workflows and README for new pipeline samples
* Dummy commit for testing
* Update README.md
* Adding descriptions for pipeline notebooks
* Updating to latest build
* Removing datastores from testing and compute from scheduled runs
* Update lightgbm-iris-sweep.ipynb
* Updating to latest build
* Update README.md
* Fixing source directory for data and updating CLI/SDK build
* Updating to latest build
* Update README.md
* Adding clean up for endpoints and workspaces
* Update workspace.ipynb
* Updating to new build
* Update README.md
* Fixing issues with workspace notebook
* Update setup.sh
* Update setup.sh
* create new branch
* adapt sweep sample to april release
* adapt to command based job creation
* Updating workflows and readme to point to the april branch
* Updating CLI README to show April branch status
* Updating CLI workflows to trigger in april branch
* Changing workflows to trigger on april branch
* Builder fix up
* fix typo
* [April SDK] Pipeline builder function samples (#1117 )
* refine the samples
* re-structure pipeline sdk examples
* merge folder
* reorgnize pipeline samples
* refine
* add image_classification sample
* update dsl_component example
* add command function sample
* update
* refine command function sample
* update 3a_tf_mnist
* add sample 3e_image_classification
* update 3a_tf_mnist
* refine samples
* fix 3a_tf_mnist
* refine samples
* refine samples
* refine samples
* refine sample
* update sample 1a
* fix: sdk-3b_tf_mnist_sweep (#1060 )
fix entry func name in component definition
* refine sample
* refine samples
* add load auto load extention
* refine samples
* tf_minst_sweep: fix sample (#1062 )
* tf_minst_sweep: fix sample
* Update train.py
* Revert metric codes (#1061 )
* update mlflow
* still use azure.core
Co-authored-by: chenyin <chenyin@microsoft.com>
* refine samples
* sdk: 3c_cifar-10 sample bash encoding error (#1063 )
* add readme
* update readme
* print link that can be clicked (#1067 )
* refine samples
* rename the samples
* refine sample 1a
* Change DataInput/Output to ArtifactInput/Output
* refine sample 1a
* update naming for image classification
* refine 1e sample
* refine READMEs
* update README
* update ci workflow
* update ci SDK version
* add ci pipeline for 2d_image_classification_with_densenet
* update readme
* update readme
* add workflows
* merge march CLI and add mltable, sweep sample
* add march cli and sweep mltable support
* feat: refine sweep & cifar examples (#1078 )
* feat: refine sweep examples with dummy components
* fix: rollback python version
* fix: fix azure-ml version
* refactor: rename & use dataset
use a dataset instead of downloading inside a node
update learning objective & motivation
* fix: fix conda environment
* fix: add troubleshooting for gpu size not matched
* feat: demo more hyperparameters
* fix: update setup azure-ml version
* feat: add ci
* fix: update ci
* fix: fix ci
* fix: fix ci
* fix: fix ci
* fix: fix ci
* remove unused file
* fix: fix naming issue
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Refine dsl components samples (#1079 )
* Refine dsl components samples
* add CI workflow to Readme
* correct instance_count usage
* remove useless code
* Set resources for component
* update working dir
* change sample order to align with march version
* mltable
* include pipeline branch in CI
* update image classification workflow (#1085 )
* Refine common code snippet of samples (#1080 )
* change sample1
* move config location
* update CI
* refine
* refine wording
* change samples
* update snippet
* update notebook
* update
* update CI
* update 1e
* update samples
* refine sample
* refine inputs
* update
* add CI
* refine workflows
* fix dockerfile
* refine readme
* update 2d
* fix sample
* refine samples
* refine samples
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
* refine readme
* update
* update nyc to use mltable and mlflow model
* update doc link
* Updating to latest build
* Update README.md
* fix typo
* remove unneccesary imports
* Updating to latest build and testing in eastus
* Fixing workflows
* Update readme.py
* Online endpoints sdk (#1059 )
* online endpoints sdk samples
* online endpoints sdk samples
* minor update
* updating readme.md
* adding back orginal sample
* minor
* update to sample folder
* minor changes
* test workflow scope
* add job stream
* remove test workflow
* fix workflow cred
* update
* update sdk version
* update mnist sample
* Adding workflows and README for the new online endpoint examples
* Updating to the RC build
* update
* fix: flush (#1094 )
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* update build number
* update version
* change to rw_mount
* replace path with uri_folder
* fix: rename & correct the usage of setting (#1116 )
pipeline_job_setting.continue_on_step_failure
* Update sdk script gen pipeline jobs workflow (#1113 )
* update setup script
* update workflows
* fix checkout
* add script to write pipeline readme
* update readme
Co-authored-by: Han Wang <wanhan@microsoft.com>
* update compute (#1093 )
* Test v2 samples in magic workspace (#1092 )
* move test workspace
* change back CI
* add script to run all samples
* skip wait for completion
* fix script
* remove ps1
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Clwan/update nyc with mlflow_model type (#1118 )
* update notebook descriptions
* update readme
* update sdk nyc sample to adopt mlflow_model
* use sub folder as worksround for mlflow model
* refine samples
* update sweep and mlflow model
* refine samples
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: lochen <cloga0216@gmail.com>
* [Pipeline builder function] merge april branch (#1126 )
* add amlarc workflow (#968 )
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* committing onnx model generator script used with scriptrunconfig (#962 )
* updating onnx model generator with sriptrunconfig
* removing duplicacy with single onnx model script
* updating with minor changes
* updating with reviews
* minor changes
* model_name comparison changed
* minor changes
* Adding CodeGen related changes to bankmarketing notebook (#970 )
* added codegen related cells to bankmarketing notebook
* running black
* run black (#978 )
* Unpinning azureml-defaults to avoid installing itsdangerous coming indirectly from Flask (#981 )
* Unpinning azureml-defaults to avoid installing itsdangerous coming indirectly from Flask
* Pinning to the latest azureml version and touching files to trigger the builds
* Removing the update from docker
* updating mlflow deployment skus (#982 )
* Adding tags to certain cells in AutoML notebook (#964 )
* added metadata cells.
* fix inf env issue (#995 )
Co-authored-by: Yuan Zhuang <yuzhua@microsoft.com>
* Update dev-requirements.txt
* Create test.yml
* Delete test.yml
* Create sdk-assets-model-model.yml (#1001 )
* Create sdk-assets-model-model.yml
* Update sdk-assets-model-model.yml
* Update sdk-assets-model-model.yml
* Update and rename sdk-assets-model-model.yml to smoke-test-sdk-notebook-tests.yml
* Update and rename smoke-test-sdk-notebook-tests.yml to smoke-test-sdk-samplejob.yml
* Delete smoke-test-sdk-samplejob.yml
* Workflows for Testing SDKv2 Sample Notebooks
* Adding more workflows for sdk v2 notebook testing
* Adding workflows for SDKv2 Notebook Testing
* add language to kernelspec as asked by compute instance team (#1015 )
Co-authored-by: Lifeng Wei <lifengwei@microsoft.com>
* Add gpu diag/perf cli job (#980 )
* add gpu diag/perf cli job
* add path in docker instead of hardcoded in job
* update structure to have distinct folder for environment
* update environment to use inline
* finalize tests
* Upgrading the azure cli version
* Lifengwei/automl nlp notebooks (#1018 )
* multilabel example notebook with preprocessed data
* upgrade azureml_env_linux to 1.39 version
* make separate update_env file for new notebook
Co-authored-by: Lifeng Wei <lifengwei@microsoft.com>
* Lifengwei/automl nlp notebooks (#1026 )
* multiclass related files
* ner related files
* add language info to all notebooks
* make multilabel notebook trained with distributed training
Co-authored-by: Lifeeng Wei <lifengwei@microsoft.com>
* Fix a weird papermill error
Adding ipython_genutils to fix a weird papermill error
* update creds in cleanup script (#1029 )
* Distributed Deep Learning image end to end tutorial (#1023 )
* write e2e object dl notebook
* implement distributed pytorch
* implement mlflow model logging
* implement profiling
* add get_output, remove extra parameters, add comments for primary_metric (#1035 )
* add get_output, remove extra parameters, add comments for primary_metric
* update experiment names
* fix a typo
* typo
Co-authored-by: Lifeeng Wei <lifengwei@microsoft.com>
* Set idle_seconds_defore_scaledown (#1039 )
* Set idle_seconds_defore_scaledown
* Applied black, which removed some spaces
* Fixed AutoML notebooks pandas -1 (#1038 )
* update tfserving byoc sample (#983 )
* update tfserving byoc sample
* minor update
* anon byoc env (#1051 )
* Automl notebook fixes (#1046 )
* Jeffshep/fixcsrmatrixissue (#1072 )
* Fix error with csr_matrix is not JSON serializable
* Applied black
* support custom ennvironment packages by environemnt.yml (#1005 )
* update version of black to fix smoke test (#1088 )
* update
* remove workflow
* Lifengwei/update nlp notebooks (#1090 )
* remove some packages
* add back a package
* update notebooks, mainly change multilabel data file
* data file
* fix typos, codes in notebooks
* update experiment name
* fix typo, remove comments
Co-authored-by: Lifeeng Wei <lifengwei@microsoft.com>
* March sdk preview (#1021 )
* Address CLI breaking changes
* Update nyc_taxi_data_regression.ipynb
* Updating pipelines
* Inline documentation updates
* Incorporating Sam's PR into this branch
* Installing new build
* Adding temp workflows for automated testing of notebooks
* Update model.ipynb
* Update model.ipynb
* Reverting model to sdk-preview version
* Update setup.sh
* Update setup.sh
* Adding a new build
* Removing all workflows from this branch
* Create sdk-assets-model-model.yml
* Update dev-requirements.txt
* Removing local path, adding @latest
* Updating to latest version of SDK/CLI and adding ipython_genutils
* Adding workflows for assets
* Update dev-requirements.txt
* Updating to Build # 58490278
* Adding tests for single step jobs
* Adding tests for endpoints
* Adding test workflows for pipelines
* Adding all test workflows
* Removing dataset notebook since we now have data
* Automated README.md
* Workflows for this branch
* Updating to latest build
* Update mnist-nonmlflow.ipynb
* Updating to latest build
* Adding description metadata to notebooks
* Improving README and Workflow generation
* Lochen/sdk march (#1041 )
* innit
* init
* Lochen/sdk march (#1042 )
* innit
* init
* merge folder
* fix pipeline sample: cifar-10, tf_mnist
* update workflows for pipeline
* Lochen/sdk march (#1044 )
* innit
* init
* merge folder
* change e2e registered, basic, nyc
* basic pipeline nyc
* add instructions.
* update nyc
* fix basic pipeline
* Upgrade to latest build
* Adjusting workflows and README for new pipeline samples
* Dummy commit for testing
* Update README.md
* Adding descriptions for pipeline notebooks
* Updating to latest build
* Removing datastores from testing and compute from scheduled runs
* Update lightgbm-iris-sweep.ipynb
* Updating to latest build
* Update README.md
* Fixing source directory for data and updating CLI/SDK build
* Updating to latest build
* Update README.md
* Adding clean up for endpoints and workspaces
* Update workspace.ipynb
* Updating to new build
* Update README.md
* Fixing issues with workspace notebook
* Update setup.sh
* Update setup.sh
* Updating to latest build
* Update README.md
* Updating to latest build and testing in eastus
* Fixing workflows
* Update readme.py
* Online endpoints sdk (#1059 )
* online endpoints sdk samples
* online endpoints sdk samples
* minor update
* updating readme.md
* adding back orginal sample
* minor
* update to sample folder
* minor changes
* Adding workflows and README for the new online endpoint examples
* Updating to the RC build
* Update README.md
* Fixing issues with MIR samples
* Standardizing kernelspec for all notebooks
* Formatting Cleaning python files with black
* Fixing readme.py
* Moving workflows from march-sdk-preview to sdk-preview
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
* Modifying SDK workflows to sync up with March SDK changes
* Update README.md
* March CLI Breaking Changes (#1020 )
* remove component job, flatten code (#1002 )
* Update .gitignore
* Addressing CLI breaking changes
* adding uri_file to path
* adding uri_file type to paths
* adding uri_file and uri_folder
* Updating basic pipeline jobs
* Update hello-pipeline-default-artifacts.yml
* Lochen/cli march (#1009 )
* remove component job, flatten code
* add r component sample
* Online deployment changes
* Batch Deployment Changes
* Bug fixes to YAML
* Update REST API examples to Feb API (#1013 )
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
* minor fix
* resolve yaml error
* environment name and version as optional in custom containers (#1016 )
* minor updates (#1017 )
* Lochen/cli march (#1019 )
* remove component job, flatten code
* add r component sample
* update cli
* Updates to cifar example
* Adding daily build of CLI for testing
* Updating from dataset to data for create
* Rename folder from asset/datasets to assets/data
* Updating data assets to use data schema directive instead of dataset
* rename description to data asset
* Changing github workflows to use the new data assets
* removing dataset from md files
* Update README.md
* Removing local_path from examples
* Fixing python:latest bug with workaround
* Update prefix.md
* Lochen/cli march (#1027 )
* remove component job, flatten code
* add r component sample
* update cli
* update after test
* Updating environments to use latest version
* Updating environment to us latest
* Update setup.sh
* Test
* streamline register from a run (#1024 )
* streamline register from a run
* update to MLflow specific
* Lochen/cli march (#1031 )
* remove component job, flatten code
* add r component sample
* update cli
* update after test
* add default_compute, dataset
* Lochen/cli march (#1032 )
* remove component job, flatten code
* add r component sample
* update cli
* update after test
* add default_compute, dataset
* fix default compute, sweep
* fix default settings on pipeline
* Using the new build for CLI
* Update requirements.txt (#1033 )
* Updating to Build # 58490278
* Fixing filenames in the workflows.
* Fixing file name in train.sh
* add default value in pipeline settings.
* update default value
* change data asset
* update job link in readme
* Updating to latest CLI build
* Merge latest changes from main related to cli/jobs/single-step/gpu_perf/ command (#1037 )
* pick and choose changes from main
* align, add readme
* Update create-datasets.sh
* Updating to latest build
* Update README.md
* Update README.md
* Fixing failures in CLI
* Dummy commit to re-run tests
* fix to align with resource in test ws
* change 4d to use uri_folder
* upgrade dependencies, use fastparquet to fix pyarrow issue (#1043 )
* Upgrading to latest build
* merge datastore uri sample
* fix file script
* Users/anksing/mlflow model mir deploy (#1047 )
* Adding example for OBO flow using v2 cli
* Updating sklearn model for MIR deployment
* Updating lightgbm mlflow model for MIR deployment
* Removing obo folder
* Users/anksing/mlflow mir deployment (#1048 )
* Adding example for OBO flow using v2 cli
* Renaming model to avoid conflict with existing model with same name
* Removing obo folder
* Update batch-score.sh
Adding az version check.
* Updating model name to avoid conflict (#1049 )
* change cifar to directly use command
* add image classification cli example
* Updating to latest build
* Making setup files Unix friendly by removing \r
* Update endpoint.yml
remove the properties config -- we are not exposing in v2
* Update blue-deployment.yml
Removed properties field - not exposing user-configurable properties metadata in V2
* Dummy Commit to force tests
* update pipeline cifar to make it work under cpu-cluster
* update cifar sample to support cpu sku
* change 5c to cpu image
* clean up yaml, change cpu image for cifar
* Updating to latest build
* Update README.md
* formatting python files to ensure smoke test does not fail
* update workflow (#1064 )
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Wanhan/add image classification CI (#1065 )
* add image classification ci
* fix create component
Co-authored-by: Han Wang <wanhan@microsoft.com>
* fix image classification ci (#1066 )
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Update entry.py
* Update top latest build
* resolve merge conflict
* update to v2 dsl.component
* update train step (#1071 )
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Updating to latest build
* Update README.md
* Update README.md
* remove pyjokes in source file
* recover sample to test CI
* add component CRUD sample in assets
* Updating to new build
* Update setup.sh
* Changing readme temporarily to show march status
* Examples for workspace connections (#1073 )
* Create python-feed-pat.yml
* More examples
* Create README.md
* Update README.md
* update component name, add CI workflow
* update readme for asset component
* Update setup.sh
* Add yml for scheduled start stop of CI
Add yml for scheduled start stop of CI
* Delete instance-schedule.yml
* fix: fix attr overwrite command (#1074 )
* fix: fix attr overwrite command
add pipeline job test shell
* fix: fix attr overwrite command
* fix: create data with random version
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Updating to latest build
* Running tests on canary
* Trying to run test cases in master
* Reverting back to eastus
* Testing a new secret file
* Delete test-new-creds.yml
* Running tests on master
* Update job.yml (#1083 )
* Run tests in canary
* remove old dataset sample
* change process per node to 1 for cifar in pipeline
* add note to use multi-gpu cards
* Running latest build on eastus
* Update README.md
* fix sample issue to workaround output empty folder case
* Updating to latest build and testing in eastus
* Cleaning up all GitHub Workflows and the README file
* Update setup.sh
* Update dev-requirements.txt
* Fixing formatting issue so that black does not error
* Updating to the RC build
* change component name (#1075 )
* change component name
* rename
* add march-cli-preview in pipeline CI
* update registered case
* swith CI to master
* add back component creation
* update 3b to use rw_mount (#1097 )
* fix: fix run-pipeline-jobs.sh
* change upload mode to rw_mount
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Update README.md
* remove pipeline sample (#1102 )
* temp delete 2 samples
* remove reference links
Co-authored-by: Anthony Hu <anthu@microsoft.com>
* Fixing workflows and getting ready for merge to main
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Kayla Seager <1675997+kseager@users.noreply.github.com>
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
Co-authored-by: Paul Shealy <paulsh@microsoft.com>
Co-authored-by: Abraham Omorogbe <abrahamomor@gmail.com>
Co-authored-by: Ankit Singhal <30610298+singankit@users.noreply.github.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
Co-authored-by: Hugo Aponte <haponte@gmail.com>
Co-authored-by: Lisa Li (STC) <lisal@microsoft.com>
Co-authored-by: mx-iao <33701412+mx-iao@users.noreply.github.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Honglin Du <0mza987@gmail.com>
Co-authored-by: Roope Astala <rastala@users.noreply.github.com>
Co-authored-by: Swati Gharse <36742198+swatig007@users.noreply.github.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: eniac871 <supereniac@live.com>
* Updating workflows and readme to point to the april branch
* Updating CLI README to show April branch status
* Updating CLI workflows to trigger in april branch
* Changing workflows to trigger on april branch
* Builder fix up
* fix typo
* update workflows
Co-authored-by: Zeliang Tian <83852443+zetiaatgithub@users.noreply.github.com>
Co-authored-by: Vadthyavath Ram <7171558+vadthyavath@users.noreply.github.com>
Co-authored-by: Razvan Tanase <ratanase@microsoft.com>
Co-authored-by: Cody Peterson <54814569+lostmygithubaccount@users.noreply.github.com>
Co-authored-by: Harneet Virk <harnvir@microsoft.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
Co-authored-by: Yuan Zhuang <40252949+yuanzhuangyuanzhuang@users.noreply.github.com>
Co-authored-by: Yuan Zhuang <yuzhua@microsoft.com>
Co-authored-by: Bala P V <33712765+balapv@users.noreply.github.com>
Co-authored-by: Lifeng Wei <lfwei@ucdavis.edu>
Co-authored-by: Lifeng Wei <lifengwei@microsoft.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
Co-authored-by: Sethu Raman <rsethur@users.noreply.github.com>
Co-authored-by: jeff-shepherd <39775772+jeff-shepherd@users.noreply.github.com>
Co-authored-by: v-pbavanari <97779269+v-pbavanari@users.noreply.github.com>
Co-authored-by: Banibrata De <banibrata@outlook.com>
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Bala P V <balapv@microsoft.com>
Co-authored-by: Kayla Seager <1675997+kseager@users.noreply.github.com>
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
Co-authored-by: Paul Shealy <paulsh@microsoft.com>
Co-authored-by: Abraham Omorogbe <abrahamomor@gmail.com>
Co-authored-by: Ankit Singhal <30610298+singankit@users.noreply.github.com>
Co-authored-by: Hugo Aponte <haponte@gmail.com>
Co-authored-by: Lisa Li (STC) <lisal@microsoft.com>
Co-authored-by: mx-iao <33701412+mx-iao@users.noreply.github.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Honglin Du <0mza987@gmail.com>
Co-authored-by: Roope Astala <rastala@users.noreply.github.com>
Co-authored-by: Swati Gharse <36742198+swatig007@users.noreply.github.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Hanchi Wang <hancwang@microsoft.com>
Co-authored-by: Daniel Schneider <danielsc@microsoft.com>
* feat: sweep cli sdk examples update (#1125 )
* feat: use mlflow.autolog instead of Run.log
* fix: fix predict code
* fix: update parameters
* add image_classification_keras_minist_convnet (#1127 )
* add image classification kerad
* update
* update
* add readme
* add workflow and update README
Co-authored-by: Clement Wang <clwan@microsoft.com>
* fix: cifar-10/image_classification output mode
* update readme
* feat: descriptions for cli pipeline job examples (#1129 )
* feat: descriptions for cli pipeline job examples
* fix: remove hello git & update setup.sh
* fix readme
* Lochen/dataset asset (#1128 )
* write model to load then move to output folder
* add dataset dataasset sample
* add assest
* change component name to lower case
* refine samples
* update workflows
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: lochen <cloga0216@gmail.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
Co-authored-by: Ying Chen <2601502859@qq.com>
Co-authored-by: chenyin <chenyin@microsoft.com>
Co-authored-by: Korin <0mza987@gmail.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: Bala P V <balapv@microsoft.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
Co-authored-by: Zeliang Tian <83852443+zetiaatgithub@users.noreply.github.com>
Co-authored-by: Vadthyavath Ram <7171558+vadthyavath@users.noreply.github.com>
Co-authored-by: Razvan Tanase <ratanase@microsoft.com>
Co-authored-by: Cody Peterson <54814569+lostmygithubaccount@users.noreply.github.com>
Co-authored-by: Harneet Virk <harnvir@microsoft.com>
Co-authored-by: Yuan Zhuang <40252949+yuanzhuangyuanzhuang@users.noreply.github.com>
Co-authored-by: Yuan Zhuang <yuzhua@microsoft.com>
Co-authored-by: Bala P V <33712765+balapv@users.noreply.github.com>
Co-authored-by: Lifeng Wei <lfwei@ucdavis.edu>
Co-authored-by: Lifeng Wei <lifengwei@microsoft.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
Co-authored-by: Sethu Raman <rsethur@users.noreply.github.com>
Co-authored-by: jeff-shepherd <39775772+jeff-shepherd@users.noreply.github.com>
Co-authored-by: v-pbavanari <97779269+v-pbavanari@users.noreply.github.com>
Co-authored-by: Banibrata De <banibrata@outlook.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Kayla Seager <1675997+kseager@users.noreply.github.com>
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
Co-authored-by: Paul Shealy <paulsh@microsoft.com>
Co-authored-by: Abraham Omorogbe <abrahamomor@gmail.com>
Co-authored-by: Ankit Singhal <30610298+singankit@users.noreply.github.com>
Co-authored-by: Hugo Aponte <haponte@gmail.com>
Co-authored-by: Lisa Li (STC) <lisal@microsoft.com>
Co-authored-by: mx-iao <33701412+mx-iao@users.noreply.github.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Roope Astala <rastala@users.noreply.github.com>
Co-authored-by: Swati Gharse <36742198+swatig007@users.noreply.github.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Hanchi Wang <hancwang@microsoft.com>
Co-authored-by: Daniel Schneider <danielsc@microsoft.com>
* Pipeline April SDK Preview (#1130 )
* minor change
* update
* update
* make prepare step pass
* refine cli yaml: display_name
* fix cifar sample
* fix notebook
* fix: upgrade build version
* fix: fix 4d_dataset_input
* update keras sample
* ci: trigger ci on april-sdk-preview
* fix: do not always checkout april sdk preview
* fix: eval_mount is supported in new sdk/cli
* fix: merge main image classification
* fix: fix ci
* update dev build version
* fix: recover dataset related ci
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: Ying Chen <2601502859@qq.com>
* Pipeline April SDK Preview (#1137 )
* fix: reduce 6a/6b instance count
* fix: add asset creation to run-job-pipeline-all
* fix: remove repeating cli job status
* fix: use uri_folder instead of path
* fix ci : assets/component
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
* update SDK/CLI test for force_rerun (#1140 )
* update sdk code for rerun
* update cli test for rerun
* revert change in notebook
* add rerun in sdk ci pipeline
* revert
* add rerun in sdk ci pipeline
* update readme.py
* Pipeline SDK Preview Features (#1138 )
* add component archive/restore, customize output
* add name to path
* refine image classification sample
* clear outputs.
* Refine component notebook
* Rai pipeline sample (#1139 )
* Add RAI CLI sample pipeline
* add github workflow
* update pipeline yaml postfix
* add pre-setup steps
* update workflow to correct name
* add rai sample
* add rai sdk sample ci workflow
* change name to 2f
* update schema refer and workflow
* update readme and workflow name
* rename workflow to correct name
* minor update
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Honglin Du <0mza987@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
* update workflow & readme
* refine rai sample
* update
* update
* update readme
* update
Co-authored-by: lochen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Honglin Du <0mza987@gmail.com>
* AutoML preview PR (Python SDK and CLI examples) (#1089 )
* Baseline of AutoML Classification task notebook
* Updated to code and wheel 3-17-2022
* Updates for new wheel
* Forecasting baseline
* Fixed Forecasting notebook now working
* Deleted not used folders
* Added current Custom Environments
* Classifier notebook updated
* Forecasting notebook updated
* Baseline for Regression sample notebook
* WIP baseline AutoML for Images
* Updated to use URI-FOLDER data asset for Images
* Updates working Image-Classification with MLTables
* Baseline for AutoML NLP Text Classif multiclass
* Updated Images notebook to use Curated Env
* Updated NLP notebook with Curated Env
* Updates on NLP and minor IMG
* Minor updates, working notebooks
* Add notebooks for rest of image tasks (#1077 )
* Special PrP properties updated for all ntbooks
* Temporal WIP MLFlow
* MLFlow updates
* Baseline AutoML CLI example
* Updates SDK AutoML-MLFlow notebook
* Updates to hello AutoML CLI YAMLs
* Updates to CLI AutoML examples
* adding test github action for automl classification
* renamed the automl workflow
* Minor updates to CLI YAML examples
* Updates to MLFlow-AutoML notebook
* Updated _automl_internal_save_mlflow
* Updated mlflow-automl notebook with more MLTables
* delete old MLTable
* Minor update to YAML config
* CLI examples updates forecasting and classif
* adding automl workflows
* modifying notebook to wait for completion
* fixing workflows.
* changing notebooks to fetch create MLClient from config and wait for experiment to complete.
* update trials to 5 to overcome bug.
* updating to latest private wheel
* fixing classification & regression notebooks
* comment out set_limits for image notebooks
* fixing image object detection
* parameterizing automl notebooks with compute name
* fixing notebook whitespaces
* adding new line and remove obsolete action workflow
* fixing mltable path
* cleaning notebooks, running nb_black
* changing automl cli scripts
* adding AutoML CLI workflows.
* Porting AutoML samples into automl-preview. (#1101 )
* adding automl workflows
* modifying notebook to wait for completion
* fixing workflows.
* changing notebooks to fetch create MLClient from config and wait for experiment to complete.
* update trials to 5 to overcome bug.
* updating to latest private wheel
* fixing classification & regression notebooks
* comment out set_limits for image notebooks
* fixing image object detection
* parameterizing automl notebooks with compute name
* fixing notebook whitespaces
* adding new line and remove obsolete action workflow
* fixing mltable path
* cleaning notebooks, running nb_black
* changing automl cli scripts
* adding AutoML CLI workflows.
* adding parameter to instance segmentation notebook.
* Updates on getting MLFlow tracking URI
* Add yaml cli examples for image tasks (#1104 )
* revert to using macros for workspace setup in Bankmarketing example.
* Fix paths in jsonl files for image notebooks (#1105 )
* Updates for MLFlow URI and other
* Fix json issue
* Subscription, workspace.. aligned..
* merged from parent branch.
* removing non-automl actions.
* Forecasting updates
* Minor updates on classifc notebook
* Update on "data" folder for Images notebook
* Workspace IDS for macros fixed
* data folder update for regression notebook
* Clean-up output images multi-class
* Several fixes to image-classif-multi-label
* fixing the resource group name macro.
* Minor clean up in image classif notebooks
* updating the automl actions to refer automl-preview branch
* branch name changes in actions.
* Object Detection updated to use "./data"
* Updates on "./data" to image segmentation
* Clean up image segmentation
* Updates working Image classif multi-label and segm
* Commented setLimits to Image multi-label
* update encoding to utf8 (#1107 )
Co-authored-by: Miseon Park <mipark@microsoft.com>
* remove set_limits from image object detection notebook.
* add NLP cli examples (#1108 )
Co-authored-by: Miseon Park <mipark@microsoft.com>
* modify automl CLI workflows
* adding sdk-preview to the list of branches to monitor for CLI actions.
* ensuring all AutoML CLI workflows get triggered.
* fixing CLI setup script
* fixing additional properties for classification & forecasting.
* CLI Regression example baseline
* pinning to latest CLI & SDK & updating actions for CLI.
* Updated Regression CLI YAML working
* attempting to fix the bankmarketing cli example.
* fix setup.sh for sdk.
* adding regression CLI action.
* remove drop_columns from featurization
* update automl cli yamls to camel case and latest schemas
* removing CLI actions from running on PRs on automl-preview branch
* Fix set_limits() for image tasks. Fix image classification yaml (#1110 )
* Fix set_limits() for image tasks. Fix image classification yaml
* CLI example fixes for schema changes
* Don't specify a version while uploading data folder
* Add scripts to prepare data for image cli jobs (#1120 )
* AutoML April preview - SDK + CLI (#1121 )
* Address CLI breaking changes
* Update nyc_taxi_data_regression.ipynb
* Updating pipelines
* Inline documentation updates
* Incorporating Sam's PR into this branch
* Installing new build
* Adding temp workflows for automated testing of notebooks
* Update model.ipynb
* Update model.ipynb
* Reverting model to sdk-preview version
* Update setup.sh
* Update setup.sh
* Adding a new build
* Removing all workflows from this branch
* Create sdk-assets-model-model.yml
* Update dev-requirements.txt
* Removing local path, adding @latest
* Updating to latest version of SDK/CLI and adding ipython_genutils
* Adding workflows for assets
* Update dev-requirements.txt
* Updating to Build # 58490278
* Adding tests for single step jobs
* Adding tests for endpoints
* Adding test workflows for pipelines
* Adding all test workflows
* Removing dataset notebook since we now have data
* Automated README.md
* Workflows for this branch
* Updating to latest build
* Update mnist-nonmlflow.ipynb
* Updating to latest build
* Adding description metadata to notebooks
* Improving README and Workflow generation
* Lochen/sdk march (#1041 )
* innit
* init
* Lochen/sdk march (#1042 )
* innit
* init
* merge folder
* fix pipeline sample: cifar-10, tf_mnist
* update workflows for pipeline
* Lochen/sdk march (#1044 )
* innit
* init
* merge folder
* change e2e registered, basic, nyc
* basic pipeline nyc
* add instructions.
* update nyc
* fix basic pipeline
* Upgrade to latest build
* Adjusting workflows and README for new pipeline samples
* Dummy commit for testing
* Update README.md
* Adding descriptions for pipeline notebooks
* Updating to latest build
* Removing datastores from testing and compute from scheduled runs
* Update lightgbm-iris-sweep.ipynb
* Updating to latest build
* Update README.md
* Fixing source directory for data and updating CLI/SDK build
* Updating to latest build
* Update README.md
* Adding clean up for endpoints and workspaces
* Update workspace.ipynb
* Updating to new build
* Update README.md
* Fixing issues with workspace notebook
* Update setup.sh
* Update setup.sh
* create new branch
* adapt sweep sample to april release
* adapt to command based job creation
* Updating workflows and readme to point to the april branch
* Updating CLI README to show April branch status
* Updating CLI workflows to trigger in april branch
* Changing workflows to trigger on april branch
* Builder fix up
* fix typo
Co-authored-by: Bala P V <balapv@microsoft.com>
Co-authored-by: Bala P V <33712765+balapv@users.noreply.github.com>
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Daniel Schneider <schneider@volkskammer.org>
Co-authored-by: Hanchi Wang <hancwang@microsoft.com>
Co-authored-by: Daniel Schneider <danielsc@microsoft.com>
Co-authored-by: Gaurav Singh <gasi@microsoft.com>
* cleanup
* renaming the folder for automl CLI jobs
* removing AutoML CLI workflows.
* regenerated all the CLI workflows
* regenerated SDK workflows
* Minor changes for Workspace prop showup
* Minor changes to AutoML-MLFlow notebook
* Undo non-automl files
* Add jsonl files pointing to GH actions workspace for automl images (#1136 )
* Added MLFlow model local test
* Removed MLFlow inference to point to next notebook
* Update cli-jobs-automl-standalone-jobs-cli-automl-classification-task-bankmarketing-cli-automl-classification-task-bankmarketing.yml
testing
* Update cli-jobs-automl-standalone-jobs-cli-automl-classification-task-bankmarketing-cli-automl-classification-task-bankmarketing.yml
revert
* ml_flow.set_experiment() not needed, removed
* Forecasting updates for MLFlow
* regenerate automl workflows for april branch merge test
* Update cli-automl-classification-task-bankmarketing.yml
* Update cli-automl-classification-task-bankmarketing.yml
* remove bankmarketing mlflow.yml from workflows
* update to specs
Co-authored-by: PhaniShekhar <30535191+PhaniShekhar@users.noreply.github.com>
Co-authored-by: Razvan Tanase <ratanase@microsoft.com>
Co-authored-by: Miseon <msp8955@gmail.com>
Co-authored-by: Miseon Park <mipark@microsoft.com>
Co-authored-by: Gaurav Singh <svaruag@outlook.com>
Co-authored-by: Bala P V <balapv@microsoft.com>
Co-authored-by: Bala P V <33712765+balapv@users.noreply.github.com>
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Daniel Schneider <schneider@volkskammer.org>
Co-authored-by: Hanchi Wang <hancwang@microsoft.com>
Co-authored-by: Daniel Schneider <danielsc@microsoft.com>
Co-authored-by: Gaurav Singh <gasi@microsoft.com>
Co-authored-by: Srujan Saggam <41802116+srsaggam@users.noreply.github.com>
* Pipeline SDK/CLI April Features (#1144 )
* update add missing pipeline workflows
* update sdk workflows & readme
* add force rerun
* move dsl files to src folder (#1145 )
* update data format
* remove invalid workflow
* update RAI samples: 1, update register data name to align with the on… (#1146 )
* update RAI samples: 1, update register data name to align with the one in RAI repo. 2, SDK sample reuse the component from CLI one. 3, SDK use a more complext pipeline. 4, update readme
* remove some unuse comment
Co-authored-by: Anthony Hu <anthu@microsoft.com>
* fix: fix sweep cli example
* parallel run notebooks
* fix encoding
* add dry run
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Korin <0mza987@gmail.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Added .TXT files on how to run CLI AutoML (#1150 )
* Added .TXT on how to run CLI AutoML
* Comment added about GPU cluster
* blocked_algos commented due to bug
* Removed dev temporal internal properties
* Prep data explanation sentence fixed
* Lochen/april release 0413 (#1147 )
* update add missing pipeline workflows
* update sdk workflows & readme
* add force rerun
* move dsl files to src folder (#1145 )
* update data format
* remove invalid workflow
* update RAI samples: 1, update register data name to align with the on… (#1146 )
* update RAI samples: 1, update register data name to align with the one in RAI repo. 2, SDK sample reuse the component from CLI one. 3, SDK use a more complext pipeline. 4, update readme
* remove some unuse comment
Co-authored-by: Anthony Hu <anthu@microsoft.com>
* fix: fix sweep cli example
* parallel run notebooks
* fix encoding
* add dry run
* data asset dataset, remove pipeline level input
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Korin <0mza987@gmail.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Updates to workflows and build Numbers
* Add files via upload
* Add files via upload
* Add files via upload
* Updating workflows
* fix links (#1156 )
* Moving to DeafultAzureCredential instead of Browser Auth. Removing branch check-out.
* Adding AutoML CLI tests, removing apr branch checkout
* Moving from commandjob to command
* Fixing pipeline credentials
* Update AutoML Job schema urls (#1160 )
* Using requirements from SDK folder. Adding pandas to requirements
* remove automl-preview reference (#1143 )
* remove mlflow WIP notebooks from github actions
* remove pandas reference
* remove pandas after merge
Co-authored-by: Miseon Park <mipark@microsoft.com>
* Pipeline April SDK/CLI preview samples refine (#1157 )
* merge
* update sample to make compute config lines fewer
* make configure credential simpler
* update output setting of nyc_tax_data_regression
* refine rai samples
* update sdk ci workflow
* update
* fix rai
* use newer dev build before nightly build issue resolved
* update workflow
* Rename sample node name to align with pipeline node restriction (#1158 )
* fix name
* fix name
* fix sample
* fix
* fix
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Update pipeline sdk samples to use azure.ml.Input/Output
* update setup.sh
* update
* refine
* feat: update run-job-pipeline-all auto-gen
* fix: fix image classification example
* update setup.sh
* update Input
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Updating to latest build
* Update prefix.md
* Update prefix.md
* Update import paths for automl entities in image notebooks (#1164 )
* Update cli-automl-classification-task-bankmarketing.yml
* Update cli-automl-forecasting-task-energy-demand.yml
* Update cli-automl-forecasting-task-energy-demand.yml
* Pipeline April CLIL/SDK Feature refine (#1161 )
* remove mlflow workaround
* remove mlflow molde workaround
* update sdk nyc sample
* resolve components ci failure
* add configuration notebook
* simplify pipeline_with_components_from_yaml notebook
* fix hello-pipeline-customize-output-folder.yml
* simplify samples
* refine
* update workflow and readme
* update
* add display name for hello pipelines
* update
Co-authored-by: lochen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Updating to latest build
* Fixing links in docs
* Delete online-inferencing-sample.ipynb
* Update automl-classification-task-bankmarketing-mlflow.ipynb
* Converting all links to configuration.ipynb file into a relative link
* Fixing links to README file
* Fixing links to other files and folders
* Hancwang/april (#1168 )
* Clean up iris
* demo notebook
* notebook used in demo
* namespace update
* more import fix
* remove studio_url
* more conflicts
* revert readme
* Fix build error
* Update hello-automl-job-basic.yml
* Pipeline April SDK Feature samples refine (#1175 )
* remove refrence of azure.ml._constants
* revert notebook rename
* update
* update to latest master build
* fix run all
* fix SDK sample link
* update pipeline sdk 2e sample (#1176 )
* update pipeline sdk 2e sample
* update compute
* fix credential part
* adapt to latest version sdk
* update
* update
* update
* update
* update conda aml sdk version
* update according to comments
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Blanca Li <39058063+likebupt@users.noreply.github.com>
* Updating nightly build
* Update namespace from azure.ml._constants to azure.ml.constants (#1174 )
Co-authored-by: Clement Wang <clwan@microsoft.com>
* Updating description for preview
* endpoints named by sample (#1180 )
* add custom featurization to tests (#1172 )
Co-authored-by: Miseon Park <mipark@microsoft.com>
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Daniel Schneider <schneider@volkskammer.org>
Co-authored-by: Hanchi Wang <hancwang@microsoft.com>
Co-authored-by: Daniel Schneider <danielsc@microsoft.com>
Co-authored-by: Clement Wang <47586720+wangchao1230@users.noreply.github.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
Co-authored-by: Ying Chen <2601502859@qq.com>
Co-authored-by: chenyin <chenyin@microsoft.com>
Co-authored-by: Korin <0mza987@gmail.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
Co-authored-by: Zeliang Tian <83852443+zetiaatgithub@users.noreply.github.com>
Co-authored-by: Vadthyavath Ram <7171558+vadthyavath@users.noreply.github.com>
Co-authored-by: Razvan Tanase <ratanase@microsoft.com>
Co-authored-by: Cody Peterson <54814569+lostmygithubaccount@users.noreply.github.com>
Co-authored-by: Harneet Virk <harnvir@microsoft.com>
Co-authored-by: Yuan Zhuang <40252949+yuanzhuangyuanzhuang@users.noreply.github.com>
Co-authored-by: Yuan Zhuang <yuzhua@microsoft.com>
Co-authored-by: Lifeng Wei <lfwei@ucdavis.edu>
Co-authored-by: Lifeng Wei <lifengwei@microsoft.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
Co-authored-by: Sethu Raman <rsethur@users.noreply.github.com>
Co-authored-by: jeff-shepherd <39775772+jeff-shepherd@users.noreply.github.com>
Co-authored-by: v-pbavanari <97779269+v-pbavanari@users.noreply.github.com>
Co-authored-by: Banibrata De <banibrata@outlook.com>
Co-authored-by: Kayla Seager <1675997+kseager@users.noreply.github.com>
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
Co-authored-by: Paul Shealy <paulsh@microsoft.com>
Co-authored-by: Abraham Omorogbe <abrahamomor@gmail.com>
Co-authored-by: Ankit Singhal <30610298+singankit@users.noreply.github.com>
Co-authored-by: Hugo Aponte <haponte@gmail.com>
Co-authored-by: Lisa Li (STC) <lisal@microsoft.com>
Co-authored-by: mx-iao <33701412+mx-iao@users.noreply.github.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Roope Astala <rastala@users.noreply.github.com>
Co-authored-by: Swati Gharse <36742198+swatig007@users.noreply.github.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Cesar De la Torre <cesardl@microsoft.com>
Co-authored-by: PhaniShekhar <30535191+PhaniShekhar@users.noreply.github.com>
Co-authored-by: Miseon <msp8955@gmail.com>
Co-authored-by: Miseon Park <mipark@microsoft.com>
Co-authored-by: Gaurav Singh <svaruag@outlook.com>
Co-authored-by: Gaurav Singh <gasi@microsoft.com>
Co-authored-by: Srujan Saggam <41802116+srsaggam@users.noreply.github.com>
Co-authored-by: Hanchi Wang <luigiking307@gmail.com>
Co-authored-by: Blanca Li <39058063+likebupt@users.noreply.github.com>
* Adjusting workflows and readme to point to correct branch
* SDK preview - pipeline samples refine (#1183 )
* remove unnecesarry files
* update imports in pipeline samples
* remove is_deterministic: False
* update 2e sample (#1184 )
* refine wording
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Blanca Li <39058063+likebupt@users.noreply.github.com>
* Reformatting python files with black
* Formatting notebooks with black
* Revise e2e vision notebook to use commands and sdk april release (#1167 )
* remove coco dependencies in component
* remove spec
* revise requirements
* revise notebook to use commands only
* adapt to april release
* restore notebooks
* remove pipeline snapshot
* update the e2e-ds-experience nb to april release (#1185 )
Co-authored-by: amah <ma.mahmoudzadeh@gmail.com>
* Adding metadata to refer from docs
* Pipeline SDK Preview Samples Refine (#1187 )
* update component notebook description
* remove enum before issue resolved
* refine component notebook
* add description
* refine 1e sample
* revert enum
* fix cli ci
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* add aml arc inferencing notebook (#1194 )
* add notebooks for amlarc inferencing
* add workflow for amlarc inferencing notebook
* fix: specify gpu related environment version (#1199 )
* Update sdk-endpoints-online-amlarc-online-endpoints-safe-rollout.yml (#1200 )
fix ipynb file name issue
* update 2e sample (#1201 )
* update pipeline job sdk 2e sample
* update description of the pipeline
* comment invoke method for aks policy (#1203 )
comment invoke out as cluster under dev subscription can't be accessed from public internet.
* Update online-endpoints-simple-deployment.ipynb
* add k8s prefix for amlarc online endpoint name (#1212 )
* Update online-endpoints-safe-rollout.ipynb
add k8s prefix for amlarc endpoint name
* feat: use mlflow in nyc-taxi-data-regression (#1208 )
* feat: use mlflow to save model in nyc_taxi_data_regression
* feat: optimize run-job-pipeline related shells
* Uncomment allowed and blocked models for tabular tasks (#1196 )
* Uncomment allowed and blocked models for tabular tasks
* fix forecasting
Co-authored-by: Ayush Mishra <ayushmishra@microsoft.com>
* add metadata to sdk notebook examples (#1217 )
Co-authored-by: Miseon Park <mipark@microsoft.com>
* update to May API version (#1223 )
* Update main.py
* rename amlarc to kubernetes (#1219 )
* rename amlarc to kubernetes
Co-authored-by: Xue Wei <xuewe@microsoft.com>
* enable download for rai pipeline (#1221 )
* enable download for rai pipeline
* remove empty cell
* refine
* Add metadata tags for automl images (#1242 )
* Add metadata tags for automl images
* Separate cells
* Minor metadat
* Update kubernetes-online-endpoints-safe-rollout.ipynb (#1251 )
* update pipeline sdk 2e sample (#1260 )
* added k8s related workflows result (#1266 )
* add cell name for doc reference (#1267 )
* Fixes to e2e dl distributed pytorch notebook from bugbash (#1255 )
* implement details
* add compute test
* update 2e sample (#1265 )
* update 2e sample
* update
* update
* Changing namespace from azure.ml to azure.ai.ml
* Changing JobInput to Input in AutoML
* Removing JobInput and azure.ml from files
* Bug FIxes
* Delete old endpoint sample test
* Adding metadata for cells - needed for docs
* [Pipeline] Fix azure-ml -> azure-ai-ml (#1282 )
* update azure-ml version in conda.yml
* update @dsl.pipeline -> @pipeline
* update build version
Co-authored-by: Clement Wang <clwan@microsoft.com>
* update 2e sample file and related reference (#1289 )
* Updating to new builds
* Update lightgbm-iris-sweep.ipynb
* sync workflow scripts from main to sdk-preview branch (#1300 )
* Update setup.sh
* Update setup.sh
* Update setup.sh
* Update setup.sh
* update train.py. r (#1299 )
emove mlflow related comment to avoid confuse audience.
* Sync notebook with doc (#1291 )
* for docs tutorial
* Updating tutorial to match docs
* run black on this
* remove dataset
* rename notebook
* add alert for cpu-name
* add note, fix intro
* Updating daily build
* Update load_component and switch other YAML loading to use load_x() (#1314 )
* Update load_component and switch other YAML loading to use load_x()
* Update to the latest private wheel
* Remove urireference
* sync scripts (#1310 )
* managed vnet for managed online endpoints (#1303 )
* vnet sample
* update
* updates
* updates
* updates
* update dir
* scripts
* add model
* updates
* update scripts
* cleanup
* cleanup
* cleanup
* parameterize the branch for testing
* test with pr branch
* test sample branch
* test branch
* test branch
* fix the branch name
* automated vnet test: fix the branch name
* validate scoring
* fix script
* fix script
* update the setup instructions
* update comments
* fix formatting
* additional comments
* fix review comments
* update scoring test scripts
* update keyvault naming
* update build script
* reorganize directories
* update path
* update pr check
* update
* remove uai to seperate file
* mlflow vnet sample
* mlflow vnet sample
* mlflow vnet example
* mlflow vnet example
* docs comments
* mirror traffic
* update to new syntax
* test
* testing with debug
* debug
* revert setup.sh changes
* debug
* latest cli
* updated cli
* update cli version
* adding local deployment
* local deployment
* test
* local deployment
* new cli build
* new cli version
* latest cli version
* testing fix for apt update error
* fix update error
* remove unused packages
* newer version of cli
* remove private preview flags
* custom cli version for testing from vm
* Users/mipark/update automl standalone examples (#1316 )
* cli updates
* sdk updates
* add model deploy and local inference examples
* update readme
Co-authored-by: Miseon Park <mipark@microsoft.com>
* Updating to latest builds
* Add notebooks for local model inference. Model deployment (#1317 )
* remove data settings from hello automl notebook (#1319 )
* cli updates
* sdk updates
* add model deploy and local inference examples
* update readme
* update automl basic example
* update workflows
Co-authored-by: Miseon Park <mipark@microsoft.com>
* Update setup.sh
* Updating to latest build
* Removing rw_mount from samples
* pick non private preview change (#1326 )
* pick non private preview change
* update samples
* simplify @pipeline usage
* remove description and name for pipeline, add doc string instead
* fix: fix readme.md for rai sample
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Honglin Du <0mza987@gmail.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
* update mldesigner install (#1333 )
Co-authored-by: Han Wang <wanhan@microsoft.com>
* fix automl examples (#1331 )
* fix examples
* Correct timeout_minutes in Object detection notebook
* Correct import paths in prepare_data.py files
Co-authored-by: Miseon Park <mipark@microsoft.com>
Co-authored-by: Phani Shekhar Mantripragada <phmantri@microsoft.com>
* updating rest api version (#1329 )
Co-authored-by: Raghu Ramaswamy <raghura@microsoft.com>
* Reformat notebooks, update latest build, improve cleanup and README generation
* Reformatting notebooks
* Reformatting some more notebooks
* Updating README and workflows for SDK v2
* Updating CLI workflows
* Updating to use defaultAzure Creds
* updated the e2e-ds-notebook to may release (#1325 )
* updated the e2e-ds-notebook to may release
* cleaned notebook cell outputs
* fixed some terms to the new ones
Co-authored-by: amah <ma.mahmoudzadeh@gmail.com>
* Changes to fix online endpoint issues
* Fixing Online Deployment scripts
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Dennis Eikelenboom <92387505+deeikele@users.noreply.github.com>
Co-authored-by: Tracy Chen <47156502+tracychms@users.noreply.github.com>
Co-authored-by: Hanchi Wang <luigiking307@gmail.com>
Co-authored-by: Cody <54814569+lostmygithubaccount@users.noreply.github.com>
Co-authored-by: Paul Shealy <paulsh@microsoft.com>
Co-authored-by: abeomor <abrahaomor@gmail.com>
Co-authored-by: amah <mamahmoudzadeh@gmail.com>
Co-authored-by: amah <ma.mahmoudzadeh@gmail.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
Co-authored-by: Janhavi Mahajan <janhavi13@users.noreply.github.com>
Co-authored-by: Banibrata De <banibrata@outlook.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
Co-authored-by: Daniel Schneider <schneider@volkskammer.org>
Co-authored-by: Hanchi Wang <hancwang@microsoft.com>
Co-authored-by: Daniel Schneider <danielsc@microsoft.com>
Co-authored-by: Clement Wang <47586720+wangchao1230@users.noreply.github.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
Co-authored-by: Ying Chen <2601502859@qq.com>
Co-authored-by: chenyin <chenyin@microsoft.com>
Co-authored-by: Korin <0mza987@gmail.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: Zeliang Tian <83852443+zetiaatgithub@users.noreply.github.com>
Co-authored-by: Vadthyavath Ram <7171558+vadthyavath@users.noreply.github.com>
Co-authored-by: Razvan Tanase <ratanase@microsoft.com>
Co-authored-by: Harneet Virk <harnvir@microsoft.com>
Co-authored-by: Yuan Zhuang <40252949+yuanzhuangyuanzhuang@users.noreply.github.com>
Co-authored-by: Yuan Zhuang <yuzhua@microsoft.com>
Co-authored-by: Lifeng Wei <lfwei@ucdavis.edu>
Co-authored-by: Lifeng Wei <lifengwei@microsoft.com>
Co-authored-by: Sethu Raman <rsethur@users.noreply.github.com>
Co-authored-by: jeff-shepherd <39775772+jeff-shepherd@users.noreply.github.com>
Co-authored-by: v-pbavanari <97779269+v-pbavanari@users.noreply.github.com>
Co-authored-by: Kayla Seager <1675997+kseager@users.noreply.github.com>
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
Co-authored-by: Abraham Omorogbe <abrahamomor@gmail.com>
Co-authored-by: Ankit Singhal <30610298+singankit@users.noreply.github.com>
Co-authored-by: Hugo Aponte <haponte@gmail.com>
Co-authored-by: Lisa Li (STC) <lisal@microsoft.com>
Co-authored-by: mx-iao <33701412+mx-iao@users.noreply.github.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Roope Astala <rastala@users.noreply.github.com>
Co-authored-by: Swati Gharse <36742198+swatig007@users.noreply.github.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Cesar De la Torre <cesardl@microsoft.com>
Co-authored-by: PhaniShekhar <30535191+PhaniShekhar@users.noreply.github.com>
Co-authored-by: Miseon <msp8955@gmail.com>
Co-authored-by: Miseon Park <mipark@microsoft.com>
Co-authored-by: Gaurav Singh <svaruag@outlook.com>
Co-authored-by: Gaurav Singh <gasi@microsoft.com>
Co-authored-by: Srujan Saggam <41802116+srsaggam@users.noreply.github.com>
Co-authored-by: Blanca Li <39058063+likebupt@users.noreply.github.com>
Co-authored-by: snowei <snowei1988@gmail.com>
Co-authored-by: Ayush Mishra <61145377+novaturient95@users.noreply.github.com>
Co-authored-by: Ayush Mishra <ayushmishra@microsoft.com>
Co-authored-by: Xiao Zhang <zhanxia@microsoft.com>
Co-authored-by: Sheri Gilley <sgilley@microsoft.com>
Co-authored-by: Phani Shekhar Mantripragada <phmantri@microsoft.com>
Co-authored-by: Raghu R <13340619+raghutillu@users.noreply.github.com>
Co-authored-by: Raghu Ramaswamy <raghura@microsoft.com>
2022-05-23 19:13:49 -07:00
bbaf9c844c
remove unused packages ( #1293 )
...
* remove unused packages
* remove unused packages
2022-05-12 10:19:51 -07:00
c46a0de2b2
feat: re-enable image/cifar-10 pipeline tests & update readme.py ( #1112 )
...
* feat: test ci
* feat: enable cifar-10 & image classification
* feat: add sample for hyperparameter sweep
* fix: fix ci
* fix: fix ci
* fix ci: update readme.py
* fix: black reformat
* Revert "fix: black reformat"
This reverts commit d47755571f
2022-04-08 09:47:34 +08:00
f50f94756a
March CLI Breaking Changes ( #1020 )
...
* remove component job, flatten code (#1002 )
* Update .gitignore
* Addressing CLI breaking changes
* adding uri_file to path
* adding uri_file type to paths
* adding uri_file and uri_folder
* Updating basic pipeline jobs
* Update hello-pipeline-default-artifacts.yml
* Lochen/cli march (#1009 )
* remove component job, flatten code
* add r component sample
* Online deployment changes
* Batch Deployment Changes
* Bug fixes to YAML
* Update REST API examples to Feb API (#1013 )
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
* minor fix
* resolve yaml error
* environment name and version as optional in custom containers (#1016 )
* minor updates (#1017 )
* Lochen/cli march (#1019 )
* remove component job, flatten code
* add r component sample
* update cli
* Updates to cifar example
* Adding daily build of CLI for testing
* Updating from dataset to data for create
* Rename folder from asset/datasets to assets/data
* Updating data assets to use data schema directive instead of dataset
* rename description to data asset
* Changing github workflows to use the new data assets
* removing dataset from md files
* Update README.md
* Removing local_path from examples
* Fixing python:latest bug with workaround
* Update prefix.md
* Lochen/cli march (#1027 )
* remove component job, flatten code
* add r component sample
* update cli
* update after test
* Updating environments to use latest version
* Updating environment to us latest
* Update setup.sh
* Test
* streamline register from a run (#1024 )
* streamline register from a run
* update to MLflow specific
* Lochen/cli march (#1031 )
* remove component job, flatten code
* add r component sample
* update cli
* update after test
* add default_compute, dataset
* Lochen/cli march (#1032 )
* remove component job, flatten code
* add r component sample
* update cli
* update after test
* add default_compute, dataset
* fix default compute, sweep
* fix default settings on pipeline
* Using the new build for CLI
* Update requirements.txt (#1033 )
* Updating to Build # 58490278
* Fixing filenames in the workflows.
* Fixing file name in train.sh
* add default value in pipeline settings.
* update default value
* change data asset
* update job link in readme
* Updating to latest CLI build
* Merge latest changes from main related to cli/jobs/single-step/gpu_perf/ command (#1037 )
* pick and choose changes from main
* align, add readme
* Update create-datasets.sh
* Updating to latest build
* Update README.md
* Update README.md
* Fixing failures in CLI
* Dummy commit to re-run tests
* fix to align with resource in test ws
* change 4d to use uri_folder
* upgrade dependencies, use fastparquet to fix pyarrow issue (#1043 )
* Upgrading to latest build
* merge datastore uri sample
* fix file script
* Users/anksing/mlflow model mir deploy (#1047 )
* Adding example for OBO flow using v2 cli
* Updating sklearn model for MIR deployment
* Updating lightgbm mlflow model for MIR deployment
* Removing obo folder
* Users/anksing/mlflow mir deployment (#1048 )
* Adding example for OBO flow using v2 cli
* Renaming model to avoid conflict with existing model with same name
* Removing obo folder
* Update batch-score.sh
Adding az version check.
* Updating model name to avoid conflict (#1049 )
* change cifar to directly use command
* add image classification cli example
* Updating to latest build
* Making setup files Unix friendly by removing \r
* Update endpoint.yml
remove the properties config -- we are not exposing in v2
* Update blue-deployment.yml
Removed properties field - not exposing user-configurable properties metadata in V2
* Dummy Commit to force tests
* update pipeline cifar to make it work under cpu-cluster
* update cifar sample to support cpu sku
* change 5c to cpu image
* clean up yaml, change cpu image for cifar
* Updating to latest build
* Update README.md
* formatting python files to ensure smoke test does not fail
* update workflow (#1064 )
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Wanhan/add image classification CI (#1065 )
* add image classification ci
* fix create component
Co-authored-by: Han Wang <wanhan@microsoft.com>
* fix image classification ci (#1066 )
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Update entry.py
* Update top latest build
* resolve merge conflict
* update to v2 dsl.component
* update train step (#1071 )
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Updating to latest build
* Update README.md
* Update README.md
* remove pyjokes in source file
* recover sample to test CI
* add component CRUD sample in assets
* Updating to new build
* Update setup.sh
* Changing readme temporarily to show march status
* Examples for workspace connections (#1073 )
* Create python-feed-pat.yml
* More examples
* Create README.md
* Update README.md
* update component name, add CI workflow
* update readme for asset component
* Update setup.sh
* Add yml for scheduled start stop of CI
Add yml for scheduled start stop of CI
* Delete instance-schedule.yml
* fix: fix attr overwrite command (#1074 )
* fix: fix attr overwrite command
add pipeline job test shell
* fix: fix attr overwrite command
* fix: create data with random version
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Updating to latest build
* Running tests on canary
* Trying to run test cases in master
* Reverting back to eastus
* Testing a new secret file
* Delete test-new-creds.yml
* Running tests on master
* Update job.yml (#1083 )
* Run tests in canary
* remove old dataset sample
* change process per node to 1 for cifar in pipeline
* add note to use multi-gpu cards
* Running latest build on eastus
* Update README.md
* fix sample issue to workaround output empty folder case
* Updating to latest build and testing in eastus
* Cleaning up all GitHub Workflows and the README file
* Update setup.sh
* Update dev-requirements.txt
* Fixing formatting issue so that black does not error
* Updating to the RC build
* change component name (#1075 )
* change component name
* rename
* add march-cli-preview in pipeline CI
* update registered case
* swith CI to master
* add back component creation
* update 3b to use rw_mount (#1097 )
* fix: fix run-pipeline-jobs.sh
* change upload mode to rw_mount
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Update README.md
* remove pipeline sample (#1102 )
* temp delete 2 samples
* remove reference links
Co-authored-by: Anthony Hu <anthu@microsoft.com>
* Fixing workflows and getting ready for merge to main
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Kayla Seager <1675997+kseager@users.noreply.github.com>
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
Co-authored-by: Paul Shealy <paulsh@microsoft.com>
Co-authored-by: Abraham Omorogbe <abrahamomor@gmail.com>
Co-authored-by: Ankit Singhal <30610298+singankit@users.noreply.github.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
Co-authored-by: Hugo Aponte <haponte@gmail.com>
Co-authored-by: Lisa Li (STC) <lisal@microsoft.com>
Co-authored-by: mx-iao <33701412+mx-iao@users.noreply.github.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Honglin Du <0mza987@gmail.com>
Co-authored-by: Roope Astala <rastala@users.noreply.github.com>
Co-authored-by: Swati Gharse <36742198+swatig007@users.noreply.github.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: eniac871 <supereniac@live.com>
2022-04-04 10:14:29 -07:00
a4837f6cb7
update examples for CLI (v2) public preview refresh ( #890 )
...
* update setup to use preview CLI
* Create setup.sh
* update schemas to preview
* Update setup.sh
* Update setup.sh
* Update setup.sh
* Update setup.sh
* Update setup.sh
* add environment examples (#736 )
* Create docker image example
* Rename docker image example to docker_image.yml
* Create Dockerfile
* Create docker_context.yml
* refactor and updates
* update creds, readme, workflows
* fix
* remove workflow
* fix mistake
* fix
Co-authored-by: Cody <54814569+lostmygithubaccount@users.noreply.github.com>
* Update docker-context.yml (#739 )
* Update setup.sh
* Update setup.sh
* Update setup.sh
* Update setup.sh
* major jobs refactor; add pipeline job (#745 )
* add basic and nyc taxi pipelines samples under jobs
* major refactor
* minor edit
* minor edit
* save
* remove sweep dir
* update inputs outputs and compute for cli 0.7
* update component yamls nyc taxi example
Co-authored-by: Sharmeelee Bijlani <shbijlan@microsoft.com>
* Create simple-compute-cluster.yml (#748 )
* update compute cluster examples (#752 )
* update compute examples
* shorten name
* low_priorty -> lowpriority
* add locaiton example
* add compute-instance
* remove instances from testing
* refactor jobs; add simple pipeline examples (#753 )
* refactor jobs
* save
* pipeline edits
* switch to canary
* switch to eastus
* fully test
* switch back to master
* reduce python-sdk testing freq; cli and setup cleanup (#759 )
* repo health
* rerun readme
* environment example updates (#760 )
* remove version pinning; add version pinning
* edits
* Update setup.sh
* add workflow_dispatch: to cli workflows (#761 )
* touch setup.sh
* add workflow_dispatch: to cli jobs
* Update setup.sh
* Update setup.sh
* fix low priority compute example (#762 )
* Update setup.sh
* public preview refresh changes (#750 )
* fixes
* changes
* revert setup.sh changes
* latest cli
* cli changes
* update dir structure
* update name
* update cli version
* cli version
* cli version
* cleanup
* update comments
Co-authored-by: Cody <54814569+lostmygithubaccount@users.noreply.github.com>
* Update setup.sh
* drop _job (#772 )
* Add files via upload
* fix validation scripts for user specified identity and system assigned identity (#747 )
* fix sai + uai scripts
* remove set x
* fix workspace name
* fix arg
* fix rest
* add sku
* change credentials to mir
* touch
* add check
* add extra line ending
* add object info to role assignment
* forgotten vars
* fix role assignment
* fix other role assignments
* use latest version of sdk
* fix arm calls
* remove hardcoded vals
* fix hardcoded values
* replace endpoint name with var
* add euap location
* remove pined version of cli
* fix sai role assignment
* update credential logic in readme for uai, sai
* format code
* fixed comments
* cleaned up
* cleanup
* revert
* fix tags
* fix uai
Co-authored-by: max <public.max@outlook.com>
Co-authored-by: Cody <54814569+lostmygithubaccount@users.noreply.github.com>
* clean up nyc taxi pipeline (#773 )
* update sweep timeout (#778 )
* pin badges to main branch (#779 )
* pin badges to main branch
* save
* switch testing to master
* manually update schemas
* Update setup.sh
* Update PyTorch example (#791 )
* Update setup.sh
* add cifar-10 pipeline example (#794 )
* save
* save - training updates
* remove cifar from untested jobs
* save
* save
* switch setup to eastus
* set CR to false
* fix some jobs
* update readme
* save
* save
* fix setup
* run readme
* add data copy script
* edit
* edit
* save
* save
* fix
* save
* edit cifar loc
* fix
* fix
* Update setup.sh
* batch GA samples (#722 )
* batch score GA samples
* update batch
* update batch
* update batch sample
* update anon env
* move non mlflow up
* update test flow
* clean up batch sample
* update batch sample
* update schema ref
* rename yml files
* update yml file name
* minor update
* Update setup.sh
* Update workspace CLI examples (#767 )
* Update workspace CLI examples
* updating examples per PR feedback
* property ordering and update app insights attr fix
* include example on adding private link endpoint
* include private link example
* simplify ple example
* disabled in capital and consistent resource suffix
* consisten resource names
* save
* fix
* add ws delete
* fixes
* fix
* fix
* remove workspace script
Co-authored-by: Cody <54814569+lostmygithubaccount@users.noreply.github.com>
* add diabetes data
* fix
* fix
* save
* Fixed typo in snippet (#796 )
* pipelines-with-components basic examples (#777 )
* pipelines-with-components
* data samples
* tf hello world
* create components and dataset in repo setup
* pytorch hello world
* update readme; format code
* update create-datasets.sh
* fix readme; remove unused job; run readme
* add missing files
* setup updates
Co-authored-by: Cody <54814569+lostmygithubaccount@users.noreply.github.com>
* create variables (#799 )
* local endpoint test script (#804 )
* script for local endpoint
* local endpoint
* Added Dataset and Datastore YAML samples. (#800 )
* Added Dataset and Datastore YAML samples.
* Updated README files.
* renames and minor updates
* Updated Dataset samples README.
Co-authored-by: Cody <54814569+lostmygithubaccount@users.noreply.github.com>
* standardization and fixes (#805 )
* update schemas
* update assets
* update resources
* update setup-repo
* save
* udpate assets
* update run logic (#806 )
* update schemas
* update assets
* update resources
* update setup-repo
* save
* udpate assets
* update run logic
* omit run-job.sh
* print job uri
* remove workflow
* edit run-job.sh
* update run-job.sh
* update run-job.sh
* fix cifar pipeline (#809 )
* Update setup.sh
* ; => : (#812 )
* model-dir => model_dir (#813 )
* update run-job.sh
* mount => ro_mount (#814 )
* fix nyc-taxi pipeline example (#815 )
* ever feel stupid? I feel stupid
* fix 4d_dataset_input (#816 )
* clean up batch endpoints (#817 )
* switch RG (#818 )
* Update setup.sh
* format code
* update jobs and things (#822 )
* save
* save
* save
* save
* save
* save
* save
* save
* format code
* remove cli/jobs/basics/data
* save
* save
* save
* update
* fix
* minor edit
* edit
* switch to eastus
* format code
* Add functional Batch Endpoint REST sample (#820 )
* add batch score rest sample
* add git workflow
* update REST sample
* update sample
* Functional October batch REST sample -- score from Dataset failing on request
* Clean up script
* run readme
* Fix some quoting issues
* reset location
* Fix modelUri
* Fail early if job submit failed
* Reset readme.py
* Fix location to be based on workspace
* Add other terminal state
Co-authored-by: Tracy Chen <47156502+tracychms@users.noreply.github.com>
Co-authored-by: Cody <54814569+lostmygithubaccount@users.noreply.github.com>
* minor edits
* save
* update
* job updates (#824 )
* save
* updates
* minor updates
* minor update
* fix cifar dataset
* Update hello-iris-file.yml
* fixes (#827 )
* fixes
* fix
* fix
* add docker context job example for sklearn (#828 )
* add docker context job example for sklearn
* run readme
* change image
* Add missing section opener (#829 )
Missing opening comment for create_environment
* minor edit
* remove arc examples (#830 )
* save
* add hello-code
* autoscale (#832 )
* fix unique output (#831 )
* fix unique output
* fix unique output
* beautify json
* update REST invoke
* fix typo
* remove delete prompt
* unique output folder
* add deployment delete
* consistency edits (#834 )
* --set experiment_name in train.sh (#835 )
* fix instance (#836 )
* minor consistency edits (#837 )
* removing aks sample from cli-preview (#844 )
* Update torchserve BYOC sample (#845 )
* add notebooks via papermill job examples (#847 )
* add notebooks via papermill job examples
* output notebook to ./outputs/
* fix tag (#848 )
* byoc tfserving sample (#853 )
* byoc tfserving sample
* deployment name fix
* r-byoc-update (#854 )
* Update setup.sh
* update
* update
* update vm attach example (#855 )
* Revert "update vm attach example" (#856 )
This reverts commit f3a7efe4fa#858 )
* fix typo in directory; rw-mount to rw_mount (#859 )
* fix spark example (#857 )
* fix spark example
* save
* try pin pip
* ?
* try older pip
* pin image
* format code
* update rest sample (#861 )
* update rest sample
* fix format
* fix typo
* fix output path
* update outputdataset to output
* update output
* move dask to named output
* add tag for wait (#866 )
* Fixing closing region tag (#867 )
read_condafile section tags were not correctly closed.
* swap install steps in setup.sh
* edit comment
* run black
* remove CIs from refresh testing (#873 )
* add defaults list to misc script (#874 )
* https => wasbs for dask folder example (#875 )
* https => wasbs for dask folder example
* test moving container
* Adding MLflow sample for online endpoints (#872 )
* Adding MLflow sample for online endpoints
* run readme.py
* run black
* Revert "run black"
This reverts commit b649bb564d#876 )
* Update triton samples to ncd - single model
* minor spelling error
* addressing build failures -1
* addressing build failures -2
* addressing missing dependencies
* adding missing dependency 2
* update pip install tritonclient[http]
* addressing build failures
* minor update to tags (#877 )
* remove extra triton examples (#881 )
* remove flaml notebook action
* add ssh settings to vm attach (#883 )
* remove .schemas; consistency edit to scripts (#887 )
* remove .schemas
* consistency edit
* remove unused workflow
* switch cli install to public
* remove broken job (#892 )
* fix merge issues
* fix job logic
Co-authored-by: Saachi Gopal <sg932@cornell.edu>
Co-authored-by: Sharmeelee Bijlani <shbijlan@microsoft.com>
Co-authored-by: vijetajo <40418529+vijetajo@users.noreply.github.com>
Co-authored-by: Sethu Raman <rsethur@users.noreply.github.com>
Co-authored-by: Bala P V <33712765+balapv@users.noreply.github.com>
Co-authored-by: max markin <maxormo@gmail.com>
Co-authored-by: max <public.max@outlook.com>
Co-authored-by: Paul Shealy <paulsh@microsoft.com>
Co-authored-by: Tracy Chen <47156502+tracychms@users.noreply.github.com>
Co-authored-by: Dennis Eikelenboom <mail@denniseik.nl>
Co-authored-by: Larry O'Brien <lobrien@knowing.net>
Co-authored-by: Manoj Bableshwar <manoj.bableshwar@outlook.com>
Co-authored-by: Yogi Pandey <20666257+ynpandey@users.noreply.github.com>
Co-authored-by: Christopher Pirillo <pirillo.chris@gmail.com>
Co-authored-by: Larry Franks <larry.franks@microsoft.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
Co-authored-by: Daniel Schneider <daniel.schneider@microsoft.com>
2021-11-04 07:46:46 -07:00
265e887885
exclude create-compute.sh script from workflows ( #696 )
2021-08-23 10:30:52 -07:00
24c96cbd3a
fix badge links for cli jobs ( #685 )
2021-08-19 07:09:05 -07:00
2fd3becf7f
improve status badge links across repo ( #682 )
...
* better badge links
* fix automl links
2021-08-19 06:35:58 -07:00
f7d7e4d5d7
remove local runs, reorganize hello world jobs ( #679 )
...
* remove local runs, reorganize hello world jobs
* fix targets
* remove unused jobs dirs
* fix misc.sh
2021-08-18 09:44:21 -07:00
cd018e7a3b
action cleanup; run all cli tests on changes to setup.sh ( #589 )
...
* updated ghactions for pre-release CLI wheel
Signed-off-by: Joe Zhou <jiazho@microsoft.com>
* updated workflow files
Signed-off-by: Joe Zhou <jiazho@microsoft.com>
* updated workflow files
Signed-off-by: Joe Zhou <jiazho@microsoft.com>
* updated cleanup script
* switch to public install
Co-authored-by: Cody <54814569+lostmygithubaccount@users.noreply.github.com>
2021-06-15 10:18:12 -07:00
15e13cf59f
commit ( #574 )
2021-06-08 13:07:59 -07:00
5371a80aaf
cli edits ( #567 )
...
* test
* test
* test
* save
* save
* save
* save
* minor edit
* code format
* fix docs action
* test
* test2
* ?
* origin/
* run code formatter
2021-06-08 07:11:10 -07:00
d5e7b8cb5d
Create env-var-job.yml ( #558 )
...
* Create env-var-job.yml
* edits
* readme edit
Co-authored-by: Cody <54814569+lostmygithubaccount@users.noreply.github.com>
2021-06-02 16:10:54 -07:00
25c3baf484
minor edit, test credential fix ( #550 )
2021-05-26 06:58:03 -07:00
7ead601426
update workspace, resource group, credentials ( #537 )
...
* switch workspaces
* add back storage key
* more fixes
* fixes
* fix cleanup
* decrement asset versions
2021-05-21 15:47:49 -07:00
1c5974102e
update to use public cli install ( #531 )
2021-05-21 09:26:59 -07:00
ea96a9cc6b
update to 2.0.0a1 ( #518 )
2021-05-17 11:20:22 -07:00
20d2fb9986
creds for MIR ( #501 )
...
* creds for MIR
* formatting
2021-05-11 20:35:19 -07:00
85942a733a
update docs workflows ( #500 )
2021-05-11 19:11:01 -07:00
3172ab93a8
refactor for CLI 2.0 preview ( #495 )
...
* scaffolding
* fixes, might break
* update contributing
* fix
* try 0.63
* fix readme
* order of cells
* fix env
* uncomment lines
* try a job
* add job test
* try miniconda image
* try mcr image
* -n -> --name for az ml workspace create
* fix workspace create
* same
* isolate rg
* fix filename
* simply env; increment version
* add mlflow
* fix enviornment, increment version
* switch to .sh
* add workspace create
* quick fix
* fix typo
* add asset creation
* debugging
* fix action
* fix
* typo
* fix typo
* workflow edits
* fix?
* edits
* fix data?
* Add more commands
* try something
* remove notebook
* update
* fix?
* try data workaround
* fix training script
* add azureml-mlflow to env
* saving
* try sweep job
* sweep edits
* fix yaml
* remove limits
* testing
* edits
* minor edits to basic.yml
* add exp name
* saving
* save; green?
* move --web to untested.sh
* hmm
* add model assets
* remove dsstore
* saving w/ model create
* add endpoints dir
* try something for docs
* try for docs
* try anon data and assets
* edit
* path edit
* fix asset path
* move stuff around
* try ./
* fix mistake
* save
* add mount
* fix
* change model to use run
* remove model register
* data hack
* test
* test registered file dataset
* saving
* testing
* whoops
* change strategy
* fix name
* switch rg back
* switch creds
* switch lightgbm back
* fix ws
* add file:
* remove deepspeed
* try tensorflow
* remove tensorflow
* edits for docs
* add verify
* unisntall -> remove
* -0
* saving
* add check_extensions
* fix mistake
* saving; might break
* update manage-assets and test it
* whoops
* minor edits
* local hello world
* edits
* simplify workflow
* try compute job
* ?
* try compute type
* fix hello world
* fix?
* major refactor
* remove line
* comments
* save
* save
* order of operations
* saving - run sweep job
* magic
* switch to data - possible typos
* fix typos?
* ...
* try dedenting for sweep
* edits
* names
* sweep hack for now
* broke stuff?
* test distributed
* update query logic
* add exp name, :1 to env
* wait for job to complete
* fix user error
* env to version 11
* readme edit
* try something
* remove redundant stuff
* version
* try downloading cifar
* try job status check
* debug bash
* fixes
* try stuff
* add back log upload
* fix bash?
* copy examples/endpoints from v2-preview
* run formatting
* add batch endpoint (#427 )
* hack
* move compute create to setup.sh; --name to -n for workspace create
* NC6 -> NC12
* try randint sweep
* remove randint from sweep; download=False for pytorch-cifar
* comment out logs upload from worklow
* slim iris lightgbm code
* try something for cifar
* fix
* preprocess -> process
* mlflow.pytorch log model for cifar
* no mlflow, fix cifar
* run formatting
* relocate hello world
* whoops
* cifar epochs 20 -> 2
* try new env for pytorch gpu
* switch pytorch to mlflow logging
* cleanup cifar code
* fix
* write to ./model?
* edit batch endpoint asset path
* force add model.pkl
* copy workspace assets
* add some schemas, more needed
* add more schemas
* fix schemas
* Updating the schema link to CDN endpoint (#428 )
* Update batch endpoint test (#429 )
* update schemas
* fix more schemas
* simplify
* Revert "simplify"
This reverts commit 38ec77e0ca#432 )
* test python/cli refactor (#431 )
* saving
* Update environment.yml (#430 )
* ?
* testing
* testing
* testing
* cli fix
* fix
* saving
* saving
* whoops
* fix readme
* fix readme
* whoops
* minor readme edit
* last readme edits
* add setup workspace step
* fix cli tests (#433 )
* test
* testing
* fixes
* save
* test
* fix how-to-manage-assets
* minor edit
* fix
* fix setup-workspace
* fix readme (#434 )
* edits for docs (#437 )
* fix readme
* remove sweep hack
* readme fix
* command => sweep in schema
* saving
* add back setup-workspace.sh
* ?
* sweep hack
* Saving
* edit readme asset logic
* -A
* minor edits for docs (#438 )
* fix readme
* remove sweep hack
* readme fix
* command => sweep in schema
* saving
* add back setup-workspace.sh
* ?
* sweep hack
* Saving
* edit readme asset logic
* -A
* >- => |
* missed hello world
* remove old env
* blind copy
* yolo
* edits
* edits
* docs edits (#440 )
* docs edits
* save
* remove upgrade
* fix
* reorder
* save (#442 )
* edit training jobs (#443 )
* saving
* minor edit
* fix world language? (#444 )
* change tf horovod env (#445 )
* change tf horovod env
* -
* move exp name
* standardize cli jobs (#446 )
* test
* standardize training jobs
* online endpoints (#435 )
* online endpoints
* run readme.py; manual edit workflows
* fix delete command
* fix
* fix endpoint
* rerun readme
Co-authored-by: Cody <54814569+lostmygithubaccount@users.noreply.github.com>
* quick fix (#448 )
* run readme (#449 )
* docs updates (#450 )
* minor edit for docs (#451 )
* docs updates
* quick fix
* undo sweep hack (#452 )
* edits for docs (#453 )
* add hello world back
* updates
* minor edits
* Update python-ml-basic-cpu.yml
* Add REST examples (#439 )
* Add scripts without blob cli
* Everything working for endpoint except discrimnator issue with online deployment
* Working simple job flow
* Use default container for data
* Rename scripts, make wait_for_completion a function, fix online deployment
* Add scoring to endpoint
* Change structure of folders
* Remove todo
* Move rest to cli folder
* Switch to lightgbm from xgboost
* Fix wait_for_completion
* Get AZURE_STORAGE_KEY from az cli
* Change MIR region to westus
* Remove exports from variables
* Fix scoring the endpoint
Co-authored-by: Trent McCormick <trmccorm@microsoft.com>
* Fix closing tag of sample to match opening tag. (#455 )
* updates for docs (#456 )
* switch cli version back (#457 )
* /
* tests for rest how-to (#454 )
* run readme
* save
* test rest with azureml-examples
* fixes
* run readme
* Use old way of getting certain variables
Co-authored-by: tmccrmck <tjmcc01@gmail.com>
* Update how-to-batch-score.sh (#436 )
Fix batch asset path
* force add files (#458 )
* endpoint script fix (#459 )
* fix endpoints
* fix endpoints
* fix endpoint script
* fix endpoints
* edits for docs (#461 )
* use
* exp name edits
* bash edits
* save
* save
* edit
* Get storage info from default datastore (#460 )
* simplify hello world
* move RG (#463 )
* switch credentials (#464 )
* debug (#465 )
* debug
* fix typo
* fjeiwpaol tefjpao
* Tweak to canary-flow shellscript (#466 )
* Fix `test_green` snippet
`test_green` snippet was dupe of `green_10pct_traffic`. Switched to syntax described in https://azure.github.io/azureml-v2-preview/_build/html/overview/concepts/endpoints/online-endpoint/scenarios/declarative-flow.html
* Results of `python readme.py`
This seems erroneous to me, but...
* Changed endpoint name to my-new-endpoint per rest of script
* fixed typo in var name
Co-authored-by: Cody <54814569+lostmygithubaccount@users.noreply.github.com>
* Add managed identity shell script for online endpoint (#467 )
* Online endpoint managed identities tests shell script
* Updated online endpoint managed identity shell scripts
Co-authored-by: pansav <pansav@microsoft.com>
* imperative flow (#469 )
* .yaml -> .yml (#470 )
* .yaml -> yml
* update python files
* edit
* Add get deployment logs to rest docs (#473 )
* Add getLogs API call
* Fix typo
* minor edit to hello world (#474 )
* add -example to exp name
* edits for docs (#476 )
* hacks
* order of operations
* edits
* increment model
* edits to batch
* fix readme
* reduce training times to 1 epoch
* Update setup.sh
* >- -> > (#477 )
* update CLI version (#480 )
* update CLI version
* try 0.71
* use 1.0.0a2
* Gopalv/tfserving sample (#479 )
* WIP
* adding logic to extract ACR
* ran readme.py
* local testing works
* sed works
* parametrized, added local and remote test
* adding workflow file
* acr login failing
* adding sleep after docker run
* workflows
* Fixed deployment yaml for online endpoint managed identity example (#482 )
Co-authored-by: pansav <pansav@microsoft.com>
* updates to cli-preview (#484 )
* testing
* update dockerfiles
* swithc back to conda yml
* test
* ?
* ?
* dead snakes
* fix job
* remove non-python jobs
* test
* job descriptions
* asset descritptions
* docs
* save
* saveing
* Fixed online endpoint automation failures issues (#489 )
* Fixed automation failures issues
* Removed try catch block from run method
* Fixed declarative yaml
* Updated docker image uri to Ubuntu 18.04
* Update system identity flow to use endpoint show to retrieve object id
Co-authored-by: pansav <pansav@microsoft.com>
* Add how-to-train-autorest with Python SDK (#481 )
* Add autorest example
* Fixes for REST scripts
* Removed linux specific code in shell script (#492 )
* Removed linux specific code in shell script
* Added delay after system identity is created and also changed secrets id
* Changed CLI version for managed identity tests
* Fixed declarative yaml file and upgraded to latest CLI version for managed identity
* Changed user identity place holder name
* Changed online endpoint schema url in yaml files
* Fixed endpoint delete command in declarative flow
* Fixed file path in declarative sample
Co-authored-by: pansav <pansav@microsoft.com>
* Fix how-to-deploy-script (#491 )
* Fix how-to-deploy-script
* Use random versions for environmetns
* Don't use headers for async operations
* Use jq -r everywhere
* Print async operation error
* Try changing endpoint
* Add mlflow model file (#486 )
* update batch sample (#493 )
* Gopalv/triton sample (#490 )
* WIP
* adding logic to extract ACR
* ran readme.py
* local testing works
* sed works
* parametrized, added local and remote test
* adding workflow file
* acr login failing
* adding sleep after docker run
* workflows
* updating yaml and script to exit properly
* adding deletion
* fixing typos
* making base endpoint more generic
* try after deletion
* renaming tfserving
* moved file to correct place
* works until MFE error
* not tested locally
* new CLI drop seems to work
* adding triton endpoint
* using anonymous environments
* adding back placeholder
* tfserving fails for some reason
* triton works
* get different acr
* install pyindex
* deleting model
* fixing model delete
* factoring cleanup utility
* adding exception handling for acr fail
* fixing syntax error
* making image url an argument
* using public image for tfserving
* add back tutorial test, rename
* manual modify of recently added workflow
* repin black; minor edit
* run black-nb
* fix typo
* test horovod env (#496 )
* test horovod env
* revert to old environment
* try v2
* revert
* try oss horovod
* try tf image
* try horoovd image
* switch horovod env
* 1.0.0a2 -> 1.0.0a4
* update batch sample (#497 )
* Removed min/max instances and changed auth_mode to key (#498 )
Co-authored-by: pansav <pansav@microsoft.com>
* saving (#499 )
* minor reqs edits
Co-authored-by: Tracy Chen <47156502+tracychms@users.noreply.github.com>
Co-authored-by: Harneet Virk <harnvir@microsoft.com>
Co-authored-by: Sethu Raman <rsethur@users.noreply.github.com>
Co-authored-by: trent <tjmcc01@gmail.com>
Co-authored-by: Trent McCormick <trmccorm@microsoft.com>
Co-authored-by: vmagelo <51368918+vmagelo@users.noreply.github.com>
Co-authored-by: Larry O'Brien <lobrien@knowing.net>
Co-authored-by: psavdekar <45885005+psavdekar@users.noreply.github.com>
Co-authored-by: pansav <pansav@microsoft.com>
Co-authored-by: Gopal Vashishtha <gopalv@microsoft.com>
2021-05-11 18:53:32 -07:00
Diondra
Vivian Li
Fred Li
kdestin
cassieesvelt
Alain LI
jeff-shepherd
Clement Wang
Jun
Yogi Pandey
Gaurav Gupta
Harneet Virk
eniac871
Neehar Duvvuri
Raghu Ramaswamy
Cody Peterson
Bala P V
Sethu Raman
elliotzh
Jiaxiao Zhou
Jordan Edwards