5285f9c7e8
Merge Build 2022 changes to main ( #1182 )
...
* Standalone Jobs and Pipelines with Core SDK
* Dataset Examples
* Adding environment examples
* Examples for models
* remove unwanted files
* Create workspace.ipynb
* Create compute.ipynb
* Update readme.md
* Removing credentials from notebook
* Lochen/pipeline dsl example (#913 )
* add DSL example
* update readme
* Formatting (#914 )
* add batch sdk sample (#918 )
* Remove unwanted files from command-job-in-pipeline
* removing unwanted files from pipelines with components
* Lochen/pipeline dsl example (#920 )
* add DSL example
* update readme
* change component name
* update folder name
* remove sample add more notebook
* Adding example for online inferencing
* Add files via upload
* Add files via upload
* Create datastore.ipynb
* Update MLClient to the latest interface and making use of promoted properties (#921 )
* Updated command job examples to align with the latest release
* ignore notebook cache
* remove cache
* Online inferencing example
* Update samples
* Pin to a specific private preview SDK version (#924 )
* Pin to a specific private preview API version
* typo
* readme consistency edits (#926 )
* readme consistency edits
* rename?
* fix errant backticks (#927 )
* Adding sweep job to samples (#931 )
* Adding sweep job to samples
* Update lightgbm-iris-sweep.ipynb
* Update .gitignore
* Update README.md
* How to use datasets in a job
* How to use datastore
* Adding Documentation the PyTorch notebook.
* Create notebook-template.ipynb
* Accidents Example for R
* Update accident-prediction.ipynb
* Adding R sample
* Updates to documentation
* Updating Documentation
* Updating documentation
* Updating documentation
* Updating documentation for single step jobs
* Update model.ipynb
* Update sklearn-mnist.ipynb
* Updating documentation for endpoints
* Paulsh/sdk dashboard (#985 )
* First pass
* Add trigger
* Fix sed commands
* Fix path to workbook
* Updates
* Updates
* Updates
* Skip some workspace creation commands and use a random workspace name
* Shorten workspace name to fit ARM requirements
* Test (#988 )
* Test
* Update
* Add config file
* Comment out config file lines
* Update (#989 )
* Workflows for testing sample notebooks (#997 )
* Adding workflows for single step jobs
* python file for workflow creation
* Adding versions to environments
* Adding workflows for assets
* Add files via upload
* Update tensorflow-mnist-distributed.ipynb
* Adding workflows for assets
* Adding workflows for endpoints
* Update tensorflow-mnist-distributed.ipynb
* Updating pipeline samples
* Adding workflows for pipelines with components
* Revert "Updating pipeline samples"
This reverts commit 468438ec4f#1004 )
* added e2e data scientist sample part I
Co-authored-by: amah <ma.mahmoudzadeh@gmail.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
* Janhavi13/model nb (#1008 )
* Fix model_uri in model notebook
* Update model upload from cloud path and run
* Update model upload from cloud path and remove run
* Move Tutorials to dedicated folder (#1034 )
* moved e2e ds tutorial to a dedicated folder
* removed from old location
Co-authored-by: amah <ma.mahmoudzadeh@gmail.com>
* Update dev-requirements.txt
* Hancwang/batch fix (#1011 )
* Batch deployment fix
* typo
* few more changes to invoke and deployment init
Co-authored-by: Banibrata De <banibrata@outlook.com>
* Distributed Deep Learning image end to end tutorial (#1023 )
* write e2e object dl notebook
* implement distributed pytorch
* implement mlflow model logging
* implement profiling
* March sdk preview (#1021 )
* Address CLI breaking changes
* Update nyc_taxi_data_regression.ipynb
* Updating pipelines
* Inline documentation updates
* Incorporating Sam's PR into this branch
* Installing new build
* Adding temp workflows for automated testing of notebooks
* Update model.ipynb
* Update model.ipynb
* Reverting model to sdk-preview version
* Update setup.sh
* Update setup.sh
* Adding a new build
* Removing all workflows from this branch
* Create sdk-assets-model-model.yml
* Update dev-requirements.txt
* Removing local path, adding @latest
* Updating to latest version of SDK/CLI and adding ipython_genutils
* Adding workflows for assets
* Update dev-requirements.txt
* Updating to Build # 58490278
* Adding tests for single step jobs
* Adding tests for endpoints
* Adding test workflows for pipelines
* Adding all test workflows
* Removing dataset notebook since we now have data
* Automated README.md
* Workflows for this branch
* Updating to latest build
* Update mnist-nonmlflow.ipynb
* Updating to latest build
* Adding description metadata to notebooks
* Improving README and Workflow generation
* Lochen/sdk march (#1041 )
* innit
* init
* Lochen/sdk march (#1042 )
* innit
* init
* merge folder
* fix pipeline sample: cifar-10, tf_mnist
* update workflows for pipeline
* Lochen/sdk march (#1044 )
* innit
* init
* merge folder
* change e2e registered, basic, nyc
* basic pipeline nyc
* add instructions.
* update nyc
* fix basic pipeline
* Upgrade to latest build
* Adjusting workflows and README for new pipeline samples
* Dummy commit for testing
* Update README.md
* Adding descriptions for pipeline notebooks
* Updating to latest build
* Removing datastores from testing and compute from scheduled runs
* Update lightgbm-iris-sweep.ipynb
* Updating to latest build
* Update README.md
* Fixing source directory for data and updating CLI/SDK build
* Updating to latest build
* Update README.md
* Adding clean up for endpoints and workspaces
* Update workspace.ipynb
* Updating to new build
* Update README.md
* Fixing issues with workspace notebook
* Update setup.sh
* Update setup.sh
* Updating to latest build
* Update README.md
* Updating to latest build and testing in eastus
* Fixing workflows
* Update readme.py
* Online endpoints sdk (#1059 )
* online endpoints sdk samples
* online endpoints sdk samples
* minor update
* updating readme.md
* adding back orginal sample
* minor
* update to sample folder
* minor changes
* Adding workflows and README for the new online endpoint examples
* Updating to the RC build
* Update README.md
* Fixing issues with MIR samples
* Standardizing kernelspec for all notebooks
* Formatting Cleaning python files with black
* Fixing readme.py
* Moving workflows from march-sdk-preview to sdk-preview
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
* Update README.md
* April Changes to SDK and CLI v2 (#1114 )
* Address CLI breaking changes
* Update nyc_taxi_data_regression.ipynb
* Updating pipelines
* Inline documentation updates
* Incorporating Sam's PR into this branch
* Installing new build
* Adding temp workflows for automated testing of notebooks
* Update model.ipynb
* Update model.ipynb
* Reverting model to sdk-preview version
* Update setup.sh
* Update setup.sh
* Adding a new build
* Removing all workflows from this branch
* Create sdk-assets-model-model.yml
* Update dev-requirements.txt
* Removing local path, adding @latest
* Updating to latest version of SDK/CLI and adding ipython_genutils
* Adding workflows for assets
* Update dev-requirements.txt
* Updating to Build # 58490278
* Adding tests for single step jobs
* Adding tests for endpoints
* Adding test workflows for pipelines
* Adding all test workflows
* Removing dataset notebook since we now have data
* Automated README.md
* Workflows for this branch
* Updating to latest build
* Update mnist-nonmlflow.ipynb
* Updating to latest build
* Adding description metadata to notebooks
* Improving README and Workflow generation
* Lochen/sdk march (#1041 )
* innit
* init
* Lochen/sdk march (#1042 )
* innit
* init
* merge folder
* fix pipeline sample: cifar-10, tf_mnist
* update workflows for pipeline
* Lochen/sdk march (#1044 )
* innit
* init
* merge folder
* change e2e registered, basic, nyc
* basic pipeline nyc
* add instructions.
* update nyc
* fix basic pipeline
* Upgrade to latest build
* Adjusting workflows and README for new pipeline samples
* Dummy commit for testing
* Update README.md
* Adding descriptions for pipeline notebooks
* Updating to latest build
* Removing datastores from testing and compute from scheduled runs
* Update lightgbm-iris-sweep.ipynb
* Updating to latest build
* Update README.md
* Fixing source directory for data and updating CLI/SDK build
* Updating to latest build
* Update README.md
* Adding clean up for endpoints and workspaces
* Update workspace.ipynb
* Updating to new build
* Update README.md
* Fixing issues with workspace notebook
* Update setup.sh
* Update setup.sh
* create new branch
* adapt sweep sample to april release
* adapt to command based job creation
* Updating workflows and readme to point to the april branch
* Updating CLI README to show April branch status
* Updating CLI workflows to trigger in april branch
* Changing workflows to trigger on april branch
* Builder fix up
* fix typo
* [April SDK] Pipeline builder function samples (#1117 )
* refine the samples
* re-structure pipeline sdk examples
* merge folder
* reorgnize pipeline samples
* refine
* add image_classification sample
* update dsl_component example
* add command function sample
* update
* refine command function sample
* update 3a_tf_mnist
* add sample 3e_image_classification
* update 3a_tf_mnist
* refine samples
* fix 3a_tf_mnist
* refine samples
* refine samples
* refine samples
* refine sample
* update sample 1a
* fix: sdk-3b_tf_mnist_sweep (#1060 )
fix entry func name in component definition
* refine sample
* refine samples
* add load auto load extention
* refine samples
* tf_minst_sweep: fix sample (#1062 )
* tf_minst_sweep: fix sample
* Update train.py
* Revert metric codes (#1061 )
* update mlflow
* still use azure.core
Co-authored-by: chenyin <chenyin@microsoft.com>
* refine samples
* sdk: 3c_cifar-10 sample bash encoding error (#1063 )
* add readme
* update readme
* print link that can be clicked (#1067 )
* refine samples
* rename the samples
* refine sample 1a
* Change DataInput/Output to ArtifactInput/Output
* refine sample 1a
* update naming for image classification
* refine 1e sample
* refine READMEs
* update README
* update ci workflow
* update ci SDK version
* add ci pipeline for 2d_image_classification_with_densenet
* update readme
* update readme
* add workflows
* merge march CLI and add mltable, sweep sample
* add march cli and sweep mltable support
* feat: refine sweep & cifar examples (#1078 )
* feat: refine sweep examples with dummy components
* fix: rollback python version
* fix: fix azure-ml version
* refactor: rename & use dataset
use a dataset instead of downloading inside a node
update learning objective & motivation
* fix: fix conda environment
* fix: add troubleshooting for gpu size not matched
* feat: demo more hyperparameters
* fix: update setup azure-ml version
* feat: add ci
* fix: update ci
* fix: fix ci
* fix: fix ci
* fix: fix ci
* fix: fix ci
* remove unused file
* fix: fix naming issue
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Refine dsl components samples (#1079 )
* Refine dsl components samples
* add CI workflow to Readme
* correct instance_count usage
* remove useless code
* Set resources for component
* update working dir
* change sample order to align with march version
* mltable
* include pipeline branch in CI
* update image classification workflow (#1085 )
* Refine common code snippet of samples (#1080 )
* change sample1
* move config location
* update CI
* refine
* refine wording
* change samples
* update snippet
* update notebook
* update
* update CI
* update 1e
* update samples
* refine sample
* refine inputs
* update
* add CI
* refine workflows
* fix dockerfile
* refine readme
* update 2d
* fix sample
* refine samples
* refine samples
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
* refine readme
* update
* update nyc to use mltable and mlflow model
* update doc link
* Updating to latest build
* Update README.md
* fix typo
* remove unneccesary imports
* Updating to latest build and testing in eastus
* Fixing workflows
* Update readme.py
* Online endpoints sdk (#1059 )
* online endpoints sdk samples
* online endpoints sdk samples
* minor update
* updating readme.md
* adding back orginal sample
* minor
* update to sample folder
* minor changes
* test workflow scope
* add job stream
* remove test workflow
* fix workflow cred
* update
* update sdk version
* update mnist sample
* Adding workflows and README for the new online endpoint examples
* Updating to the RC build
* update
* fix: flush (#1094 )
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* update build number
* update version
* change to rw_mount
* replace path with uri_folder
* fix: rename & correct the usage of setting (#1116 )
pipeline_job_setting.continue_on_step_failure
* Update sdk script gen pipeline jobs workflow (#1113 )
* update setup script
* update workflows
* fix checkout
* add script to write pipeline readme
* update readme
Co-authored-by: Han Wang <wanhan@microsoft.com>
* update compute (#1093 )
* Test v2 samples in magic workspace (#1092 )
* move test workspace
* change back CI
* add script to run all samples
* skip wait for completion
* fix script
* remove ps1
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Clwan/update nyc with mlflow_model type (#1118 )
* update notebook descriptions
* update readme
* update sdk nyc sample to adopt mlflow_model
* use sub folder as worksround for mlflow model
* refine samples
* update sweep and mlflow model
* refine samples
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: lochen <cloga0216@gmail.com>
* [Pipeline builder function] merge april branch (#1126 )
* add amlarc workflow (#968 )
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* committing onnx model generator script used with scriptrunconfig (#962 )
* updating onnx model generator with sriptrunconfig
* removing duplicacy with single onnx model script
* updating with minor changes
* updating with reviews
* minor changes
* model_name comparison changed
* minor changes
* Adding CodeGen related changes to bankmarketing notebook (#970 )
* added codegen related cells to bankmarketing notebook
* running black
* run black (#978 )
* Unpinning azureml-defaults to avoid installing itsdangerous coming indirectly from Flask (#981 )
* Unpinning azureml-defaults to avoid installing itsdangerous coming indirectly from Flask
* Pinning to the latest azureml version and touching files to trigger the builds
* Removing the update from docker
* updating mlflow deployment skus (#982 )
* Adding tags to certain cells in AutoML notebook (#964 )
* added metadata cells.
* fix inf env issue (#995 )
Co-authored-by: Yuan Zhuang <yuzhua@microsoft.com>
* Update dev-requirements.txt
* Create test.yml
* Delete test.yml
* Create sdk-assets-model-model.yml (#1001 )
* Create sdk-assets-model-model.yml
* Update sdk-assets-model-model.yml
* Update sdk-assets-model-model.yml
* Update and rename sdk-assets-model-model.yml to smoke-test-sdk-notebook-tests.yml
* Update and rename smoke-test-sdk-notebook-tests.yml to smoke-test-sdk-samplejob.yml
* Delete smoke-test-sdk-samplejob.yml
* Workflows for Testing SDKv2 Sample Notebooks
* Adding more workflows for sdk v2 notebook testing
* Adding workflows for SDKv2 Notebook Testing
* add language to kernelspec as asked by compute instance team (#1015 )
Co-authored-by: Lifeng Wei <lifengwei@microsoft.com>
* Add gpu diag/perf cli job (#980 )
* add gpu diag/perf cli job
* add path in docker instead of hardcoded in job
* update structure to have distinct folder for environment
* update environment to use inline
* finalize tests
* Upgrading the azure cli version
* Lifengwei/automl nlp notebooks (#1018 )
* multilabel example notebook with preprocessed data
* upgrade azureml_env_linux to 1.39 version
* make separate update_env file for new notebook
Co-authored-by: Lifeng Wei <lifengwei@microsoft.com>
* Lifengwei/automl nlp notebooks (#1026 )
* multiclass related files
* ner related files
* add language info to all notebooks
* make multilabel notebook trained with distributed training
Co-authored-by: Lifeeng Wei <lifengwei@microsoft.com>
* Fix a weird papermill error
Adding ipython_genutils to fix a weird papermill error
* update creds in cleanup script (#1029 )
* Distributed Deep Learning image end to end tutorial (#1023 )
* write e2e object dl notebook
* implement distributed pytorch
* implement mlflow model logging
* implement profiling
* add get_output, remove extra parameters, add comments for primary_metric (#1035 )
* add get_output, remove extra parameters, add comments for primary_metric
* update experiment names
* fix a typo
* typo
Co-authored-by: Lifeeng Wei <lifengwei@microsoft.com>
* Set idle_seconds_defore_scaledown (#1039 )
* Set idle_seconds_defore_scaledown
* Applied black, which removed some spaces
* Fixed AutoML notebooks pandas -1 (#1038 )
* update tfserving byoc sample (#983 )
* update tfserving byoc sample
* minor update
* anon byoc env (#1051 )
* Automl notebook fixes (#1046 )
* Jeffshep/fixcsrmatrixissue (#1072 )
* Fix error with csr_matrix is not JSON serializable
* Applied black
* support custom ennvironment packages by environemnt.yml (#1005 )
* update version of black to fix smoke test (#1088 )
* update
* remove workflow
* Lifengwei/update nlp notebooks (#1090 )
* remove some packages
* add back a package
* update notebooks, mainly change multilabel data file
* data file
* fix typos, codes in notebooks
* update experiment name
* fix typo, remove comments
Co-authored-by: Lifeeng Wei <lifengwei@microsoft.com>
* March sdk preview (#1021 )
* Address CLI breaking changes
* Update nyc_taxi_data_regression.ipynb
* Updating pipelines
* Inline documentation updates
* Incorporating Sam's PR into this branch
* Installing new build
* Adding temp workflows for automated testing of notebooks
* Update model.ipynb
* Update model.ipynb
* Reverting model to sdk-preview version
* Update setup.sh
* Update setup.sh
* Adding a new build
* Removing all workflows from this branch
* Create sdk-assets-model-model.yml
* Update dev-requirements.txt
* Removing local path, adding @latest
* Updating to latest version of SDK/CLI and adding ipython_genutils
* Adding workflows for assets
* Update dev-requirements.txt
* Updating to Build # 58490278
* Adding tests for single step jobs
* Adding tests for endpoints
* Adding test workflows for pipelines
* Adding all test workflows
* Removing dataset notebook since we now have data
* Automated README.md
* Workflows for this branch
* Updating to latest build
* Update mnist-nonmlflow.ipynb
* Updating to latest build
* Adding description metadata to notebooks
* Improving README and Workflow generation
* Lochen/sdk march (#1041 )
* innit
* init
* Lochen/sdk march (#1042 )
* innit
* init
* merge folder
* fix pipeline sample: cifar-10, tf_mnist
* update workflows for pipeline
* Lochen/sdk march (#1044 )
* innit
* init
* merge folder
* change e2e registered, basic, nyc
* basic pipeline nyc
* add instructions.
* update nyc
* fix basic pipeline
* Upgrade to latest build
* Adjusting workflows and README for new pipeline samples
* Dummy commit for testing
* Update README.md
* Adding descriptions for pipeline notebooks
* Updating to latest build
* Removing datastores from testing and compute from scheduled runs
* Update lightgbm-iris-sweep.ipynb
* Updating to latest build
* Update README.md
* Fixing source directory for data and updating CLI/SDK build
* Updating to latest build
* Update README.md
* Adding clean up for endpoints and workspaces
* Update workspace.ipynb
* Updating to new build
* Update README.md
* Fixing issues with workspace notebook
* Update setup.sh
* Update setup.sh
* Updating to latest build
* Update README.md
* Updating to latest build and testing in eastus
* Fixing workflows
* Update readme.py
* Online endpoints sdk (#1059 )
* online endpoints sdk samples
* online endpoints sdk samples
* minor update
* updating readme.md
* adding back orginal sample
* minor
* update to sample folder
* minor changes
* Adding workflows and README for the new online endpoint examples
* Updating to the RC build
* Update README.md
* Fixing issues with MIR samples
* Standardizing kernelspec for all notebooks
* Formatting Cleaning python files with black
* Fixing readme.py
* Moving workflows from march-sdk-preview to sdk-preview
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
* Modifying SDK workflows to sync up with March SDK changes
* Update README.md
* March CLI Breaking Changes (#1020 )
* remove component job, flatten code (#1002 )
* Update .gitignore
* Addressing CLI breaking changes
* adding uri_file to path
* adding uri_file type to paths
* adding uri_file and uri_folder
* Updating basic pipeline jobs
* Update hello-pipeline-default-artifacts.yml
* Lochen/cli march (#1009 )
* remove component job, flatten code
* add r component sample
* Online deployment changes
* Batch Deployment Changes
* Bug fixes to YAML
* Update REST API examples to Feb API (#1013 )
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
* minor fix
* resolve yaml error
* environment name and version as optional in custom containers (#1016 )
* minor updates (#1017 )
* Lochen/cli march (#1019 )
* remove component job, flatten code
* add r component sample
* update cli
* Updates to cifar example
* Adding daily build of CLI for testing
* Updating from dataset to data for create
* Rename folder from asset/datasets to assets/data
* Updating data assets to use data schema directive instead of dataset
* rename description to data asset
* Changing github workflows to use the new data assets
* removing dataset from md files
* Update README.md
* Removing local_path from examples
* Fixing python:latest bug with workaround
* Update prefix.md
* Lochen/cli march (#1027 )
* remove component job, flatten code
* add r component sample
* update cli
* update after test
* Updating environments to use latest version
* Updating environment to us latest
* Update setup.sh
* Test
* streamline register from a run (#1024 )
* streamline register from a run
* update to MLflow specific
* Lochen/cli march (#1031 )
* remove component job, flatten code
* add r component sample
* update cli
* update after test
* add default_compute, dataset
* Lochen/cli march (#1032 )
* remove component job, flatten code
* add r component sample
* update cli
* update after test
* add default_compute, dataset
* fix default compute, sweep
* fix default settings on pipeline
* Using the new build for CLI
* Update requirements.txt (#1033 )
* Updating to Build # 58490278
* Fixing filenames in the workflows.
* Fixing file name in train.sh
* add default value in pipeline settings.
* update default value
* change data asset
* update job link in readme
* Updating to latest CLI build
* Merge latest changes from main related to cli/jobs/single-step/gpu_perf/ command (#1037 )
* pick and choose changes from main
* align, add readme
* Update create-datasets.sh
* Updating to latest build
* Update README.md
* Update README.md
* Fixing failures in CLI
* Dummy commit to re-run tests
* fix to align with resource in test ws
* change 4d to use uri_folder
* upgrade dependencies, use fastparquet to fix pyarrow issue (#1043 )
* Upgrading to latest build
* merge datastore uri sample
* fix file script
* Users/anksing/mlflow model mir deploy (#1047 )
* Adding example for OBO flow using v2 cli
* Updating sklearn model for MIR deployment
* Updating lightgbm mlflow model for MIR deployment
* Removing obo folder
* Users/anksing/mlflow mir deployment (#1048 )
* Adding example for OBO flow using v2 cli
* Renaming model to avoid conflict with existing model with same name
* Removing obo folder
* Update batch-score.sh
Adding az version check.
* Updating model name to avoid conflict (#1049 )
* change cifar to directly use command
* add image classification cli example
* Updating to latest build
* Making setup files Unix friendly by removing \r
* Update endpoint.yml
remove the properties config -- we are not exposing in v2
* Update blue-deployment.yml
Removed properties field - not exposing user-configurable properties metadata in V2
* Dummy Commit to force tests
* update pipeline cifar to make it work under cpu-cluster
* update cifar sample to support cpu sku
* change 5c to cpu image
* clean up yaml, change cpu image for cifar
* Updating to latest build
* Update README.md
* formatting python files to ensure smoke test does not fail
* update workflow (#1064 )
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Wanhan/add image classification CI (#1065 )
* add image classification ci
* fix create component
Co-authored-by: Han Wang <wanhan@microsoft.com>
* fix image classification ci (#1066 )
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Update entry.py
* Update top latest build
* resolve merge conflict
* update to v2 dsl.component
* update train step (#1071 )
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Updating to latest build
* Update README.md
* Update README.md
* remove pyjokes in source file
* recover sample to test CI
* add component CRUD sample in assets
* Updating to new build
* Update setup.sh
* Changing readme temporarily to show march status
* Examples for workspace connections (#1073 )
* Create python-feed-pat.yml
* More examples
* Create README.md
* Update README.md
* update component name, add CI workflow
* update readme for asset component
* Update setup.sh
* Add yml for scheduled start stop of CI
Add yml for scheduled start stop of CI
* Delete instance-schedule.yml
* fix: fix attr overwrite command (#1074 )
* fix: fix attr overwrite command
add pipeline job test shell
* fix: fix attr overwrite command
* fix: create data with random version
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Updating to latest build
* Running tests on canary
* Trying to run test cases in master
* Reverting back to eastus
* Testing a new secret file
* Delete test-new-creds.yml
* Running tests on master
* Update job.yml (#1083 )
* Run tests in canary
* remove old dataset sample
* change process per node to 1 for cifar in pipeline
* add note to use multi-gpu cards
* Running latest build on eastus
* Update README.md
* fix sample issue to workaround output empty folder case
* Updating to latest build and testing in eastus
* Cleaning up all GitHub Workflows and the README file
* Update setup.sh
* Update dev-requirements.txt
* Fixing formatting issue so that black does not error
* Updating to the RC build
* change component name (#1075 )
* change component name
* rename
* add march-cli-preview in pipeline CI
* update registered case
* swith CI to master
* add back component creation
* update 3b to use rw_mount (#1097 )
* fix: fix run-pipeline-jobs.sh
* change upload mode to rw_mount
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Update README.md
* remove pipeline sample (#1102 )
* temp delete 2 samples
* remove reference links
Co-authored-by: Anthony Hu <anthu@microsoft.com>
* Fixing workflows and getting ready for merge to main
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Kayla Seager <1675997+kseager@users.noreply.github.com>
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
Co-authored-by: Paul Shealy <paulsh@microsoft.com>
Co-authored-by: Abraham Omorogbe <abrahamomor@gmail.com>
Co-authored-by: Ankit Singhal <30610298+singankit@users.noreply.github.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
Co-authored-by: Hugo Aponte <haponte@gmail.com>
Co-authored-by: Lisa Li (STC) <lisal@microsoft.com>
Co-authored-by: mx-iao <33701412+mx-iao@users.noreply.github.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Honglin Du <0mza987@gmail.com>
Co-authored-by: Roope Astala <rastala@users.noreply.github.com>
Co-authored-by: Swati Gharse <36742198+swatig007@users.noreply.github.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: eniac871 <supereniac@live.com>
* Updating workflows and readme to point to the april branch
* Updating CLI README to show April branch status
* Updating CLI workflows to trigger in april branch
* Changing workflows to trigger on april branch
* Builder fix up
* fix typo
* update workflows
Co-authored-by: Zeliang Tian <83852443+zetiaatgithub@users.noreply.github.com>
Co-authored-by: Vadthyavath Ram <7171558+vadthyavath@users.noreply.github.com>
Co-authored-by: Razvan Tanase <ratanase@microsoft.com>
Co-authored-by: Cody Peterson <54814569+lostmygithubaccount@users.noreply.github.com>
Co-authored-by: Harneet Virk <harnvir@microsoft.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
Co-authored-by: Yuan Zhuang <40252949+yuanzhuangyuanzhuang@users.noreply.github.com>
Co-authored-by: Yuan Zhuang <yuzhua@microsoft.com>
Co-authored-by: Bala P V <33712765+balapv@users.noreply.github.com>
Co-authored-by: Lifeng Wei <lfwei@ucdavis.edu>
Co-authored-by: Lifeng Wei <lifengwei@microsoft.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
Co-authored-by: Sethu Raman <rsethur@users.noreply.github.com>
Co-authored-by: jeff-shepherd <39775772+jeff-shepherd@users.noreply.github.com>
Co-authored-by: v-pbavanari <97779269+v-pbavanari@users.noreply.github.com>
Co-authored-by: Banibrata De <banibrata@outlook.com>
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Bala P V <balapv@microsoft.com>
Co-authored-by: Kayla Seager <1675997+kseager@users.noreply.github.com>
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
Co-authored-by: Paul Shealy <paulsh@microsoft.com>
Co-authored-by: Abraham Omorogbe <abrahamomor@gmail.com>
Co-authored-by: Ankit Singhal <30610298+singankit@users.noreply.github.com>
Co-authored-by: Hugo Aponte <haponte@gmail.com>
Co-authored-by: Lisa Li (STC) <lisal@microsoft.com>
Co-authored-by: mx-iao <33701412+mx-iao@users.noreply.github.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Honglin Du <0mza987@gmail.com>
Co-authored-by: Roope Astala <rastala@users.noreply.github.com>
Co-authored-by: Swati Gharse <36742198+swatig007@users.noreply.github.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Hanchi Wang <hancwang@microsoft.com>
Co-authored-by: Daniel Schneider <danielsc@microsoft.com>
* feat: sweep cli sdk examples update (#1125 )
* feat: use mlflow.autolog instead of Run.log
* fix: fix predict code
* fix: update parameters
* add image_classification_keras_minist_convnet (#1127 )
* add image classification kerad
* update
* update
* add readme
* add workflow and update README
Co-authored-by: Clement Wang <clwan@microsoft.com>
* fix: cifar-10/image_classification output mode
* update readme
* feat: descriptions for cli pipeline job examples (#1129 )
* feat: descriptions for cli pipeline job examples
* fix: remove hello git & update setup.sh
* fix readme
* Lochen/dataset asset (#1128 )
* write model to load then move to output folder
* add dataset dataasset sample
* add assest
* change component name to lower case
* refine samples
* update workflows
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: lochen <cloga0216@gmail.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
Co-authored-by: Ying Chen <2601502859@qq.com>
Co-authored-by: chenyin <chenyin@microsoft.com>
Co-authored-by: Korin <0mza987@gmail.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: Bala P V <balapv@microsoft.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
Co-authored-by: Zeliang Tian <83852443+zetiaatgithub@users.noreply.github.com>
Co-authored-by: Vadthyavath Ram <7171558+vadthyavath@users.noreply.github.com>
Co-authored-by: Razvan Tanase <ratanase@microsoft.com>
Co-authored-by: Cody Peterson <54814569+lostmygithubaccount@users.noreply.github.com>
Co-authored-by: Harneet Virk <harnvir@microsoft.com>
Co-authored-by: Yuan Zhuang <40252949+yuanzhuangyuanzhuang@users.noreply.github.com>
Co-authored-by: Yuan Zhuang <yuzhua@microsoft.com>
Co-authored-by: Bala P V <33712765+balapv@users.noreply.github.com>
Co-authored-by: Lifeng Wei <lfwei@ucdavis.edu>
Co-authored-by: Lifeng Wei <lifengwei@microsoft.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
Co-authored-by: Sethu Raman <rsethur@users.noreply.github.com>
Co-authored-by: jeff-shepherd <39775772+jeff-shepherd@users.noreply.github.com>
Co-authored-by: v-pbavanari <97779269+v-pbavanari@users.noreply.github.com>
Co-authored-by: Banibrata De <banibrata@outlook.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Kayla Seager <1675997+kseager@users.noreply.github.com>
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
Co-authored-by: Paul Shealy <paulsh@microsoft.com>
Co-authored-by: Abraham Omorogbe <abrahamomor@gmail.com>
Co-authored-by: Ankit Singhal <30610298+singankit@users.noreply.github.com>
Co-authored-by: Hugo Aponte <haponte@gmail.com>
Co-authored-by: Lisa Li (STC) <lisal@microsoft.com>
Co-authored-by: mx-iao <33701412+mx-iao@users.noreply.github.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Roope Astala <rastala@users.noreply.github.com>
Co-authored-by: Swati Gharse <36742198+swatig007@users.noreply.github.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Hanchi Wang <hancwang@microsoft.com>
Co-authored-by: Daniel Schneider <danielsc@microsoft.com>
* Pipeline April SDK Preview (#1130 )
* minor change
* update
* update
* make prepare step pass
* refine cli yaml: display_name
* fix cifar sample
* fix notebook
* fix: upgrade build version
* fix: fix 4d_dataset_input
* update keras sample
* ci: trigger ci on april-sdk-preview
* fix: do not always checkout april sdk preview
* fix: eval_mount is supported in new sdk/cli
* fix: merge main image classification
* fix: fix ci
* update dev build version
* fix: recover dataset related ci
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: Ying Chen <2601502859@qq.com>
* Pipeline April SDK Preview (#1137 )
* fix: reduce 6a/6b instance count
* fix: add asset creation to run-job-pipeline-all
* fix: remove repeating cli job status
* fix: use uri_folder instead of path
* fix ci : assets/component
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
* update SDK/CLI test for force_rerun (#1140 )
* update sdk code for rerun
* update cli test for rerun
* revert change in notebook
* add rerun in sdk ci pipeline
* revert
* add rerun in sdk ci pipeline
* update readme.py
* Pipeline SDK Preview Features (#1138 )
* add component archive/restore, customize output
* add name to path
* refine image classification sample
* clear outputs.
* Refine component notebook
* Rai pipeline sample (#1139 )
* Add RAI CLI sample pipeline
* add github workflow
* update pipeline yaml postfix
* add pre-setup steps
* update workflow to correct name
* add rai sample
* add rai sdk sample ci workflow
* change name to 2f
* update schema refer and workflow
* update readme and workflow name
* rename workflow to correct name
* minor update
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Honglin Du <0mza987@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
* update workflow & readme
* refine rai sample
* update
* update
* update readme
* update
Co-authored-by: lochen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Honglin Du <0mza987@gmail.com>
* AutoML preview PR (Python SDK and CLI examples) (#1089 )
* Baseline of AutoML Classification task notebook
* Updated to code and wheel 3-17-2022
* Updates for new wheel
* Forecasting baseline
* Fixed Forecasting notebook now working
* Deleted not used folders
* Added current Custom Environments
* Classifier notebook updated
* Forecasting notebook updated
* Baseline for Regression sample notebook
* WIP baseline AutoML for Images
* Updated to use URI-FOLDER data asset for Images
* Updates working Image-Classification with MLTables
* Baseline for AutoML NLP Text Classif multiclass
* Updated Images notebook to use Curated Env
* Updated NLP notebook with Curated Env
* Updates on NLP and minor IMG
* Minor updates, working notebooks
* Add notebooks for rest of image tasks (#1077 )
* Special PrP properties updated for all ntbooks
* Temporal WIP MLFlow
* MLFlow updates
* Baseline AutoML CLI example
* Updates SDK AutoML-MLFlow notebook
* Updates to hello AutoML CLI YAMLs
* Updates to CLI AutoML examples
* adding test github action for automl classification
* renamed the automl workflow
* Minor updates to CLI YAML examples
* Updates to MLFlow-AutoML notebook
* Updated _automl_internal_save_mlflow
* Updated mlflow-automl notebook with more MLTables
* delete old MLTable
* Minor update to YAML config
* CLI examples updates forecasting and classif
* adding automl workflows
* modifying notebook to wait for completion
* fixing workflows.
* changing notebooks to fetch create MLClient from config and wait for experiment to complete.
* update trials to 5 to overcome bug.
* updating to latest private wheel
* fixing classification & regression notebooks
* comment out set_limits for image notebooks
* fixing image object detection
* parameterizing automl notebooks with compute name
* fixing notebook whitespaces
* adding new line and remove obsolete action workflow
* fixing mltable path
* cleaning notebooks, running nb_black
* changing automl cli scripts
* adding AutoML CLI workflows.
* Porting AutoML samples into automl-preview. (#1101 )
* adding automl workflows
* modifying notebook to wait for completion
* fixing workflows.
* changing notebooks to fetch create MLClient from config and wait for experiment to complete.
* update trials to 5 to overcome bug.
* updating to latest private wheel
* fixing classification & regression notebooks
* comment out set_limits for image notebooks
* fixing image object detection
* parameterizing automl notebooks with compute name
* fixing notebook whitespaces
* adding new line and remove obsolete action workflow
* fixing mltable path
* cleaning notebooks, running nb_black
* changing automl cli scripts
* adding AutoML CLI workflows.
* adding parameter to instance segmentation notebook.
* Updates on getting MLFlow tracking URI
* Add yaml cli examples for image tasks (#1104 )
* revert to using macros for workspace setup in Bankmarketing example.
* Fix paths in jsonl files for image notebooks (#1105 )
* Updates for MLFlow URI and other
* Fix json issue
* Subscription, workspace.. aligned..
* merged from parent branch.
* removing non-automl actions.
* Forecasting updates
* Minor updates on classifc notebook
* Update on "data" folder for Images notebook
* Workspace IDS for macros fixed
* data folder update for regression notebook
* Clean-up output images multi-class
* Several fixes to image-classif-multi-label
* fixing the resource group name macro.
* Minor clean up in image classif notebooks
* updating the automl actions to refer automl-preview branch
* branch name changes in actions.
* Object Detection updated to use "./data"
* Updates on "./data" to image segmentation
* Clean up image segmentation
* Updates working Image classif multi-label and segm
* Commented setLimits to Image multi-label
* update encoding to utf8 (#1107 )
Co-authored-by: Miseon Park <mipark@microsoft.com>
* remove set_limits from image object detection notebook.
* add NLP cli examples (#1108 )
Co-authored-by: Miseon Park <mipark@microsoft.com>
* modify automl CLI workflows
* adding sdk-preview to the list of branches to monitor for CLI actions.
* ensuring all AutoML CLI workflows get triggered.
* fixing CLI setup script
* fixing additional properties for classification & forecasting.
* CLI Regression example baseline
* pinning to latest CLI & SDK & updating actions for CLI.
* Updated Regression CLI YAML working
* attempting to fix the bankmarketing cli example.
* fix setup.sh for sdk.
* adding regression CLI action.
* remove drop_columns from featurization
* update automl cli yamls to camel case and latest schemas
* removing CLI actions from running on PRs on automl-preview branch
* Fix set_limits() for image tasks. Fix image classification yaml (#1110 )
* Fix set_limits() for image tasks. Fix image classification yaml
* CLI example fixes for schema changes
* Don't specify a version while uploading data folder
* Add scripts to prepare data for image cli jobs (#1120 )
* AutoML April preview - SDK + CLI (#1121 )
* Address CLI breaking changes
* Update nyc_taxi_data_regression.ipynb
* Updating pipelines
* Inline documentation updates
* Incorporating Sam's PR into this branch
* Installing new build
* Adding temp workflows for automated testing of notebooks
* Update model.ipynb
* Update model.ipynb
* Reverting model to sdk-preview version
* Update setup.sh
* Update setup.sh
* Adding a new build
* Removing all workflows from this branch
* Create sdk-assets-model-model.yml
* Update dev-requirements.txt
* Removing local path, adding @latest
* Updating to latest version of SDK/CLI and adding ipython_genutils
* Adding workflows for assets
* Update dev-requirements.txt
* Updating to Build # 58490278
* Adding tests for single step jobs
* Adding tests for endpoints
* Adding test workflows for pipelines
* Adding all test workflows
* Removing dataset notebook since we now have data
* Automated README.md
* Workflows for this branch
* Updating to latest build
* Update mnist-nonmlflow.ipynb
* Updating to latest build
* Adding description metadata to notebooks
* Improving README and Workflow generation
* Lochen/sdk march (#1041 )
* innit
* init
* Lochen/sdk march (#1042 )
* innit
* init
* merge folder
* fix pipeline sample: cifar-10, tf_mnist
* update workflows for pipeline
* Lochen/sdk march (#1044 )
* innit
* init
* merge folder
* change e2e registered, basic, nyc
* basic pipeline nyc
* add instructions.
* update nyc
* fix basic pipeline
* Upgrade to latest build
* Adjusting workflows and README for new pipeline samples
* Dummy commit for testing
* Update README.md
* Adding descriptions for pipeline notebooks
* Updating to latest build
* Removing datastores from testing and compute from scheduled runs
* Update lightgbm-iris-sweep.ipynb
* Updating to latest build
* Update README.md
* Fixing source directory for data and updating CLI/SDK build
* Updating to latest build
* Update README.md
* Adding clean up for endpoints and workspaces
* Update workspace.ipynb
* Updating to new build
* Update README.md
* Fixing issues with workspace notebook
* Update setup.sh
* Update setup.sh
* create new branch
* adapt sweep sample to april release
* adapt to command based job creation
* Updating workflows and readme to point to the april branch
* Updating CLI README to show April branch status
* Updating CLI workflows to trigger in april branch
* Changing workflows to trigger on april branch
* Builder fix up
* fix typo
Co-authored-by: Bala P V <balapv@microsoft.com>
Co-authored-by: Bala P V <33712765+balapv@users.noreply.github.com>
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Daniel Schneider <schneider@volkskammer.org>
Co-authored-by: Hanchi Wang <hancwang@microsoft.com>
Co-authored-by: Daniel Schneider <danielsc@microsoft.com>
Co-authored-by: Gaurav Singh <gasi@microsoft.com>
* cleanup
* renaming the folder for automl CLI jobs
* removing AutoML CLI workflows.
* regenerated all the CLI workflows
* regenerated SDK workflows
* Minor changes for Workspace prop showup
* Minor changes to AutoML-MLFlow notebook
* Undo non-automl files
* Add jsonl files pointing to GH actions workspace for automl images (#1136 )
* Added MLFlow model local test
* Removed MLFlow inference to point to next notebook
* Update cli-jobs-automl-standalone-jobs-cli-automl-classification-task-bankmarketing-cli-automl-classification-task-bankmarketing.yml
testing
* Update cli-jobs-automl-standalone-jobs-cli-automl-classification-task-bankmarketing-cli-automl-classification-task-bankmarketing.yml
revert
* ml_flow.set_experiment() not needed, removed
* Forecasting updates for MLFlow
* regenerate automl workflows for april branch merge test
* Update cli-automl-classification-task-bankmarketing.yml
* Update cli-automl-classification-task-bankmarketing.yml
* remove bankmarketing mlflow.yml from workflows
* update to specs
Co-authored-by: PhaniShekhar <30535191+PhaniShekhar@users.noreply.github.com>
Co-authored-by: Razvan Tanase <ratanase@microsoft.com>
Co-authored-by: Miseon <msp8955@gmail.com>
Co-authored-by: Miseon Park <mipark@microsoft.com>
Co-authored-by: Gaurav Singh <svaruag@outlook.com>
Co-authored-by: Bala P V <balapv@microsoft.com>
Co-authored-by: Bala P V <33712765+balapv@users.noreply.github.com>
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Daniel Schneider <schneider@volkskammer.org>
Co-authored-by: Hanchi Wang <hancwang@microsoft.com>
Co-authored-by: Daniel Schneider <danielsc@microsoft.com>
Co-authored-by: Gaurav Singh <gasi@microsoft.com>
Co-authored-by: Srujan Saggam <41802116+srsaggam@users.noreply.github.com>
* Pipeline SDK/CLI April Features (#1144 )
* update add missing pipeline workflows
* update sdk workflows & readme
* add force rerun
* move dsl files to src folder (#1145 )
* update data format
* remove invalid workflow
* update RAI samples: 1, update register data name to align with the on… (#1146 )
* update RAI samples: 1, update register data name to align with the one in RAI repo. 2, SDK sample reuse the component from CLI one. 3, SDK use a more complext pipeline. 4, update readme
* remove some unuse comment
Co-authored-by: Anthony Hu <anthu@microsoft.com>
* fix: fix sweep cli example
* parallel run notebooks
* fix encoding
* add dry run
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Korin <0mza987@gmail.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Added .TXT files on how to run CLI AutoML (#1150 )
* Added .TXT on how to run CLI AutoML
* Comment added about GPU cluster
* blocked_algos commented due to bug
* Removed dev temporal internal properties
* Prep data explanation sentence fixed
* Lochen/april release 0413 (#1147 )
* update add missing pipeline workflows
* update sdk workflows & readme
* add force rerun
* move dsl files to src folder (#1145 )
* update data format
* remove invalid workflow
* update RAI samples: 1, update register data name to align with the on… (#1146 )
* update RAI samples: 1, update register data name to align with the one in RAI repo. 2, SDK sample reuse the component from CLI one. 3, SDK use a more complext pipeline. 4, update readme
* remove some unuse comment
Co-authored-by: Anthony Hu <anthu@microsoft.com>
* fix: fix sweep cli example
* parallel run notebooks
* fix encoding
* add dry run
* data asset dataset, remove pipeline level input
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Korin <0mza987@gmail.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Updates to workflows and build Numbers
* Add files via upload
* Add files via upload
* Add files via upload
* Updating workflows
* fix links (#1156 )
* Moving to DeafultAzureCredential instead of Browser Auth. Removing branch check-out.
* Adding AutoML CLI tests, removing apr branch checkout
* Moving from commandjob to command
* Fixing pipeline credentials
* Update AutoML Job schema urls (#1160 )
* Using requirements from SDK folder. Adding pandas to requirements
* remove automl-preview reference (#1143 )
* remove mlflow WIP notebooks from github actions
* remove pandas reference
* remove pandas after merge
Co-authored-by: Miseon Park <mipark@microsoft.com>
* Pipeline April SDK/CLI preview samples refine (#1157 )
* merge
* update sample to make compute config lines fewer
* make configure credential simpler
* update output setting of nyc_tax_data_regression
* refine rai samples
* update sdk ci workflow
* update
* fix rai
* use newer dev build before nightly build issue resolved
* update workflow
* Rename sample node name to align with pipeline node restriction (#1158 )
* fix name
* fix name
* fix sample
* fix
* fix
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Update pipeline sdk samples to use azure.ml.Input/Output
* update setup.sh
* update
* refine
* feat: update run-job-pipeline-all auto-gen
* fix: fix image classification example
* update setup.sh
* update Input
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Updating to latest build
* Update prefix.md
* Update prefix.md
* Update import paths for automl entities in image notebooks (#1164 )
* Update cli-automl-classification-task-bankmarketing.yml
* Update cli-automl-forecasting-task-energy-demand.yml
* Update cli-automl-forecasting-task-energy-demand.yml
* Pipeline April CLIL/SDK Feature refine (#1161 )
* remove mlflow workaround
* remove mlflow molde workaround
* update sdk nyc sample
* resolve components ci failure
* add configuration notebook
* simplify pipeline_with_components_from_yaml notebook
* fix hello-pipeline-customize-output-folder.yml
* simplify samples
* refine
* update workflow and readme
* update
* add display name for hello pipelines
* update
Co-authored-by: lochen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Updating to latest build
* Fixing links in docs
* Delete online-inferencing-sample.ipynb
* Update automl-classification-task-bankmarketing-mlflow.ipynb
* Converting all links to configuration.ipynb file into a relative link
* Fixing links to README file
* Fixing links to other files and folders
* Hancwang/april (#1168 )
* Clean up iris
* demo notebook
* notebook used in demo
* namespace update
* more import fix
* remove studio_url
* more conflicts
* revert readme
* Fix build error
* Update hello-automl-job-basic.yml
* Pipeline April SDK Feature samples refine (#1175 )
* remove refrence of azure.ml._constants
* revert notebook rename
* update
* update to latest master build
* fix run all
* fix SDK sample link
* update pipeline sdk 2e sample (#1176 )
* update pipeline sdk 2e sample
* update compute
* fix credential part
* adapt to latest version sdk
* update
* update
* update
* update
* update conda aml sdk version
* update according to comments
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Blanca Li <39058063+likebupt@users.noreply.github.com>
* Updating nightly build
* Update namespace from azure.ml._constants to azure.ml.constants (#1174 )
Co-authored-by: Clement Wang <clwan@microsoft.com>
* Updating description for preview
* endpoints named by sample (#1180 )
* add custom featurization to tests (#1172 )
Co-authored-by: Miseon Park <mipark@microsoft.com>
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Daniel Schneider <schneider@volkskammer.org>
Co-authored-by: Hanchi Wang <hancwang@microsoft.com>
Co-authored-by: Daniel Schneider <danielsc@microsoft.com>
Co-authored-by: Clement Wang <47586720+wangchao1230@users.noreply.github.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
Co-authored-by: Ying Chen <2601502859@qq.com>
Co-authored-by: chenyin <chenyin@microsoft.com>
Co-authored-by: Korin <0mza987@gmail.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
Co-authored-by: Zeliang Tian <83852443+zetiaatgithub@users.noreply.github.com>
Co-authored-by: Vadthyavath Ram <7171558+vadthyavath@users.noreply.github.com>
Co-authored-by: Razvan Tanase <ratanase@microsoft.com>
Co-authored-by: Cody Peterson <54814569+lostmygithubaccount@users.noreply.github.com>
Co-authored-by: Harneet Virk <harnvir@microsoft.com>
Co-authored-by: Yuan Zhuang <40252949+yuanzhuangyuanzhuang@users.noreply.github.com>
Co-authored-by: Yuan Zhuang <yuzhua@microsoft.com>
Co-authored-by: Lifeng Wei <lfwei@ucdavis.edu>
Co-authored-by: Lifeng Wei <lifengwei@microsoft.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
Co-authored-by: Sethu Raman <rsethur@users.noreply.github.com>
Co-authored-by: jeff-shepherd <39775772+jeff-shepherd@users.noreply.github.com>
Co-authored-by: v-pbavanari <97779269+v-pbavanari@users.noreply.github.com>
Co-authored-by: Banibrata De <banibrata@outlook.com>
Co-authored-by: Kayla Seager <1675997+kseager@users.noreply.github.com>
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
Co-authored-by: Paul Shealy <paulsh@microsoft.com>
Co-authored-by: Abraham Omorogbe <abrahamomor@gmail.com>
Co-authored-by: Ankit Singhal <30610298+singankit@users.noreply.github.com>
Co-authored-by: Hugo Aponte <haponte@gmail.com>
Co-authored-by: Lisa Li (STC) <lisal@microsoft.com>
Co-authored-by: mx-iao <33701412+mx-iao@users.noreply.github.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Roope Astala <rastala@users.noreply.github.com>
Co-authored-by: Swati Gharse <36742198+swatig007@users.noreply.github.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Cesar De la Torre <cesardl@microsoft.com>
Co-authored-by: PhaniShekhar <30535191+PhaniShekhar@users.noreply.github.com>
Co-authored-by: Miseon <msp8955@gmail.com>
Co-authored-by: Miseon Park <mipark@microsoft.com>
Co-authored-by: Gaurav Singh <svaruag@outlook.com>
Co-authored-by: Gaurav Singh <gasi@microsoft.com>
Co-authored-by: Srujan Saggam <41802116+srsaggam@users.noreply.github.com>
Co-authored-by: Hanchi Wang <luigiking307@gmail.com>
Co-authored-by: Blanca Li <39058063+likebupt@users.noreply.github.com>
* Adjusting workflows and readme to point to correct branch
* SDK preview - pipeline samples refine (#1183 )
* remove unnecesarry files
* update imports in pipeline samples
* remove is_deterministic: False
* update 2e sample (#1184 )
* refine wording
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Blanca Li <39058063+likebupt@users.noreply.github.com>
* Reformatting python files with black
* Formatting notebooks with black
* Revise e2e vision notebook to use commands and sdk april release (#1167 )
* remove coco dependencies in component
* remove spec
* revise requirements
* revise notebook to use commands only
* adapt to april release
* restore notebooks
* remove pipeline snapshot
* update the e2e-ds-experience nb to april release (#1185 )
Co-authored-by: amah <ma.mahmoudzadeh@gmail.com>
* Adding metadata to refer from docs
* Pipeline SDK Preview Samples Refine (#1187 )
* update component notebook description
* remove enum before issue resolved
* refine component notebook
* add description
* refine 1e sample
* revert enum
* fix cli ci
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* add aml arc inferencing notebook (#1194 )
* add notebooks for amlarc inferencing
* add workflow for amlarc inferencing notebook
* fix: specify gpu related environment version (#1199 )
* Update sdk-endpoints-online-amlarc-online-endpoints-safe-rollout.yml (#1200 )
fix ipynb file name issue
* update 2e sample (#1201 )
* update pipeline job sdk 2e sample
* update description of the pipeline
* comment invoke method for aks policy (#1203 )
comment invoke out as cluster under dev subscription can't be accessed from public internet.
* Update online-endpoints-simple-deployment.ipynb
* add k8s prefix for amlarc online endpoint name (#1212 )
* Update online-endpoints-safe-rollout.ipynb
add k8s prefix for amlarc endpoint name
* feat: use mlflow in nyc-taxi-data-regression (#1208 )
* feat: use mlflow to save model in nyc_taxi_data_regression
* feat: optimize run-job-pipeline related shells
* Uncomment allowed and blocked models for tabular tasks (#1196 )
* Uncomment allowed and blocked models for tabular tasks
* fix forecasting
Co-authored-by: Ayush Mishra <ayushmishra@microsoft.com>
* add metadata to sdk notebook examples (#1217 )
Co-authored-by: Miseon Park <mipark@microsoft.com>
* update to May API version (#1223 )
* Update main.py
* rename amlarc to kubernetes (#1219 )
* rename amlarc to kubernetes
Co-authored-by: Xue Wei <xuewe@microsoft.com>
* enable download for rai pipeline (#1221 )
* enable download for rai pipeline
* remove empty cell
* refine
* Add metadata tags for automl images (#1242 )
* Add metadata tags for automl images
* Separate cells
* Minor metadat
* Update kubernetes-online-endpoints-safe-rollout.ipynb (#1251 )
* update pipeline sdk 2e sample (#1260 )
* added k8s related workflows result (#1266 )
* add cell name for doc reference (#1267 )
* Fixes to e2e dl distributed pytorch notebook from bugbash (#1255 )
* implement details
* add compute test
* update 2e sample (#1265 )
* update 2e sample
* update
* update
* Changing namespace from azure.ml to azure.ai.ml
* Changing JobInput to Input in AutoML
* Removing JobInput and azure.ml from files
* Bug FIxes
* Delete old endpoint sample test
* Adding metadata for cells - needed for docs
* [Pipeline] Fix azure-ml -> azure-ai-ml (#1282 )
* update azure-ml version in conda.yml
* update @dsl.pipeline -> @pipeline
* update build version
Co-authored-by: Clement Wang <clwan@microsoft.com>
* update 2e sample file and related reference (#1289 )
* Updating to new builds
* Update lightgbm-iris-sweep.ipynb
* sync workflow scripts from main to sdk-preview branch (#1300 )
* Update setup.sh
* Update setup.sh
* Update setup.sh
* Update setup.sh
* update train.py. r (#1299 )
emove mlflow related comment to avoid confuse audience.
* Sync notebook with doc (#1291 )
* for docs tutorial
* Updating tutorial to match docs
* run black on this
* remove dataset
* rename notebook
* add alert for cpu-name
* add note, fix intro
* Updating daily build
* Update load_component and switch other YAML loading to use load_x() (#1314 )
* Update load_component and switch other YAML loading to use load_x()
* Update to the latest private wheel
* Remove urireference
* sync scripts (#1310 )
* managed vnet for managed online endpoints (#1303 )
* vnet sample
* update
* updates
* updates
* updates
* update dir
* scripts
* add model
* updates
* update scripts
* cleanup
* cleanup
* cleanup
* parameterize the branch for testing
* test with pr branch
* test sample branch
* test branch
* test branch
* fix the branch name
* automated vnet test: fix the branch name
* validate scoring
* fix script
* fix script
* update the setup instructions
* update comments
* fix formatting
* additional comments
* fix review comments
* update scoring test scripts
* update keyvault naming
* update build script
* reorganize directories
* update path
* update pr check
* update
* remove uai to seperate file
* mlflow vnet sample
* mlflow vnet sample
* mlflow vnet example
* mlflow vnet example
* docs comments
* mirror traffic
* update to new syntax
* test
* testing with debug
* debug
* revert setup.sh changes
* debug
* latest cli
* updated cli
* update cli version
* adding local deployment
* local deployment
* test
* local deployment
* new cli build
* new cli version
* latest cli version
* testing fix for apt update error
* fix update error
* remove unused packages
* newer version of cli
* remove private preview flags
* custom cli version for testing from vm
* Users/mipark/update automl standalone examples (#1316 )
* cli updates
* sdk updates
* add model deploy and local inference examples
* update readme
Co-authored-by: Miseon Park <mipark@microsoft.com>
* Updating to latest builds
* Add notebooks for local model inference. Model deployment (#1317 )
* remove data settings from hello automl notebook (#1319 )
* cli updates
* sdk updates
* add model deploy and local inference examples
* update readme
* update automl basic example
* update workflows
Co-authored-by: Miseon Park <mipark@microsoft.com>
* Update setup.sh
* Updating to latest build
* Removing rw_mount from samples
* pick non private preview change (#1326 )
* pick non private preview change
* update samples
* simplify @pipeline usage
* remove description and name for pipeline, add doc string instead
* fix: fix readme.md for rai sample
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Honglin Du <0mza987@gmail.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
* update mldesigner install (#1333 )
Co-authored-by: Han Wang <wanhan@microsoft.com>
* fix automl examples (#1331 )
* fix examples
* Correct timeout_minutes in Object detection notebook
* Correct import paths in prepare_data.py files
Co-authored-by: Miseon Park <mipark@microsoft.com>
Co-authored-by: Phani Shekhar Mantripragada <phmantri@microsoft.com>
* updating rest api version (#1329 )
Co-authored-by: Raghu Ramaswamy <raghura@microsoft.com>
* Reformat notebooks, update latest build, improve cleanup and README generation
* Reformatting notebooks
* Reformatting some more notebooks
* Updating README and workflows for SDK v2
* Updating CLI workflows
* Updating to use defaultAzure Creds
* updated the e2e-ds-notebook to may release (#1325 )
* updated the e2e-ds-notebook to may release
* cleaned notebook cell outputs
* fixed some terms to the new ones
Co-authored-by: amah <ma.mahmoudzadeh@gmail.com>
* Changes to fix online endpoint issues
* Fixing Online Deployment scripts
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Dennis Eikelenboom <92387505+deeikele@users.noreply.github.com>
Co-authored-by: Tracy Chen <47156502+tracychms@users.noreply.github.com>
Co-authored-by: Hanchi Wang <luigiking307@gmail.com>
Co-authored-by: Cody <54814569+lostmygithubaccount@users.noreply.github.com>
Co-authored-by: Paul Shealy <paulsh@microsoft.com>
Co-authored-by: abeomor <abrahaomor@gmail.com>
Co-authored-by: amah <mamahmoudzadeh@gmail.com>
Co-authored-by: amah <ma.mahmoudzadeh@gmail.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
Co-authored-by: Janhavi Mahajan <janhavi13@users.noreply.github.com>
Co-authored-by: Banibrata De <banibrata@outlook.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
Co-authored-by: Daniel Schneider <schneider@volkskammer.org>
Co-authored-by: Hanchi Wang <hancwang@microsoft.com>
Co-authored-by: Daniel Schneider <danielsc@microsoft.com>
Co-authored-by: Clement Wang <47586720+wangchao1230@users.noreply.github.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
Co-authored-by: Ying Chen <2601502859@qq.com>
Co-authored-by: chenyin <chenyin@microsoft.com>
Co-authored-by: Korin <0mza987@gmail.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: Zeliang Tian <83852443+zetiaatgithub@users.noreply.github.com>
Co-authored-by: Vadthyavath Ram <7171558+vadthyavath@users.noreply.github.com>
Co-authored-by: Razvan Tanase <ratanase@microsoft.com>
Co-authored-by: Harneet Virk <harnvir@microsoft.com>
Co-authored-by: Yuan Zhuang <40252949+yuanzhuangyuanzhuang@users.noreply.github.com>
Co-authored-by: Yuan Zhuang <yuzhua@microsoft.com>
Co-authored-by: Lifeng Wei <lfwei@ucdavis.edu>
Co-authored-by: Lifeng Wei <lifengwei@microsoft.com>
Co-authored-by: Sethu Raman <rsethur@users.noreply.github.com>
Co-authored-by: jeff-shepherd <39775772+jeff-shepherd@users.noreply.github.com>
Co-authored-by: v-pbavanari <97779269+v-pbavanari@users.noreply.github.com>
Co-authored-by: Kayla Seager <1675997+kseager@users.noreply.github.com>
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
Co-authored-by: Abraham Omorogbe <abrahamomor@gmail.com>
Co-authored-by: Ankit Singhal <30610298+singankit@users.noreply.github.com>
Co-authored-by: Hugo Aponte <haponte@gmail.com>
Co-authored-by: Lisa Li (STC) <lisal@microsoft.com>
Co-authored-by: mx-iao <33701412+mx-iao@users.noreply.github.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Roope Astala <rastala@users.noreply.github.com>
Co-authored-by: Swati Gharse <36742198+swatig007@users.noreply.github.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Cesar De la Torre <cesardl@microsoft.com>
Co-authored-by: PhaniShekhar <30535191+PhaniShekhar@users.noreply.github.com>
Co-authored-by: Miseon <msp8955@gmail.com>
Co-authored-by: Miseon Park <mipark@microsoft.com>
Co-authored-by: Gaurav Singh <svaruag@outlook.com>
Co-authored-by: Gaurav Singh <gasi@microsoft.com>
Co-authored-by: Srujan Saggam <41802116+srsaggam@users.noreply.github.com>
Co-authored-by: Blanca Li <39058063+likebupt@users.noreply.github.com>
Co-authored-by: snowei <snowei1988@gmail.com>
Co-authored-by: Ayush Mishra <61145377+novaturient95@users.noreply.github.com>
Co-authored-by: Ayush Mishra <ayushmishra@microsoft.com>
Co-authored-by: Xiao Zhang <zhanxia@microsoft.com>
Co-authored-by: Sheri Gilley <sgilley@microsoft.com>
Co-authored-by: Phani Shekhar Mantripragada <phmantri@microsoft.com>
Co-authored-by: Raghu R <13340619+raghutillu@users.noreply.github.com>
Co-authored-by: Raghu Ramaswamy <raghura@microsoft.com>
2022-05-23 19:13:49 -07:00
f50f94756a
March CLI Breaking Changes ( #1020 )
...
* remove component job, flatten code (#1002 )
* Update .gitignore
* Addressing CLI breaking changes
* adding uri_file to path
* adding uri_file type to paths
* adding uri_file and uri_folder
* Updating basic pipeline jobs
* Update hello-pipeline-default-artifacts.yml
* Lochen/cli march (#1009 )
* remove component job, flatten code
* add r component sample
* Online deployment changes
* Batch Deployment Changes
* Bug fixes to YAML
* Update REST API examples to Feb API (#1013 )
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
* minor fix
* resolve yaml error
* environment name and version as optional in custom containers (#1016 )
* minor updates (#1017 )
* Lochen/cli march (#1019 )
* remove component job, flatten code
* add r component sample
* update cli
* Updates to cifar example
* Adding daily build of CLI for testing
* Updating from dataset to data for create
* Rename folder from asset/datasets to assets/data
* Updating data assets to use data schema directive instead of dataset
* rename description to data asset
* Changing github workflows to use the new data assets
* removing dataset from md files
* Update README.md
* Removing local_path from examples
* Fixing python:latest bug with workaround
* Update prefix.md
* Lochen/cli march (#1027 )
* remove component job, flatten code
* add r component sample
* update cli
* update after test
* Updating environments to use latest version
* Updating environment to us latest
* Update setup.sh
* Test
* streamline register from a run (#1024 )
* streamline register from a run
* update to MLflow specific
* Lochen/cli march (#1031 )
* remove component job, flatten code
* add r component sample
* update cli
* update after test
* add default_compute, dataset
* Lochen/cli march (#1032 )
* remove component job, flatten code
* add r component sample
* update cli
* update after test
* add default_compute, dataset
* fix default compute, sweep
* fix default settings on pipeline
* Using the new build for CLI
* Update requirements.txt (#1033 )
* Updating to Build # 58490278
* Fixing filenames in the workflows.
* Fixing file name in train.sh
* add default value in pipeline settings.
* update default value
* change data asset
* update job link in readme
* Updating to latest CLI build
* Merge latest changes from main related to cli/jobs/single-step/gpu_perf/ command (#1037 )
* pick and choose changes from main
* align, add readme
* Update create-datasets.sh
* Updating to latest build
* Update README.md
* Update README.md
* Fixing failures in CLI
* Dummy commit to re-run tests
* fix to align with resource in test ws
* change 4d to use uri_folder
* upgrade dependencies, use fastparquet to fix pyarrow issue (#1043 )
* Upgrading to latest build
* merge datastore uri sample
* fix file script
* Users/anksing/mlflow model mir deploy (#1047 )
* Adding example for OBO flow using v2 cli
* Updating sklearn model for MIR deployment
* Updating lightgbm mlflow model for MIR deployment
* Removing obo folder
* Users/anksing/mlflow mir deployment (#1048 )
* Adding example for OBO flow using v2 cli
* Renaming model to avoid conflict with existing model with same name
* Removing obo folder
* Update batch-score.sh
Adding az version check.
* Updating model name to avoid conflict (#1049 )
* change cifar to directly use command
* add image classification cli example
* Updating to latest build
* Making setup files Unix friendly by removing \r
* Update endpoint.yml
remove the properties config -- we are not exposing in v2
* Update blue-deployment.yml
Removed properties field - not exposing user-configurable properties metadata in V2
* Dummy Commit to force tests
* update pipeline cifar to make it work under cpu-cluster
* update cifar sample to support cpu sku
* change 5c to cpu image
* clean up yaml, change cpu image for cifar
* Updating to latest build
* Update README.md
* formatting python files to ensure smoke test does not fail
* update workflow (#1064 )
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Wanhan/add image classification CI (#1065 )
* add image classification ci
* fix create component
Co-authored-by: Han Wang <wanhan@microsoft.com>
* fix image classification ci (#1066 )
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Update entry.py
* Update top latest build
* resolve merge conflict
* update to v2 dsl.component
* update train step (#1071 )
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Updating to latest build
* Update README.md
* Update README.md
* remove pyjokes in source file
* recover sample to test CI
* add component CRUD sample in assets
* Updating to new build
* Update setup.sh
* Changing readme temporarily to show march status
* Examples for workspace connections (#1073 )
* Create python-feed-pat.yml
* More examples
* Create README.md
* Update README.md
* update component name, add CI workflow
* update readme for asset component
* Update setup.sh
* Add yml for scheduled start stop of CI
Add yml for scheduled start stop of CI
* Delete instance-schedule.yml
* fix: fix attr overwrite command (#1074 )
* fix: fix attr overwrite command
add pipeline job test shell
* fix: fix attr overwrite command
* fix: create data with random version
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Updating to latest build
* Running tests on canary
* Trying to run test cases in master
* Reverting back to eastus
* Testing a new secret file
* Delete test-new-creds.yml
* Running tests on master
* Update job.yml (#1083 )
* Run tests in canary
* remove old dataset sample
* change process per node to 1 for cifar in pipeline
* add note to use multi-gpu cards
* Running latest build on eastus
* Update README.md
* fix sample issue to workaround output empty folder case
* Updating to latest build and testing in eastus
* Cleaning up all GitHub Workflows and the README file
* Update setup.sh
* Update dev-requirements.txt
* Fixing formatting issue so that black does not error
* Updating to the RC build
* change component name (#1075 )
* change component name
* rename
* add march-cli-preview in pipeline CI
* update registered case
* swith CI to master
* add back component creation
* update 3b to use rw_mount (#1097 )
* fix: fix run-pipeline-jobs.sh
* change upload mode to rw_mount
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Update README.md
* remove pipeline sample (#1102 )
* temp delete 2 samples
* remove reference links
Co-authored-by: Anthony Hu <anthu@microsoft.com>
* Fixing workflows and getting ready for merge to main
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Kayla Seager <1675997+kseager@users.noreply.github.com>
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
Co-authored-by: Paul Shealy <paulsh@microsoft.com>
Co-authored-by: Abraham Omorogbe <abrahamomor@gmail.com>
Co-authored-by: Ankit Singhal <30610298+singankit@users.noreply.github.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
Co-authored-by: Hugo Aponte <haponte@gmail.com>
Co-authored-by: Lisa Li (STC) <lisal@microsoft.com>
Co-authored-by: mx-iao <33701412+mx-iao@users.noreply.github.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Honglin Du <0mza987@gmail.com>
Co-authored-by: Roope Astala <rastala@users.noreply.github.com>
Co-authored-by: Swati Gharse <36742198+swatig007@users.noreply.github.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: eniac871 <supereniac@live.com>
2022-04-04 10:14:29 -07:00
3172ab93a8
refactor for CLI 2.0 preview ( #495 )
...
* scaffolding
* fixes, might break
* update contributing
* fix
* try 0.63
* fix readme
* order of cells
* fix env
* uncomment lines
* try a job
* add job test
* try miniconda image
* try mcr image
* -n -> --name for az ml workspace create
* fix workspace create
* same
* isolate rg
* fix filename
* simply env; increment version
* add mlflow
* fix enviornment, increment version
* switch to .sh
* add workspace create
* quick fix
* fix typo
* add asset creation
* debugging
* fix action
* fix
* typo
* fix typo
* workflow edits
* fix?
* edits
* fix data?
* Add more commands
* try something
* remove notebook
* update
* fix?
* try data workaround
* fix training script
* add azureml-mlflow to env
* saving
* try sweep job
* sweep edits
* fix yaml
* remove limits
* testing
* edits
* minor edits to basic.yml
* add exp name
* saving
* save; green?
* move --web to untested.sh
* hmm
* add model assets
* remove dsstore
* saving w/ model create
* add endpoints dir
* try something for docs
* try for docs
* try anon data and assets
* edit
* path edit
* fix asset path
* move stuff around
* try ./
* fix mistake
* save
* add mount
* fix
* change model to use run
* remove model register
* data hack
* test
* test registered file dataset
* saving
* testing
* whoops
* change strategy
* fix name
* switch rg back
* switch creds
* switch lightgbm back
* fix ws
* add file:
* remove deepspeed
* try tensorflow
* remove tensorflow
* edits for docs
* add verify
* unisntall -> remove
* -0
* saving
* add check_extensions
* fix mistake
* saving; might break
* update manage-assets and test it
* whoops
* minor edits
* local hello world
* edits
* simplify workflow
* try compute job
* ?
* try compute type
* fix hello world
* fix?
* major refactor
* remove line
* comments
* save
* save
* order of operations
* saving - run sweep job
* magic
* switch to data - possible typos
* fix typos?
* ...
* try dedenting for sweep
* edits
* names
* sweep hack for now
* broke stuff?
* test distributed
* update query logic
* add exp name, :1 to env
* wait for job to complete
* fix user error
* env to version 11
* readme edit
* try something
* remove redundant stuff
* version
* try downloading cifar
* try job status check
* debug bash
* fixes
* try stuff
* add back log upload
* fix bash?
* copy examples/endpoints from v2-preview
* run formatting
* add batch endpoint (#427 )
* hack
* move compute create to setup.sh; --name to -n for workspace create
* NC6 -> NC12
* try randint sweep
* remove randint from sweep; download=False for pytorch-cifar
* comment out logs upload from worklow
* slim iris lightgbm code
* try something for cifar
* fix
* preprocess -> process
* mlflow.pytorch log model for cifar
* no mlflow, fix cifar
* run formatting
* relocate hello world
* whoops
* cifar epochs 20 -> 2
* try new env for pytorch gpu
* switch pytorch to mlflow logging
* cleanup cifar code
* fix
* write to ./model?
* edit batch endpoint asset path
* force add model.pkl
* copy workspace assets
* add some schemas, more needed
* add more schemas
* fix schemas
* Updating the schema link to CDN endpoint (#428 )
* Update batch endpoint test (#429 )
* update schemas
* fix more schemas
* simplify
* Revert "simplify"
This reverts commit 38ec77e0ca#432 )
* test python/cli refactor (#431 )
* saving
* Update environment.yml (#430 )
* ?
* testing
* testing
* testing
* cli fix
* fix
* saving
* saving
* whoops
* fix readme
* fix readme
* whoops
* minor readme edit
* last readme edits
* add setup workspace step
* fix cli tests (#433 )
* test
* testing
* fixes
* save
* test
* fix how-to-manage-assets
* minor edit
* fix
* fix setup-workspace
* fix readme (#434 )
* edits for docs (#437 )
* fix readme
* remove sweep hack
* readme fix
* command => sweep in schema
* saving
* add back setup-workspace.sh
* ?
* sweep hack
* Saving
* edit readme asset logic
* -A
* minor edits for docs (#438 )
* fix readme
* remove sweep hack
* readme fix
* command => sweep in schema
* saving
* add back setup-workspace.sh
* ?
* sweep hack
* Saving
* edit readme asset logic
* -A
* >- => |
* missed hello world
* remove old env
* blind copy
* yolo
* edits
* edits
* docs edits (#440 )
* docs edits
* save
* remove upgrade
* fix
* reorder
* save (#442 )
* edit training jobs (#443 )
* saving
* minor edit
* fix world language? (#444 )
* change tf horovod env (#445 )
* change tf horovod env
* -
* move exp name
* standardize cli jobs (#446 )
* test
* standardize training jobs
* online endpoints (#435 )
* online endpoints
* run readme.py; manual edit workflows
* fix delete command
* fix
* fix endpoint
* rerun readme
Co-authored-by: Cody <54814569+lostmygithubaccount@users.noreply.github.com>
* quick fix (#448 )
* run readme (#449 )
* docs updates (#450 )
* minor edit for docs (#451 )
* docs updates
* quick fix
* undo sweep hack (#452 )
* edits for docs (#453 )
* add hello world back
* updates
* minor edits
* Update python-ml-basic-cpu.yml
* Add REST examples (#439 )
* Add scripts without blob cli
* Everything working for endpoint except discrimnator issue with online deployment
* Working simple job flow
* Use default container for data
* Rename scripts, make wait_for_completion a function, fix online deployment
* Add scoring to endpoint
* Change structure of folders
* Remove todo
* Move rest to cli folder
* Switch to lightgbm from xgboost
* Fix wait_for_completion
* Get AZURE_STORAGE_KEY from az cli
* Change MIR region to westus
* Remove exports from variables
* Fix scoring the endpoint
Co-authored-by: Trent McCormick <trmccorm@microsoft.com>
* Fix closing tag of sample to match opening tag. (#455 )
* updates for docs (#456 )
* switch cli version back (#457 )
* /
* tests for rest how-to (#454 )
* run readme
* save
* test rest with azureml-examples
* fixes
* run readme
* Use old way of getting certain variables
Co-authored-by: tmccrmck <tjmcc01@gmail.com>
* Update how-to-batch-score.sh (#436 )
Fix batch asset path
* force add files (#458 )
* endpoint script fix (#459 )
* fix endpoints
* fix endpoints
* fix endpoint script
* fix endpoints
* edits for docs (#461 )
* use
* exp name edits
* bash edits
* save
* save
* edit
* Get storage info from default datastore (#460 )
* simplify hello world
* move RG (#463 )
* switch credentials (#464 )
* debug (#465 )
* debug
* fix typo
* fjeiwpaol tefjpao
* Tweak to canary-flow shellscript (#466 )
* Fix `test_green` snippet
`test_green` snippet was dupe of `green_10pct_traffic`. Switched to syntax described in https://azure.github.io/azureml-v2-preview/_build/html/overview/concepts/endpoints/online-endpoint/scenarios/declarative-flow.html
* Results of `python readme.py`
This seems erroneous to me, but...
* Changed endpoint name to my-new-endpoint per rest of script
* fixed typo in var name
Co-authored-by: Cody <54814569+lostmygithubaccount@users.noreply.github.com>
* Add managed identity shell script for online endpoint (#467 )
* Online endpoint managed identities tests shell script
* Updated online endpoint managed identity shell scripts
Co-authored-by: pansav <pansav@microsoft.com>
* imperative flow (#469 )
* .yaml -> .yml (#470 )
* .yaml -> yml
* update python files
* edit
* Add get deployment logs to rest docs (#473 )
* Add getLogs API call
* Fix typo
* minor edit to hello world (#474 )
* add -example to exp name
* edits for docs (#476 )
* hacks
* order of operations
* edits
* increment model
* edits to batch
* fix readme
* reduce training times to 1 epoch
* Update setup.sh
* >- -> > (#477 )
* update CLI version (#480 )
* update CLI version
* try 0.71
* use 1.0.0a2
* Gopalv/tfserving sample (#479 )
* WIP
* adding logic to extract ACR
* ran readme.py
* local testing works
* sed works
* parametrized, added local and remote test
* adding workflow file
* acr login failing
* adding sleep after docker run
* workflows
* Fixed deployment yaml for online endpoint managed identity example (#482 )
Co-authored-by: pansav <pansav@microsoft.com>
* updates to cli-preview (#484 )
* testing
* update dockerfiles
* swithc back to conda yml
* test
* ?
* ?
* dead snakes
* fix job
* remove non-python jobs
* test
* job descriptions
* asset descritptions
* docs
* save
* saveing
* Fixed online endpoint automation failures issues (#489 )
* Fixed automation failures issues
* Removed try catch block from run method
* Fixed declarative yaml
* Updated docker image uri to Ubuntu 18.04
* Update system identity flow to use endpoint show to retrieve object id
Co-authored-by: pansav <pansav@microsoft.com>
* Add how-to-train-autorest with Python SDK (#481 )
* Add autorest example
* Fixes for REST scripts
* Removed linux specific code in shell script (#492 )
* Removed linux specific code in shell script
* Added delay after system identity is created and also changed secrets id
* Changed CLI version for managed identity tests
* Fixed declarative yaml file and upgraded to latest CLI version for managed identity
* Changed user identity place holder name
* Changed online endpoint schema url in yaml files
* Fixed endpoint delete command in declarative flow
* Fixed file path in declarative sample
Co-authored-by: pansav <pansav@microsoft.com>
* Fix how-to-deploy-script (#491 )
* Fix how-to-deploy-script
* Use random versions for environmetns
* Don't use headers for async operations
* Use jq -r everywhere
* Print async operation error
* Try changing endpoint
* Add mlflow model file (#486 )
* update batch sample (#493 )
* Gopalv/triton sample (#490 )
* WIP
* adding logic to extract ACR
* ran readme.py
* local testing works
* sed works
* parametrized, added local and remote test
* adding workflow file
* acr login failing
* adding sleep after docker run
* workflows
* updating yaml and script to exit properly
* adding deletion
* fixing typos
* making base endpoint more generic
* try after deletion
* renaming tfserving
* moved file to correct place
* works until MFE error
* not tested locally
* new CLI drop seems to work
* adding triton endpoint
* using anonymous environments
* adding back placeholder
* tfserving fails for some reason
* triton works
* get different acr
* install pyindex
* deleting model
* fixing model delete
* factoring cleanup utility
* adding exception handling for acr fail
* fixing syntax error
* making image url an argument
* using public image for tfserving
* add back tutorial test, rename
* manual modify of recently added workflow
* repin black; minor edit
* run black-nb
* fix typo
* test horovod env (#496 )
* test horovod env
* revert to old environment
* try v2
* revert
* try oss horovod
* try tf image
* try horoovd image
* switch horovod env
* 1.0.0a2 -> 1.0.0a4
* update batch sample (#497 )
* Removed min/max instances and changed auth_mode to key (#498 )
Co-authored-by: pansav <pansav@microsoft.com>
* saving (#499 )
* minor reqs edits
Co-authored-by: Tracy Chen <47156502+tracychms@users.noreply.github.com>
Co-authored-by: Harneet Virk <harnvir@microsoft.com>
Co-authored-by: Sethu Raman <rsethur@users.noreply.github.com>
Co-authored-by: trent <tjmcc01@gmail.com>
Co-authored-by: Trent McCormick <trmccorm@microsoft.com>
Co-authored-by: vmagelo <51368918+vmagelo@users.noreply.github.com>
Co-authored-by: Larry O'Brien <lobrien@knowing.net>
Co-authored-by: psavdekar <45885005+psavdekar@users.noreply.github.com>
Co-authored-by: pansav <pansav@microsoft.com>
Co-authored-by: Gopal Vashishtha <gopalv@microsoft.com>
2021-05-11 18:53:32 -07:00
Facundo Santiago
kdestin
Ying Chen
Bala P V
Cody
Sethu Raman