Merge Build 2022 changes to main (#1182)
* Standalone Jobs and Pipelines with Core SDK
* Dataset Examples
* Adding environment examples
* Examples for models
* remove unwanted files
* Create workspace.ipynb
* Create compute.ipynb
* Update readme.md
* Removing credentials from notebook
* Lochen/pipeline dsl example (#913)
* add DSL example
* update readme
* Formatting (#914)
* add batch sdk sample (#918)
* Remove unwanted files from command-job-in-pipeline
* removing unwanted files from pipelines with components
* Lochen/pipeline dsl example (#920)
* add DSL example
* update readme
* change component name
* update folder name
* remove sample add more notebook
* Adding example for online inferencing
* Add files via upload
* Add files via upload
* Create datastore.ipynb
* Update MLClient to the latest interface and making use of promoted properties (#921)
* Updated command job examples to align with the latest release
* ignore notebook cache
* remove cache
* Online inferencing example
* Update samples
* Pin to a specific private preview SDK version (#924)
* Pin to a specific private preview API version
* typo
* readme consistency edits (#926)
* readme consistency edits
* rename?
* fix errant backticks (#927)
* Adding sweep job to samples (#931)
* Adding sweep job to samples
* Update lightgbm-iris-sweep.ipynb
* Update .gitignore
* Update README.md
* How to use datasets in a job
* How to use datastore
* Adding Documentation the PyTorch notebook.
* Create notebook-template.ipynb
* Accidents Example for R
* Update accident-prediction.ipynb
* Adding R sample
* Updates to documentation
* Updating Documentation
* Updating documentation
* Updating documentation
* Updating documentation for single step jobs
* Update model.ipynb
* Update sklearn-mnist.ipynb
* Updating documentation for endpoints
* Paulsh/sdk dashboard (#985)
* First pass
* Add trigger
* Fix sed commands
* Fix path to workbook
* Updates
* Updates
* Updates
* Skip some workspace creation commands and use a random workspace name
* Shorten workspace name to fit ARM requirements
* Test (#988)
* Test
* Update
* Add config file
* Comment out config file lines
* Update (#989)
* Workflows for testing sample notebooks (#997)
* Adding workflows for single step jobs
* python file for workflow creation
* Adding versions to environments
* Adding workflows for assets
* Add files via upload
* Update tensorflow-mnist-distributed.ipynb
* Adding workflows for assets
* Adding workflows for endpoints
* Update tensorflow-mnist-distributed.ipynb
* Updating pipeline samples
* Adding workflows for pipelines with components
* Revert "Updating pipeline samples"
This reverts commit 468438ec4f.
* Updating pipeline with component samples
* Name uniqueness changes for CI
* Update online-inferencing-sample.ipynb
* Update nyc_taxi_data_regression.ipynb
* Update pipeline-dsl-example.ipynb
* Update pipeline-dsl-example.ipynb
* Adding workflows for non-DSL pipelines
* Editing Readme and removing some workflows
* Delete python-sdk-resources-workspace-notebook.yml
* Update write-workflows.py
* Cleaning up workflows based on pauls comments
* Update README.md
* Update README.md
* Update sdk-assets-model-model.yml
* Update sdk-assets-model-model.yml
* updated model notebook
* more updates to model.ipynb
* Removing workflows from branch. Will add them to main
* Update README.md
* Update README.md
* Update README.md
* Update basic_pipline.ipynb
* Update e2e_registered_components.ipynb
* Providing more information in README
* SDKV2 Data science 101 e2e sample (#1004)
* added e2e data scientist sample part I
Co-authored-by: amah <ma.mahmoudzadeh@gmail.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
* Janhavi13/model nb (#1008)
* Fix model_uri in model notebook
* Update model upload from cloud path and run
* Update model upload from cloud path and remove run
* Move Tutorials to dedicated folder (#1034)
* moved e2e ds tutorial to a dedicated folder
* removed from old location
Co-authored-by: amah <ma.mahmoudzadeh@gmail.com>
* Update dev-requirements.txt
* Hancwang/batch fix (#1011)
* Batch deployment fix
* typo
* few more changes to invoke and deployment init
Co-authored-by: Banibrata De <banibrata@outlook.com>
* Distributed Deep Learning image end to end tutorial (#1023)
* write e2e object dl notebook
* implement distributed pytorch
* implement mlflow model logging
* implement profiling
* March sdk preview (#1021)
* Address CLI breaking changes
* Update nyc_taxi_data_regression.ipynb
* Updating pipelines
* Inline documentation updates
* Incorporating Sam's PR into this branch
* Installing new build
* Adding temp workflows for automated testing of notebooks
* Update model.ipynb
* Update model.ipynb
* Reverting model to sdk-preview version
* Update setup.sh
* Update setup.sh
* Adding a new build
* Removing all workflows from this branch
* Create sdk-assets-model-model.yml
* Update dev-requirements.txt
* Removing local path, adding @latest
* Updating to latest version of SDK/CLI and adding ipython_genutils
* Adding workflows for assets
* Update dev-requirements.txt
* Updating to Build # 58490278
* Adding tests for single step jobs
* Adding tests for endpoints
* Adding test workflows for pipelines
* Adding all test workflows
* Removing dataset notebook since we now have data
* Automated README.md
* Workflows for this branch
* Updating to latest build
* Update mnist-nonmlflow.ipynb
* Updating to latest build
* Adding description metadata to notebooks
* Improving README and Workflow generation
* Lochen/sdk march (#1041)
* innit
* init
* Lochen/sdk march (#1042)
* innit
* init
* merge folder
* fix pipeline sample: cifar-10, tf_mnist
* update workflows for pipeline
* Lochen/sdk march (#1044)
* innit
* init
* merge folder
* change e2e registered, basic, nyc
* basic pipeline nyc
* add instructions.
* update nyc
* fix basic pipeline
* Upgrade to latest build
* Adjusting workflows and README for new pipeline samples
* Dummy commit for testing
* Update README.md
* Adding descriptions for pipeline notebooks
* Updating to latest build
* Removing datastores from testing and compute from scheduled runs
* Update lightgbm-iris-sweep.ipynb
* Updating to latest build
* Update README.md
* Fixing source directory for data and updating CLI/SDK build
* Updating to latest build
* Update README.md
* Adding clean up for endpoints and workspaces
* Update workspace.ipynb
* Updating to new build
* Update README.md
* Fixing issues with workspace notebook
* Update setup.sh
* Update setup.sh
* Updating to latest build
* Update README.md
* Updating to latest build and testing in eastus
* Fixing workflows
* Update readme.py
* Online endpoints sdk (#1059)
* online endpoints sdk samples
* online endpoints sdk samples
* minor update
* updating readme.md
* adding back orginal sample
* minor
* update to sample folder
* minor changes
* Adding workflows and README for the new online endpoint examples
* Updating to the RC build
* Update README.md
* Fixing issues with MIR samples
* Standardizing kernelspec for all notebooks
* Formatting Cleaning python files with black
* Fixing readme.py
* Moving workflows from march-sdk-preview to sdk-preview
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
* Update README.md
* April Changes to SDK and CLI v2 (#1114)
* Address CLI breaking changes
* Update nyc_taxi_data_regression.ipynb
* Updating pipelines
* Inline documentation updates
* Incorporating Sam's PR into this branch
* Installing new build
* Adding temp workflows for automated testing of notebooks
* Update model.ipynb
* Update model.ipynb
* Reverting model to sdk-preview version
* Update setup.sh
* Update setup.sh
* Adding a new build
* Removing all workflows from this branch
* Create sdk-assets-model-model.yml
* Update dev-requirements.txt
* Removing local path, adding @latest
* Updating to latest version of SDK/CLI and adding ipython_genutils
* Adding workflows for assets
* Update dev-requirements.txt
* Updating to Build # 58490278
* Adding tests for single step jobs
* Adding tests for endpoints
* Adding test workflows for pipelines
* Adding all test workflows
* Removing dataset notebook since we now have data
* Automated README.md
* Workflows for this branch
* Updating to latest build
* Update mnist-nonmlflow.ipynb
* Updating to latest build
* Adding description metadata to notebooks
* Improving README and Workflow generation
* Lochen/sdk march (#1041)
* innit
* init
* Lochen/sdk march (#1042)
* innit
* init
* merge folder
* fix pipeline sample: cifar-10, tf_mnist
* update workflows for pipeline
* Lochen/sdk march (#1044)
* innit
* init
* merge folder
* change e2e registered, basic, nyc
* basic pipeline nyc
* add instructions.
* update nyc
* fix basic pipeline
* Upgrade to latest build
* Adjusting workflows and README for new pipeline samples
* Dummy commit for testing
* Update README.md
* Adding descriptions for pipeline notebooks
* Updating to latest build
* Removing datastores from testing and compute from scheduled runs
* Update lightgbm-iris-sweep.ipynb
* Updating to latest build
* Update README.md
* Fixing source directory for data and updating CLI/SDK build
* Updating to latest build
* Update README.md
* Adding clean up for endpoints and workspaces
* Update workspace.ipynb
* Updating to new build
* Update README.md
* Fixing issues with workspace notebook
* Update setup.sh
* Update setup.sh
* create new branch
* adapt sweep sample to april release
* adapt to command based job creation
* Updating workflows and readme to point to the april branch
* Updating CLI README to show April branch status
* Updating CLI workflows to trigger in april branch
* Changing workflows to trigger on april branch
* Builder fix up
* fix typo
* [April SDK] Pipeline builder function samples (#1117)
* refine the samples
* re-structure pipeline sdk examples
* merge folder
* reorgnize pipeline samples
* refine
* add image_classification sample
* update dsl_component example
* add command function sample
* update
* refine command function sample
* update 3a_tf_mnist
* add sample 3e_image_classification
* update 3a_tf_mnist
* refine samples
* fix 3a_tf_mnist
* refine samples
* refine samples
* refine samples
* refine sample
* update sample 1a
* fix: sdk-3b_tf_mnist_sweep (#1060)
fix entry func name in component definition
* refine sample
* refine samples
* add load auto load extention
* refine samples
* tf_minst_sweep: fix sample (#1062)
* tf_minst_sweep: fix sample
* Update train.py
* Revert metric codes (#1061)
* update mlflow
* still use azure.core
Co-authored-by: chenyin <chenyin@microsoft.com>
* refine samples
* sdk: 3c_cifar-10 sample bash encoding error (#1063)
* add readme
* update readme
* print link that can be clicked (#1067)
* refine samples
* rename the samples
* refine sample 1a
* Change DataInput/Output to ArtifactInput/Output
* refine sample 1a
* update naming for image classification
* refine 1e sample
* refine READMEs
* update README
* update ci workflow
* update ci SDK version
* add ci pipeline for 2d_image_classification_with_densenet
* update readme
* update readme
* add workflows
* merge march CLI and add mltable, sweep sample
* add march cli and sweep mltable support
* feat: refine sweep & cifar examples (#1078)
* feat: refine sweep examples with dummy components
* fix: rollback python version
* fix: fix azure-ml version
* refactor: rename & use dataset
use a dataset instead of downloading inside a node
update learning objective & motivation
* fix: fix conda environment
* fix: add troubleshooting for gpu size not matched
* feat: demo more hyperparameters
* fix: update setup azure-ml version
* feat: add ci
* fix: update ci
* fix: fix ci
* fix: fix ci
* fix: fix ci
* fix: fix ci
* remove unused file
* fix: fix naming issue
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Refine dsl components samples (#1079)
* Refine dsl components samples
* add CI workflow to Readme
* correct instance_count usage
* remove useless code
* Set resources for component
* update working dir
* change sample order to align with march version
* mltable
* include pipeline branch in CI
* update image classification workflow (#1085)
* Refine common code snippet of samples (#1080)
* change sample1
* move config location
* update CI
* refine
* refine wording
* change samples
* update snippet
* update notebook
* update
* update CI
* update 1e
* update samples
* refine sample
* refine inputs
* update
* add CI
* refine workflows
* fix dockerfile
* refine readme
* update 2d
* fix sample
* refine samples
* refine samples
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
* refine readme
* update
* update nyc to use mltable and mlflow model
* update doc link
* Updating to latest build
* Update README.md
* fix typo
* remove unneccesary imports
* Updating to latest build and testing in eastus
* Fixing workflows
* Update readme.py
* Online endpoints sdk (#1059)
* online endpoints sdk samples
* online endpoints sdk samples
* minor update
* updating readme.md
* adding back orginal sample
* minor
* update to sample folder
* minor changes
* test workflow scope
* add job stream
* remove test workflow
* fix workflow cred
* update
* update sdk version
* update mnist sample
* Adding workflows and README for the new online endpoint examples
* Updating to the RC build
* update
* fix: flush (#1094)
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* update build number
* update version
* change to rw_mount
* replace path with uri_folder
* fix: rename & correct the usage of setting (#1116)
pipeline_job_setting.continue_on_step_failure
* Update sdk script gen pipeline jobs workflow (#1113)
* update setup script
* update workflows
* fix checkout
* add script to write pipeline readme
* update readme
Co-authored-by: Han Wang <wanhan@microsoft.com>
* update compute (#1093)
* Test v2 samples in magic workspace (#1092)
* move test workspace
* change back CI
* add script to run all samples
* skip wait for completion
* fix script
* remove ps1
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Clwan/update nyc with mlflow_model type (#1118)
* update notebook descriptions
* update readme
* update sdk nyc sample to adopt mlflow_model
* use sub folder as worksround for mlflow model
* refine samples
* update sweep and mlflow model
* refine samples
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: lochen <cloga0216@gmail.com>
* [Pipeline builder function] merge april branch (#1126)
* add amlarc workflow (#968)
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* add amlarc workflow
* committing onnx model generator script used with scriptrunconfig (#962)
* updating onnx model generator with sriptrunconfig
* removing duplicacy with single onnx model script
* updating with minor changes
* updating with reviews
* minor changes
* model_name comparison changed
* minor changes
* Adding CodeGen related changes to bankmarketing notebook (#970)
* added codegen related cells to bankmarketing notebook
* running black
* run black (#978)
* Unpinning azureml-defaults to avoid installing itsdangerous coming indirectly from Flask (#981)
* Unpinning azureml-defaults to avoid installing itsdangerous coming indirectly from Flask
* Pinning to the latest azureml version and touching files to trigger the builds
* Removing the update from docker
* updating mlflow deployment skus (#982)
* Adding tags to certain cells in AutoML notebook (#964)
* added metadata cells.
* fix inf env issue (#995)
Co-authored-by: Yuan Zhuang <yuzhua@microsoft.com>
* Update dev-requirements.txt
* Create test.yml
* Delete test.yml
* Create sdk-assets-model-model.yml (#1001)
* Create sdk-assets-model-model.yml
* Update sdk-assets-model-model.yml
* Update sdk-assets-model-model.yml
* Update and rename sdk-assets-model-model.yml to smoke-test-sdk-notebook-tests.yml
* Update and rename smoke-test-sdk-notebook-tests.yml to smoke-test-sdk-samplejob.yml
* Delete smoke-test-sdk-samplejob.yml
* Workflows for Testing SDKv2 Sample Notebooks
* Adding more workflows for sdk v2 notebook testing
* Adding workflows for SDKv2 Notebook Testing
* add language to kernelspec as asked by compute instance team (#1015)
Co-authored-by: Lifeng Wei <lifengwei@microsoft.com>
* Add gpu diag/perf cli job (#980)
* add gpu diag/perf cli job
* add path in docker instead of hardcoded in job
* update structure to have distinct folder for environment
* update environment to use inline
* finalize tests
* Upgrading the azure cli version
* Lifengwei/automl nlp notebooks (#1018)
* multilabel example notebook with preprocessed data
* upgrade azureml_env_linux to 1.39 version
* make separate update_env file for new notebook
Co-authored-by: Lifeng Wei <lifengwei@microsoft.com>
* Lifengwei/automl nlp notebooks (#1026)
* multiclass related files
* ner related files
* add language info to all notebooks
* make multilabel notebook trained with distributed training
Co-authored-by: Lifeeng Wei <lifengwei@microsoft.com>
* Fix a weird papermill error
Adding ipython_genutils to fix a weird papermill error
* update creds in cleanup script (#1029)
* Distributed Deep Learning image end to end tutorial (#1023)
* write e2e object dl notebook
* implement distributed pytorch
* implement mlflow model logging
* implement profiling
* add get_output, remove extra parameters, add comments for primary_metric (#1035)
* add get_output, remove extra parameters, add comments for primary_metric
* update experiment names
* fix a typo
* typo
Co-authored-by: Lifeeng Wei <lifengwei@microsoft.com>
* Set idle_seconds_defore_scaledown (#1039)
* Set idle_seconds_defore_scaledown
* Applied black, which removed some spaces
* Fixed AutoML notebooks pandas -1 (#1038)
* update tfserving byoc sample (#983)
* update tfserving byoc sample
* minor update
* anon byoc env (#1051)
* Automl notebook fixes (#1046)
* Jeffshep/fixcsrmatrixissue (#1072)
* Fix error with csr_matrix is not JSON serializable
* Applied black
* support custom ennvironment packages by environemnt.yml (#1005)
* update version of black to fix smoke test (#1088)
* update
* remove workflow
* Lifengwei/update nlp notebooks (#1090)
* remove some packages
* add back a package
* update notebooks, mainly change multilabel data file
* data file
* fix typos, codes in notebooks
* update experiment name
* fix typo, remove comments
Co-authored-by: Lifeeng Wei <lifengwei@microsoft.com>
* March sdk preview (#1021)
* Address CLI breaking changes
* Update nyc_taxi_data_regression.ipynb
* Updating pipelines
* Inline documentation updates
* Incorporating Sam's PR into this branch
* Installing new build
* Adding temp workflows for automated testing of notebooks
* Update model.ipynb
* Update model.ipynb
* Reverting model to sdk-preview version
* Update setup.sh
* Update setup.sh
* Adding a new build
* Removing all workflows from this branch
* Create sdk-assets-model-model.yml
* Update dev-requirements.txt
* Removing local path, adding @latest
* Updating to latest version of SDK/CLI and adding ipython_genutils
* Adding workflows for assets
* Update dev-requirements.txt
* Updating to Build # 58490278
* Adding tests for single step jobs
* Adding tests for endpoints
* Adding test workflows for pipelines
* Adding all test workflows
* Removing dataset notebook since we now have data
* Automated README.md
* Workflows for this branch
* Updating to latest build
* Update mnist-nonmlflow.ipynb
* Updating to latest build
* Adding description metadata to notebooks
* Improving README and Workflow generation
* Lochen/sdk march (#1041)
* innit
* init
* Lochen/sdk march (#1042)
* innit
* init
* merge folder
* fix pipeline sample: cifar-10, tf_mnist
* update workflows for pipeline
* Lochen/sdk march (#1044)
* innit
* init
* merge folder
* change e2e registered, basic, nyc
* basic pipeline nyc
* add instructions.
* update nyc
* fix basic pipeline
* Upgrade to latest build
* Adjusting workflows and README for new pipeline samples
* Dummy commit for testing
* Update README.md
* Adding descriptions for pipeline notebooks
* Updating to latest build
* Removing datastores from testing and compute from scheduled runs
* Update lightgbm-iris-sweep.ipynb
* Updating to latest build
* Update README.md
* Fixing source directory for data and updating CLI/SDK build
* Updating to latest build
* Update README.md
* Adding clean up for endpoints and workspaces
* Update workspace.ipynb
* Updating to new build
* Update README.md
* Fixing issues with workspace notebook
* Update setup.sh
* Update setup.sh
* Updating to latest build
* Update README.md
* Updating to latest build and testing in eastus
* Fixing workflows
* Update readme.py
* Online endpoints sdk (#1059)
* online endpoints sdk samples
* online endpoints sdk samples
* minor update
* updating readme.md
* adding back orginal sample
* minor
* update to sample folder
* minor changes
* Adding workflows and README for the new online endpoint examples
* Updating to the RC build
* Update README.md
* Fixing issues with MIR samples
* Standardizing kernelspec for all notebooks
* Formatting Cleaning python files with black
* Fixing readme.py
* Moving workflows from march-sdk-preview to sdk-preview
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
* Modifying SDK workflows to sync up with March SDK changes
* Update README.md
* March CLI Breaking Changes (#1020)
* remove component job, flatten code (#1002)
* Update .gitignore
* Addressing CLI breaking changes
* adding uri_file to path
* adding uri_file type to paths
* adding uri_file and uri_folder
* Updating basic pipeline jobs
* Update hello-pipeline-default-artifacts.yml
* Lochen/cli march (#1009)
* remove component job, flatten code
* add r component sample
* Online deployment changes
* Batch Deployment Changes
* Bug fixes to YAML
* Update REST API examples to Feb API (#1013)
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
* minor fix
* resolve yaml error
* environment name and version as optional in custom containers (#1016)
* minor updates (#1017)
* Lochen/cli march (#1019)
* remove component job, flatten code
* add r component sample
* update cli
* Updates to cifar example
* Adding daily build of CLI for testing
* Updating from dataset to data for create
* Rename folder from asset/datasets to assets/data
* Updating data assets to use data schema directive instead of dataset
* rename description to data asset
* Changing github workflows to use the new data assets
* removing dataset from md files
* Update README.md
* Removing local_path from examples
* Fixing python:latest bug with workaround
* Update prefix.md
* Lochen/cli march (#1027)
* remove component job, flatten code
* add r component sample
* update cli
* update after test
* Updating environments to use latest version
* Updating environment to us latest
* Update setup.sh
* Test
* streamline register from a run (#1024)
* streamline register from a run
* update to MLflow specific
* Lochen/cli march (#1031)
* remove component job, flatten code
* add r component sample
* update cli
* update after test
* add default_compute, dataset
* Lochen/cli march (#1032)
* remove component job, flatten code
* add r component sample
* update cli
* update after test
* add default_compute, dataset
* fix default compute, sweep
* fix default settings on pipeline
* Using the new build for CLI
* Update requirements.txt (#1033)
* Updating to Build # 58490278
* Fixing filenames in the workflows.
* Fixing file name in train.sh
* add default value in pipeline settings.
* update default value
* change data asset
* update job link in readme
* Updating to latest CLI build
* Merge latest changes from main related to cli/jobs/single-step/gpu_perf/ command (#1037)
* pick and choose changes from main
* align, add readme
* Update create-datasets.sh
* Updating to latest build
* Update README.md
* Update README.md
* Fixing failures in CLI
* Dummy commit to re-run tests
* fix to align with resource in test ws
* change 4d to use uri_folder
* upgrade dependencies, use fastparquet to fix pyarrow issue (#1043)
* Upgrading to latest build
* merge datastore uri sample
* fix file script
* Users/anksing/mlflow model mir deploy (#1047)
* Adding example for OBO flow using v2 cli
* Updating sklearn model for MIR deployment
* Updating lightgbm mlflow model for MIR deployment
* Removing obo folder
* Users/anksing/mlflow mir deployment (#1048)
* Adding example for OBO flow using v2 cli
* Renaming model to avoid conflict with existing model with same name
* Removing obo folder
* Update batch-score.sh
Adding az version check.
* Updating model name to avoid conflict (#1049)
* change cifar to directly use command
* add image classification cli example
* Updating to latest build
* Making setup files Unix friendly by removing \r
* Update endpoint.yml
remove the properties config -- we are not exposing in v2
* Update blue-deployment.yml
Removed properties field - not exposing user-configurable properties metadata in V2
* Dummy Commit to force tests
* update pipeline cifar to make it work under cpu-cluster
* update cifar sample to support cpu sku
* change 5c to cpu image
* clean up yaml, change cpu image for cifar
* Updating to latest build
* Update README.md
* formatting python files to ensure smoke test does not fail
* update workflow (#1064)
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Wanhan/add image classification CI (#1065)
* add image classification ci
* fix create component
Co-authored-by: Han Wang <wanhan@microsoft.com>
* fix image classification ci (#1066)
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Update entry.py
* Update top latest build
* resolve merge conflict
* update to v2 dsl.component
* update train step (#1071)
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Updating to latest build
* Update README.md
* Update README.md
* remove pyjokes in source file
* recover sample to test CI
* add component CRUD sample in assets
* Updating to new build
* Update setup.sh
* Changing readme temporarily to show march status
* Examples for workspace connections (#1073)
* Create python-feed-pat.yml
* More examples
* Create README.md
* Update README.md
* update component name, add CI workflow
* update readme for asset component
* Update setup.sh
* Add yml for scheduled start stop of CI
Add yml for scheduled start stop of CI
* Delete instance-schedule.yml
* fix: fix attr overwrite command (#1074)
* fix: fix attr overwrite command
add pipeline job test shell
* fix: fix attr overwrite command
* fix: create data with random version
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Updating to latest build
* Running tests on canary
* Trying to run test cases in master
* Reverting back to eastus
* Testing a new secret file
* Delete test-new-creds.yml
* Running tests on master
* Update job.yml (#1083)
* Run tests in canary
* remove old dataset sample
* change process per node to 1 for cifar in pipeline
* add note to use multi-gpu cards
* Running latest build on eastus
* Update README.md
* fix sample issue to workaround output empty folder case
* Updating to latest build and testing in eastus
* Cleaning up all GitHub Workflows and the README file
* Update setup.sh
* Update dev-requirements.txt
* Fixing formatting issue so that black does not error
* Updating to the RC build
* change component name (#1075)
* change component name
* rename
* add march-cli-preview in pipeline CI
* update registered case
* swith CI to master
* add back component creation
* update 3b to use rw_mount (#1097)
* fix: fix run-pipeline-jobs.sh
* change upload mode to rw_mount
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Update README.md
* remove pipeline sample (#1102)
* temp delete 2 samples
* remove reference links
Co-authored-by: Anthony Hu <anthu@microsoft.com>
* Fixing workflows and getting ready for merge to main
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Kayla Seager <1675997+kseager@users.noreply.github.com>
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
Co-authored-by: Paul Shealy <paulsh@microsoft.com>
Co-authored-by: Abraham Omorogbe <abrahamomor@gmail.com>
Co-authored-by: Ankit Singhal <30610298+singankit@users.noreply.github.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
Co-authored-by: Hugo Aponte <haponte@gmail.com>
Co-authored-by: Lisa Li (STC) <lisal@microsoft.com>
Co-authored-by: mx-iao <33701412+mx-iao@users.noreply.github.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Honglin Du <0mza987@gmail.com>
Co-authored-by: Roope Astala <rastala@users.noreply.github.com>
Co-authored-by: Swati Gharse <36742198+swatig007@users.noreply.github.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: eniac871 <supereniac@live.com>
* Updating workflows and readme to point to the april branch
* Updating CLI README to show April branch status
* Updating CLI workflows to trigger in april branch
* Changing workflows to trigger on april branch
* Builder fix up
* fix typo
* update workflows
Co-authored-by: Zeliang Tian <83852443+zetiaatgithub@users.noreply.github.com>
Co-authored-by: Vadthyavath Ram <7171558+vadthyavath@users.noreply.github.com>
Co-authored-by: Razvan Tanase <ratanase@microsoft.com>
Co-authored-by: Cody Peterson <54814569+lostmygithubaccount@users.noreply.github.com>
Co-authored-by: Harneet Virk <harnvir@microsoft.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
Co-authored-by: Yuan Zhuang <40252949+yuanzhuangyuanzhuang@users.noreply.github.com>
Co-authored-by: Yuan Zhuang <yuzhua@microsoft.com>
Co-authored-by: Bala P V <33712765+balapv@users.noreply.github.com>
Co-authored-by: Lifeng Wei <lfwei@ucdavis.edu>
Co-authored-by: Lifeng Wei <lifengwei@microsoft.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
Co-authored-by: Sethu Raman <rsethur@users.noreply.github.com>
Co-authored-by: jeff-shepherd <39775772+jeff-shepherd@users.noreply.github.com>
Co-authored-by: v-pbavanari <97779269+v-pbavanari@users.noreply.github.com>
Co-authored-by: Banibrata De <banibrata@outlook.com>
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Bala P V <balapv@microsoft.com>
Co-authored-by: Kayla Seager <1675997+kseager@users.noreply.github.com>
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
Co-authored-by: Paul Shealy <paulsh@microsoft.com>
Co-authored-by: Abraham Omorogbe <abrahamomor@gmail.com>
Co-authored-by: Ankit Singhal <30610298+singankit@users.noreply.github.com>
Co-authored-by: Hugo Aponte <haponte@gmail.com>
Co-authored-by: Lisa Li (STC) <lisal@microsoft.com>
Co-authored-by: mx-iao <33701412+mx-iao@users.noreply.github.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Honglin Du <0mza987@gmail.com>
Co-authored-by: Roope Astala <rastala@users.noreply.github.com>
Co-authored-by: Swati Gharse <36742198+swatig007@users.noreply.github.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Hanchi Wang <hancwang@microsoft.com>
Co-authored-by: Daniel Schneider <danielsc@microsoft.com>
* feat: sweep cli sdk examples update (#1125)
* feat: use mlflow.autolog instead of Run.log
* fix: fix predict code
* fix: update parameters
* add image_classification_keras_minist_convnet (#1127)
* add image classification kerad
* update
* update
* add readme
* add workflow and update README
Co-authored-by: Clement Wang <clwan@microsoft.com>
* fix: cifar-10/image_classification output mode
* update readme
* feat: descriptions for cli pipeline job examples (#1129)
* feat: descriptions for cli pipeline job examples
* fix: remove hello git & update setup.sh
* fix readme
* Lochen/dataset asset (#1128)
* write model to load then move to output folder
* add dataset dataasset sample
* add assest
* change component name to lower case
* refine samples
* update workflows
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: lochen <cloga0216@gmail.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
Co-authored-by: Ying Chen <2601502859@qq.com>
Co-authored-by: chenyin <chenyin@microsoft.com>
Co-authored-by: Korin <0mza987@gmail.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: Bala P V <balapv@microsoft.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
Co-authored-by: Zeliang Tian <83852443+zetiaatgithub@users.noreply.github.com>
Co-authored-by: Vadthyavath Ram <7171558+vadthyavath@users.noreply.github.com>
Co-authored-by: Razvan Tanase <ratanase@microsoft.com>
Co-authored-by: Cody Peterson <54814569+lostmygithubaccount@users.noreply.github.com>
Co-authored-by: Harneet Virk <harnvir@microsoft.com>
Co-authored-by: Yuan Zhuang <40252949+yuanzhuangyuanzhuang@users.noreply.github.com>
Co-authored-by: Yuan Zhuang <yuzhua@microsoft.com>
Co-authored-by: Bala P V <33712765+balapv@users.noreply.github.com>
Co-authored-by: Lifeng Wei <lfwei@ucdavis.edu>
Co-authored-by: Lifeng Wei <lifengwei@microsoft.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
Co-authored-by: Sethu Raman <rsethur@users.noreply.github.com>
Co-authored-by: jeff-shepherd <39775772+jeff-shepherd@users.noreply.github.com>
Co-authored-by: v-pbavanari <97779269+v-pbavanari@users.noreply.github.com>
Co-authored-by: Banibrata De <banibrata@outlook.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Kayla Seager <1675997+kseager@users.noreply.github.com>
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
Co-authored-by: Paul Shealy <paulsh@microsoft.com>
Co-authored-by: Abraham Omorogbe <abrahamomor@gmail.com>
Co-authored-by: Ankit Singhal <30610298+singankit@users.noreply.github.com>
Co-authored-by: Hugo Aponte <haponte@gmail.com>
Co-authored-by: Lisa Li (STC) <lisal@microsoft.com>
Co-authored-by: mx-iao <33701412+mx-iao@users.noreply.github.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Roope Astala <rastala@users.noreply.github.com>
Co-authored-by: Swati Gharse <36742198+swatig007@users.noreply.github.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Hanchi Wang <hancwang@microsoft.com>
Co-authored-by: Daniel Schneider <danielsc@microsoft.com>
* Pipeline April SDK Preview (#1130)
* minor change
* update
* update
* make prepare step pass
* refine cli yaml: display_name
* fix cifar sample
* fix notebook
* fix: upgrade build version
* fix: fix 4d_dataset_input
* update keras sample
* ci: trigger ci on april-sdk-preview
* fix: do not always checkout april sdk preview
* fix: eval_mount is supported in new sdk/cli
* fix: merge main image classification
* fix: fix ci
* update dev build version
* fix: recover dataset related ci
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: Ying Chen <2601502859@qq.com>
* Pipeline April SDK Preview (#1137)
* fix: reduce 6a/6b instance count
* fix: add asset creation to run-job-pipeline-all
* fix: remove repeating cli job status
* fix: use uri_folder instead of path
* fix ci : assets/component
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
* update SDK/CLI test for force_rerun (#1140)
* update sdk code for rerun
* update cli test for rerun
* revert change in notebook
* add rerun in sdk ci pipeline
* revert
* add rerun in sdk ci pipeline
* update readme.py
* Pipeline SDK Preview Features (#1138)
* add component archive/restore, customize output
* add name to path
* refine image classification sample
* clear outputs.
* Refine component notebook
* Rai pipeline sample (#1139)
* Add RAI CLI sample pipeline
* add github workflow
* update pipeline yaml postfix
* add pre-setup steps
* update workflow to correct name
* add rai sample
* add rai sdk sample ci workflow
* change name to 2f
* update schema refer and workflow
* update readme and workflow name
* rename workflow to correct name
* minor update
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Honglin Du <0mza987@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
* update workflow & readme
* refine rai sample
* update
* update
* update readme
* update
Co-authored-by: lochen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Honglin Du <0mza987@gmail.com>
* AutoML preview PR (Python SDK and CLI examples) (#1089)
* Baseline of AutoML Classification task notebook
* Updated to code and wheel 3-17-2022
* Updates for new wheel
* Forecasting baseline
* Fixed Forecasting notebook now working
* Deleted not used folders
* Added current Custom Environments
* Classifier notebook updated
* Forecasting notebook updated
* Baseline for Regression sample notebook
* WIP baseline AutoML for Images
* Updated to use URI-FOLDER data asset for Images
* Updates working Image-Classification with MLTables
* Baseline for AutoML NLP Text Classif multiclass
* Updated Images notebook to use Curated Env
* Updated NLP notebook with Curated Env
* Updates on NLP and minor IMG
* Minor updates, working notebooks
* Add notebooks for rest of image tasks (#1077)
* Special PrP properties updated for all ntbooks
* Temporal WIP MLFlow
* MLFlow updates
* Baseline AutoML CLI example
* Updates SDK AutoML-MLFlow notebook
* Updates to hello AutoML CLI YAMLs
* Updates to CLI AutoML examples
* adding test github action for automl classification
* renamed the automl workflow
* Minor updates to CLI YAML examples
* Updates to MLFlow-AutoML notebook
* Updated _automl_internal_save_mlflow
* Updated mlflow-automl notebook with more MLTables
* delete old MLTable
* Minor update to YAML config
* CLI examples updates forecasting and classif
* adding automl workflows
* modifying notebook to wait for completion
* fixing workflows.
* changing notebooks to fetch create MLClient from config and wait for experiment to complete.
* update trials to 5 to overcome bug.
* updating to latest private wheel
* fixing classification & regression notebooks
* comment out set_limits for image notebooks
* fixing image object detection
* parameterizing automl notebooks with compute name
* fixing notebook whitespaces
* adding new line and remove obsolete action workflow
* fixing mltable path
* cleaning notebooks, running nb_black
* changing automl cli scripts
* adding AutoML CLI workflows.
* Porting AutoML samples into automl-preview. (#1101)
* adding automl workflows
* modifying notebook to wait for completion
* fixing workflows.
* changing notebooks to fetch create MLClient from config and wait for experiment to complete.
* update trials to 5 to overcome bug.
* updating to latest private wheel
* fixing classification & regression notebooks
* comment out set_limits for image notebooks
* fixing image object detection
* parameterizing automl notebooks with compute name
* fixing notebook whitespaces
* adding new line and remove obsolete action workflow
* fixing mltable path
* cleaning notebooks, running nb_black
* changing automl cli scripts
* adding AutoML CLI workflows.
* adding parameter to instance segmentation notebook.
* Updates on getting MLFlow tracking URI
* Add yaml cli examples for image tasks (#1104)
* revert to using macros for workspace setup in Bankmarketing example.
* Fix paths in jsonl files for image notebooks (#1105)
* Updates for MLFlow URI and other
* Fix json issue
* Subscription, workspace.. aligned..
* merged from parent branch.
* removing non-automl actions.
* Forecasting updates
* Minor updates on classifc notebook
* Update on "data" folder for Images notebook
* Workspace IDS for macros fixed
* data folder update for regression notebook
* Clean-up output images multi-class
* Several fixes to image-classif-multi-label
* fixing the resource group name macro.
* Minor clean up in image classif notebooks
* updating the automl actions to refer automl-preview branch
* branch name changes in actions.
* Object Detection updated to use "./data"
* Updates on "./data" to image segmentation
* Clean up image segmentation
* Updates working Image classif multi-label and segm
* Commented setLimits to Image multi-label
* update encoding to utf8 (#1107)
Co-authored-by: Miseon Park <mipark@microsoft.com>
* remove set_limits from image object detection notebook.
* add NLP cli examples (#1108)
Co-authored-by: Miseon Park <mipark@microsoft.com>
* modify automl CLI workflows
* adding sdk-preview to the list of branches to monitor for CLI actions.
* ensuring all AutoML CLI workflows get triggered.
* fixing CLI setup script
* fixing additional properties for classification & forecasting.
* CLI Regression example baseline
* pinning to latest CLI & SDK & updating actions for CLI.
* Updated Regression CLI YAML working
* attempting to fix the bankmarketing cli example.
* fix setup.sh for sdk.
* adding regression CLI action.
* remove drop_columns from featurization
* update automl cli yamls to camel case and latest schemas
* removing CLI actions from running on PRs on automl-preview branch
* Fix set_limits() for image tasks. Fix image classification yaml (#1110)
* Fix set_limits() for image tasks. Fix image classification yaml
* CLI example fixes for schema changes
* Don't specify a version while uploading data folder
* Add scripts to prepare data for image cli jobs (#1120)
* AutoML April preview - SDK + CLI (#1121)
* Address CLI breaking changes
* Update nyc_taxi_data_regression.ipynb
* Updating pipelines
* Inline documentation updates
* Incorporating Sam's PR into this branch
* Installing new build
* Adding temp workflows for automated testing of notebooks
* Update model.ipynb
* Update model.ipynb
* Reverting model to sdk-preview version
* Update setup.sh
* Update setup.sh
* Adding a new build
* Removing all workflows from this branch
* Create sdk-assets-model-model.yml
* Update dev-requirements.txt
* Removing local path, adding @latest
* Updating to latest version of SDK/CLI and adding ipython_genutils
* Adding workflows for assets
* Update dev-requirements.txt
* Updating to Build # 58490278
* Adding tests for single step jobs
* Adding tests for endpoints
* Adding test workflows for pipelines
* Adding all test workflows
* Removing dataset notebook since we now have data
* Automated README.md
* Workflows for this branch
* Updating to latest build
* Update mnist-nonmlflow.ipynb
* Updating to latest build
* Adding description metadata to notebooks
* Improving README and Workflow generation
* Lochen/sdk march (#1041)
* innit
* init
* Lochen/sdk march (#1042)
* innit
* init
* merge folder
* fix pipeline sample: cifar-10, tf_mnist
* update workflows for pipeline
* Lochen/sdk march (#1044)
* innit
* init
* merge folder
* change e2e registered, basic, nyc
* basic pipeline nyc
* add instructions.
* update nyc
* fix basic pipeline
* Upgrade to latest build
* Adjusting workflows and README for new pipeline samples
* Dummy commit for testing
* Update README.md
* Adding descriptions for pipeline notebooks
* Updating to latest build
* Removing datastores from testing and compute from scheduled runs
* Update lightgbm-iris-sweep.ipynb
* Updating to latest build
* Update README.md
* Fixing source directory for data and updating CLI/SDK build
* Updating to latest build
* Update README.md
* Adding clean up for endpoints and workspaces
* Update workspace.ipynb
* Updating to new build
* Update README.md
* Fixing issues with workspace notebook
* Update setup.sh
* Update setup.sh
* create new branch
* adapt sweep sample to april release
* adapt to command based job creation
* Updating workflows and readme to point to the april branch
* Updating CLI README to show April branch status
* Updating CLI workflows to trigger in april branch
* Changing workflows to trigger on april branch
* Builder fix up
* fix typo
Co-authored-by: Bala P V <balapv@microsoft.com>
Co-authored-by: Bala P V <33712765+balapv@users.noreply.github.com>
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Daniel Schneider <schneider@volkskammer.org>
Co-authored-by: Hanchi Wang <hancwang@microsoft.com>
Co-authored-by: Daniel Schneider <danielsc@microsoft.com>
Co-authored-by: Gaurav Singh <gasi@microsoft.com>
* cleanup
* renaming the folder for automl CLI jobs
* removing AutoML CLI workflows.
* regenerated all the CLI workflows
* regenerated SDK workflows
* Minor changes for Workspace prop showup
* Minor changes to AutoML-MLFlow notebook
* Undo non-automl files
* Add jsonl files pointing to GH actions workspace for automl images (#1136)
* Added MLFlow model local test
* Removed MLFlow inference to point to next notebook
* Update cli-jobs-automl-standalone-jobs-cli-automl-classification-task-bankmarketing-cli-automl-classification-task-bankmarketing.yml
testing
* Update cli-jobs-automl-standalone-jobs-cli-automl-classification-task-bankmarketing-cli-automl-classification-task-bankmarketing.yml
revert
* ml_flow.set_experiment() not needed, removed
* Forecasting updates for MLFlow
* regenerate automl workflows for april branch merge test
* Update cli-automl-classification-task-bankmarketing.yml
* Update cli-automl-classification-task-bankmarketing.yml
* remove bankmarketing mlflow.yml from workflows
* update to specs
Co-authored-by: PhaniShekhar <30535191+PhaniShekhar@users.noreply.github.com>
Co-authored-by: Razvan Tanase <ratanase@microsoft.com>
Co-authored-by: Miseon <msp8955@gmail.com>
Co-authored-by: Miseon Park <mipark@microsoft.com>
Co-authored-by: Gaurav Singh <svaruag@outlook.com>
Co-authored-by: Bala P V <balapv@microsoft.com>
Co-authored-by: Bala P V <33712765+balapv@users.noreply.github.com>
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Daniel Schneider <schneider@volkskammer.org>
Co-authored-by: Hanchi Wang <hancwang@microsoft.com>
Co-authored-by: Daniel Schneider <danielsc@microsoft.com>
Co-authored-by: Gaurav Singh <gasi@microsoft.com>
Co-authored-by: Srujan Saggam <41802116+srsaggam@users.noreply.github.com>
* Pipeline SDK/CLI April Features (#1144)
* update add missing pipeline workflows
* update sdk workflows & readme
* add force rerun
* move dsl files to src folder (#1145)
* update data format
* remove invalid workflow
* update RAI samples: 1, update register data name to align with the on… (#1146)
* update RAI samples: 1, update register data name to align with the one in RAI repo. 2, SDK sample reuse the component from CLI one. 3, SDK use a more complext pipeline. 4, update readme
* remove some unuse comment
Co-authored-by: Anthony Hu <anthu@microsoft.com>
* fix: fix sweep cli example
* parallel run notebooks
* fix encoding
* add dry run
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Korin <0mza987@gmail.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Added .TXT files on how to run CLI AutoML (#1150)
* Added .TXT on how to run CLI AutoML
* Comment added about GPU cluster
* blocked_algos commented due to bug
* Removed dev temporal internal properties
* Prep data explanation sentence fixed
* Lochen/april release 0413 (#1147)
* update add missing pipeline workflows
* update sdk workflows & readme
* add force rerun
* move dsl files to src folder (#1145)
* update data format
* remove invalid workflow
* update RAI samples: 1, update register data name to align with the on… (#1146)
* update RAI samples: 1, update register data name to align with the one in RAI repo. 2, SDK sample reuse the component from CLI one. 3, SDK use a more complext pipeline. 4, update readme
* remove some unuse comment
Co-authored-by: Anthony Hu <anthu@microsoft.com>
* fix: fix sweep cli example
* parallel run notebooks
* fix encoding
* add dry run
* data asset dataset, remove pipeline level input
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Korin <0mza987@gmail.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Updates to workflows and build Numbers
* Add files via upload
* Add files via upload
* Add files via upload
* Updating workflows
* fix links (#1156)
* Moving to DeafultAzureCredential instead of Browser Auth. Removing branch check-out.
* Adding AutoML CLI tests, removing apr branch checkout
* Moving from commandjob to command
* Fixing pipeline credentials
* Update AutoML Job schema urls (#1160)
* Using requirements from SDK folder. Adding pandas to requirements
* remove automl-preview reference (#1143)
* remove mlflow WIP notebooks from github actions
* remove pandas reference
* remove pandas after merge
Co-authored-by: Miseon Park <mipark@microsoft.com>
* Pipeline April SDK/CLI preview samples refine (#1157)
* merge
* update sample to make compute config lines fewer
* make configure credential simpler
* update output setting of nyc_tax_data_regression
* refine rai samples
* update sdk ci workflow
* update
* fix rai
* use newer dev build before nightly build issue resolved
* update workflow
* Rename sample node name to align with pipeline node restriction (#1158)
* fix name
* fix name
* fix sample
* fix
* fix
Co-authored-by: Han Wang <wanhan@microsoft.com>
* Update pipeline sdk samples to use azure.ml.Input/Output
* update setup.sh
* update
* refine
* feat: update run-job-pipeline-all auto-gen
* fix: fix image classification example
* update setup.sh
* update Input
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Updating to latest build
* Update prefix.md
* Update prefix.md
* Update import paths for automl entities in image notebooks (#1164)
* Update cli-automl-classification-task-bankmarketing.yml
* Update cli-automl-forecasting-task-energy-demand.yml
* Update cli-automl-forecasting-task-energy-demand.yml
* Pipeline April CLIL/SDK Feature refine (#1161)
* remove mlflow workaround
* remove mlflow molde workaround
* update sdk nyc sample
* resolve components ci failure
* add configuration notebook
* simplify pipeline_with_components_from_yaml notebook
* fix hello-pipeline-customize-output-folder.yml
* simplify samples
* refine
* update workflow and readme
* update
* add display name for hello pipelines
* update
Co-authored-by: lochen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* Updating to latest build
* Fixing links in docs
* Delete online-inferencing-sample.ipynb
* Update automl-classification-task-bankmarketing-mlflow.ipynb
* Converting all links to configuration.ipynb file into a relative link
* Fixing links to README file
* Fixing links to other files and folders
* Hancwang/april (#1168)
* Clean up iris
* demo notebook
* notebook used in demo
* namespace update
* more import fix
* remove studio_url
* more conflicts
* revert readme
* Fix build error
* Update hello-automl-job-basic.yml
* Pipeline April SDK Feature samples refine (#1175)
* remove refrence of azure.ml._constants
* revert notebook rename
* update
* update to latest master build
* fix run all
* fix SDK sample link
* update pipeline sdk 2e sample (#1176)
* update pipeline sdk 2e sample
* update compute
* fix credential part
* adapt to latest version sdk
* update
* update
* update
* update
* update conda aml sdk version
* update according to comments
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: Blanca Li <39058063+likebupt@users.noreply.github.com>
* Updating nightly build
* Update namespace from azure.ml._constants to azure.ml.constants (#1174)
Co-authored-by: Clement Wang <clwan@microsoft.com>
* Updating description for preview
* endpoints named by sample (#1180)
* add custom featurization to tests (#1172)
Co-authored-by: Miseon Park <mipark@microsoft.com>
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Daniel Schneider <schneider@volkskammer.org>
Co-authored-by: Hanchi Wang <hancwang@microsoft.com>
Co-authored-by: Daniel Schneider <danielsc@microsoft.com>
Co-authored-by: Clement Wang <47586720+wangchao1230@users.noreply.github.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
Co-authored-by: Ying Chen <2601502859@qq.com>
Co-authored-by: chenyin <chenyin@microsoft.com>
Co-authored-by: Korin <0mza987@gmail.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
Co-authored-by: Zeliang Tian <83852443+zetiaatgithub@users.noreply.github.com>
Co-authored-by: Vadthyavath Ram <7171558+vadthyavath@users.noreply.github.com>
Co-authored-by: Razvan Tanase <ratanase@microsoft.com>
Co-authored-by: Cody Peterson <54814569+lostmygithubaccount@users.noreply.github.com>
Co-authored-by: Harneet Virk <harnvir@microsoft.com>
Co-authored-by: Yuan Zhuang <40252949+yuanzhuangyuanzhuang@users.noreply.github.com>
Co-authored-by: Yuan Zhuang <yuzhua@microsoft.com>
Co-authored-by: Lifeng Wei <lfwei@ucdavis.edu>
Co-authored-by: Lifeng Wei <lifengwei@microsoft.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
Co-authored-by: Sethu Raman <rsethur@users.noreply.github.com>
Co-authored-by: jeff-shepherd <39775772+jeff-shepherd@users.noreply.github.com>
Co-authored-by: v-pbavanari <97779269+v-pbavanari@users.noreply.github.com>
Co-authored-by: Banibrata De <banibrata@outlook.com>
Co-authored-by: Kayla Seager <1675997+kseager@users.noreply.github.com>
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
Co-authored-by: Paul Shealy <paulsh@microsoft.com>
Co-authored-by: Abraham Omorogbe <abrahamomor@gmail.com>
Co-authored-by: Ankit Singhal <30610298+singankit@users.noreply.github.com>
Co-authored-by: Hugo Aponte <haponte@gmail.com>
Co-authored-by: Lisa Li (STC) <lisal@microsoft.com>
Co-authored-by: mx-iao <33701412+mx-iao@users.noreply.github.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Roope Astala <rastala@users.noreply.github.com>
Co-authored-by: Swati Gharse <36742198+swatig007@users.noreply.github.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Cesar De la Torre <cesardl@microsoft.com>
Co-authored-by: PhaniShekhar <30535191+PhaniShekhar@users.noreply.github.com>
Co-authored-by: Miseon <msp8955@gmail.com>
Co-authored-by: Miseon Park <mipark@microsoft.com>
Co-authored-by: Gaurav Singh <svaruag@outlook.com>
Co-authored-by: Gaurav Singh <gasi@microsoft.com>
Co-authored-by: Srujan Saggam <41802116+srsaggam@users.noreply.github.com>
Co-authored-by: Hanchi Wang <luigiking307@gmail.com>
Co-authored-by: Blanca Li <39058063+likebupt@users.noreply.github.com>
* Adjusting workflows and readme to point to correct branch
* SDK preview - pipeline samples refine (#1183)
* remove unnecesarry files
* update imports in pipeline samples
* remove is_deterministic: False
* update 2e sample (#1184)
* refine wording
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Blanca Li <39058063+likebupt@users.noreply.github.com>
* Reformatting python files with black
* Formatting notebooks with black
* Revise e2e vision notebook to use commands and sdk april release (#1167)
* remove coco dependencies in component
* remove spec
* revise requirements
* revise notebook to use commands only
* adapt to april release
* restore notebooks
* remove pipeline snapshot
* update the e2e-ds-experience nb to april release (#1185)
Co-authored-by: amah <ma.mahmoudzadeh@gmail.com>
* Adding metadata to refer from docs
* Pipeline SDK Preview Samples Refine (#1187)
* update component notebook description
* remove enum before issue resolved
* refine component notebook
* add description
* refine 1e sample
* revert enum
* fix cli ci
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
* add aml arc inferencing notebook (#1194)
* add notebooks for amlarc inferencing
* add workflow for amlarc inferencing notebook
* fix: specify gpu related environment version (#1199)
* Update sdk-endpoints-online-amlarc-online-endpoints-safe-rollout.yml (#1200)
fix ipynb file name issue
* update 2e sample (#1201)
* update pipeline job sdk 2e sample
* update description of the pipeline
* comment invoke method for aks policy (#1203)
comment invoke out as cluster under dev subscription can't be accessed from public internet.
* Update online-endpoints-simple-deployment.ipynb
* add k8s prefix for amlarc online endpoint name (#1212)
* Update online-endpoints-safe-rollout.ipynb
add k8s prefix for amlarc endpoint name
* feat: use mlflow in nyc-taxi-data-regression (#1208)
* feat: use mlflow to save model in nyc_taxi_data_regression
* feat: optimize run-job-pipeline related shells
* Uncomment allowed and blocked models for tabular tasks (#1196)
* Uncomment allowed and blocked models for tabular tasks
* fix forecasting
Co-authored-by: Ayush Mishra <ayushmishra@microsoft.com>
* add metadata to sdk notebook examples (#1217)
Co-authored-by: Miseon Park <mipark@microsoft.com>
* update to May API version (#1223)
* Update main.py
* rename amlarc to kubernetes (#1219)
* rename amlarc to kubernetes
Co-authored-by: Xue Wei <xuewe@microsoft.com>
* enable download for rai pipeline (#1221)
* enable download for rai pipeline
* remove empty cell
* refine
* Add metadata tags for automl images (#1242)
* Add metadata tags for automl images
* Separate cells
* Minor metadat
* Update kubernetes-online-endpoints-safe-rollout.ipynb (#1251)
* update pipeline sdk 2e sample (#1260)
* added k8s related workflows result (#1266)
* add cell name for doc reference (#1267)
* Fixes to e2e dl distributed pytorch notebook from bugbash (#1255)
* implement details
* add compute test
* update 2e sample (#1265)
* update 2e sample
* update
* update
* Changing namespace from azure.ml to azure.ai.ml
* Changing JobInput to Input in AutoML
* Removing JobInput and azure.ml from files
* Bug FIxes
* Delete old endpoint sample test
* Adding metadata for cells - needed for docs
* [Pipeline] Fix azure-ml -> azure-ai-ml (#1282)
* update azure-ml version in conda.yml
* update @dsl.pipeline -> @pipeline
* update build version
Co-authored-by: Clement Wang <clwan@microsoft.com>
* update 2e sample file and related reference (#1289)
* Updating to new builds
* Update lightgbm-iris-sweep.ipynb
* sync workflow scripts from main to sdk-preview branch (#1300)
* Update setup.sh
* Update setup.sh
* Update setup.sh
* Update setup.sh
* update train.py. r (#1299)
emove mlflow related comment to avoid confuse audience.
* Sync notebook with doc (#1291)
* for docs tutorial
* Updating tutorial to match docs
* run black on this
* remove dataset
* rename notebook
* add alert for cpu-name
* add note, fix intro
* Updating daily build
* Update load_component and switch other YAML loading to use load_x() (#1314)
* Update load_component and switch other YAML loading to use load_x()
* Update to the latest private wheel
* Remove urireference
* sync scripts (#1310)
* managed vnet for managed online endpoints (#1303)
* vnet sample
* update
* updates
* updates
* updates
* update dir
* scripts
* add model
* updates
* update scripts
* cleanup
* cleanup
* cleanup
* parameterize the branch for testing
* test with pr branch
* test sample branch
* test branch
* test branch
* fix the branch name
* automated vnet test: fix the branch name
* validate scoring
* fix script
* fix script
* update the setup instructions
* update comments
* fix formatting
* additional comments
* fix review comments
* update scoring test scripts
* update keyvault naming
* update build script
* reorganize directories
* update path
* update pr check
* update
* remove uai to seperate file
* mlflow vnet sample
* mlflow vnet sample
* mlflow vnet example
* mlflow vnet example
* docs comments
* mirror traffic
* update to new syntax
* test
* testing with debug
* debug
* revert setup.sh changes
* debug
* latest cli
* updated cli
* update cli version
* adding local deployment
* local deployment
* test
* local deployment
* new cli build
* new cli version
* latest cli version
* testing fix for apt update error
* fix update error
* remove unused packages
* newer version of cli
* remove private preview flags
* custom cli version for testing from vm
* Users/mipark/update automl standalone examples (#1316)
* cli updates
* sdk updates
* add model deploy and local inference examples
* update readme
Co-authored-by: Miseon Park <mipark@microsoft.com>
* Updating to latest builds
* Add notebooks for local model inference. Model deployment (#1317)
* remove data settings from hello automl notebook (#1319)
* cli updates
* sdk updates
* add model deploy and local inference examples
* update readme
* update automl basic example
* update workflows
Co-authored-by: Miseon Park <mipark@microsoft.com>
* Update setup.sh
* Updating to latest build
* Removing rw_mount from samples
* pick non private preview change (#1326)
* pick non private preview change
* update samples
* simplify @pipeline usage
* remove description and name for pipeline, add doc string instead
* fix: fix readme.md for rai sample
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Honglin Du <0mza987@gmail.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
* update mldesigner install (#1333)
Co-authored-by: Han Wang <wanhan@microsoft.com>
* fix automl examples (#1331)
* fix examples
* Correct timeout_minutes in Object detection notebook
* Correct import paths in prepare_data.py files
Co-authored-by: Miseon Park <mipark@microsoft.com>
Co-authored-by: Phani Shekhar Mantripragada <phmantri@microsoft.com>
* updating rest api version (#1329)
Co-authored-by: Raghu Ramaswamy <raghura@microsoft.com>
* Reformat notebooks, update latest build, improve cleanup and README generation
* Reformatting notebooks
* Reformatting some more notebooks
* Updating README and workflows for SDK v2
* Updating CLI workflows
* Updating to use defaultAzure Creds
* updated the e2e-ds-notebook to may release (#1325)
* updated the e2e-ds-notebook to may release
* cleaned notebook cell outputs
* fixed some terms to the new ones
Co-authored-by: amah <ma.mahmoudzadeh@gmail.com>
* Changes to fix online endpoint issues
* Fixing Online Deployment scripts
Co-authored-by: Cloga Chen <cloga0216@gmail.com>
Co-authored-by: Dennis Eikelenboom <92387505+deeikele@users.noreply.github.com>
Co-authored-by: Tracy Chen <47156502+tracychms@users.noreply.github.com>
Co-authored-by: Hanchi Wang <luigiking307@gmail.com>
Co-authored-by: Cody <54814569+lostmygithubaccount@users.noreply.github.com>
Co-authored-by: Paul Shealy <paulsh@microsoft.com>
Co-authored-by: abeomor <abrahaomor@gmail.com>
Co-authored-by: amah <mamahmoudzadeh@gmail.com>
Co-authored-by: amah <ma.mahmoudzadeh@gmail.com>
Co-authored-by: Jeff Omhover <jf.omhover@gmail.com>
Co-authored-by: Janhavi Mahajan <janhavi13@users.noreply.github.com>
Co-authored-by: Banibrata De <banibrata@outlook.com>
Co-authored-by: Clement Wang <clwan@microsoft.com>
Co-authored-by: Ami Patel <85967395+amibp@users.noreply.github.com>
Co-authored-by: Shivani Santosh Sambare <61255242+shivanissambare@users.noreply.github.com>
Co-authored-by: Daniel Schneider <schneider@volkskammer.org>
Co-authored-by: Hanchi Wang <hancwang@microsoft.com>
Co-authored-by: Daniel Schneider <danielsc@microsoft.com>
Co-authored-by: Clement Wang <47586720+wangchao1230@users.noreply.github.com>
Co-authored-by: Han Wang <wanhan@microsoft.com>
Co-authored-by: elliotzh <37076709+elliotzh@users.noreply.github.com>
Co-authored-by: Ying Chen <2601502859@qq.com>
Co-authored-by: chenyin <chenyin@microsoft.com>
Co-authored-by: Korin <0mza987@gmail.com>
Co-authored-by: zhangxingzhi <zhangxingzhi@microsoft.com>
Co-authored-by: Han Wang <phoenix.seek@gmail.com>
Co-authored-by: Zeliang Tian <83852443+zetiaatgithub@users.noreply.github.com>
Co-authored-by: Vadthyavath Ram <7171558+vadthyavath@users.noreply.github.com>
Co-authored-by: Razvan Tanase <ratanase@microsoft.com>
Co-authored-by: Harneet Virk <harnvir@microsoft.com>
Co-authored-by: Yuan Zhuang <40252949+yuanzhuangyuanzhuang@users.noreply.github.com>
Co-authored-by: Yuan Zhuang <yuzhua@microsoft.com>
Co-authored-by: Lifeng Wei <lfwei@ucdavis.edu>
Co-authored-by: Lifeng Wei <lifengwei@microsoft.com>
Co-authored-by: Sethu Raman <rsethur@users.noreply.github.com>
Co-authored-by: jeff-shepherd <39775772+jeff-shepherd@users.noreply.github.com>
Co-authored-by: v-pbavanari <97779269+v-pbavanari@users.noreply.github.com>
Co-authored-by: Kayla Seager <1675997+kseager@users.noreply.github.com>
Co-authored-by: Kayla Ames <kseager@users.noreply.github.com>
Co-authored-by: Abraham Omorogbe <abrahamomor@gmail.com>
Co-authored-by: Ankit Singhal <30610298+singankit@users.noreply.github.com>
Co-authored-by: Hugo Aponte <haponte@gmail.com>
Co-authored-by: Lisa Li (STC) <lisal@microsoft.com>
Co-authored-by: mx-iao <33701412+mx-iao@users.noreply.github.com>
Co-authored-by: Anthony Hu <anthu@microsoft.com>
Co-authored-by: Roope Astala <rastala@users.noreply.github.com>
Co-authored-by: Swati Gharse <36742198+swatig007@users.noreply.github.com>
Co-authored-by: eniac871 <supereniac@live.com>
Co-authored-by: Cesar De la Torre <cesardl@microsoft.com>
Co-authored-by: PhaniShekhar <30535191+PhaniShekhar@users.noreply.github.com>
Co-authored-by: Miseon <msp8955@gmail.com>
Co-authored-by: Miseon Park <mipark@microsoft.com>
Co-authored-by: Gaurav Singh <svaruag@outlook.com>
Co-authored-by: Gaurav Singh <gasi@microsoft.com>
Co-authored-by: Srujan Saggam <41802116+srsaggam@users.noreply.github.com>
Co-authored-by: Blanca Li <39058063+likebupt@users.noreply.github.com>
Co-authored-by: snowei <snowei1988@gmail.com>
Co-authored-by: Ayush Mishra <61145377+novaturient95@users.noreply.github.com>
Co-authored-by: Ayush Mishra <ayushmishra@microsoft.com>
Co-authored-by: Xiao Zhang <zhanxia@microsoft.com>
Co-authored-by: Sheri Gilley <sgilley@microsoft.com>
Co-authored-by: Phani Shekhar Mantripragada <phmantri@microsoft.com>
Co-authored-by: Raghu R <13340619+raghutillu@users.noreply.github.com>
Co-authored-by: Raghu Ramaswamy <raghura@microsoft.com>
5285f9c7e8
