From 255f6a63e045b05f655dbdd65ea00eedabe19d28 Mon Sep 17 00:00:00 2001 From: konabuta Date: Thu, 24 Jun 2021 05:40:20 +0000 Subject: [PATCH 1/8] command to start docsify --- .devcontainer/devcontainer.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json index 0f0b13d..f460705 100644 --- a/.devcontainer/devcontainer.json +++ b/.devcontainer/devcontainer.json @@ -22,7 +22,7 @@ // "forwardPorts": [], // Use 'postCreateCommand' to run commands after the container is created. - // "postCreateCommand": "yarn install", + // "postCreateCommand": "docsify serve .", // Comment out connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root. "remoteUser": "node" From fcc1faa2fa61cc01e458284b7e20fb44acf36ced Mon Sep 17 00:00:00 2001 From: konabuta Date: Thu, 24 Jun 2021 05:40:57 +0000 Subject: [PATCH 2/8] add new terraform templates --- technical/architecture.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/technical/architecture.md b/technical/architecture.md index 7c90eca..d6e9c21 100644 --- a/technical/architecture.md +++ b/technical/architecture.md @@ -34,6 +34,8 @@ Azure Kubernetes Services (AKS) 利用時の構成です。
Private Endpoint / Private Link を利用した環境構築の手順ガイド - [Welcome to step-by-step guide to provision secure workspace](https://github.com/jhirono/amlsecurity) +Terraform を用いた環境構築テンプレート +- [Terraform examples for provisioning Azure Machine Learning](https://github.com/microsoft/azureml-terraform-examples)
## Data Ingestion Pattern From 287b424dee8d1147cdaebf0315fdb7e7fa051abc Mon Sep 17 00:00:00 2001 From: konabuta Date: Tue, 17 Aug 2021 06:58:37 +0000 Subject: [PATCH 3/8] workshop 2021, onnx runtime for training sample --- technical/sample.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/technical/sample.md b/technical/sample.md index 1f1a2d8..bfe2568 100644 --- a/technical/sample.md +++ b/technical/sample.md @@ -83,6 +83,10 @@ - [Responsible AI Workshop](https://github.com/konabuta/responsible-ai) - モデル解釈・説明性のテクノジーについてのワークショップコンテンツ。`Explainable Boosting Machine` と `SHAP` の実装。 +- [Azure Machine Learning Workshop 2021](https://github.com/konabuta/azureml-workshop-2021) - Azure Machine Learning 4 days ワークショップ + +- [ONNX Runtime for model training](https://github.com/microsoft/onnxruntime-training-examples) - ONNX Runtime を用いた Transformer モデル学習の高速化サンプル +
From fd362e2b3fbf4ab0c9be1ee07b03e5f80a0fcf8b Mon Sep 17 00:00:00 2001 From: konabuta Date: Tue, 17 Aug 2021 14:15:47 +0000 Subject: [PATCH 4/8] add coursera --- technical/learning.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/technical/learning.md b/technical/learning.md index 1200f00..247eb97 100644 --- a/technical/learning.md +++ b/technical/learning.md @@ -23,6 +23,10 @@ - [Udacity : Become a Machine Learning Engineer for Microsoft Azure](https://www.udacity.com/course/machine-learning-engineer-for-microsoft-azure-nanodegree--nd00333) - Azure Machine Learning の Nanodegree Program +- [Coursera - Microsoft](https://www.coursera.org/instructor/microsoft) + - [Azure Machine Learning](https://www.coursera.org/learn/microsoft-azure-machine-learning) + - [Artificial Intelligence on Microsoft Azure](https://www.coursera.org/learn/artificial-intelligence-microsoft-azure) + - [Modern Data Warehouse Analytics in Microsoft Azure](https://www.coursera.org/learn/data-warehouse-analytics-microsoft-azure) ## コミュニティ From 52891e11f228955ab28011b76919130b2dfa5595 Mon Sep 17 00:00:00 2001 From: konabuta Date: Thu, 4 Nov 2021 14:38:18 +0000 Subject: [PATCH 5/8] add Feast, modify nlp sample repo, modify automl --- technical/sample.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/technical/sample.md b/technical/sample.md index bfe2568..cb0ca8d 100644 --- a/technical/sample.md +++ b/technical/sample.md @@ -8,7 +8,11 @@ - [Azure ML 2.0 Developer Experience](https://github.com/Azure/azureml-v2-preview) - Azure ML 2.0 API/CLI/SDK (private preview) -- [AutoML for Computer Vision](https://github.com/swatig007/automlForImages) - sample code of automl for computer vision tasks (classification, object detection, instance segmentation) +- [AutoML for Image](https://github.com/swatig007/automlForImages) - sample code of automl for computer vision tasks (classification, object detection, instance segmentation) + +## Tool + +- [Feast on Azure](https://github.com/Azure/feast-azure) - Azure plugins for Feast (FEAture STore) ## Solution Accelerators @@ -53,7 +57,7 @@ ||年収予測モデルの開発と公平性の評価と軽減| [Responsible AI Widgets - Samples](https://github.com/microsoft/responsible-ai-widgets/blob/main/notebooks/fairness-dashboard-loan-allocation.ipynb)| ||後方互換性を考慮した年収予測モデルの開発|[Backward Compatibility ML - Samples](https://github.com/microsoft/BackwardCompatibilityML/blob/dev/examples/compatibility-analysis-adult.ipynb)| ||患者のガン診断モデルと影響した変数の抽出|[InterpretML - Samples](https://github.com/interpretml/interpret-community/blob/master/notebooks/explain-binary-classification-local.ipynb)| -|Text Analytics |Livedoor ニュース記事の分類|[AzureML-NLP (日本語)](https://github.com/konabuta/AzureML-NLP) | +|Text Analytics |Livedoor ニュース記事の分類|[NLP Samples (日本語)](https://github.com/Azure/nlp-samples) | |Credit Analytics |住宅ローンの与信モデル構築と反実仮想サンプルの生成 | :runner: under construction ([DiCE](https://github.com/interpretml/DiCE)をベースに作成中) | |Demand Forecasting |時系列予測 (統計手法、機械学習、深層学習)|[Forecasting Best Practices](https://github.com/microsoft/forecasting)| ||各商品ブランドの店舗ごとの大量モデル開発と推論| [Many Models Solution Accelerator](https://github.com/microsoft/solution-accelerator-many-models)| From 004598df1300dba06c01ccbd7c6e168dd3490345 Mon Sep 17 00:00:00 2001 From: konabuta Date: Thu, 4 Nov 2021 14:38:26 +0000 Subject: [PATCH 6/8] add portal site --- technical/techinfo.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/technical/techinfo.md b/technical/techinfo.md index ebdb310..f832f29 100644 --- a/technical/techinfo.md +++ b/technical/techinfo.md @@ -1,10 +1,15 @@ # :cloud: 技術情報 -## ドキュメント -* [Microsoft Docs の網羅検索](https://docs.microsoft.com/ja-jp/search/) - + +## サイト * Azure Machine Learning - [ホームページ](https://azure.microsoft.com/ja-jp/services/machine-learning/ ) - [製品ドキュメント](https://docs.microsoft.com/ja-JP/azure/machine-learning/) +- [Machine learning for data scientists](https://azure.microsoft.com/en-us/overview/ai-platform/data-scientist-resources/#overview) - Data Scientist 向けのポータルサイト +- [Artificial intelligence for developers](https://azure.microsoft.com/en-us/overview/ai-platform/dev-resources/) - Developer 向けのポータルサイト + + +## ドキュメント +* [Microsoft Docs の網羅検索](https://docs.microsoft.com/ja-jp/search/) ## ブログ From ce37dc09aa5d7ba7c96541230350e294f4ea9e18 Mon Sep 17 00:00:00 2001 From: konabuta Date: Thu, 4 Nov 2021 14:52:03 +0000 Subject: [PATCH 7/8] add solution accelerators --- technical/sample.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/technical/sample.md b/technical/sample.md index cb0ca8d..80772ac 100644 --- a/technical/sample.md +++ b/technical/sample.md @@ -24,6 +24,13 @@ - [Databricks Solution Accelerators](https://databricks.com/solutions/accelerators) - Databricks 社が提供するユースケースごとの Solution Accelerators +- [Purview Machine Learning Lineage Solution Accelerator](https://github.com/microsoft/Purview-Machine-Learning-Lineage-Solution-Accelerator) - Purview による Azure Machine Learning Lineage + +- [Commodity Price Prediction Solution Accelerator](https://github.com/microsoft/Azure-Synapse-Solution-Accelerator-Commodity-Price-Prediction) - 商品価格予測 の Solution Accelerator + +- [Content Recommendations Solution Accelerator](https://github.com/microsoft/Azure-Synapse-Content-Recommendations-Solution-Accelerator) - コンテンツ推薦システムの Solution Accelerator + +- [Customer Complaint Management Solution Accelerator](https://github.com/microsoft/Azure-Solution-Accelerator-Customer-Complaint-Management) - 顧客クレーム管理システムの Solution Accelerator ## Recipes - [Computer Vision](https://github.com/microsoft/computervision-recipes) - コンピュータービジョンのサンプル集 From 38ab9c9fd97192d43596e25f08a600df91302d60 Mon Sep 17 00:00:00 2001 From: konabuta Date: Fri, 5 Nov 2021 08:29:57 +0000 Subject: [PATCH 8/8] add tips --- technical/tips.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/technical/tips.md b/technical/tips.md index a19cd39..3a5769a 100644 --- a/technical/tips.md +++ b/technical/tips.md @@ -105,6 +105,16 @@ print(df.head()) +* 重複行データを取り除く方法 + + Azure Synapse Pipeline (Azure Data Factory) の Data Flow で簡単に除外すること可能です。 + - 参考 : [Azure Data Factory - Data Flowで重複行を取り除く (最初の行だけ選択する)方法メモ](https://zenn.dev/shohei_aio/articles/5c9716ac817b79) + +* Excel ファイル の各シートごとの処理の実行方法 + Azure Functions を用いてシート一覧を取得し、Azure Synapse Pipeline (Azure Data Factory) の DataFlow にて繰り返し処理することができます。 + - 参考 : [Excelのシート一覧をFunctionsで出力しForEach&DataFlowで加工する @Azure Synapse Pipeline](https://zenn.dev/shohei_aio/articles/a155ddfc8c9cab) + + ## データ探索 :runner: _coming soon_