microsoft
/
ai4eutils
зеркало из
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
Shared utility scripts for AI for Earth projects and team members
133 коммитов 8 Ветки 0 Теги 2,7 MiB
Jupyter Notebook 76.1%
HTML 13.8%
Python 10%
Shell 0.1%
Перейти к файлу
Dan Morris ecaa5ef90c Added scripts for enumerating folders in a blob container a specified depth, enumerating blobs in parallel in a large blob container, and removing empty folder in a local file system 2020-10-29 12:55:09 -07:00
TF_OD_API Added dockerfile and sh script to install TFODAPI (TF v 1.15) on Azure Ubuntu 16.04 DSVM (#6) 2020-01-22 12:17:48 -08:00
azcopy-bandwidth-test no message 2020-01-06 11:11:34 -08:00
azure-metrics-calc-storage-size ws only 2019-12-06 10:08:54 -08:00
azure-sdk-calc-storage-size Finish updating everything to Azure Blob Storage Python SDK v12 2020-07-16 16:05:44 -06:00
geospatial Minor modification to terminology section. 2020-10-01 15:47:09 -07:00
tests Update sas_blob_utils.py and unit tests 2020-09-11 01:10:20 +00:00
.gitignore Add utility functions for dealing with Azure blob storage SAS tokens. 2019-03-01 10:24:54 -08:00
LICENSE Initial commit 2018-12-07 14:40:39 -08:00
README.md Add directory structure to /geospatial. (#9) 2020-06-30 14:38:30 -07:00
ai4e_azure_utils.py Optionally allow overwrite in file upload to blob 2020-10-01 10:21:53 -07:00
ai4e_string_utils.py Removed string return value from human_readable_to_bytes 2020-08-11 11:49:53 -07:00
ai4e_web_utils.py verbosity control in download_url 2020-05-01 08:59:18 -07:00
back_up_blob_containers.py typo 2020-03-20 11:22:41 -07:00
directory_listing.py Sample invocation for directory_listing.py 2020-10-07 10:20:36 -07:00
enumerate_blob_folders_at_depth.py Added scripts for enumerating folders in a blob container a specified depth, enumerating blobs in parallel in a large blob container, and removing empty folder in a local file system 2020-10-29 12:55:09 -07:00
gDrive_download.py Added gDrive_download.py 2019-03-14 16:47:33 -07:00
matlab_porting_tools.py moved some path functions from matlab_porting_tools to the much-more-sensible path_utils 2019-08-09 15:27:48 -07:00
mypy.ini Add mypy.ini 2020-08-13 15:16:15 -07:00
parallel_enumerate_blobs.py Added scripts for enumerating folders in a blob container a specified depth, enumerating blobs in parallel in a large blob container, and removing empty folder in a local file system 2020-10-29 12:55:09 -07:00
path_utils.py Minor ws changes in path_utils 2020-09-21 12:50:47 -07:00
pylintrc Add directory structure to /geospatial. (#9) 2020-06-30 14:38:30 -07:00
remove_empty_folders.py Added scripts for enumerating folders in a blob container a specified depth, enumerating blobs in parallel in a large blob container, and removing empty folder in a local file system 2020-10-29 12:55:09 -07:00
sas_blob_utils.py Allowing optional leading question marks in SAS tokens 2020-09-22 07:22:35 -07:00
write_html_image_list.py Now removing unicode characters in write_html_image_list; unicode causes fwrite to fail in some scenarios. 2019-05-26 08:07:42 -07:00

README.md

Overview

Shared utilities developed by the Microsoft AI for Earth team

The general convention in this repo is that users who want to consume these utilities will add the top-level path of the repo to their Python path, so it's okay to assume that other packages/modules within the repo are available. The "scrap" directory can be used for standalone, one-time-use scripts that you might otherwise have emailed to someone.

Contents

  • path_utils.py: Miscellaneous useful utils for path manipulation, things that could almost be in os.path, but aren't.

  • matlab_porting_tools.py: A few ported Matlab functions that makes it easier to port other, larger Matlab functions to Python.

  • write_html_image_list.py: Given a list of image file names, writes an HTML file that shows all those images, with optional one-line headers above each.

  • sas_blob_utils.py: Helper functions for dealing with Shared Access Signatures (SAS) tokens for Azure Blob Storage.

  • TF_OD_API: A Dockerfile and a script to prepare a Docker image for use with the TensorFlow Object Detection API.

  • gDrive_download.py: Semi-automatic script for bulk download from shared Google Drives using the gDrive Python SDK.

  • azure-sdk-calc-storage-size: Script for recursively computing the size of all blobs and files in an Azure subscription.

  • azure-metrics-calc-storage-size: Script for computing the total size of all storage accounts in an Azure subscription (using Azure Metrics).

  • ai4e_azure_utils.py: Functions for interacting with the Azure Storage SDK

  • ai4e_web_utils.py: Functions for interacting with http requests

  • geospatial: Classes and utility functions for processing geospatial data for machine learning applications

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.