Highly concurrent data transfer tool for Azure Blob Storage.
Перейти к файлу
microsoft-github-policy-service[bot] ac23782d56
Auto merge mandatory file pr
This pr is auto merged as it contains a mandatory file and is opened for more than 10 days.
2023-05-31 18:49:40 +00:00
.vscode comments added 2016-12-24 13:10:30 -05:00
Godeps - Upgrade to new Azure storage SDK for Go 2018-01-25 21:41:03 -05:00
_build - zip removed 2018-02-26 15:57:43 -05:00
docs Update examples.rst 2019-05-14 18:54:47 -04:00
img readme update 2017-03-24 12:15:36 -04:00
internal docs updates and refactoring 2019-02-25 19:39:37 -05:00
pipeline test fix 2018-04-19 22:53:55 -04:00
sources docs updates and refactoring 2019-02-25 19:39:37 -05:00
targets docs updates and refactoring 2019-02-25 19:39:37 -05:00
transfer docs updates and refactoring 2019-02-25 19:39:37 -05:00
util 0.6.20rc 2019-02-23 19:12:34 -05:00
vendor 0.6.20rc 2019-02-23 19:12:34 -05:00
.gitignore 0.6.20rc 2019-02-23 19:12:34 -05:00
.travis.yml docs updates and refactoring 2019-02-25 19:39:37 -05:00
Gopkg.lock 0.6.20rc 2019-02-23 19:12:34 -05:00
Gopkg.toml 0.6.20rc 2019-02-23 19:12:34 -05:00
LICENSE v0522 2018-01-16 02:48:47 -05:00
README.md document update 2019-02-25 19:51:16 -05:00
SECURITY.md Microsoft mandatory file 2023-03-28 16:47:02 +00:00
_config.yml Set theme jekyll-theme-cayman 2017-11-29 22:40:54 -05:00
args.go 0.6.20rc 2019-02-23 19:12:34 -05:00
args_test.go -Expiration of the readonly access token for S3/blob is configurable 2018-02-19 15:01:34 -05:00
blobporter.go docs updates and refactoring 2019-02-25 19:39:37 -05:00
files.sh - handleman for target pipeline 2018-02-10 13:40:56 -05:00
go.mod docs updates and refactoring 2019-02-25 19:39:37 -05:00
go.sum docs updates and refactoring 2019-02-25 19:39:37 -05:00
inttest._m.sh docs updates and refactoring 2019-02-25 19:39:37 -05:00
inttest.sh - Batch transfers, 200 files at the time. 2017-11-13 13:27:12 -05:00
pipelinefactory.go 0.6.20rc 2019-02-23 19:12:34 -05:00
progstate.go 0.6.20rc 2019-02-23 19:12:34 -05:00

README.md

BlobPorter

Build Status Go Report Card

Introduction

BlobPorter is a data transfer tool for Azure Blob Storage that maximizes throughput through concurrent reads and writes that can scale up and down independently.

Sources and targets are decoupled, this design enables the composition of various transfer scenarios.

From/To Azure Block Blob Azure Page Blob File (Download)
File (Upload) Yes Yes NA
HTTP/HTTPS* Yes Yes Yes
Azure Block Blob Yes Yes Yes
Azure Page Blob Yes Yes Yes
S3 Endpoint Yes Yes No

* The HTTP/HTTPS source must support HTTP byte ranges and return the file size as a response to a HTTP HEAD request.

What's new (0.6.20) ?

  • Blob to Block Blobs transfers use the Put Block from URL API.
  • Proxy support.
  • Minor fixes and performance improvements.
  • Upgrade to Go 1.11

Documentation

Issues and Feedback

If you have a question or find a bug, open a new issue in this repository. BlobPorter is an OSS project maintained by the contributors.

Contribute

If you would like to become an active contributor to this project please follow the instructions provided in Microsoft Azure Projects Contribution Guidelines.


This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.