Граф коммитов

30 Коммитов

Автор SHA1 Сообщение Дата
Brian Hoang 9e9b4942f4
Release: v0.8.0 (#357) 2019-06-20 14:08:27 -07:00
Brian Hoang 93f3fbc6fd
v0.7.3 (#356) 2019-06-18 12:46:28 -07:00
Brian Hoang 844ebde125
Release: v0.7.2 (#317)
* Updated DESCRIPTION version

* Updated CHANGELOG.md to v0.7.2
2018-10-08 10:18:28 -07:00
Brian Hoang b97f447ca2
Feature: Add Batch Service Built-In Container support (#291)
* Replaced Virtual Configuration

* Added containerSettings in addTask helper function

* Using new Dockerfile with blobxfer and R

* Removed container settings

* Fixed containerConfiguration lintr

* Fixed description dependency

* Fixed cluster setting

* Fixed R packages installation on pool level
2018-08-14 15:34:50 -07:00
Brian Hoang 737bf498e8
CHANGELOG for v0.7.1 (#285) 2018-07-13 15:13:03 -07:00
Brian 958d84f6db
Enable AAD and VNet Support (#252)
* Added config property for service principal

* Fixed property names for service principal

* Added makeBatchClient and makeStorageClient functions

* Added ARM calls

* Fixed configuration

* Added more working features for AAD auth

* Switched sas token generator

* Added storage client generate sas tokens into doAzureParallel

* basic foreach loop works

* Long running jobs with AAD validated

* Removed credentials output

* Added docs for vnets

* Fixed network configurations for doazp

* Replaced rAzureBatch namespace with batchClient

* Fixed url auth for storage

* Changed cluster parameter naming

* Renamed helper file and fixed lintr

* Wrong file name

* Removed new line

* Removed lowerCamelCase for R6 classes

* Fixed sas token

* Fixed createBlobUrl

* Fixed configuration endpoint suffix

* Fixed endpoint suffix for download merge results

* Added environment parameter in setCredentials

* Added default for endpoint suffix for svp

* Changed default credentials file to shared key

* Updated docs for sharedkey/serviceprincipal

* Updated documentation

* Broken test for doc example

* Fixed lintr for docs

* Updated version to 0.7.0
2018-04-27 17:43:06 -07:00
Brian 80ddceadfb
Updated version v0.6.3 (#256) 2018-04-27 17:25:56 -07:00
zfengms cea055058c
Feature/nationalcloud (#239)
* support national cloud

* fix hardcoded domain name in createOutputFile

* update rAzureBatch version etc

* auto discovery of storage account endpoint suffix

* styling fix

* fix test failure

* add back endpointSuffix for storage account

* add storage account endpoint suffix to downloadBlob call

* update docs
2018-04-04 15:40:57 -07:00
Brian 5fb2fe07c2
Upgraded description for fix resize cluster (#225) 2018-02-22 18:47:48 -08:00
zfengms ec815fa5ef
Feature/asynccluster (#197)
* support for async cluster creation

* fix lintr bot errors

* remove test files

* use private function for duplicate code

* update change log

* Feature/asynccluster1 (#200)

* show node status in getCluster

* workaround error

* minor fixes

* add getClusterList api

* add metadata to pool indicating pool is created by doAzureParallel

* add test for getClusterList(), add 'other' state for nodes for getCluster()
2018-01-18 13:52:42 -08:00
Brian 920336bd1d
Updated DESCRIPTION's reference rAzureBatch to v0.5.4 (#184) 2017-12-07 08:43:16 -08:00
Pablo Selem e29422749a Feature/docker registry auth (#182)
* initial changes to support auth

* Temp changes to pull install scripts from local branch

* Updated docs to expose dockerAuth field

* Minor tweaks to doc language

* revert change to setup scripts back to master

* fix linter line too long error

* add links to cluster configuration docs

* remove whitespace

* PR feedback

* write and read docker password from disk
2017-12-06 17:57:57 -08:00
Pablo Selem a6e51c964e
Feature/container (#153)
* force add PATH to current user

* checkin docker setup script

* Update cluster_setup.sh

* install docker and start container on cluster setup

* WIP: Run task in container

* fix merge conflict

* run tasks and merge task from within container

* refactor code to proper docker commands and make a single R container per job

* refactor command line utils into its own file

* refactor job utilities into its own file

* move cluster setup script to inst folder

* remove unnecessary curl installs

* remove starting container from setup script

* check in WIP

* add apt_install file

* make required directories

* update cluster setup files as needed

* include libxml2 packages in apt installs

* working cluster create with cran and github dependencies

* update job prep to install apt-get and not each task

* use rocker containers instead of r-base

* remove unused & commented code

* remove unused install function

* address several lintr issues

* initial test dockerfile

* add spacing between commands

* temporarily point wget to feature branch

* update bioconductor install for non-jobPrep installs

* Delete Dockerfile

* minor changes to install bioc

* resolve merge conflicts

* update cluster to correctly install BioC packages using install_bioconductor

* fix issue where some packages were not getting installed

* add missing BioConductorCommand initializer

* remove print lines

* initial dockerfile implementations

* update docker files

* Only install packages if they are required

* Remove requirement on bioconductor installer script on start task

* remove duplicate environment variable entry

* update docs for container support

* update version to 0.6.0

* refactor changes updates

* remove poorly formatted whitespaces

* add full path to pacakges directory

* fix docker command line

* update file share sample

* update azure files cluster name

* update mandelbrot sample

* update package management sample

* update plyr samples

* make montecarlo sample more consistent

* update montecarlo sample

* remove plyr example

* fix bad environment pointer

* fix linter issues

* more linter fixes

* more linter issues

* use latest rAzureBatch version

* update resource files example

* remove reference to deleted sample

* pr feedback

* PR docs feedback

* Print errors from worker (#154)

* Fixed pool package command line lintr test

* Package installation tests fixed - too long lines

* Fixed json in customize cluster docs

* Fix: Typos in customize cluster docs

* Cleaning up files

* Feature/githubbiopackage (#150)

* install github package worked for foreach loop

* fix lintr error

* tests for github and bioc packages installation

* lintr fix

* add back lost code due to merge and update docs

* The Travis CI build failed for feature/githubbiopackage

* remove incorrect parameter for install_github

* Updated job prep task to have default command

* Use the latest version of rAzureBatch

* Updated description + Generate cluster config

* Fix: Bioconductor and Github packages installation (#155)

* Added multiple package install test and fix obj reading args

* Fixed naming for packages install

* Replaced validation exclusion for linter

* Fixed test validate test

* Fixing all interactive tests with skip

* Fixed renaming validation

* Removed default test - cannot be tested

* Removed  in validation

* Added cluster package install tests (#156)
2017-11-03 10:06:40 -07:00
Rich Calaway 40a2cf036a Update DESCRIPTION to point to LICENSE file (#148) 2017-10-17 19:14:04 -07:00
Brian a25c73581d Validate job names and pool names (#129)
* Added validator class

* Added validators for lintr

* Added exclusion list for validators
2017-09-29 16:57:49 -07:00
zfengms 2e8aff58c3 Merge from feature/getjobresult for long running job support (#130)
* Added set chunk size

* Added cluster configuration validation function (#30)

* Added pool config test validation

* Added a fix for validation

* Added if checks for null tests and more validation tests

* Install R packages at job run time (#29)

* Added cran/github installation scripts

* Added package installation tests

* Upgraded package version to 0.3.2

* Output file support (#40)

* Output files support

* Added createOutputFile method

* output files readme documentation

* added tests and find container sas

* Added more detailed variable names

* Enable/disable merge task (#39)

* Merge task pass params

* Fixed enableMerge cases

* Merge task documentation on README.md

* Fixed typo on merge task description

* Update doAzureParallel.R

* Changed enableMerge to enableCloudCombine

* convert getJobResult output from binary to text

* Only write vector to temp file

* save cloud merge enabled, chunk size and packages as job metadata

* update cloudMergeEnabled to cloudCombineEnabled

* Fix/backwards compatible (#68)

* Added backwards compatible in make cluster

* Added deprecated config validator

* Added mismatch label

* Added validation for quota limits and bad getPool requests in waitForNodesToComplete (#52)

* Added validation for quota limits and bad getPool requests

* Fixed based on PR

* Fixed progress bar layout to use switch statements instead of if statements

* Changed clusterId to poolId

* Added comments and fixed messages

* Added running state to the node status

* Reformatted lines for function

* Added end statement for node completion

* Feature/custom script and reduce (#70)

* Added custom scripts and removed dependencies parameter

* Updated roxygen tool version

* Added parallelThreads support

* Added test coverage

* Removed verbose message on command line

* Added Reduce function for group of tasks

* Fix build because of doc semantics mismatch with function

* Removed unused function

* Added command line arg

* Added docs for custom script

* Moved customize cluster to separate doc for future usage

* Fixed typo

* Bug - Waiting for tasks to completion function ends too early (#69)

* Moved wait for tasks to complete to doAzureParallel utility

* Removed unneeded variables and progress

* Fixed camel case for skiptoken

* Travis/lintr (#72)

* Added lintr config file

* Added travis github package installation

* Removed snake case rule

* Fixed documents on doAzureParallel

* Based on lintr default_settins docs, correctly added default rules

* Updated lintr package to use object_name_style

* Added package :: operator

* Reformatted after merge

* Fixed command line tests

* Upgraded roxygen to 6.0.1

* Cluster config docs

* Removed additional delete job

* add getJob api (#84)

* add getJob api

* reformat code

* update styling in utility file

* fix code styling

* update chunksize to chunkSize and remove unused code

* handle job metadata in getJob api

* fix styling issue

* update getJobList parameter from list of job ids to filter object, and output jobs status in data frame (#128)

long running job support, getJob, getJobList and getJobResult implementation

* reformat code

* update styling in utility file

* fix code styling

* update chunksize to chunkSize and remove unused code

* handle job metadata in getJob api

* fix styling issue

* use counting service api in getJobList

* fix coding style

* return data frame from getJobList

* update getJobList parameter from job id list to filter by state

* reformat code

* update description for getJobList

* remove dup code

* address review feedback

* jobId parameter check for getJobResult

* update documentation for long run job

* update version to 0.5.0

* update version

* address review feedback

* update chunkSizeValue to chunkSizeKeyValuePair
2017-09-29 16:49:57 -07:00
Brian 59dac737c2 v0.4.3 Release (#131)
* Upgraded description to use rAzureBatch v0.5.1

* Updated change log for job failure
2017-09-29 10:50:42 -07:00
Pablo Selem 1b06f891b1 Update DESCRIPTION (#117)
Update version for new milestone.
2017-09-18 10:52:08 -07:00
Brian 64e6da3b5d Feature/cluster logs (#98)
* download merge result gets content raw

* Added setHttpTraffic and logging functions docs

* Fixed broken links

* Shorten lines down to 120 characters

* download merge result gets content raw

* Added setHttpTraffic and logging functions docs

* Fixed broken links

* Shorten lines down to 120 characters

* Renamed function names from past discussion

* Fixed log documentation

* Added new operations for storage management

* Added dont run examples

* Fixed unused arg for running example

* Updated docs for storage management

* Added a new doc dedicated for managing storage

* Added attribute for container name in data frame

* Fixed downloadBlob to work with new rAzureBatch function

* Updated docs based on PR comments

* Changed dependency version to razurebatch 0.5.0
2017-09-07 13:06:21 -07:00
Brian 320885c07c Check if existing pool is deleted when makeCluster is called (#99)
* Added deleting pool check for makeCluster

* Fixed double quotes

* cluster logs renamed from pool to cluster

* Added correct imports and fix range
2017-09-01 11:42:35 -07:00
Brian c12e451cae Fix/GitHub authentication token (#92)
* Removed this list case to check if it's an empty string or not

* Upgraded DESCRIPTION for fix
2017-08-29 15:55:57 -07:00
Gray 53b80585ea Update DESCRIPTION (#77)
At the moment, rAzureBatch is not available on CRAN, so it can be accessible from github repo.
Vignette https://github.com/hadley/devtools/blob/master/vignettes/dependencies.Rmd
Example https://github.com/hadley/devtools/blob/master/DESCRIPTION
2017-08-24 11:48:22 -07:00
Brian 3ef734500a Milestone/0.4.0 (#74)
* Added set chunk size

* Added cluster configuration validation function (#30)

* Added pool config test validation

* Added a fix for validation

* Added if checks for null tests and more validation tests

* Install R packages at job run time (#29)

* Added cran/github installation scripts

* Added package installation tests

* Upgraded package version to 0.3.2

* Output file support (#40)

* Output files support

* Added createOutputFile method

* output files readme documentation

* added tests and find container sas

* Added more detailed variable names

* Enable/disable merge task (#39)

* Merge task pass params

* Fixed enableMerge cases

* Merge task documentation on README.md

* Fixed typo on merge task description

* Update doAzureParallel.R

* Changed enableMerge to enableCloudCombine

* Fix/backwards compatible (#68)

* Added backwards compatible in make cluster

* Added deprecated config validator

* Added mismatch label

* Added validation for quota limits and bad getPool requests in waitForNodesToComplete (#52)

* Added validation for quota limits and bad getPool requests

* Fixed based on PR

* Fixed progress bar layout to use switch statements instead of if statements

* Changed clusterId to poolId

* Added comments and fixed messages

* Added running state to the node status

* Reformatted lines for function

* Added end statement for node completion

* Feature/custom script and reduce (#70)

* Added custom scripts and removed dependencies parameter

* Updated roxygen tool version

* Added parallelThreads support

* Added test coverage

* Removed verbose message on command line

* Added Reduce function for group of tasks

* Fix build because of doc semantics mismatch with function

* Removed unused function

* Added command line arg

* Added docs for custom script

* Moved customize cluster to separate doc for future usage

* Fixed typo

* Bug - Waiting for tasks to completion function ends too early (#69)

* Moved wait for tasks to complete to doAzureParallel utility

* Removed unneeded variables and progress

* Fixed camel case for skiptoken

* Travis/lintr (#72)

* Added lintr config file

* Added travis github package installation

* Removed snake case rule

* Fixed documents on doAzureParallel

* Based on lintr default_settins docs, correctly added default rules

* Updated lintr package to use object_name_style

* Added package :: operator

* Reformatted after merge

* Fixed command line tests

* Upgraded roxygen to 6.0.1

* Cluster config docs

* Removed additional delete job

* Fixed warning descriptions in makeCluster

* Ramped up versions for DESCRIPTION

* Updated CHANGELOG to 0.4.0

* Fixed environmentSettings style from merge conflict

* Fixed cluster warning naming style
2017-08-22 16:49:28 -07:00
Brian 3a0f2e971f Fixed doAzureParallel query viewer pane (#22) 2017-05-25 15:53:58 -07:00
Brian 02c5eac6bf v0.3.0 Release (#20)
* Added set chunk size

* Adding resource files on pool creation

* renaming generate file functions

* Moved worker/merger scripts to doAzureParallel and created common job env

* Added stdout and stderr logs in uploads

* added to docs / README

* Switched params for cluster and added examples

* setCreds, resizeCluster, job management

* cred generator update

* Added samples, moved autoscale, and low-pri/output files

* Added documentation on methods for ??R feature

* Added export for makeCluster

* Namespace missing export

* clusterSetting param name

* cluster id param name

* NumOfNodes param for wait nodes completion fix

* Added proper naming for registerDoAzureParallel

* readme update'

* typo readme

* low pri in readme

* monte carlo simulation

* Added new sample for sas resource files

* caret + annotation on montecarlo sim

* samples readme.md

* samples readme

* Fixed the resource files to use proper storage account for example

* Update README.md

* Update 11-autoscale.md

* Fixed autoscale formula for task queue to take maxTaskPerNode

* Added named args to createSasToken

* Update resource-files-example.R

* Update 21-distributing-data.md

* Renamed samples files to underscore format

* Update 21-distributing-data.md

* Update README.md

* Update README.md

* Update README.md

* Edited changelog file

* Update plyr_example.R

* Update README.md
2017-05-22 17:22:40 -07:00
Brian 6f75e4fc1f Added resourceFiles back 2017-04-12 09:10:20 -07:00
Brian 31ada95f5f Changed R installation command line 2017-04-05 11:53:25 -07:00
Brian 9a21dbc5ca Updated version 2.0 2017-03-14 12:48:42 -07:00
Brian 3ab29e8cb6 Added test integration and documentation 2017-03-14 12:47:08 -07:00
Brian 502799f0ca Added doAzureParallel package 2017-02-15 17:57:06 -08:00