Граф коммитов

134 Коммитов

Автор SHA1 Сообщение Дата
zfengms bb0e136c70 Merge branch 'master' into feature/DEoptim 2018-01-26 15:24:09 -08:00
zfengms 00df9c5e58 merge with remote 2018-01-26 15:23:06 -08:00
zfengms 1f2e105606 minor updates 2018-01-26 15:18:59 -08:00
Zhiqiang Feng 86241b9390 working sample 2018-01-25 12:39:07 -08:00
Brian f4bfaebcf8 Created an issue template (#207) 2018-01-25 08:43:06 -08:00
zfengms d050285bc4 DEoptim integration test 2018-01-24 16:31:26 -08:00
Ronan O'Malley 00373b9a1e Update long_running_job.R (#206)
renamed a misnamed azure options variable
2018-01-24 10:11:35 -08:00
Brian 2d3ad396cf
In order correction (#202) 2018-01-22 12:54:47 -08:00
Pablo Selem 45fe532d50
Update documentation on using private docker registries (#201)
* update documentation for private docker registry

* update docs to be less confusing
2018-01-22 08:54:30 -08:00
zfengms ec815fa5ef
Feature/asynccluster (#197)
* support for async cluster creation

* fix lintr bot errors

* remove test files

* use private function for duplicate code

* update change log

* Feature/asynccluster1 (#200)

* show node status in getCluster

* workaround error

* minor fixes

* add getClusterList api

* add metadata to pool indicating pool is created by doAzureParallel

* add test for getClusterList(), add 'other' state for nodes for getCluster()
2018-01-18 13:52:42 -08:00
Pablo Selem fc05cdf71c
fix link to generate config doc (#199) 2018-01-17 09:19:22 -08:00
Brian afde92fe01
Fix/add task perf (#195)
* Added task id range

* Removed upload blob methods

* Removed upload blob

* Fixed trailing whitespace

* Discarded job id on merge task id name

* Adding chunk logic for argsList

* Added check for args containing data sets

* Removed container name for docker run command for all tasks

* Added test for hasDataSet

* Fix travis yml

* Adding before_install for R

* Removed before install, added github package of nycflights13
2018-01-09 18:15:35 -08:00
Brian 9d50403846
Fixed order of package installation (#196) 2017-12-27 22:49:50 -08:00
Brian a0d5537b7b
Improve R console UI experience (#193)
* Improve UI experience

* Added verbose mode to deleteJob and deleteStorageContainer

* Refactor print method
2017-12-14 13:10:52 -08:00
zfengms e257b4ea8e DEoptim sample 2017-12-13 16:50:21 -08:00
Brian 89dbba9164
Collapsing pool package installation on start task command line (#191)
* Collasping the R package installations for pool installation

* Renamed script variable

* Fixed pool installation test

* Fixed length test
2017-12-13 10:20:56 -08:00
Pablo Selem 68b2fa4a05
address issue where empty docker auth credentials are used to create … (#190)
* address issue where empty docker auth credentials are used to create the cluster

* remove unnecessary null check
2017-12-11 15:16:57 -08:00
Brian 920336bd1d
Updated DESCRIPTION's reference rAzureBatch to v0.5.4 (#184) 2017-12-07 08:43:16 -08:00
Brian a2b8c0b7f0
Fixed based on recent feedback (#185) 2017-12-07 08:42:47 -08:00
Pablo Selem e29422749a Feature/docker registry auth (#182)
* initial changes to support auth

* Temp changes to pull install scripts from local branch

* Updated docs to expose dockerAuth field

* Minor tweaks to doc language

* revert change to setup scripts back to master

* fix linter line too long error

* add links to cluster configuration docs

* remove whitespace

* PR feedback

* write and read docker password from disk
2017-12-06 17:57:57 -08:00
zfengms cbbe32b9d2
Feature/longrunjob, long running job improvement, add deleteJob and terminateJob (#174)
* add documentation and sample for long running job

* update sample file name

* update long running job doc and test

* update metadata code

* add errorHandling to job metadata

* add deleteJob to delete both job defintion and job result

* styling fix

* save foreach wait setting to metadata

* implement retry logic in getjobresult

* add terminateJob

* handle various corner cases

* regenerate document

* add job state in getJob

* do not fail getJobResult if getMetadata failed for backward compatibility

* add deleteJob option to foreach, by default it is true for wait=TRUE job

* styling fix

* update version and changelog

* address review feedback

* add setJobAutoDelete function

* rename jobAutoDelete to autoDeleteJob to workaround R bugs and update docs

* update faq

* fix styling issues

* more styling fix

* roll back manual update to DESCRIPTION

* add namespace to api call
2017-12-06 16:51:50 -08:00
Brian 8f90cd9a4b
Changing UI of progress bar (#183)
* Redone the progress bar

* Added string utilities
2017-12-06 12:04:29 -08:00
Brian fdd91d9c10
Implemented progress bar with verbose details (#181)
* Implemented a more detailed verbose progress bar

* Fixed infix operator spacing

* Added newline for 'Tasks have completed' message
2017-12-05 10:17:26 -08:00
Pablo Selem 24f0cbb1d7
updated docs (#172) 2017-11-20 15:30:19 -08:00
zfengms 5f1afc7929
Feature/configfromobj support programmatically created credentials and cluster configs (#168)
* support programmatically generated credentials/cluster config

* update docs for programmatically generated config

* reformat code

* styling fixes

* combine credentials cluster config methods

* fix set credentials issue and test input

* do not create az_config.json

* update messages

* move githubAuthenticationToken from cluster config to credentials
2017-11-16 23:18:09 -08:00
Pablo Selem bc529f00d3
initial docs for docker users (#166)
* initial docs for docker users

* Fixes plus PR feedback
2017-11-14 14:54:45 -08:00
Pablo Selem dfd18d6132
remove unused validator file (#167) 2017-11-14 12:30:24 -08:00
JS d0a3848f79
Update README.md (#164) 2017-11-10 15:36:14 -08:00
Pablo Selem 6e78b096bb
Merge branch 'stable' into master 2017-11-03 14:37:43 -07:00
Pablo Selem 52319d3677
long running job support (#136) (#161)
* treat warnings as failures and fail the creation of the cluster (#91)

* treat warnings as failures and fail the creation of the cluster

* fix unit tests

* fix lintr lines too long issue

* escape single quotes

* Check if existing pool is deleted when makeCluster is called (#99)

* Added deleting pool check for makeCluster

* Fixed double quotes

* cluster logs renamed from pool to cluster

* Added correct imports and fix range

* Feature/bio conductor docs (#106)

* initial command line instructions for bioconductor

* initial startup scripts for installing bioconductor

* fix if then syntax

* force update node environment with update path for R runtime

* install bioconductor

* wrap bioconductor install command in Rscript

* bioconductor sample docs

* update bioC docs

* remove .gitignore rule for .json files

* add pointer to BioC cluster config from docs

* Feature/cluster logs (#98)

* download merge result gets content raw

* Added setHttpTraffic and logging functions docs

* Fixed broken links

* Shorten lines down to 120 characters

* download merge result gets content raw

* Added setHttpTraffic and logging functions docs

* Fixed broken links

* Shorten lines down to 120 characters

* Renamed function names from past discussion

* Fixed log documentation

* Added new operations for storage management

* Added dont run examples

* Fixed unused arg for running example

* Updated docs for storage management

* Added a new doc dedicated for managing storage

* Added attribute for container name in data frame

* Fixed downloadBlob to work with new rAzureBatch function

* Updated docs based on PR comments

* Changed dependency version to razurebatch 0.5.0

* Feature/add azure files cluster config (#108)

* initial command line instructions for bioconductor

* initial startup scripts for installing bioconductor

* fix if then syntax

* force update node environment with update path for R runtime

* install bioconductor

* wrap bioconductor install command in Rscript

* bioconductor sample docs

* update bioC docs

* remove .gitignore rule for .json files

* add pointer to BioC cluster config from docs

* add missing azureFiles cluster config to samples

* Add 0.4.2 CHANGELOG comments (#111)

* Added live scenario test (#107)

* Added live scenario test so users do not have to write their own sample code to test

* Added file names for test live

* Removed single quote linter

* Added comment about the reason for this test

* Wait for job preparation task function (#109)

* Fixed verbose for getDoParWorkers (#112)

* Feature/faq (#110)

* initial FAQ

* rename faq to FAQ

* merge FAQ and Troubleshooting docs

* add info on how to reboot a node

* refrence TSG and FAQ from main docs index page

* add more info as per PR feedback

* PR feedback

* point raw scripts at master branch (#118)

* Update DESCRIPTION (#117)

Update version for new milestone.

* Fix: Removed anaconda from path (#119)

* Removed anaconda from environment path

* Line is too long for blobxfer command

* For BioConductor install, force remove MRO 3.3 prior to installing MRO 3.4 (#120)

* force add PATH to current user

* Update bioc_setup.sh

* Check verbose null case (#121)

* Change True/False to TRUE/FALSE in README example (#124)

* add .gitiattrributes file to track line endings

* True and False are not valid in R; changed to TRUE and FALSE

* Fixed worker and merger scripts (#116)

* Fixed worker and merger scripts

* Fixed verbose logs based on PR comments

* Added documentation on error handling

* Fixed header on table markdown

* Fixed based on PR comments

* v0.4.3 Release (#131)

* Upgraded description to use rAzureBatch v0.5.1

* Updated change log for job failure

* readme.md update

* Merge from feature/getjobresult for long running job support (#130)

* Added set chunk size

* Added cluster configuration validation function (#30)

* Added pool config test validation

* Added a fix for validation

* Added if checks for null tests and more validation tests

* Install R packages at job run time (#29)

* Added cran/github installation scripts

* Added package installation tests

* Upgraded package version to 0.3.2

* Output file support (#40)

* Output files support

* Added createOutputFile method

* output files readme documentation

* added tests and find container sas

* Added more detailed variable names

* Enable/disable merge task (#39)

* Merge task pass params

* Fixed enableMerge cases

* Merge task documentation on README.md

* Fixed typo on merge task description

* Update doAzureParallel.R

* Changed enableMerge to enableCloudCombine

* convert getJobResult output from binary to text

* Only write vector to temp file

* save cloud merge enabled, chunk size and packages as job metadata

* update cloudMergeEnabled to cloudCombineEnabled

* Fix/backwards compatible (#68)

* Added backwards compatible in make cluster

* Added deprecated config validator

* Added mismatch label

* Added validation for quota limits and bad getPool requests in waitForNodesToComplete (#52)

* Added validation for quota limits and bad getPool requests

* Fixed based on PR

* Fixed progress bar layout to use switch statements instead of if statements

* Changed clusterId to poolId

* Added comments and fixed messages

* Added running state to the node status

* Reformatted lines for function

* Added end statement for node completion

* Feature/custom script and reduce (#70)

* Added custom scripts and removed dependencies parameter

* Updated roxygen tool version

* Added parallelThreads support

* Added test coverage

* Removed verbose message on command line

* Added Reduce function for group of tasks

* Fix build because of doc semantics mismatch with function

* Removed unused function

* Added command line arg

* Added docs for custom script

* Moved customize cluster to separate doc for future usage

* Fixed typo

* Bug - Waiting for tasks to completion function ends too early (#69)

* Moved wait for tasks to complete to doAzureParallel utility

* Removed unneeded variables and progress

* Fixed camel case for skiptoken

* Travis/lintr (#72)

* Added lintr config file

* Added travis github package installation

* Removed snake case rule

* Fixed documents on doAzureParallel

* Based on lintr default_settins docs, correctly added default rules

* Updated lintr package to use object_name_style

* Added package :: operator

* Reformatted after merge

* Fixed command line tests

* Upgraded roxygen to 6.0.1

* Cluster config docs

* Removed additional delete job

* add getJob api (#84)

* add getJob api

* reformat code

* update styling in utility file

* fix code styling

* update chunksize to chunkSize and remove unused code

* handle job metadata in getJob api

* fix styling issue

* update getJobList parameter from list of job ids to filter object, and output jobs status in data frame (#128)

long running job support, getJob, getJobList and getJobResult implementation

* reformat code

* update styling in utility file

* fix code styling

* update chunksize to chunkSize and remove unused code

* handle job metadata in getJob api

* fix styling issue

* use counting service api in getJobList

* fix coding style

* return data frame from getJobList

* update getJobList parameter from job id list to filter by state

* reformat code

* update description for getJobList

* remove dup code

* address review feedback

* jobId parameter check for getJobResult

* update documentation for long run job

* update version to 0.5.0

* update version

* address review feedback

* update chunkSizeValue to chunkSizeKeyValuePair

* Validate job names and pool names (#129)

* Added validator class

* Added validators for lintr

* Added exclusion list for validators

* fix bug in metadata handling for packages and enableCloudCombine (#133)

* fix bug in metadata handling for packages and enableCloudCombine

* call long running job api in test

* update test

* add test for long running job feature

* code style fix

* update job state description in readme

* use list for job state filter

* address review feedback
2017-11-03 14:20:48 -07:00
Pablo Selem aae6587b56
fix pointers to master branch (#160) 2017-11-03 12:48:10 -07:00
Pablo Selem 6553c1db59
Changelog/v0.6.0 (#158)
* update changelog for 0.6.0

* update typo

* capitalize repo names correctly
2017-11-03 12:17:39 -07:00
Pablo Selem a6e51c964e
Feature/container (#153)
* force add PATH to current user

* checkin docker setup script

* Update cluster_setup.sh

* install docker and start container on cluster setup

* WIP: Run task in container

* fix merge conflict

* run tasks and merge task from within container

* refactor code to proper docker commands and make a single R container per job

* refactor command line utils into its own file

* refactor job utilities into its own file

* move cluster setup script to inst folder

* remove unnecessary curl installs

* remove starting container from setup script

* check in WIP

* add apt_install file

* make required directories

* update cluster setup files as needed

* include libxml2 packages in apt installs

* working cluster create with cran and github dependencies

* update job prep to install apt-get and not each task

* use rocker containers instead of r-base

* remove unused & commented code

* remove unused install function

* address several lintr issues

* initial test dockerfile

* add spacing between commands

* temporarily point wget to feature branch

* update bioconductor install for non-jobPrep installs

* Delete Dockerfile

* minor changes to install bioc

* resolve merge conflicts

* update cluster to correctly install BioC packages using install_bioconductor

* fix issue where some packages were not getting installed

* add missing BioConductorCommand initializer

* remove print lines

* initial dockerfile implementations

* update docker files

* Only install packages if they are required

* Remove requirement on bioconductor installer script on start task

* remove duplicate environment variable entry

* update docs for container support

* update version to 0.6.0

* refactor changes updates

* remove poorly formatted whitespaces

* add full path to pacakges directory

* fix docker command line

* update file share sample

* update azure files cluster name

* update mandelbrot sample

* update package management sample

* update plyr samples

* make montecarlo sample more consistent

* update montecarlo sample

* remove plyr example

* fix bad environment pointer

* fix linter issues

* more linter fixes

* more linter issues

* use latest rAzureBatch version

* update resource files example

* remove reference to deleted sample

* pr feedback

* PR docs feedback

* Print errors from worker (#154)

* Fixed pool package command line lintr test

* Package installation tests fixed - too long lines

* Fixed json in customize cluster docs

* Fix: Typos in customize cluster docs

* Cleaning up files

* Feature/githubbiopackage (#150)

* install github package worked for foreach loop

* fix lintr error

* tests for github and bioc packages installation

* lintr fix

* add back lost code due to merge and update docs

* The Travis CI build failed for feature/githubbiopackage

* remove incorrect parameter for install_github

* Updated job prep task to have default command

* Use the latest version of rAzureBatch

* Updated description + Generate cluster config

* Fix: Bioconductor and Github packages installation (#155)

* Added multiple package install test and fix obj reading args

* Fixed naming for packages install

* Replaced validation exclusion for linter

* Fixed test validate test

* Fixing all interactive tests with skip

* Fixed renaming validation

* Removed default test - cannot be tested

* Removed  in validation

* Added cluster package install tests (#156)
2017-11-03 10:06:40 -07:00
Brian 4eb3773738 Fix: Do not use task chunk size during cloud combine function (#152)
* Fix task counts when group of tasks are not equal

* Added chunksize test

* Fixed pass case
2017-10-20 11:38:12 -07:00
Brian 169e75f9b4 Fixed job creation (#138) 2017-10-19 09:47:18 -07:00
Rich Calaway 40a2cf036a Update DESCRIPTION to point to LICENSE file (#148) 2017-10-17 19:14:04 -07:00
zfengms 36fadf4287 Feature/longrunjobdoc (#139)
* add documentation and sample for long running job

* update sample file name

* update long running job doc and test
2017-10-10 17:27:18 -07:00
Pablo Selem 0744c437b6 fix bioconductor package install docs for multi-task race condition (#135)
* fix bioconductor package install docs for multi-task race condition

* fix typos

* remove docs on updating an existing cluster's packages from within the foreach loop

* support an install_bioconductor.R script

* update docs with working sample
2017-10-03 08:28:05 -07:00
zfengms f6ab94a590 long running job support (#136)
* treat warnings as failures and fail the creation of the cluster (#91)

* treat warnings as failures and fail the creation of the cluster

* fix unit tests

* fix lintr lines too long issue

* escape single quotes

* Check if existing pool is deleted when makeCluster is called (#99)

* Added deleting pool check for makeCluster

* Fixed double quotes

* cluster logs renamed from pool to cluster

* Added correct imports and fix range

* Feature/bio conductor docs (#106)

* initial command line instructions for bioconductor

* initial startup scripts for installing bioconductor

* fix if then syntax

* force update node environment with update path for R runtime

* install bioconductor

* wrap bioconductor install command in Rscript

* bioconductor sample docs

* update bioC docs

* remove .gitignore rule for .json files

* add pointer to BioC cluster config from docs

* Feature/cluster logs (#98)

* download merge result gets content raw

* Added setHttpTraffic and logging functions docs

* Fixed broken links

* Shorten lines down to 120 characters

* download merge result gets content raw

* Added setHttpTraffic and logging functions docs

* Fixed broken links

* Shorten lines down to 120 characters

* Renamed function names from past discussion

* Fixed log documentation

* Added new operations for storage management

* Added dont run examples

* Fixed unused arg for running example

* Updated docs for storage management

* Added a new doc dedicated for managing storage

* Added attribute for container name in data frame

* Fixed downloadBlob to work with new rAzureBatch function

* Updated docs based on PR comments

* Changed dependency version to razurebatch 0.5.0

* Feature/add azure files cluster config (#108)

* initial command line instructions for bioconductor

* initial startup scripts for installing bioconductor

* fix if then syntax

* force update node environment with update path for R runtime

* install bioconductor

* wrap bioconductor install command in Rscript

* bioconductor sample docs

* update bioC docs

* remove .gitignore rule for .json files

* add pointer to BioC cluster config from docs

* add missing azureFiles cluster config to samples

* Add 0.4.2 CHANGELOG comments (#111)

* Added live scenario test (#107)

* Added live scenario test so users do not have to write their own sample code to test

* Added file names for test live

* Removed single quote linter

* Added comment about the reason for this test

* Wait for job preparation task function (#109)

* Fixed verbose for getDoParWorkers (#112)

* Feature/faq (#110)

* initial FAQ

* rename faq to FAQ

* merge FAQ and Troubleshooting docs

* add info on how to reboot a node

* refrence TSG and FAQ from main docs index page

* add more info as per PR feedback

* PR feedback

* point raw scripts at master branch (#118)

* Update DESCRIPTION (#117)

Update version for new milestone.

* Fix: Removed anaconda from path (#119)

* Removed anaconda from environment path

* Line is too long for blobxfer command

* For BioConductor install, force remove MRO 3.3 prior to installing MRO 3.4 (#120)

* force add PATH to current user

* Update bioc_setup.sh

* Check verbose null case (#121)

* Change True/False to TRUE/FALSE in README example (#124)

* add .gitiattrributes file to track line endings

* True and False are not valid in R; changed to TRUE and FALSE

* Fixed worker and merger scripts (#116)

* Fixed worker and merger scripts

* Fixed verbose logs based on PR comments

* Added documentation on error handling

* Fixed header on table markdown

* Fixed based on PR comments

* v0.4.3 Release (#131)

* Upgraded description to use rAzureBatch v0.5.1

* Updated change log for job failure

* readme.md update

* Merge from feature/getjobresult for long running job support (#130)

* Added set chunk size

* Added cluster configuration validation function (#30)

* Added pool config test validation

* Added a fix for validation

* Added if checks for null tests and more validation tests

* Install R packages at job run time (#29)

* Added cran/github installation scripts

* Added package installation tests

* Upgraded package version to 0.3.2

* Output file support (#40)

* Output files support

* Added createOutputFile method

* output files readme documentation

* added tests and find container sas

* Added more detailed variable names

* Enable/disable merge task (#39)

* Merge task pass params

* Fixed enableMerge cases

* Merge task documentation on README.md

* Fixed typo on merge task description

* Update doAzureParallel.R

* Changed enableMerge to enableCloudCombine

* convert getJobResult output from binary to text

* Only write vector to temp file

* save cloud merge enabled, chunk size and packages as job metadata

* update cloudMergeEnabled to cloudCombineEnabled

* Fix/backwards compatible (#68)

* Added backwards compatible in make cluster

* Added deprecated config validator

* Added mismatch label

* Added validation for quota limits and bad getPool requests in waitForNodesToComplete (#52)

* Added validation for quota limits and bad getPool requests

* Fixed based on PR

* Fixed progress bar layout to use switch statements instead of if statements

* Changed clusterId to poolId

* Added comments and fixed messages

* Added running state to the node status

* Reformatted lines for function

* Added end statement for node completion

* Feature/custom script and reduce (#70)

* Added custom scripts and removed dependencies parameter

* Updated roxygen tool version

* Added parallelThreads support

* Added test coverage

* Removed verbose message on command line

* Added Reduce function for group of tasks

* Fix build because of doc semantics mismatch with function

* Removed unused function

* Added command line arg

* Added docs for custom script

* Moved customize cluster to separate doc for future usage

* Fixed typo

* Bug - Waiting for tasks to completion function ends too early (#69)

* Moved wait for tasks to complete to doAzureParallel utility

* Removed unneeded variables and progress

* Fixed camel case for skiptoken

* Travis/lintr (#72)

* Added lintr config file

* Added travis github package installation

* Removed snake case rule

* Fixed documents on doAzureParallel

* Based on lintr default_settins docs, correctly added default rules

* Updated lintr package to use object_name_style

* Added package :: operator

* Reformatted after merge

* Fixed command line tests

* Upgraded roxygen to 6.0.1

* Cluster config docs

* Removed additional delete job

* add getJob api (#84)

* add getJob api

* reformat code

* update styling in utility file

* fix code styling

* update chunksize to chunkSize and remove unused code

* handle job metadata in getJob api

* fix styling issue

* update getJobList parameter from list of job ids to filter object, and output jobs status in data frame (#128)

long running job support, getJob, getJobList and getJobResult implementation

* reformat code

* update styling in utility file

* fix code styling

* update chunksize to chunkSize and remove unused code

* handle job metadata in getJob api

* fix styling issue

* use counting service api in getJobList

* fix coding style

* return data frame from getJobList

* update getJobList parameter from job id list to filter by state

* reformat code

* update description for getJobList

* remove dup code

* address review feedback

* jobId parameter check for getJobResult

* update documentation for long run job

* update version to 0.5.0

* update version

* address review feedback

* update chunkSizeValue to chunkSizeKeyValuePair

* Validate job names and pool names (#129)

* Added validator class

* Added validators for lintr

* Added exclusion list for validators

* fix bug in metadata handling for packages and enableCloudCombine (#133)

* fix bug in metadata handling for packages and enableCloudCombine

* call long running job api in test

* update test

* add test for long running job feature

* code style fix

* update job state description in readme

* use list for job state filter

* address review feedback
2017-10-02 17:19:07 -07:00
zfengms e11afbcb9b Merge branch 'stable' into master 2017-10-02 17:04:09 -07:00
zfengms 907812228c address review feedback 2017-10-02 16:58:56 -07:00
zfengms 7ec1306aeb fix bug in metadata handling for packages and enableCloudCombine (#133)
* fix bug in metadata handling for packages and enableCloudCombine

* call long running job api in test

* update test

* add test for long running job feature

* code style fix

* update job state description in readme

* use list for job state filter
2017-09-30 21:45:25 -07:00
Brian a25c73581d Validate job names and pool names (#129)
* Added validator class

* Added validators for lintr

* Added exclusion list for validators
2017-09-29 16:57:49 -07:00
zfengms 2e8aff58c3 Merge from feature/getjobresult for long running job support (#130)
* Added set chunk size

* Added cluster configuration validation function (#30)

* Added pool config test validation

* Added a fix for validation

* Added if checks for null tests and more validation tests

* Install R packages at job run time (#29)

* Added cran/github installation scripts

* Added package installation tests

* Upgraded package version to 0.3.2

* Output file support (#40)

* Output files support

* Added createOutputFile method

* output files readme documentation

* added tests and find container sas

* Added more detailed variable names

* Enable/disable merge task (#39)

* Merge task pass params

* Fixed enableMerge cases

* Merge task documentation on README.md

* Fixed typo on merge task description

* Update doAzureParallel.R

* Changed enableMerge to enableCloudCombine

* convert getJobResult output from binary to text

* Only write vector to temp file

* save cloud merge enabled, chunk size and packages as job metadata

* update cloudMergeEnabled to cloudCombineEnabled

* Fix/backwards compatible (#68)

* Added backwards compatible in make cluster

* Added deprecated config validator

* Added mismatch label

* Added validation for quota limits and bad getPool requests in waitForNodesToComplete (#52)

* Added validation for quota limits and bad getPool requests

* Fixed based on PR

* Fixed progress bar layout to use switch statements instead of if statements

* Changed clusterId to poolId

* Added comments and fixed messages

* Added running state to the node status

* Reformatted lines for function

* Added end statement for node completion

* Feature/custom script and reduce (#70)

* Added custom scripts and removed dependencies parameter

* Updated roxygen tool version

* Added parallelThreads support

* Added test coverage

* Removed verbose message on command line

* Added Reduce function for group of tasks

* Fix build because of doc semantics mismatch with function

* Removed unused function

* Added command line arg

* Added docs for custom script

* Moved customize cluster to separate doc for future usage

* Fixed typo

* Bug - Waiting for tasks to completion function ends too early (#69)

* Moved wait for tasks to complete to doAzureParallel utility

* Removed unneeded variables and progress

* Fixed camel case for skiptoken

* Travis/lintr (#72)

* Added lintr config file

* Added travis github package installation

* Removed snake case rule

* Fixed documents on doAzureParallel

* Based on lintr default_settins docs, correctly added default rules

* Updated lintr package to use object_name_style

* Added package :: operator

* Reformatted after merge

* Fixed command line tests

* Upgraded roxygen to 6.0.1

* Cluster config docs

* Removed additional delete job

* add getJob api (#84)

* add getJob api

* reformat code

* update styling in utility file

* fix code styling

* update chunksize to chunkSize and remove unused code

* handle job metadata in getJob api

* fix styling issue

* update getJobList parameter from list of job ids to filter object, and output jobs status in data frame (#128)

long running job support, getJob, getJobList and getJobResult implementation

* reformat code

* update styling in utility file

* fix code styling

* update chunksize to chunkSize and remove unused code

* handle job metadata in getJob api

* fix styling issue

* use counting service api in getJobList

* fix coding style

* return data frame from getJobList

* update getJobList parameter from job id list to filter by state

* reformat code

* update description for getJobList

* remove dup code

* address review feedback

* jobId parameter check for getJobResult

* update documentation for long run job

* update version to 0.5.0

* update version

* address review feedback

* update chunkSizeValue to chunkSizeKeyValuePair
2017-09-29 16:49:57 -07:00
zfengms 91db8a959e readme.md update 2017-09-29 14:17:09 -07:00
Brian 737c1d5a51 Merge master to stable for version 0.4.3 (#132)
* treat warnings as failures and fail the creation of the cluster (#91)

* treat warnings as failures and fail the creation of the cluster

* fix unit tests

* fix lintr lines too long issue

* escape single quotes

* Check if existing pool is deleted when makeCluster is called (#99)

* Added deleting pool check for makeCluster

* Fixed double quotes

* cluster logs renamed from pool to cluster

* Added correct imports and fix range

* Feature/bio conductor docs (#106)

* initial command line instructions for bioconductor

* initial startup scripts for installing bioconductor

* fix if then syntax

* force update node environment with update path for R runtime

* install bioconductor

* wrap bioconductor install command in Rscript

* bioconductor sample docs

* update bioC docs

* remove .gitignore rule for .json files

* add pointer to BioC cluster config from docs

* Feature/cluster logs (#98)

* download merge result gets content raw

* Added setHttpTraffic and logging functions docs

* Fixed broken links

* Shorten lines down to 120 characters

* download merge result gets content raw

* Added setHttpTraffic and logging functions docs

* Fixed broken links

* Shorten lines down to 120 characters

* Renamed function names from past discussion

* Fixed log documentation

* Added new operations for storage management

* Added dont run examples

* Fixed unused arg for running example

* Updated docs for storage management

* Added a new doc dedicated for managing storage

* Added attribute for container name in data frame

* Fixed downloadBlob to work with new rAzureBatch function

* Updated docs based on PR comments

* Changed dependency version to razurebatch 0.5.0

* Feature/add azure files cluster config (#108)

* initial command line instructions for bioconductor

* initial startup scripts for installing bioconductor

* fix if then syntax

* force update node environment with update path for R runtime

* install bioconductor

* wrap bioconductor install command in Rscript

* bioconductor sample docs

* update bioC docs

* remove .gitignore rule for .json files

* add pointer to BioC cluster config from docs

* add missing azureFiles cluster config to samples

* Add 0.4.2 CHANGELOG comments (#111)

* Added live scenario test (#107)

* Added live scenario test so users do not have to write their own sample code to test

* Added file names for test live

* Removed single quote linter

* Added comment about the reason for this test

* Wait for job preparation task function (#109)

* Fixed verbose for getDoParWorkers (#112)

* Feature/faq (#110)

* initial FAQ

* rename faq to FAQ

* merge FAQ and Troubleshooting docs

* add info on how to reboot a node

* refrence TSG and FAQ from main docs index page

* add more info as per PR feedback

* PR feedback

* point raw scripts at master branch (#118)

* Update DESCRIPTION (#117)

Update version for new milestone.

* Fix: Removed anaconda from path (#119)

* Removed anaconda from environment path

* Line is too long for blobxfer command

* For BioConductor install, force remove MRO 3.3 prior to installing MRO 3.4 (#120)

* force add PATH to current user

* Update bioc_setup.sh

* Check verbose null case (#121)

* Change True/False to TRUE/FALSE in README example (#124)

* add .gitiattrributes file to track line endings

* True and False are not valid in R; changed to TRUE and FALSE

* Fixed worker and merger scripts (#116)

* Fixed worker and merger scripts

* Fixed verbose logs based on PR comments

* Added documentation on error handling

* Fixed header on table markdown

* Fixed based on PR comments

* v0.4.3 Release (#131)

* Upgraded description to use rAzureBatch v0.5.1

* Updated change log for job failure
2017-09-29 13:23:21 -07:00
Brian 59dac737c2 v0.4.3 Release (#131)
* Upgraded description to use rAzureBatch v0.5.1

* Updated change log for job failure
2017-09-29 10:50:42 -07:00
Brian 3dadf0803e Fixed worker and merger scripts (#116)
* Fixed worker and merger scripts

* Fixed verbose logs based on PR comments

* Added documentation on error handling

* Fixed header on table markdown

* Fixed based on PR comments
2017-09-28 15:18:06 -07:00
Dan Tenenbaum 368eeb85c8 Change True/False to TRUE/FALSE in README example (#124)
* add .gitiattrributes file to track line endings

* True and False are not valid in R; changed to TRUE and FALSE
2017-09-27 14:29:32 -07:00
Brian e1769ebb84 Check verbose null case (#121) 2017-09-19 11:03:49 -07:00