Feature/pkgmgmtdoc (#231)

* merge package management doc

* merge packagement docs

* address review feedback (#232)

* address review feedback

* add reference to github and bioconductor packages in worker

* update package management sample

* update package management doc (#233)

* address review feedback

* add reference to github and bioconductor packages in worker

* update package management sample

* update package management doc

* remove cluster.json

* remove package installation

docs/20-package-management.md

@@ -4,7 +4,26 @@ The doAzureParallel package allows you to install packages to your pool in two w
 - Installing on pool creation
 - Installing per-*foreach* loop
 
+Packages installed at the pool level benefit from only needing to be installed once per node. Each iteration of the foreach can load the library without needing to install them again. Packages installed in the foreach benefit from specifying any dependencies required only for that instance of the loop.
+
 ## Installing Packages on Pool Creation
+
+Pool level packages support CRAN, GitHub and BioConductor packages. The packages are installed in a shared directory on the node. It is important to note that it is required to add it to .packages parameter (or github or bioconductor for github or bioconductor packages), or explicitly load any packages installed at the pool level within the foreach loop. For example, if you installed xml2 on the cluster, you must explicitly load it or add it to .packages before using it.
+
+```R
+foreach (i = 1:4) %dopar% {
+  # Load the libraries you want to use.
+  library(xml2)
+  xml2::as_list(...)
+}
+```
+or
+```R
+foreach (i = 1:4, .packages=c('xml2')) %dopar% {
+  xml2::as_list(...)
+}
+```
+
 You can install packages by specifying the package(s) in your JSON pool configuration file. This will then install the specified packages at the time of pool creation.
 
 ```R

inst/startup/worker.R

@@ -92,6 +92,14 @@ for (package in azbatchenv$packages) {
   library(package, character.only = TRUE)
 }
 
+for (package in azbatchenv$github) {
+  library(package, character.only = TRUE)
+}
+
+for (package in azbatchenv$bioconductor) {
+  library(package, character.only = TRUE)
+}
+
 ls(azbatchenv)
 parent.env(azbatchenv$exportenv) <- getparentenv(azbatchenv$pkgName)
 

samples/package_management/README.md

@@ -1,69 +0,0 @@
-# Using package management
-
-doAzureParallel supports installing packages at either the cluster level or during the execution of the foreach loop. Packages installed at the cluster level benefit from only needing to be installed once per node. Each iteration of the foreach can load the library without needing to install them again. Packages installed in the foreach benefit from specifying any specific dependencies required only for that instance of the loop.
-
-## Cluster level packages
-
-Cluster level packages support CRAN, GitHub and BioConductor packages. The packages are installed in a shared directory on the node. It is important to note that it is required to explicitly load any packages installed at the cluster level within the foreach loop. For example, if you installed xml2 on the cluster, you must explicityly load it before using it.
-
-```R
-foreach (i = 1:4) %dopar% {
-  # Load the libraries you want to use.
-  library(xml2)
-  xml2::as_list(...)
-}
-```
-
-### CRAN
-
-CRAN packages can be insatlled on the cluster by adding them to the collection of _cran_ packages in the cluster specification. 
-
-```json
-"rPackages": {
-    "cran": ["package1", "package2", "..."],
-    "github": [],
-    "bioconductor": []
-  }
-```
-
-### GitHub
-
-GitHub packages can be insatlled on the cluster by adding them to the collection of _github_ packages in the cluster specification. 
-
-```json
-"rPackages": {
-    "cran": [],
-    "github": ["repo1/name1", "repo1/name2", "repo2/name1", "..."],
-    "bioconductor": []
-  }
-```
-
-**NOTE** When using packages from a private GitHub repository, you must add your GitHub authentication token to your credentials.json file.
-
-### BioConductor
-
-Installing bioconductor packages is now supported via the cluster configuration. Simply add the list of packages you want to have installed in the cluster configuration file and they will get automatically applied
-
-```json
-"rPackages": {
-    "cran": [],
-    "github": [],
-    "bioconductor": ["IRanges", "GenomeInofDb"]
-  }
-```
-
-**IMPORTANT** doAzureParallel uses the rocker/tidyverse Docker images by default, which comes with BioConductor pre-installed. If you use a different container image, make sure that bioconductor is installed on it.
-
-
-## Foreach level packages
-
-Foreach level packages currently only support CRAN packages. Unlike cluster level pacakges, when specifying packages on the foreach loop, packages will be automatically installed _and loaded_ for use.
-
-### CRAN
-
-```R
-foreach(i = 1:4, .packages = c("xml2")) %dopar% {
-  # xml2 is automatically loaded an can be used without calling library(xml2)
-  xml2::as_list(...)
-}
-```

samples/package_management/bioconductor.r

@@ -1,21 +1,19 @@
-# install packages
+#Please see documentation at docs/20-package-management.md for more details on packagement management.
-library(devtools)
-install_github("azure/doazureparallel")
 
 # import the doAzureParallel library and its dependencies
 library(doAzureParallel)
 
 # set your credentials
-setCredentials("credentials.json")
+doAzureParallel::setCredentials("credentials.json")
 
 # Create your cluster if not exist
-cluster <- makeCluster("bioconductor_cluster.json")
+cluster <- doAzureParallel::makeCluster("bioconductor_cluster.json")
 
 # register your parallel backend
-registerDoAzureParallel(cluster)
+doAzureParallel::registerDoAzureParallel(cluster)
 
 # check that your workers are up
-getDoParWorkers()
+doAzureParallel::getDoParWorkers()
 
 summary <- foreach(i = 1:1) %dopar% {
   library(GenomeInfoDb) # Already installed as part of the cluster configuration
@@ -23,7 +21,13 @@ summary <- foreach(i = 1:1) %dopar% {
 
   sessionInfo()
   # Your algorithm
-
+}
+
+summary
+
+summary <- foreach(i = 1:1, bioconductor=c('GenomeInfoDb', 'IRanges')) %dopar% {
+  sessionInfo()
+  # Your algorithm
 }
 
 summary