Граф коммитов

127 Коммитов

Автор SHA1 Сообщение Дата
Fred Park 3e1b5ac930
Update docs/recipes for Singularity schema change 2019-11-13 03:15:13 +00:00
Fred Park 141ba27d36
Update OSUMicroBenchmarks recipe to MVAPICH-2.3.2 2019-11-04 18:29:14 +00:00
Fred Park 3521739088
Fix remotefs bootstrap
- Samba options were being invoked for non-samba enabled clusters
2019-10-09 15:48:40 +00:00
Fred Park 74d339c413
Add doc note regarding Windows env vars
- Update DotNet recipe to use container default working dir option
- Partially addresses #311
2019-09-09 15:26:21 +00:00
Fred Park 3a91511e50
Fix possible null node agent info on list nodes
- Resolves #307
2019-08-30 15:30:49 +00:00
Fred Park 4ab382761a
Minor post-release fixes
- Update 3rd party notices
- Move docker directory to build
- Fix old info in a few recipes
2019-08-14 15:13:28 +00:00
Fred Park 3052e98c8b
Add MVAPICH support
- More changes for #287
- Automatically source environment modules if it exists
- Fix some typos
2019-08-12 01:58:39 +00:00
Fred Park b6044b3489
Update GPU support
- Update to Docker CE 19.03.1
- Use "native" Docker/containerd GPU support
- Breaking change in jobs configuration to allow arbitrary configuration
- Update docs
- Resolves #293
2019-08-08 20:36:41 +00:00
Vincent Labonté b64c3cb324 Add Infiniband support with Open MPI and MPICH (#297)
* Add Infinibnad support with Open MPI

* Add mpiBench-Infiniband-OpenMPI recipe

* Add setup script for OpenFOAM-Infiniband-OpenMPI recipe

* Update setup script for OpenFOAM-Infiniband-OpenMPI recipe

* Add OpenFOAM-Infiniband-OpenMPI recipe

* Add documentation for recipes

* Add Infiniband support with MPICH

* Add mpiBench-Infiniband-MPICH recipe
2019-08-05 10:39:08 -04:00
Fred Park 4d69c96d79
Merge branch 'sriov-merge' into singularity3 2019-07-23 21:02:52 +00:00
Vincent Labonté cc42916cba Fixes and update of recipes (#290)
* Fix multi-instance tasks that are not a MPI task

* Add setup task script for CNTK-CPU-Infiniband-IntelMPI

* Update CNTK-CPU-Infiniband-IntelMPI recipe

* Add MPI executable path option

* Update CNTK-CPU-OpenMPI recipe

* Change the default MPI executable_path to mpirun

* Modify CNTK-CPU-Infiniband-IntelMPI recipe

* Add setup task script for CNTK-GPU-Infiniband-IntelMPI

* Update CNTK-GPU-Infiniband-IntelMPI recipe

* Add setup task script for CNTK-GPU-OpenMPI

* Add setup task script for NAMD-Infiniband-IntelMPI

* Update NAMD-Infiniband-IntelMPI recipe

* Add setup task script for OpenFOAM-Infiniband-IntelMPI

* Update OpenFOAM-Infiniband-IntelMPI recipe

* Update TensorFlow-GPU Singularity recipe

* Add setup task script for OpenFOAM-TCP-OpenMPI

* Update OpenFOAM-TCP-OpenMPI recipe

* Add support for arbitrary commands with the MPI processes_per_node option

* Fix MPI with native images

* Modify CNTK-CPU-Infiniband-IntelMPI recipe

* Modify CNTK-GPU-Infiniband-IntelMPI recipe

* Modify NAMD-Infiniband-IntelMPI recipe

* Update processes_per_node documentation

* Fix `pool images list` with Singularity images

* Modify OpenFOAM-Infiniband-IntelMPI set up script

* Add check for mpi setting with Windows

* Add auto scratch support with OpenFOAM-Infiniband-IntelMPI recipe

* Modify OpenFOAM-TCP-OpenMPI set up script

* Add auto scratch support with OpenFOAM-TCP-OpenMPI recipe

* Add mpiBench-IntelMPI recipe

* Add mpiBench-MPICH recipe

* Add mpiBench-OpenMPI recipe

* Resolve PR comments

* Resolve PR comments
2019-07-17 18:57:06 -07:00
Fred Park ec3c6f49be
Update Intel Benchmarks using Hc
- Docker and Singularity3 updates
2019-07-15 03:32:24 +00:00
Fred Park 25fec92273
Support Hc/Hb
- Support RDMA bifurcation
- Update platform docs for CentOS-HPC 7.6
2019-07-15 03:32:04 +00:00
Fred Park 9736cea593
Update NAMD-TCP recipe
- Fix connection issues
2019-07-09 21:46:34 +00:00
Fred Park a16e125056
Doc updates 2019-05-10 14:26:14 +00:00
Fred Park 6767ce2326
Update recipe README 2019-04-04 08:10:45 -07:00
Fred Park ec7af5b7c1
Various updates
- Update docs
- Update azure-batch dependency
- Set Slurm scheduling option defer mode
2019-03-22 14:50:26 -07:00
Fred Park 314037f76f
Slurm on Batch feature
- Package and use Slurm 18.08 instead of default from distro repo
- Slurm "master" contains separate controller and login nodes
- Integrate RemoteFS shared file system into Slurm cluster
- Auto feature tagging on Slurm nodes
- Support CentOS 7, Ubuntu 16.04, Ubuntu 18.04 Batch pools as Slurm
  node targets
- Unify login and Batch pools on cluster user based on login user
- Auto provision passwordless SSH user on compute nodes with login user
  context
- Add slurm cluster commands, including orchestrate command
- Add separate SSH for controller, login, nodes
- Add Slurm configuration doc
- Add Slurm guide
- Add Slurm recipe
- Update usage doc
- Remove deprecated MSI VM extension from monitoring and federation
- Fix pool nodes count on non-existent pool
- Refactor SSH info to allow offsets
- Add fs cluster orchestrate command
2019-02-28 12:11:10 -08:00
Fred Park 296c39b39b
Add Windows recipes
- Resolves #246
2018-12-10 10:58:47 -08:00
Fred Park 70532fa4ae
Add Genomics recipes
- BLAST and RNASeq pipelines
- Fix adding tasks to an existing job with existing merge tasks
- Add support for force_enable_task_dependencies at the job level
- Fix doc typos
2018-11-29 08:58:01 -08:00
Fred Park 151d546bed
Add PyTorch recipes 2018-11-29 08:57:56 -08:00
Fred Park 02c6e110d7
Kata containers support
- Make Singularity runtime install optional
- Add `restrict_default_bind_mounts` option to jobs spec
- Provide a default container runtime option
2018-11-05 11:24:17 -08:00
Fred Park db009a3dc9
Various minor updates 2018-10-04 13:51:27 -07:00
Fred Park 6f35a50bf7
Rename MADL recipe to HPMLA 2018-10-04 09:59:19 -07:00
Fred Park f2aa2a4eba
Fix Docker Hub private registry login 2018-07-27 08:13:28 -07:00
Fred Park a36bc0c4c1
Add MADL recipe link from README
- Fix line endings and non-line length PEP8 issues in MADL recipe py helper
2018-06-27 12:23:58 -07:00
danyrouh baf9ce0685 Microsoft Azure Distributed Linear Learner Recipe (#195) 2018-06-27 12:10:22 -07:00
Fred Park c2cd8f5e67
Update TensorFlow Docker image ref 2018-06-11 13:37:24 -07:00
Fred Park 7f2200a31d
Update recipes to refer to platform image docs
- Resolves #186
2018-04-18 12:35:26 -07:00
Fred Park 850b54936f
Fix AAD support for non-public Azure cloud regions
- Add authority_url option
- Update default management endpoint
2018-02-16 09:21:00 -08:00
Fred Park 13b1c0e2d4
Allow CentOS-HPC 7.1 provisioning 2018-02-09 09:33:25 -08:00
Fred Park 663548d91e
Minor updates
- Update dependencies
- Check for out of date deps
2018-02-07 14:32:32 -08:00
Fred Park 59fca99319 Fix public ip configs in RemoteFS recipes 2017-11-15 08:59:06 -08:00
Fred Park 061bf59906 Update recipes SSH username 2017-11-13 09:25:20 -08:00
Fred Park 5b2af24f00 Retry image configuration errors
- Add TensorFlow-GPU Singularity recipe
2017-10-29 20:29:34 -07:00
Fred Park efb9f404f7 Rename `image` to `docker_image` in recipes 2017-10-29 20:28:29 -07:00
Fred Park edd602aed4 Add HPCG Singularity recipe 2017-10-29 09:38:46 -07:00
Fred Park 6f62740292 Tag for 3.0.0a2 release 2017-10-27 11:35:48 -07:00
Fred Park adc0c865ea docker_volumes is now volumes 2017-10-17 12:57:02 -07:00
Fred Park cbddcdfbff Use docker_image in favor of image in tasks 2017-10-03 10:05:17 -07:00
Fred Park 2c8f6b299a Update all recipes to use YAML (#122) 2017-10-03 10:04:03 -07:00
Fred Park 01c2f89ba5 Handle package db conflicts
- TensorFlow recipe typos
2017-09-22 14:59:59 -07:00
Fred Park c67225c3e9 Add Caffe2 recipes
- Resolves #112
2017-08-14 10:39:15 -07:00
Fred Park 05e9773741 Update recipes
- `remove_container_after_exit` is now defaulted enabled
- Move to CentOS-HPC 7.3 for ib recipes
2017-08-03 19:13:57 -07:00
Fred Park e5ffd492ab Update CNTK CPU infiniband recipe to 2.1 2017-08-03 16:28:23 -07:00
Fred Park a8f721a752 Add CNTK GPU ib recipe 2017-08-03 13:40:32 -07:00
Fred Park dadf574691 Update CNTK-GPU-OpenMPI to 2.1 2017-08-03 08:44:48 -07:00
Fred Park fee0d0e4ae Update CNTK-CPU-OpenMPI recipes for 2.1 2017-08-02 15:12:45 -07:00
Fred Park 804169d48b Update TensorFlow recipes to 1.2.1
- Fix Distributed TF recipe and make launcher generalized
2017-08-02 11:00:53 -07:00
Fred Park 8eb2197d23 Allow CentOS 7.3 on NC/NV 2017-07-06 11:12:05 -07:00