Граф коммитов

41 Коммитов

Автор SHA1 Сообщение Дата
Fred Park 08204092be Add CentOS GlusterFS support
- Update recipes
2016-09-15 12:47:43 -07:00
Fred Park 646cff6631 Add TensorFlow-Distributed recipe
- Fix SSH user expiry within 1 day
- Fix some README/dockerfile typos
2016-09-13 11:43:28 -07:00
Fred Park af98bdfb57 Add sample configs for all existing recipes
- Fix temp file creation for cross-platform
2016-09-09 13:49:59 -07:00
Fred Park e8d5e7a8a3 Automatically detect nvidia driver version
- Fix azure-storage dependencies for non-shipyard docker image setup
- Add no-install-recommends to apt-gets in node prep
2016-09-08 21:06:21 -07:00
Fred Park 0338fc612f Add gettaskfile/getnodefile actions
- Add .gitattributes to designate text files eol as LF
- Update configuration doc with required/optional tags
2016-09-08 20:15:11 -07:00
Fred Park 028768a61a Add CNTK recipes
- Add TCP optimization
- Fix job autocompletion
- Update azure-storage requirement to 0.33.0
2016-09-07 21:40:57 -07:00
Fred Park eea254aeb9 Add job auto-completion for multi-instance tasks 2016-09-06 10:32:42 -07:00
Fred Park ada1feb00a Add GPU support 2016-09-02 01:42:54 -07:00
Fred Park 6a414ca829 Add multi-instance task doc
- Ensure --net=host for multi-instance task coordination command
2016-09-01 14:00:14 -07:00
Fred Park 53c6e5a673 Add del mi cleanup job and stream file actions
- Add base recipes readme
2016-09-01 09:07:05 -07:00
Fred Park a666d7d9f4 Add explicit inter node comm property for pool
- Allow inter node comm to work independently of p2p transfers
2016-08-31 22:20:18 -07:00
Fred Park 75a27746a4 Complete guide/docs
- Add pool_current_dedicated support for multi-instance tasks
2016-08-31 20:44:44 -07:00
Fred Park 7f49641074 First part of the guide/docs
- Modify placement of some configuration settings
2016-08-31 15:35:33 -07:00
Fred Park d53436b6ca Prep for ghpages
- Update for pathlib2
- Add bare start point for docs
2016-08-31 11:16:43 -07:00
Fred Park e813bbeae7 Enforce VM size check for infiniband tasks 2016-08-31 09:07:33 -07:00
Fred Park c5368207cd Add more features to pool/tasks
- Add infiniband support
- Add max tasks per node
- Properly handle multi-instance tasks with docker run <-> exec
- Add docker multi-instance cleanup helper
2016-08-31 02:21:50 -07:00
Fred Park 351800344d Add multi-instance support for tasks 2016-08-30 15:31:23 -07:00
Fred Park 0d48919afa Azure file dvd support for all supported hosts
- Add default init scripts to avoid surprise config changes
- Add block flag to pool for image ready
2016-08-29 12:12:50 -07:00
Fred Park 35fb3f588b Add support for more host OSes
- Ubuntu 14.04, Debian 8, CentOS 7.x, RHEL 7.x, OpenSUSE 13.2/42.1,
  SLES 12/12-sp1
- Improve graphing
- Prevent metadata clear on existing pool
2016-08-28 19:43:53 -07:00
Fred Park 4cb1f79c07 Update docstrings and typing information
- Add MIT license text to all py files
2016-08-27 17:42:36 -07:00
Fred Park 703e9e7fd1 Reorganize project 2016-08-27 11:35:32 -07:00
Fred Park 0da28d7e0a Add stderr redirect to logger
- Refactor some common functions
- Rename services table to images
2016-08-27 11:30:55 -07:00
Fred Park e9ce32e69f Fix shipyard container issues
- Downgrade libtorrent to 1.0.9 due to DHT issues
- Add task dependency support
- Add CONTRIBUTING.md, requirements.txt and .travis.yml
2016-08-26 23:09:10 -07:00
Fred Park aa4add34d5 Support shipyard as docker container
- Add Dockerfile
- Add command file for container
2016-08-26 15:34:30 -07:00
Fred Park bc30023330 Scale fixes
- Add timing recording toggle
2016-08-25 13:07:57 -07:00
Fred Park 1fbad08ee5 Add ssh docker user/tunnel generation
- Compact some vars
2016-08-24 20:32:53 -07:00
Fred Park 359dfdad85 Use proper logging in shipyard and cascade
- Fix shipyard.py for Python2.7 compatibility
- Allow option to reboot nodes that go into start task failed state
- Correctly pin gr-done events
- Reduce chattiness of torrent info dumps
2016-08-24 15:36:23 -07:00
Fred Park 39dd9bf5c6 Fix storage env var issues
- Add support for additional resource files on docker jobs
2016-08-24 09:09:37 -07:00
Fred Park a99bee7100 Pass through more storage endpoint info
- Compact env vars
- Azure file endpoint
- Docker private registry realm
2016-08-23 15:32:49 -07:00
Fred Park ee242c21d6 Split configuration files further
- Allow docker public hub passthrough in private context
- Pass config through in more places in shipyard
2016-08-23 14:50:17 -07:00
Fred Park 6fb2d7e221 Add azurefile docker volume driver support
- Add auto packaging of registry:2 docker image if not present
2016-08-19 15:13:33 -07:00
Fred Park 8182a73847 Add docker job/task support
- Fix cascade compression bug
- Add full job and task spec support
2016-08-19 11:28:29 -07:00
Fred Park f1a666db41 Add redirect for torrents to blob
- Always install private registry to every node
- Fix uncompressed torrenting
2016-08-18 15:17:04 -07:00
Fred Park e4f58c6ff4 Allow non-p2p and no dpr modes
- Add jp block script
- Fix some data graph bugs
- Begin fixing non-reproducible .tar.gz docker save images
2016-08-17 08:30:53 -07:00
Fred Park d4c44f811c Add more configurable options for P2P mode
- Add WIP of graph.py
2016-08-12 15:31:05 -07:00
Fred Park d06d3649a0 Configuration changes
- Move more private registry settings from hardcode to config
- Split config file into two
2016-08-12 09:21:33 -07:00
Fred Park 0271e54b1a Upload perf timings to table 2016-08-09 15:23:15 -07:00
Fred Park 2e2b2c0f96 Migrate to DHT table, load/register functionality 2016-08-09 12:47:58 -07:00
Fred Park 83a422c64c Updates for single session/DHT fixes 2016-08-08 09:38:12 -07:00
Fred Park 553380df8a Break out private registry setup from cascade
- Fix torrent start, queue msg get and torrentinfo table merge
2016-07-20 13:12:20 -07:00
Fred Park 41a6f7c258 Populate with initial code 2016-07-20 13:12:07 -07:00