Azure
/
batch-shipyard
зеркало из
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

Doc updates 

- Emulate "set -e" for cmd wrap for Windows
This commit is contained in:
Fred Park 2017-11-05 11:48:08 -08:00
Родитель 2e8b43df55
Коммит f6c74bd8b7
8 изменённых файлов: 152 добавлений и 103 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

1
CHANGELOG.md
Просмотреть файл

@ -16,6 +16,7 @@ more information.
- Mount `/opt/intel` into Singularity containers - Mount `/opt/intel` into Singularity containers
- Retry image configuration error pulls from Docker registries - Retry image configuration error pulls from Docker registries
- AAD MFA token cache on Python2 - AAD MFA token cache on Python2
- Non-native coordination command fix, if not specified
## [3.0.0a2] - 2017-10-27 ## [3.0.0a2] - 2017-10-27
### Added ### Added

11
README.md
Просмотреть файл

@ -87,6 +87,8 @@ tunneling to Docker Hosts on compute nodes
* Support for execution on an * Support for execution on an
[Azure Function App environment](http://batch-shipyard.readthedocs.io/en/latest/60-batch-shipyard-site-extension/) [Azure Function App environment](http://batch-shipyard.readthedocs.io/en/latest/60-batch-shipyard-site-extension/)
* Support for [custom host images](http://batch-shipyard.readthedocs.io/en/latest/63-batch-shipyard-custom-images/) * Support for [custom host images](http://batch-shipyard.readthedocs.io/en/latest/63-batch-shipyard-custom-images/)
* Support for [Windows Containers](https://docs.microsoft.com/en-us/virtualization/windowscontainers/about/)
on compliant Windows compute node pools
## Installation ## Installation
### Azure Cloud Shell ### Azure Cloud Shell
@ -111,9 +113,12 @@ section for various sample Docker workloads using Azure Batch and Batch
Shipyard. Shipyard.
## Batch Shipyard Compute Node OS Support ## Batch Shipyard Compute Node OS Support
Batch Shipyard is currently compatible with Azure Batch supported Marketplace Batch Shipyard is currently compatible with most Azure Batch supported
Linux VMs and Marketplace Linux VMs,
[compliant Linux custom images](http://batch-shipyard.readthedocs.io/en/latest/63-batch-shipyard-custom-images/). [compliant Linux custom images](http://batch-shipyard.readthedocs.io/en/latest/63-batch-shipyard-custom-images/),
and native Azure Batch
[Windows Server with Containers](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/Microsoft.WindowsServer?tab=Overview)
VMs.
## Change Log ## Change Log
Please see the Please see the

3
convoy/util.py
Просмотреть файл

@ -274,7 +274,8 @@ def wrap_commands_in_shell(commands, windows=False, wait=True):
:return: wrapped commands :return: wrapped commands
""" """
if windows: if windows:
return 'cmd.exe /c {}'.format(' && '.join(commands)) tmp = ['(({}) || exit /b)'.format(x) for x in commands]
return 'cmd.exe /c "{}"'.format(' && '.join(tmp))
else: else:
return '/bin/bash -c \'set -e; set -o pipefail; {}{}\''.format( return '/bin/bash -c \'set -e; set -o pipefail; {}{}\''.format(
'; '.join(commands), '; wait' if wait else '') '; '.join(commands), '; wait' if wait else '')

224
docs/20-batch-shipyard-usage.md
Просмотреть файл

@ -216,40 +216,45 @@ then invoked the `cert add` command. Please see the
## `data` Command ## `data` Command
The `data` command has the following sub-commands: The `data` command has the following sub-commands:
``` ```
getfile Retrieve file(s) from a job/task files Compute node file actions
getfilenode Retrieve file(s) from a compute node ingress Ingress data into Azure
ingress Ingress data into Azure
listfiles List files for tasks in jobs
stream Stream a text file to the local console
``` ```
* `getfile` will retrieve a file with job, task, filename semantics The `data files` sub-command has the following sub-sub-commands:
* `--all --filespec <jobid>,<taskid>,<include pattern>` can be given to ```
download all files for the job and task with an optional include pattern list List files for tasks in jobs
* `--filespec <jobid>,<taskid>,<filename>` can be given to download one node Retrieve file(s) from a compute node
specific file from the job and task. If `<taskid>` is set to stream Stream a file as text to the local console or...
`@FIRSTRUNNING`, then the first running task within the job of `<jobid>` task Retrieve file(s) from a job/task
will be used to locate the `<filename>`. ```
* `getfilenode` will retrieve a file with node id and filename semantics
* `files list` will list files for all tasks in jobs
* `--jobid` force scope to just this job id
* `--taskid` force scope to just this task id
* `files node ` will retrieve a file with node id and filename semantics
* `--all --filespec <nodeid>,<include pattern>` can be given to download * `--all --filespec <nodeid>,<include pattern>` can be given to download
all files from the compute node with the optional include pattern all files from the compute node with the optional include pattern
* `--filespec <nodeid>,<filename>` can be given to download one * `--filespec <nodeid>,<filename>` can be given to download one
specific file from compute node specific file from compute node
* `ingress` will ingress data as specified in configuration files * `files stream` will stream a file as text (UTF-8 decoded) to the local
* `--to-fs <STORAGE_CLUSTER_ID>` transfers data as specified in console or binary if streamed to disk
configuration files to the specified remote file system storage cluster
instead of Azure Storage
* `listfiles` will list files for all tasks in jobs
* `--jobid` force scope to just this job id
* `--taskid` force scope to just this task id
* `stream` will stream a file as text (UTF-8 decoded) to the local console
or binary if streamed to disk
* `--disk` will write the streamed data as binary to disk instead of output * `--disk` will write the streamed data as binary to disk instead of output
to local console to local console
* `--filespec <jobid>,<taskid>,<filename>` can be given to stream a * `--filespec <jobid>,<taskid>,<filename>` can be given to stream a
specific file. If `<taskid>` is set to `@FIRSTRUNNING`, then the first specific file. If `<taskid>` is set to `@FIRSTRUNNING`, then the first
running task within the job of `<jobid>` will be used to locate the running task within the job of `<jobid>` will be used to locate the
`<filename>`. `<filename>`.
* `files task` will retrieve a file with job, task, filename semantics
* `--all --filespec <jobid>,<taskid>,<include pattern>` can be given to
download all files for the job and task with an optional include pattern
* `--filespec <jobid>,<taskid>,<filename>` can be given to download one
specific file from the job and task. If `<taskid>` is set to
`@FIRSTRUNNING`, then the first running task within the job of `<jobid>`
will be used to locate the `<filename>`.
* `ingress` will ingress data as specified in configuration files
* `--to-fs <STORAGE_CLUSTER_ID>` transfers data as specified in
configuration files to the specified remote file system storage cluster
instead of Azure Storage
## `fs` Command ## `fs` Command
The `fs` command has the following sub-commands which work on two different The `fs` command has the following sub-commands which work on two different
@ -345,18 +350,23 @@ assumed.
## `jobs` Command ## `jobs` Command
The `jobs` command has the following sub-commands: The `jobs` command has the following sub-commands:
``` ```
add Add jobs add Add jobs
cmi Cleanup multi-instance jobs cmi Cleanup non-native multi-instance jobs
del Delete jobs and job schedules del Delete jobs and job schedules
deltasks Delete specified tasks in jobs disable Disable jobs and job schedules
disable Disable jobs and job schedules enable Enable jobs and job schedules
enable Enable jobs and job schedules list List jobs
list List jobs migrate Migrate jobs or job schedules to another pool
listtasks List tasks within jobs stats Get statistics about jobs
migrate Migrate jobs or job schedules to another pool tasks Tasks actions
stats Get statistics about jobs term Terminate jobs and job schedules
term Terminate jobs and job schedules ```
termtasks Terminate specified tasks in jobs
The `jobs tasks` sub-command has the following sub-sub-commands:
```
del Delete specified tasks in jobs
list List tasks within jobs
term Terminate specified tasks in jobs
``` ```
* `add` will add all jobs and tasks defined in the jobs configuration file * `add` will add all jobs and tasks defined in the jobs configuration file
@ -364,8 +374,8 @@ to the Batch pool
* `--recreate` will recreate any completed jobs with the same id * `--recreate` will recreate any completed jobs with the same id
* `--tail` will tail the specified file of the last job and task added * `--tail` will tail the specified file of the last job and task added
with this command invocation with this command invocation
* `cmi` will cleanup any stale multi-instance tasks and jobs. Note that this * `cmi` will cleanup any stale non-native multi-instance tasks and jobs. Note
sub-command is typically not required if `multi_instance_auto_complete` is that this sub-command is typically not required if `auto_complete` is
set to `true` in the job specification for the job. set to `true` in the job specification for the job.
* `--delete` will delete any stale cleanup jobs * `--delete` will delete any stale cleanup jobs
* `del` will delete jobs and job scheudles specified in the jobs * `del` will delete jobs and job scheudles specified in the jobs
@ -381,12 +391,6 @@ cleaned up.
Termination of running tasks requires a valid SSH user if the tasks Termination of running tasks requires a valid SSH user if the tasks
are running on a non-`native` container support pool. are running on a non-`native` container support pool.
* `--wait` will wait for deletion to complete * `--wait` will wait for deletion to complete
* `deltasks` will delete tasks within jobs specified in the jobs
configuration file. Active or running tasks will be terminated first on
non-`native` container support pools.
* `--jobid` force deletion scope to just this job id
* `--taskid` force deletion scope to just this task id
* `--wait` will wait for deletion to complete
* `disable` will disable jobs or job schedules * `disable` will disable jobs or job schedules
* `--jobid` force disable scope to just this job id * `--jobid` force disable scope to just this job id
* `--jobscheduleid` force disable scope to just this job schedule id * `--jobscheduleid` force disable scope to just this job schedule id
@ -397,11 +401,6 @@ non-`native` container support pools.
* `--jobid` force enable scope to just this job id * `--jobid` force enable scope to just this job id
* `--jobscheduleid` force enable scope to just this job schedule id * `--jobscheduleid` force enable scope to just this job schedule id
* `list` will list all jobs in the Batch account * `list` will list all jobs in the Batch account
* `listtasks` will list tasks from jobs specified in the jobs configuration
file
* `--all` list all tasks in all jobs in the account
* `--jobid` force scope to just this job id
* `--poll-until-tasks-complete` will poll until all tasks have completed
* `migrate` will migrate jobs or job schedules to another pool. Ensure that * `migrate` will migrate jobs or job schedules to another pool. Ensure that
the new target pool has the Docker images required to run the job. the new target pool has the Docker images required to run the job.
* `--jobid` force migration scope to just this job id * `--jobid` force migration scope to just this job id
@ -412,6 +411,24 @@ the new target pool has the Docker images required to run the job.
* `--wait` wait for running tasks to complete * `--wait` wait for running tasks to complete
* `stats` will generate a statistics summary of a job or jobs * `stats` will generate a statistics summary of a job or jobs
* `--jobid` will query the specified job instead of all jobs * `--jobid` will query the specified job instead of all jobs
* `tasks del` will delete tasks within jobs specified in the jobs
configuration file. Active or running tasks will be terminated first on
non-`native` container support pools.
* `--jobid` force deletion scope to just this job id
* `--taskid` force deletion scope to just this task id
* `--wait` will wait for deletion to complete
* `tasks list` will list tasks from jobs specified in the jobs
configuration file
* `--all` list all tasks in all jobs in the account
* `--jobid` force scope to just this job id
* `--poll-until-tasks-complete` will poll until all tasks have completed
* `tasks term` will terminate tasks within jobs specified in the jobs
configuration file. Termination of running tasks requires a valid SSH
user if tasks are running on a non-`native` container support pool.
* `--force` force send docker kill signal regardless of task state
* `--jobid` force termination scope to just this job id
* `--taskid` force termination scope to just this task id
* `--wait` will wait for termination to complete
* `term` will terminate jobs and job schedules found in the jobs * `term` will terminate jobs and job schedules found in the jobs
configuration file. If an autopool is specified for all jobs and a jobid configuration file. If an autopool is specified for all jobs and a jobid
option is not specified, the storage associated with the autopool will be option is not specified, the storage associated with the autopool will be
@ -425,13 +442,6 @@ cleaned up.
Termination of running tasks requires a valid SSH user if tasks are Termination of running tasks requires a valid SSH user if tasks are
running on a non-`native` container support pool. running on a non-`native` container support pool.
* `--wait` will wait for termination to complete * `--wait` will wait for termination to complete
* `termtasks` will terminate tasks within jobs specified in the jobs
configuration file. Termination of running tasks requires a valid SSH
user if tasks are running on a non-`native` container support pool.
* `--force` force send docker kill signal regardless of task state
* `--jobid` force termination scope to just this job id
* `--taskid` force termination scope to just this task id
* `--wait` will wait for termination to complete
## `keyvault` Command ## `keyvault` Command
The `keyvault` command has the following sub-commands: The `keyvault` command has the following sub-commands:
@ -480,35 +490,54 @@ or has run the specified task
## `pool` Command ## `pool` Command
The `pool` command has the following sub-commands: The `pool` command has the following sub-commands:
``` ```
add Add a pool to the Batch account add Add a pool to the Batch account
asu Add an SSH user to all nodes in pool autoscale Autoscale actions
autoscale Pool autoscale actions del Delete a pool from the Batch account
del Delete a pool from the Batch account images Container images actions
delnode Delete a node from a pool list List all pools in the Batch account
dsu Delete an SSH user from all nodes in pool listskus List available VM configurations available to...
grls Get remote login settings for all nodes in... nodes Compute node actions
list List all pools in the Batch account resize Resize a pool
listimages List Docker images in a pool ssh Interactively login via SSH to a node in a...
listnodes List nodes in pool stats Get statistics about a pool
listskus List available VM configurations available to... user Remote user actions
rebootnode Reboot a node or nodes in a pool ```
resize Resize a pool
ssh Interactively login via SSH to a node in a... The `pool autoscale` sub-command has the following sub-sub-commands:
stats Get statistics about a pool ```
udi Update Docker images in a pool disable Disable autoscale on a pool
enable Enable autoscale on a pool
evaluate Evaluate autoscale formula
lastexec Get the result of the last execution of the...
```
The `pool images` sub-command has the following sub-sub-commands:
```
list List container images in a pool
update Update container images in a pool
```
The `pool nodes` sub-command has the following sub-sub-commands:
```
del Delete a node from a pool
grls Get remote login settings for all nodes in...
list List nodes in pool
reboot Reboot a node or nodes in a pool
```
The `pool user` sub-command has the following sub-sub-commands:
```
add Add a remote user to all nodes in pool
del Delete a remote user from all nodes in pool
``` ```
* `add` will add the pool defined in the pool configuration file to the * `add` will add the pool defined in the pool configuration file to the
Batch account Batch account
* `asu` will add the SSH user defined in the pool configuration file to * `autoscale disable` will disable autoscale on the pool
all nodes in the specified pool * `autoscale enable` will enable autoscale on the pool
* `autoscale` will invoke the autoscale subcommand. The autoscale * `autoscale evaluate` will evaluate the autoscale formula in the pool
subcommand has 4 subcommands: configuration file
* `disable` will disable autoscale on the pool * `autoscale lastexec` will query the last execution information for autoscale
* `enable` will enable autoscale on the pool
* `evaluate` will evaluate the autoscale formula in the pool configuration
file
* `lastexec` will query the last execution information for autoscale
* `del` will delete the pool defined in the pool configuration file from * `del` will delete the pool defined in the pool configuration file from
the Batch account along with associated metadata in Azure Storage used by the Batch account along with associated metadata in Azure Storage used by
Batch Shipyard. It is recommended to use this command instead of deleting Batch Shipyard. It is recommended to use this command instead of deleting
@ -518,22 +547,28 @@ Azure Storage.
* `--poolid` will delete the specified pool instead of the pool from the * `--poolid` will delete the specified pool instead of the pool from the
pool configuration file pool configuration file
* `--wait` will wait for deletion to complete * `--wait` will wait for deletion to complete
* `delnode` will delete the specified node from the pool * `images list` will query the nodes in the pool for Docker images. Common
and mismatched images will be listed. Requires a provisioned SSH user and
private key.
* `images update` will update container images on all compute nodes of the
pool. This command may require a valid SSH user.
* `--docker-image` will restrict the update to just the Docker image or
image:tag
* `--docker-image-digest` will restrict the update to just the Docker
image or image:tag and a specific digest
* `--singularity-image` will restrict the update to just the Singularity
image or image:tag
* `list` will list all pools in the Batch account
* `nodes del` will delete the specified node from the pool
* `--all-start-task-failed` will delete all nodes in the start task * `--all-start-task-failed` will delete all nodes in the start task
failed state failed state
* `--all-starting` will delete all nodes in the starting state * `--all-starting` will delete all nodes in the starting state
* `--all-unusable` will delete all nodes in the unusable state * `--all-unusable` will delete all nodes in the unusable state
* `--nodeid` is the node id to delete * `--nodeid` is the node id to delete
* `dsu` will delete the SSH user defined in the pool configuration file * `nodes grls` will retrieve all of the remote login settings for every node
from all nodes in the specified pool
* `grls` will retrieve all of the remote login settings for every node
in the specified pool in the specified pool
* `list` will list all pools in the Batch account * `nodes list` will list all nodes in the specified pool
* `listimages` will query the nodes in the pool for Docker images. Common * `nodes reboot` will reboot a specified node in the pool
and mismatched images will be listed. Requires a provisioned SSH user and
private key.
* `listnodes` will list all nodes in the specified pool
* `rebootnode` will reboot a specified node in the pool
* `--all-start-task-failed` will reboot all nodes in the start task * `--all-start-task-failed` will reboot all nodes in the start task
failed state failed state
* `--nodeid` is the node id to reboot * `--nodeid` is the node id to reboot
@ -552,11 +587,10 @@ configuration file
* `stats` will generate a statistics summary of the pool * `stats` will generate a statistics summary of the pool
* `--poolid` will query the specified pool instead of the pool from the * `--poolid` will query the specified pool instead of the pool from the
pool configuration file pool configuration file
* `udi` will update Docker images on all compute nodes of the pool. This * `user add` will add an SSH or RDP user defined in the pool
command requires a valid SSH user. configuration file to all nodes in the specified pool
* `--image` will restrict the update to just the image or image:tag * `user del` will delete the SSH or RDP user defined in the pool
* `--digest` will restrict the update to just the image or image:tag and configuration file from all nodes in the specified pool
a specific digest
## `storage` Command ## `storage` Command
The `storage` command has the following sub-commands: The `storage` command has the following sub-commands:
@ -578,6 +612,8 @@ for metadata purposes
shipyard pool add --credentials credentials.yaml --config config.yaml --pool pool.yaml shipyard pool add --credentials credentials.yaml --config config.yaml --pool pool.yaml
# ... or if all config files are in the current working directory named as above ... # ... or if all config files are in the current working directory named as above ...
# (note this is strictly not necessary as Batch Shipyard will search the
# current working directory if the options above are not explicitly specified
shipyard pool add --configdir . shipyard pool add --configdir .

2
docs/70-batch-shipyard-data-movement.md
Просмотреть файл

@ -138,6 +138,8 @@ Note that `files` is an array, therefore, Batch Shipyard accepts any number
of `source`/`destination` pairings and even mixed GlusterFS and Azure Storage of `source`/`destination` pairings and even mixed GlusterFS and Azure Storage
ingress objects. ingress objects.
Data ingress from on-premises to Windows pools is not supported.
### From Azure Storage (Blob and File) ### From Azure Storage (Blob and File)
Data from Azure Storage can be ingressed to compute nodes in many different Data from Azure Storage can be ingressed to compute nodes in many different
ways with Batch Shipyard. The recommended method when using Batch Shipyard ways with Batch Shipyard. The recommended method when using Batch Shipyard

4
docs/97-faq.md
Просмотреть файл

@ -85,5 +85,5 @@ possible.
* Yes, please see [the guide](63-batch-shipyard-custom-images.md). * Yes, please see [the guide](63-batch-shipyard-custom-images.md).
#### Does Batch Shipyard support Windows Server Containers? #### Does Batch Shipyard support Windows Server Containers?
* Not at this time, we are tracking the issue * Yes, but with some feature, configuration, and CLI limitations. Please see
[here](https://github.com/Azure/batch-shipyard/issues/7). the [current limitations](99-current-limitations.md) doc for more information.

8
docs/99-current-limitations.md
Просмотреть файл

@ -49,8 +49,12 @@ no longer performed before a pool is created and will instead result in
a ResizeError on the pool if not all compute nodes can be allocated. a ResizeError on the pool if not all compute nodes can be allocated.
* Data movement between Batch tasks as defined by `input_data`:`azure_batch` * Data movement between Batch tasks as defined by `input_data`:`azure_batch`
is restricted to Batch accounts with keys (non-AAD). is restricted to Batch accounts with keys (non-AAD).
* Windows Server 2016, Clear Linux, and Oracle Linux are not supported with * Clear Linux and Oracle Linux are not supported with Batch Shipyard at this
Batch Shipyard at this time. time.
* Certain features, configuration options and usage commands are not
supported with Windows compute node pools such as on-premises to compute
node direct data ingress, storage cluster mounting, and credential
encryption support.
* Task dependencies are incompatible with multi-instance tasks. This is a * Task dependencies are incompatible with multi-instance tasks. This is a
current limitation of the underlying Azure Batch service. current limitation of the underlying Azure Batch service.
* Only Intel MPI (or polling-based IB uverbs) can be used in conjunction * Only Intel MPI (or polling-based IB uverbs) can be used in conjunction

2
shipyard.py
Просмотреть файл

@ -1472,7 +1472,7 @@ def jobs_del(
@aad_options @aad_options
@pass_cli_context @pass_cli_context
def jobs_cmi(ctx, delete): def jobs_cmi(ctx, delete):
"""Cleanup multi-instance jobs""" """Cleanup non-native multi-instance jobs"""
ctx.initialize_for_batch() ctx.initialize_for_batch()
convoy.fleet.action_jobs_cmi(ctx.batch_client, ctx.config, delete) convoy.fleet.action_jobs_cmi(ctx.batch_client, ctx.config, delete)