Граф коммитов

48 Коммитов

Автор SHA1 Сообщение Дата
mpage 1025e577f0 [warden] Add privileged option to run/spawn
This adds an options has that may be passed to spawn. The initial
option allowed is the boolean "privileged," which, if supplied,
will cause the command to be run as root.

Change-Id: I7f90553e7e3092b17d9326bb4d339c52d0a55035
2012-05-03 18:45:08 -07:00
Jesse Zhang e4ecaee5e9 [warden] No panic on receiving invalid json
The warden daemon currently faints when it sees malformed json. This
change makes the daemon slightly more robust by disconnecting the
client when warden can't parse the request and keeps the warden daemon
running.

  Test plan: pass unit tests

Change-Id: I90301efd86a18cda63db6483b9b89c7c0f454709
2012-04-19 20:16:27 -07:00
Pieter Noordhuis 5351e51ec1 warden: Allow clients to specify the container's disk size
Change-Id: I1542fba9206de25c512e32f96367f32394638324
2012-04-10 15:20:17 -07:00
Pieter Noordhuis f86f081a9a warden: Rename for consistency
Change-Id: I9f61ab52490a4fd0e54b058c2a9b9d8b577b589b
2012-03-29 15:37:35 -07:00
Pieter Noordhuis 919b9f9811 warden: Include IP addresses in INFO output
Change-Id: If34301257375fb48c2249a7e73801d259d0ccb4a
2012-03-29 15:37:35 -07:00
Pieter Noordhuis 3af322a694 warden: Use string keys in unchecked hashes
Change-Id: I4056bea0bcbe673d030747ba25b3f168b81769ad
2012-03-29 15:37:35 -07:00
Pieter Noordhuis abfa27cb34 warden: Sanitize container grace time
Change-Id: Ice6ed09239f9ff7dfc82aac17c039bd5e933ff55
2012-03-29 15:37:35 -07:00
Pieter Noordhuis d61441fa09 warden: Grow memory limit in OOM test to fix false positive
Change-Id: I4a13464ecb2c2ea99b1ea5928f6e74e4784e215e
2012-03-28 17:18:30 -07:00
mpage 07b448e065 [warden] Don't write to /etc/mtab when bind mounting
The ``mount'' command writes to /etc/mtab by default, expecting
a corresponding ``unmount'' command to remove the added entries.
However, we rely on the destruction of the container's mount
namespace to handle unmounting bind mounts, so there is no
corresponding ``unmount'' to the initial ``mount''. This leaves
/etc/mtab out of sync.

Change-Id: Id71bfc0361e7595b616e160704cea115927a72fa
2012-03-26 19:08:49 -07:00
Pieter Noordhuis 41d459a827 warden: Sanitize config to CREATE before using it
In addition, this patch makes the "bind_mounts" configuration option use
an Array to keep its mount specifications instead of a Hash. The order
in which mount specifications are passed may be important.

Change-Id: I6083dde1fbcbab61dc7a5d50f29a4e707eb270b0
2012-03-15 13:38:50 -07:00
mpage cfb4a4fe7e [warden] Add support for bind mounting paths at creation
This add support for bind mounting parts of the host filesystem into
the container during creation. Mount permissions may also be supplied.

Change-Id: I37b5daf73b57507dc1cbde659ffa4e93a4e128c7
2012-03-05 10:56:14 -08:00
mpage 985bcd1944 [warden] Add support for copying files to/from containers
This diff provides support for copying files/directories to/from containers.
Callers may also supply a "owner" (in the form of <user>:<group>) when
copying files out of a container.

Test plan:
- Spot checking with warden-repl
- Added new specs

Change-Id: I2629d29b1a90b8b2d2b4484c5baebe73496053cf
2012-02-27 17:37:51 -08:00
Pieter Noordhuis 179f830028 warden: Spec grace timer behavior
Change-Id: Iff9b8239aa59147508430db6ac4cb6a33912d4d6
2012-02-23 15:01:28 -08:00
Pieter Noordhuis d314adeab7 warden: Specs for container base class
Change-Id: Icc4e65caa509acc90b93836fcc3fa7227fcef606
2012-02-23 15:01:28 -08:00
Pieter Noordhuis 1f1a77b782 warden: Move address release delay logic to NetworkPool
Change-Id: I9f7fee68bd37df5515f239e68b45a5d459e43f1d
2012-02-23 15:01:28 -08:00
Pieter Noordhuis d3427a88a2 warden: Remove quota-related code
Change-Id: Ia9d5280070c2905320c9e0e8460637f3b265f317
2012-02-23 09:49:22 -08:00
Pieter Noordhuis 1e9791c71d warden: Fix error where container could be destroyed twice
Change-Id: Ic724f89a764505f375211169fdea5c99255998e2
2012-02-17 14:05:10 -08:00
Pieter Noordhuis a61e4304b0 warden: Remove artifacts after spec run via clear.sh
Change-Id: I1ed5160704f87edb272f99dcd5f04f3db97c8d47
2012-02-17 13:47:05 -08:00
Pieter Noordhuis eedbb297e5 warden: Jobs now return full stdout and stderr
Change-Id: I2394d9e27692f3ee558f4dcb7947bcaaa8ad3510
2012-02-16 15:27:55 -08:00
Pieter Noordhuis bdc2b5ca37 warden: Don't allow the container to swap
This requires both memory.limit_in_bytes and memory.memsw.limit_in_bytes
in the container's cgroup to be set to the memory limit.

Change-Id: I1a989d55394fd243fd8fe21a955d461c71215648
2012-02-16 15:27:55 -08:00
Pieter Noordhuis 788994c1ec warden: Only include Linux-related features on Linux
Change-Id: I5f4057f145d307ad6f0a484359550dd15eb828d8
2012-02-08 15:04:29 -08:00
Pieter Noordhuis e2cffce255 warden: Expect the SIGKILL exit status after OOM
Change-Id: Ifb9e5c32b795a3988034f9e91d59ab0b5a89ef83
2012-01-20 16:17:18 -08:00
mpage aabe6be56e [warden] Add script to enable quotas, fix error with test uid pool name
Change-Id: I425f64a0a501749c421a28ebb8d919fd2d124fe4
2012-01-20 13:10:13 -08:00
mpage cc5931aaa8 [warden] Add option to specify config file to warden:start
Change-Id: I08ffd0b8801459fb7a26bd16d40608741d5f5c15
2012-01-20 13:10:13 -08:00
Pieter Noordhuis 23d702b442 warden: Remove stale files
Change-Id: I08f1d98beb7286a62850b3e476c8805da1673662
2012-01-19 09:52:01 -08:00
Pieter Noordhuis 8609330c2e warden: Rework container management scripts
Change-Id: I3588b0a3cee13817903d8c2e103734b8ff55cf6d
2012-01-13 09:47:31 -08:00
Pieter Noordhuis 1ee0321d1e warden: Exit status is nil when it is unknown
Change-Id: Id36df57a136746d6d2a8644237748618bf5628e9
2011-12-20 15:05:03 -08:00
Pieter Noordhuis c70b3dc66b warden: Use new network offset per test
This solves resource contention issues when running the test suite.

Change-Id: If7bd81a7edb86856ad9f7e9dd4db3fb257df9ef4
2011-12-20 14:35:53 -08:00
mpage a9c1598002 Add "info" command
This merges the old information obtained via "stats" into the
"info" command and adds the following extra information: limits
and container state.

Test plan:
- New unit tests pass

Change-Id: Ia18b5663513552d2876aebf84f8595329fd241b0
2011-12-20 12:19:50 -08:00
Pieter Noordhuis ae6e59fd5a warden: Switch to a line-based JSON
Switch protocol because of current shortcomings in hiredis. This commit
also removes the EM-based client and adds a new blocking client for the
new protocol.

Change-Id: I6527a4ca0cf949470df8d7fa465997a4b25a234e
2011-12-20 10:40:04 -08:00
mpage 2b3b646c7b Minor test cleanup
Change-Id: Id3c4c1e62e416a54ba824098ef4bb1c4ab8ab44b
2011-12-16 21:04:18 -08:00
mpage da2b9e0c87 Add "stopped" state to containers, transition there on oom/quota events
This adds the "stopped" state to containers. When stopped, all processes
inside the container are killed. After entering the stopped state, the
container may be introspected, existing spawns may be linked, and the
container can be destroyed.

We now transition to the "stopped" state after an oom or quota violation
occurs in order to allow users to poke around inside the container and
retrieve artifacts.

Test plan:
- Unit tests pass

Change-Id: I13a6b19ba9b3fc0bcd208de6b677f4fdbb21d757
2011-12-16 11:57:34 -08:00
mpage 20d7f09482 Add 'stats' command to warden
This diff add support for the "stats" command. It returns a list
of key value pairs that correspond to container-specific metrics.
Currently, "disk_usage_B" and "mem_usage_B" are returned for LXC.

Test plan:
- New unit tests pass

Change-Id: I1314410412095f1d37f2b1d675fd70295b41ca73
2011-12-13 15:28:34 -08:00
mpage 3370c4b48f Add command to set memory limit for LXC containers
This allows users to set the memory limit on their containers
via 'memory.limit_in_bytes'. Containers whose cgroup exceeds the supplied
limit will be destroyed automatically.

Test plan:
- New unit test passes

Change-Id: I52e3eb74bcaf5f43f186d98bc599d06f02f359c0
2011-12-13 11:59:11 -08:00
Pieter Noordhuis ba261795dc warden: Allow whitelisting of hosts/ports at runtime
Change-Id: Ib81e0af0bd1a0fec6a32c76b326d06da60f6bf7e
2011-12-12 15:02:50 -08:00
Pieter Noordhuis 81e3e7706d warden: Support outbound network traffic filtering
This adds support for black/whitelisting of networks. These
configuration parameters are applied as follows:

* If destination IP is whitelisted => allow
* If destination IP is blacklisted => deny
* Default => allow

To use a deny-by-default policy, the 0.0.0.0/0 network may be added as a
blacklisted and all traffic that is not explicitly allowed is denied.

Change-Id: I3b610e15d02dc95acce04d27902b172afa27c4d3
2011-12-09 11:59:30 -08:00
mpage 284d00d8ab Add quota monitor to tear down containers that exceed their disk usage limits
This diff includes a few of things:
1. A class for monitor quota usage and tearing down containers that exceed their limits.
2. A small C program for reporting quota usage. This is necessary because repquota
   has several bugs that prevent us from using it to reliably check quota usage.
3. Small refactor that moves everything under the :quota config hash. This makes sense,
   as we only need the uidpool if quota support is enabled.

Test plan:
- Existing unit tests pass
- New unit tests pass

Change-Id: I8dccd91bb4571f051c7a328d22436b77c6fb3f0c
2011-12-07 15:34:15 -08:00
mpage e0cfcc15ea Add command for setting/retrieving disk usage limit.
This add plumbing for setting limits on containers. The syntax is:

    limit <handle> <name>             # Retrieve limit named <name>
    limit <handle> <name> <limit>+    # Set limit named <name>

This diff also implements support for setting and retrieving disk usage
limits (enforced via quotas).

Test plan:
- All existing tests pass
- New tests pass.

Change-Id: I9b5d3c7ffdd22e06a2dcfd2605146a3a3ae48d23
2011-12-05 16:33:11 -08:00
mpage ce10fc982b Add uid pool to LXC container
This is in preparation for adding quota support to the LXC container.
If a uid pool is supplied the LXC container will now attempt to acquire a
uid for each container that is created. A WardenError will be raised in the
event that a uid cannot be acquired. Note that the uid is the same both inside
and outside the container in order to facilitate enforcement of disk quotas.

Test plan:
- All existing tests pass.
- New tests pass.

Change-Id: Ic509c0cd9e40275f92207d35bb9bfb943d700026
2011-12-05 14:30:00 -08:00
Pieter Noordhuis ca29ace2ba warden: Allow external ports to be mapped to containers
Change-Id: Id24aee6a6c78137cfefe82b4cbc4bf1e7c771c92
2011-11-29 11:02:06 -08:00
Pieter Noordhuis f68086a3bc warden: Abstraction for executing sequence of commands
Instead of simply running commands in a fire and forget manner, we want
to optionally run rollback hooks to undo (or try to undo) the mutation
caused by the initial command. When a command halfway through the
sequence fails, it should execute a rollback for all commands up to the
failing one.

This is best-effort approach, as commands may irreversibly mutate global
state. It can therefore be seen best as an abstraction to a set of nested
begin/rescue/end blocks.

Change-Id: I5132a0135fce07c69760e77fa6517d28048a3a77
2011-11-28 17:27:17 -08:00
Pieter Noordhuis 2f15735ef8 warden: Make container grace time configurable
Change-Id: Ia85dab81f15b9c689c666157d1fa0bc64e1eb448
2011-11-21 15:34:40 -08:00
Pieter Noordhuis 22b6809712 warden: Add spawn/link tests; refactor command dispatch
Change-Id: If54c7f4a3d0e305a9892e777f62b1a6ad1282413
2011-11-21 13:59:58 -08:00
Pieter Noordhuis 56411bb83a warden: Support spawn/link semantics
This enables client connections to go away without killing the scripts
that it started. Clients can reconnect and re-attach to the scripts they
started to wait until completion, or to reap the status when the script
has already exited.

Change-Id: Id9b89f241d9bba89b5ea211b5737d957ca48c62a
2011-11-18 18:36:16 -08:00
Pieter Noordhuis 42175351fb warden: Implement insecure handler
The insecure handler does not try to jail the container in any way.
Scripts run in insecure containers have access to the full file system,
network interfaces, process table, etc. They are run as the user that
runs warden.

Change-Id: Ia5eeed8fb0df1f67750eebe1cd69cf86e7c552f7
2011-11-18 15:24:00 -08:00
Pieter Noordhuis 248335e082 warden: More prep work to support both LXC and insecure
Change-Id: I886d7de9244bfd60ba8c2dea72f5857d3c1d4e73
2011-11-17 19:32:38 -08:00
Pieter Noordhuis 7e9405dd99 warden: Move separate concerns to their own files
This also includes prep work for supporting execution environments that
are less secure than LXC. This is required for MCF, OSX, dev, etc.

Change-Id: I82ee0aef3dce60ea0f174598663b9d78ab89bb6c
2011-11-17 17:21:06 -08:00
Pieter Noordhuis 33459df52c Add warden: manager for ephemeral Linux containers
Change-Id: I3913e5be6392ffec183e9eb58d933988c15deafb
2011-11-17 13:37:25 -08:00