This makes it possible to pass the rest of the testsuite without aufs in the kernel.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
All archive that are created from somewhere generally have to be closed, because
at some point there is a file or a pipe or something that backs them. So, we
make archive.Archive a ReadCloser. However, code consuming archives does not
typically close them so we add an archive.ArchiveReader and use that when we're
only reading.
We then change all the Tar/Archive places to create ReadClosers, and to properly
close them everywhere.
As an added bonus we can use ReadCloserWrapper rather than EofReader in several places,
which is good as EofReader doesn't always work right. For instance, many compression
schemes like gzip knows it is EOF before having read the EOF from the stream, so the
EofCloser never sees an EOF.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
We no longer pass "pool" anywhere that uses byHash() per the last
commit, so we can now remove this hack.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Shutdown contains debug warnings like:
[debug] deviceset.go:699 [deviceset docker-0:33-17945897] waitRemove(/dev/mapper/docker-0:33-17945897-pool)
[debug] deviceset.go:380 libdevmapper(3): libdm-common.c:552 (-1) Device /dev/mapper/docker-0:33-17945897-pool not found
This is because shutdown is using removeDeviceAndWait() to remove the pool device and the
wait part fails because the pool is gone.
We fix this by adding a pool specific removal function which avoids all the trickiness of the
normal remove.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
If /dev/loop-control exists on the system running the test then
ioctlLoopCtlGetFree() will be called, but if not it won't. It does
not exist in the standard docker build environment, so the tests
currently require this to not be called. This makes it instead
optional, allowing the tests to run on e.g. Fedora.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
We're currently leaving around lots of empty directories in
/var/lib/docker/devicemapper/mnt/ for removed images and containers.
Fix this by removing the directory when the device is removed.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
This makes the device mapper not zero out blocks allocated on the
thinp device. This is safe in our use case, as we access the device
via a filesystem that doesn't leak any uninitialized data to userspace.
This partially helps with https://github.com/dotcloud/docker/issues/3280
and should generally improve preformance on the devicemapper backend.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
For some reason we seem to get transient EBUSY when removing
thinp devices, which prohibit removing containers. When
this happens we retry a few times which seems to fix the
issue for me.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
This makes sure the device is removed just like in deactivateDevice.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
We already have some kind of refcounting in DeviceSet, this fleshes
it out to allow it to completely subsume the refcounting in
devmapper.Driver.
This allows us to drop the double refcounting, and the locking inside
devmapper.Driver. This, in particular the locking simplification will
make it easier in the future to parallelize the device mapper.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
This adds a function that calls the lowlevel removeDevice
and then waits for it to finish.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
"Remove" is a bit overloaded, as it is also what deactivating a
device mapper device is called. Using "delete" is more clear here.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Several of the error messages were wrong, but not one of them was helpful, so this fixes them to include the important information (the actual error), and a correct description of which operation failed.
Docker-DCO-1.1-Signed-off-by: Andrew Page <admwiggin@gmail.com> (github: tianon)
This fixes compilation issues when btrfs.h isn't available (because we just need the relevant structs, which for userspace programs are supposed to come from btrfs-progs instead of the kernel headers).
Docker-DCO-1.1-Signed-off-by: Andrew Page <admwiggin@gmail.com> (github: tianon)
Rather than creating a new directory and moving it there before
deleting that new directory, just move the directory we intend to
delete.
In the old way, the Mkdirall could fail, which meant that you
couldn't delete containers when the disk was full.
Tested.
Docker-DCO-1.1-Signed-off-by: Peter Waller <p@pwaller.net> (github: pwaller)
This is an experimental btrfs driver. To use it you must have
/var/lib/docker mounted on a btrfs filesystem and explicitly
specify DOCKER_DRIVER=btrfs in the docker daemon environment.
It works by using subvolumes for the docker image/container layers.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
This implements the new Put() operation such that
Get()/Put() maintains a refcount for each ID, mounting
only on first Get() and unmounting on the last Get().
This means we avoid littering the system with lots of mounts
and free resources related to them.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
This implements the new Put() operation such that
Get()/Put() maintains a refcount for each ID, mounting
only on first Get() and unmounting on the last Get().
This means we avoid littering the system with lots of mounts
and active devicemapper devices and free resources related
to them.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
asdfljk
This makes all users of Put() have a corresponding call
to Get() which means we will be able to track whether
any particular ID is in use and if not unmount it.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
This simplifies that code that calls out to tar by removing support
for now unused features.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Mount is self contained and generic, it should be in pkg, to allow other pkg modules to use it.
Docker-DCO-1.1-Signed-off-by: Paul Nasrat <pnasrat@gmail.com> (github: pnasrat)
This works around the fact that deleting a device in a thin pool
doesn't discard the free space. Unfortunately even this is not perfect,
as it seems discards are respected only for blocks that has never been
shared in the thin device code. However, this has been fixed in the
upstream kernel device-mapper tree:
http://git.kernel.org/cgit/linux/kernel/git/device-mapper/linux-dm.git/commit/?h=for-next&id=0ab1c92ff748b745c1ed7cde31bb37ad2c5f901a
When this hits the kernel I belive this will fully return space
for removed images/containers to the host FS. For now it only
helps partially (which is better than nothing).
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Paul Morie
Alexander Larsson
Guillaume J. Charmes
unclejack
Michael Crosby
Tianon Gravi
Guillaume J. Charmes
Peter Waller
Paul Nasrat
Clayton Coleman
Michael Crosby
Guillaume J. Charmes
Josh Poimboeuf
Victor Vieux
Andrews Medina
Guillaume J. Charmes