Add a custom report_garbage handler to collect and remove
garbage .idx files from the pack directory.
Signed-off-by: Doug Kelly <dougk.ff7@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Pack garbage, noticeably stale .idx files, can be cleaned up during
a garbage collection. This tests to ensure such garbage is properly
cleaned up.
Note that the prior test for checking pack garbage with count-objects
left some stale garbage after the test exited. This has also been
corrected.
Signed-off-by: Doug Kelly <dougk.ff7@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When for_each_packed_object is called, we call
prepare_packed_git() to make sure we have the actual list of
packs. But the latter does not actually open the pack
indices, meaning that pack->nr_objects may simply be 0 if
the pack has not otherwise been used since the program
started.
In practice, this didn't come up for the current callers,
because they iterate the packed objects only after iterating
all reachable objects (so for it to matter you would have to
have a pack consisting only of unreachable objects). But it
is a dangerous and confusing interface that should be fixed
for future callers.
Note that we do not end the iteration when a pack cannot be
opened, but we do return an error. That lets you complete
the iteration even in actively-repacked repository where an
.idx file may racily go away, but it also lets callers know
that they may not have gotten the complete list (which the
current reachability-check caller does care about).
We have to tweak one of the prune tests due to the changed
return value; an earlier test creates bogus .idx files and
does not clean them up. Having to make this tweak is a good
thing; it means we will not prune in a broken repository,
and the test confirms that we do not negatively impact a
more lenient caller, count-objects.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The string in 'base' contains a path suffix to a specific object;
when its value is used, the suffix must either be filled (as in
stat_sha1_file, open_sha1_file, check_and_freshen_nonlocal) or
cleared (as in prepare_packed_git) to avoid junk at the end.
660c889e (sha1_file: add for_each iterators for loose and packed
objects, 2014-10-15) introduced loose_from_alt_odb(), but this did
neither and treated 'base' as a complete path to the "base" object
directory, instead of a pointer to the "base" of the full path
string.
The trailing path after 'base' is still initialized to NUL, hiding
the bug in some common cases. Additionally the descendent
for_each_file_in_obj_subdir() function swallows ENOENT, so an error
only shows if the alternate's path was last filled with a valid
object (where statting /path/to/existing/00/0bjectfile/00 fails).
Signed-off-by: Jonathon Mah <me@JonathonMah.com>
Helped-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
For small outputs, we sometimes use:
test "$(some_cmd)" = "something we expect"
instead of a full test_cmp. The downside of this is that
when it fails, there is no output at all from the script.
Let's introduce a small helper to make tests easier to
debug.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is slightly more robust (checking "! test -f" would not
notice a directory of the same name, though that is not
likely to happen here). It also makes debugging easier, as
the test script will output a message on failure.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
HEAD is not explicitly used as a starting commit for
calculating reachability, so if it's detached and reflogs
are disabled it may be pruned.
Add tests which demonstrate it. Test 'prune: prune former HEAD after checking
out branch' also reverts changes to repository.
Signed-off-by: Max Kirillov <max@max630.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch teaches "prune" to remove shallow roots that are no longer
reachable from any refs (e.g. when the relevant refs are removed).
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
prepare_packed_git_one() is modified to allow count-objects to hook a
report function to so we don't need to duplicate the pack searching
logic in count-objects.c. When report_pack_garbage is NULL, the
overhead is insignificant.
The garbage is reported with warning() instead of error() in packed
garbage case because it's not an error to have garbage. Loose garbage
is still reported as errors and will be converted to warnings later.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
On systems where the local time and file modification time may be out of
sync (e.g. test directory on NFS) t3306 and t5305 can fail because prune
compares times such as "now" (client time) with file modification times
(server times for remote file systems). I.e., these are spurious test
failures.
Avoid this by setting the relevant modification times to the local time.
Noticed on a system with as little as 2s time skew.
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Previously, prune treated an expiration time of 0 to mean that no
expire argument was supplied, and everything should be pruned. As a
result, "prune --expire=never" would prune all unreachable objects,
regardless of their timestamp.
prune can be called with --expire=never automatically by gc, when the
gc.pruneExpire configuration is set to "never".
Signed-off-by: Adam Simpkins <simpkins@facebook.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Test the effect of an earlier change by f7835a2 (preserve mtime of local
clone, 2009-09-12) to keep stale loose object files stale in the new
repository when a local clone is made by copying files in .git/
directory.
Signed-off-by: Clemens Buchacher <drizzd@aon.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With this patch, "git gc --no-prune" will not prune any loose (and
dangling) object, and "git gc --prune=5.minutes.ago" will prune
all loose objects older than 5 minutes.
This patch benefitted from suggestions by Thomas Rast and Jan Krï¿œger.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
test-chmtime can adjust the mtime of a file based on the file's mtime, or
based on the system time. For files accessed over NFS, the file's mtime is
set by the NFS server, and as such may vary a great deal from the NFS
client's system time if the clocks of the client and server are out of
sync. Since these tests are testing the expire feature of git-prune, an
incorrect mtime could cause a file to be expired or not expired incorrectly
and produce a test failure.
Avoid this NFS pitfall by modifying the calls to test-chmtime so that the
mtime is adjusted based on the system time, rather than the file's mtime.
Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Finally, this resurrects the documented behaviour to protect other
objects listed on the command line from getting pruned.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It seems that git prune changed behaviour with respect to revisions added
from command line, probably when it became a builtin. Currently, it prints
a short usage and exits: instead, it should take those revisions into
account and not prune them. So add a couple of test to point this out.
We'll be fixing this by switching to parse_options(), so add tests to
detect bogus command line parameters as well, to keep ourselves from
introducing regressions.
Signed-off-by: Michele Ballabio <barra_cuda@katamail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The only reason we did not call "prune" in git-gc was that it is an
inherently dangerous operation: if there is a commit going on, you will
prune loose objects that were just created, and are, in fact, needed by the
commit object just about to be created.
Since it is dangerous, we told users so. That led to many users not even
daring to run it when it was actually safe. Besides, they are users, and
should not have to remember such details as when to call git-gc with
--prune, or to call git-prune directly.
Of course, the consequence was that "git gc --auto" gets triggered much
more often than we would like, since unreferenced loose objects (such as
left-overs from a rebase or a reset --hard) were never pruned.
Alas, git-prune recently learnt the option --expire <minimum-age>, which
makes it a much safer operation. This allows us to call prune from git-gc,
with a grace period of 2 weeks for the unreferenced loose objects (this
value was determined in a discussion on the git list as a safe one).
If you want to override this grace period, just set the config variable
gc.pruneExpire to a different value; an example would be
[gc]
pruneExpire = 6.months.ago
or even "never", if you feel really paranoid.
Note that this new behaviour makes "--prune" be a no-op.
While adding a test to t5304-prune.sh (since it really tests the implicit
call to "prune"), also the original test for "prune --expire" was moved
there from t1410-reflog.sh, where it did not belong.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Write errors when repacking (eg, due to out-of-space conditions)
can leave temporary packs (and possibly other files beginning
with "tmp_") lying around which no existing
codepath removes and which aren't obvious to the casual user.
These can also be multi-megabyte files wasting noticeable space.
Unfortunately there's no way to definitely tell in builtin-prune
that a tmp_ file is not being used by a concurrent process,
such as a fetch. However, it is documented that pruning should
only be done on a quiet repository and --expire is honoured
(using code from Johannes Schindelin, along with a test case
he wrote) so that its safety is the same as that of loose
object pruning.
Since they might be signs of a problem (unlike orphaned loose
objects) the names of any removed files are printed.
Signed-off-by: David Tweed (david.tweed@gmail.com)
Acked-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Doug Kelly
Jeff King
Jonathon Mah
Max Kirillov
Justin Lebar
Nguyễn Thái Ngọc Duy
Michael J Gruber
Adam Simpkins
Clemens Buchacher
Johannes Schindelin
Brandon Casey
Junio C Hamano
Michele Ballabio
David Steven Tweed