Merge pull request #293: Merge in current ds/maintenance-part-3

This includes all changes from #292, but then also `ds/maintenance-part-3` from upstream. This is _not_ the final maintenance builtin, but is very close. It's time to start making full updates in Scalar that depend on them.
This commit is contained in:
Derrick Stolee 2020-10-02 15:40:16 -04:00
Родитель 606ed55b32 1bbf20ec01
Коммит 4135e3a839
22 изменённых файлов: 1307 добавлений и 28 удалений

1
.gitignore поставляемый
Просмотреть файл

@ -67,6 +67,7 @@
/git-filter-branch
/git-fmt-merge-msg
/git-for-each-ref
/git-for-each-repo
/git-format-patch
/git-fsck
/git-fsck-objects

Просмотреть файл

@ -627,8 +627,8 @@ core.useReplaceRefs::
core.multiPackIndex::
Use the multi-pack-index file to track multiple packfiles using a
single index. See link:technical/multi-pack-index.html[the
multi-pack-index design document].
single index. See linkgit:git-multi-pack-index[1] for more
information. Defaults to true.
core.gvfs::
Enable the features needed for GVFS. This value can be set to true

Просмотреть файл

@ -1,3 +1,8 @@
maintenance.auto::
This boolean config option controls whether some commands run
`git maintenance run --auto` after doing their normal work. Defaults
to true.
maintenance.<task>.enabled::
This boolean config option controls whether the maintenance task
with name `<task>` is run when no `--task` option is specified to
@ -5,6 +10,11 @@ maintenance.<task>.enabled::
`--task` option exists. By default, only `maintenance.gc.enabled`
is true.
maintenance.<task>.schedule::
This config option controls whether or not the given `<task>` runs
during a `git maintenance run --schedule=<frequency>` command. The
value must be one of "hourly", "daily", or "weekly".
maintenance.commit-graph.auto::
This integer config option controls how often the `commit-graph` task
should be run as part of `git maintenance run --auto`. If zero, then
@ -14,3 +24,21 @@ maintenance.commit-graph.auto::
reachable commits that are not in the commit-graph file is at least
the value of `maintenance.commit-graph.auto`. The default value is
100.
maintenance.loose-objects.auto::
This integer config option controls how often the `loose-objects` task
should be run as part of `git maintenance run --auto`. If zero, then
the `loose-objects` task will not run with the `--auto` option. A
negative value will force the task to run every time. Otherwise, a
positive value implies the command should run when the number of
loose objects is at least the value of `maintenance.loose-objects.auto`.
The default value is 100.
maintenance.incremental-repack.auto::
This integer config option controls how often the `incremental-repack`
task should be run as part of `git maintenance run --auto`. If zero,
then the `incremental-repack` task will not run with the `--auto`
option. A negative value will force the task to run every time.
Otherwise, a positive value implies the command should run when the
number of pack-files not in the multi-pack-index is at least the value
of `maintenance.incremental-repack.auto`. The default value is 10.

Просмотреть файл

@ -0,0 +1,59 @@
git-for-each-repo(1)
====================
NAME
----
git-for-each-repo - Run a Git command on a list of repositories
SYNOPSIS
--------
[verse]
'git for-each-repo' --config=<config> [--] <arguments>
DESCRIPTION
-----------
Run a Git command on a list of repositories. The arguments after the
known options or `--` indicator are used as the arguments for the Git
subprocess.
THIS COMMAND IS EXPERIMENTAL. THE BEHAVIOR MAY CHANGE.
For example, we could run maintenance on each of a list of repositories
stored in a `maintenance.repo` config variable using
-------------
git for-each-repo --config=maintenance.repo maintenance run
-------------
This will run `git -C <repo> maintenance run` for each value `<repo>`
in the multi-valued config variable `maintenance.repo`.
OPTIONS
-------
--config=<config>::
Use the given config variable as a multi-valued list storing
absolute path names. Iterate on that list of paths to run
the given arguments.
+
These config values are loaded from system, global, and local Git config,
as available. If `git for-each-repo` is run in a directory that is not a
Git repository, then only the system and global config is used.
SUBPROCESS BEHAVIOR
-------------------
If any `git -C <repo> <arguments>` subprocess returns a non-zero exit code,
then the `git for-each-repo` process returns that exit code without running
more subprocesses.
Each `git -C <repo> <arguments>` subprocess inherits the standard file
descriptors `stdin`, `stdout`, and `stderr`.
GIT
---
Part of the linkgit:git[1] suite

Просмотреть файл

@ -29,6 +29,30 @@ Git repository.
SUBCOMMANDS
-----------
register::
Initialize Git config values so any scheduled maintenance will
start running on this repository. This adds the repository to the
`maintenance.repo` config variable in the current user's global
config and enables some recommended configuration values for
`maintenance.<task>.schedule`. The tasks that are enabled are safe
for running in the background without disrupting foreground
processes.
+
If your repository has no `maintenance.<task>.schedule` configuration
values set, then Git will use a recommended default schedule that performs
background maintenance that will not interrupt foreground commands. The
default schedule is as follows:
+
* `gc`: disabled.
* `commit-graph`: hourly.
* `prefetch`: hourly.
* `loose-objects`: daily.
* `incremental-repack`: daily.
+
`git maintenance register` will also disable foreground maintenance by
setting `maintenance.auto = false` in the current repository. This config
setting will remain after a `git maintenance unregister` command.
run::
Run one or more maintenance tasks. If one or more `--task` options
are specified, then those tasks are run in that order. Otherwise,
@ -36,6 +60,22 @@ run::
config options are true. By default, only `maintenance.gc.enabled`
is true.
start::
Start running maintenance on the current repository. This performs
the same config updates as the `register` subcommand, then updates
the background scheduler to run `git maintenance run --scheduled`
on an hourly basis.
stop::
Halt the background maintenance schedule. The current repository
is not removed from the list of maintained repositories, in case
the background maintenance is restarted later.
unregister::
Remove the current repository from background maintenance. This
only removes the repository from the configured list. It does not
stop the background maintenance processes from running.
TASKS
-----
@ -47,6 +87,21 @@ commit-graph::
`commit-graph-chain` file. They will be deleted by a later run based
on the expiration delay.
prefetch::
The `prefetch` task updates the object directory with the latest
objects from all registered remotes. For each remote, a `git fetch`
command is run. The refmap is custom to avoid updating local or remote
branches (those in `refs/heads` or `refs/remotes`). Instead, the
remote refs are stored in `refs/prefetch/<remote>/`. Also, tags are
not updated.
+
This is done to avoid disrupting the remote-tracking branches. The end users
expect these refs to stay unmoved unless they initiate a fetch. With prefetch
task, however, the objects necessary to complete a later real fetch would
already be obtained, so the real fetch would go faster. In the ideal case,
it will just become an update to bunch of remote-tracking branches without
any object transfer.
gc::
Clean up unnecessary files and optimize the local repository. "GC"
stands for "garbage collection," but this task performs many
@ -55,6 +110,39 @@ gc::
be disruptive in some situations, as it deletes stale data. See
linkgit:git-gc[1] for more details on garbage collection in Git.
loose-objects::
The `loose-objects` job cleans up loose objects and places them into
pack-files. In order to prevent race conditions with concurrent Git
commands, it follows a two-step process. First, it deletes any loose
objects that already exist in a pack-file; concurrent Git processes
will examine the pack-file for the object data instead of the loose
object. Second, it creates a new pack-file (starting with "loose-")
containing a batch of loose objects. The batch size is limited to 50
thousand objects to prevent the job from taking too long on a
repository with many loose objects. The `gc` task writes unreachable
objects as loose objects to be cleaned up by a later step only if
they are not re-added to a pack-file; for this reason it is not
advisable to enable both the `loose-objects` and `gc` tasks at the
same time.
incremental-repack::
The `incremental-repack` job repacks the object directory
using the `multi-pack-index` feature. In order to prevent race
conditions with concurrent Git commands, it follows a two-step
process. First, it calls `git multi-pack-index expire` to delete
pack-files unreferenced by the `multi-pack-index` file. Second, it
calls `git multi-pack-index repack` to select several small
pack-files and repack them into a bigger one, and then update the
`multi-pack-index` entries that refer to the small pack-files to
refer to the new pack-file. This prepares those small pack-files
for deletion upon the next run of `git multi-pack-index expire`.
The selection of the small pack-files is such that the expected
size of the big pack-file is at least the batch size; see the
`--batch-size` option for the `repack` subcommand in
linkgit:git-multi-pack-index[1]. The default batch-size is zero,
which is a special case that attempts to repack all pack-files
into a single pack-file.
OPTIONS
-------
--auto::
@ -62,7 +150,18 @@ OPTIONS
only if certain thresholds are met. For example, the `gc` task
runs when the number of loose objects exceeds the number stored
in the `gc.auto` config setting, or when the number of pack-files
exceeds the `gc.autoPackLimit` config setting.
exceeds the `gc.autoPackLimit` config setting. Not compatible with
the `--schedule` option.
--schedule::
When combined with the `run` subcommand, run maintenance tasks
only if certain time conditions are met, as specified by the
`maintenance.<task>.schedule` config value for each `<task>`.
This config value specifies a number of seconds since the last
time that task ran, according to the `maintenance.<task>.lastRun`
config value. The tasks that are tested are those provided by
the `--task=<task>` option(s) or those with
`maintenance.<task>.enabled` set to true.
--quiet::
Do not report progress or other information over `stderr`.
@ -74,6 +173,50 @@ OPTIONS
`maintenance.<task>.enabled` configured as `true` are considered.
See the 'TASKS' section for the list of accepted `<task>` values.
TROUBLESHOOTING
---------------
The `git maintenance` command is designed to simplify the repository
maintenance patterns while minimizing user wait time during Git commands.
A variety of configuration options are available to allow customizing this
process. The default maintenance options focus on operations that complete
quickly, even on large repositories.
Users may find some cases where scheduled maintenance tasks do not run as
frequently as intended. Each `git maintenance run` command takes a lock on
the repository's object database, and this prevents other concurrent
`git maintenance run` commands from running on the same repository. Without
this safeguard, competing processes could leave the repository in an
unpredictable state.
The background maintenance schedule runs `git maintenance run` processes
on an hourly basis. Each run executes the "hourly" tasks. At midnight,
that process also executes the "daily" tasks. At midnight on the first day
of the week, that process also executes the "weekly" tasks. A single
process iterates over each registered repository, performing the scheduled
tasks for that frequency. Depending on the number of registered
repositories and their sizes, this process may take longer than an hour.
In this case, multiple `git maintenance run` commands may run on the same
repository at the same time, colliding on the object database lock. This
results in one of the two tasks not running.
If you find that some maintenance windows are taking longer than one hour
to complete, then consider reducing the complexity of your maintenance
tasks. For example, the `gc` task is much slower than the
`incremental-repack` task. However, this comes at a cost of a slightly
larger object database. Consider moving more expensive tasks to be run
less frequently.
Expert users may consider scheduling their own maintenance tasks using a
different schedule than is available through `git maintenance start` and
Git configuration options. These users should be aware of the object
database lock and how concurrent `git maintenance run` commands behave.
Further, the `git gc` command should not be combined with
`git maintenance run` commands. `git gc` modifies the object database
but does not take the lock in the same way as `git maintenance run`. If
possible, use `git maintenance run --task=gc` instead of `git gc`.
GIT
---
Part of the linkgit:git[1] suite

Просмотреть файл

@ -695,6 +695,7 @@ TEST_BUILTINS_OBJS += test-bloom.o
TEST_BUILTINS_OBJS += test-chmtime.o
TEST_BUILTINS_OBJS += test-cmp.o
TEST_BUILTINS_OBJS += test-config.o
TEST_BUILTINS_OBJS += test-crontab.o
TEST_BUILTINS_OBJS += test-ctype.o
TEST_BUILTINS_OBJS += test-date.o
TEST_BUILTINS_OBJS += test-delta.o
@ -1096,6 +1097,7 @@ BUILTIN_OBJS += builtin/fetch-pack.o
BUILTIN_OBJS += builtin/fetch.o
BUILTIN_OBJS += builtin/fmt-merge-msg.o
BUILTIN_OBJS += builtin/for-each-ref.o
BUILTIN_OBJS += builtin/for-each-repo.o
BUILTIN_OBJS += builtin/fsck.o
BUILTIN_OBJS += builtin/gc.o
BUILTIN_OBJS += builtin/get-tar-commit-id.o

Просмотреть файл

@ -155,6 +155,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix);
int cmd_fetch_pack(int argc, const char **argv, const char *prefix);
int cmd_fmt_merge_msg(int argc, const char **argv, const char *prefix);
int cmd_for_each_ref(int argc, const char **argv, const char *prefix);
int cmd_for_each_repo(int argc, const char **argv, const char *prefix);
int cmd_format_patch(int argc, const char **argv, const char *prefix);
int cmd_fsck(int argc, const char **argv, const char *prefix);
int cmd_gc(int argc, const char **argv, const char *prefix);

58
builtin/for-each-repo.c Normal file
Просмотреть файл

@ -0,0 +1,58 @@
#include "cache.h"
#include "config.h"
#include "builtin.h"
#include "parse-options.h"
#include "run-command.h"
#include "string-list.h"
static const char * const for_each_repo_usage[] = {
N_("git for-each-repo --config=<config> <command-args>"),
NULL
};
static int run_command_on_repo(const char *path,
void *cbdata)
{
int i;
struct child_process child = CHILD_PROCESS_INIT;
struct strvec *args = (struct strvec *)cbdata;
child.git_cmd = 1;
strvec_pushl(&child.args, "-C", path, NULL);
for (i = 0; i < args->nr; i++)
strvec_push(&child.args, args->v[i]);
return run_command(&child);
}
int cmd_for_each_repo(int argc, const char **argv, const char *prefix)
{
static const char *config_key = NULL;
int i, result = 0;
const struct string_list *values;
struct strvec args = STRVEC_INIT;
const struct option options[] = {
OPT_STRING(0, "config", &config_key, N_("config"),
N_("config key storing a list of repository paths")),
OPT_END()
};
argc = parse_options(argc, argv, prefix, options, for_each_repo_usage,
PARSE_OPT_STOP_AT_NON_OPTION);
if (!config_key)
die(_("missing --config=<config>"));
for (i = 0; i < argc; i++)
strvec_push(&args, argv[i]);
values = repo_config_get_value_multi(the_repository,
config_key);
for (i = 0; !result && i < values->nr; i++)
result = run_command_on_repo(values->items[i].string, &args);
return result;
}

Просмотреть файл

@ -30,6 +30,9 @@
#include "tree.h"
#include "promisor-remote.h"
#include "refs.h"
#include "remote.h"
#include "object-store.h"
#include "exec-cmd.h"
#define FAILED_RUN "failed to run %s"
@ -706,13 +709,50 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
}
static const char *const builtin_maintenance_run_usage[] = {
N_("git maintenance run [--auto] [--[no-]quiet] [--task=<task>]"),
N_("git maintenance run [--auto] [--[no-]quiet] [--task=<task>] [--schedule]"),
NULL
};
enum schedule_priority {
SCHEDULE_NONE = 0,
SCHEDULE_WEEKLY = 1,
SCHEDULE_DAILY = 2,
SCHEDULE_HOURLY = 3,
};
static enum schedule_priority parse_schedule(const char *value)
{
if (!value)
return SCHEDULE_NONE;
if (!strcasecmp(value, "hourly"))
return SCHEDULE_HOURLY;
if (!strcasecmp(value, "daily"))
return SCHEDULE_DAILY;
if (!strcasecmp(value, "weekly"))
return SCHEDULE_WEEKLY;
return SCHEDULE_NONE;
}
static int maintenance_opt_schedule(const struct option *opt, const char *arg,
int unset)
{
enum schedule_priority *priority = opt->value;
if (unset)
die(_("--no-schedule is not allowed"));
*priority = parse_schedule(arg);
if (!*priority)
die(_("unrecognized --schedule argument '%s'"), arg);
return 0;
}
struct maintenance_run_opts {
int auto_flag;
int quiet;
enum schedule_priority schedule;
};
/* Remember to update object flag allocation in object.h */
@ -820,6 +860,51 @@ static int maintenance_task_commit_graph(struct maintenance_run_opts *opts)
return 0;
}
static int fetch_remote(const char *remote, struct maintenance_run_opts *opts)
{
struct child_process child = CHILD_PROCESS_INIT;
child.git_cmd = 1;
strvec_pushl(&child.args, "fetch", remote, "--prune", "--no-tags",
"--no-write-fetch-head", "--recurse-submodules=no",
"--refmap=", NULL);
if (opts->quiet)
strvec_push(&child.args, "--quiet");
strvec_pushf(&child.args, "+refs/heads/*:refs/prefetch/%s/*", remote);
return !!run_command(&child);
}
static int append_remote(struct remote *remote, void *cbdata)
{
struct string_list *remotes = (struct string_list *)cbdata;
string_list_append(remotes, remote->name);
return 0;
}
static int maintenance_task_prefetch(struct maintenance_run_opts *opts)
{
int result = 0;
struct string_list_item *item;
struct string_list remotes = STRING_LIST_INIT_DUP;
if (for_each_remote(append_remote, &remotes)) {
error(_("failed to fill remotes"));
result = 1;
goto cleanup;
}
for_each_string_list_item(item, &remotes)
result |= fetch_remote(item->string, opts);
cleanup:
string_list_clear(&remotes, 0);
return result;
}
static int maintenance_task_gc(struct maintenance_run_opts *opts)
{
struct child_process child = CHILD_PROCESS_INIT;
@ -838,6 +923,268 @@ static int maintenance_task_gc(struct maintenance_run_opts *opts)
return run_command(&child);
}
static int prune_packed(struct maintenance_run_opts *opts)
{
struct child_process child = CHILD_PROCESS_INIT;
child.git_cmd = 1;
strvec_push(&child.args, "prune-packed");
if (opts->quiet)
strvec_push(&child.args, "--quiet");
return !!run_command(&child);
}
struct write_loose_object_data {
FILE *in;
int count;
int batch_size;
};
static int loose_object_auto_limit = 100;
static int loose_object_count(const struct object_id *oid,
const char *path,
void *data)
{
int *count = (int*)data;
if (++(*count) >= loose_object_auto_limit)
return 1;
return 0;
}
static int loose_object_auto_condition(void)
{
int count = 0;
git_config_get_int("maintenance.loose-objects.auto",
&loose_object_auto_limit);
if (!loose_object_auto_limit)
return 0;
if (loose_object_auto_limit < 0)
return 1;
return for_each_loose_file_in_objdir(the_repository->objects->odb->path,
loose_object_count,
NULL, NULL, &count);
}
static int bail_on_loose(const struct object_id *oid,
const char *path,
void *data)
{
return 1;
}
static int write_loose_object_to_stdin(const struct object_id *oid,
const char *path,
void *data)
{
struct write_loose_object_data *d = (struct write_loose_object_data *)data;
fprintf(d->in, "%s\n", oid_to_hex(oid));
return ++(d->count) > d->batch_size;
}
static int pack_loose(struct maintenance_run_opts *opts)
{
struct repository *r = the_repository;
int result = 0;
struct write_loose_object_data data;
struct child_process pack_proc = CHILD_PROCESS_INIT;
/*
* Do not start pack-objects process
* if there are no loose objects.
*/
if (!for_each_loose_file_in_objdir(r->objects->odb->path,
bail_on_loose,
NULL, NULL, NULL))
return 0;
pack_proc.git_cmd = 1;
strvec_push(&pack_proc.args, "pack-objects");
if (opts->quiet)
strvec_push(&pack_proc.args, "--quiet");
strvec_pushf(&pack_proc.args, "%s/pack/loose", r->objects->odb->path);
pack_proc.in = -1;
if (start_command(&pack_proc)) {
error(_("failed to start 'git pack-objects' process"));
return 1;
}
data.in = xfdopen(pack_proc.in, "w");
data.count = 0;
data.batch_size = 50000;
for_each_loose_file_in_objdir(r->objects->odb->path,
write_loose_object_to_stdin,
NULL,
NULL,
&data);
fclose(data.in);
if (finish_command(&pack_proc)) {
error(_("failed to finish 'git pack-objects' process"));
result = 1;
}
return result;
}
static int maintenance_task_loose_objects(struct maintenance_run_opts *opts)
{
return prune_packed(opts) || pack_loose(opts);
}
static int incremental_repack_auto_condition(void)
{
struct packed_git *p;
int enabled;
int incremental_repack_auto_limit = 10;
int count = 0;
if (git_config_get_bool("core.multiPackIndex", &enabled) ||
!enabled)
return 0;
git_config_get_int("maintenance.incremental-repack.auto",
&incremental_repack_auto_limit);
if (!incremental_repack_auto_limit)
return 0;
if (incremental_repack_auto_limit < 0)
return 1;
for (p = get_packed_git(the_repository);
count < incremental_repack_auto_limit && p;
p = p->next) {
if (!p->multi_pack_index)
count++;
}
return count >= incremental_repack_auto_limit;
}
static int multi_pack_index_write(struct maintenance_run_opts *opts)
{
struct child_process child = CHILD_PROCESS_INIT;
child.git_cmd = 1;
strvec_pushl(&child.args, "multi-pack-index", "write", NULL);
if (opts->quiet)
strvec_push(&child.args, "--no-progress");
if (run_command(&child))
return error(_("failed to write multi-pack-index"));
return 0;
}
static int multi_pack_index_expire(struct maintenance_run_opts *opts)
{
struct child_process child = CHILD_PROCESS_INIT;
child.git_cmd = 1;
strvec_pushl(&child.args, "multi-pack-index", "expire", NULL);
if (opts->quiet)
strvec_push(&child.args, "--no-progress");
close_object_store(the_repository->objects);
if (run_command(&child))
return error(_("'git multi-pack-index expire' failed"));
return 0;
}
#define TWO_GIGABYTES (INT32_MAX)
static off_t get_auto_pack_size(void)
{
/*
* The "auto" value is special: we optimize for
* one large pack-file (i.e. from a clone) and
* expect the rest to be small and they can be
* repacked quickly.
*
* The strategy we select here is to select a
* size that is one more than the second largest
* pack-file. This ensures that we will repack
* at least two packs if there are three or more
* packs.
*/
off_t max_size = 0;
off_t second_largest_size = 0;
off_t result_size;
struct packed_git *p;
struct repository *r = the_repository;
reprepare_packed_git(r);
for (p = get_all_packs(r); p; p = p->next) {
if (p->pack_size > max_size) {
second_largest_size = max_size;
max_size = p->pack_size;
} else if (p->pack_size > second_largest_size)
second_largest_size = p->pack_size;
}
result_size = second_largest_size + 1;
/* But limit ourselves to a batch size of 2g */
if (result_size > TWO_GIGABYTES)
result_size = TWO_GIGABYTES;
return result_size;
}
static int multi_pack_index_repack(struct maintenance_run_opts *opts)
{
struct child_process child = CHILD_PROCESS_INIT;
child.git_cmd = 1;
strvec_pushl(&child.args, "multi-pack-index", "repack", NULL);
if (opts->quiet)
strvec_push(&child.args, "--no-progress");
strvec_pushf(&child.args, "--batch-size=%"PRIuMAX,
(uintmax_t)get_auto_pack_size());
close_object_store(the_repository->objects);
if (run_command(&child))
return error(_("'git multi-pack-index repack' failed"));
return 0;
}
static int maintenance_task_incremental_repack(struct maintenance_run_opts *opts)
{
prepare_repo_settings(the_repository);
if (!the_repository->settings.core_multi_pack_index) {
warning(_("skipping incremental-repack task because core.multiPackIndex is disabled"));
return 0;
}
if (multi_pack_index_write(opts))
return 1;
if (multi_pack_index_expire(opts))
return 1;
if (multi_pack_index_repack(opts))
return 1;
return 0;
}
typedef int maintenance_task_fn(struct maintenance_run_opts *opts);
/*
@ -853,11 +1200,16 @@ struct maintenance_task {
maintenance_auto_fn *auto_condition;
unsigned enabled:1;
enum schedule_priority schedule;
/* -1 if not selected. */
int selected_order;
};
enum maintenance_task_label {
TASK_PREFETCH,
TASK_LOOSE_OBJECTS,
TASK_INCREMENTAL_REPACK,
TASK_GC,
TASK_COMMIT_GRAPH,
@ -866,6 +1218,20 @@ enum maintenance_task_label {
};
static struct maintenance_task tasks[] = {
[TASK_PREFETCH] = {
"prefetch",
maintenance_task_prefetch,
},
[TASK_LOOSE_OBJECTS] = {
"loose-objects",
maintenance_task_loose_objects,
loose_object_auto_condition,
},
[TASK_INCREMENTAL_REPACK] = {
"incremental-repack",
maintenance_task_incremental_repack,
incremental_repack_auto_condition,
},
[TASK_GC] = {
"gc",
maintenance_task_gc,
@ -889,6 +1255,59 @@ static int compare_tasks_by_selection(const void *a_, const void *b_)
return b->selected_order - a->selected_order;
}
static int has_schedule_config(void)
{
int i, found = 0;
struct strbuf config_name = STRBUF_INIT;
size_t prefix;
strbuf_addstr(&config_name, "maintenance.");
prefix = config_name.len;
for (i = 0; !found && i < TASK__COUNT; i++) {
char *value;
strbuf_setlen(&config_name, prefix);
strbuf_addf(&config_name, "%s.schedule", tasks[i].name);
if (!git_config_get_string(config_name.buf, &value)) {
found = 1;
FREE_AND_NULL(value);
}
strbuf_setlen(&config_name, prefix);
strbuf_addf(&config_name, "%s.enabled", tasks[i].name);
if (!git_config_get_string(config_name.buf, &value)) {
found = 1;
FREE_AND_NULL(value);
}
}
strbuf_release(&config_name);
return found;
}
static void set_recommended_schedule(void)
{
if (has_schedule_config())
return;
tasks[TASK_GC].enabled = 0;
tasks[TASK_PREFETCH].enabled = 1;
tasks[TASK_PREFETCH].schedule = SCHEDULE_HOURLY;
tasks[TASK_COMMIT_GRAPH].enabled = 1;
tasks[TASK_COMMIT_GRAPH].schedule = SCHEDULE_HOURLY;
tasks[TASK_LOOSE_OBJECTS].enabled = 1;
tasks[TASK_LOOSE_OBJECTS].schedule = SCHEDULE_DAILY;
tasks[TASK_INCREMENTAL_REPACK].enabled = 1;
tasks[TASK_INCREMENTAL_REPACK].schedule = SCHEDULE_DAILY;
}
static int maintenance_run_tasks(struct maintenance_run_opts *opts)
{
int i, found_selected = 0;
@ -918,6 +1337,8 @@ static int maintenance_run_tasks(struct maintenance_run_opts *opts)
if (found_selected)
QSORT(tasks, TASK__COUNT, compare_tasks_by_selection);
else if (opts->schedule != SCHEDULE_NONE)
set_recommended_schedule();
for (i = 0; i < TASK__COUNT; i++) {
if (found_selected && tasks[i].selected_order < 0)
@ -931,6 +1352,9 @@ static int maintenance_run_tasks(struct maintenance_run_opts *opts)
!tasks[i].auto_condition()))
continue;
if (opts->schedule && tasks[i].schedule < opts->schedule)
continue;
trace2_region_enter("maintenance", tasks[i].name, r);
if (tasks[i].fn(opts)) {
error(_("task '%s' failed"), tasks[i].name);
@ -951,13 +1375,23 @@ static void initialize_task_config(void)
for (i = 0; i < TASK__COUNT; i++) {
int config_value;
char *config_str;
strbuf_setlen(&config_name, 0);
strbuf_reset(&config_name);
strbuf_addf(&config_name, "maintenance.%s.enabled",
tasks[i].name);
if (!git_config_get_bool(config_name.buf, &config_value))
tasks[i].enabled = config_value;
strbuf_reset(&config_name);
strbuf_addf(&config_name, "maintenance.%s.schedule",
tasks[i].name);
if (!git_config_get_string(config_name.buf, &config_str)) {
tasks[i].schedule = parse_schedule(config_str);
free(config_str);
}
}
strbuf_release(&config_name);
@ -1001,6 +1435,9 @@ static int maintenance_run(int argc, const char **argv, const char *prefix)
struct option builtin_maintenance_run_options[] = {
OPT_BOOL(0, "auto", &opts.auto_flag,
N_("run tasks based on the state of the repository")),
OPT_CALLBACK(0, "schedule", &opts.schedule, N_("frequency"),
N_("run tasks based on frequency"),
maintenance_opt_schedule),
OPT_BOOL(0, "quiet", &opts.quiet,
N_("do not report progress or other information over stderr")),
OPT_CALLBACK_F(0, "task", NULL, N_("task"),
@ -1021,13 +1458,187 @@ static int maintenance_run(int argc, const char **argv, const char *prefix)
builtin_maintenance_run_usage,
PARSE_OPT_STOP_AT_NON_OPTION);
if (opts.auto_flag && opts.schedule)
die(_("use at most one of --auto and --schedule=<frequency>"));
if (argc != 0)
usage_with_options(builtin_maintenance_run_usage,
builtin_maintenance_run_options);
return maintenance_run_tasks(&opts);
}
static const char builtin_maintenance_usage[] = N_("git maintenance run [<options>]");
static int maintenance_register(void)
{
struct child_process config_set = CHILD_PROCESS_INIT;
struct child_process config_get = CHILD_PROCESS_INIT;
/* There is no current repository, so skip registering it */
if (!the_repository || !the_repository->gitdir)
return 0;
/* Disable foreground maintenance */
git_config_set("maintenance.auto", "false");
config_get.git_cmd = 1;
strvec_pushl(&config_get.args, "config", "--global", "--get", "maintenance.repo",
the_repository->worktree ? the_repository->worktree
: the_repository->gitdir,
NULL);
config_get.out = -1;
if (start_command(&config_get))
return error(_("failed to run 'git config'"));
/* We already have this value in our config! */
if (!finish_command(&config_get))
return 0;
config_set.git_cmd = 1;
strvec_pushl(&config_set.args, "config", "--add", "--global", "maintenance.repo",
the_repository->worktree ? the_repository->worktree
: the_repository->gitdir,
NULL);
return run_command(&config_set);
}
static int maintenance_unregister(void)
{
struct child_process config_unset = CHILD_PROCESS_INIT;
if (!the_repository || !the_repository->gitdir)
return error(_("no current repository to unregister"));
config_unset.git_cmd = 1;
strvec_pushl(&config_unset.args, "config", "--global", "--unset",
"maintenance.repo",
the_repository->worktree ? the_repository->worktree
: the_repository->gitdir,
NULL);
return run_command(&config_unset);
}
#define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
#define END_LINE "# END GIT MAINTENANCE SCHEDULE"
static int update_background_schedule(int run_maintenance)
{
int result = 0;
int in_old_region = 0;
struct child_process crontab_list = CHILD_PROCESS_INIT;
struct child_process crontab_edit = CHILD_PROCESS_INIT;
FILE *cron_list, *cron_in;
const char *crontab_name;
struct strbuf line = STRBUF_INIT;
struct lock_file lk;
char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
return error(_("another process is scheduling background maintenance"));
crontab_name = getenv("GIT_TEST_CRONTAB");
if (!crontab_name)
crontab_name = "crontab";
strvec_split(&crontab_list.args, crontab_name);
strvec_push(&crontab_list.args, "-l");
crontab_list.in = -1;
crontab_list.out = dup(lk.tempfile->fd);
crontab_list.git_cmd = 0;
if (start_command(&crontab_list)) {
result = error(_("failed to run 'crontab -l'; your system might not support 'cron'"));
goto cleanup;
}
/* Ignore exit code, as an empty crontab will return error. */
finish_command(&crontab_list);
/*
* Read from the .lock file, filtering out the old
* schedule while appending the new schedule.
*/
cron_list = fdopen(lk.tempfile->fd, "r");
rewind(cron_list);
strvec_split(&crontab_edit.args, crontab_name);
crontab_edit.in = -1;
crontab_edit.git_cmd = 0;
if (start_command(&crontab_edit)) {
result = error(_("failed to run 'crontab'; your system might not support 'cron'"));
goto cleanup;
}
cron_in = fdopen(crontab_edit.in, "w");
if (!cron_in) {
result = error(_("failed to open stdin of 'crontab'"));
goto done_editing;
}
while (!strbuf_getline_lf(&line, cron_list)) {
if (!in_old_region && !strcmp(line.buf, BEGIN_LINE))
in_old_region = 1;
if (in_old_region)
continue;
fprintf(cron_in, "%s\n", line.buf);
if (in_old_region && !strcmp(line.buf, END_LINE))
in_old_region = 0;
}
if (run_maintenance) {
struct strbuf line_format = STRBUF_INIT;
const char *exec_path = git_exec_path();
fprintf(cron_in, "%s\n", BEGIN_LINE);
fprintf(cron_in,
"# The following schedule was created by Git\n");
fprintf(cron_in, "# Any edits made in this region might be\n");
fprintf(cron_in,
"# replaced in the future by a Git command.\n\n");
strbuf_addf(&line_format,
"%%s %%s * * %%s \"%s/git\" --exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%s\n",
exec_path, exec_path);
fprintf(cron_in, line_format.buf, "0", "1-23", "*", "hourly");
fprintf(cron_in, line_format.buf, "0", "0", "1-6", "daily");
fprintf(cron_in, line_format.buf, "0", "0", "0", "weekly");
strbuf_release(&line_format);
fprintf(cron_in, "\n%s\n", END_LINE);
}
fflush(cron_in);
fclose(cron_in);
close(crontab_edit.in);
done_editing:
if (finish_command(&crontab_edit)) {
result = error(_("'crontab' died"));
goto cleanup;
}
fclose(cron_list);
cleanup:
rollback_lock_file(&lk);
return result;
}
static int maintenance_start(void)
{
if (maintenance_register())
warning(_("failed to add repo to global config"));
return update_background_schedule(1);
}
static int maintenance_stop(void)
{
return update_background_schedule(0);
}
static const char builtin_maintenance_usage[] = N_("git maintenance <subcommand> [<options>]");
int cmd_maintenance(int argc, const char **argv, const char *prefix)
{
@ -1037,6 +1648,14 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
if (!strcmp(argv[1], "run"))
return maintenance_run(argc - 1, argv + 1, prefix);
if (!strcmp(argv[1], "start"))
return maintenance_start();
if (!strcmp(argv[1], "stop"))
return maintenance_stop();
if (!strcmp(argv[1], "register"))
return maintenance_register();
if (!strcmp(argv[1], "unregister"))
return maintenance_unregister();
die(_("invalid subcommand: %s"), argv[1]);
}

Просмотреть файл

@ -94,6 +94,7 @@ git-fetch-pack synchingrepositories
git-filter-branch ancillarymanipulators
git-fmt-merge-msg purehelpers
git-for-each-ref plumbinginterrogators
git-for-each-repo plumbinginterrogators
git-format-patch mainporcelain
git-fsck ancillaryinterrogators complete
git-gc mainporcelain

1
git.c
Просмотреть файл

@ -585,6 +585,7 @@ static struct cmd_struct commands[] = {
{ "fetch-pack", cmd_fetch_pack, RUN_SETUP | NO_PARSEOPT },
{ "fmt-merge-msg", cmd_fmt_merge_msg, RUN_SETUP },
{ "for-each-ref", cmd_for_each_ref, RUN_SETUP },
{ "for-each-repo", cmd_for_each_repo, RUN_SETUP_GENTLY },
{ "format-patch", cmd_format_patch, RUN_SETUP },
{ "fsck", cmd_fsck, RUN_SETUP | BLOCK_ON_GVFS_REPO},
{ "fsck-objects", cmd_fsck, RUN_SETUP },

21
midx.c
Просмотреть файл

@ -10,6 +10,7 @@
#include "progress.h"
#include "trace2.h"
#include "run-command.h"
#include "repository.h"
#define MIDX_SIGNATURE 0x4d494458 /* "MIDX" */
#define MIDX_VERSION 1
@ -398,15 +399,9 @@ int prepare_multi_pack_index_one(struct repository *r, const char *object_dir, i
{
struct multi_pack_index *m;
struct multi_pack_index *m_search;
int config_value;
static int env_value = -1;
if (env_value < 0)
env_value = git_env_bool(GIT_TEST_MULTI_PACK_INDEX, 0);
if (!env_value &&
(repo_config_get_bool(r, "core.multipackindex", &config_value) ||
!config_value))
prepare_repo_settings(r);
if (!r->settings.core_multi_pack_index)
return 0;
for (m_search = r->objects->multi_pack_index; m_search; m_search = m_search->next)
@ -850,7 +845,7 @@ static int write_midx_internal(const char *object_dir, struct multi_pack_index *
packs.pack_paths_checked = 0;
if (flags & MIDX_PROGRESS)
packs.progress = start_progress(_("Adding packfiles to multi-pack-index"), 0);
packs.progress = start_delayed_progress(_("Adding packfiles to multi-pack-index"), 0);
else
packs.progress = NULL;
@ -987,7 +982,7 @@ static int write_midx_internal(const char *object_dir, struct multi_pack_index *
}
if (flags & MIDX_PROGRESS)
progress = start_progress(_("Writing chunks to multi-pack-index"),
progress = start_delayed_progress(_("Writing chunks to multi-pack-index"),
num_chunks);
for (i = 0; i < num_chunks; i++) {
if (written != chunk_offsets[i])
@ -1129,7 +1124,7 @@ int verify_midx_file(struct repository *r, const char *object_dir, unsigned flag
}
if (flags & MIDX_PROGRESS)
progress = start_progress(_("Looking for referenced packfiles"),
progress = start_delayed_progress(_("Looking for referenced packfiles"),
m->num_packs);
for (i = 0; i < m->num_packs; i++) {
if (prepare_midx_pack(r, m, i))
@ -1250,7 +1245,7 @@ int expire_midx_packs(struct repository *r, const char *object_dir, unsigned fla
count = xcalloc(m->num_packs, sizeof(uint32_t));
if (flags & MIDX_PROGRESS)
progress = start_progress(_("Counting referenced objects"),
progress = start_delayed_progress(_("Counting referenced objects"),
m->num_objects);
for (i = 0; i < m->num_objects; i++) {
int pack_int_id = nth_midxed_pack_int_id(m, i);
@ -1260,7 +1255,7 @@ int expire_midx_packs(struct repository *r, const char *object_dir, unsigned fla
stop_progress(&progress);
if (flags & MIDX_PROGRESS)
progress = start_progress(_("Finding and deleting unreferenced packfiles"),
progress = start_delayed_progress(_("Finding and deleting unreferenced packfiles"),
m->num_packs);
for (i = 0; i < m->num_packs; i++) {
char *pack_name;

Просмотреть файл

@ -1,6 +1,7 @@
#include "cache.h"
#include "config.h"
#include "repository.h"
#include "midx.h"
#define UPDATE_DEFAULT_BOOL(s,v) do { if (s == -1) { s = v; } } while(0)
@ -52,6 +53,11 @@ void prepare_repo_settings(struct repository *r)
r->settings.pack_use_sparse = value;
UPDATE_DEFAULT_BOOL(r->settings.pack_use_sparse, 1);
value = git_env_bool(GIT_TEST_MULTI_PACK_INDEX, 0);
if (value || !repo_config_get_bool(r, "core.multipackindex", &value))
r->settings.core_multi_pack_index = value;
UPDATE_DEFAULT_BOOL(r->settings.core_multi_pack_index, 1);
if (!repo_config_get_bool(r, "feature.manyfiles", &value) && value) {
UPDATE_DEFAULT_BOOL(r->settings.index_version, 4);
UPDATE_DEFAULT_BOOL(r->settings.core_untracked_cache, UNTRACKED_CACHE_WRITE);

Просмотреть файл

@ -39,6 +39,8 @@ struct repo_settings {
int pack_use_sparse;
enum fetch_negotiation_setting fetch_negotiation_algorithm;
int core_multi_pack_index;
};
struct repository {

Просмотреть файл

@ -1951,8 +1951,13 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task,
int run_auto_maintenance(int quiet)
{
int enabled;
struct child_process maint = CHILD_PROCESS_INIT;
if (!git_config_get_bool("maintenance.auto", &enabled) &&
!enabled)
return 0;
maint.git_cmd = 1;
strvec_pushl(&maint.args, "maintenance", "run", "--auto", NULL);
strvec_push(&maint.args, quiet ? "--quiet" : "--no-quiet");

35
t/helper/test-crontab.c Normal file
Просмотреть файл

@ -0,0 +1,35 @@
#include "test-tool.h"
#include "cache.h"
/*
* Usage: test-tool cron <file> [-l]
*
* If -l is specified, then write the contents of <file> to stdout.
* Otherwise, write from stdin into <file>.
*/
int cmd__crontab(int argc, const char **argv)
{
int a;
FILE *from, *to;
if (argc == 3 && !strcmp(argv[2], "-l")) {
from = fopen(argv[1], "r");
if (!from)
return 0;
to = stdout;
} else if (argc == 2) {
from = stdin;
to = fopen(argv[1], "w");
} else
return error("unknown arguments");
while ((a = fgetc(from)) != EOF)
fputc(a, to);
if (argc == 3)
fclose(from);
else
fclose(to);
return 0;
}

Просмотреть файл

@ -19,6 +19,7 @@ static struct test_cmd cmds[] = {
{ "chmtime", cmd__chmtime },
{ "cmp", cmd__cmp },
{ "config", cmd__config },
{ "crontab", cmd__crontab },
{ "ctype", cmd__ctype },
{ "date", cmd__date },
{ "delta", cmd__delta },

Просмотреть файл

@ -9,6 +9,7 @@ int cmd__bloom(int argc, const char **argv);
int cmd__chmtime(int argc, const char **argv);
int cmd__cmp(int argc, const char **argv);
int cmd__config(int argc, const char **argv);
int cmd__crontab(int argc, const char **argv);
int cmd__ctype(int argc, const char **argv);
int cmd__date(int argc, const char **argv);
int cmd__delta(int argc, const char **argv);

30
t/t0068-for-each-repo.sh Executable file
Просмотреть файл

@ -0,0 +1,30 @@
#!/bin/sh
test_description='git for-each-repo builtin'
. ./test-lib.sh
test_expect_success 'run based on configured value' '
git init one &&
git init two &&
git init three &&
git -C two commit --allow-empty -m "DID NOT RUN" &&
git config run.key "$TRASH_DIRECTORY/one" &&
git config --add run.key "$TRASH_DIRECTORY/three" &&
git for-each-repo --config=run.key commit --allow-empty -m "ran" &&
git -C one log -1 --pretty=format:%s >message &&
grep ran message &&
git -C two log -1 --pretty=format:%s >message &&
! grep ran message &&
git -C three log -1 --pretty=format:%s >message &&
grep ran message &&
git for-each-repo --config=run.key -- commit --allow-empty -m "ran again" &&
git -C one log -1 --pretty=format:%s >message &&
grep again message &&
git -C two log -1 --pretty=format:%s >message &&
! grep again message &&
git -C three log -1 --pretty=format:%s >message &&
grep again message
'
test_done

Просмотреть файл

@ -3,6 +3,7 @@
test_description='multi-pack-indexes'
. ./test-lib.sh
GIT_TEST_MULTI_PACK_INDEX=0
objdir=.git/objects
HASH_LEN=$(test_oid rawsz)
@ -173,12 +174,12 @@ test_expect_success 'write progress off for redirected stderr' '
'
test_expect_success 'write force progress on for stderr' '
git multi-pack-index --object-dir=$objdir --progress write 2>err &&
GIT_PROGRESS_DELAY=0 git multi-pack-index --object-dir=$objdir --progress write 2>err &&
test_file_not_empty err
'
test_expect_success 'write with the --no-progress option' '
git multi-pack-index --object-dir=$objdir --no-progress write 2>err &&
GIT_PROGRESS_DELAY=0 git multi-pack-index --object-dir=$objdir --no-progress write 2>err &&
test_line_count = 0 err
'
@ -368,17 +369,17 @@ test_expect_success 'git-fsck incorrect offset' '
'
test_expect_success 'repack progress off for redirected stderr' '
git multi-pack-index --object-dir=$objdir repack 2>err &&
GIT_PROGRESS_DELAY=0 git multi-pack-index --object-dir=$objdir repack 2>err &&
test_line_count = 0 err
'
test_expect_success 'repack force progress on for stderr' '
git multi-pack-index --object-dir=$objdir --progress repack 2>err &&
GIT_PROGRESS_DELAY=0 git multi-pack-index --object-dir=$objdir --progress repack 2>err &&
test_file_not_empty err
'
test_expect_success 'repack with the --no-progress option' '
git multi-pack-index --object-dir=$objdir --no-progress repack 2>err &&
GIT_PROGRESS_DELAY=0 git multi-pack-index --object-dir=$objdir --no-progress repack 2>err &&
test_line_count = 0 err
'
@ -562,7 +563,7 @@ test_expect_success 'expire progress off for redirected stderr' '
test_expect_success 'expire force progress on for stderr' '
(
cd dup &&
git multi-pack-index --progress expire 2>err &&
GIT_PROGRESS_DELAY=0 git multi-pack-index --progress expire 2>err &&
test_file_not_empty err
)
'
@ -570,7 +571,7 @@ test_expect_success 'expire force progress on for stderr' '
test_expect_success 'expire with the --no-progress option' '
(
cd dup &&
git multi-pack-index --no-progress expire 2>err &&
GIT_PROGRESS_DELAY=0 git multi-pack-index --no-progress expire 2>err &&
test_line_count = 0 err
)
'

Просмотреть файл

@ -5,10 +5,11 @@ test_description='git maintenance builtin'
. ./test-lib.sh
GIT_TEST_COMMIT_GRAPH=0
GIT_TEST_MULTI_PACK_INDEX=0
test_expect_success 'help text' '
test_expect_code 129 git maintenance -h 2>err &&
test_i18ngrep "usage: git maintenance run" err &&
test_i18ngrep "usage: git maintenance <subcommand>" err &&
test_expect_code 128 git maintenance barf 2>err &&
test_i18ngrep "invalid subcommand: barf" err &&
test_expect_code 129 git maintenance 2>err &&
@ -27,6 +28,19 @@ test_expect_success 'run [--auto|--quiet]' '
test_subcommand git gc --no-quiet <run-no-quiet.txt
'
test_expect_success 'maintenance.auto config option' '
GIT_TRACE2_EVENT="$(pwd)/default" git commit --quiet --allow-empty -m 1 &&
test_subcommand git maintenance run --auto --quiet <default &&
GIT_TRACE2_EVENT="$(pwd)/true" \
git -c maintenance.auto=true \
commit --quiet --allow-empty -m 2 &&
test_subcommand git maintenance run --auto --quiet <true &&
GIT_TRACE2_EVENT="$(pwd)/false" \
git -c maintenance.auto=false \
commit --quiet --allow-empty -m 3 &&
test_subcommand ! git maintenance run --auto --quiet <false
'
test_expect_success 'maintenance.<task>.enabled' '
git config maintenance.gc.enabled false &&
git config maintenance.commit-graph.enabled true &&
@ -62,4 +76,274 @@ test_expect_success 'run --task duplicate' '
test_i18ngrep "cannot be selected multiple times" err
'
test_expect_success 'run --task=prefetch with no remotes' '
git maintenance run --task=prefetch 2>err &&
test_must_be_empty err
'
test_expect_success 'prefetch multiple remotes' '
git clone . clone1 &&
git clone . clone2 &&
git remote add remote1 "file://$(pwd)/clone1" &&
git remote add remote2 "file://$(pwd)/clone2" &&
git -C clone1 switch -c one &&
git -C clone2 switch -c two &&
test_commit -C clone1 one &&
test_commit -C clone2 two &&
GIT_TRACE2_EVENT="$(pwd)/run-prefetch.txt" git maintenance run --task=prefetch 2>/dev/null &&
fetchargs="--prune --no-tags --no-write-fetch-head --recurse-submodules=no --refmap= --quiet" &&
test_subcommand git fetch remote1 $fetchargs +refs/heads/\\*:refs/prefetch/remote1/\\* <run-prefetch.txt &&
test_subcommand git fetch remote2 $fetchargs +refs/heads/\\*:refs/prefetch/remote2/\\* <run-prefetch.txt &&
test_path_is_missing .git/refs/remotes &&
git log prefetch/remote1/one &&
git log prefetch/remote2/two &&
git fetch --all &&
test_cmp_rev refs/remotes/remote1/one refs/prefetch/remote1/one &&
test_cmp_rev refs/remotes/remote2/two refs/prefetch/remote2/two
'
test_expect_success 'loose-objects task' '
# Repack everything so we know the state of the object dir
git repack -adk &&
# Hack to stop maintenance from running during "git commit"
echo in use >.git/objects/maintenance.lock &&
# Assuming that "git commit" creates at least one loose object
test_commit create-loose-object &&
rm .git/objects/maintenance.lock &&
ls .git/objects >obj-dir-before &&
test_file_not_empty obj-dir-before &&
ls .git/objects/pack/*.pack >packs-before &&
test_line_count = 1 packs-before &&
# The first run creates a pack-file
# but does not delete loose objects.
git maintenance run --task=loose-objects &&
ls .git/objects >obj-dir-between &&
test_cmp obj-dir-before obj-dir-between &&
ls .git/objects/pack/*.pack >packs-between &&
test_line_count = 2 packs-between &&
ls .git/objects/pack/loose-*.pack >loose-packs &&
test_line_count = 1 loose-packs &&
# The second run deletes loose objects
# but does not create a pack-file.
git maintenance run --task=loose-objects &&
ls .git/objects >obj-dir-after &&
cat >expect <<-\EOF &&
info
pack
EOF
test_cmp expect obj-dir-after &&
ls .git/objects/pack/*.pack >packs-after &&
test_cmp packs-between packs-after
'
test_expect_success 'maintenance.loose-objects.auto' '
git repack -adk &&
GIT_TRACE2_EVENT="$(pwd)/trace-lo1.txt" \
git -c maintenance.loose-objects.auto=1 maintenance \
run --auto --task=loose-objects 2>/dev/null &&
test_subcommand ! git prune-packed --quiet <trace-lo1.txt &&
test_write_lines data-A | git hash-object -t blob --stdin -w &&
GIT_TRACE2_EVENT="$(pwd)/trace-loA" \
git -c maintenance.loose-objects.auto=2 \
maintenance run --auto --task=loose-objects 2>/dev/null &&
test_subcommand ! git prune-packed --quiet <trace-loA &&
test_write_lines data-B | git hash-object -t blob --stdin -w &&
GIT_TRACE2_EVENT="$(pwd)/trace-loB" \
git -c maintenance.loose-objects.auto=2 \
maintenance run --auto --task=loose-objects 2>/dev/null &&
test_subcommand git prune-packed --quiet <trace-loB &&
GIT_TRACE2_EVENT="$(pwd)/trace-loC" \
git -c maintenance.loose-objects.auto=2 \
maintenance run --auto --task=loose-objects 2>/dev/null &&
test_subcommand git prune-packed --quiet <trace-loC
'
test_expect_success 'incremental-repack task' '
packDir=.git/objects/pack &&
for i in $(test_seq 1 5)
do
test_commit $i || return 1
done &&
# Create three disjoint pack-files with size BIG, small, small.
echo HEAD~2 | git pack-objects --revs $packDir/test-1 &&
test_tick &&
git pack-objects --revs $packDir/test-2 <<-\EOF &&
HEAD~1
^HEAD~2
EOF
test_tick &&
git pack-objects --revs $packDir/test-3 <<-\EOF &&
HEAD
^HEAD~1
EOF
rm -f $packDir/pack-* &&
rm -f $packDir/loose-* &&
ls $packDir/*.pack >packs-before &&
test_line_count = 3 packs-before &&
# the job repacks the two into a new pack, but does not
# delete the old ones.
git maintenance run --task=incremental-repack &&
ls $packDir/*.pack >packs-between &&
test_line_count = 4 packs-between &&
# the job deletes the two old packs, and does not write
# a new one because the batch size is not high enough to
# pack the largest pack-file.
git maintenance run --task=incremental-repack &&
ls .git/objects/pack/*.pack >packs-after &&
test_line_count = 2 packs-after
'
test_expect_success EXPENSIVE 'incremental-repack 2g limit' '
for i in $(test_seq 1 5)
do
test-tool genrandom foo$i $((512 * 1024 * 1024 + 1)) >>big ||
return 1
done &&
git add big &&
git commit -m "Add big file (1)" &&
# ensure any possible loose objects are in a pack-file
git maintenance run --task=loose-objects &&
rm big &&
for i in $(test_seq 6 10)
do
test-tool genrandom foo$i $((512 * 1024 * 1024 + 1)) >>big ||
return 1
done &&
git add big &&
git commit -m "Add big file (2)" &&
# ensure any possible loose objects are in a pack-file
git maintenance run --task=loose-objects &&
# Now run the incremental-repack task and check the batch-size
GIT_TRACE2_EVENT="$(pwd)/run-2g.txt" git maintenance run \
--task=incremental-repack 2>/dev/null &&
test_subcommand git multi-pack-index repack \
--no-progress --batch-size=2147483647 <run-2g.txt
'
test_expect_success 'maintenance.incremental-repack.auto' '
git repack -adk &&
git config core.multiPackIndex true &&
git multi-pack-index write &&
GIT_TRACE2_EVENT="$(pwd)/midx-init.txt" git \
-c maintenance.incremental-repack.auto=1 \
maintenance run --auto --task=incremental-repack 2>/dev/null &&
test_subcommand ! git multi-pack-index write --no-progress <midx-init.txt &&
test_commit A &&
git pack-objects --revs .git/objects/pack/pack <<-\EOF &&
HEAD
^HEAD~1
EOF
GIT_TRACE2_EVENT=$(pwd)/trace-A git \
-c maintenance.incremental-repack.auto=2 \
maintenance run --auto --task=incremental-repack 2>/dev/null &&
test_subcommand ! git multi-pack-index write --no-progress <trace-A &&
test_commit B &&
git pack-objects --revs .git/objects/pack/pack <<-\EOF &&
HEAD
^HEAD~1
EOF
GIT_TRACE2_EVENT=$(pwd)/trace-B git \
-c maintenance.incremental-repack.auto=2 \
maintenance run --auto --task=incremental-repack 2>/dev/null &&
test_subcommand git multi-pack-index write --no-progress <trace-B
'
test_expect_success '--auto and --schedule incompatible' '
test_must_fail git maintenance run --auto --schedule=daily 2>err &&
test_i18ngrep "at most one" err
'
test_expect_success 'invalid --schedule value' '
test_must_fail git maintenance run --schedule=annually 2>err &&
test_i18ngrep "unrecognized --schedule" err
'
test_expect_success '--schedule inheritance weekly -> daily -> hourly' '
git config maintenance.loose-objects.enabled true &&
git config maintenance.loose-objects.schedule hourly &&
git config maintenance.commit-graph.enabled true &&
git config maintenance.commit-graph.schedule daily &&
git config maintenance.incremental-repack.enabled true &&
git config maintenance.incremental-repack.schedule weekly &&
GIT_TRACE2_EVENT="$(pwd)/hourly.txt" \
git maintenance run --schedule=hourly 2>/dev/null &&
test_subcommand git prune-packed --quiet <hourly.txt &&
test_subcommand ! git commit-graph write --split --reachable \
--no-progress <hourly.txt &&
test_subcommand ! git multi-pack-index write --no-progress <hourly.txt &&
GIT_TRACE2_EVENT="$(pwd)/daily.txt" \
git maintenance run --schedule=daily 2>/dev/null &&
test_subcommand git prune-packed --quiet <daily.txt &&
test_subcommand git commit-graph write --split --reachable \
--no-progress <daily.txt &&
test_subcommand ! git multi-pack-index write --no-progress <daily.txt &&
GIT_TRACE2_EVENT="$(pwd)/weekly.txt" \
git maintenance run --schedule=weekly 2>/dev/null &&
test_subcommand git prune-packed --quiet <weekly.txt &&
test_subcommand git commit-graph write --split --reachable \
--no-progress <weekly.txt &&
test_subcommand git multi-pack-index write --no-progress <weekly.txt
'
test_expect_success 'register and unregister' '
test_when_finished git config --global --unset-all maintenance.repo &&
git config --global --add maintenance.repo /existing1 &&
git config --global --add maintenance.repo /existing2 &&
git config --global --get-all maintenance.repo >before &&
git maintenance register &&
test_cmp_config false maintenance.auto &&
git config --global --get-all maintenance.repo >between &&
cp before expect &&
pwd >>expect &&
test_cmp expect between &&
git maintenance unregister &&
git config --global --get-all maintenance.repo >actual &&
test_cmp before actual
'
test_expect_success 'start from empty cron table' '
GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
# start registers the repo
git config --get --global maintenance.repo "$(pwd)" &&
grep "for-each-repo --config=maintenance.repo maintenance run --schedule=daily" cron.txt &&
grep "for-each-repo --config=maintenance.repo maintenance run --schedule=hourly" cron.txt &&
grep "for-each-repo --config=maintenance.repo maintenance run --schedule=weekly" cron.txt
'
test_expect_success 'stop from existing schedule' '
GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
# stop does not unregister the repo
git config --get --global maintenance.repo "$(pwd)" &&
# Operation is idempotent
GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
test_must_be_empty cron.txt
'
test_expect_success 'start preserves existing schedule' '
echo "Important information!" >cron.txt &&
GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
grep "Important information!" cron.txt
'
test_done

Просмотреть файл

@ -1729,3 +1729,9 @@ test_lazy_prereq SHA1 '
test_lazy_prereq REBASE_P '
test -z "$GIT_TEST_SKIP_REBASE_P"
'
# Ensure that no test accidentally triggers a Git command
# that runs 'crontab', affecting a user's cron schedule.
# Tests that verify the cron integration must set this locally
# to avoid errors.
GIT_TEST_CRONTAB="exit 1"