Merge pull request #301: Update 'git maintenance' to match upstream

This PR updates our `vfs-2.29.0` branch's version of `git maintenance` to match the latest in upstream. Unfortunately, not all of these commits made it to the `2.30` release candidate, but there are more commits from the series making it in. They will cause a conflict in the `vfs-2.30.0` rebase, so merge them in here. This also includes the `fixup!` reverts of the earlier versions.

Finally, I also noticed that we started depending on `git maintenance start` in Scalar for macOS, but we never checked that this worked with the shared object cache. It doesn't! 😨  The very tip commit of this PR includes logic to make `git maintenance run` care about `gvfs.sharedCache`. Functional test updates in Scalar will follow.
This commit is contained in:
Derrick Stolee 2020-12-15 08:59:18 -05:00 коммит произвёл GitHub
Родитель 5e62c5d6e0 2864f7cbfd
Коммит fe73a588a0
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
5 изменённых файлов: 748 добавлений и 101 удалений

Просмотреть файл

@ -3,6 +3,21 @@ maintenance.auto::
`git maintenance run --auto` after doing their normal work. Defaults `git maintenance run --auto` after doing their normal work. Defaults
to true. to true.
maintenance.strategy::
This string config option provides a way to specify one of a few
recommended schedules for background maintenance. This only affects
which tasks are run during `git maintenance run --schedule=X`
commands, provided no `--task=<task>` arguments are provided.
Further, if a `maintenance.<task>.schedule` config value is set,
then that value is used instead of the one provided by
`maintenance.strategy`. The possible strategy strings are:
+
* `none`: This default setting implies no task are run at any schedule.
* `incremental`: This setting optimizes for performing small maintenance
activities that do not delete any data. This does not schedule the `gc`
task, but runs the `prefetch` and `commit-graph` tasks hourly and the
`loose-objects` and `incremental-repack` tasks daily.
maintenance.<task>.enabled:: maintenance.<task>.enabled::
This boolean config option controls whether the maintenance task This boolean config option controls whether the maintenance task
with name `<task>` is run when no `--task` option is specified to with name `<task>` is run when no `--task` option is specified to

Просмотреть файл

@ -38,16 +38,18 @@ register::
for running in the background without disrupting foreground for running in the background without disrupting foreground
processes. processes.
+ +
If your repository has no `maintenance.<task>.schedule` configuration The `register` subcomand will also set the `maintenance.strategy` config
values set, then Git will use a recommended default schedule that performs value to `incremental`, if this value is not previously set. The
background maintenance that will not interrupt foreground commands. The `incremental` strategy uses the following schedule for each maintenance
default schedule is as follows: task:
+ +
--
* `gc`: disabled. * `gc`: disabled.
* `commit-graph`: hourly. * `commit-graph`: hourly.
* `prefetch`: hourly. * `prefetch`: hourly.
* `loose-objects`: daily. * `loose-objects`: daily.
* `incremental-repack`: daily. * `incremental-repack`: daily.
--
+ +
`git maintenance register` will also disable foreground maintenance by `git maintenance register` will also disable foreground maintenance by
setting `maintenance.auto = false` in the current repository. This config setting `maintenance.auto = false` in the current repository. This config
@ -216,6 +218,122 @@ Further, the `git gc` command should not be combined with
but does not take the lock in the same way as `git maintenance run`. If but does not take the lock in the same way as `git maintenance run`. If
possible, use `git maintenance run --task=gc` instead of `git gc`. possible, use `git maintenance run --task=gc` instead of `git gc`.
The following sections describe the mechanisms put in place to run
background maintenance by `git maintenance start` and how to customize
them.
BACKGROUND MAINTENANCE ON POSIX SYSTEMS
---------------------------------------
The standard mechanism for scheduling background tasks on POSIX systems
is cron(8). This tool executes commands based on a given schedule. The
current list of user-scheduled tasks can be found by running `crontab -l`.
The schedule written by `git maintenance start` is similar to this:
-----------------------------------------------------------------------
# BEGIN GIT MAINTENANCE SCHEDULE
# The following schedule was created by Git
# Any edits made in this region might be
# replaced in the future by a Git command.
0 1-23 * * * "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=hourly
0 0 * * 1-6 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=daily
0 0 * * 0 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=weekly
# END GIT MAINTENANCE SCHEDULE
-----------------------------------------------------------------------
The comments are used as a region to mark the schedule as written by Git.
Any modifications within this region will be completely deleted by
`git maintenance stop` or overwritten by `git maintenance start`.
The `crontab` entry specifies the full path of the `git` executable to
ensure that the executed `git` command is the same one with which
`git maintenance start` was issued independent of `PATH`. If the same user
runs `git maintenance start` with multiple Git executables, then only the
latest executable is used.
These commands use `git for-each-repo --config=maintenance.repo` to run
`git maintenance run --schedule=<frequency>` on each repository listed in
the multi-valued `maintenance.repo` config option. These are typically
loaded from the user-specific global config. The `git maintenance` process
then determines which maintenance tasks are configured to run on each
repository with each `<frequency>` using the `maintenance.<task>.schedule`
config options. These values are loaded from the global or repository
config values.
If the config values are insufficient to achieve your desired background
maintenance schedule, then you can create your own schedule. If you run
`crontab -e`, then an editor will load with your user-specific `cron`
schedule. In that editor, you can add your own schedule lines. You could
start by adapting the default schedule listed earlier, or you could read
the crontab(5) documentation for advanced scheduling techniques. Please
do use the full path and `--exec-path` techniques from the default
schedule to ensure you are executing the correct binaries in your
schedule.
BACKGROUND MAINTENANCE ON MACOS SYSTEMS
---------------------------------------
While macOS technically supports `cron`, using `crontab -e` requires
elevated privileges and the executed process does not have a full user
context. Without a full user context, Git and its credential helpers
cannot access stored credentials, so some maintenance tasks are not
functional.
Instead, `git maintenance start` interacts with the `launchctl` tool,
which is the recommended way to schedule timed jobs in macOS. Scheduling
maintenance through `git maintenance (start|stop)` requires some
`launchctl` features available only in macOS 10.11 or later.
Your user-specific scheduled tasks are stored as XML-formatted `.plist`
files in `~/Library/LaunchAgents/`. You can see the currently-registered
tasks using the following command:
-----------------------------------------------------------------------
$ ls ~/Library/LaunchAgents/org.git-scm.git*
org.git-scm.git.daily.plist
org.git-scm.git.hourly.plist
org.git-scm.git.weekly.plist
-----------------------------------------------------------------------
One task is registered for each `--schedule=<frequency>` option. To
inspect how the XML format describes each schedule, open one of these
`.plist` files in an editor and inspect the `<array>` element following
the `<key>StartCalendarInterval</key>` element.
`git maintenance start` will overwrite these files and register the
tasks again with `launchctl`, so any customizations should be done by
creating your own `.plist` files with distinct names. Similarly, the
`git maintenance stop` command will unregister the tasks with `launchctl`
and delete the `.plist` files.
To create more advanced customizations to your background tasks, see
launchctl.plist(5) for more information.
BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS
-----------------------------------------
Windows does not support `cron` and instead has its own system for
scheduling background tasks. The `git maintenance start` command uses
the `schtasks` command to submit tasks to this system. You can inspect
all background tasks using the Task Scheduler application. The tasks
added by Git have names of the form `Git Maintenance (<frequency>)`.
The Task Scheduler GUI has ways to inspect these tasks, but you can also
export the tasks to XML files and view the details there.
Note that since Git is a console application, these background tasks
create a console window visible to the current user. This can be changed
manually by selecting the "Run whether user is logged in or not" option
in Task Scheduler. This change requires a password input, which is why
`git maintenance start` does not select it by default.
If you want to customize the background tasks, please rename the tasks
so future calls to `git maintenance (start|stop)` do not overwrite your
custom tasks.
GIT GIT
--- ---

Просмотреть файл

@ -989,6 +989,8 @@ static int write_loose_object_to_stdin(const struct object_id *oid,
return ++(d->count) > d->batch_size; return ++(d->count) > d->batch_size;
} }
static const char *object_dir = NULL;
static int pack_loose(struct maintenance_run_opts *opts) static int pack_loose(struct maintenance_run_opts *opts)
{ {
struct repository *r = the_repository; struct repository *r = the_repository;
@ -996,11 +998,14 @@ static int pack_loose(struct maintenance_run_opts *opts)
struct write_loose_object_data data; struct write_loose_object_data data;
struct child_process pack_proc = CHILD_PROCESS_INIT; struct child_process pack_proc = CHILD_PROCESS_INIT;
if (!object_dir)
object_dir = r->objects->odb->path;
/* /*
* Do not start pack-objects process * Do not start pack-objects process
* if there are no loose objects. * if there are no loose objects.
*/ */
if (!for_each_loose_file_in_objdir(r->objects->odb->path, if (!for_each_loose_file_in_objdir(object_dir,
bail_on_loose, bail_on_loose,
NULL, NULL, NULL)) NULL, NULL, NULL))
return 0; return 0;
@ -1010,7 +1015,7 @@ static int pack_loose(struct maintenance_run_opts *opts)
strvec_push(&pack_proc.args, "pack-objects"); strvec_push(&pack_proc.args, "pack-objects");
if (opts->quiet) if (opts->quiet)
strvec_push(&pack_proc.args, "--quiet"); strvec_push(&pack_proc.args, "--quiet");
strvec_pushf(&pack_proc.args, "%s/pack/loose", r->objects->odb->path); strvec_pushf(&pack_proc.args, "%s/pack/loose", object_dir);
pack_proc.in = -1; pack_proc.in = -1;
@ -1023,7 +1028,7 @@ static int pack_loose(struct maintenance_run_opts *opts)
data.count = 0; data.count = 0;
data.batch_size = 50000; data.batch_size = 50000;
for_each_loose_file_in_objdir(r->objects->odb->path, for_each_loose_file_in_objdir(object_dir,
write_loose_object_to_stdin, write_loose_object_to_stdin,
NULL, NULL,
NULL, NULL,
@ -1255,59 +1260,6 @@ static int compare_tasks_by_selection(const void *a_, const void *b_)
return b->selected_order - a->selected_order; return b->selected_order - a->selected_order;
} }
static int has_schedule_config(void)
{
int i, found = 0;
struct strbuf config_name = STRBUF_INIT;
size_t prefix;
strbuf_addstr(&config_name, "maintenance.");
prefix = config_name.len;
for (i = 0; !found && i < TASK__COUNT; i++) {
char *value;
strbuf_setlen(&config_name, prefix);
strbuf_addf(&config_name, "%s.schedule", tasks[i].name);
if (!git_config_get_string(config_name.buf, &value)) {
found = 1;
FREE_AND_NULL(value);
}
strbuf_setlen(&config_name, prefix);
strbuf_addf(&config_name, "%s.enabled", tasks[i].name);
if (!git_config_get_string(config_name.buf, &value)) {
found = 1;
FREE_AND_NULL(value);
}
}
strbuf_release(&config_name);
return found;
}
static void set_recommended_schedule(void)
{
if (has_schedule_config())
return;
tasks[TASK_GC].enabled = 0;
tasks[TASK_PREFETCH].enabled = 1;
tasks[TASK_PREFETCH].schedule = SCHEDULE_HOURLY;
tasks[TASK_COMMIT_GRAPH].enabled = 1;
tasks[TASK_COMMIT_GRAPH].schedule = SCHEDULE_HOURLY;
tasks[TASK_LOOSE_OBJECTS].enabled = 1;
tasks[TASK_LOOSE_OBJECTS].schedule = SCHEDULE_DAILY;
tasks[TASK_INCREMENTAL_REPACK].enabled = 1;
tasks[TASK_INCREMENTAL_REPACK].schedule = SCHEDULE_DAILY;
}
static int maintenance_run_tasks(struct maintenance_run_opts *opts) static int maintenance_run_tasks(struct maintenance_run_opts *opts)
{ {
int i, found_selected = 0; int i, found_selected = 0;
@ -1337,8 +1289,6 @@ static int maintenance_run_tasks(struct maintenance_run_opts *opts)
if (found_selected) if (found_selected)
QSORT(tasks, TASK__COUNT, compare_tasks_by_selection); QSORT(tasks, TASK__COUNT, compare_tasks_by_selection);
else if (opts->schedule != SCHEDULE_NONE)
set_recommended_schedule();
for (i = 0; i < TASK__COUNT; i++) { for (i = 0; i < TASK__COUNT; i++) {
if (found_selected && tasks[i].selected_order < 0) if (found_selected && tasks[i].selected_order < 0)
@ -1367,12 +1317,35 @@ static int maintenance_run_tasks(struct maintenance_run_opts *opts)
return result; return result;
} }
static void initialize_task_config(void) static void initialize_maintenance_strategy(void)
{
char *config_str;
if (git_config_get_string("maintenance.strategy", &config_str))
return;
if (!strcasecmp(config_str, "incremental")) {
tasks[TASK_GC].schedule = SCHEDULE_NONE;
tasks[TASK_COMMIT_GRAPH].enabled = 1;
tasks[TASK_COMMIT_GRAPH].schedule = SCHEDULE_HOURLY;
tasks[TASK_PREFETCH].enabled = 1;
tasks[TASK_PREFETCH].schedule = SCHEDULE_HOURLY;
tasks[TASK_INCREMENTAL_REPACK].enabled = 1;
tasks[TASK_INCREMENTAL_REPACK].schedule = SCHEDULE_DAILY;
tasks[TASK_LOOSE_OBJECTS].enabled = 1;
tasks[TASK_LOOSE_OBJECTS].schedule = SCHEDULE_DAILY;
}
}
static void initialize_task_config(int schedule)
{ {
int i; int i;
struct strbuf config_name = STRBUF_INIT; struct strbuf config_name = STRBUF_INIT;
gc_config(); gc_config();
if (schedule)
initialize_maintenance_strategy();
for (i = 0; i < TASK__COUNT; i++) { for (i = 0; i < TASK__COUNT; i++) {
int config_value; int config_value;
char *config_str; char *config_str;
@ -1448,7 +1421,6 @@ static int maintenance_run(int argc, const char **argv, const char *prefix)
memset(&opts, 0, sizeof(opts)); memset(&opts, 0, sizeof(opts));
opts.quiet = !isatty(2); opts.quiet = !isatty(2);
initialize_task_config();
for (i = 0; i < TASK__COUNT; i++) for (i = 0; i < TASK__COUNT; i++)
tasks[i].selected_order = -1; tasks[i].selected_order = -1;
@ -1461,14 +1433,27 @@ static int maintenance_run(int argc, const char **argv, const char *prefix)
if (opts.auto_flag && opts.schedule) if (opts.auto_flag && opts.schedule)
die(_("use at most one of --auto and --schedule=<frequency>")); die(_("use at most one of --auto and --schedule=<frequency>"));
initialize_task_config(opts.schedule);
if (argc != 0) if (argc != 0)
usage_with_options(builtin_maintenance_run_usage, usage_with_options(builtin_maintenance_run_usage,
builtin_maintenance_run_options); builtin_maintenance_run_options);
/*
* To enable the VFS for Git/Scalar shared object cache, use
* the gvfs.sharedcache config option to redirect the
* maintenance to that location.
*/
if (!git_config_get_value("gvfs.sharedcache", &object_dir) &&
object_dir)
setenv(DB_ENVIRONMENT, object_dir, 1);
return maintenance_run_tasks(&opts); return maintenance_run_tasks(&opts);
} }
static int maintenance_register(void) static int maintenance_register(void)
{ {
char *config_value;
struct child_process config_set = CHILD_PROCESS_INIT; struct child_process config_set = CHILD_PROCESS_INIT;
struct child_process config_get = CHILD_PROCESS_INIT; struct child_process config_get = CHILD_PROCESS_INIT;
@ -1479,6 +1464,12 @@ static int maintenance_register(void)
/* Disable foreground maintenance */ /* Disable foreground maintenance */
git_config_set("maintenance.auto", "false"); git_config_set("maintenance.auto", "false");
/* Set maintenance strategy, if unset */
if (!git_config_get_string("maintenance.strategy", &config_value))
free(config_value);
else
git_config_set("maintenance.strategy", "incremental");
config_get.git_cmd = 1; config_get.git_cmd = 1;
strvec_pushl(&config_get.args, "config", "--global", "--get", "maintenance.repo", strvec_pushl(&config_get.args, "config", "--global", "--get", "maintenance.repo",
the_repository->worktree ? the_repository->worktree the_repository->worktree ? the_repository->worktree
@ -1519,38 +1510,367 @@ static int maintenance_unregister(void)
return run_command(&config_unset); return run_command(&config_unset);
} }
static const char *get_frequency(enum schedule_priority schedule)
{
switch (schedule) {
case SCHEDULE_HOURLY:
return "hourly";
case SCHEDULE_DAILY:
return "daily";
case SCHEDULE_WEEKLY:
return "weekly";
default:
BUG("invalid schedule %d", schedule);
}
}
static char *launchctl_service_name(const char *frequency)
{
struct strbuf label = STRBUF_INIT;
strbuf_addf(&label, "org.git-scm.git.%s", frequency);
return strbuf_detach(&label, NULL);
}
static char *launchctl_service_filename(const char *name)
{
char *expanded;
struct strbuf filename = STRBUF_INIT;
strbuf_addf(&filename, "~/Library/LaunchAgents/%s.plist", name);
expanded = expand_user_path(filename.buf, 1);
if (!expanded)
die(_("failed to expand path '%s'"), filename.buf);
strbuf_release(&filename);
return expanded;
}
static char *launchctl_get_uid(void)
{
return xstrfmt("gui/%d", getuid());
}
static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
{
int result;
struct child_process child = CHILD_PROCESS_INIT;
char *uid = launchctl_get_uid();
strvec_split(&child.args, cmd);
if (enable)
strvec_push(&child.args, "bootstrap");
else
strvec_push(&child.args, "bootout");
strvec_push(&child.args, uid);
strvec_push(&child.args, filename);
child.no_stderr = 1;
child.no_stdout = 1;
if (start_command(&child))
die(_("failed to start launchctl"));
result = finish_command(&child);
free(uid);
return result;
}
static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)
{
const char *frequency = get_frequency(schedule);
char *name = launchctl_service_name(frequency);
char *filename = launchctl_service_filename(name);
int result = launchctl_boot_plist(0, filename, cmd);
unlink(filename);
free(filename);
free(name);
return result;
}
static int launchctl_remove_plists(const char *cmd)
{
return launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||
launchctl_remove_plist(SCHEDULE_DAILY, cmd) ||
launchctl_remove_plist(SCHEDULE_WEEKLY, cmd);
}
static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)
{
FILE *plist;
int i;
const char *preamble, *repeat;
const char *frequency = get_frequency(schedule);
char *name = launchctl_service_name(frequency);
char *filename = launchctl_service_filename(name);
if (safe_create_leading_directories(filename))
die(_("failed to create directories for '%s'"), filename);
plist = xfopen(filename, "w");
preamble = "<?xml version=\"1.0\" encoding=\"US-ASCII\"?>\n"
"<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n"
"<plist version=\"1.0\">"
"<dict>\n"
"<key>Label</key><string>%s</string>\n"
"<key>ProgramArguments</key>\n"
"<array>\n"
"<string>%s/git</string>\n"
"<string>--exec-path=%s</string>\n"
"<string>for-each-repo</string>\n"
"<string>--config=maintenance.repo</string>\n"
"<string>maintenance</string>\n"
"<string>run</string>\n"
"<string>--schedule=%s</string>\n"
"</array>\n"
"<key>StartCalendarInterval</key>\n"
"<array>\n";
fprintf(plist, preamble, name, exec_path, exec_path, frequency);
switch (schedule) {
case SCHEDULE_HOURLY:
repeat = "<dict>\n"
"<key>Hour</key><integer>%d</integer>\n"
"<key>Minute</key><integer>0</integer>\n"
"</dict>\n";
for (i = 1; i <= 23; i++)
fprintf(plist, repeat, i);
break;
case SCHEDULE_DAILY:
repeat = "<dict>\n"
"<key>Day</key><integer>%d</integer>\n"
"<key>Hour</key><integer>0</integer>\n"
"<key>Minute</key><integer>0</integer>\n"
"</dict>\n";
for (i = 1; i <= 6; i++)
fprintf(plist, repeat, i);
break;
case SCHEDULE_WEEKLY:
fprintf(plist,
"<dict>\n"
"<key>Day</key><integer>0</integer>\n"
"<key>Hour</key><integer>0</integer>\n"
"<key>Minute</key><integer>0</integer>\n"
"</dict>\n");
break;
default:
/* unreachable */
break;
}
fprintf(plist, "</array>\n</dict>\n</plist>\n");
fclose(plist);
/* bootout might fail if not already running, so ignore */
launchctl_boot_plist(0, filename, cmd);
if (launchctl_boot_plist(1, filename, cmd))
die(_("failed to bootstrap service %s"), filename);
free(filename);
free(name);
return 0;
}
static int launchctl_add_plists(const char *cmd)
{
const char *exec_path = git_exec_path();
return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||
launchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||
launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
}
static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)
{
if (run_maintenance)
return launchctl_add_plists(cmd);
else
return launchctl_remove_plists(cmd);
}
static char *schtasks_task_name(const char *frequency)
{
struct strbuf label = STRBUF_INIT;
strbuf_addf(&label, "Git Maintenance (%s)", frequency);
return strbuf_detach(&label, NULL);
}
static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)
{
int result;
struct strvec args = STRVEC_INIT;
const char *frequency = get_frequency(schedule);
char *name = schtasks_task_name(frequency);
strvec_split(&args, cmd);
strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
result = run_command_v_opt(args.v, 0);
strvec_clear(&args);
free(name);
return result;
}
static int schtasks_remove_tasks(const char *cmd)
{
return schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||
schtasks_remove_task(SCHEDULE_DAILY, cmd) ||
schtasks_remove_task(SCHEDULE_WEEKLY, cmd);
}
static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
{
int result;
struct child_process child = CHILD_PROCESS_INIT;
const char *xml;
struct tempfile *tfile;
const char *frequency = get_frequency(schedule);
char *name = schtasks_task_name(frequency);
struct strbuf tfilename = STRBUF_INIT;
strbuf_addf(&tfilename, "schedule_%s_XXXXXX", frequency);
tfile = xmks_tempfile(tfilename.buf);
strbuf_release(&tfilename);
if (!fdopen_tempfile(tfile, "w"))
die(_("failed to create temp xml file"));
xml = "<?xml version=\"1.0\" encoding=\"US-ASCII\"?>\n"
"<Task version=\"1.4\" xmlns=\"http://schemas.microsoft.com/windows/2004/02/mit/task\">\n"
"<Triggers>\n"
"<CalendarTrigger>\n";
fputs(xml, tfile->fp);
switch (schedule) {
case SCHEDULE_HOURLY:
fprintf(tfile->fp,
"<StartBoundary>2020-01-01T01:00:00</StartBoundary>\n"
"<Enabled>true</Enabled>\n"
"<ScheduleByDay>\n"
"<DaysInterval>1</DaysInterval>\n"
"</ScheduleByDay>\n"
"<Repetition>\n"
"<Interval>PT1H</Interval>\n"
"<Duration>PT23H</Duration>\n"
"<StopAtDurationEnd>false</StopAtDurationEnd>\n"
"</Repetition>\n");
break;
case SCHEDULE_DAILY:
fprintf(tfile->fp,
"<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
"<Enabled>true</Enabled>\n"
"<ScheduleByWeek>\n"
"<DaysOfWeek>\n"
"<Monday />\n"
"<Tuesday />\n"
"<Wednesday />\n"
"<Thursday />\n"
"<Friday />\n"
"<Saturday />\n"
"</DaysOfWeek>\n"
"<WeeksInterval>1</WeeksInterval>\n"
"</ScheduleByWeek>\n");
break;
case SCHEDULE_WEEKLY:
fprintf(tfile->fp,
"<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
"<Enabled>true</Enabled>\n"
"<ScheduleByWeek>\n"
"<DaysOfWeek>\n"
"<Sunday />\n"
"</DaysOfWeek>\n"
"<WeeksInterval>1</WeeksInterval>\n"
"</ScheduleByWeek>\n");
break;
default:
break;
}
xml = "</CalendarTrigger>\n"
"</Triggers>\n"
"<Principals>\n"
"<Principal id=\"Author\">\n"
"<LogonType>InteractiveToken</LogonType>\n"
"<RunLevel>LeastPrivilege</RunLevel>\n"
"</Principal>\n"
"</Principals>\n"
"<Settings>\n"
"<MultipleInstancesPolicy>IgnoreNew</MultipleInstancesPolicy>\n"
"<Enabled>true</Enabled>\n"
"<Hidden>true</Hidden>\n"
"<UseUnifiedSchedulingEngine>true</UseUnifiedSchedulingEngine>\n"
"<WakeToRun>false</WakeToRun>\n"
"<ExecutionTimeLimit>PT72H</ExecutionTimeLimit>\n"
"<Priority>7</Priority>\n"
"</Settings>\n"
"<Actions Context=\"Author\">\n"
"<Exec>\n"
"<Command>\"%s\\git.exe\"</Command>\n"
"<Arguments>--exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%s</Arguments>\n"
"</Exec>\n"
"</Actions>\n"
"</Task>\n";
fprintf(tfile->fp, xml, exec_path, exec_path, frequency);
strvec_split(&child.args, cmd);
strvec_pushl(&child.args, "/create", "/tn", name, "/f", "/xml",
get_tempfile_path(tfile), NULL);
close_tempfile_gently(tfile);
child.no_stdout = 1;
child.no_stderr = 1;
if (start_command(&child))
die(_("failed to start schtasks"));
result = finish_command(&child);
delete_tempfile(&tfile);
free(name);
return result;
}
static int schtasks_schedule_tasks(const char *cmd)
{
const char *exec_path = git_exec_path();
return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||
schtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||
schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
}
static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)
{
if (run_maintenance)
return schtasks_schedule_tasks(cmd);
else
return schtasks_remove_tasks(cmd);
}
#define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE" #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
#define END_LINE "# END GIT MAINTENANCE SCHEDULE" #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
static int update_background_schedule(int run_maintenance) static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
{ {
int result = 0; int result = 0;
int in_old_region = 0; int in_old_region = 0;
struct child_process crontab_list = CHILD_PROCESS_INIT; struct child_process crontab_list = CHILD_PROCESS_INIT;
struct child_process crontab_edit = CHILD_PROCESS_INIT; struct child_process crontab_edit = CHILD_PROCESS_INIT;
FILE *cron_list, *cron_in; FILE *cron_list, *cron_in;
const char *crontab_name;
struct strbuf line = STRBUF_INIT; struct strbuf line = STRBUF_INIT;
struct lock_file lk;
char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) strvec_split(&crontab_list.args, cmd);
return error(_("another process is scheduling background maintenance"));
crontab_name = getenv("GIT_TEST_CRONTAB");
if (!crontab_name)
crontab_name = "crontab";
strvec_split(&crontab_list.args, crontab_name);
strvec_push(&crontab_list.args, "-l"); strvec_push(&crontab_list.args, "-l");
crontab_list.in = -1; crontab_list.in = -1;
crontab_list.out = dup(lk.tempfile->fd); crontab_list.out = dup(fd);
crontab_list.git_cmd = 0; crontab_list.git_cmd = 0;
if (start_command(&crontab_list)) { if (start_command(&crontab_list))
result = error(_("failed to run 'crontab -l'; your system might not support 'cron'")); return error(_("failed to run 'crontab -l'; your system might not support 'cron'"));
goto cleanup;
}
/* Ignore exit code, as an empty crontab will return error. */ /* Ignore exit code, as an empty crontab will return error. */
finish_command(&crontab_list); finish_command(&crontab_list);
@ -1559,17 +1879,15 @@ static int update_background_schedule(int run_maintenance)
* Read from the .lock file, filtering out the old * Read from the .lock file, filtering out the old
* schedule while appending the new schedule. * schedule while appending the new schedule.
*/ */
cron_list = fdopen(lk.tempfile->fd, "r"); cron_list = fdopen(fd, "r");
rewind(cron_list); rewind(cron_list);
strvec_split(&crontab_edit.args, crontab_name); strvec_split(&crontab_edit.args, cmd);
crontab_edit.in = -1; crontab_edit.in = -1;
crontab_edit.git_cmd = 0; crontab_edit.git_cmd = 0;
if (start_command(&crontab_edit)) { if (start_command(&crontab_edit))
result = error(_("failed to run 'crontab'; your system might not support 'cron'")); return error(_("failed to run 'crontab'; your system might not support 'cron'"));
goto cleanup;
}
cron_in = fdopen(crontab_edit.in, "w"); cron_in = fdopen(crontab_edit.in, "w");
if (!cron_in) { if (!cron_in) {
@ -1614,14 +1932,54 @@ static int update_background_schedule(int run_maintenance)
close(crontab_edit.in); close(crontab_edit.in);
done_editing: done_editing:
if (finish_command(&crontab_edit)) { if (finish_command(&crontab_edit))
result = error(_("'crontab' died")); result = error(_("'crontab' died"));
goto cleanup; else
} fclose(cron_list);
fclose(cron_list); return result;
}
#if defined(__APPLE__)
static const char platform_scheduler[] = "launchctl";
#elif defined(GIT_WINDOWS_NATIVE)
static const char platform_scheduler[] = "schtasks";
#else
static const char platform_scheduler[] = "crontab";
#endif
static int update_background_schedule(int enable)
{
int result;
const char *scheduler = platform_scheduler;
const char *cmd = scheduler;
char *testing;
struct lock_file lk;
char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
if (testing) {
char *sep = strchr(testing, ':');
if (!sep)
die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
*sep = '\0';
scheduler = testing;
cmd = sep + 1;
}
if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
return error(_("another process is scheduling background maintenance"));
if (!strcmp(scheduler, "launchctl"))
result = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);
else if (!strcmp(scheduler, "schtasks"))
result = schtasks_update_schedule(enable, lk.tempfile->fd, cmd);
else if (!strcmp(scheduler, "crontab"))
result = crontab_update_schedule(enable, lk.tempfile->fd, cmd);
else
die("unknown background scheduler: %s", scheduler);
cleanup:
rollback_lock_file(&lk); rollback_lock_file(&lk);
free(testing);
return result; return result;
} }

Просмотреть файл

@ -7,6 +7,19 @@ test_description='git maintenance builtin'
GIT_TEST_COMMIT_GRAPH=0 GIT_TEST_COMMIT_GRAPH=0
GIT_TEST_MULTI_PACK_INDEX=0 GIT_TEST_MULTI_PACK_INDEX=0
test_lazy_prereq XMLLINT '
xmllint --version
'
test_xmllint () {
if test_have_prereq XMLLINT
then
xmllint --noout "$@"
else
true
fi
}
test_expect_success 'help text' ' test_expect_success 'help text' '
test_expect_code 129 git maintenance -h 2>err && test_expect_code 129 git maintenance -h 2>err &&
test_i18ngrep "usage: git maintenance <subcommand>" err && test_i18ngrep "usage: git maintenance <subcommand>" err &&
@ -147,12 +160,12 @@ test_expect_success 'maintenance.loose-objects.auto' '
git -c maintenance.loose-objects.auto=1 maintenance \ git -c maintenance.loose-objects.auto=1 maintenance \
run --auto --task=loose-objects 2>/dev/null && run --auto --task=loose-objects 2>/dev/null &&
test_subcommand ! git prune-packed --quiet <trace-lo1.txt && test_subcommand ! git prune-packed --quiet <trace-lo1.txt &&
test_write_lines data-A | git hash-object -t blob --stdin -w && printf data-A | git hash-object -t blob --stdin -w &&
GIT_TRACE2_EVENT="$(pwd)/trace-loA" \ GIT_TRACE2_EVENT="$(pwd)/trace-loA" \
git -c maintenance.loose-objects.auto=2 \ git -c maintenance.loose-objects.auto=2 \
maintenance run --auto --task=loose-objects 2>/dev/null && maintenance run --auto --task=loose-objects 2>/dev/null &&
test_subcommand ! git prune-packed --quiet <trace-loA && test_subcommand ! git prune-packed --quiet <trace-loA &&
test_write_lines data-B | git hash-object -t blob --stdin -w && printf data-B | git hash-object -t blob --stdin -w &&
GIT_TRACE2_EVENT="$(pwd)/trace-loB" \ GIT_TRACE2_EVENT="$(pwd)/trace-loB" \
git -c maintenance.loose-objects.auto=2 \ git -c maintenance.loose-objects.auto=2 \
maintenance run --auto --task=loose-objects 2>/dev/null && maintenance run --auto --task=loose-objects 2>/dev/null &&
@ -300,6 +313,55 @@ test_expect_success '--schedule inheritance weekly -> daily -> hourly' '
test_subcommand git multi-pack-index write --no-progress <weekly.txt test_subcommand git multi-pack-index write --no-progress <weekly.txt
' '
test_expect_success 'maintenance.strategy inheritance' '
for task in commit-graph loose-objects incremental-repack
do
git config --unset maintenance.$task.schedule || return 1
done &&
test_when_finished git config --unset maintenance.strategy &&
git config maintenance.strategy incremental &&
GIT_TRACE2_EVENT="$(pwd)/incremental-hourly.txt" \
git maintenance run --schedule=hourly --quiet &&
GIT_TRACE2_EVENT="$(pwd)/incremental-daily.txt" \
git maintenance run --schedule=daily --quiet &&
test_subcommand git commit-graph write --split --reachable \
--no-progress <incremental-hourly.txt &&
test_subcommand ! git prune-packed --quiet <incremental-hourly.txt &&
test_subcommand ! git multi-pack-index write --no-progress \
<incremental-hourly.txt &&
test_subcommand git commit-graph write --split --reachable \
--no-progress <incremental-daily.txt &&
test_subcommand git prune-packed --quiet <incremental-daily.txt &&
test_subcommand git multi-pack-index write --no-progress \
<incremental-daily.txt &&
# Modify defaults
git config maintenance.commit-graph.schedule daily &&
git config maintenance.loose-objects.schedule hourly &&
git config maintenance.incremental-repack.enabled false &&
GIT_TRACE2_EVENT="$(pwd)/modified-hourly.txt" \
git maintenance run --schedule=hourly --quiet &&
GIT_TRACE2_EVENT="$(pwd)/modified-daily.txt" \
git maintenance run --schedule=daily --quiet &&
test_subcommand ! git commit-graph write --split --reachable \
--no-progress <modified-hourly.txt &&
test_subcommand git prune-packed --quiet <modified-hourly.txt &&
test_subcommand ! git multi-pack-index write --no-progress \
<modified-hourly.txt &&
test_subcommand git commit-graph write --split --reachable \
--no-progress <modified-daily.txt &&
test_subcommand git prune-packed --quiet <modified-daily.txt &&
test_subcommand ! git multi-pack-index write --no-progress \
<modified-daily.txt
'
test_expect_success 'register and unregister' ' test_expect_success 'register and unregister' '
test_when_finished git config --global --unset-all maintenance.repo && test_when_finished git config --global --unset-all maintenance.repo &&
git config --global --add maintenance.repo /existing1 && git config --global --add maintenance.repo /existing1 &&
@ -319,7 +381,7 @@ test_expect_success 'register and unregister' '
' '
test_expect_success 'start from empty cron table' ' test_expect_success 'start from empty cron table' '
GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start && GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
# start registers the repo # start registers the repo
git config --get --global maintenance.repo "$(pwd)" && git config --get --global maintenance.repo "$(pwd)" &&
@ -330,20 +392,113 @@ test_expect_success 'start from empty cron table' '
' '
test_expect_success 'stop from existing schedule' ' test_expect_success 'stop from existing schedule' '
GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop && GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance stop &&
# stop does not unregister the repo # stop does not unregister the repo
git config --get --global maintenance.repo "$(pwd)" && git config --get --global maintenance.repo "$(pwd)" &&
# Operation is idempotent # Operation is idempotent
GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop && GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance stop &&
test_must_be_empty cron.txt test_must_be_empty cron.txt
' '
test_expect_success 'start preserves existing schedule' ' test_expect_success 'start preserves existing schedule' '
echo "Important information!" >cron.txt && echo "Important information!" >cron.txt &&
GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start && GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
grep "Important information!" cron.txt grep "Important information!" cron.txt
' '
test_expect_success 'start and stop macOS maintenance' '
# ensure $HOME can be compared against hook arguments on all platforms
pfx=$(cd "$HOME" && pwd) &&
write_script print-args <<-\EOF &&
echo $* | sed "s:gui/[0-9][0-9]*:gui/[UID]:" >>args
EOF
rm -f args &&
GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&
# start registers the repo
git config --get --global maintenance.repo "$(pwd)" &&
ls "$HOME/Library/LaunchAgents" >actual &&
cat >expect <<-\EOF &&
org.git-scm.git.daily.plist
org.git-scm.git.hourly.plist
org.git-scm.git.weekly.plist
EOF
test_cmp expect actual &&
rm -f expect &&
for frequency in hourly daily weekly
do
PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
test_xmllint "$PLIST" &&
grep schedule=$frequency "$PLIST" &&
echo "bootout gui/[UID] $PLIST" >>expect &&
echo "bootstrap gui/[UID] $PLIST" >>expect || return 1
done &&
test_cmp expect args &&
rm -f args &&
GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance stop &&
# stop does not unregister the repo
git config --get --global maintenance.repo "$(pwd)" &&
printf "bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
hourly daily weekly >expect &&
test_cmp expect args &&
ls "$HOME/Library/LaunchAgents" >actual &&
test_line_count = 0 actual
'
test_expect_success 'start and stop Windows maintenance' '
write_script print-args <<-\EOF &&
echo $* >>args
while test $# -gt 0
do
case "$1" in
/xml) shift; xmlfile=$1; break ;;
*) shift ;;
esac
done
test -z "$xmlfile" || cp "$xmlfile" "$xmlfile.xml"
EOF
rm -f args &&
GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" GIT_TRACE2_PERF=1 git maintenance start &&
# start registers the repo
git config --get --global maintenance.repo "$(pwd)" &&
for frequency in hourly daily weekly
do
grep "/create /tn Git Maintenance ($frequency) /f /xml" args &&
file=$(ls schedule_$frequency*.xml) &&
test_xmllint "$file" &&
grep "encoding=.US-ASCII." "$file" || return 1
done &&
rm -f args &&
GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance stop &&
# stop does not unregister the repo
git config --get --global maintenance.repo "$(pwd)" &&
printf "/delete /tn Git Maintenance (%s) /f\n" \
hourly daily weekly >expect &&
test_cmp expect args
'
test_expect_success 'register preserves existing strategy' '
git config maintenance.strategy none &&
git maintenance register &&
test_config maintenance.strategy none &&
git config --unset maintenance.strategy &&
git maintenance register &&
test_config maintenance.strategy incremental
'
test_done test_done

Просмотреть файл

@ -1731,7 +1731,8 @@ test_lazy_prereq REBASE_P '
' '
# Ensure that no test accidentally triggers a Git command # Ensure that no test accidentally triggers a Git command
# that runs 'crontab', affecting a user's cron schedule. # that runs the actual maintenance scheduler, affecting a user's
# Tests that verify the cron integration must set this locally # system permanently.
# Tests that verify the scheduler integration must set this locally
# to avoid errors. # to avoid errors.
GIT_TEST_CRONTAB="exit 1" GIT_TEST_MAINT_SCHEDULER="none:exit 1"