All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] maintenance: use systemd timers on Linux
@ 2021-05-01 14:52 Lénaïc Huard
  2021-05-01 20:02 ` brian m. carlson
                   ` (5 more replies)
  0 siblings, 6 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-05-01 14:52 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine, Lénaïc Huard

The existing mechanism for scheduling background maintenance is done
through cron. On Linux systems managed by systemd, systemd provides an
alternative to schedule recurring tasks: systemd timers.

The main motivations to implement systemd timers in addition to cron
are:
* cron is optional and Linux systems running systemd might not have it
  installed.
* The execution of `crontab -l` can tell us if cron is installed but not
  if the daemon is actually running.
* With systemd, each service is run in its own cgroup and its logs are
  tagged by the service inside journald. With cron, all scheduled tasks
  are running in the cron daemon cgroup and all the logs of the
  user-scheduled tasks are pretended to belong to the system cron
  service.
  Concretely, a user that doesn’t have access to the system logs won’t
  have access to the log of its own tasks scheduled by cron whereas he
  will have access to the log of its own tasks scheduled by systemd
  timer.

In order to schedule git maintenance, we need two unit template files:
* ~/.config/systemd/user/git-maintenance@.service
  to define the command to be started by systemd and
* ~/.config/systemd/user/git-maintenance@.timer
  to define the schedule at which the command should be run.

Those units are templates that are parametrized by the frequency.

Based on those templates, 3 timers are started:
* git-maintenance@hourly.timer
* git-maintenance@daily.timer
* git-maintenance@weekly.timer

The command launched by those three timers are the same than with the
other scheduling methods:

git for-each-repo --config=maintenance.repo maintenance run
--schedule=%i

with the full path for git to ensure that the version of git launched
for the scheduled maintenance is the same as the one used to run
`maintenance start`.

The timer unit contains `Persistent=true` so that, if the computer is
powered down when a maintenance task should run, the task will be run
when the computer is back powered on.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 Documentation/git-maintenance.txt |  49 ++++++++
 builtin/gc.c                      | 188 +++++++++++++++++++++++++++++-
 t/t7900-maintenance.sh            |  51 ++++++++
 3 files changed, 287 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 80ddd33ceb..30443b417a 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -279,6 +279,55 @@ schedule to ensure you are executing the correct binaries in your
 schedule.
 
 
+BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMD
+-----------------------------------------------
+
+While Linux supports `cron`, depending on the distribution, `cron` may
+be an optional package not necessarily installed. On modern Linux
+distributions, systemd timers are superseding it.
+
+If user systemd timers are available, they will be used as a replacement
+of `cron`.
+
+In this case, `git maintenance start` will create user systemd timer units
+and start the timers. The current list of user-scheduled tasks can be found
+by running `systemctl --user list-timers`. The timers written by `git
+maintenance start` are similar to this:
+
+-----------------------------------------------------------------------
+$ systemctl --user list-timers
+NEXT                         LEFT          LAST                         PASSED     UNIT                         ACTIVATES
+Thu 2021-04-29 19:00:00 CEST 42min left    Thu 2021-04-29 18:00:11 CEST 17min ago  git-maintenance@hourly.timer git-maintenance@hourly.service
+Fri 2021-04-30 00:00:00 CEST 5h 42min left Thu 2021-04-29 00:00:11 CEST 18h ago    git-maintenance@daily.timer  git-maintenance@daily.service
+Mon 2021-05-03 00:00:00 CEST 3 days left   Mon 2021-04-26 00:00:11 CEST 3 days ago git-maintenance@weekly.timer git-maintenance@weekly.service
+
+3 timers listed.
+Pass --all to see loaded but inactive timers, too.
+-----------------------------------------------------------------------
+
+One timer is registered for each `--schedule=<frequency>` option.
+
+The definition of the systemd units can be inspected in the following files:
+
+-----------------------------------------------------------------------
+~/.config/systemd/user/git-maintenance@.timer
+~/.config/systemd/user/git-maintenance@.service
+~/.config/systemd/user/timers.target.wants/git-maintenance@hourly.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@daily.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@weekly.timer
+-----------------------------------------------------------------------
+
+`git maintenance start` will overwrite these files and start the timer
+again with `systemctl --user`, so any customization should be done by
+creating a drop-in file
+`~/.config/systemd/user/git-maintenance@.service.d/*.conf`.
+
+`git maintenance stop` will stop the user systemd timers and delete
+the above mentioned files.
+
+For more details, see systemd.timer(5)
+
+
 BACKGROUND MAINTENANCE ON MACOS SYSTEMS
 ---------------------------------------
 
diff --git a/builtin/gc.c b/builtin/gc.c
index ef7226d7bc..913fcfc882 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1872,6 +1872,25 @@ static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd
 		return schtasks_remove_tasks(cmd);
 }
 
+static int is_crontab_available(const char *cmd)
+{
+	struct child_process child = CHILD_PROCESS_INIT;
+
+	strvec_split(&child.args, cmd);
+	strvec_push(&child.args, "-l");
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child))
+		return 0;
+	/* Ignore exit code, as an empty crontab will return error. */
+	finish_command(&child);
+
+	return 1;
+}
+
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
@@ -1959,10 +1978,164 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 	return result;
 }
 
+static int is_systemd_timer_available(const char *cmd)
+{
+	struct child_process child = CHILD_PROCESS_INIT;
+
+	strvec_split(&child.args, cmd);
+	strvec_pushl(&child.args, "--user", "list-timers", NULL);
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child))
+		return 0;
+	if (finish_command(&child))
+		return 0;
+
+	return 1;
+}
+
+static char *systemd_timer_timer_filename()
+{
+	const char *filename = "~/.config/systemd/user/git-maintenance@.timer";
+	char *expanded = expand_user_path(filename, 0);
+	if (!expanded)
+		die(_("failed to expand path '%s'"), filename);
+
+	return expanded;
+}
+
+static char *systemd_timer_service_filename()
+{
+	const char *filename =
+		"~/.config/systemd/user/git-maintenance@.service";
+	char *expanded = expand_user_path(filename, 0);
+	if (!expanded)
+		die(_("failed to expand path '%s'"), filename);
+
+	return expanded;
+}
+
+static int systemd_timer_enable_unit(int enable,
+				     enum schedule_priority schedule,
+				     const char *cmd)
+{
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *frequency = get_frequency(schedule);
+
+	strvec_split(&child.args, cmd);
+	strvec_push(&child.args, "--user");
+	if (enable)
+		strvec_push(&child.args, "enable");
+	else
+		strvec_push(&child.args, "disable");
+	strvec_push(&child.args, "--now");
+	strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency);
+
+	if (start_command(&child))
+		die(_("failed to run systemctl"));
+	return finish_command(&child);
+}
+
+static int systemd_timer_delete_unit_templates()
+{
+	char *filename = systemd_timer_timer_filename();
+	unlink(filename);
+	free(filename);
+
+	filename = systemd_timer_service_filename();
+	unlink(filename);
+	free(filename);
+
+	return 0;
+}
+
+static int systemd_timer_delete_units(const char *cmd)
+{
+	return systemd_timer_enable_unit(0, SCHEDULE_HOURLY, cmd) ||
+	       systemd_timer_enable_unit(0, SCHEDULE_DAILY, cmd) ||
+	       systemd_timer_enable_unit(0, SCHEDULE_WEEKLY, cmd) ||
+	       systemd_timer_delete_unit_templates();
+}
+
+static int systemd_timer_write_unit_templates(const char *exec_path)
+{
+	char *filename;
+	FILE *file;
+	const char *unit;
+
+	filename = systemd_timer_timer_filename();
+	if (safe_create_leading_directories(filename))
+		die(_("failed to create directories for '%s'"), filename);
+	file = xfopen(filename, "w");
+	free(filename);
+
+	unit = "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Timer]\n"
+	       "OnCalendar=%i\n"
+	       "Persistent=true\n"
+	       "\n"
+	       "[Install]\n"
+	       "WantedBy=timers.target\n";
+	fputs(unit, file);
+	fclose(file);
+
+	filename = systemd_timer_service_filename();
+	if (safe_create_leading_directories(filename))
+		die(_("failed to create directories for '%s'"), filename);
+	file = xfopen(filename, "w");
+	free(filename);
+
+	unit = "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Service]\n"
+	       "Type=oneshot\n"
+	       "ExecStart=\"%1$s/git\" --exec-path=\"%1$s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
+	       "LockPersonality=yes\n"
+	       "MemoryDenyWriteExecute=yes\n"
+	       "NoNewPrivileges=yes\n"
+	       "RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\n"
+	       "RestrictNamespaces=yes\n"
+	       "RestrictRealtime=yes\n"
+	       "RestrictSUIDSGID=yes\n"
+	       "SystemCallArchitectures=native\n"
+	       "SystemCallFilter=@system-service\n";
+	fprintf(file, unit, exec_path);
+	fclose(file);
+
+	return 0;
+}
+
+static int systemd_timer_setup_units(const char *cmd)
+{
+	const char *exec_path = git_exec_path();
+
+	return systemd_timer_write_unit_templates(exec_path) ||
+	       systemd_timer_enable_unit(1, SCHEDULE_HOURLY, cmd) ||
+	       systemd_timer_enable_unit(1, SCHEDULE_DAILY, cmd) ||
+	       systemd_timer_enable_unit(1, SCHEDULE_WEEKLY, cmd);
+}
+
+static int systemd_timer_update_schedule(int run_maintenance, int fd,
+					 const char *cmd)
+{
+	if (run_maintenance)
+		return systemd_timer_setup_units(cmd);
+	else
+		return systemd_timer_delete_units(cmd);
+}
+
 #if defined(__APPLE__)
 static const char platform_scheduler[] = "launchctl";
 #elif defined(GIT_WINDOWS_NATIVE)
 static const char platform_scheduler[] = "schtasks";
+#elif defined(__linux__)
+static const char platform_scheduler[] = "crontab_or_systemctl";
 #else
 static const char platform_scheduler[] = "crontab";
 #endif
@@ -1986,6 +2159,15 @@ static int update_background_schedule(int enable)
 		cmd = sep + 1;
 	}
 
+	if (!strcmp(scheduler, "crontab_or_systemctl")) {
+		if (is_systemd_timer_available("systemctl"))
+			scheduler = cmd = "systemctl";
+		else if (is_crontab_available("crontab"))
+			scheduler = cmd = "crontab";
+		else
+			die(_("Neither systemd timers nor crontab are available"));
+	}
+
 	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
 		return error(_("another process is scheduling background maintenance"));
 
@@ -1995,10 +2177,14 @@ static int update_background_schedule(int enable)
 		result = schtasks_update_schedule(enable, get_lock_file_fd(&lk), cmd);
 	else if (!strcmp(scheduler, "crontab"))
 		result = crontab_update_schedule(enable, get_lock_file_fd(&lk), cmd);
+	else if (!strcmp(scheduler, "systemctl"))
+		result = systemd_timer_update_schedule(
+			enable, get_lock_file_fd(&lk), cmd);
 	else
-		die("unknown background scheduler: %s", scheduler);
+		die(_("unknown background scheduler: %s"), scheduler);
 
 	rollback_lock_file(&lk);
+	free(lock_path);
 	free(testing);
 	return result;
 }
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 2412d8c5c0..dd281789f4 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -20,6 +20,20 @@ test_xmllint () {
 	fi
 }
 
+test_lazy_prereq SYSTEMD_ANALYZE '
+	systemd-analyze --help >out &&
+	grep -w verify out
+'
+
+test_systemd_analyze_verify () {
+	if test_have_prereq SYSTEMD_ANALYZE
+	then
+		systemd-analyze verify "$@"
+	else
+		true
+	fi
+}
+
 test_expect_success 'help text' '
 	test_expect_code 129 git maintenance -h 2>err &&
 	test_i18ngrep "usage: git maintenance <subcommand>" err &&
@@ -615,6 +629,43 @@ test_expect_success 'start and stop Windows maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success 'start and stop Linux/systemd maintenance' '
+	write_script print-args <<-\EOF &&
+	echo $* >>args
+	EOF
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance start &&
+
+	# start registers the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_systemd_analyze_verify "$HOME/.config/systemd/user/git-maintenance@.service" &&
+
+	rm -f expect &&
+	for frequency in hourly daily weekly
+	do
+		echo "--user enable --now git-maintenance@${frequency}.timer" >>expect || return 1
+	done &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_path_is_missing "$HOME/.config/systemd/user/git-maintenance@.timer" &&
+	test_path_is_missing "$HOME/.config/systemd/user/git-maintenance@.service" &&
+
+	rm -f expect &&
+	for frequency in hourly daily weekly
+	do
+		echo "--user disable --now git-maintenance@${frequency}.timer" >>expect || return 1
+	done &&
+	test_cmp expect args
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
2.31.1


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* Re: [PATCH] maintenance: use systemd timers on Linux
  2021-05-01 14:52 [PATCH] maintenance: use systemd timers on Linux Lénaïc Huard
@ 2021-05-01 20:02 ` brian m. carlson
  2021-05-02  5:28 ` Bagas Sanjaya
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 138+ messages in thread
From: brian m. carlson @ 2021-05-01 20:02 UTC (permalink / raw)
  To: Lénaïc Huard; +Cc: git, Junio C Hamano, Derrick Stolee, Eric Sunshine

[-- Attachment #1: Type: text/plain, Size: 1568 bytes --]

On 2021-05-01 at 14:52:20, Lénaïc Huard wrote:
> The existing mechanism for scheduling background maintenance is done
> through cron. On Linux systems managed by systemd, systemd provides an
> alternative to schedule recurring tasks: systemd timers.
> 
> The main motivations to implement systemd timers in addition to cron
> are:
> * cron is optional and Linux systems running systemd might not have it
>   installed.
> * The execution of `crontab -l` can tell us if cron is installed but not
>   if the daemon is actually running.
> * With systemd, each service is run in its own cgroup and its logs are
>   tagged by the service inside journald. With cron, all scheduled tasks
>   are running in the cron daemon cgroup and all the logs of the
>   user-scheduled tasks are pretended to belong to the system cron
>   service.
>   Concretely, a user that doesn’t have access to the system logs won’t
>   have access to the log of its own tasks scheduled by cron whereas he
>   will have access to the log of its own tasks scheduled by systemd
>   timer.

I would prefer to see this as a configurable option.  I have systemd
installed (because it's not really optional to have a functional desktop
on Linux) but I want to restrict it to starting and stopping services,
not performing the tasks of cron.  cron is portable across a wide
variety of systems, including Linux variants (and WSL) that don't use
systemd, and I prefer to use more standard tooling when possible.
-- 
brian m. carlson (he/him or they/them)
Houston, Texas, US

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH] maintenance: use systemd timers on Linux
  2021-05-01 14:52 [PATCH] maintenance: use systemd timers on Linux Lénaïc Huard
  2021-05-01 20:02 ` brian m. carlson
@ 2021-05-02  5:28 ` Bagas Sanjaya
  2021-05-02  6:49   ` Eric Sunshine
  2021-05-02  6:45 ` Eric Sunshine
                   ` (3 subsequent siblings)
  5 siblings, 1 reply; 138+ messages in thread
From: Bagas Sanjaya @ 2021-05-02  5:28 UTC (permalink / raw)
  To: Lénaïc Huard, git; +Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine

On 01/05/21 21.52, Lénaïc Huard wrote:
> The existing mechanism for scheduling background maintenance is done
> through cron. On Linux systems managed by systemd, systemd provides an
> alternative to schedule recurring tasks: systemd timers.
> 
> The main motivations to implement systemd timers in addition to cron
> are:
> * cron is optional and Linux systems running systemd might not have it
>    installed.

Supposed that I have Linux box with systemd and classical cron. Should
systemd timers be preferred over cron?

> * The execution of `crontab -l` can tell us if cron is installed but not
>    if the daemon is actually running.
> * With systemd, each service is run in its own cgroup and its logs are
>    tagged by the service inside journald. With cron, all scheduled tasks
>    are running in the cron daemon cgroup and all the logs of the
>    user-scheduled tasks are pretended to belong to the system cron
>    service.
>    Concretely, a user that doesn’t have access to the system logs won’t
>    have access to the log of its own tasks scheduled by cron whereas he
>    will have access to the log of its own tasks scheduled by systemd
>    timer.
> 
> In order to schedule git maintenance, we need two unit template files:
> * ~/.config/systemd/user/git-maintenance@.service
>    to define the command to be started by systemd and
> * ~/.config/systemd/user/git-maintenance@.timer
>    to define the schedule at which the command should be run.
> 
> Those units are templates that are parametrized by the frequency.
> 
> Based on those templates, 3 timers are started:
> * git-maintenance@hourly.timer
> * git-maintenance@daily.timer
> * git-maintenance@weekly.timer
> 
> The command launched by those three timers are the same than with the
> other scheduling methods:
> 
> git for-each-repo --config=maintenance.repo maintenance run
> --schedule=%i
> 
> with the full path for git to ensure that the version of git launched
> for the scheduled maintenance is the same as the one used to run
> `maintenance start`.
> 
> The timer unit contains `Persistent=true` so that, if the computer is
> powered down when a maintenance task should run, the task will be run
> when the computer is back powered on.
> 
> Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
Nevertheless, because we are dealing with external dependency (systemd), it
should makes sense to enforce this dependency requirement when user choose to use
systemd timers so that users on non-systemd boxes (such as Gentoo with OpenRC)
don't see errors that forcing them to use systemd.

-- 
An old man doll... just what I always wanted! - Clara

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH] maintenance: use systemd timers on Linux
  2021-05-01 14:52 [PATCH] maintenance: use systemd timers on Linux Lénaïc Huard
  2021-05-01 20:02 ` brian m. carlson
  2021-05-02  5:28 ` Bagas Sanjaya
@ 2021-05-02  6:45 ` Eric Sunshine
  2021-05-02 14:10   ` Phillip Wood
  2021-05-05 12:01   ` Ævar Arnfjörð Bjarmason
  2021-05-02 11:12 ` Bagas Sanjaya
                   ` (2 subsequent siblings)
  5 siblings, 2 replies; 138+ messages in thread
From: Eric Sunshine @ 2021-05-02  6:45 UTC (permalink / raw)
  To: Lénaïc Huard
  Cc: Git List, Junio C Hamano, Derrick Stolee, brian m. carlson

On Sat, May 1, 2021 at 10:59 AM Lénaïc Huard <lenaic@lhuard.fr> wrote:
> The existing mechanism for scheduling background maintenance is done
> through cron. On Linux systems managed by systemd, systemd provides an
> alternative to schedule recurring tasks: systemd timers.

Thanks for working on this. While `cron` has been the go-to standard
for decades, `systemd` is certainly widespread enough that it makes
sense to support it, as well.

> The main motivations to implement systemd timers in addition to cron
> are:
> * cron is optional and Linux systems running systemd might not have it
>   installed.
> * The execution of `crontab -l` can tell us if cron is installed but not
>   if the daemon is actually running.
> * With systemd, each service is run in its own cgroup and its logs are
>   tagged by the service inside journald. With cron, all scheduled tasks
>   are running in the cron daemon cgroup and all the logs of the
>   user-scheduled tasks are pretended to belong to the system cron
>   service.
>   Concretely, a user that doesn’t have access to the system logs won’t
>   have access to the log of its own tasks scheduled by cron whereas he
>   will have access to the log of its own tasks scheduled by systemd
>   timer.

The last point is somewhat compelling. A potential counterargument is
that `cron` does send email to the user by default if any output is
generated by the cron job. However, it seems quite likely these days
that many systems either won't have local mail service enabled or the
user won't bother checking the local mailbox. It's a minor point, but
if you re-roll it might make sense for the commit message to expand
the last point by saying that although `cron` attempts to send email,
that email may go unseen by the user.

> In order to schedule git maintenance, we need two unit template files:
> * ~/.config/systemd/user/git-maintenance@.service
>   to define the command to be started by systemd and
> * ~/.config/systemd/user/git-maintenance@.timer
>   to define the schedule at which the command should be run.
> [...]
> The timer unit contains `Persistent=true` so that, if the computer is
> powered down when a maintenance task should run, the task will be run
> when the computer is back powered on.

It would be nice for the commit message to also give some high-level
information about how git-maintenance chooses between `cron` and
`systemd` and whether the user can influence that decision. (I know
the answer because I read the patch, but this is the sort of
information which is good to have in the commit message; readers want
to know why certain choices were made.)

Although I avoid Linux distros with `systemd`, my knee-jerk reaction,
like brian's upthread, is that there should be some escape hatch or
direct mechanism to allow the user to choose between `systemd` and
`cron`.

The patch itself is straightforward enough and nicely follows the
pattern established for already-implemented schedulers, so I don't
have a lot to say about it. I did leave a few comments below, most of
which are subjective nits and minor observations, though there are two
or three actionable items.

> Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
> ---
> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
> @@ -279,6 +279,55 @@ schedule to ensure you are executing the correct binaries in your
> +BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMD
> +-----------------------------------------------

Is there a reason for the duplicated "SYSTEMD" that I'm missing? I
suppose you probably mean "SYSTEMD SYSTEMS".

> +In this case, `git maintenance start` will create user systemd timer units
> +and start the timers. The current list of user-scheduled tasks can be found
> +by running `systemctl --user list-timers`. The timers written by `git
> +maintenance start` are similar to this:
> +
> +-----------------------------------------------------------------------
> +$ systemctl --user list-timers
> +NEXT                         LEFT          LAST                         PASSED     UNIT                         ACTIVATES
> +Thu 2021-04-29 19:00:00 CEST 42min left    Thu 2021-04-29 18:00:11 CEST 17min ago  git-maintenance@hourly.timer git-maintenance@hourly.service
> +Fri 2021-04-30 00:00:00 CEST 5h 42min left Thu 2021-04-29 00:00:11 CEST 18h ago    git-maintenance@daily.timer  git-maintenance@daily.service
> +Mon 2021-05-03 00:00:00 CEST 3 days left   Mon 2021-04-26 00:00:11 CEST 3 days ago git-maintenance@weekly.timer git-maintenance@weekly.service
> +
> +3 timers listed.
> +Pass --all to see loaded but inactive timers, too.
> +-----------------------------------------------------------------------

I suspect that the "3 timers listed" and "Pass --all" lines don't add
value and can be dropped without hurting the example.

> +`git maintenance start` will overwrite these files and start the timer
> +again with `systemctl --user`, so any customization should be done by
> +creating a drop-in file
> +`~/.config/systemd/user/git-maintenance@.service.d/*.conf`.

Will `systemd` users generally understand what filename to create in
the "...@.service.d/" directory, and will they know what to populate
the file with? (Genuine question; I've never dealt with that.)

> diff --git a/builtin/gc.c b/builtin/gc.c
> @@ -1872,6 +1872,25 @@ static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd
> +static int is_crontab_available(const char *cmd)
> +{
> +       struct child_process child = CHILD_PROCESS_INIT;
> +
> +       strvec_split(&child.args, cmd);
> +       strvec_push(&child.args, "-l");
> +       child.no_stdin = 1;
> +       child.no_stdout = 1;
> +       child.no_stderr = 1;
> +       child.silent_exec_failure = 1;
> +
> +       if (start_command(&child))
> +               return 0;
> +       /* Ignore exit code, as an empty crontab will return error. */
> +       finish_command(&child);
> +
> +       return 1;
> +}

Ignoring the error from `crontab -l` is an already-established idiom
in this file. Okay.

Nit: There doesn't seem to be a need for the blank line before `return
1`, and other maintenance-related functions don't have such a blank
line. The same comment about blank lines before `return` applies to
other newly-added functions, as well. But it's subjective, and not
necessarily worth changing.

> +static char *systemd_timer_timer_filename()
> +{
> +       const char *filename = "~/.config/systemd/user/git-maintenance@.timer";
> +       char *expanded = expand_user_path(filename, 0);
> +       if (!expanded)
> +               die(_("failed to expand path '%s'"), filename);
> +
> +       return expanded;
> +}

I was curious whether this would fail if `.config/systemd/user/`
didn't already exist, but looking at the implementation of
expand_user_path() , I see that it doesn't require the path to already
exist if you pass 0 for the second argument as you do here. Okay.

> +static char *systemd_timer_service_filename()
> +{
> +       const char *filename =
> +               "~/.config/systemd/user/git-maintenance@.service";
> +       char *expanded = expand_user_path(filename, 0);
> +       if (!expanded)
> +               die(_("failed to expand path '%s'"), filename);
> +
> +       return expanded;
> +}

The duplication of code between systemd_timer_timer_filename() and
systemd_timer_service_filename() is probably too minor to worry about.
Okay.

> +static int systemd_timer_enable_unit(int enable,
> +                                    enum schedule_priority schedule,
> +                                    const char *cmd)
> +{
> +       struct child_process child = CHILD_PROCESS_INIT;
> +       const char *frequency = get_frequency(schedule);
> +
> +       strvec_split(&child.args, cmd);
> +       strvec_push(&child.args, "--user");
> +       if (enable)
> +               strvec_push(&child.args, "enable");
> +       else
> +               strvec_push(&child.args, "disable");

It's subjective, but this might be more nicely expressed as:

    strvec_push(&child.args, enable ? "enable" : "disable");

> +       strvec_push(&child.args, "--now");
> +       strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency);
> +
> +       if (start_command(&child))
> +               die(_("failed to run systemctl"));
> +       return finish_command(&child);
> +}
> +static int systemd_timer_write_unit_templates(const char *exec_path)
> +{
> +       unit = "[Unit]\n"
> +              "Description=Optimize Git repositories data\n"
> +              "\n"
> +              "[Service]\n"
> +              "Type=oneshot\n"
> +              "ExecStart=\"%1$s/git\" --exec-path=\"%1$s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"

I see that it's in POSIX, but do we use this `%n$s` directive
elsewhere in the Git source code? If not, I'd be cautious of
introducing it here. Maybe it's better to just use plain `%s` twice...

> +              "LockPersonality=yes\n"
> +              "MemoryDenyWriteExecute=yes\n"
> +              "NoNewPrivileges=yes\n"
> +              "RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\n"
> +              "RestrictNamespaces=yes\n"
> +              "RestrictRealtime=yes\n"
> +              "RestrictSUIDSGID=yes\n"
> +              "SystemCallArchitectures=native\n"
> +              "SystemCallFilter=@system-service\n";
> +       fprintf(file, unit, exec_path);

... and then:

    fprintf(file, unit, exec_path, exec_path);

> +       fclose(file);
> +
> +       return 0;
> +}
> @@ -1986,6 +2159,15 @@ static int update_background_schedule(int enable)
> +       if (!strcmp(scheduler, "crontab_or_systemctl")) {
> +               if (is_systemd_timer_available("systemctl"))
> +                       scheduler = cmd = "systemctl";
> +               else if (is_crontab_available("crontab"))
> +                       scheduler = cmd = "crontab";
> +               else
> +                       die(_("Neither systemd timers nor crontab are available"));
> +       }

Other messages emitted by git-maintenance are entirely lowercase, so
downcasing "Neither" would be appropriate.

> @@ -1995,10 +2177,14 @@ static int update_background_schedule(int enable)
> -               die("unknown background scheduler: %s", scheduler);
> +               die(_("unknown background scheduler: %s"), scheduler);

This change is unrelated to the rest of the patch. Normally, such a
"fix" would be made as a separate patch. This one is somewhat minor,
so perhaps it doesn't matter whether it's in this patch...

>         rollback_lock_file(&lk);
> +       free(lock_path);
>         free(testing);
>         return result;

... however, this leak fix probably deserves its own patch. Or, at the
very least, mention these two fixes in this commit message.

> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
> @@ -20,6 +20,20 @@ test_xmllint () {
> +test_lazy_prereq SYSTEMD_ANALYZE '
> +       systemd-analyze --help >out &&
> +       grep -w verify out
> +'

Unportable use of `grep -w`. It's neither in POSIX nor understood by
BSD-lineage `grep` (including macOS `grep`).

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH] maintenance: use systemd timers on Linux
  2021-05-02  5:28 ` Bagas Sanjaya
@ 2021-05-02  6:49   ` Eric Sunshine
  0 siblings, 0 replies; 138+ messages in thread
From: Eric Sunshine @ 2021-05-02  6:49 UTC (permalink / raw)
  To: Bagas Sanjaya
  Cc: Lénaïc Huard, Git List, Junio C Hamano, Derrick Stolee

On Sun, May 2, 2021 at 1:28 AM Bagas Sanjaya <bagasdotme@gmail.com> wrote:
> On 01/05/21 21.52, Lénaïc Huard wrote:
> > The existing mechanism for scheduling background maintenance is done
> > through cron. On Linux systems managed by systemd, systemd provides an
> > alternative to schedule recurring tasks: systemd timers.
> >
> > The main motivations to implement systemd timers in addition to cron
> > are:
> > * cron is optional and Linux systems running systemd might not have it
> >    installed.
>
> Supposed that I have Linux box with systemd and classical cron. Should
> systemd timers be preferred over cron?

The implementation in this patch unconditionally prefers `systemd`
over `cron`. Whether that's a good idea is subject to question (as
both brian and I mentioned in our reviews).

> Nevertheless, because we are dealing with external dependency (systemd), it
> should makes sense to enforce this dependency requirement when user choose to use
> systemd timers so that users on non-systemd boxes (such as Gentoo with OpenRC)
> don't see errors that forcing them to use systemd.

If you scan through the patch itself, you will find that it is careful
to choose the appropriate scheduler and not to spit out errors when
one or the other scheduler is unavailable.

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH] maintenance: use systemd timers on Linux
  2021-05-01 14:52 [PATCH] maintenance: use systemd timers on Linux Lénaïc Huard
                   ` (2 preceding siblings ...)
  2021-05-02  6:45 ` Eric Sunshine
@ 2021-05-02 11:12 ` Bagas Sanjaya
  2021-05-03 12:04 ` Derrick Stolee
  2021-05-09 21:32 ` [PATCH v2 0/1] " Lénaïc Huard
  5 siblings, 0 replies; 138+ messages in thread
From: Bagas Sanjaya @ 2021-05-02 11:12 UTC (permalink / raw)
  To: Lénaïc Huard, git; +Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine

On 01/05/21 21.52, Lénaïc Huard wrote:
> +BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMD
> +-----------------------------------------------
> +

systemd was repeated twice above. It should be
`BACKGROUND MAINTENANCE WITH SYSTEMD TIMERS`

>   test_expect_success 'help text' '
>   	test_expect_code 129 git maintenance -h 2>err &&

Why exit code 129 there?

-- 
An old man doll... just what I always wanted! - Clara

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH] maintenance: use systemd timers on Linux
  2021-05-02  6:45 ` Eric Sunshine
@ 2021-05-02 14:10   ` Phillip Wood
  2021-05-05 12:19     ` Đoàn Trần Công Danh
  2021-05-05 12:01   ` Ævar Arnfjörð Bjarmason
  1 sibling, 1 reply; 138+ messages in thread
From: Phillip Wood @ 2021-05-02 14:10 UTC (permalink / raw)
  To: Eric Sunshine, Lénaïc Huard
  Cc: Git List, Junio C Hamano, Derrick Stolee, brian m. carlson

On 02/05/2021 07:45, Eric Sunshine wrote:
> On Sat, May 1, 2021 at 10:59 AM Lénaïc Huard <lenaic@lhuard.fr> wrote:
>> The existing mechanism for scheduling background maintenance is done
>> through cron. On Linux systems managed by systemd, systemd provides an
>> alternative to schedule recurring tasks: systemd timers.
> 
> Thanks for working on this. While `cron` has been the go-to standard
> for decades, `systemd` is certainly widespread enough that it makes
> sense to support it, as well.

Yes, thank you for working on this, it will be very useful to users like 
me who use a linux distribution that does not install a cron daemon by 
default but relies on systemd instead.

>> The main motivations to implement systemd timers in addition to cron
>> are:
>> * cron is optional and Linux systems running systemd might not have it
>>    installed.
>> * The execution of `crontab -l` can tell us if cron is installed but not
>>    if the daemon is actually running.

Can we use systemctl to see if it is running (and enabled so we know it 
will be restarted after a reboot)?

>> * With systemd, each service is run in its own cgroup and its logs are
>>    tagged by the service inside journald. With cron, all scheduled tasks
>>    are running in the cron daemon cgroup and all the logs of the
>>    user-scheduled tasks are pretended to belong to the system cron
>>    service.
>>    Concretely, a user that doesn’t have access to the system logs won’t
>>    have access to the log of its own tasks scheduled by cron whereas he
>>    will have access to the log of its own tasks scheduled by systemd
>>    timer.
> 
> The last point is somewhat compelling. A potential counterargument is
> that `cron` does send email to the user by default if any output is
> generated by the cron job. However, it seems quite likely these days
> that many systems either won't have local mail service enabled or the
> user won't bother checking the local mailbox. It's a minor point, but
> if you re-roll it might make sense for the commit message to expand
> the last point by saying that although `cron` attempts to send email,
> that email may go unseen by the user.
> 
>> In order to schedule git maintenance, we need two unit template files:
>> * ~/.config/systemd/user/git-maintenance@.service
>>    to define the command to be started by systemd and
>> * ~/.config/systemd/user/git-maintenance@.timer
>>    to define the schedule at which the command should be run.
>> [...]
>> The timer unit contains `Persistent=true` so that, if the computer is
>> powered down when a maintenance task should run, the task will be run
>> when the computer is back powered on.
> 
> It would be nice for the commit message to also give some high-level
> information about how git-maintenance chooses between `cron` and
> `systemd` and whether the user can influence that decision. (I know
> the answer because I read the patch, but this is the sort of
> information which is good to have in the commit message; readers want
> to know why certain choices were made.)
> 
> Although I avoid Linux distros with `systemd`, my knee-jerk reaction,
> like brian's upthread, is that there should be some escape hatch or
> direct mechanism to allow the user to choose between `systemd` and
> `cron`.

I agree that if both are present the user should be able to choose one. 
I'm not sure what the default should be in that case - before I read the 
commit message and Eric's comments I was inclined to say that if the 
user has cron installed we should take that as a sign they preferred 
cron over systemd timers and use that. However, given the arguments 
above about not knowing if cron is running and the user not necessarily 
getting emails from cron or being able to read the logs than maybe 
defaulting to systemd timers makes sense.

> The patch itself is straightforward enough and nicely follows the
> pattern established for already-implemented schedulers, so I don't
> have a lot to say about it. I did leave a few comments below, most of
> which are subjective nits and minor observations, though there are two
> or three actionable items.
> 
>> Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
>> ---
>> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
>> @@ -279,6 +279,55 @@ schedule to ensure you are executing the correct binaries in your
>> +BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMD
>> +-----------------------------------------------
> 
> Is there a reason for the duplicated "SYSTEMD" that I'm missing? I
> suppose you probably mean "SYSTEMD SYSTEMS".
> 
>> +In this case, `git maintenance start` will create user systemd timer units
>> +and start the timers. The current list of user-scheduled tasks can be found
>> +by running `systemctl --user list-timers`. The timers written by `git
>> +maintenance start` are similar to this:
>> +
>> +-----------------------------------------------------------------------
>> +$ systemctl --user list-timers
>> +NEXT                         LEFT          LAST                         PASSED     UNIT                         ACTIVATES
>> +Thu 2021-04-29 19:00:00 CEST 42min left    Thu 2021-04-29 18:00:11 CEST 17min ago  git-maintenance@hourly.timer git-maintenance@hourly.service
>> +Fri 2021-04-30 00:00:00 CEST 5h 42min left Thu 2021-04-29 00:00:11 CEST 18h ago    git-maintenance@daily.timer  git-maintenance@daily.service
>> +Mon 2021-05-03 00:00:00 CEST 3 days left   Mon 2021-04-26 00:00:11 CEST 3 days ago git-maintenance@weekly.timer git-maintenance@weekly.service
>> +
>> +3 timers listed.
>> +Pass --all to see loaded but inactive timers, too.
>> +-----------------------------------------------------------------------
> 
> I suspect that the "3 timers listed" and "Pass --all" lines don't add
> value and can be dropped without hurting the example.

If the idea is to show the output of `systemctl --user list-timers` then 
I don't think we should be editing it. I also think having the column 
headers helps as it shows what the fields are.

>> +`git maintenance start` will overwrite these files and start the timer
>> +again with `systemctl --user`, so any customization should be done by
>> +creating a drop-in file
>> +`~/.config/systemd/user/git-maintenance@.service.d/*.conf`.
> 
> Will `systemd` users generally understand what filename to create in
> the "...@.service.d/" directory, and will they know what to populate
> the file with? (Genuine question; I've never dealt with that.)

I think it would be helpful to explicitly mention the file names (I 
don't think I could tell you what they are without reading the relevant 
systemd man page)

>> diff --git a/builtin/gc.c b/builtin/gc.c
>> @@ -1872,6 +1872,25 @@ static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd
>> +static int is_crontab_available(const char *cmd)
>> +{
>> +       struct child_process child = CHILD_PROCESS_INIT;
>> +
>> +       strvec_split(&child.args, cmd);
>> +       strvec_push(&child.args, "-l");
>> +       child.no_stdin = 1;
>> +       child.no_stdout = 1;
>> +       child.no_stderr = 1;
>> +       child.silent_exec_failure = 1;
>> +
>> +       if (start_command(&child))
>> +               return 0;
>> +       /* Ignore exit code, as an empty crontab will return error. */
>> +       finish_command(&child);
>> +
>> +       return 1;
>> +}
> 
> Ignoring the error from `crontab -l` is an already-established idiom
> in this file. Okay.
> 
> Nit: There doesn't seem to be a need for the blank line before `return
> 1`, and other maintenance-related functions don't have such a blank
> line. The same comment about blank lines before `return` applies to
> other newly-added functions, as well. But it's subjective, and not
> necessarily worth changing.
> 
>> +static char *systemd_timer_timer_filename()
>> +{
>> +       const char *filename = "~/.config/systemd/user/git-maintenance@.timer";
>> +       char *expanded = expand_user_path(filename, 0);
>> +       if (!expanded)
>> +               die(_("failed to expand path '%s'"), filename);
>> +
>> +       return expanded;
>> +}
> 
> I was curious whether this would fail if `.config/systemd/user/`
> didn't already exist, but looking at the implementation of
> expand_user_path() , I see that it doesn't require the path to already
> exist if you pass 0 for the second argument as you do here. Okay.

Do we need to worry about $XDG_CONFIG_HOME rather than hard coding 
"~/.config/". There is a function xdg_config_home() that takes care of this.

Thanks again for working on this

Best Wishes

Phillip

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH] maintenance: use systemd timers on Linux
  2021-05-01 14:52 [PATCH] maintenance: use systemd timers on Linux Lénaïc Huard
                   ` (3 preceding siblings ...)
  2021-05-02 11:12 ` Bagas Sanjaya
@ 2021-05-03 12:04 ` Derrick Stolee
  2021-05-09 21:32 ` [PATCH v2 0/1] " Lénaïc Huard
  5 siblings, 0 replies; 138+ messages in thread
From: Derrick Stolee @ 2021-05-03 12:04 UTC (permalink / raw)
  To: Lénaïc Huard, git; +Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine

On 5/1/2021 10:52 AM, Lénaïc Huard wrote:
> The existing mechanism for scheduling background maintenance is done
> through cron. On Linux systems managed by systemd, systemd provides an
> alternative to schedule recurring tasks: systemd timers.
> 
> The main motivations to implement systemd timers in addition to cron
> are:
...

Thank you for working on this. Users have questioned "why cron?"
since the release of background maintenance, so I appreciate you
taking the time to port this feature to systemd.

I won't do a deep code review here since that seems to already be
covered, and a v2 seems required. Ensuring that users can choose
which of the two backends is a good idea. We might even want to
start with 'cron' as the default and 'systemd' as an opt-in.

The other concern I wanted to discuss was the upgrade scenario.
If users have already enabled background maintenance with the
cron backend, how can we help users disable the cron backend
before they upgrade to the systemd version? I imagine that we
should disable cron when enabling systemd, using
crontab_update_schedule() with run_maintenance given as 0.
We might want to enable quiet errors in that method for the
case that cron does not exist.

It is important to make it clear that we only accept one
scheduler at a time, since they would be competing to run
'git for-each-repo' over the same list of repos. A single user
cannot schedule some repositories in 'cron' and another set in
'systemd'. This seems like an acceptable technical limitation.

Thanks, and I look forward to your v2!
-Stolee

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH] maintenance: use systemd timers on Linux
  2021-05-02  6:45 ` Eric Sunshine
  2021-05-02 14:10   ` Phillip Wood
@ 2021-05-05 12:01   ` Ævar Arnfjörð Bjarmason
  2021-05-09 22:34     ` Lénaïc Huard
  1 sibling, 1 reply; 138+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-05 12:01 UTC (permalink / raw)
  To: Eric Sunshine
  Cc: Lénaïc Huard, Git List, Junio C Hamano, Derrick Stolee,
	brian m. carlson


On Sun, May 02 2021, Eric Sunshine wrote:

> On Sat, May 1, 2021 at 10:59 AM Lénaïc Huard <lenaic@lhuard.fr> wrote:
>> +       strvec_push(&child.args, "--now");
>> +       strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency);
>> +
>> +       if (start_command(&child))
>> +               die(_("failed to run systemctl"));
>> +       return finish_command(&child);
>> +}
>> +static int systemd_timer_write_unit_templates(const char *exec_path)
>> +{
>> +       unit = "[Unit]\n"
>> +              "Description=Optimize Git repositories data\n"
>> +              "\n"
>> +              "[Service]\n"
>> +              "Type=oneshot\n"
>> +              "ExecStart=\"%1$s/git\" --exec-path=\"%1$s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
>
> I see that it's in POSIX, but do we use this `%n$s` directive
> elsewhere in the Git source code? If not, I'd be cautious of
> introducing it here. Maybe it's better to just use plain `%s` twice...

We use it in po/, so for sprintf() on systems that don't have
NO_GETTEXT=Y we already test it in the wild.

But no, I don't think anything in the main source uses it, FWIW I have a
WIP series in my own fork that I've cooked for a while that uses this, I
haven't run into any issues with it in either GitHub's CI
(e.g. Windows), or on the systems I myself test on.

I think it would be a useful canary to just take a change like this, we
can always change it to the form you suggest if it doesn't work out.

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH] maintenance: use systemd timers on Linux
  2021-05-02 14:10   ` Phillip Wood
@ 2021-05-05 12:19     ` Đoàn Trần Công Danh
  2021-05-05 14:57       ` Phillip Wood
  0 siblings, 1 reply; 138+ messages in thread
From: Đoàn Trần Công Danh @ 2021-05-05 12:19 UTC (permalink / raw)
  To: phillip.wood
  Cc: Eric Sunshine, Lénaïc Huard, Git List, Junio C Hamano,
	Derrick Stolee, brian m. carlson

On 2021-05-02 15:10:05+0100, Phillip Wood <phillip.wood123@gmail.com> wrote:
> On 02/05/2021 07:45, Eric Sunshine wrote:
> > On Sat, May 1, 2021 at 10:59 AM Lénaïc Huard <lenaic@lhuard.fr> wrote:
> > > The existing mechanism for scheduling background maintenance is done
> > > through cron. On Linux systems managed by systemd, systemd provides an
> > > alternative to schedule recurring tasks: systemd timers.
> > 
> > Thanks for working on this. While `cron` has been the go-to standard
> > for decades, `systemd` is certainly widespread enough that it makes
> > sense to support it, as well.
> 
> Yes, thank you for working on this, it will be very useful to users like me
> who use a linux distribution that does not install a cron daemon by default
> but relies on systemd instead.
> 
> > > The main motivations to implement systemd timers in addition to cron
> > > are:
> > > * cron is optional and Linux systems running systemd might not have it
> > >    installed.
> > > * The execution of `crontab -l` can tell us if cron is installed but not
> > >    if the daemon is actually running.
> 
> Can we use systemctl to see if it is running (and enabled so we know it will
> be restarted after a reboot)?

Not sure if I understand this suggestion.
However, non-systemd systems doesn't have systemctl command to begin
with.

> > > * With systemd, each service is run in its own cgroup and its logs are
> > >    tagged by the service inside journald. With cron, all scheduled tasks
> > >    are running in the cron daemon cgroup and all the logs of the
> > >    user-scheduled tasks are pretended to belong to the system cron
> > >    service.
> > >    Concretely, a user that doesn’t have access to the system logs won’t
> > >    have access to the log of its own tasks scheduled by cron whereas he
> > >    will have access to the log of its own tasks scheduled by systemd
> > >    timer.
> > 
> > The last point is somewhat compelling. A potential counterargument is
> > that `cron` does send email to the user by default if any output is
> > generated by the cron job. However, it seems quite likely these days
> > that many systems either won't have local mail service enabled or the
> > user won't bother checking the local mailbox. It's a minor point, but
> > if you re-roll it might make sense for the commit message to expand
> > the last point by saying that although `cron` attempts to send email,
> > that email may go unseen by the user.
> > 
> > > In order to schedule git maintenance, we need two unit template files:
> > > * ~/.config/systemd/user/git-maintenance@.service
> > >    to define the command to be started by systemd and
> > > * ~/.config/systemd/user/git-maintenance@.timer
> > >    to define the schedule at which the command should be run.

I think it would be better to change ~/.config here to
$XDG_CONFIG_HOME, as others also points out in another comments.

[..snip..]

> > > +`git maintenance start` will overwrite these files and start the timer
> > > +again with `systemctl --user`, so any customization should be done by
> > > +creating a drop-in file
> > > +`~/.config/systemd/user/git-maintenance@.service.d/*.conf`.

Ditto.

> > Will `systemd` users generally understand what filename to create in
> > the "...@.service.d/" directory, and will they know what to populate
> > the file with? (Genuine question; I've never dealt with that.)
> 
> I think it would be helpful to explicitly mention the file names (I don't
> think I could tell you what they are without reading the relevant systemd
> man page)

[..snip..]

> > > +static char *systemd_timer_timer_filename()
> > > +{
> > > +       const char *filename = "~/.config/systemd/user/git-maintenance@.timer";
> > > +       char *expanded = expand_user_path(filename, 0);
> > > +       if (!expanded)
> > > +               die(_("failed to expand path '%s'"), filename);
> > > +
> > > +       return expanded;
> > > +}
> > 
> > I was curious whether this would fail if `.config/systemd/user/`
> > didn't already exist, but looking at the implementation of
> > expand_user_path() , I see that it doesn't require the path to already
> > exist if you pass 0 for the second argument as you do here. Okay.
> 
> Do we need to worry about $XDG_CONFIG_HOME rather than hard coding
> "~/.config/". There is a function xdg_config_home() that takes care of this.

-- 
Danh

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH] maintenance: use systemd timers on Linux
  2021-05-05 12:19     ` Đoàn Trần Công Danh
@ 2021-05-05 14:57       ` Phillip Wood
  0 siblings, 0 replies; 138+ messages in thread
From: Phillip Wood @ 2021-05-05 14:57 UTC (permalink / raw)
  To: Đoàn Trần Công Danh, phillip.wood
  Cc: Eric Sunshine, Lénaïc Huard, Git List, Junio C Hamano,
	Derrick Stolee, brian m. carlson

Hi Đoàn

On 05/05/2021 13:19, Đoàn Trần Công Danh wrote:
> On 2021-05-02 15:10:05+0100, Phillip Wood <phillip.wood123@gmail.com> wrote:
>> On 02/05/2021 07:45, Eric Sunshine wrote:
>>> On Sat, May 1, 2021 at 10:59 AM Lénaïc Huard <lenaic@lhuard.fr> wrote:
>>>> The existing mechanism for scheduling background maintenance is done
>>>> through cron. On Linux systems managed by systemd, systemd provides an
>>>> alternative to schedule recurring tasks: systemd timers.
>>>
>>> Thanks for working on this. While `cron` has been the go-to standard
>>> for decades, `systemd` is certainly widespread enough that it makes
>>> sense to support it, as well.
>>
>> Yes, thank you for working on this, it will be very useful to users like me
>> who use a linux distribution that does not install a cron daemon by default
>> but relies on systemd instead.
>>
>>>> The main motivations to implement systemd timers in addition to cron
>>>> are:
>>>> * cron is optional and Linux systems running systemd might not have it
>>>>     installed.
>>>> * The execution of `crontab -l` can tell us if cron is installed but not
>>>>     if the daemon is actually running.
>>
>> Can we use systemctl to see if it is running (and enabled so we know it will
>> be restarted after a reboot)?
> 
> Not sure if I understand this suggestion.
> However, non-systemd systems doesn't have systemctl command to begin
> with.

I was wondering if on systems with both cron and systemd installed we 
could use systemctl to determine if crond is actually running as Lénaïc 
pointed out that being able to run `crontab -l` does not tell us if 
crond is running.

Best Wishes

Phillip

>>>> * With systemd, each service is run in its own cgroup and its logs are
>>>>     tagged by the service inside journald. With cron, all scheduled tasks
>>>>     are running in the cron daemon cgroup and all the logs of the
>>>>     user-scheduled tasks are pretended to belong to the system cron
>>>>     service.
>>>>     Concretely, a user that doesn’t have access to the system logs won’t
>>>>     have access to the log of its own tasks scheduled by cron whereas he
>>>>     will have access to the log of its own tasks scheduled by systemd
>>>>     timer.
>>>
>>> The last point is somewhat compelling. A potential counterargument is
>>> that `cron` does send email to the user by default if any output is
>>> generated by the cron job. However, it seems quite likely these days
>>> that many systems either won't have local mail service enabled or the
>>> user won't bother checking the local mailbox. It's a minor point, but
>>> if you re-roll it might make sense for the commit message to expand
>>> the last point by saying that although `cron` attempts to send email,
>>> that email may go unseen by the user.
>>>
>>>> In order to schedule git maintenance, we need two unit template files:
>>>> * ~/.config/systemd/user/git-maintenance@.service
>>>>     to define the command to be started by systemd and
>>>> * ~/.config/systemd/user/git-maintenance@.timer
>>>>     to define the schedule at which the command should be run.
> 
> I think it would be better to change ~/.config here to
> $XDG_CONFIG_HOME, as others also points out in another comments.
> 
> [..snip..]
> 
>>>> +`git maintenance start` will overwrite these files and start the timer
>>>> +again with `systemctl --user`, so any customization should be done by
>>>> +creating a drop-in file
>>>> +`~/.config/systemd/user/git-maintenance@.service.d/*.conf`.
> 
> Ditto.
> 
>>> Will `systemd` users generally understand what filename to create in
>>> the "...@.service.d/" directory, and will they know what to populate
>>> the file with? (Genuine question; I've never dealt with that.)
>>
>> I think it would be helpful to explicitly mention the file names (I don't
>> think I could tell you what they are without reading the relevant systemd
>> man page)
> 
> [..snip..]
> 
>>>> +static char *systemd_timer_timer_filename()
>>>> +{
>>>> +       const char *filename = "~/.config/systemd/user/git-maintenance@.timer";
>>>> +       char *expanded = expand_user_path(filename, 0);
>>>> +       if (!expanded)
>>>> +               die(_("failed to expand path '%s'"), filename);
>>>> +
>>>> +       return expanded;
>>>> +}
>>>
>>> I was curious whether this would fail if `.config/systemd/user/`
>>> didn't already exist, but looking at the implementation of
>>> expand_user_path() , I see that it doesn't require the path to already
>>> exist if you pass 0 for the second argument as you do here. Okay.
>>
>> Do we need to worry about $XDG_CONFIG_HOME rather than hard coding
>> "~/.config/". There is a function xdg_config_home() that takes care of this.
> 

^ permalink raw reply	[flat|nested] 138+ messages in thread

* [PATCH v2 0/1] maintenance: use systemd timers on Linux
  2021-05-01 14:52 [PATCH] maintenance: use systemd timers on Linux Lénaïc Huard
                   ` (4 preceding siblings ...)
  2021-05-03 12:04 ` Derrick Stolee
@ 2021-05-09 21:32 ` Lénaïc Huard
  2021-05-09 21:32   ` [PATCH v2 1/1] " Lénaïc Huard
  2021-05-20 22:13   ` [PATCH v3 0/4] " Lénaïc Huard
  5 siblings, 2 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-05-09 21:32 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine, brian m . carlson,
	Bagas Sanjaya, Phillip Wood,
	Đoàn Trần Công Danh,
	Ævar Arnfjörð Bjarmason, Lénaïc Huard

Hello,

Thank you all for your valuable feedback!
I tried to address all the discussed points in this new version of the
patch.
Do not hesitate to let me know if I forgot anything.

The main new thing in this version is the `--scheduler=<scheduler>`
parameter that has been added to `git maintenance start` command. It
allows the end user to choose between `cron` or user systemd timers
for scheduling git maintenance tasks.

I also addressed the migration problematic during an upgrade.
If a user invokes `git maintenance start --scheduler=systemd-timer`
first, and then invokes `git maintenance start --scheduler=cron`
without invoking `git maintenance stop` in between, the git
maintenance tasks will be removed from `systemd-timer` to be sure that
the same tasks won’t be scheduled concurrently twice by both
`systemd-timer` and `cron`. And the same in the other way round.

On its side, `git maintenance stop` don’t have any
`--scheduler=<scheduler>` parameter as it will try to remove the git
maintenance tasks from all the schedulers available on the system.

The default scheduler when `--scheduler=<scheduler>` isn’t specified
is `auto` which means “choose an appropriate scheduler”.

On Windows and MacOS, it always chooses the specific scheduler on
those platforms, `schtasks` and `launchctl`.

On Linux, it chooses user systemd timers if they are available and
`cron` otherwise.
This order has been a subject of discussion so, let me explain why I
chose this order.

On my system, I uses systemd and all the packages of my distribution
that are needing regular scheduled tasks are defining systemd timers
instead of cron task.
So, I don’t use crontab anymore. However, a cron package is installed
because it is an indirect dependency of a package I installed through
my Linux distribution package manager. But the cron daemon isn’t
started.

`systemctl --user list-timers` is the CLI command that actually talks
to the daemon in charge of scheduling timers. So, if `systemctl --user
list-timers` succeed, we can be sure that user systemd timers are
functional.
Concretely, the `is_systemd_timer_available` function of this patch is
reliable.

`crontab`, on the other hand, only reads and writes to
`/var/spool/cron/$USER` but this command can work also if the `cron`
daemon isn’t running. In this case, the scheduled tasks will never
run.
Concretely, the `is_crontab_available` function of this patch is less
reliable.

We would need to check if the `cron` daemon is really running.
Relying on `systemctl` to check if the service is enabled and running
isn’t ideal since we want to support systems that don’t have systemd.
Parsing directly `/proc` is challenging since its content is OS
specific. `/proc` content on a Solaris system is different than its
content on a Linux system.
We could rely on `ps` but we must keep in mind its interface is very
different from one system to another. It doesn’t implement both the
standard syntax and the BSD syntax for its arguments on all platforms
for example.

Moreover, Linux distributions are proposing several different
implementations of `cron`: `cronie`, `fcron`, `dcron`, `vixie-cron`,
`scron`, `bcron`.

See:
* https://wiki.archlinux.org/title/Cron#Installation
* https://wiki.gentoo.org/wiki/Cron#Which_cron_is_right_for_the_job.3F

Depending on the `cron` implementation, the name of the `cron` daemon
process might differ.
So, reliably detecting if a `cron` daemon is running may require to
review each `cron` implementation.

With this new version of the patch, advanced users that care about
systemd timers versus `cron` can explicitly choose which one they want to
use.
For less advanced users that don’t care, I prefer to choose the method
which has the higher probability of working.

But since this order is a subject of debate, what I can propose is a
`./configure` compile time option to select which `cron` or systemd
timers should be chosen in priority if both are available.



In addition to this big change, this new version of the patch also honors
the `XDG_CONFIG_HOME` environment variable and removes the code
duplication between `systemd_timer_timer_filename()` and
`systemd_timer_service_filename()`.

It also fixes other points raised in the code review.

Lénaïc Huard (1):
  maintenance: use systemd timers on Linux

 Documentation/git-maintenance.txt |  60 +++++
 builtin/gc.c                      | 375 ++++++++++++++++++++++++++++--
 t/t7900-maintenance.sh            |  51 ++++
 3 files changed, 462 insertions(+), 24 deletions(-)

-- 
2.31.1


^ permalink raw reply	[flat|nested] 138+ messages in thread

* [PATCH v2 1/1] maintenance: use systemd timers on Linux
  2021-05-09 21:32 ` [PATCH v2 0/1] " Lénaïc Huard
@ 2021-05-09 21:32   ` Lénaïc Huard
  2021-05-10  1:20     ` Đoàn Trần Công Danh
                       ` (4 more replies)
  2021-05-20 22:13   ` [PATCH v3 0/4] " Lénaïc Huard
  1 sibling, 5 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-05-09 21:32 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine, brian m . carlson,
	Bagas Sanjaya, Phillip Wood,
	Đoàn Trần Công Danh,
	Ævar Arnfjörð Bjarmason, Lénaïc Huard

The existing mechanism for scheduling background maintenance is done
through cron. On Linux systems managed by systemd, systemd provides an
alternative to schedule recurring tasks: systemd timers.

The main motivations to implement systemd timers in addition to cron
are:
* cron is optional and Linux systems running systemd might not have it
  installed.
* The execution of `crontab -l` can tell us if cron is installed but not
  if the daemon is actually running.
* With systemd, each service is run in its own cgroup and its logs are
  tagged by the service inside journald. With cron, all scheduled tasks
  are running in the cron daemon cgroup and all the logs of the
  user-scheduled tasks are pretended to belong to the system cron
  service.
  Concretely, a user that doesn’t have access to the system logs won’t
  have access to the log of its own tasks scheduled by cron whereas he
  will have access to the log of its own tasks scheduled by systemd
  timer.
  Although `cron` attempts to send email, that email may go unseen by
  the user because these days, local mailboxes are not heavily used
  anymore.

In order to choose which scheduler to use between `cron` and user
systemd timers, a new option
`--scheduler=auto|cron|systemd|launchctl|schtasks` has been added to
`git maintenance start`.
When `git maintenance start --scheduler=XXX` is run, it not only
registers `git maintenance run` tasks in the scheduler XXX, it also
removes the `git maintenance run` tasks from all the other schedulers to
ensure we cannot have two schedulers launching concurrent identical
tasks.

The default value is `auto` which chooses a suitable scheduler for the
system.
On Linux, if user systemd timers are available, they will be used as git
maintenance scheduler. If not, `cron` will be used if it is available.
If none is available, it will fail.

`git maintenance stop` doesn't have any `--scheduler` parameter because
this command will try to remove the `git maintenance run` tasks from all
the available schedulers.

In order to schedule git maintenance, we need two unit template files:
* ~/.config/systemd/user/git-maintenance@.service
  to define the command to be started by systemd and
* ~/.config/systemd/user/git-maintenance@.timer
  to define the schedule at which the command should be run.

Those units are templates that are parametrized by the frequency.

Based on those templates, 3 timers are started:
* git-maintenance@hourly.timer
* git-maintenance@daily.timer
* git-maintenance@weekly.timer

The command launched by those three timers are the same than with the
other scheduling methods:

git for-each-repo --config=maintenance.repo maintenance run
--schedule=%i

with the full path for git to ensure that the version of git launched
for the scheduled maintenance is the same as the one used to run
`maintenance start`.

The timer unit contains `Persistent=true` so that, if the computer is
powered down when a maintenance task should run, the task will be run
when the computer is back powered on.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 Documentation/git-maintenance.txt |  60 +++++
 builtin/gc.c                      | 375 ++++++++++++++++++++++++++++--
 t/t7900-maintenance.sh            |  51 ++++
 3 files changed, 462 insertions(+), 24 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 80ddd33ceb..f012923333 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -181,6 +181,20 @@ OPTIONS
 	`maintenance.<task>.enabled` configured as `true` are considered.
 	See the 'TASKS' section for the list of accepted `<task>` values.
 
+--scheduler=auto|crontab|systemd-timer|launchctl|schtasks::
+	When combined with the `start` subcommand, specify the scheduler
+	to use to run the hourly, daily and weekly executions of
+	`git maintenance run`.
+	The possible values for `<scheduler>` depend on the system: `crontab`
+	is available on POSIX systems, `systemd-timer` is available on Linux
+	systems, `launchctl` is available on MacOS and `schtasks` is available
+	on Windows.
+	By default or when `auto` is specified, the most appropriate scheduler
+	for the system is used. On MacOS, `launchctl` is used. On Windows,
+	`schtasks` is used. On Linux, `systemd-timers` is used if user systemd
+	timers are available, otherwise, `crontab` is used. On all other systems,
+	`crontab` is used.
+
 
 TROUBLESHOOTING
 ---------------
@@ -279,6 +293,52 @@ schedule to ensure you are executing the correct binaries in your
 schedule.
 
 
+BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMS
+-----------------------------------------------
+
+While Linux supports `cron`, depending on the distribution, `cron` may
+be an optional package not necessarily installed. On modern Linux
+distributions, systemd timers are superseding it.
+
+If user systemd timers are available, they will be used as a replacement
+of `cron`.
+
+In this case, `git maintenance start` will create user systemd timer units
+and start the timers. The current list of user-scheduled tasks can be found
+by running `systemctl --user list-timers`. The timers written by `git
+maintenance start` are similar to this:
+
+-----------------------------------------------------------------------
+$ systemctl --user list-timers
+NEXT                         LEFT          LAST                         PASSED     UNIT                         ACTIVATES
+Thu 2021-04-29 19:00:00 CEST 42min left    Thu 2021-04-29 18:00:11 CEST 17min ago  git-maintenance@hourly.timer git-maintenance@hourly.service
+Fri 2021-04-30 00:00:00 CEST 5h 42min left Thu 2021-04-29 00:00:11 CEST 18h ago    git-maintenance@daily.timer  git-maintenance@daily.service
+Mon 2021-05-03 00:00:00 CEST 3 days left   Mon 2021-04-26 00:00:11 CEST 3 days ago git-maintenance@weekly.timer git-maintenance@weekly.service
+-----------------------------------------------------------------------
+
+One timer is registered for each `--schedule=<frequency>` option.
+
+The definition of the systemd units can be inspected in the following files:
+
+-----------------------------------------------------------------------
+~/.config/systemd/user/git-maintenance@.timer
+~/.config/systemd/user/git-maintenance@.service
+~/.config/systemd/user/timers.target.wants/git-maintenance@hourly.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@daily.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@weekly.timer
+-----------------------------------------------------------------------
+
+`git maintenance start` will overwrite these files and start the timer
+again with `systemctl --user`, so any customization should be done by
+creating a drop-in file, i.e. a `.conf` suffixed file in the
+`~/.config/systemd/user/git-maintenance@.service.d` directory.
+
+`git maintenance stop` will stop the user systemd timers and delete
+the above mentioned files.
+
+For more details, see systemd.timer(5)
+
+
 BACKGROUND MAINTENANCE ON MACOS SYSTEMS
 ---------------------------------------
 
diff --git a/builtin/gc.c b/builtin/gc.c
index ef7226d7bc..7c72aa3b99 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1544,6 +1544,15 @@ static const char *get_frequency(enum schedule_priority schedule)
 	}
 }
 
+static int is_launchctl_available(const char *cmd)
+{
+#ifdef __APPLE__
+	return 1;
+#else
+	return 0;
+#endif
+}
+
 static char *launchctl_service_name(const char *frequency)
 {
 	struct strbuf label = STRBUF_INIT;
@@ -1710,6 +1719,15 @@ static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm
 		return launchctl_remove_plists(cmd);
 }
 
+static int is_schtasks_available(const char *cmd)
+{
+#ifdef GIT_WINDOWS_NATIVE
+	return 1;
+#else
+	return 0;
+#endif
+}
+
 static char *schtasks_task_name(const char *frequency)
 {
 	struct strbuf label = STRBUF_INIT;
@@ -1872,6 +1890,28 @@ static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd
 		return schtasks_remove_tasks(cmd);
 }
 
+static int is_crontab_available(const char *cmd)
+{
+	static int cached_result = -1;
+	struct child_process child = CHILD_PROCESS_INIT;
+
+	if (cached_result != -1)
+		return cached_result;
+
+	strvec_split(&child.args, cmd);
+	strvec_push(&child.args, "-l");
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child))
+		return cached_result = 0;
+	/* Ignore exit code, as an empty crontab will return error. */
+	finish_command(&child);
+	return cached_result = 1;
+}
+
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
@@ -1959,61 +1999,348 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 	return result;
 }
 
+static int is_systemd_timer_available(const char *cmd)
+{
+#ifdef __linux__
+	static int cached_result = -1;
+	struct child_process child = CHILD_PROCESS_INIT;
+
+	if (cached_result != -1)
+		return cached_result;
+
+	strvec_split(&child.args, cmd);
+	strvec_pushl(&child.args, "--user", "list-timers", NULL);
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child))
+		return cached_result = 0;
+	if (finish_command(&child))
+		return cached_result = 0;
+	return cached_result = 1;
+#else
+	return 0;
+#endif
+}
+
+static char *xdg_config_home_systemd(const char *filename)
+{
+	const char *home, *config_home;
+
+	assert(filename);
+	config_home = getenv("XDG_CONFIG_HOME");
+	if (config_home && *config_home)
+		return mkpathdup("%s/systemd/user/%s", config_home, filename);
+
+	home = getenv("HOME");
+	if (home)
+		return mkpathdup("%s/.config/systemd/user/%s", home, filename);
+
+	die(_("failed to get $XDG_CONFIG_HOME and $HOME"));
+}
+
+static int systemd_timer_enable_unit(int enable,
+				     enum schedule_priority schedule,
+				     const char *cmd)
+{
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *frequency = get_frequency(schedule);
+
+	strvec_split(&child.args, cmd);
+	strvec_pushl(&child.args, "--user", enable ? "enable" : "disable",
+		     "--now", NULL);
+	strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency);
+
+	if (start_command(&child))
+		die(_("failed to run systemctl"));
+	return finish_command(&child);
+}
+
+static int systemd_timer_delete_unit_templates(void)
+{
+	char *filename = xdg_config_home_systemd("git-maintenance@.timer");
+	unlink(filename);
+	free(filename);
+
+	filename = xdg_config_home_systemd("git-maintenance@.service");
+	unlink(filename);
+	free(filename);
+
+	return 0;
+}
+
+static int systemd_timer_delete_units(const char *cmd)
+{
+	return systemd_timer_enable_unit(0, SCHEDULE_HOURLY, cmd) ||
+	       systemd_timer_enable_unit(0, SCHEDULE_DAILY, cmd) ||
+	       systemd_timer_enable_unit(0, SCHEDULE_WEEKLY, cmd) ||
+	       systemd_timer_delete_unit_templates();
+}
+
+static int systemd_timer_write_unit_templates(const char *exec_path)
+{
+	char *filename;
+	FILE *file;
+	const char *unit;
+
+	filename = xdg_config_home_systemd("git-maintenance@.timer");
+	if (safe_create_leading_directories(filename))
+		die(_("failed to create directories for '%s'"), filename);
+	file = xfopen(filename, "w");
+	free(filename);
+
+	unit = "# This file was created and is maintained by Git.\n"
+	       "# Any edits made in this file might be replaced in the future\n"
+	       "# by a Git command.\n"
+	       "\n"
+	       "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Timer]\n"
+	       "OnCalendar=%i\n"
+	       "Persistent=true\n"
+	       "\n"
+	       "[Install]\n"
+	       "WantedBy=timers.target\n";
+	fputs(unit, file);
+	fclose(file);
+
+	filename = xdg_config_home_systemd("git-maintenance@.service");
+	if (safe_create_leading_directories(filename))
+		die(_("failed to create directories for '%s'"), filename);
+	file = xfopen(filename, "w");
+	free(filename);
+
+	unit = "# This file was created and is maintained by Git.\n"
+	       "# Any edits made in this file might be replaced in the future\n"
+	       "# by a Git command.\n"
+	       "\n"
+	       "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Service]\n"
+	       "Type=oneshot\n"
+	       "ExecStart=\"%1$s/git\" --exec-path=\"%1$s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
+	       "LockPersonality=yes\n"
+	       "MemoryDenyWriteExecute=yes\n"
+	       "NoNewPrivileges=yes\n"
+	       "RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\n"
+	       "RestrictNamespaces=yes\n"
+	       "RestrictRealtime=yes\n"
+	       "RestrictSUIDSGID=yes\n"
+	       "SystemCallArchitectures=native\n"
+	       "SystemCallFilter=@system-service\n";
+	fprintf(file, unit, exec_path);
+	fclose(file);
+
+	return 0;
+}
+
+static int systemd_timer_setup_units(const char *cmd)
+{
+	const char *exec_path = git_exec_path();
+
+	return systemd_timer_write_unit_templates(exec_path) ||
+	       systemd_timer_enable_unit(1, SCHEDULE_HOURLY, cmd) ||
+	       systemd_timer_enable_unit(1, SCHEDULE_DAILY, cmd) ||
+	       systemd_timer_enable_unit(1, SCHEDULE_WEEKLY, cmd);
+}
+
+static int systemd_timer_update_schedule(int run_maintenance, int fd,
+					 const char *cmd)
+{
+	if (run_maintenance)
+		return systemd_timer_setup_units(cmd);
+	else
+		return systemd_timer_delete_units(cmd);
+}
+
+enum scheduler {
+	SCHEDULER_INVALID = -1,
+	SCHEDULER_AUTO = 0,
+	SCHEDULER_CRON = 1,
+	SCHEDULER_SYSTEMD = 2,
+	SCHEDULER_LAUNCHCTL = 3,
+	SCHEDULER_SCHTASKS = 4,
+};
+
+static const struct {
+	int (*is_available)(const char *cmd);
+	int (*update_schedule)(int run_maintenance, int fd, const char *cmd);
+	const char *cmd;
+} scheduler_fn[] = {
+	[SCHEDULER_CRON] = { is_crontab_available, crontab_update_schedule,
+			     "crontab" },
+	[SCHEDULER_SYSTEMD] = { is_systemd_timer_available,
+				systemd_timer_update_schedule, "systemctl" },
+	[SCHEDULER_LAUNCHCTL] = { is_launchctl_available,
+				  launchctl_update_schedule, "launchctl" },
+	[SCHEDULER_SCHTASKS] = { is_schtasks_available,
+				 schtasks_update_schedule, "schtasks" },
+};
+
+static enum scheduler parse_scheduler(const char *value)
+{
+	if (!value)
+		return SCHEDULER_INVALID;
+	else if (!strcasecmp(value, "auto"))
+		return SCHEDULER_AUTO;
+	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
+		return SCHEDULER_CRON;
+	else if (!strcasecmp(value, "systemd") ||
+		 !strcasecmp(value, "systemd-timer"))
+		return SCHEDULER_SYSTEMD;
+	else if (!strcasecmp(value, "launchctl"))
+		return SCHEDULER_LAUNCHCTL;
+	else if (!strcasecmp(value, "schtasks"))
+		return SCHEDULER_SCHTASKS;
+	else
+		return SCHEDULER_INVALID;
+}
+
+static int maintenance_opt_scheduler(const struct option *opt, const char *arg,
+				     int unset)
+{
+	enum scheduler *scheduler = opt->value;
+
+	if (unset)
+		die(_("--no-scheduler is not allowed"));
+
+	*scheduler = parse_scheduler(arg);
+
+	if (*scheduler == SCHEDULER_INVALID)
+		die(_("unrecognized --scheduler argument '%s'"), arg);
+
+	return 0;
+}
+
+struct maintenance_start_opts {
+	enum scheduler scheduler;
+};
+
+static void resolve_auto_scheduler(enum scheduler *scheduler)
+{
+	if (*scheduler != SCHEDULER_AUTO)
+		return;
+
 #if defined(__APPLE__)
-static const char platform_scheduler[] = "launchctl";
+	*scheduler = SCHEDULER_LAUNCHCTL;
+	return;
+
 #elif defined(GIT_WINDOWS_NATIVE)
-static const char platform_scheduler[] = "schtasks";
+	*scheduler = SCHEDULER_SCHTASKS;
+	return;
+
+#elif defined(__linux__)
+	if (is_systemd_timer_available("systemctl"))
+		*scheduler = SCHEDULER_SYSTEMD;
+	else if (is_crontab_available("crontab"))
+		*scheduler = SCHEDULER_CRON;
+	else
+		die(_("neither systemd timers nor crontab are available"));
+	return;
+
 #else
-static const char platform_scheduler[] = "crontab";
+	*scheduler = SCHEDULER_CRON;
+	return;
 #endif
+}
 
-static int update_background_schedule(int enable)
+static void validate_scheduler(enum scheduler scheduler)
 {
-	int result;
-	const char *scheduler = platform_scheduler;
-	const char *cmd = scheduler;
+	const char *cmd;
+
+	if (scheduler == SCHEDULER_INVALID)
+		BUG("invalid scheduler");
+	if (scheduler == SCHEDULER_AUTO)
+		BUG("resolve_auto_scheduler should have been called before");
+
+	cmd = scheduler_fn[scheduler].cmd;
+	if (!scheduler_fn[scheduler].is_available(cmd))
+		die(_("%s scheduler is not available"), cmd);
+}
+
+static int update_background_schedule(const struct maintenance_start_opts *opts,
+				      int enable)
+{
+	unsigned int i;
+	int res, result = 0;
+	enum scheduler scheduler;
+	const char *cmd = NULL;
 	char *testing;
 	struct lock_file lk;
 	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
 
+	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
+		return error(_("another process is scheduling background maintenance"));
+
 	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
 	if (testing) {
 		char *sep = strchr(testing, ':');
 		if (!sep)
 			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
 		*sep = '\0';
-		scheduler = testing;
+		scheduler = parse_scheduler(testing);
 		cmd = sep + 1;
+		result = scheduler_fn[scheduler].update_schedule(
+			enable, get_lock_file_fd(&lk), cmd);
+		goto done;
 	}
 
-	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
-		return error(_("another process is scheduling background maintenance"));
-
-	if (!strcmp(scheduler, "launchctl"))
-		result = launchctl_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else if (!strcmp(scheduler, "schtasks"))
-		result = schtasks_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else if (!strcmp(scheduler, "crontab"))
-		result = crontab_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else
-		die("unknown background scheduler: %s", scheduler);
+	for (i = 1; i < ARRAY_SIZE(scheduler_fn); i++) {
+		int enable_scheduler = enable && (opts->scheduler == i);
+		cmd = scheduler_fn[i].cmd;
+		if (!scheduler_fn[i].is_available(cmd))
+			continue;
+		res = scheduler_fn[i].update_schedule(
+			enable_scheduler, get_lock_file_fd(&lk), cmd);
+		if (enable_scheduler)
+			result = res;
+	}
 
+done:
 	rollback_lock_file(&lk);
 	free(testing);
 	return result;
 }
 
-static int maintenance_start(void)
+static const char *const builtin_maintenance_start_usage[] = {
+	N_("git maintenance start [--scheduler=<scheduler>]"), NULL
+};
+
+static int maintenance_start(int argc, const char **argv, const char *prefix)
 {
+	struct maintenance_start_opts opts;
+	struct option builtin_maintenance_start_options[] = {
+		OPT_CALLBACK(
+			0, "scheduler", &opts.scheduler, N_("scheduler"),
+			N_("scheduler to use to trigger git maintenance run"),
+			maintenance_opt_scheduler),
+		OPT_END()
+	};
+	memset(&opts, 0, sizeof(opts));
+
+	argc = parse_options(argc, argv, prefix,
+			     builtin_maintenance_start_options,
+			     builtin_maintenance_start_usage, 0);
+
+	resolve_auto_scheduler(&opts.scheduler);
+	validate_scheduler(opts.scheduler);
+
+	if (argc > 0)
+		usage_with_options(builtin_maintenance_start_usage,
+				   builtin_maintenance_start_options);
+
 	if (maintenance_register())
 		warning(_("failed to add repo to global config"));
-
-	return update_background_schedule(1);
+	return update_background_schedule(&opts, 1);
 }
 
 static int maintenance_stop(void)
 {
-	return update_background_schedule(0);
+	return update_background_schedule(NULL, 0);
 }
 
 static const char builtin_maintenance_usage[] =	N_("git maintenance <subcommand> [<options>]");
@@ -2027,7 +2354,7 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
 	if (!strcmp(argv[1], "run"))
 		return maintenance_run(argc - 1, argv + 1, prefix);
 	if (!strcmp(argv[1], "start"))
-		return maintenance_start();
+		return maintenance_start(argc - 1, argv + 1, prefix);
 	if (!strcmp(argv[1], "stop"))
 		return maintenance_stop();
 	if (!strcmp(argv[1], "register"))
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 2412d8c5c0..6e6316cd90 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -20,6 +20,20 @@ test_xmllint () {
 	fi
 }
 
+test_lazy_prereq SYSTEMD_ANALYZE '
+	systemd-analyze --help >out &&
+	grep verify out
+'
+
+test_systemd_analyze_verify () {
+	if test_have_prereq SYSTEMD_ANALYZE
+	then
+		systemd-analyze verify "$@"
+	else
+		true
+	fi
+}
+
 test_expect_success 'help text' '
 	test_expect_code 129 git maintenance -h 2>err &&
 	test_i18ngrep "usage: git maintenance <subcommand>" err &&
@@ -615,6 +629,43 @@ test_expect_success 'start and stop Windows maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success 'start and stop Linux/systemd maintenance' '
+	write_script print-args <<-\EOF &&
+	echo $* >>args
+	EOF
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemd:./print-args" git maintenance start &&
+
+	# start registers the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_systemd_analyze_verify "$HOME/.config/systemd/user/git-maintenance@.service" &&
+
+	rm -f expect &&
+	for frequency in hourly daily weekly
+	do
+		echo "--user enable --now git-maintenance@${frequency}.timer" >>expect || return 1
+	done &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemd:./print-args" git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_path_is_missing "$HOME/.config/systemd/user/git-maintenance@.timer" &&
+	test_path_is_missing "$HOME/.config/systemd/user/git-maintenance@.service" &&
+
+	rm -f expect &&
+	for frequency in hourly daily weekly
+	do
+		echo "--user disable --now git-maintenance@${frequency}.timer" >>expect || return 1
+	done &&
+	test_cmp expect args
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
2.31.1


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* Re: [PATCH] maintenance: use systemd timers on Linux
  2021-05-05 12:01   ` Ævar Arnfjörð Bjarmason
@ 2021-05-09 22:34     ` Lénaïc Huard
  2021-05-10 13:03       ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 138+ messages in thread
From: Lénaïc Huard @ 2021-05-09 22:34 UTC (permalink / raw)
  To: Eric Sunshine, Ævar Arnfjörð Bjarmason
  Cc: Git List, Junio C Hamano, Derrick Stolee, brian m. carlson

Le mercredi 5 mai 2021, 14:01:25 CEST Ævar Arnfjörð Bjarmason a écrit :
> On Sun, May 02 2021, Eric Sunshine wrote:
> > On Sat, May 1, 2021 at 10:59 AM Lénaïc Huard <lenaic@lhuard.fr> wrote:
> >> +       strvec_push(&child.args, "--now");
> >> +       strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency);
> >> +
> >> +       if (start_command(&child))
> >> +               die(_("failed to run systemctl"));
> >> +       return finish_command(&child);
> >> +}
> >> +static int systemd_timer_write_unit_templates(const char *exec_path)
> >> +{
> >> +       unit = "[Unit]\n"
> >> +              "Description=Optimize Git repositories data\n"
> >> +              "\n"
> >> +              "[Service]\n"
> >> +              "Type=oneshot\n"
> >> +              "ExecStart=\"%1$s/git\" --exec-path=\"%1$s\" for-each-repo
> >> --config=maintenance.repo maintenance run --schedule=%%i\n"> 
> > I see that it's in POSIX, but do we use this `%n$s` directive
> > elsewhere in the Git source code? If not, I'd be cautious of
> > introducing it here. Maybe it's better to just use plain `%s` twice...
> 
> We use it in po/, so for sprintf() on systems that don't have
> NO_GETTEXT=Y we already test it in the wild.
> 
> But no, I don't think anything in the main source uses it, FWIW I have a
> WIP series in my own fork that I've cooked for a while that uses this, I
> haven't run into any issues with it in either GitHub's CI
> (e.g. Windows), or on the systems I myself test on.
> 
> I think it would be a useful canary to just take a change like this, we
> can always change it to the form you suggest if it doesn't work out.

Based on this latest comment, I left the `%n$s` directive in the v2 of the 
patch.

Let me know if that’s still OK. Otherwise, I’d be happy to implement Eric’s 
suggestion.

Note however that this would be a “poor” canary to check if that directive is 
supported on all the platforms on which git has been ported.
Indeed, this code is executed only on systemd platforms, which means quite 
recent Linux systems.
Should this directive not be supported, I suppose it would be on more exotic 
systems.



^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v2 1/1] maintenance: use systemd timers on Linux
  2021-05-09 21:32   ` [PATCH v2 1/1] " Lénaïc Huard
@ 2021-05-10  1:20     ` Đoàn Trần Công Danh
  2021-05-10  2:48       ` Eric Sunshine
  2021-05-10 18:03     ` Phillip Wood
                       ` (3 subsequent siblings)
  4 siblings, 1 reply; 138+ messages in thread
From: Đoàn Trần Công Danh @ 2021-05-10  1:20 UTC (permalink / raw)
  To: Lénaïc Huard
  Cc: git, Junio C Hamano, Derrick Stolee, Eric Sunshine,
	brian m . carlson, Bagas Sanjaya, Phillip Wood,
	Ævar Arnfjörð Bjarmason

On 2021-05-09 23:32:17+0200, Lénaïc Huard <lenaic@lhuard.fr> wrote:
> The existing mechanism for scheduling background maintenance is done
> through cron. On Linux systems managed by systemd, systemd provides an
> alternative to schedule recurring tasks: systemd timers.
> 
> +static int systemd_timer_enable_unit(int enable,
> +				     enum schedule_priority schedule,
> +				     const char *cmd)
> +{
> +	struct child_process child = CHILD_PROCESS_INIT;
> +	const char *frequency = get_frequency(schedule);
> +
> +	strvec_split(&child.args, cmd);
> +	strvec_pushl(&child.args, "--user", enable ? "enable" : "disable",
> +		     "--now", NULL);
> +	strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency);
> +
> +	if (start_command(&child))
> +		die(_("failed to run systemctl"));
> +	return finish_command(&child);
> +}
> +
> +static int systemd_timer_delete_unit_templates(void)
> +{
> +	char *filename = xdg_config_home_systemd("git-maintenance@.timer");
> +	unlink(filename);
> +	free(filename);
> +
> +	filename = xdg_config_home_systemd("git-maintenance@.service");
> +	unlink(filename);
> +	free(filename);
> +
> +	return 0;
> +}
> +
> +static int systemd_timer_delete_units(const char *cmd)
> +{
> +	return systemd_timer_enable_unit(0, SCHEDULE_HOURLY, cmd) ||
> +	       systemd_timer_enable_unit(0, SCHEDULE_DAILY, cmd) ||
> +	       systemd_timer_enable_unit(0, SCHEDULE_WEEKLY, cmd) ||
> +	       systemd_timer_delete_unit_templates();
> +}

I'm not using any systemd-based distros. However, isn't this try to
enable all systemd's {hourly,daily,weekly} user's timer, then delete
the templates?^W^W^W^W^W^W^W^W^W^W^W^W^W^W^W^W^W^W^W^W^W^W^W^W^W^W^W

Argh, we're disabling those systemd timer units first, by passing 0 as
first argument of systemd_timer_delete_units.

The fact that I read that twice, and still wrote down above reply
makes me think that above code is not self-explanatory enough.
May we switch to something else? Let's say using enum?


> +static int systemd_timer_write_unit_templates(const char *exec_path)
> +{
> +	char *filename;
> +	FILE *file;
> +	const char *unit;
> +
> +	filename = xdg_config_home_systemd("git-maintenance@.timer");
> +	if (safe_create_leading_directories(filename))
> +		die(_("failed to create directories for '%s'"), filename);

This message is used by other codes, less works for translator, nice!

> +	file = xfopen(filename, "w");
> +	free(filename);

I'm sure if we should use FREE_AND_NULL(filename) instead?
Since, filename will be reused later.

> +
> +	unit = "# This file was created and is maintained by Git.\n"
> +	       "# Any edits made in this file might be replaced in the future\n"
> +	       "# by a Git command.\n"
> +	       "\n"
> +	       "[Unit]\n"
> +	       "Description=Optimize Git repositories data\n"
> +	       "\n"
> +	       "[Timer]\n"
> +	       "OnCalendar=%i\n"
> +	       "Persistent=true\n"
> +	       "\n"
> +	       "[Install]\n"
> +	       "WantedBy=timers.target\n";
> +	fputs(unit, file);
> +	fclose(file);
> +
> +	filename = xdg_config_home_systemd("git-maintenance@.service");
> +	if (safe_create_leading_directories(filename))
> +		die(_("failed to create directories for '%s'"), filename);
> +	file = xfopen(filename, "w");
> +	free(filename);
> +
> +	unit = "# This file was created and is maintained by Git.\n"
> +	       "# Any edits made in this file might be replaced in the future\n"
> +	       "# by a Git command.\n"
> +	       "\n"
> +	       "[Unit]\n"
> +	       "Description=Optimize Git repositories data\n"
> +	       "\n"
> +	       "[Service]\n"
> +	       "Type=oneshot\n"
> +	       "ExecStart=\"%1$s/git\" --exec-path=\"%1$s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
> +	       "LockPersonality=yes\n"
> +	       "MemoryDenyWriteExecute=yes\n"
> +	       "NoNewPrivileges=yes\n"
> +	       "RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\n"
> +	       "RestrictNamespaces=yes\n"
> +	       "RestrictRealtime=yes\n"
> +	       "RestrictSUIDSGID=yes\n"
> +	       "SystemCallArchitectures=native\n"
> +	       "SystemCallFilter=@system-service\n";
> +	fprintf(file, unit, exec_path);

I think others have strong opinion on not using "%1$s",
and prefer simple "%s" and using "exec_path" twice instead.

> +	fclose(file);
> +
> +	return 0;
> +}
> +
> +static int systemd_timer_setup_units(const char *cmd)
> +{
> +	const char *exec_path = git_exec_path();
> +
> +	return systemd_timer_write_unit_templates(exec_path) ||
> +	       systemd_timer_enable_unit(1, SCHEDULE_HOURLY, cmd) ||
> +	       systemd_timer_enable_unit(1, SCHEDULE_DAILY, cmd) ||
> +	       systemd_timer_enable_unit(1, SCHEDULE_WEEKLY, cmd);
> +}
> +
> +static int systemd_timer_update_schedule(int run_maintenance, int fd,
> +					 const char *cmd)
> +{
> +	if (run_maintenance)
> +		return systemd_timer_setup_units(cmd);
> +	else
> +		return systemd_timer_delete_units(cmd);
> +}
> +
> +enum scheduler {
> +	SCHEDULER_INVALID = -1,
> +	SCHEDULER_AUTO = 0,
> +	SCHEDULER_CRON = 1,
> +	SCHEDULER_SYSTEMD = 2,
> +	SCHEDULER_LAUNCHCTL = 3,
> +	SCHEDULER_SCHTASKS = 4,

I think explicitly writing down values doesn't make things clearer,
-1 would be nice, not a strong opinion, though.

Anyway, would it be better to move those type declaration to top of
file?

> +};
> +
> +static const struct {
> +	int (*is_available)(const char *cmd);
> +	int (*update_schedule)(int run_maintenance, int fd, const char *cmd);
> +	const char *cmd;
> +} scheduler_fn[] = {
> +	[SCHEDULER_CRON] = { is_crontab_available, crontab_update_schedule,
> +			     "crontab" },
> +	[SCHEDULER_SYSTEMD] = { is_systemd_timer_available,
> +				systemd_timer_update_schedule, "systemctl" },
> +	[SCHEDULER_LAUNCHCTL] = { is_launchctl_available,
> +				  launchctl_update_schedule, "launchctl" },
> +	[SCHEDULER_SCHTASKS] = { is_schtasks_available,
> +				 schtasks_update_schedule, "schtasks" },
> +};
> +
> +static enum scheduler parse_scheduler(const char *value)
> +{
> +	if (!value)
> +		return SCHEDULER_INVALID;
> +	else if (!strcasecmp(value, "auto"))
> +		return SCHEDULER_AUTO;
> +	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
> +		return SCHEDULER_CRON;
> +	else if (!strcasecmp(value, "systemd") ||
> +		 !strcasecmp(value, "systemd-timer"))
> +		return SCHEDULER_SYSTEMD;
> +	else if (!strcasecmp(value, "launchctl"))
> +		return SCHEDULER_LAUNCHCTL;
> +	else if (!strcasecmp(value, "schtasks"))
> +		return SCHEDULER_SCHTASKS;
> +	else
> +		return SCHEDULER_INVALID;
> +}
> +
> +static int maintenance_opt_scheduler(const struct option *opt, const char *arg,
> +				     int unset)
> +{
> +	enum scheduler *scheduler = opt->value;
> +
> +	if (unset)
> +		die(_("--no-scheduler is not allowed"));

I think it's better to use OPT_CALLBACK_F in the options list and
we will write below code instead:

	BUG_ON_OPT_NEG(unset)

> +
> +	*scheduler = parse_scheduler(arg);
> +
> +	if (*scheduler == SCHEDULER_INVALID)
> +		die(_("unrecognized --scheduler argument '%s'"), arg);

Most of other callbacks do this instead:

	return error(_("messsage.... '%s'"), arg);

> +
> +	return 0;
> +}
> +
> +struct maintenance_start_opts {
> +	enum scheduler scheduler;
> +};
> +
> +static void resolve_auto_scheduler(enum scheduler *scheduler)
> +{
> +	if (*scheduler != SCHEDULER_AUTO)
> +		return;
> +
>  #if defined(__APPLE__)
> -static const char platform_scheduler[] = "launchctl";
> +	*scheduler = SCHEDULER_LAUNCHCTL;
> +	return;
> +
>  #elif defined(GIT_WINDOWS_NATIVE)
> -static const char platform_scheduler[] = "schtasks";
> +	*scheduler = SCHEDULER_SCHTASKS;
> +	return;
> +
> +#elif defined(__linux__)
> +	if (is_systemd_timer_available("systemctl"))
> +		*scheduler = SCHEDULER_SYSTEMD;
> +	else if (is_crontab_available("crontab"))
> +		*scheduler = SCHEDULER_CRON;
> +	else
> +		die(_("neither systemd timers nor crontab are available"));
> +	return;
> +
>  #else
> -static const char platform_scheduler[] = "crontab";
> +	*scheduler = SCHEDULER_CRON;
> +	return;
>  #endif
> +}
>  
> -static int update_background_schedule(int enable)
> +static void validate_scheduler(enum scheduler scheduler)
>  {
> -	int result;
> -	const char *scheduler = platform_scheduler;
> -	const char *cmd = scheduler;
> +	const char *cmd;
> +
> +	if (scheduler == SCHEDULER_INVALID)
> +		BUG("invalid scheduler");
> +	if (scheduler == SCHEDULER_AUTO)
> +		BUG("resolve_auto_scheduler should have been called before");
> +
> +	cmd = scheduler_fn[scheduler].cmd;
> +	if (!scheduler_fn[scheduler].is_available(cmd))
> +		die(_("%s scheduler is not available"), cmd);
> +}
> +
> +static int update_background_schedule(const struct maintenance_start_opts *opts,
> +				      int enable)
> +{
> +	unsigned int i;
> +	int res, result = 0;
> +	enum scheduler scheduler;
> +	const char *cmd = NULL;
>  	char *testing;
>  	struct lock_file lk;
>  	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
>  
> +	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
> +		return error(_("another process is scheduling background maintenance"));
> +
>  	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
>  	if (testing) {
>  		char *sep = strchr(testing, ':');
>  		if (!sep)
>  			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
>  		*sep = '\0';
> -		scheduler = testing;
> +		scheduler = parse_scheduler(testing);
>  		cmd = sep + 1;
> +		result = scheduler_fn[scheduler].update_schedule(
> +			enable, get_lock_file_fd(&lk), cmd);
> +		goto done;
>  	}
>  
> -	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
> -		return error(_("another process is scheduling background maintenance"));
> -
> -	if (!strcmp(scheduler, "launchctl"))
> -		result = launchctl_update_schedule(enable, get_lock_file_fd(&lk), cmd);
> -	else if (!strcmp(scheduler, "schtasks"))
> -		result = schtasks_update_schedule(enable, get_lock_file_fd(&lk), cmd);
> -	else if (!strcmp(scheduler, "crontab"))
> -		result = crontab_update_schedule(enable, get_lock_file_fd(&lk), cmd);
> -	else
> -		die("unknown background scheduler: %s", scheduler);
> +	for (i = 1; i < ARRAY_SIZE(scheduler_fn); i++) {
> +		int enable_scheduler = enable && (opts->scheduler == i);
> +		cmd = scheduler_fn[i].cmd;
> +		if (!scheduler_fn[i].is_available(cmd))
> +			continue;
> +		res = scheduler_fn[i].update_schedule(
> +			enable_scheduler, get_lock_file_fd(&lk), cmd);
> +		if (enable_scheduler)
> +			result = res;
> +	}
>  
> +done:
>  	rollback_lock_file(&lk);
>  	free(testing);
>  	return result;
>  }
>  
> -static int maintenance_start(void)
> +static const char *const builtin_maintenance_start_usage[] = {
> +	N_("git maintenance start [--scheduler=<scheduler>]"), NULL
> +};
> +
> +static int maintenance_start(int argc, const char **argv, const char *prefix)
>  {
> +	struct maintenance_start_opts opts;
> +	struct option builtin_maintenance_start_options[] = {
> +		OPT_CALLBACK(
> +			0, "scheduler", &opts.scheduler, N_("scheduler"),
> +			N_("scheduler to use to trigger git maintenance run"),
> +			maintenance_opt_scheduler),

Following up my comment above, we're better to use:

		OPT_CALLBACK_F(0, "scheduler", &opts.scheduler, N_("scheduler"),
			N_("............"),
			PARSE_OPT_NONEG, maintenance_opt_scheduler),

> +		OPT_END()
> +	};
> +	memset(&opts, 0, sizeof(opts));
> +
> +	argc = parse_options(argc, argv, prefix,
> +			     builtin_maintenance_start_options,
> +			     builtin_maintenance_start_usage, 0);
> +
> +	resolve_auto_scheduler(&opts.scheduler);
> +	validate_scheduler(opts.scheduler);
> +
> +	if (argc > 0)
> +		usage_with_options(builtin_maintenance_start_usage,
> +				   builtin_maintenance_start_options);
> +
>  	if (maintenance_register())
>  		warning(_("failed to add repo to global config"));
> -
> -	return update_background_schedule(1);
> +	return update_background_schedule(&opts, 1);
>  }
>  
>  static int maintenance_stop(void)
>  {
> -	return update_background_schedule(0);
> +	return update_background_schedule(NULL, 0);
>  }
>  
>  static const char builtin_maintenance_usage[] =	N_("git maintenance <subcommand> [<options>]");
> @@ -2027,7 +2354,7 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
>  	if (!strcmp(argv[1], "run"))
>  		return maintenance_run(argc - 1, argv + 1, prefix);
>  	if (!strcmp(argv[1], "start"))
> -		return maintenance_start();
> +		return maintenance_start(argc - 1, argv + 1, prefix);
>  	if (!strcmp(argv[1], "stop"))
>  		return maintenance_stop();
>  	if (!strcmp(argv[1], "register"))
> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
> index 2412d8c5c0..6e6316cd90 100755
> --- a/t/t7900-maintenance.sh
> +++ b/t/t7900-maintenance.sh
> @@ -20,6 +20,20 @@ test_xmllint () {
>  	fi
>  }
>  
> +test_lazy_prereq SYSTEMD_ANALYZE '
> +	systemd-analyze --help >out &&
> +	grep verify out
> +'
> +
> +test_systemd_analyze_verify () {
> +	if test_have_prereq SYSTEMD_ANALYZE
> +	then
> +		systemd-analyze verify "$@"
> +	else
> +		true

The "else" leg is not necessary.

> +	fi
> +}
> +
>  test_expect_success 'help text' '
>  	test_expect_code 129 git maintenance -h 2>err &&
>  	test_i18ngrep "usage: git maintenance <subcommand>" err &&
> @@ -615,6 +629,43 @@ test_expect_success 'start and stop Windows maintenance' '
>  	test_cmp expect args
>  '
>  
> +test_expect_success 'start and stop Linux/systemd maintenance' '
> +	write_script print-args <<-\EOF &&
> +	echo $* >>args

To avoid any possible incompatibility with zillion echo implementation
out there. printf should be prefered over echo. Not a in this test
case, however, it costs us nothing anyway.

	printf "%s\n" "$*"

> +	EOF
> +
> +	rm -f args &&
> +	GIT_TEST_MAINT_SCHEDULER="systemd:./print-args" git maintenance start &&
> +
> +	# start registers the repo
> +	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
> +
> +	test_systemd_analyze_verify "$HOME/.config/systemd/user/git-maintenance@.service" &&
> +
> +	rm -f expect &&
> +	for frequency in hourly daily weekly
> +	do
> +		echo "--user enable --now git-maintenance@${frequency}.timer" >>expect || return 1
> +	done &&

And here, we can have a nicer syntax with printf:

	printf "--user enable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&

With printf, we don't even need "rm -f expect" above.

> +	test_cmp expect args &&
> +
> +	rm -f args &&
> +	GIT_TEST_MAINT_SCHEDULER="systemd:./print-args" git maintenance stop &&
> +
> +	# stop does not unregister the repo
> +	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
> +
> +	test_path_is_missing "$HOME/.config/systemd/user/git-maintenance@.timer" &&
> +	test_path_is_missing "$HOME/.config/systemd/user/git-maintenance@.service" &&
> +
> +	rm -f expect &&
> +	for frequency in hourly daily weekly
> +	do
> +		echo "--user disable --now git-maintenance@${frequency}.timer" >>expect || return 1
> +	done &&

Ditto.

All of this was written without testing, because I don't have any
systemd based system near my hand, right now.

So, please take it with a grain of salt.


-- 
Danh

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v2 1/1] maintenance: use systemd timers on Linux
  2021-05-10  1:20     ` Đoàn Trần Công Danh
@ 2021-05-10  2:48       ` Eric Sunshine
  2021-05-10  6:25         ` Junio C Hamano
  0 siblings, 1 reply; 138+ messages in thread
From: Eric Sunshine @ 2021-05-10  2:48 UTC (permalink / raw)
  To: Đoàn Trần Công Danh
  Cc: Lénaïc Huard, Git List, Junio C Hamano, Derrick Stolee,
	brian m . carlson, Bagas Sanjaya, Phillip Wood,
	Ævar Arnfjörð Bjarmason

On Sun, May 9, 2021 at 9:20 PM Đoàn Trần Công Danh <congdanhqx@gmail.com> wrote:
> On 2021-05-09 23:32:17+0200, Lénaïc Huard <lenaic@lhuard.fr> wrote:
> > +static int systemd_timer_delete_units(const char *cmd)
> > +{
> > +     return systemd_timer_enable_unit(0, SCHEDULE_HOURLY, cmd) ||
> > +            systemd_timer_enable_unit(0, SCHEDULE_DAILY, cmd) ||
> > +            systemd_timer_enable_unit(0, SCHEDULE_WEEKLY, cmd) ||
> > +            systemd_timer_delete_unit_templates();
> > +}
>
> Argh, we're disabling those systemd timer units first, by passing 0 as
> first argument of systemd_timer_delete_units.
>
> The fact that I read that twice, and still wrote down above reply
> makes me think that above code is not self-explanatory enough.
> May we switch to something else? Let's say using enum?

This is modeled after existing scheduler functions in this file, in
which the `enable` argument is a simple 0 or 1, so changing this to an
enum just for this function would be inconsistent. Changing all the
functions to `enum` in a preparatory patch could indeed improve
readability, however, that's tangential cleanup which may be outside
the scope of this submission.

> > +     file = xfopen(filename, "w");
> > +     free(filename);
>
> I'm sure if we should use FREE_AND_NULL(filename) instead?
> Since, filename will be reused later.

Indeed, probably a good idea, as it would make catching mistakes easier.

> > +            "ExecStart=\"%1$s/git\" --exec-path=\"%1$s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
>
> I think others have strong opinion on not using "%1$s",
> and prefer simple "%s" and using "exec_path" twice instead.

I brought it up only because I hadn't seen it in Git sources, and
wasn't sure if we'd want to start using it. Aside from Ævar, who
seemed reasonably in favor of it, nobody else chimed in, so it could
go either way, I suppose.

> > +test_systemd_analyze_verify () {
> > +     if test_have_prereq SYSTEMD_ANALYZE
> > +     then
> > +             systemd-analyze verify "$@"
> > +     else
> > +             true
>
> The "else" leg is not necessary.

This was patterned after the existing test_xmllint() function in this
file which has the `else true` leg. I wrote test_xmllint(), so I'll
take blame. But you're right, the `if` will return success if the
prerequisite is not set, so the `else` leg is indeed not needed.
(Cleaning up the test_xmllint() function is outside the scope of this
patch.)

> > +test_expect_success 'start and stop Linux/systemd maintenance' '
> > +     write_script print-args <<-\EOF &&
> > +     echo $* >>args
>
> To avoid any possible incompatibility with zillion echo implementation
> out there. printf should be prefered over echo. Not a in this test
> case, however, it costs us nothing anyway.
>
>         printf "%s\n" "$*"

This, too, is patterned after existing auxiliary scripts created by
these test functions, and you're correct that it's potentially
dangerous. A manual inspection of all the existing instances shows
that `echo $*` happens to be safe for those cases but that doesn't
excuse being sloppy about it, so the existing cases probably ought to
be cleaned up. But, again, that is outside the scope of this series.
For this particular case, though...

> > +     for frequency in hourly daily weekly
> > +     do
> > +             echo "--user enable --now git-maintenance@${frequency}.timer" >>expect || return 1
> > +     done &&
>
> And here, we can have a nicer syntax with printf:
>
>         printf "--user enable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
>
> With printf, we don't even need "rm -f expect" above.

... you're quite correct that `printf` is the way to go both here and
in the generated `print-args` script since these arguments start with
hyphen.

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v2 1/1] maintenance: use systemd timers on Linux
  2021-05-10  2:48       ` Eric Sunshine
@ 2021-05-10  6:25         ` Junio C Hamano
  2021-05-12  0:29           ` Đoàn Trần Công Danh
  0 siblings, 1 reply; 138+ messages in thread
From: Junio C Hamano @ 2021-05-10  6:25 UTC (permalink / raw)
  To: Eric Sunshine
  Cc: Đoàn Trần Công Danh, Lénaïc Huard,
	Git List, Derrick Stolee, brian m . carlson, Bagas Sanjaya,
	Phillip Wood, Ævar Arnfjörð Bjarmason

Eric Sunshine <sunshine@sunshineco.com> writes:

>> I think others have strong opinion on not using "%1$s",
>> and prefer simple "%s" and using "exec_path" twice instead.
>
> I brought it up only because I hadn't seen it in Git sources, and
> wasn't sure if we'd want to start using it. Aside from Ævar, who
> seemed reasonably in favor of it, nobody else chimed in, so it could
> go either way, I suppose.

If this were a piece of code that _everybody_ would use on _all_ the
supported platforms, I would suggest declaring that this is a
weather-balloon to see if some platforms have trouble using it.  But
unfortunately this is not such a piece of code.  Dependence on
systemd should strictly be opt-in.

So my preference is

 - here, just do it in the dumb and simple way

 - somewhere else, find code that is compiled and run for everybody
   on all platforms that feeds two same arguments to printf format,
   and update it to use "%1$x" twice, mark it clearly as a weather
   balloon, and document it (see how what 512f41cf did is documented
   in Documentation/CodingGuidelines and mimick, but tone it down as
   we haven't declared it safe to use (yet).

It is likely that we need rearrangement of argument order for po/
files anyway, but a misimplementation might not handle using the
same placeholder twice, and that is why I'd like to be a bit extra
careful.

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH] maintenance: use systemd timers on Linux
  2021-05-09 22:34     ` Lénaïc Huard
@ 2021-05-10 13:03       ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 138+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-10 13:03 UTC (permalink / raw)
  To: Lénaïc Huard
  Cc: Eric Sunshine, Git List, Junio C Hamano, Derrick Stolee,
	brian m. carlson


On Mon, May 10 2021, Lénaïc Huard wrote:

> Le mercredi 5 mai 2021, 14:01:25 CEST Ævar Arnfjörð Bjarmason a écrit :
>> On Sun, May 02 2021, Eric Sunshine wrote:
>> > On Sat, May 1, 2021 at 10:59 AM Lénaïc Huard <lenaic@lhuard.fr> wrote:
>> >> +       strvec_push(&child.args, "--now");
>> >> +       strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency);
>> >> +
>> >> +       if (start_command(&child))
>> >> +               die(_("failed to run systemctl"));
>> >> +       return finish_command(&child);
>> >> +}
>> >> +static int systemd_timer_write_unit_templates(const char *exec_path)
>> >> +{
>> >> +       unit = "[Unit]\n"
>> >> +              "Description=Optimize Git repositories data\n"
>> >> +              "\n"
>> >> +              "[Service]\n"
>> >> +              "Type=oneshot\n"
>> >> +              "ExecStart=\"%1$s/git\" --exec-path=\"%1$s\" for-each-repo
>> >> --config=maintenance.repo maintenance run --schedule=%%i\n"> 
>> > I see that it's in POSIX, but do we use this `%n$s` directive
>> > elsewhere in the Git source code? If not, I'd be cautious of
>> > introducing it here. Maybe it's better to just use plain `%s` twice...
>> 
>> We use it in po/, so for sprintf() on systems that don't have
>> NO_GETTEXT=Y we already test it in the wild.
>> 
>> But no, I don't think anything in the main source uses it, FWIW I have a
>> WIP series in my own fork that I've cooked for a while that uses this, I
>> haven't run into any issues with it in either GitHub's CI
>> (e.g. Windows), or on the systems I myself test on.
>> 
>> I think it would be a useful canary to just take a change like this, we
>> can always change it to the form you suggest if it doesn't work out.
>
> Based on this latest comment, I left the `%n$s` directive in the v2 of the 
> patch.
>
> Let me know if that’s still OK. Otherwise, I’d be happy to implement Eric’s 
> suggestion.
>
> Note however that this would be a “poor” canary to check if that directive is 
> supported on all the platforms on which git has been ported.
> Indeed, this code is executed only on systemd platforms, which means quite 
> recent Linux systems.
> Should this directive not be supported, I suppose it would be on more exotic 
> systems.

Indeed, although we compile it on non-Linux platforms, so we'd expect to
get complaints from smarter non-Linux compilers if fprintf() is given an
unknown formatting directive.

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v2 1/1] maintenance: use systemd timers on Linux
  2021-05-09 21:32   ` [PATCH v2 1/1] " Lénaïc Huard
  2021-05-10  1:20     ` Đoàn Trần Công Danh
@ 2021-05-10 18:03     ` Phillip Wood
  2021-05-10 18:25       ` Eric Sunshine
  2021-06-08 14:55       ` Lénaïc Huard
  2021-05-10 19:15     ` Martin Ågren
                       ` (2 subsequent siblings)
  4 siblings, 2 replies; 138+ messages in thread
From: Phillip Wood @ 2021-05-10 18:03 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine, brian m . carlson,
	Bagas Sanjaya, Đoàn Trần Công Danh,
	Ævar Arnfjörð Bjarmason

Hi Lénaïc

Thanks for updating the patch, I've left some comments below. Aside from 
one possible bug, they mostly revolve around not passing the name of 
scheduler command around when that name is fixed and functions which do 
not check for errors but return a success/failure value (either they 
should check the return values of the system calls they make or be 
declared as void if it is safe to ignore the failures)

On 09/05/2021 22:32, Lénaïc Huard wrote:
> The existing mechanism for scheduling background maintenance is done
> through cron. On Linux systems managed by systemd, systemd provides an
> alternative to schedule recurring tasks: systemd timers.
> 
> The main motivations to implement systemd timers in addition to cron
> are:
> * cron is optional and Linux systems running systemd might not have it
>    installed.
> * The execution of `crontab -l` can tell us if cron is installed but not
>    if the daemon is actually running.
> * With systemd, each service is run in its own cgroup and its logs are
>    tagged by the service inside journald. With cron, all scheduled tasks
>    are running in the cron daemon cgroup and all the logs of the
>    user-scheduled tasks are pretended to belong to the system cron
>    service.
>    Concretely, a user that doesn’t have access to the system logs won’t
>    have access to the log of its own tasks scheduled by cron whereas he
>    will have access to the log of its own tasks scheduled by systemd
>    timer.
>    Although `cron` attempts to send email, that email may go unseen by
>    the user because these days, local mailboxes are not heavily used
>    anymore.
> 
> In order to choose which scheduler to use between `cron` and user
> systemd timers, a new option
> `--scheduler=auto|cron|systemd|launchctl|schtasks` has been added to
> `git maintenance start`.
> When `git maintenance start --scheduler=XXX` is run, it not only
> registers `git maintenance run` tasks in the scheduler XXX, it also
> removes the `git maintenance run` tasks from all the other schedulers

I'm not sure it is actually doing that at the moment - see my comment in 
update_background_schedule()

> to
> ensure we cannot have two schedulers launching concurrent identical
> tasks.
> 
> The default value is `auto` which chooses a suitable scheduler for the
> system.
> On Linux, if user systemd timers are available, they will be used as git
> maintenance scheduler. If not, `cron` will be used if it is available.
> If none is available, it will fail.

I think defaulting to systemd timers when both systemd and cron are 
installed is sensible given the problems associated with testing whether 
crond is actually running in that scenario.

> `git maintenance stop` doesn't have any `--scheduler` parameter because
> this command will try to remove the `git maintenance run` tasks from all
> the available schedulers.
> 
> In order to schedule git maintenance, we need two unit template files:
> * ~/.config/systemd/user/git-maintenance@.service
>    to define the command to be started by systemd and
> * ~/.config/systemd/user/git-maintenance@.timer
>    to define the schedule at which the command should be run.
> 
> Those units are templates that are parametrized by the frequency.
> 
> Based on those templates, 3 timers are started:
> * git-maintenance@hourly.timer
> * git-maintenance@daily.timer
> * git-maintenance@weekly.timer
> 
> The command launched by those three timers are the same than with the
> other scheduling methods:
> 
> git for-each-repo --config=maintenance.repo maintenance run
> --schedule=%i
> 
> with the full path for git to ensure that the version of git launched
> for the scheduled maintenance is the same as the one used to run
> `maintenance start`.
> 
> The timer unit contains `Persistent=true` so that, if the computer is
> powered down when a maintenance task should run, the task will be run
> when the computer is back powered on.
> 
> Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
> ---
>   Documentation/git-maintenance.txt |  60 +++++
>   builtin/gc.c                      | 375 ++++++++++++++++++++++++++++--
>   t/t7900-maintenance.sh            |  51 ++++
>   3 files changed, 462 insertions(+), 24 deletions(-)
> 
> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
> index 80ddd33ceb..f012923333 100644
> --- a/Documentation/git-maintenance.txt
> +++ b/Documentation/git-maintenance.txt
> @@ -181,6 +181,20 @@ OPTIONS
>   	`maintenance.<task>.enabled` configured as `true` are considered.
>   	See the 'TASKS' section for the list of accepted `<task>` values.
>   
> +--scheduler=auto|crontab|systemd-timer|launchctl|schtasks::
> +	When combined with the `start` subcommand, specify the scheduler
> +	to use to run the hourly, daily and weekly executions of
> +	`git maintenance run`.
> +	The possible values for `<scheduler>` depend on the system: `crontab`
> +	is available on POSIX systems, `systemd-timer` is available on Linux
> +	systems, `launchctl` is available on MacOS and `schtasks` is available
> +	on Windows.
> +	By default or when `auto` is specified, the most appropriate scheduler
> +	for the system is used. On MacOS, `launchctl` is used. On Windows,
> +	`schtasks` is used. On Linux, `systemd-timers` is used if user systemd
> +	timers are available, otherwise, `crontab` is used. On all other systems,
> +	`crontab` is used.
> +
>   
>   TROUBLESHOOTING
>   ---------------
> @@ -279,6 +293,52 @@ schedule to ensure you are executing the correct binaries in your
>   schedule.
>   
>   
> +BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMS
> +-----------------------------------------------
> +
> +While Linux supports `cron`, depending on the distribution, `cron` may
> +be an optional package not necessarily installed. On modern Linux
> +distributions, systemd timers are superseding it.
> +
> +If user systemd timers are available, they will be used as a replacement
> +of `cron`.
> +
> +In this case, `git maintenance start` will create user systemd timer units
> +and start the timers. The current list of user-scheduled tasks can be found
> +by running `systemctl --user list-timers`. The timers written by `git
> +maintenance start` are similar to this:
> +
> +-----------------------------------------------------------------------
> +$ systemctl --user list-timers
> +NEXT                         LEFT          LAST                         PASSED     UNIT                         ACTIVATES
> +Thu 2021-04-29 19:00:00 CEST 42min left    Thu 2021-04-29 18:00:11 CEST 17min ago  git-maintenance@hourly.timer git-maintenance@hourly.service
> +Fri 2021-04-30 00:00:00 CEST 5h 42min left Thu 2021-04-29 00:00:11 CEST 18h ago    git-maintenance@daily.timer  git-maintenance@daily.service
> +Mon 2021-05-03 00:00:00 CEST 3 days left   Mon 2021-04-26 00:00:11 CEST 3 days ago git-maintenance@weekly.timer git-maintenance@weekly.service
> +-----------------------------------------------------------------------
> +
> +One timer is registered for each `--schedule=<frequency>` option.
> +
> +The definition of the systemd units can be inspected in the following files:
> +
> +-----------------------------------------------------------------------
> +~/.config/systemd/user/git-maintenance@.timer
> +~/.config/systemd/user/git-maintenance@.service
> +~/.config/systemd/user/timers.target.wants/git-maintenance@hourly.timer
> +~/.config/systemd/user/timers.target.wants/git-maintenance@daily.timer
> +~/.config/systemd/user/timers.target.wants/git-maintenance@weekly.timer
> +-----------------------------------------------------------------------
> +
> +`git maintenance start` will overwrite these files and start the timer
> +again with `systemctl --user`, so any customization should be done by
> +creating a drop-in file, i.e. a `.conf` suffixed file in the
> +`~/.config/systemd/user/git-maintenance@.service.d` directory.
> +
> +`git maintenance stop` will stop the user systemd timers and delete
> +the above mentioned files.
> +
> +For more details, see systemd.timer(5)
> +
> +
>   BACKGROUND MAINTENANCE ON MACOS SYSTEMS
>   ---------------------------------------
>   
> diff --git a/builtin/gc.c b/builtin/gc.c
> index ef7226d7bc..7c72aa3b99 100644
> --- a/builtin/gc.c
> +++ b/builtin/gc.c
> @@ -1544,6 +1544,15 @@ static const char *get_frequency(enum schedule_priority schedule)
>   	}
>   }
>   
> +static int is_launchctl_available(const char *cmd)

None of these is_..._available() function needs the cmd parameter. It 
matches the existing pattern of the existing ..._update_schedule() 
functions but they don't really need the cmd argument either so I would 
drop the argument for the new functions.

> +{
> +#ifdef __APPLE__
> +	return 1;
> +#else
> +	return 0;
> +#endif
> +}
> +
>   static char *launchctl_service_name(const char *frequency)
>   {
>   	struct strbuf label = STRBUF_INIT;
> @@ -1710,6 +1719,15 @@ static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm
>   		return launchctl_remove_plists(cmd);
>   }
>   
> +static int is_schtasks_available(const char *cmd)
> +{
> +#ifdef GIT_WINDOWS_NATIVE
> +	return 1;
> +#else
> +	return 0;
> +#endif
> +}
> +
>   static char *schtasks_task_name(const char *frequency)
>   {
>   	struct strbuf label = STRBUF_INIT;
> @@ -1872,6 +1890,28 @@ static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd
>   		return schtasks_remove_tasks(cmd);
>   }
>   
> +static int is_crontab_available(const char *cmd)
> +{
> +	static int cached_result = -1;
> +	struct child_process child = CHILD_PROCESS_INIT;
> +
> +	if (cached_result != -1)
> +		return cached_result;
> +
> +	strvec_split(&child.args, cmd);
> +	strvec_push(&child.args, "-l");
> +	child.no_stdin = 1;
> +	child.no_stdout = 1;
> +	child.no_stderr = 1;
> +	child.silent_exec_failure = 1;
> +
> +	if (start_command(&child))
> +		return cached_result = 0;
> +	/* Ignore exit code, as an empty crontab will return error. */
> +	finish_command(&child);
> +	return cached_result = 1;
> +}
> +
>   #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
>   #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
>   
> @@ -1959,61 +1999,348 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
>   	return result;
>   }
>   
> +static int is_systemd_timer_available(const char *cmd)
> +{
> +#ifdef __linux__
> +	static int cached_result = -1;
> +	struct child_process child = CHILD_PROCESS_INIT;
> +
> +	if (cached_result != -1)
> +		return cached_result;
> +
> +	strvec_split(&child.args, cmd);
> +	strvec_pushl(&child.args, "--user", "list-timers", NULL);
> +	child.no_stdin = 1;
> +	child.no_stdout = 1;
> +	child.no_stderr = 1;
> +	child.silent_exec_failure = 1;
> +
> +	if (start_command(&child))
> +		return cached_result = 0;

This is maybe a bit too clever. It would be clearer to separate the 
assignment of the cached result and the return statement.

> +	if (finish_command(&child))
> +		return cached_result = 0;
> +	return cached_result = 1;
> +#else
> +	return 0;
> +#endif
> +}
> +
> +static char *xdg_config_home_systemd(const char *filename)
> +{
> +	const char *home, *config_home;
> +
> +	assert(filename);
> +	config_home = getenv("XDG_CONFIG_HOME");
> +	if (config_home && *config_home)
> +		return mkpathdup("%s/systemd/user/%s", config_home, filename);
> +
> +	home = getenv("HOME");
> +	if (home)
> +		return mkpathdup("%s/.config/systemd/user/%s", home, filename);
> +
> +	die(_("failed to get $XDG_CONFIG_HOME and $HOME"));
> +}

This is largely a copy of xdg_config_home(), I would prefer to see this 
function calling that rather than duplicating it.

static char *xdg_config_home_systemd(const char *filename)
{
     struct strbuf buf = STRBUF_INIT;
     char *path;

     strbuf_addf(&buf, "systemd/user/%s", filename);
     path = xdg_config_home(buf.buf);
     strbuf_release(&buf);
     return path;
}

Even better would be to modify xdg_config_home() to take a format string.

> +static int systemd_timer_enable_unit(int enable,
> +				     enum schedule_priority schedule,
> +				     const char *cmd)

The cmd argument is pointless, it will always be "systemctl" and you 
have even hard coded that value into the error message below.

> +{
> +	struct child_process child = CHILD_PROCESS_INIT;
> +	const char *frequency = get_frequency(schedule);
> +
> +	strvec_split(&child.args, cmd);
> +	strvec_pushl(&child.args, "--user", enable ? "enable" : "disable",
> +		     "--now", NULL);
> +	strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency);
> +
> +	if (start_command(&child))
> +		die(_("failed to run systemctl"));
> +	return finish_command(&child);

There is a strange mix of dying and returning an error here. Also should 
this be printing a message if finish_command() returns a non-zero value? 
If systemctl fails then there is not much we can do to recover so dying 
on any error might be ok unless we need to perform some cleanup if it 
fails like removing the timer unit files.

What is the exit code of systemctl if a unit is already enabled and we 
try to enbale it again (and the same for disabling a disabled unit)?

> +}
> +
> +static int systemd_timer_delete_unit_templates(void)
> +{
> +	char *filename = xdg_config_home_systemd("git-maintenance@.timer");
> +	unlink(filename);

This is missing error handling (and below). If there is a good reason to 
ignore the errors then there is no point in returning a value from this 
function

> +	free(filename);
> +
> +	filename = xdg_config_home_systemd("git-maintenance@.service");
> +	unlink(filename);
> +	free(filename);
> +
> +	return 0;
> +}
> +
> +static int systemd_timer_delete_units(const char *cmd)
> +{
> +	return systemd_timer_enable_unit(0, SCHEDULE_HOURLY, cmd) ||
> +	       systemd_timer_enable_unit(0, SCHEDULE_DAILY, cmd) ||
> +	       systemd_timer_enable_unit(0, SCHEDULE_WEEKLY, cmd) ||
> +	       systemd_timer_delete_unit_templates();
> +}
> +
> +static int systemd_timer_write_unit_templates(const char *exec_path)
> +{
> +	char *filename;
> +	FILE *file;
> +	const char *unit;
> +
> +	filename = xdg_config_home_systemd("git-maintenance@.timer");
> +	if (safe_create_leading_directories(filename))
> +		die(_("failed to create directories for '%s'"), filename);
> +	file = xfopen(filename, "w");
> +	free(filename);
> +
> +	unit = "# This file was created and is maintained by Git.\n"
> +	       "# Any edits made in this file might be replaced in the future\n"
> +	       "# by a Git command.\n"
> +	       "\n"
> +	       "[Unit]\n"
> +	       "Description=Optimize Git repositories data\n"
> +	       "\n"
> +	       "[Timer]\n"
> +	       "OnCalendar=%i\n"
> +	       "Persistent=true\n"
> +	       "\n"
> +	       "[Install]\n"
> +	       "WantedBy=timers.target\n";
> +	fputs(unit, file);
> +	fclose(file);
> +
> +	filename = xdg_config_home_systemd("git-maintenance@.service");
> +	if (safe_create_leading_directories(filename))

Haven't we already created this path if it was missing above for the 
first file?

> +		die(_("failed to create directories for '%s'"), filename);
> +	file = xfopen(filename, "w");

Do we want to try and remove the first file if we cannot write the 
second file to leave the file system in a consitent state? If so you'll 
need to use something like fopen_or_warn() and check the return value.

> +	free(filename);
> +
> +	unit = "# This file was created and is maintained by Git.\n"
> +	       "# Any edits made in this file might be replaced in the future\n"
> +	       "# by a Git command.\n"
> +	       "\n"
> +	       "[Unit]\n"
> +	       "Description=Optimize Git repositories data\n"
> +	       "\n"
> +	       "[Service]\n"
> +	       "Type=oneshot\n"
> +	       "ExecStart=\"%1$s/git\" --exec-path=\"%1$s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
> +	       "LockPersonality=yes\n"
> +	       "MemoryDenyWriteExecute=yes\n"
> +	       "NoNewPrivileges=yes\n"
> +	       "RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\n"
> +	       "RestrictNamespaces=yes\n"
> +	       "RestrictRealtime=yes\n"
> +	       "RestrictSUIDSGID=yes\n"

After a quick read of the systemd.exec man page it is unclear to me if 
these Restrict... lines are needed as we already have 
NoNewPrivileges=yes - maybe they have some effect if `git maintence` is 
run as root?

> +	       "SystemCallArchitectures=native\n"
> +	       "SystemCallFilter=@system-service\n";
> +	fprintf(file, unit, exec_path);
> +	fclose(file);
> +
> +	return 0;

As we don't return any errors but die instead there is no need to return 
a value. On the other hand maybe we should be checking the return values 
of fclose() and possibly fputs() / fprint().

> +}
> +
> +static int systemd_timer_setup_units(const char *cmd)
> +{
> +	const char *exec_path = git_exec_path();
> +
> +	return systemd_timer_write_unit_templates(exec_path) ||
> +	       systemd_timer_enable_unit(1, SCHEDULE_HOURLY, cmd) ||
> +	       systemd_timer_enable_unit(1, SCHEDULE_DAILY, cmd) ||
> +	       systemd_timer_enable_unit(1, SCHEDULE_WEEKLY, cmd);

If any step fails do we need to try and reverse the preceding steps to 
avoid leaving so units enabled but not others. In practice this is 
probably pretty unlikely so maybe we don't need to worry.

> +}
> +
> +static int systemd_timer_update_schedule(int run_maintenance, int fd,
> +					 const char *cmd)

No need for the cmd argument

> +{
> +	if (run_maintenance)
> +		return systemd_timer_setup_units(cmd);
> +	else
> +		return systemd_timer_delete_units(cmd);
> +}
> +
> +enum scheduler {
> +	SCHEDULER_INVALID = -1,
> +	SCHEDULER_AUTO = 0,
> +	SCHEDULER_CRON = 1,
> +	SCHEDULER_SYSTEMD = 2,
> +	SCHEDULER_LAUNCHCTL = 3,
> +	SCHEDULER_SCHTASKS = 4,
> +};
> +
> +static const struct {
> +	int (*is_available)(const char *cmd);
> +	int (*update_schedule)(int run_maintenance, int fd, const char *cmd);
> +	const char *cmd;
> +} scheduler_fn[] = {
> +	[SCHEDULER_CRON] = { is_crontab_available, crontab_update_schedule,
> +			     "crontab" },
> +	[SCHEDULER_SYSTEMD] = { is_systemd_timer_available,
> +				systemd_timer_update_schedule, "systemctl" },
> +	[SCHEDULER_LAUNCHCTL] = { is_launchctl_available,
> +				  launchctl_update_schedule, "launchctl" },
> +	[SCHEDULER_SCHTASKS] = { is_schtasks_available,
> +				 schtasks_update_schedule, "schtasks" },
> +};
> +
> +static enum scheduler parse_scheduler(const char *value)
> +{
> +	if (!value)
> +		return SCHEDULER_INVALID;
> +	else if (!strcasecmp(value, "auto"))
> +		return SCHEDULER_AUTO;
> +	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
> +		return SCHEDULER_CRON;
> +	else if (!strcasecmp(value, "systemd") ||
> +		 !strcasecmp(value, "systemd-timer"))
> +		return SCHEDULER_SYSTEMD;
> +	else if (!strcasecmp(value, "launchctl"))
> +		return SCHEDULER_LAUNCHCTL;
> +	else if (!strcasecmp(value, "schtasks"))
> +		return SCHEDULER_SCHTASKS;
> +	else
> +		return SCHEDULER_INVALID;
> +}
> +
> +static int maintenance_opt_scheduler(const struct option *opt, const char *arg,
> +				     int unset)
> +{
> +	enum scheduler *scheduler = opt->value;
> +
> +	if (unset)
> +		die(_("--no-scheduler is not allowed"));

As others have said it would be better to BUG() here and pass 
PARSE_OPT_NONEG when defining the option below.

> +
> +	*scheduler = parse_scheduler(arg);
> +
> +	if (*scheduler == SCHEDULER_INVALID)
> +		die(_("unrecognized --scheduler argument '%s'"), arg);
> +
> +	return 0;
> +}
> +
> +struct maintenance_start_opts {
> +	enum scheduler scheduler;
> +};
> +
> +static void resolve_auto_scheduler(enum scheduler *scheduler)
> +{
> +	if (*scheduler != SCHEDULER_AUTO)
> +		return;
> +
>   #if defined(__APPLE__)
> -static const char platform_scheduler[] = "launchctl";
> +	*scheduler = SCHEDULER_LAUNCHCTL;
> +	return;
> +
>   #elif defined(GIT_WINDOWS_NATIVE)
> -static const char platform_scheduler[] = "schtasks";
> +	*scheduler = SCHEDULER_SCHTASKS;
> +	return;
> +
> +#elif defined(__linux__)
> +	if (is_systemd_timer_available("systemctl"))
> +		*scheduler = SCHEDULER_SYSTEMD;
> +	else if (is_crontab_available("crontab"))
> +		*scheduler = SCHEDULER_CRON;
> +	else
> +		die(_("neither systemd timers nor crontab are available"));
> +	return;
> +
>   #else
> -static const char platform_scheduler[] = "crontab";
> +	*scheduler = SCHEDULER_CRON;
> +	return;
>   #endif
> +}
>   
> -static int update_background_schedule(int enable)
> +static void validate_scheduler(enum scheduler scheduler)
>   {
> -	int result;
> -	const char *scheduler = platform_scheduler;
> -	const char *cmd = scheduler;
> +	const char *cmd;
> +
> +	if (scheduler == SCHEDULER_INVALID)
> +		BUG("invalid scheduler");
> +	if (scheduler == SCHEDULER_AUTO)
> +		BUG("resolve_auto_scheduler should have been called before");
> +
> +	cmd = scheduler_fn[scheduler].cmd;
> +	if (!scheduler_fn[scheduler].is_available(cmd))
> +		die(_("%s scheduler is not available"), cmd);
> +}
> +
> +static int update_background_schedule(const struct maintenance_start_opts *opts,
> +				      int enable)
> +{
> +	unsigned int i;
> +	int res, result = 0;
> +	enum scheduler scheduler;
> +	const char *cmd = NULL;
>   	char *testing;
>   	struct lock_file lk;
>   	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
>   
> +	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
> +		return error(_("another process is scheduling background maintenance"));
> +
>   	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
>   	if (testing) {
>   		char *sep = strchr(testing, ':');
>   		if (!sep)
>   			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
>   		*sep = '\0';
> -		scheduler = testing;
> +		scheduler = parse_scheduler(testing);
>   		cmd = sep + 1;
> +		result = scheduler_fn[scheduler].update_schedule(
> +			enable, get_lock_file_fd(&lk), cmd);
> +		goto done;
>   	}
>   
> -	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
> -		return error(_("another process is scheduling background maintenance"));
> -
> -	if (!strcmp(scheduler, "launchctl"))
> -		result = launchctl_update_schedule(enable, get_lock_file_fd(&lk), cmd);
> -	else if (!strcmp(scheduler, "schtasks"))
> -		result = schtasks_update_schedule(enable, get_lock_file_fd(&lk), cmd);
> -	else if (!strcmp(scheduler, "crontab"))
> -		result = crontab_update_schedule(enable, get_lock_file_fd(&lk), cmd);
> -	else
> -		die("unknown background scheduler: %s", scheduler);
> +	for (i = 1; i < ARRAY_SIZE(scheduler_fn); i++) {
> +		int enable_scheduler = enable && (opts->scheduler == i);
> +		cmd = scheduler_fn[i].cmd;
> +		if (!scheduler_fn[i].is_available(cmd))
> +			continue;
> +		res = scheduler_fn[i].update_schedule(
> +			enable_scheduler, get_lock_file_fd(&lk), cmd);
> +		if (enable_scheduler)
> +			result = res;

If I have understood the code correctly then if systemd timers are 
currently enabled and the user runs `git maintenance --scheduler=cron` 
then cron will be enabled and the loop will quit before we get the the 
entry to disable systemd timers leaving both running. I think it would 
be cleaner and clearer to loop over the schedulers disabling them first 
and then enable the one the user has selected.

> +	}
>   
> +done:
>   	rollback_lock_file(&lk);
>   	free(testing);
>   	return result;
>   }
>   
> -static int maintenance_start(void)
> +static const char *const builtin_maintenance_start_usage[] = {
> +	N_("git maintenance start [--scheduler=<scheduler>]"), NULL
> +};
> +
> +static int maintenance_start(int argc, const char **argv, const char *prefix)
>   {
> +	struct maintenance_start_opts opts;

struct maintenance_start_opts opts = { 0 };

would avoid the need for the memset() call below

Thanks again for working on this. Best Wishes

Phillip

> +	struct option builtin_maintenance_start_options[] = {
> +		OPT_CALLBACK(
> +			0, "scheduler", &opts.scheduler, N_("scheduler"),
> +			N_("scheduler to use to trigger git maintenance run"),
> +			maintenance_opt_scheduler),
> +		OPT_END()
> +	};
> +	memset(&opts, 0, sizeof(opts));
> +
> +	argc = parse_options(argc, argv, prefix,
> +			     builtin_maintenance_start_options,
> +			     builtin_maintenance_start_usage, 0);
> +
> +	resolve_auto_scheduler(&opts.scheduler);
> +	validate_scheduler(opts.scheduler);
> +
> +	if (argc > 0)
> +		usage_with_options(builtin_maintenance_start_usage,
> +				   builtin_maintenance_start_options);
> +
>   	if (maintenance_register())
>   		warning(_("failed to add repo to global config"));
> -
> -	return update_background_schedule(1);
> +	return update_background_schedule(&opts, 1);
>   }
>   
>   static int maintenance_stop(void)
>   {
> -	return update_background_schedule(0);
> +	return update_background_schedule(NULL, 0);
>   }
>   
>   static const char builtin_maintenance_usage[] =	N_("git maintenance <subcommand> [<options>]");
> @@ -2027,7 +2354,7 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
>   	if (!strcmp(argv[1], "run"))
>   		return maintenance_run(argc - 1, argv + 1, prefix);
>   	if (!strcmp(argv[1], "start"))
> -		return maintenance_start();
> +		return maintenance_start(argc - 1, argv + 1, prefix);
>   	if (!strcmp(argv[1], "stop"))
>   		return maintenance_stop();
>   	if (!strcmp(argv[1], "register"))
> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
> index 2412d8c5c0..6e6316cd90 100755
> --- a/t/t7900-maintenance.sh
> +++ b/t/t7900-maintenance.sh
> @@ -20,6 +20,20 @@ test_xmllint () {
>   	fi
>   }
>   
> +test_lazy_prereq SYSTEMD_ANALYZE '
> +	systemd-analyze --help >out &&
> +	grep verify out
> +'
> +
> +test_systemd_analyze_verify () {
> +	if test_have_prereq SYSTEMD_ANALYZE
> +	then
> +		systemd-analyze verify "$@"
> +	else
> +		true
> +	fi
> +}
> +
>   test_expect_success 'help text' '
>   	test_expect_code 129 git maintenance -h 2>err &&
>   	test_i18ngrep "usage: git maintenance <subcommand>" err &&
> @@ -615,6 +629,43 @@ test_expect_success 'start and stop Windows maintenance' '
>   	test_cmp expect args
>   '
>   
> +test_expect_success 'start and stop Linux/systemd maintenance' '
> +	write_script print-args <<-\EOF &&
> +	echo $* >>args
> +	EOF
> +
> +	rm -f args &&
> +	GIT_TEST_MAINT_SCHEDULER="systemd:./print-args" git maintenance start &&
> +
> +	# start registers the repo
> +	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
> +
> +	test_systemd_analyze_verify "$HOME/.config/systemd/user/git-maintenance@.service" &&
> +
> +	rm -f expect &&
> +	for frequency in hourly daily weekly
> +	do
> +		echo "--user enable --now git-maintenance@${frequency}.timer" >>expect || return 1
> +	done &&
> +	test_cmp expect args &&
> +
> +	rm -f args &&
> +	GIT_TEST_MAINT_SCHEDULER="systemd:./print-args" git maintenance stop &&
> +
> +	# stop does not unregister the repo
> +	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
> +
> +	test_path_is_missing "$HOME/.config/systemd/user/git-maintenance@.timer" &&
> +	test_path_is_missing "$HOME/.config/systemd/user/git-maintenance@.service" &&
> +
> +	rm -f expect &&
> +	for frequency in hourly daily weekly
> +	do
> +		echo "--user disable --now git-maintenance@${frequency}.timer" >>expect || return 1
> +	done &&
> +	test_cmp expect args
> +'
> +
>   test_expect_success 'register preserves existing strategy' '
>   	git config maintenance.strategy none &&
>   	git maintenance register &&
> 

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v2 1/1] maintenance: use systemd timers on Linux
  2021-05-10 18:03     ` Phillip Wood
@ 2021-05-10 18:25       ` Eric Sunshine
  2021-05-10 20:09         ` Phillip Wood
  2021-06-08 14:55       ` Lénaïc Huard
  1 sibling, 1 reply; 138+ messages in thread
From: Eric Sunshine @ 2021-05-10 18:25 UTC (permalink / raw)
  To: Phillip Wood
  Cc: Lénaïc Huard, Git List, Junio C Hamano, Derrick Stolee,
	brian m . carlson, Bagas Sanjaya,
	Đoàn Trần Công Danh,
	Ævar Arnfjörð Bjarmason

On Mon, May 10, 2021 at 2:04 PM Phillip Wood <phillip.wood123@gmail.com> wrote:
> On 09/05/2021 22:32, Lénaïc Huard wrote:
> > +static int systemd_timer_enable_unit(int enable,
> > +                                  enum schedule_priority schedule,
> > +                                  const char *cmd)
>
> The cmd argument is pointless, it will always be "systemctl" and you
> have even hard coded that value into the error message below.

The reason that `cmd` is passed around everywhere is that the actual
command can be overridden by GIT_TEST_MAINT_SCHEDULER which allows the
test script to mock up a scheduler command rather than running the
real scheduler command. I haven't read the new version of the patch
closely yet, but after a quick scan, I'm pretty confident that this is
still the case (despite the aggressive changes the patch makes to the
areas around GIT_TEST_MAINT_SCHEDULER).

As for hardcoding the command name in the error message, that seems
perfectly fine since, under normal circumstances, it _will_ be that
command (it's only different when testing).

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v2 1/1] maintenance: use systemd timers on Linux
  2021-05-09 21:32   ` [PATCH v2 1/1] " Lénaïc Huard
  2021-05-10  1:20     ` Đoàn Trần Công Danh
  2021-05-10 18:03     ` Phillip Wood
@ 2021-05-10 19:15     ` Martin Ågren
  2021-05-11 14:50     ` Phillip Wood
  2021-05-11 17:31     ` Derrick Stolee
  4 siblings, 0 replies; 138+ messages in thread
From: Martin Ågren @ 2021-05-10 19:15 UTC (permalink / raw)
  To: Lénaïc Huard
  Cc: Git Mailing List, Junio C Hamano, Derrick Stolee, Eric Sunshine,
	brian m . carlson, Bagas Sanjaya, Phillip Wood,
	Đoàn Trần Công Danh,
	Ævar Arnfjörð Bjarmason

Hi Lénaïc,

On Sun, 9 May 2021 at 23:37, Lénaïc Huard <lenaic@lhuard.fr> wrote:
> The default value is `auto` which chooses a suitable scheduler for the
> system.
> On Linux, if user systemd timers are available, they will be used as git
> maintenance scheduler. If not, `cron` will be used if it is available.
> If none is available, it will fail.

I understand your reasoning for going with systemd-timer over cron,
especially the part about knowing that the thing is actually running.

> +--scheduler=auto|crontab|systemd-timer|launchctl|schtasks::

This says "systemd-timer"...

> +       By default or when `auto` is specified, the most appropriate scheduler
> +       for the system is used. On MacOS, `launchctl` is used. On Windows,
> +       `schtasks` is used. On Linux, `systemd-timers` is used if user systemd

... this says "systemd-timers". Should those two be the same? (Which?)

> +       timers are available, otherwise, `crontab` is used. On all other systems,
> +       `crontab` is used.

So to be clear, I don't have a horse in this race. A few years ago I
would have foreseen all kinds of reactions to the implication that
systemd-timers would be "the most appropriate scheduler [...] on Linux".
Maybe those times are behind us now. In the commit message, you say "a
suitable", which reads a little bit less opinionated (to me).

That's just a minor point; feel free to disregard.

> +For more details, see systemd.timer(5)

Missing trailing ".".

A cursory grepping of our docs suggests this should be monospace
(`systemd.timer(5)`). There aren't that many places where we refer to
non-git manpages, thanks for doing so.

That's the only nit I found to make about the markup in the
documentation. Thanks for your attention to details. :-)

Martin

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v2 1/1] maintenance: use systemd timers on Linux
  2021-05-10 18:25       ` Eric Sunshine
@ 2021-05-10 20:09         ` Phillip Wood
  2021-05-10 20:52           ` Eric Sunshine
  0 siblings, 1 reply; 138+ messages in thread
From: Phillip Wood @ 2021-05-10 20:09 UTC (permalink / raw)
  To: Eric Sunshine, Phillip Wood
  Cc: Lénaïc Huard, Git List, Junio C Hamano, Derrick Stolee,
	brian m . carlson, Bagas Sanjaya,
	Đoàn Trần Công Danh,
	Ævar Arnfjörð Bjarmason

Hi Eric

On 10/05/2021 19:25, Eric Sunshine wrote:
> On Mon, May 10, 2021 at 2:04 PM Phillip Wood <phillip.wood123@gmail.com> wrote:
>> On 09/05/2021 22:32, Lénaïc Huard wrote:
>>> +static int systemd_timer_enable_unit(int enable,
>>> +                                  enum schedule_priority schedule,
>>> +                                  const char *cmd)
>>
>> The cmd argument is pointless, it will always be "systemctl" and you
>> have even hard coded that value into the error message below.
> 
> The reason that `cmd` is passed around everywhere is that the actual
> command can be overridden by GIT_TEST_MAINT_SCHEDULER which allows the
> test script to mock up a scheduler command rather than running the
> real scheduler command. I haven't read the new version of the patch
> closely yet, but after a quick scan, I'm pretty confident that this is
> still the case (despite the aggressive changes the patch makes to the
> areas around GIT_TEST_MAINT_SCHEDULER).

Thanks for pointing that out I'd somehow glossed over the 
GIT_TEST_MAINT_SCHEDULER code, I agree it looks like the patch takes 
care to keep it working.

It is outside the scope of this patch but a possibly nicer pattern would 
be to have a function get_command_name(const char *default) that checks 
GIT_TEST_MAINT_SCHEDULER and returns the command name from that or the 
default if it is not set. We would then call that function to get the 
command name when we want to run a command. That way all the extra 
complexity is localized around the command call (and consists of a 
single function call), the usual command name is visible in the function 
calling the command and we'd avoid littering all the function signatures 
with a argument that is only relevant for testing.

> As for hardcoding the command name in the error message, that seems
> perfectly fine since, under normal circumstances, it _will_ be that
> command (it's only different when testing).

I agree, thanks

Phillip

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v2 1/1] maintenance: use systemd timers on Linux
  2021-05-10 20:09         ` Phillip Wood
@ 2021-05-10 20:52           ` Eric Sunshine
  0 siblings, 0 replies; 138+ messages in thread
From: Eric Sunshine @ 2021-05-10 20:52 UTC (permalink / raw)
  To: Phillip Wood
  Cc: Lénaïc Huard, Git List, Junio C Hamano, Derrick Stolee,
	brian m . carlson, Bagas Sanjaya,
	Đoàn Trần Công Danh,
	Ævar Arnfjörð Bjarmason

On Mon, May 10, 2021 at 4:09 PM Phillip Wood <phillip.wood123@gmail.com> wrote:
> It is outside the scope of this patch but a possibly nicer pattern would
> be to have a function get_command_name(const char *default) that checks
> GIT_TEST_MAINT_SCHEDULER and returns the command name from that or the
> default if it is not set. We would then call that function to get the
> command name when we want to run a command. That way all the extra
> complexity is localized around the command call (and consists of a
> single function call), the usual command name is visible in the function
> calling the command and we'd avoid littering all the function signatures
> with a argument that is only relevant for testing.

Yup, that would be a nice eventual cleanup. I agree that it is outside
the scope of this submission.

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v2 1/1] maintenance: use systemd timers on Linux
  2021-05-09 21:32   ` [PATCH v2 1/1] " Lénaïc Huard
                       ` (2 preceding siblings ...)
  2021-05-10 19:15     ` Martin Ågren
@ 2021-05-11 14:50     ` Phillip Wood
  2021-05-11 17:31     ` Derrick Stolee
  4 siblings, 0 replies; 138+ messages in thread
From: Phillip Wood @ 2021-05-11 14:50 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine, brian m . carlson,
	Bagas Sanjaya, Đoàn Trần Công Danh,
	Ævar Arnfjörð Bjarmason

Hi Lénaïc

I've added some comments about testing

On 09/05/2021 22:32, Lénaïc Huard wrote:
>[...]
> +struct maintenance_start_opts {
> +	enum scheduler scheduler;
> +};
> +
> +static void resolve_auto_scheduler(enum scheduler *scheduler)
> +{
> +	if (*scheduler != SCHEDULER_AUTO)
> +		return;
> +
>   #if defined(__APPLE__)
> -static const char platform_scheduler[] = "launchctl";
> +	*scheduler = SCHEDULER_LAUNCHCTL;
> +	return;
> +
>   #elif defined(GIT_WINDOWS_NATIVE)
> -static const char platform_scheduler[] = "schtasks";
> +	*scheduler = SCHEDULER_SCHTASKS;
> +	return;
> +
> +#elif defined(__linux__)
> +	if (is_systemd_timer_available("systemctl"))
> +		*scheduler = SCHEDULER_SYSTEMD;
> +	else if (is_crontab_available("crontab"))
> +		*scheduler = SCHEDULER_CRON;
> +	else
> +		die(_("neither systemd timers nor crontab are available"));
> +	return;
> +
>   #else
> -static const char platform_scheduler[] = "crontab";
> +	*scheduler = SCHEDULER_CRON;
> +	return;
>   #endif
> +}

As it stands this function is untested and there is no way to test it 
with the current setup. There are two difficulties with testing it (i) 
it uses conditional compilation and (ii) there is no way to fake crontab 
and systemctl with the current test setup. I think we can address the 
latter (see below) and handle the condition compilation issues using 
test prerequisites if necessary.

> -static int update_background_schedule(int enable)
> +static void validate_scheduler(enum scheduler scheduler)
>   {
> -	int result;
> -	const char *scheduler = platform_scheduler;
> -	const char *cmd = scheduler;
> +	const char *cmd;
> +
> +	if (scheduler == SCHEDULER_INVALID)
> +		BUG("invalid scheduler");
> +	if (scheduler == SCHEDULER_AUTO)
> +		BUG("resolve_auto_scheduler should have been called before");
> +
> +	cmd = scheduler_fn[scheduler].cmd;
> +	if (!scheduler_fn[scheduler].is_available(cmd))
> +		die(_("%s scheduler is not available"), cmd);
> +}
> +
> +static int update_background_schedule(const struct maintenance_start_opts *opts,
> +				      int enable)
> +{
> +	unsigned int i;
> +	int res, result = 0;
> +	enum scheduler scheduler;
> +	const char *cmd = NULL;
>   	char *testing;
>   	struct lock_file lk;
>   	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
>   
> +	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
> +		return error(_("another process is scheduling background maintenance"));
> +
>   	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
>   	if (testing) {
>   		char *sep = strchr(testing, ':');
>   		if (!sep)
>   			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
>   		*sep = '\0';
> -		scheduler = testing;
> +		scheduler = parse_scheduler(testing);
>   		cmd = sep + 1;
> +		result = scheduler_fn[scheduler].update_schedule(
> +			enable, get_lock_file_fd(&lk), cmd);
> +		goto done;
>   	}
>   
> -	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
> -		return error(_("another process is scheduling background maintenance"));
> -
> -	if (!strcmp(scheduler, "launchctl"))
> -		result = launchctl_update_schedule(enable, get_lock_file_fd(&lk), cmd);
> -	else if (!strcmp(scheduler, "schtasks"))
> -		result = schtasks_update_schedule(enable, get_lock_file_fd(&lk), cmd);
> -	else if (!strcmp(scheduler, "crontab"))
> -		result = crontab_update_schedule(enable, get_lock_file_fd(&lk), cmd);
> -	else
> -		die("unknown background scheduler: %s", scheduler);
> +	for (i = 1; i < ARRAY_SIZE(scheduler_fn); i++) {
> +		int enable_scheduler = enable && (opts->scheduler == i);
> +		cmd = scheduler_fn[i].cmd;
> +		if (!scheduler_fn[i].is_available(cmd))
> +			continue;
> +		res = scheduler_fn[i].update_schedule(
> +			enable_scheduler, get_lock_file_fd(&lk), cmd);
> +		if (enable_scheduler)
> +			result = res;
> +	}

This loop which is responsible for disabling the existing scheduler and 
enabling a new one is completely untested. I think that before this 
patch special casing the testing above still ran the important code, 
however now we fail to test an important aspect of the business logic. 
As we can only fake one command name the current test setup is 
insufficient to fix this. I think the best solution would be to mock 
systemctl, crontab, and the other commands in the test environment.

	mkdir bin &&
	PATH="$(pwd)/bin:$PATH" &&
	test_write_script systemctl <<-\EOF &&
	# Mock systemctl, set FAKE_SYSTEMD=0 in the
	# environment to fake systemctl missing
	case "$*" in
		"--user list-timers") test ${FAKE_SYSTEMD:-1} = 1;
				      exit $?;;
		*) printf "%s\n" "$*";;
	esac	
	EOF
	# and so on for crontab etc

Then it would be possible to test the various values 
--scheduler=<scheduler> including 'auto' (by setting FAKE_SYSTEMD=0 or 
FAKE_CRONTAB=0) and that we disable cron when enabling systemd and vice 
versa. Mocking the commands would also allow us to cleanup the code in 
gc.c as it would no longer need to pass command names around.

> +done:
>   	rollback_lock_file(&lk);
>   	free(testing);
>   	return result;
>   }
>   
> -static int maintenance_start(void)
> +static const char *const builtin_maintenance_start_usage[] = {
> +	N_("git maintenance start [--scheduler=<scheduler>]"), NULL
> +};
> +
> +static int maintenance_start(int argc, const char **argv, const char *prefix)
>   {
> +	struct maintenance_start_opts opts;
> +	struct option builtin_maintenance_start_options[] = {
> +		OPT_CALLBACK(
> +			0, "scheduler", &opts.scheduler, N_("scheduler"),
> +			N_("scheduler to use to trigger git maintenance run"),
> +			maintenance_opt_scheduler),
> +		OPT_END()
> +	};
> +	memset(&opts, 0, sizeof(opts));
> +
> +	argc = parse_options(argc, argv, prefix,
> +			     builtin_maintenance_start_options,
> +			     builtin_maintenance_start_usage, 0);

This new command line option is completely untested

Best Wishes

Phillip

> +
> +	resolve_auto_scheduler(&opts.scheduler);
> +	validate_scheduler(opts.scheduler);
> +
> +	if (argc > 0)
> +		usage_with_options(builtin_maintenance_start_usage,
> +				   builtin_maintenance_start_options);
> +
>   	if (maintenance_register())
>   		warning(_("failed to add repo to global config"));
> -
> -	return update_background_schedule(1);
> +	return update_background_schedule(&opts, 1);
>   }
>   
>   static int maintenance_stop(void)
>   {
> -	return update_background_schedule(0);
> +	return update_background_schedule(NULL, 0);
>   }
>   
>   static const char builtin_maintenance_usage[] =	N_("git maintenance <subcommand> [<options>]");
> @@ -2027,7 +2354,7 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
>   	if (!strcmp(argv[1], "run"))
>   		return maintenance_run(argc - 1, argv + 1, prefix);
>   	if (!strcmp(argv[1], "start"))
> -		return maintenance_start();
> +		return maintenance_start(argc - 1, argv + 1, prefix);
>   	if (!strcmp(argv[1], "stop"))
>   		return maintenance_stop();
>   	if (!strcmp(argv[1], "register"))
> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
> index 2412d8c5c0..6e6316cd90 100755
> --- a/t/t7900-maintenance.sh
> +++ b/t/t7900-maintenance.sh
> @@ -20,6 +20,20 @@ test_xmllint () {
>   	fi
>   }
>   
> +test_lazy_prereq SYSTEMD_ANALYZE '
> +	systemd-analyze --help >out &&
> +	grep verify out
> +'
> +
> +test_systemd_analyze_verify () {
> +	if test_have_prereq SYSTEMD_ANALYZE
> +	then
> +		systemd-analyze verify "$@"
> +	else
> +		true
> +	fi
> +}
> +
>   test_expect_success 'help text' '
>   	test_expect_code 129 git maintenance -h 2>err &&
>   	test_i18ngrep "usage: git maintenance <subcommand>" err &&
> @@ -615,6 +629,43 @@ test_expect_success 'start and stop Windows maintenance' '
>   	test_cmp expect args
>   '
>   
> +test_expect_success 'start and stop Linux/systemd maintenance' '
> +	write_script print-args <<-\EOF &&
> +	echo $* >>args
> +	EOF
> +
> +	rm -f args &&
> +	GIT_TEST_MAINT_SCHEDULER="systemd:./print-args" git maintenance start &&
> +
> +	# start registers the repo
> +	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
> +
> +	test_systemd_analyze_verify "$HOME/.config/systemd/user/git-maintenance@.service" &&
> +
> +	rm -f expect &&
> +	for frequency in hourly daily weekly
> +	do
> +		echo "--user enable --now git-maintenance@${frequency}.timer" >>expect || return 1
> +	done &&
> +	test_cmp expect args &&
> +
> +	rm -f args &&
> +	GIT_TEST_MAINT_SCHEDULER="systemd:./print-args" git maintenance stop &&
> +
> +	# stop does not unregister the repo
> +	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
> +
> +	test_path_is_missing "$HOME/.config/systemd/user/git-maintenance@.timer" &&
> +	test_path_is_missing "$HOME/.config/systemd/user/git-maintenance@.service" &&
> +
> +	rm -f expect &&
> +	for frequency in hourly daily weekly
> +	do
> +		echo "--user disable --now git-maintenance@${frequency}.timer" >>expect || return 1
> +	done &&
> +	test_cmp expect args
> +'
> +
>   test_expect_success 'register preserves existing strategy' '
>   	git config maintenance.strategy none &&
>   	git maintenance register &&
> 

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v2 1/1] maintenance: use systemd timers on Linux
  2021-05-09 21:32   ` [PATCH v2 1/1] " Lénaïc Huard
                       ` (3 preceding siblings ...)
  2021-05-11 14:50     ` Phillip Wood
@ 2021-05-11 17:31     ` Derrick Stolee
  4 siblings, 0 replies; 138+ messages in thread
From: Derrick Stolee @ 2021-05-11 17:31 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine, brian m . carlson,
	Bagas Sanjaya, Phillip Wood,
	Đoàn Trần Công Danh,
	Ævar Arnfjörð Bjarmason

On 5/9/2021 5:32 PM, Lénaïc Huard wrote:
> The existing mechanism for scheduling background maintenance is done
> through cron. On Linux systems managed by systemd, systemd provides an
> alternative to schedule recurring tasks: systemd timers.

Thank you for working so hard on this systemd integration. I see you
have already received significant feedback on that portion, so I
wanted to focus my review on the other piece that exists in this patch.

> In order to choose which scheduler to use between `cron` and user
> systemd timers, a new option
> `--scheduler=auto|cron|systemd|launchctl|schtasks` has been added to
> `git maintenance start`.
> When `git maintenance start --scheduler=XXX` is run, it not only
> registers `git maintenance run` tasks in the scheduler XXX, it also
> removes the `git maintenance run` tasks from all the other schedulers to
> ensure we cannot have two schedulers launching concurrent identical
> tasks.
> 
> The default value is `auto` which chooses a suitable scheduler for the
> system.

This addition of the --scheduler option should be split into a patch
on its own. It requires significant refactoring of the existing code
in a way that distracts from your systemd work.

I'll highlight the portions of the diff that you could include in
a preliminary patch and save the systemd stuff for an addition on top
of that.

> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
> index 80ddd33ceb..f012923333 100644
> --- a/Documentation/git-maintenance.txt
> +++ b/Documentation/git-maintenance.txt
> @@ -181,6 +181,20 @@ OPTIONS
>  	`maintenance.<task>.enabled` configured as `true` are considered.
>  	See the 'TASKS' section for the list of accepted `<task>` values.
>  
> +--scheduler=auto|crontab|systemd-timer|launchctl|schtasks::
> +	When combined with the `start` subcommand, specify the scheduler
> +	to use to run the hourly, daily and weekly executions of
> +	`git maintenance run`.
> +	The possible values for `<scheduler>` depend on the system: `crontab`
> +	is available on POSIX systems, `systemd-timer` is available on Linux
> +	systems, `launchctl` is available on MacOS and `schtasks` is available
> +	on Windows.
> +	By default or when `auto` is specified, the most appropriate scheduler
> +	for the system is used. On MacOS, `launchctl` is used. On Windows,
> +	`schtasks` is used. On Linux, `systemd-timers` is used if user systemd
> +	timers are available, otherwise, `crontab` is used. On all other systems,
> +	`crontab` is used.
> +

This portion of the docs can be updated on its own (minus the systemd bits).
> diff --git a/builtin/gc.c b/builtin/gc.c
> index ef7226d7bc..7c72aa3b99 100644
> --- a/builtin/gc.c
> +++ b/builtin/gc.c
> @@ -1544,6 +1544,15 @@ static const char *get_frequency(enum schedule_priority schedule)
>  	}
>  }
>  
> +static int is_launchctl_available(const char *cmd)
> +{
> +#ifdef __APPLE__
> +	return 1;
> +#else
> +	return 0;
> +#endif
> +}
> +
>  static char *launchctl_service_name(const char *frequency)
>  {
>  	struct strbuf label = STRBUF_INIT;
> @@ -1710,6 +1719,15 @@ static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm
>  		return launchctl_remove_plists(cmd);
>  }
>  
> +static int is_schtasks_available(const char *cmd)
> +{
> +#ifdef GIT_WINDOWS_NATIVE
> +	return 1;
> +#else
> +	return 0;
> +#endif
> +}
> +
>  static char *schtasks_task_name(const char *frequency)
>  {
>  	struct strbuf label = STRBUF_INIT;
> @@ -1872,6 +1890,28 @@ static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd
>  		return schtasks_remove_tasks(cmd);
>  }
>  
> +static int is_crontab_available(const char *cmd)
> +{
> +	static int cached_result = -1;
> +	struct child_process child = CHILD_PROCESS_INIT;
> +
> +	if (cached_result != -1)
> +		return cached_result;
> +
> +	strvec_split(&child.args, cmd);
> +	strvec_push(&child.args, "-l");
> +	child.no_stdin = 1;
> +	child.no_stdout = 1;
> +	child.no_stderr = 1;
> +	child.silent_exec_failure = 1;
> +
> +	if (start_command(&child))
> +		return cached_result = 0;
> +	/* Ignore exit code, as an empty crontab will return error. */
> +	finish_command(&child);
> +	return cached_result = 1;
> +}
> +

These is_X_available() methods are valuable helpers. Adding
is_systemd_timer_available() in the second patch will be
simpler with this framework in place.

> +enum scheduler {
> +	SCHEDULER_INVALID = -1,
> +	SCHEDULER_AUTO = 0,
> +	SCHEDULER_CRON = 1,
> +	SCHEDULER_SYSTEMD = 2,
> +	SCHEDULER_LAUNCHCTL = 3,
> +	SCHEDULER_SCHTASKS = 4,
> +};
> +
> +static const struct {
> +	int (*is_available)(const char *cmd);
> +	int (*update_schedule)(int run_maintenance, int fd, const char *cmd);
> +	const char *cmd;
> +} scheduler_fn[] = {
> +	[SCHEDULER_CRON] = { is_crontab_available, crontab_update_schedule,
> +			     "crontab" },
> +	[SCHEDULER_SYSTEMD] = { is_systemd_timer_available,
> +				systemd_timer_update_schedule, "systemctl" },
> +	[SCHEDULER_LAUNCHCTL] = { is_launchctl_available,
> +				  launchctl_update_schedule, "launchctl" },
> +	[SCHEDULER_SCHTASKS] = { is_schtasks_available,
> +				 schtasks_update_schedule, "schtasks" },
> +};

This is also good to include, minus the systemd lines.

I would also like to see this declaration reformatted.
Something like this would be good:

static const struct {
	int (*is_available)(const char *cmd);
	int (*update_schedule)(int run_maintenance, int fd, const char *cmd);
	const char *cmd;
} scheduler_fn[] = {
	[SCHEDULER_CRON] = {
		.is_available = is_crontab_available,
		.update_schedule = crontab_update_schedule,
		.cmd = "crontab",
	},
	[SCHEDULER_LAUNCHCTL] = {
		.is_available = is_launchctl_available,
		.update_schedule = launchctl_update_schedule,
		.cmd = "launchctl",
	},
	[SCHEDULER_SCHTASKS] = {
		.is_available = is_schtasks_available,
		.update_schedule = schtasks_update_schedule,
		.cmd = "schtasks",
	},
};

The use of member names can help if we need to augment this
struct later, and the use of commas after the final terms of
each block helps the future diff if we add items to the end.

> +
> +static enum scheduler parse_scheduler(const char *value)
> +{
> +	if (!value)
> +		return SCHEDULER_INVALID;
> +	else if (!strcasecmp(value, "auto"))
> +		return SCHEDULER_AUTO;
> +	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
> +		return SCHEDULER_CRON;
> +	else if (!strcasecmp(value, "systemd") ||
> +		 !strcasecmp(value, "systemd-timer"))
> +		return SCHEDULER_SYSTEMD;
> +	else if (!strcasecmp(value, "launchctl"))
> +		return SCHEDULER_LAUNCHCTL;
> +	else if (!strcasecmp(value, "schtasks"))
> +		return SCHEDULER_SCHTASKS;
> +	else
> +		return SCHEDULER_INVALID;
> +}

Good. The systemd stuff can be added in the second patch,
making a clear integration point.

> +static int maintenance_opt_scheduler(const struct option *opt, const char *arg,
> +				     int unset)
> +{
> +	enum scheduler *scheduler = opt->value;
> +
> +	if (unset)
> +		die(_("--no-scheduler is not allowed"));
> +
> +	*scheduler = parse_scheduler(arg);
> +
> +	if (*scheduler == SCHEDULER_INVALID)
> +		die(_("unrecognized --scheduler argument '%s'"), arg);
> +
> +	return 0;
> +}
> +
> +struct maintenance_start_opts {
> +	enum scheduler scheduler;
> +};

This struct that contains only the enum seems confusing to me.
Maybe it will make sense later.

> +static void resolve_auto_scheduler(enum scheduler *scheduler)
> +{
> +	if (*scheduler != SCHEDULER_AUTO)
> +		return;
> +
>  #if defined(__APPLE__)
> -static const char platform_scheduler[] = "launchctl";
> +	*scheduler = SCHEDULER_LAUNCHCTL;
> +	return;
> +
>  #elif defined(GIT_WINDOWS_NATIVE)
> -static const char platform_scheduler[] = "schtasks";
> +	*scheduler = SCHEDULER_SCHTASKS;
> +	return;
> +
> +#elif defined(__linux__)
> +	if (is_systemd_timer_available("systemctl"))
> +		*scheduler = SCHEDULER_SYSTEMD;
> +	else if (is_crontab_available("crontab"))
> +		*scheduler = SCHEDULER_CRON;
> +	else
> +		die(_("neither systemd timers nor crontab are available"));
> +	return;
> +
>  #else
> -static const char platform_scheduler[] = "crontab";
> +	*scheduler = SCHEDULER_CRON;
> +	return;
>  #endif
> +}

This diff looks pretty rough. I see that you are making
systemctl the default for Linux. Ok. This also seems like
it will not be testable in the test suite.

>  
> -static int update_background_schedule(int enable)
> +static void validate_scheduler(enum scheduler scheduler)
>  {
> -	int result;
> -	const char *scheduler = platform_scheduler;
> -	const char *cmd = scheduler;
> +	const char *cmd;
> +
> +	if (scheduler == SCHEDULER_INVALID)
> +		BUG("invalid scheduler");
> +	if (scheduler == SCHEDULER_AUTO)
> +		BUG("resolve_auto_scheduler should have been called before");
> +
> +	cmd = scheduler_fn[scheduler].cmd;
> +	if (!scheduler_fn[scheduler].is_available(cmd))
> +		die(_("%s scheduler is not available"), cmd);
> +}
> +
> +static int update_background_schedule(const struct maintenance_start_opts *opts,
> +				      int enable)
> +{
> +	unsigned int i;
> +	int res, result = 0;
> +	enum scheduler scheduler;
> +	const char *cmd = NULL;
>  	char *testing;
>  	struct lock_file lk;
>  	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
>  
> +	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
> +		return error(_("another process is scheduling background maintenance"));
> +
>  	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
>  	if (testing) {
>  		char *sep = strchr(testing, ':');
>  		if (!sep)
>  			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
>  		*sep = '\0';
> -		scheduler = testing;
> +		scheduler = parse_scheduler(testing);
>  		cmd = sep + 1;
> +		result = scheduler_fn[scheduler].update_schedule(
> +			enable, get_lock_file_fd(&lk), cmd);
> +		goto done;

I see this 'goto done' is the reason we need to take the lock earlier. The
other option would be to put the 'goto done' after the rollback_lock_file(),
but this is fine, too.

>  	}
>  
> -	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
> -		return error(_("another process is scheduling background maintenance"));
> -
> -	if (!strcmp(scheduler, "launchctl"))
> -		result = launchctl_update_schedule(enable, get_lock_file_fd(&lk), cmd);
> -	else if (!strcmp(scheduler, "schtasks"))
> -		result = schtasks_update_schedule(enable, get_lock_file_fd(&lk), cmd);
> -	else if (!strcmp(scheduler, "crontab"))
> -		result = crontab_update_schedule(enable, get_lock_file_fd(&lk), cmd);
> -	else
> -		die("unknown background scheduler: %s", scheduler);
> +	for (i = 1; i < ARRAY_SIZE(scheduler_fn); i++) {
> +		int enable_scheduler = enable && (opts->scheduler == i);
> +		cmd = scheduler_fn[i].cmd;
> +		if (!scheduler_fn[i].is_available(cmd))
> +			continue;
> +		res = scheduler_fn[i].update_schedule(
> +			enable_scheduler, get_lock_file_fd(&lk), cmd);
> +		if (enable_scheduler)
> +			result = res;
> +	}

This loop is cleaner than the list of else-ifs.

>  
> +done:
>  	rollback_lock_file(&lk);
>  	free(testing);
>  	return result;
>  }
>  
> -static int maintenance_start(void)
> +static const char *const builtin_maintenance_start_usage[] = {
> +	N_("git maintenance start [--scheduler=<scheduler>]"), NULL
> +};
> +
> +static int maintenance_start(int argc, const char **argv, const char *prefix)
>  {
> +	struct maintenance_start_opts opts;

I see you using the struct now and in the helper methods above.
While the creation of the struct looks a little strange now, this
use matches other patterns in this file and is more flexible to
additional options in the future. Thanks.

> +	struct option builtin_maintenance_start_options[] = {
> +		OPT_CALLBACK(
> +			0, "scheduler", &opts.scheduler, N_("scheduler"),
> +			N_("scheduler to use to trigger git maintenance run"),
> +			maintenance_opt_scheduler),

nit: these would typically be aligned with the end of the
OPT_CALLBACK( opener, with the first set of parameters being on
the same line:

		OPT_CALLBACK(0, "scheduler", &opts.scheduler, N_("scheduler"),
			     N_("scheduler to use to trigger 'git maintenance run'"),
			     maintenance_opt_scheduler),

These options frequently run a little long on the line width,
which might have been your motivation in adding an extra line.

> +		OPT_END()
> +	};
> +	memset(&opts, 0, sizeof(opts));
> +
> +	argc = parse_options(argc, argv, prefix,
> +			     builtin_maintenance_start_options,
> +			     builtin_maintenance_start_usage, 0);
> +
> +	resolve_auto_scheduler(&opts.scheduler);
> +	validate_scheduler(opts.scheduler);
> +
> +	if (argc > 0)
> +		usage_with_options(builtin_maintenance_start_usage,
> +				   builtin_maintenance_start_options);

nit: "if (argc)" is the more typical pattern in the Git codebase.

Also, this check should come right after parse_options().

>  	if (maintenance_register())
>  		warning(_("failed to add repo to global config"));
> -
> -	return update_background_schedule(1);
> +	return update_background_schedule(&opts, 1);
>  }
>  
>  static int maintenance_stop(void)
>  {
> -	return update_background_schedule(0);
> +	return update_background_schedule(NULL, 0);
>  }
>  
>  static const char builtin_maintenance_usage[] =	N_("git maintenance <subcommand> [<options>]");
> @@ -2027,7 +2354,7 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
>  	if (!strcmp(argv[1], "run"))
>  		return maintenance_run(argc - 1, argv + 1, prefix);
>  	if (!strcmp(argv[1], "start"))
> -		return maintenance_start();
> +		return maintenance_start(argc - 1, argv + 1, prefix);
>  	if (!strcmp(argv[1], "stop"))
>  		return maintenance_stop();
>  	if (!strcmp(argv[1], "register"))
> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
> index 2412d8c5c0..6e6316cd90 100755
> --- a/t/t7900-maintenance.sh
> +++ b/t/t7900-maintenance.sh

I'm not sure how we could achieve it, but it might be good to demonstrate
a use of the --scheduler option here in the test script.

Thanks,
-Stolee

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v2 1/1] maintenance: use systemd timers on Linux
  2021-05-10  6:25         ` Junio C Hamano
@ 2021-05-12  0:29           ` Đoàn Trần Công Danh
  2021-05-12  6:59             ` Felipe Contreras
  2021-05-12 13:38             ` Phillip Wood
  0 siblings, 2 replies; 138+ messages in thread
From: Đoàn Trần Công Danh @ 2021-05-12  0:29 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Eric Sunshine, Lénaïc Huard, Git List, Derrick Stolee,
	brian m . carlson, Bagas Sanjaya, Phillip Wood,
	Ævar Arnfjörð Bjarmason

On 2021-05-10 15:25:07+0900, Junio C Hamano <gitster@pobox.com> wrote:
> Eric Sunshine <sunshine@sunshineco.com> writes:
> 
> >> I think others have strong opinion on not using "%1$s",
> >> and prefer simple "%s" and using "exec_path" twice instead.
> >
> > I brought it up only because I hadn't seen it in Git sources, and
> > wasn't sure if we'd want to start using it. Aside from Ævar, who
> > seemed reasonably in favor of it, nobody else chimed in, so it could
> > go either way, I suppose.
> 
> If this were a piece of code that _everybody_ would use on _all_ the
> supported platforms, I would suggest declaring that this is a
> weather-balloon to see if some platforms have trouble using it.  But
> unfortunately this is not such a piece of code.  Dependence on
> systemd should strictly be opt-in.

Yes, dependence on systemd should be strictly opt-in.
Although, I don't use systemd-based distro, so it is irrelevant to me.
I think it's none of Git (the project) business to decide which
scheduler should be given higher priority. It's crontab when
maintenance was introduced, it should be crontab, now.

Another point for eternal bikeshedding: why do we limit ourselves in
crontab and systemd, how about other homebrew schedulers? What should
we do if another scheduler raise to be the big star in the scheduler
world?

I guess we should take some templates for running on {,un}register
instead? However, I think such design may open another can of worms.
So, I don't know.


-- 
Danh

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v2 1/1] maintenance: use systemd timers on Linux
  2021-05-12  0:29           ` Đoàn Trần Công Danh
@ 2021-05-12  6:59             ` Felipe Contreras
  2021-05-12 13:26               ` Phillip Wood
  2021-05-12 13:38             ` Phillip Wood
  1 sibling, 1 reply; 138+ messages in thread
From: Felipe Contreras @ 2021-05-12  6:59 UTC (permalink / raw)
  To: Đoàn Trần Công Danh, Junio C Hamano
  Cc: Eric Sunshine, Lénaïc Huard, Git List, Derrick Stolee,
	brian m . carlson, Bagas Sanjaya, Phillip Wood,
	Ævar Arnfjörð Bjarmason

Đoàn Trần Công Danh wrote:
> Yes, dependence on systemd should be strictly opt-in.
> Although, I don't use systemd-based distro, so it is irrelevant to me.
> I think it's none of Git (the project) business to decide which
> scheduler should be given higher priority. It's crontab when
> maintenance was introduced, it should be crontab, now.

I do use a systemd-based distro, and I like the option to use systemd
units, but let's be honest...

100% of systems with systemd have cron... So...

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v2 1/1] maintenance: use systemd timers on Linux
  2021-05-12  6:59             ` Felipe Contreras
@ 2021-05-12 13:26               ` Phillip Wood
  0 siblings, 0 replies; 138+ messages in thread
From: Phillip Wood @ 2021-05-12 13:26 UTC (permalink / raw)
  To: Felipe Contreras, Đoàn Trần Công Danh,
	Junio C Hamano
  Cc: Eric Sunshine, Lénaïc Huard, Git List, Derrick Stolee,
	brian m . carlson, Bagas Sanjaya,
	Ævar Arnfjörð Bjarmason

On 12/05/2021 07:59, Felipe Contreras wrote:
> Đoàn Trần Công Danh wrote:
>> Yes, dependence on systemd should be strictly opt-in.
>> Although, I don't use systemd-based distro, so it is irrelevant to me.
>> I think it's none of Git (the project) business to decide which
>> scheduler should be given higher priority. It's crontab when
>> maintenance was introduced, it should be crontab, now.
> 
> I do use a systemd-based distro, and I like the option to use systemd
> units, but let's be honest...
> 
> 100% of systems with systemd have cron...

This is untrue, as the commit message points out cron is optional on 
systems running systemd and there are distributions such as Arch Linux 
that do not install a cron daemon without explicit user intervention.

  So...
> 

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v2 1/1] maintenance: use systemd timers on Linux
  2021-05-12  0:29           ` Đoàn Trần Công Danh
  2021-05-12  6:59             ` Felipe Contreras
@ 2021-05-12 13:38             ` Phillip Wood
  2021-05-12 15:41               ` Đoàn Trần Công Danh
  1 sibling, 1 reply; 138+ messages in thread
From: Phillip Wood @ 2021-05-12 13:38 UTC (permalink / raw)
  To: Đoàn Trần Công Danh, Junio C Hamano
  Cc: Eric Sunshine, Lénaïc Huard, Git List, Derrick Stolee,
	brian m . carlson, Bagas Sanjaya,
	Ævar Arnfjörð Bjarmason

Hi Đoàn

On 12/05/2021 01:29, Đoàn Trần Công Danh wrote:
> On 2021-05-10 15:25:07+0900, Junio C Hamano <gitster@pobox.com> wrote:
>> Eric Sunshine <sunshine@sunshineco.com> writes:
>>
>>>> I think others have strong opinion on not using "%1$s",
>>>> and prefer simple "%s" and using "exec_path" twice instead.
>>>
>>> I brought it up only because I hadn't seen it in Git sources, and
>>> wasn't sure if we'd want to start using it. Aside from Ævar, who
>>> seemed reasonably in favor of it, nobody else chimed in, so it could
>>> go either way, I suppose.
>>
>> If this were a piece of code that _everybody_ would use on _all_ the
>> supported platforms, I would suggest declaring that this is a
>> weather-balloon to see if some platforms have trouble using it.  But
>> unfortunately this is not such a piece of code.  Dependence on
>> systemd should strictly be opt-in.
> 
> Yes, dependence on systemd should be strictly opt-in.
> Although, I don't use systemd-based distro, so it is irrelevant to me.
> I think it's none of Git (the project) business to decide which
> scheduler should be given higher priority. It's crontab when
> maintenance was introduced, it should be crontab, now.

You seem to be simultaneously arguing that git should be neutral on the 
choice of scheduler while saying it should prioritize crontab. The 
commit message and cover letter list a number of difficulties with the 
strategy of prioritizing crontab over systemd when both are installed. I 
think we should aim for the solution that has the most chance of working 
without user intervention.

> Another point for eternal bikeshedding: why do we limit ourselves in
> crontab and systemd, how about other homebrew schedulers? What should
> we do if another scheduler raise to be the big star in the scheduler
> world?

We should support the default scheduler on each platform - that was the 
rod we made for our own back when we decided to use the platform's 
scheduler rather than having a cross platform git maintenance daemon. It 
just happens that there are two possible default schedulers on linux so 
we need to support both of them.

Best Wishes

Phillip

> I guess we should take some templates for running on {,un}register
> instead? However, I think such design may open another can of worms.
> So, I don't know.
> 
> 

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v2 1/1] maintenance: use systemd timers on Linux
  2021-05-12 13:38             ` Phillip Wood
@ 2021-05-12 15:41               ` Đoàn Trần Công Danh
  0 siblings, 0 replies; 138+ messages in thread
From: Đoàn Trần Công Danh @ 2021-05-12 15:41 UTC (permalink / raw)
  To: phillip.wood
  Cc: Junio C Hamano, Eric Sunshine, Lénaïc Huard, Git List,
	Derrick Stolee, brian m . carlson, Bagas Sanjaya,
	Ævar Arnfjörð Bjarmason

On 2021-05-12 14:38:26+0100, Phillip Wood <phillip.wood123@gmail.com> wrote:
> > Yes, dependence on systemd should be strictly opt-in.
> > Although, I don't use systemd-based distro, so it is irrelevant to me.
> > I think it's none of Git (the project) business to decide which
> > scheduler should be given higher priority. It's crontab when
> > maintenance was introduced, it should be crontab, now.
> 
> You seem to be simultaneously arguing that git should be neutral on the
> choice of scheduler while saying it should prioritize crontab.

Yes, I'm arguing for git should be neutral on the choice of scheduler.

No, I'm not arguing for git should be prioritize crontab, I'm arguing
for "princible of least surprise" for no known break-through advantage.

FWIW, whatever default scheduler chosen won't affect me, since I don't
have systemd-timers to begin with. So ...

In addition, I was one of those people pointed out that beside
crontab, Linux users nowaday employed different schedulers [1],
and the consensus, some how, settled on crontab.

I think  we shouldn't switch away from crontab if we don't have any
compelling reasons.

> The commit
> message and cover letter list a number of difficulties with the strategy of
> prioritizing crontab over systemd when both are installed. I think we should
> aim for the solution that has the most chance of working without user
> intervention.

The solution that has the most chance of working without user
intervention is the solution that is the status quo. Promoting
systemd-timers to higher priority is a solution requires either
user intervention or our supports (which will be carried over our
lifetime).

> > Another point for eternal bikeshedding: why do we limit ourselves in
> > crontab and systemd, how about other homebrew schedulers? What should
> > we do if another scheduler raise to be the big star in the scheduler
> > world?
> 
> We should support the default scheduler on each platform - that was the rod
> we made for our own back when we decided to use the platform's scheduler
> rather than having a cross platform git maintenance daemon. It just happens
> that there are two possible default schedulers on linux so we need to
> support both of them.

As noted in [1], some home-brew solutions are very popular solutions
among those some community.
I'm not arguing that crontab or systemd-timers aren't popular.
In fact, I think they're *very* popular, I listed systemd-timers as
*first* alternative in the linked email.
I'm not against supporting both of them, I was arguing about a generic
solution.

1: https://lore.kernel.org/git/20200407005828.GC2568@danh.dev/

-- 
Danh

^ permalink raw reply	[flat|nested] 138+ messages in thread

* [PATCH v3 0/4] maintenance: use systemd timers on Linux
  2021-05-09 21:32 ` [PATCH v2 0/1] " Lénaïc Huard
  2021-05-09 21:32   ` [PATCH v2 1/1] " Lénaïc Huard
@ 2021-05-20 22:13   ` Lénaïc Huard
  2021-05-20 22:13     ` [PATCH v3 1/4] cache.h: rename "xdg_config_home" to "xdg_config_home_git" Lénaïc Huard
                       ` (4 more replies)
  1 sibling, 5 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-05-20 22:13 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Lénaïc Huard

Hello,

Thanks again for your valuable feedback!
I’ve reworked this submission to hopefully address all the raised
concerns.

As it’s becoming bigger, I’ve split it in several patches:

* cache.h: rename "xdg_config_home" to "xdg_config_home_git"

In order to honor `$XDG_CONFIG_HOME` to built the path of systemd units,
a lot of code from the `xdg_config_home()` function was
duplicated. This existing function couldn’t be used as-is because
it was hard-coding the `git` sub-directory below `$XDG_CONFIG_HOME`.

This first preparatory patch re-purposes `xdg_config_home()` to make
it more generic by removing the `git` path it used to append to
`$XDG_CONFIG_HOME`.
Thanks to this refactoring, `xdg_config_home()` can be used to build
paths like:
* `$XDG_CONFIG_HOME/git/…` it’s the new `xdg_config_home_git()`
function and
* `$XDG_CONFIG_HOME/systemd/user/…` it’s the new
`xdg_config_home_systemd()` function which will be introduced later,
in the last patch of this series.

* maintenance: introduce ENABLE/DISABLE for code clarity

Some functions in git maintenance are doing one thing and the exact
opposite, depending on a parameter that could be `0` or `1`.
This could sometimes be confusing.
This patch introduces an enum with `ENABLE` and `DISABLE` values to
make the code more explicit.

* maintenance: `git maintenance run` learned `--scheduler=<scheduler>`

This patch contains all the code that is related to the addition of
the new `--scheduler` parameter of the `git maintenance start`
command, independently of the systemd timers.

This part has significantly changed to review the testing mechanisms.
The `GIT_TEST_MAINT_SCHEDULER` environment variable can now take a
comma separated list of schedulers and their mock in order to be able
to test cases where multiple schedulers are available.

It allows to test that `git maintenance start --scheduler=XXX` not
only enables the scheduler XXX, but it also disables all the other
available ones.
We can also test that `git maintenance stop`, stops all the available
schedulers.

In order to remove the `cmd` parameter that was passed to a lot of
functions, the parsing of `GIT_TEST_MAINT_SCHEDULER` is now factorized
in a `get_schedule_cmd()` function that is invoked from the leaves of
the function call tree.
It means that this function is called many times to parse
`GIT_TEST_MAINT_SCHEDULER` again and again.
It is inefficient but as it is used for test only, it shouldn’t be a
concern.

`int get_schdule_cmd(const char **cmd, int *is_available)`

This function returns `true` if `GIT_TEST_MAINT_SCHEDULER` is
defined. `false` otherwise.

`*is_available` is set to `true` if `*cmd` is present in
`$GIT_TEST_MAINT_SCHEDULER`, and to `false` otherwise.

If `*cmd` is present in `$GIT_TEST_MAINT_SCHEDULER`, it’s value is
updated to its mock.

* maintenance: optionally use systemd timers on Linux

This last patch adds the support for systemd timers.

Lénaïc Huard (4):
  cache.h: rename "xdg_config_home" to "xdg_config_home_git"
  maintenance: introduce ENABLE/DISABLE for code clarity
  maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  maintenance: optionally use systemd timers on Linux

 Documentation/git-maintenance.txt |  60 ++++
 attr.c                            |   2 +-
 builtin/config.c                  |   2 +-
 builtin/credential-cache.c        |   2 +-
 builtin/credential-store.c        |   2 +-
 builtin/gc.c                      | 549 ++++++++++++++++++++++++++----
 cache.h                           |  12 +-
 config.c                          |   2 +-
 dir.c                             |   2 +-
 path.c                            |  35 +-
 sequencer.c                       |   2 +-
 t/t7900-maintenance.sh            | 110 +++++-
 12 files changed, 685 insertions(+), 95 deletions(-)

-- 
2.31.1


^ permalink raw reply	[flat|nested] 138+ messages in thread

* [PATCH v3 1/4] cache.h: rename "xdg_config_home" to "xdg_config_home_git"
  2021-05-20 22:13   ` [PATCH v3 0/4] " Lénaïc Huard
@ 2021-05-20 22:13     ` Lénaïc Huard
  2021-05-20 23:44       ` Đoàn Trần Công Danh
  2021-05-20 22:13     ` [PATCH v3 2/4] maintenance: introduce ENABLE/DISABLE for code clarity Lénaïc Huard
                       ` (3 subsequent siblings)
  4 siblings, 1 reply; 138+ messages in thread
From: Lénaïc Huard @ 2021-05-20 22:13 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Lénaïc Huard

Current implementation of `xdg_config_home(str)` returns
`$XDG_CONFIG_HOME/git/$str`, with the `git` subdirectory inserted
between the `XDG_CONFIG_HOME` environment variable and the parameter.

This patch re-purposes `xdg_config_home(…)` to be more generic. It now
only concatenates "$XDG_CONFIG_HOME", or "$HOME/.config" if the former
isn’t defined, with the parameter, without adding `git` in between.
Its parameter is now a format string.

The previous functionality is now provided by a new
`xdg_config_home_git(…)` function whose implementation leverages
`xdg_config_home(…)`.

`xdg_cache_home(…)` has been renamed `xdg_cache_home_git(…)` for
consistency.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 attr.c                     |  2 +-
 builtin/config.c           |  2 +-
 builtin/credential-cache.c |  2 +-
 builtin/credential-store.c |  2 +-
 cache.h                    | 12 ++++++++++--
 config.c                   |  2 +-
 dir.c                      |  2 +-
 path.c                     | 35 +++++++++++++++++++++++++++--------
 sequencer.c                |  2 +-
 9 files changed, 44 insertions(+), 17 deletions(-)

diff --git a/attr.c b/attr.c
index ac8ec7ce51..0efafbd308 100644
--- a/attr.c
+++ b/attr.c
@@ -831,7 +831,7 @@ static const char *git_etc_gitattributes(void)
 static const char *get_home_gitattributes(void)
 {
 	if (!git_attributes_file)
-		git_attributes_file = xdg_config_home("attributes");
+		git_attributes_file = xdg_config_home_git("attributes");
 
 	return git_attributes_file;
 }
diff --git a/builtin/config.c b/builtin/config.c
index f71fa39b38..8cf2394eb8 100644
--- a/builtin/config.c
+++ b/builtin/config.c
@@ -672,7 +672,7 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 
 	if (use_global_config) {
 		char *user_config = expand_user_path("~/.gitconfig", 0);
-		char *xdg_config = xdg_config_home("config");
+		char *xdg_config = xdg_config_home_git("config");
 
 		if (!user_config)
 			/*
diff --git a/builtin/credential-cache.c b/builtin/credential-cache.c
index 76a6ba3722..4c0b7c4d43 100644
--- a/builtin/credential-cache.c
+++ b/builtin/credential-cache.c
@@ -94,7 +94,7 @@ static char *get_socket_path(void)
 	if (old_dir && !stat(old_dir, &sb) && S_ISDIR(sb.st_mode))
 		socket = xstrfmt("%s/socket", old_dir);
 	else
-		socket = xdg_cache_home("credential/socket");
+		socket = xdg_cache_home_git("credential/socket");
 	free(old_dir);
 	return socket;
 }
diff --git a/builtin/credential-store.c b/builtin/credential-store.c
index ae3c1ba75f..34ca419bb6 100644
--- a/builtin/credential-store.c
+++ b/builtin/credential-store.c
@@ -175,7 +175,7 @@ int cmd_credential_store(int argc, const char **argv, const char *prefix)
 	} else {
 		if ((file = expand_user_path("~/.git-credentials", 0)))
 			string_list_append_nodup(&fns, file);
-		file = xdg_config_home("credentials");
+		file = xdg_config_home_git("credentials");
 		if (file)
 			string_list_append_nodup(&fns, file);
 	}
diff --git a/cache.h b/cache.h
index 148d9ab5f1..34fa48a438 100644
--- a/cache.h
+++ b/cache.h
@@ -1263,19 +1263,27 @@ int is_ntfs_dotgitattributes(const char *name);
  */
 int looks_like_command_line_option(const char *str);
 
+/**
+ * Return a newly allocated string with the evaluation of
+ * "$XDG_CONFIG_HOME/$fmt..." if $XDG_CONFIG_HOME is non-empty, otherwise
+ * "$HOME/.config/$fmt...". Return NULL upon error.
+ */
+char *xdg_config_home(const char *fmt, ...)
+	__attribute__((format (printf, 1, 2)));
+
 /**
  * Return a newly allocated string with the evaluation of
  * "$XDG_CONFIG_HOME/git/$filename" if $XDG_CONFIG_HOME is non-empty, otherwise
  * "$HOME/.config/git/$filename". Return NULL upon error.
  */
-char *xdg_config_home(const char *filename);
+char *xdg_config_home_git(const char *filename);
 
 /**
  * Return a newly allocated string with the evaluation of
  * "$XDG_CACHE_HOME/git/$filename" if $XDG_CACHE_HOME is non-empty, otherwise
  * "$HOME/.cache/git/$filename". Return NULL upon error.
  */
-char *xdg_cache_home(const char *filename);
+char *xdg_cache_home_git(const char *filename);
 
 int git_open_cloexec(const char *name, int flags);
 #define git_open(name) git_open_cloexec(name, O_RDONLY)
diff --git a/config.c b/config.c
index 6428393a41..648ae9b918 100644
--- a/config.c
+++ b/config.c
@@ -1883,7 +1883,7 @@ static int do_git_config_sequence(const struct config_options *opts,
 				  config_fn_t fn, void *data)
 {
 	int ret = 0;
-	char *xdg_config = xdg_config_home("config");
+	char *xdg_config = xdg_config_home_git("config");
 	char *user_config = expand_user_path("~/.gitconfig", 0);
 	char *repo_config;
 	enum config_scope prev_parsing_scope = current_parsing_scope;
diff --git a/dir.c b/dir.c
index 3474e67e8f..3fdba7b6fe 100644
--- a/dir.c
+++ b/dir.c
@@ -2990,7 +2990,7 @@ void setup_standard_excludes(struct dir_struct *dir)
 
 	/* core.excludesfile defaulting to $XDG_CONFIG_HOME/git/ignore */
 	if (!excludes_file)
-		excludes_file = xdg_config_home("ignore");
+		excludes_file = xdg_config_home_git("ignore");
 	if (excludes_file && !access_or_warn(excludes_file, R_OK, 0))
 		add_patterns_from_file_1(dir, excludes_file,
 					 dir->untracked ? &dir->ss_excludes_file : NULL);
diff --git a/path.c b/path.c
index 7b385e5eb2..15e2143e9f 100644
--- a/path.c
+++ b/path.c
@@ -1498,22 +1498,41 @@ int looks_like_command_line_option(const char *str)
 	return str && str[0] == '-';
 }
 
-char *xdg_config_home(const char *filename)
+char *xdg_config_home(const char *fmt, ...)
 {
 	const char *home, *config_home;
+	struct strbuf buf = STRBUF_INIT;
+	char *out = NULL;
+	va_list args;
+
+	va_start(args, fmt);
+	strbuf_vaddf(&buf, fmt, args);
+	va_end(args);
 
-	assert(filename);
 	config_home = getenv("XDG_CONFIG_HOME");
-	if (config_home && *config_home)
-		return mkpathdup("%s/git/%s", config_home, filename);
+	if (config_home && *config_home) {
+		out = mkpathdup("%s/%s", config_home, buf.buf);
+		goto done;
+	}
 
 	home = getenv("HOME");
-	if (home)
-		return mkpathdup("%s/.config/git/%s", home, filename);
-	return NULL;
+	if (home) {
+		out = mkpathdup("%s/.config/%s", home, buf.buf);
+		goto done;
+	}
+
+done:
+	strbuf_release(&buf);
+	return out;
+}
+
+char *xdg_config_home_git(const char *filename)
+{
+	assert(filename);
+	return xdg_config_home("git/%s", filename);
 }
 
-char *xdg_cache_home(const char *filename)
+char *xdg_cache_home_git(const char *filename)
 {
 	const char *home, *cache_home;
 
diff --git a/sequencer.c b/sequencer.c
index fd183b5593..25f467e685 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1241,7 +1241,7 @@ N_("Your name and email address were configured automatically based\n"
 static const char *implicit_ident_advice(void)
 {
 	char *user_config = expand_user_path("~/.gitconfig", 0);
-	char *xdg_config = xdg_config_home("config");
+	char *xdg_config = xdg_config_home_git("config");
 	int config_exists = file_exists(user_config) || file_exists(xdg_config);
 
 	free(user_config);
-- 
2.31.1


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* [PATCH v3 2/4] maintenance: introduce ENABLE/DISABLE for code clarity
  2021-05-20 22:13   ` [PATCH v3 0/4] " Lénaïc Huard
  2021-05-20 22:13     ` [PATCH v3 1/4] cache.h: rename "xdg_config_home" to "xdg_config_home_git" Lénaïc Huard
@ 2021-05-20 22:13     ` Lénaïc Huard
  2021-05-20 22:13     ` [PATCH v3 3/4] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
                       ` (2 subsequent siblings)
  4 siblings, 0 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-05-20 22:13 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Lénaïc Huard

The first parameter of `XXX_update_schedule` and alike functions is a
boolean specifying if the tasks should be scheduled or unscheduled.

Using an `enum` with `ENABLE` and `DISABLE` values can make the code
clearer.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 builtin/gc.c | 49 +++++++++++++++++++++++++++++++------------------
 1 file changed, 31 insertions(+), 18 deletions(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index ef7226d7bc..0caf8d45c4 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1570,19 +1570,21 @@ static char *launchctl_get_uid(void)
 	return xstrfmt("gui/%d", getuid());
 }
 
-static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
+enum enable_or_disable {
+	DISABLE,
+	ENABLE
+};
+
+static int launchctl_boot_plist(enum enable_or_disable enable,
+				const char *filename, const char *cmd)
 {
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
 	char *uid = launchctl_get_uid();
 
 	strvec_split(&child.args, cmd);
-	if (enable)
-		strvec_push(&child.args, "bootstrap");
-	else
-		strvec_push(&child.args, "bootout");
-	strvec_push(&child.args, uid);
-	strvec_push(&child.args, filename);
+	strvec_pushl(&child.args, enable == ENABLE ? "bootstrap" : "bootout",
+		     uid, filename, NULL);
 
 	child.no_stderr = 1;
 	child.no_stdout = 1;
@@ -1601,7 +1603,7 @@ static int launchctl_remove_plist(enum schedule_priority schedule, const char *c
 	const char *frequency = get_frequency(schedule);
 	char *name = launchctl_service_name(frequency);
 	char *filename = launchctl_service_filename(name);
-	int result = launchctl_boot_plist(0, filename, cmd);
+	int result = launchctl_boot_plist(DISABLE, filename, cmd);
 	unlink(filename);
 	free(filename);
 	free(name);
@@ -1684,8 +1686,8 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 	fclose(plist);
 
 	/* bootout might fail if not already running, so ignore */
-	launchctl_boot_plist(0, filename, cmd);
-	if (launchctl_boot_plist(1, filename, cmd))
+	launchctl_boot_plist(DISABLE, filename, cmd);
+	if (launchctl_boot_plist(ENABLE, filename, cmd))
 		die(_("failed to bootstrap service %s"), filename);
 
 	free(filename);
@@ -1702,12 +1704,17 @@ static int launchctl_add_plists(const char *cmd)
 		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
 }
 
-static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int launchctl_update_schedule(enum enable_or_disable run_maintenance,
+				     int fd, const char *cmd)
 {
-	if (run_maintenance)
+	switch (run_maintenance) {
+	case ENABLE:
 		return launchctl_add_plists(cmd);
-	else
+	case DISABLE:
 		return launchctl_remove_plists(cmd);
+	default:
+		BUG("invalid enable_or_disable value");
+	}
 }
 
 static char *schtasks_task_name(const char *frequency)
@@ -1864,18 +1871,24 @@ static int schtasks_schedule_tasks(const char *cmd)
 		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
 }
 
-static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int schtasks_update_schedule(enum enable_or_disable run_maintenance,
+				    int fd, const char *cmd)
 {
-	if (run_maintenance)
+	switch (run_maintenance) {
+	case ENABLE:
 		return schtasks_schedule_tasks(cmd);
-	else
+	case DISABLE:
 		return schtasks_remove_tasks(cmd);
+	default:
+		BUG("invalid enable_or_disable value");
+	}
 }
 
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
-static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int crontab_update_schedule(enum enable_or_disable run_maintenance,
+				   int fd, const char *cmd)
 {
 	int result = 0;
 	int in_old_region = 0;
@@ -1925,7 +1938,7 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 			fprintf(cron_in, "%s\n", line.buf);
 	}
 
-	if (run_maintenance) {
+	if (run_maintenance == ENABLE) {
 		struct strbuf line_format = STRBUF_INIT;
 		const char *exec_path = git_exec_path();
 
-- 
2.31.1


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* [PATCH v3 3/4] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-05-20 22:13   ` [PATCH v3 0/4] " Lénaïc Huard
  2021-05-20 22:13     ` [PATCH v3 1/4] cache.h: rename "xdg_config_home" to "xdg_config_home_git" Lénaïc Huard
  2021-05-20 22:13     ` [PATCH v3 2/4] maintenance: introduce ENABLE/DISABLE for code clarity Lénaïc Huard
@ 2021-05-20 22:13     ` Lénaïc Huard
  2021-05-21  9:52       ` Bagas Sanjaya
  2021-05-20 22:13     ` [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux Lénaïc Huard
  2021-05-24  7:15     ` [PATCH v4 0/4] add support for " Lénaïc Huard
  4 siblings, 1 reply; 138+ messages in thread
From: Lénaïc Huard @ 2021-05-20 22:13 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Lénaïc Huard

Depending on the system, different schedulers can be used to schedule
the hourly, daily and weekly executions of `git maintenance run`:
* `launchctl` for MacOS,
* `schtasks` for Windows and
* `crontab` for everything else.

`git maintenance run` now has an option to let the end-user explicitly
choose which scheduler he wants to use:
`--scheduler=auto|crontab|launchctl|schtasks`.

When `git maintenance start --scheduler=XXX` is run, it not only
registers `git maintenance run` tasks in the scheduler XXX, it also
removes the `git maintenance run` tasks from all the other schedulers to
ensure we cannot have two schedulers launching concurrent identical
tasks.

The default value is `auto` which chooses a suitable scheduler for the
system.

`git maintenance stop` doesn't have any `--scheduler` parameter because
this command will try to remove the `git maintenance run` tasks from all
the available schedulers.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 Documentation/git-maintenance.txt |  11 +
 builtin/gc.c                      | 333 ++++++++++++++++++++++++------
 t/t7900-maintenance.sh            |  56 ++++-
 3 files changed, 333 insertions(+), 67 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 80ddd33ceb..7c4bb38a2f 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -181,6 +181,17 @@ OPTIONS
 	`maintenance.<task>.enabled` configured as `true` are considered.
 	See the 'TASKS' section for the list of accepted `<task>` values.
 
+--scheduler=auto|crontab|launchctl|schtasks::
+	When combined with the `start` subcommand, specify the scheduler
+	to use to run the hourly, daily and weekly executions of
+	`git maintenance run`.
+	The possible values for `<scheduler>` depend on the system: `crontab`
+	is available on POSIX systems, `launchctl` is available on
+	MacOS and `schtasks` is available on Windows.
+	By default or when `auto` is specified, a suitable scheduler for
+	the system is used. On MacOS, `launchctl` is used. On Windows,
+	`schtasks` is used. On all other systems, `crontab` is used.
+
 
 TROUBLESHOOTING
 ---------------
diff --git a/builtin/gc.c b/builtin/gc.c
index 0caf8d45c4..bf21cec059 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1544,6 +1544,60 @@ static const char *get_frequency(enum schedule_priority schedule)
 	}
 }
 
+static int get_schedule_cmd(const char **cmd, int *is_available)
+{
+	char *item;
+	static char test_cmd[32];
+	char *testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
+
+	if (!testing)
+		return 0;
+
+	if (is_available)
+		*is_available = 0;
+
+	for(item = testing;;) {
+		char *sep;
+		char *end_item = strchr(item, ',');
+		if (end_item)
+			*end_item = '\0';
+
+		sep = strchr(item, ':');
+		if (!sep)
+			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
+		*sep = '\0';
+
+		if (!strcmp(*cmd, item)) {
+			strlcpy(test_cmd, sep+1, ARRAY_SIZE(test_cmd));
+			*cmd = test_cmd;
+			if (is_available)
+				*is_available = 1;
+			break;
+		}
+
+		if (!end_item)
+			break;
+		item = end_item + 1;
+	}
+
+	free(testing);
+	return 1;
+}
+
+static int is_launchctl_available(void)
+{
+	const char *cmd = "launchctl";
+	int is_available;
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+#ifdef __APPLE__
+	return 1;
+#else
+	return 0;
+#endif
+}
+
 static char *launchctl_service_name(const char *frequency)
 {
 	struct strbuf label = STRBUF_INIT;
@@ -1576,12 +1630,14 @@ enum enable_or_disable {
 };
 
 static int launchctl_boot_plist(enum enable_or_disable enable,
-				const char *filename, const char *cmd)
+				const char *filename)
 {
+	const char *cmd = "launchctl";
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
 	char *uid = launchctl_get_uid();
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&child.args, cmd);
 	strvec_pushl(&child.args, enable == ENABLE ? "bootstrap" : "bootout",
 		     uid, filename, NULL);
@@ -1598,26 +1654,26 @@ static int launchctl_boot_plist(enum enable_or_disable enable,
 	return result;
 }
 
-static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)
+static int launchctl_remove_plist(enum schedule_priority schedule)
 {
 	const char *frequency = get_frequency(schedule);
 	char *name = launchctl_service_name(frequency);
 	char *filename = launchctl_service_filename(name);
-	int result = launchctl_boot_plist(DISABLE, filename, cmd);
+	int result = launchctl_boot_plist(DISABLE, filename);
 	unlink(filename);
 	free(filename);
 	free(name);
 	return result;
 }
 
-static int launchctl_remove_plists(const char *cmd)
+static int launchctl_remove_plists(void)
 {
-	return launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||
-		launchctl_remove_plist(SCHEDULE_DAILY, cmd) ||
-		launchctl_remove_plist(SCHEDULE_WEEKLY, cmd);
+	return launchctl_remove_plist(SCHEDULE_HOURLY) ||
+		launchctl_remove_plist(SCHEDULE_DAILY) ||
+		launchctl_remove_plist(SCHEDULE_WEEKLY);
 }
 
-static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule)
 {
 	FILE *plist;
 	int i;
@@ -1686,8 +1742,8 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 	fclose(plist);
 
 	/* bootout might fail if not already running, so ignore */
-	launchctl_boot_plist(DISABLE, filename, cmd);
-	if (launchctl_boot_plist(ENABLE, filename, cmd))
+	launchctl_boot_plist(DISABLE, filename);
+	if (launchctl_boot_plist(ENABLE, filename))
 		die(_("failed to bootstrap service %s"), filename);
 
 	free(filename);
@@ -1695,28 +1751,42 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 	return 0;
 }
 
-static int launchctl_add_plists(const char *cmd)
+static int launchctl_add_plists(void)
 {
 	const char *exec_path = git_exec_path();
 
-	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||
-		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||
-		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
+	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY) ||
+		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY) ||
+		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY);
 }
 
 static int launchctl_update_schedule(enum enable_or_disable run_maintenance,
-				     int fd, const char *cmd)
+				     int fd)
 {
 	switch (run_maintenance) {
 	case ENABLE:
-		return launchctl_add_plists(cmd);
+		return launchctl_add_plists();
 	case DISABLE:
-		return launchctl_remove_plists(cmd);
+		return launchctl_remove_plists();
 	default:
 		BUG("invalid enable_or_disable value");
 	}
 }
 
+static int is_schtasks_available(void)
+{
+	const char *cmd = "schtasks";
+	int is_available;
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+#ifdef GIT_WINDOWS_NATIVE
+	return 1;
+#else
+	return 0;
+#endif
+}
+
 static char *schtasks_task_name(const char *frequency)
 {
 	struct strbuf label = STRBUF_INIT;
@@ -1724,13 +1794,15 @@ static char *schtasks_task_name(const char *frequency)
 	return strbuf_detach(&label, NULL);
 }
 
-static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)
+static int schtasks_remove_task(enum schedule_priority schedule)
 {
+	const char *cmd = "schtasks";
 	int result;
 	struct strvec args = STRVEC_INIT;
 	const char *frequency = get_frequency(schedule);
 	char *name = schtasks_task_name(frequency);
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&args, cmd);
 	strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
 
@@ -1741,15 +1813,16 @@ static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd
 	return result;
 }
 
-static int schtasks_remove_tasks(const char *cmd)
+static int schtasks_remove_tasks(void)
 {
-	return schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||
-		schtasks_remove_task(SCHEDULE_DAILY, cmd) ||
-		schtasks_remove_task(SCHEDULE_WEEKLY, cmd);
+	return schtasks_remove_task(SCHEDULE_HOURLY) ||
+		schtasks_remove_task(SCHEDULE_DAILY) ||
+		schtasks_remove_task(SCHEDULE_WEEKLY);
 }
 
-static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule)
 {
+	const char *cmd = "schtasks";
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
 	const char *xml;
@@ -1758,6 +1831,8 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
 	char *name = schtasks_task_name(frequency);
 	struct strbuf tfilename = STRBUF_INIT;
 
+	get_schedule_cmd(&cmd, NULL);
+
 	strbuf_addf(&tfilename, "%s/schedule_%s_XXXXXX",
 		    get_git_common_dir(), frequency);
 	tfile = xmks_tempfile(tfilename.buf);
@@ -1862,34 +1937,65 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
 	return result;
 }
 
-static int schtasks_schedule_tasks(const char *cmd)
+static int schtasks_schedule_tasks(void)
 {
 	const char *exec_path = git_exec_path();
 
-	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||
-		schtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||
-		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
+	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY) ||
+		schtasks_schedule_task(exec_path, SCHEDULE_DAILY) ||
+		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY);
 }
 
 static int schtasks_update_schedule(enum enable_or_disable run_maintenance,
-				    int fd, const char *cmd)
+				    int fd)
 {
 	switch (run_maintenance) {
 	case ENABLE:
-		return schtasks_schedule_tasks(cmd);
+		return schtasks_schedule_tasks();
 	case DISABLE:
-		return schtasks_remove_tasks(cmd);
+		return schtasks_remove_tasks();
 	default:
 		BUG("invalid enable_or_disable value");
 	}
 }
 
+static int is_crontab_available(void)
+{
+	const char *cmd = "crontab";
+	int is_available;
+	static int cached_result = -1;
+	struct child_process child = CHILD_PROCESS_INIT;
+
+	if (cached_result != -1)
+		return cached_result;
+
+	if (get_schedule_cmd(&cmd, &is_available) && !is_available)
+		return 0;
+
+	strvec_split(&child.args, cmd);
+	strvec_push(&child.args, "-l");
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child)) {
+		cached_result = 0;
+		return cached_result;
+	}
+	/* Ignore exit code, as an empty crontab will return error. */
+	finish_command(&child);
+	cached_result = 1;
+	return cached_result;
+}
+
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
 static int crontab_update_schedule(enum enable_or_disable run_maintenance,
-				   int fd, const char *cmd)
+				   int fd)
 {
+	const char *cmd = "crontab";
 	int result = 0;
 	int in_old_region = 0;
 	struct child_process crontab_list = CHILD_PROCESS_INIT;
@@ -1897,6 +2003,7 @@ static int crontab_update_schedule(enum enable_or_disable run_maintenance,
 	FILE *cron_list, *cron_in;
 	struct strbuf line = STRBUF_INIT;
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&crontab_list.args, cmd);
 	strvec_push(&crontab_list.args, "-l");
 	crontab_list.in = -1;
@@ -1972,61 +2079,161 @@ static int crontab_update_schedule(enum enable_or_disable run_maintenance,
 	return result;
 }
 
+enum scheduler {
+	SCHEDULER_INVALID = -1,
+	SCHEDULER_AUTO,
+	SCHEDULER_CRON,
+	SCHEDULER_LAUNCHCTL,
+	SCHEDULER_SCHTASKS,
+};
+
+static const struct {
+	const char *name;
+	int (*is_available)(void);
+	int (*update_schedule)(enum enable_or_disable run_maintenance, int fd);
+} scheduler_fn[] = {
+	[SCHEDULER_CRON] = {
+		.name = "crontab",
+		.is_available = is_crontab_available,
+		.update_schedule = crontab_update_schedule,
+	},
+	[SCHEDULER_LAUNCHCTL] = {
+		.name = "launchctl",
+		.is_available = is_launchctl_available,
+		.update_schedule = launchctl_update_schedule,
+	},
+	[SCHEDULER_SCHTASKS] = {
+		.name = "schtasks",
+		.is_available = is_schtasks_available,
+		.update_schedule = schtasks_update_schedule,
+	},
+};
+
+static enum scheduler parse_scheduler(const char *value)
+{
+	if (!value)
+		return SCHEDULER_INVALID;
+	else if (!strcasecmp(value, "auto"))
+		return SCHEDULER_AUTO;
+	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
+		return SCHEDULER_CRON;
+	else if (!strcasecmp(value, "launchctl"))
+		return SCHEDULER_LAUNCHCTL;
+	else if (!strcasecmp(value, "schtasks"))
+		return SCHEDULER_SCHTASKS;
+	else
+		return SCHEDULER_INVALID;
+}
+
+static int maintenance_opt_scheduler(const struct option *opt, const char *arg,
+				     int unset)
+{
+	enum scheduler *scheduler = opt->value;
+
+	BUG_ON_OPT_NEG(unset);
+
+	*scheduler = parse_scheduler(arg);
+	if (*scheduler == SCHEDULER_INVALID)
+		return error(_("unrecognized --scheduler argument '%s'"), arg);
+	return 0;
+}
+
+struct maintenance_start_opts {
+	enum scheduler scheduler;
+};
+
+static void resolve_auto_scheduler(enum scheduler *scheduler)
+{
+	if (*scheduler != SCHEDULER_AUTO)
+		return;
+
 #if defined(__APPLE__)
-static const char platform_scheduler[] = "launchctl";
+	*scheduler = SCHEDULER_LAUNCHCTL;
+	return;
+
 #elif defined(GIT_WINDOWS_NATIVE)
-static const char platform_scheduler[] = "schtasks";
+	*scheduler = SCHEDULER_SCHTASKS;
+	return;
+
 #else
-static const char platform_scheduler[] = "crontab";
+	*scheduler = SCHEDULER_CRON;
+	return;
 #endif
+}
 
-static int update_background_schedule(int enable)
+static void validate_scheduler(enum scheduler scheduler)
 {
-	int result;
-	const char *scheduler = platform_scheduler;
-	const char *cmd = scheduler;
-	char *testing;
+	if (scheduler == SCHEDULER_INVALID)
+		BUG("invalid scheduler");
+	if (scheduler == SCHEDULER_AUTO)
+		BUG("resolve_auto_scheduler should have been called before");
+
+	if (!scheduler_fn[scheduler].is_available())
+		die(_("%s scheduler is not available"),
+		    scheduler_fn[scheduler].name);
+}
+
+static int update_background_schedule(const struct maintenance_start_opts *opts,
+				      enum enable_or_disable enable)
+{
+	unsigned int i;
+	int res, result = 0;
 	struct lock_file lk;
 	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
 
-	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
-	if (testing) {
-		char *sep = strchr(testing, ':');
-		if (!sep)
-			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
-		*sep = '\0';
-		scheduler = testing;
-		cmd = sep + 1;
-	}
-
 	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
 		return error(_("another process is scheduling background maintenance"));
 
-	if (!strcmp(scheduler, "launchctl"))
-		result = launchctl_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else if (!strcmp(scheduler, "schtasks"))
-		result = schtasks_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else if (!strcmp(scheduler, "crontab"))
-		result = crontab_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else
-		die("unknown background scheduler: %s", scheduler);
+	for (i = 1; i < ARRAY_SIZE(scheduler_fn); i++) {
+		enum enable_or_disable enable_scheduler =
+			(enable == ENABLE && (opts->scheduler == i)) ?
+			ENABLE : DISABLE;
+		if (!scheduler_fn[i].is_available())
+			continue;
+		res = scheduler_fn[i].update_schedule(
+			enable_scheduler, get_lock_file_fd(&lk));
+		if (enable_scheduler)
+			result = res;
+	}
 
 	rollback_lock_file(&lk);
-	free(testing);
 	return result;
 }
 
-static int maintenance_start(void)
+static const char *const builtin_maintenance_start_usage[] = {
+	N_("git maintenance start [--scheduler=<scheduler>]"), NULL
+};
+
+static int maintenance_start(int argc, const char **argv, const char *prefix)
 {
+	struct maintenance_start_opts opts;
+	struct option builtin_maintenance_start_options[] = {
+		OPT_CALLBACK_F(
+			0, "scheduler", &opts.scheduler, N_("scheduler"),
+			N_("scheduler to use to trigger git maintenance run"),
+			PARSE_OPT_NONEG, maintenance_opt_scheduler),
+		OPT_END()
+	};
+	memset(&opts, 0, sizeof(opts));
+
+	argc = parse_options(argc, argv, prefix,
+			     builtin_maintenance_start_options,
+			     builtin_maintenance_start_usage, 0);
+	if (argc)
+		usage_with_options(builtin_maintenance_start_usage,
+				   builtin_maintenance_start_options);
+
+	resolve_auto_scheduler(&opts.scheduler);
+	validate_scheduler(opts.scheduler);
+
 	if (maintenance_register())
 		warning(_("failed to add repo to global config"));
-
-	return update_background_schedule(1);
+	return update_background_schedule(&opts, 1);
 }
 
 static int maintenance_stop(void)
 {
-	return update_background_schedule(0);
+	return update_background_schedule(NULL, 0);
 }
 
 static const char builtin_maintenance_usage[] =	N_("git maintenance <subcommand> [<options>]");
@@ -2040,7 +2247,7 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
 	if (!strcmp(argv[1], "run"))
 		return maintenance_run(argc - 1, argv + 1, prefix);
 	if (!strcmp(argv[1], "start"))
-		return maintenance_start();
+		return maintenance_start(argc - 1, argv + 1, prefix);
 	if (!strcmp(argv[1], "stop"))
 		return maintenance_stop();
 	if (!strcmp(argv[1], "register"))
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 2412d8c5c0..9eac260307 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -488,8 +488,21 @@ test_expect_success !MINGW 'register and unregister with regex metacharacters' '
 		maintenance.repo "$(pwd)/$META"
 '
 
+test_expect_success 'start --scheduler=<scheduler>' '
+	test_expect_code 129 git maintenance start --scheduler=foo 2>err &&
+	test_i18ngrep "unrecognized --scheduler argument" err &&
+
+	test_expect_code 129 git maintenance start --no-scheduler 2>err &&
+	test_i18ngrep "unknown option" err &&
+
+	test_expect_code 128 \
+		env GIT_TEST_MAINT_SCHEDULER="launchctl:true,schtasks:true" \
+		git maintenance start --scheduler=crontab 2>err &&
+	test_i18ngrep "fatal: crontab scheduler is not available" err
+'
+
 test_expect_success 'start from empty cron table' '
-	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -512,7 +525,7 @@ test_expect_success 'stop from existing schedule' '
 
 test_expect_success 'start preserves existing schedule' '
 	echo "Important information!" >cron.txt &&
-	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
 	grep "Important information!" cron.txt
 '
 
@@ -541,7 +554,7 @@ test_expect_success 'start and stop macOS maintenance' '
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start --scheduler=launchctl &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -592,7 +605,7 @@ test_expect_success 'start and stop Windows maintenance' '
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start --scheduler=schtasks &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -615,6 +628,41 @@ test_expect_success 'start and stop Windows maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success 'start and stop when several schedulers are available' '
+	write_script print-args <<-\EOF &&
+	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
+	EOF
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
+	rm -f expect &&
+	for frequency in hourly daily weekly
+	do
+		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
+		echo "launchctl bootout gui/[UID] $PLIST" >>expect &&
+		echo "launchctl bootstrap gui/[UID] $PLIST" >>expect || return 1
+	done &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
2.31.1


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux
  2021-05-20 22:13   ` [PATCH v3 0/4] " Lénaïc Huard
                       ` (2 preceding siblings ...)
  2021-05-20 22:13     ` [PATCH v3 3/4] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
@ 2021-05-20 22:13     ` Lénaïc Huard
  2021-05-21  9:59       ` Bagas Sanjaya
  2021-05-24  7:15     ` [PATCH v4 0/4] add support for " Lénaïc Huard
  4 siblings, 1 reply; 138+ messages in thread
From: Lénaïc Huard @ 2021-05-20 22:13 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Lénaïc Huard

The existing mechanism for scheduling background maintenance is done
through cron. On Linux systems managed by systemd, systemd provides an
alternative to schedule recurring tasks: systemd timers.

The main motivations to implement systemd timers in addition to cron
are:
* cron is optional and Linux systems running systemd might not have it
  installed.
* The execution of `crontab -l` can tell us if cron is installed but not
  if the daemon is actually running.
* With systemd, each service is run in its own cgroup and its logs are
  tagged by the service inside journald. With cron, all scheduled tasks
  are running in the cron daemon cgroup and all the logs of the
  user-scheduled tasks are pretended to belong to the system cron
  service.
  Concretely, a user that doesn’t have access to the system logs won’t
  have access to the log of its own tasks scheduled by cron whereas he
  will have access to the log of its own tasks scheduled by systemd
  timer.
  Although `cron` attempts to send email, that email may go unseen by
  the user because these days, local mailboxes are not heavily used
  anymore.

In order to schedule git maintenance, we need two unit template files:
* ~/.config/systemd/user/git-maintenance@.service
  to define the command to be started by systemd and
* ~/.config/systemd/user/git-maintenance@.timer
  to define the schedule at which the command should be run.

Those units are templates that are parameterized by the frequency.

Based on those templates, 3 timers are started:
* git-maintenance@hourly.timer
* git-maintenance@daily.timer
* git-maintenance@weekly.timer

The command launched by those three timers are the same as with the
other scheduling methods:

git for-each-repo --config=maintenance.repo maintenance run
--schedule=%i

with the full path for git to ensure that the version of git launched
for the scheduled maintenance is the same as the one used to run
`maintenance start`.

The timer unit contains `Persistent=true` so that, if the computer is
powered down when a maintenance task should run, the task will be run
when the computer is back powered on.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 Documentation/git-maintenance.txt |  57 +++++++++-
 builtin/gc.c                      | 181 ++++++++++++++++++++++++++++++
 t/t7900-maintenance.sh            |  66 ++++++++++-
 3 files changed, 294 insertions(+), 10 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 7c4bb38a2f..50179e010f 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -181,16 +181,19 @@ OPTIONS
 	`maintenance.<task>.enabled` configured as `true` are considered.
 	See the 'TASKS' section for the list of accepted `<task>` values.
 
---scheduler=auto|crontab|launchctl|schtasks::
+--scheduler=auto|crontab|systemd-timer|launchctl|schtasks::
 	When combined with the `start` subcommand, specify the scheduler
 	to use to run the hourly, daily and weekly executions of
 	`git maintenance run`.
 	The possible values for `<scheduler>` depend on the system: `crontab`
-	is available on POSIX systems, `launchctl` is available on
-	MacOS and `schtasks` is available on Windows.
+	is available on POSIX systems, `systemd-timer` is available on Linux
+	systems, `launchctl` is available on MacOS and `schtasks` is available
+	on Windows.
 	By default or when `auto` is specified, a suitable scheduler for
 	the system is used. On MacOS, `launchctl` is used. On Windows,
-	`schtasks` is used. On all other systems, `crontab` is used.
+	`schtasks` is used. On Linux, `systemd-timer` is used if user systemd
+	timers are available, otherwise, `crontab` is used. On all other systems,
+	`crontab` is used.
 
 
 TROUBLESHOOTING
@@ -290,6 +293,52 @@ schedule to ensure you are executing the correct binaries in your
 schedule.
 
 
+BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMS
+-----------------------------------------------
+
+While Linux supports `cron`, depending on the distribution, `cron` may
+be an optional package not necessarily installed. On modern Linux
+distributions, systemd timers are superseding it.
+
+If user systemd timers are available, they will be used as a replacement
+of `cron`.
+
+In this case, `git maintenance start` will create user systemd timer units
+and start the timers. The current list of user-scheduled tasks can be found
+by running `systemctl --user list-timers`. The timers written by `git
+maintenance start` are similar to this:
+
+-----------------------------------------------------------------------
+$ systemctl --user list-timers
+NEXT                         LEFT          LAST                         PASSED     UNIT                         ACTIVATES
+Thu 2021-04-29 19:00:00 CEST 42min left    Thu 2021-04-29 18:00:11 CEST 17min ago  git-maintenance@hourly.timer git-maintenance@hourly.service
+Fri 2021-04-30 00:00:00 CEST 5h 42min left Thu 2021-04-29 00:00:11 CEST 18h ago    git-maintenance@daily.timer  git-maintenance@daily.service
+Mon 2021-05-03 00:00:00 CEST 3 days left   Mon 2021-04-26 00:00:11 CEST 3 days ago git-maintenance@weekly.timer git-maintenance@weekly.service
+-----------------------------------------------------------------------
+
+One timer is registered for each `--schedule=<frequency>` option.
+
+The definition of the systemd units can be inspected in the following files:
+
+-----------------------------------------------------------------------
+~/.config/systemd/user/git-maintenance@.timer
+~/.config/systemd/user/git-maintenance@.service
+~/.config/systemd/user/timers.target.wants/git-maintenance@hourly.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@daily.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@weekly.timer
+-----------------------------------------------------------------------
+
+`git maintenance start` will overwrite these files and start the timer
+again with `systemctl --user`, so any customization should be done by
+creating a drop-in file, i.e. a `.conf` suffixed file in the
+`~/.config/systemd/user/git-maintenance@.service.d` directory.
+
+`git maintenance stop` will stop the user systemd timers and delete
+the above mentioned files.
+
+For more details, see `systemd.timer(5)`.
+
+
 BACKGROUND MAINTENANCE ON MACOS SYSTEMS
 ---------------------------------------
 
diff --git a/builtin/gc.c b/builtin/gc.c
index bf21cec059..d2432ee04f 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -2079,10 +2079,174 @@ static int crontab_update_schedule(enum enable_or_disable run_maintenance,
 	return result;
 }
 
+static int is_systemd_timer_available(void)
+{
+	const char *cmd = "systemctl";
+	int is_available;
+	static int cached_result = -1;
+#ifdef __linux__
+	struct child_process child = CHILD_PROCESS_INIT;
+#endif
+
+	if (cached_result != -1)
+		return cached_result;
+
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+#ifdef __linux__
+	strvec_split(&child.args, cmd);
+	strvec_pushl(&child.args, "--user", "list-timers", NULL);
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child)) {
+		cached_result = 0;
+		return cached_result;
+	}
+	if (finish_command(&child)) {
+		cached_result = 0;
+		return cached_result;
+	}
+	cached_result = 1;
+	return cached_result;
+#else
+	return 0;
+#endif
+}
+
+static char *xdg_config_home_systemd(const char *filename)
+{
+	assert(filename);
+	return xdg_config_home("systemd/user/%s", filename);
+}
+
+static int systemd_timer_enable_unit(int enable,
+				     enum schedule_priority schedule)
+{
+	const char *cmd = "systemctl";
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *frequency = get_frequency(schedule);
+
+	get_schedule_cmd(&cmd, NULL);
+	strvec_split(&child.args, cmd);
+	strvec_pushl(&child.args, "--user", enable ? "enable" : "disable",
+		     "--now", NULL);
+	strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency);
+
+	if (start_command(&child))
+		die(_("failed to run systemctl"));
+	return finish_command(&child);
+}
+
+static int systemd_timer_delete_unit_templates(void)
+{
+	char *filename = xdg_config_home_systemd("git-maintenance@.timer");
+	unlink(filename);
+	free(filename);
+
+	filename = xdg_config_home_systemd("git-maintenance@.service");
+	unlink(filename);
+	free(filename);
+
+	return 0;
+}
+
+static int systemd_timer_delete_units(void)
+{
+	return systemd_timer_enable_unit(DISABLE, SCHEDULE_HOURLY) ||
+	       systemd_timer_enable_unit(DISABLE, SCHEDULE_DAILY) ||
+	       systemd_timer_enable_unit(DISABLE, SCHEDULE_WEEKLY) ||
+	       systemd_timer_delete_unit_templates();
+}
+
+static int systemd_timer_write_unit_templates(const char *exec_path)
+{
+	char *filename;
+	FILE *file;
+	const char *unit;
+
+	filename = xdg_config_home_systemd("git-maintenance@.timer");
+	if (safe_create_leading_directories(filename))
+		die(_("failed to create directories for '%s'"), filename);
+	file = xfopen(filename, "w");
+	FREE_AND_NULL(filename);
+
+	unit = "# This file was created and is maintained by Git.\n"
+	       "# Any edits made in this file might be replaced in the future\n"
+	       "# by a Git command.\n"
+	       "\n"
+	       "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Timer]\n"
+	       "OnCalendar=%i\n"
+	       "Persistent=true\n"
+	       "\n"
+	       "[Install]\n"
+	       "WantedBy=timers.target\n";
+	fputs(unit, file);
+	fclose(file);
+
+	filename = xdg_config_home_systemd("git-maintenance@.service");
+	file = xfopen(filename, "w");
+	free(filename);
+
+	unit = "# This file was created and is maintained by Git.\n"
+	       "# Any edits made in this file might be replaced in the future\n"
+	       "# by a Git command.\n"
+	       "\n"
+	       "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Service]\n"
+	       "Type=oneshot\n"
+	       "ExecStart=\"%s/git\" --exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
+	       "LockPersonality=yes\n"
+	       "MemoryDenyWriteExecute=yes\n"
+	       "NoNewPrivileges=yes\n"
+	       "RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\n"
+	       "RestrictNamespaces=yes\n"
+	       "RestrictRealtime=yes\n"
+	       "RestrictSUIDSGID=yes\n"
+	       "SystemCallArchitectures=native\n"
+	       "SystemCallFilter=@system-service\n";
+	fprintf(file, unit, exec_path, exec_path);
+	fclose(file);
+
+	return 0;
+}
+
+static int systemd_timer_setup_units(void)
+{
+	const char *exec_path = git_exec_path();
+
+	return systemd_timer_write_unit_templates(exec_path) ||
+	       systemd_timer_enable_unit(ENABLE, SCHEDULE_HOURLY) ||
+	       systemd_timer_enable_unit(ENABLE, SCHEDULE_DAILY) ||
+	       systemd_timer_enable_unit(ENABLE, SCHEDULE_WEEKLY);
+}
+
+static int systemd_timer_update_schedule(enum enable_or_disable run_maintenance,
+					 int fd)
+{
+	switch (run_maintenance) {
+	case ENABLE:
+		return systemd_timer_setup_units();
+	case DISABLE:
+		return systemd_timer_delete_units();
+	default:
+		BUG("invalid enable_or_disable value");
+	}
+}
+
 enum scheduler {
 	SCHEDULER_INVALID = -1,
 	SCHEDULER_AUTO,
 	SCHEDULER_CRON,
+	SCHEDULER_SYSTEMD,
 	SCHEDULER_LAUNCHCTL,
 	SCHEDULER_SCHTASKS,
 };
@@ -2097,6 +2261,11 @@ static const struct {
 		.is_available = is_crontab_available,
 		.update_schedule = crontab_update_schedule,
 	},
+	[SCHEDULER_SYSTEMD] = {
+		.name = "systemctl",
+		.is_available = is_systemd_timer_available,
+		.update_schedule = systemd_timer_update_schedule,
+	},
 	[SCHEDULER_LAUNCHCTL] = {
 		.name = "launchctl",
 		.is_available = is_launchctl_available,
@@ -2117,6 +2286,9 @@ static enum scheduler parse_scheduler(const char *value)
 		return SCHEDULER_AUTO;
 	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
 		return SCHEDULER_CRON;
+	else if (!strcasecmp(value, "systemd") ||
+		 !strcasecmp(value, "systemd-timer"))
+		return SCHEDULER_SYSTEMD;
 	else if (!strcasecmp(value, "launchctl"))
 		return SCHEDULER_LAUNCHCTL;
 	else if (!strcasecmp(value, "schtasks"))
@@ -2155,6 +2327,15 @@ static void resolve_auto_scheduler(enum scheduler *scheduler)
 	*scheduler = SCHEDULER_SCHTASKS;
 	return;
 
+#elif defined(__linux__)
+	if (is_systemd_timer_available())
+		*scheduler = SCHEDULER_SYSTEMD;
+	else if (is_crontab_available())
+		*scheduler = SCHEDULER_CRON;
+	else
+		die(_("neither systemd timers nor crontab are available"));
+	return;
+
 #else
 	*scheduler = SCHEDULER_CRON;
 	return;
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 9eac260307..c8a6f19ebc 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -20,6 +20,18 @@ test_xmllint () {
 	fi
 }
 
+test_lazy_prereq SYSTEMD_ANALYZE '
+	systemd-analyze --help >out &&
+	grep verify out
+'
+
+test_systemd_analyze_verify () {
+	if test_have_prereq SYSTEMD_ANALYZE
+	then
+		systemd-analyze verify "$@"
+	fi
+}
+
 test_expect_success 'help text' '
 	test_expect_code 129 git maintenance -h 2>err &&
 	test_i18ngrep "usage: git maintenance <subcommand>" err &&
@@ -628,14 +640,54 @@ test_expect_success 'start and stop Windows maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success 'start and stop Linux/systemd maintenance' '
+	write_script print-args <<-\EOF &&
+	printf "%s\n" "$*" >>args
+	EOF
+
+	XDG_CONFIG_HOME="$PWD" &&
+	export XDG_CONFIG_HOME &&
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance start --scheduler=systemd-timer &&
+
+	# start registers the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_systemd_analyze_verify "systemd/user/git-maintenance@.service" &&
+
+	printf -- "--user enable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_path_is_missing "systemd/user/git-maintenance@.timer" &&
+	test_path_is_missing "systemd/user/git-maintenance@.service" &&
+
+	printf -- "--user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'start and stop when several schedulers are available' '
 	write_script print-args <<-\EOF &&
 	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
-	rm -f expect &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=systemd-timer &&
+	printf -- "systemctl --user enable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >>expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
 	for frequency in hourly daily weekly
 	do
 		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
@@ -647,17 +699,19 @@ test_expect_success 'start and stop when several schedulers are available' '
 	test_cmp expect args &&
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
 	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
-		hourly daily weekly >expect &&
+		hourly daily weekly >>expect &&
 	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
 		hourly daily weekly >>expect &&
 	test_cmp expect args &&
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
 	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
-		hourly daily weekly >expect &&
+		hourly daily weekly >>expect &&
 	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
 		hourly daily weekly >>expect &&
 	test_cmp expect args
-- 
2.31.1


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* Re: [PATCH v3 1/4] cache.h: rename "xdg_config_home" to "xdg_config_home_git"
  2021-05-20 22:13     ` [PATCH v3 1/4] cache.h: rename "xdg_config_home" to "xdg_config_home_git" Lénaïc Huard
@ 2021-05-20 23:44       ` Đoàn Trần Công Danh
  0 siblings, 0 replies; 138+ messages in thread
From: Đoàn Trần Công Danh @ 2021-05-20 23:44 UTC (permalink / raw)
  To: Lénaïc Huard
  Cc: git, Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Felipe Contreras, Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson

On 2021-05-21 00:13:56+0200, Lénaïc Huard <lenaic@lhuard.fr> wrote:
> Current implementation of `xdg_config_home(str)` returns
> `$XDG_CONFIG_HOME/git/$str`, with the `git` subdirectory inserted
> between the `XDG_CONFIG_HOME` environment variable and the parameter.
> 
> This patch re-purposes `xdg_config_home(…)` to be more generic. It now
> only concatenates "$XDG_CONFIG_HOME", or "$HOME/.config" if the former
> isn’t defined, with the parameter, without adding `git` in between.
> Its parameter is now a format string.

Intended or not, this change is going to make a logical conflict,
should other topics also call to xdg_config_home, i.e. no textual
conflicts, programs compiled successfully but run into failure.

I think we shouldn't re-purpose xdg_config_home, we should add a new
function named something like
xdg_config_home_{for,nongit,other,generic} instead.

> -char *xdg_config_home(const char *filename)
> +char *xdg_config_home(const char *fmt, ...)

In my opinion, we don't even need to over-engineer this function
with variadic arguments, I think below function should be enough for
most (all?) cases:

	char *xdg_config_home_prog(const char *prog, const char *filename)

>  {
>  	const char *home, *config_home;
> +	struct strbuf buf = STRBUF_INIT;
> +	char *out = NULL;
> +	va_list args;
> +
> +	va_start(args, fmt);
> +	strbuf_vaddf(&buf, fmt, args);
> +	va_end(args);

If my imagination is sensible, it's not necessary to use a temporary
strbuf and strbuf_vaddf here ...

>  
> -	assert(filename);
>  	config_home = getenv("XDG_CONFIG_HOME");
> -	if (config_home && *config_home)
> -		return mkpathdup("%s/git/%s", config_home, filename);
> +	if (config_home && *config_home) {
> +		out = mkpathdup("%s/%s", config_home, buf.buf);
> +		goto done;
> +	}
>  
>  	home = getenv("HOME");
> -	if (home)
> -		return mkpathdup("%s/.config/git/%s", home, filename);
> -	return NULL;
> +	if (home) {
> +		out = mkpathdup("%s/.config/%s", home, buf.buf);
> +		goto done;
> +	}
> +
> +done:
> +	strbuf_release(&buf);

... and go though the restructure of this function.


-- 
Danh

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v3 3/4] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-05-20 22:13     ` [PATCH v3 3/4] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
@ 2021-05-21  9:52       ` Bagas Sanjaya
  0 siblings, 0 replies; 138+ messages in thread
From: Bagas Sanjaya @ 2021-05-21  9:52 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, brian m . carlson

On 21/05/21 05.13, Lénaïc Huard wrote:
> Depending on the system, different schedulers can be used to schedule
> the hourly, daily and weekly executions of `git maintenance run`:
> * `launchctl` for MacOS,
> * `schtasks` for Windows and
> * `crontab` for everything else.
> 

...and soon to be supported systemd timers.

> `git maintenance run` now has an option to let the end-user explicitly
> choose which scheduler he wants to use:
> `--scheduler=auto|crontab|launchctl|schtasks`.
> 
> When `git maintenance start --scheduler=XXX` is run, it not only
> registers `git maintenance run` tasks in the scheduler XXX, it also
> removes the `git maintenance run` tasks from all the other schedulers to
> ensure we cannot have two schedulers launching concurrent identical
> tasks.
> 
> The default value is `auto` which chooses a suitable scheduler for the
> system.
>
  
Until this point, we haven't supported systemd timers yet, but in the next
patch we add support for it. So the patch description looks OK.

-- 
An old man doll... just what I always wanted! - Clara

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux
  2021-05-20 22:13     ` [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux Lénaïc Huard
@ 2021-05-21  9:59       ` Bagas Sanjaya
  2021-05-21 16:59         ` Derrick Stolee
  0 siblings, 1 reply; 138+ messages in thread
From: Bagas Sanjaya @ 2021-05-21  9:59 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, brian m . carlson

On 21/05/21 05.13, Lénaïc Huard wrote:
> The existing mechanism for scheduling background maintenance is done
> through cron. On Linux systems managed by systemd, systemd provides an
> alternative to schedule recurring tasks: systemd timers.
> 
> The main motivations to implement systemd timers in addition to cron
> are:
> * cron is optional and Linux systems running systemd might not have it
>    installed.
> * The execution of `crontab -l` can tell us if cron is installed but not
>    if the daemon is actually running.
> * With systemd, each service is run in its own cgroup and its logs are
>    tagged by the service inside journald. With cron, all scheduled tasks
>    are running in the cron daemon cgroup and all the logs of the
>    user-scheduled tasks are pretended to belong to the system cron
>    service.
>    Concretely, a user that doesn’t have access to the system logs won’t
>    have access to the log of its own tasks scheduled by cron whereas he
>    will have access to the log of its own tasks scheduled by systemd
>    timer.

For gender neutrality, we can use he/she instead.

>    Although `cron` attempts to send email, that email may go unseen by
>    the user because these days, local mailboxes are not heavily used
>    anymore.
> 
> In order to schedule git maintenance, we need two unit template files:
> * ~/.config/systemd/user/git-maintenance@.service
>    to define the command to be started by systemd and
> * ~/.config/systemd/user/git-maintenance@.timer
>    to define the schedule at which the command should be run.
> 
> Those units are templates that are parameterized by the frequency.
> 
> Based on those templates, 3 timers are started:
> * git-maintenance@hourly.timer
> * git-maintenance@daily.timer
> * git-maintenance@weekly.timer
> 
> The command launched by those three timers are the same as with the
> other scheduling methods:
> 
> git for-each-repo --config=maintenance.repo maintenance run
> --schedule=%i
> 
> with the full path for git to ensure that the version of git launched
> for the scheduled maintenance is the same as the one used to run
> `maintenance start`.
> 

Wouldn't it be `/path/to/git for-each-repo <options>...`?

> The timer unit contains `Persistent=true` so that, if the computer is
> powered down when a maintenance task should run, the task will be run
> when the computer is back powered on.
> 

The title for this patch implied that users running Linux can choose
between classic crontab or systemd timers. However, the intent of this
patch is we add support for systemd timers. So let's say the title
should be "maintenance: add support for systemd timers on Linux".

-- 
An old man doll... just what I always wanted! - Clara

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux
  2021-05-21  9:59       ` Bagas Sanjaya
@ 2021-05-21 16:59         ` Derrick Stolee
  2021-05-22  6:57           ` Johannes Schindelin
  0 siblings, 1 reply; 138+ messages in thread
From: Derrick Stolee @ 2021-05-21 16:59 UTC (permalink / raw)
  To: Bagas Sanjaya, Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, brian m . carlson

On 5/21/2021 5:59 AM, Bagas Sanjaya wrote:
> On 21/05/21 05.13, Lénaïc Huard wrote:
>> The existing mechanism for scheduling background maintenance is done
>> through cron. On Linux systems managed by systemd, systemd provides an
>> alternative to schedule recurring tasks: systemd timers.
>>
>> The main motivations to implement systemd timers in addition to cron
>> are:
>> * cron is optional and Linux systems running systemd might not have it
>>    installed.
>> * The execution of `crontab -l` can tell us if cron is installed but not
>>    if the daemon is actually running.
>> * With systemd, each service is run in its own cgroup and its logs are
>>    tagged by the service inside journald. With cron, all scheduled tasks
>>    are running in the cron daemon cgroup and all the logs of the
>>    user-scheduled tasks are pretended to belong to the system cron
>>    service.
>>    Concretely, a user that doesn’t have access to the system logs won’t
>>    have access to the log of its own tasks scheduled by cron whereas he
>>    will have access to the log of its own tasks scheduled by systemd
>>    timer.
> 
> For gender neutrality, we can use he/she instead.

Singular "they" is better. Fully accurate and less awkward.

Thanks,
-Stolee

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux
  2021-05-21 16:59         ` Derrick Stolee
@ 2021-05-22  6:57           ` Johannes Schindelin
  2021-05-23 18:36             ` Felipe Contreras
  0 siblings, 1 reply; 138+ messages in thread
From: Johannes Schindelin @ 2021-05-22  6:57 UTC (permalink / raw)
  To: Derrick Stolee
  Cc: Bagas Sanjaya, Lénaïc Huard, git, Junio C Hamano,
	Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, brian m . carlson

[-- Attachment #1: Type: text/plain, Size: 1583 bytes --]

Hi,

On Fri, 21 May 2021, Derrick Stolee wrote:

> On 5/21/2021 5:59 AM, Bagas Sanjaya wrote:
> > On 21/05/21 05.13, Lénaïc Huard wrote:
> >> The existing mechanism for scheduling background maintenance is done
> >> through cron. On Linux systems managed by systemd, systemd provides an
> >> alternative to schedule recurring tasks: systemd timers.
> >>
> >> The main motivations to implement systemd timers in addition to cron
> >> are:
> >> * cron is optional and Linux systems running systemd might not have it
> >>    installed.
> >> * The execution of `crontab -l` can tell us if cron is installed but not
> >>    if the daemon is actually running.
> >> * With systemd, each service is run in its own cgroup and its logs are
> >>    tagged by the service inside journald. With cron, all scheduled tasks
> >>    are running in the cron daemon cgroup and all the logs of the
> >>    user-scheduled tasks are pretended to belong to the system cron
> >>    service.
> >>    Concretely, a user that doesn’t have access to the system logs won’t
> >>    have access to the log of its own tasks scheduled by cron whereas he
> >>    will have access to the log of its own tasks scheduled by systemd
> >>    timer.
> >
> > For gender neutrality, we can use he/she instead.
>
> Singular "they" is better. Fully accurate and less awkward.

I agree. If the singular they was good enough for Shakespeare, it is good
enough for anyone. See for yourself:
http://itre.cis.upenn.edu/~myl/languagelog/archives/002748.html

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux
  2021-05-22  6:57           ` Johannes Schindelin
@ 2021-05-23 18:36             ` Felipe Contreras
  2021-05-23 23:27               ` brian m. carlson
  0 siblings, 1 reply; 138+ messages in thread
From: Felipe Contreras @ 2021-05-23 18:36 UTC (permalink / raw)
  To: Johannes Schindelin, Derrick Stolee
  Cc: Bagas Sanjaya, Lénaïc Huard, git, Junio C Hamano,
	Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, brian m . carlson

Johannes Schindelin wrote:
> On Fri, 21 May 2021, Derrick Stolee wrote:
> 
> > On 5/21/2021 5:59 AM, Bagas Sanjaya wrote:
> > > On 21/05/21 05.13, Lénaïc Huard wrote:
> > >> The existing mechanism for scheduling background maintenance is done
> > >> through cron. On Linux systems managed by systemd, systemd provides an
> > >> alternative to schedule recurring tasks: systemd timers.
> > >>
> > >> The main motivations to implement systemd timers in addition to cron
> > >> are:
> > >> * cron is optional and Linux systems running systemd might not have it
> > >>    installed.
> > >> * The execution of `crontab -l` can tell us if cron is installed but not
> > >>    if the daemon is actually running.
> > >> * With systemd, each service is run in its own cgroup and its logs are
> > >>    tagged by the service inside journald. With cron, all scheduled tasks
> > >>    are running in the cron daemon cgroup and all the logs of the
> > >>    user-scheduled tasks are pretended to belong to the system cron
> > >>    service.
> > >>    Concretely, a user that doesn’t have access to the system logs won’t
> > >>    have access to the log of its own tasks scheduled by cron whereas he
> > >>    will have access to the log of its own tasks scheduled by systemd
> > >>    timer.
> > >
> > > For gender neutrality, we can use he/she instead.
> >
> > Singular "they" is better. Fully accurate and less awkward.
> 
> I agree.

I disagree.

> If the singular they was good enough for Shakespeare,

Shakespeare:

 1. Did not know gammar
 2. Invented words as we went along
 3. Was no writing prose

This is not the kind of English we wish to replicate:

  "This was the most unkindest cut of all."

> See for yourself:
> http://itre.cis.upenn.edu/~myl/languagelog/archives/002748.html

I do not see a single instance of a singular antecedent there.

Not that it matters, because unlike Shakespeare we are wriing classic
prose style. The styles could not be more different.


The singular they is a controversial topic[1][2], even among linguists.
This is a software project, we must not make decrees about proper use of
English language, especially when linguists themselves have not yet
fully decided.

If you want to use "they", go ahead, other people want to use "he/she".
The git project should steer cleer of value judgements that one is
_better_ than the other.

Not to mention that these kinds of promulgations invite the culture war.

Cheers.

[1] https://time.com/5748649/word-of-year-they-merriam-webster/
[2] https://www.theatlantic.com/culture/archive/2013/01/singular-their-affront-good-writing/319329/

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux
  2021-05-23 18:36             ` Felipe Contreras
@ 2021-05-23 23:27               ` brian m. carlson
  2021-05-24  1:18                 ` Felipe Contreras
  2021-05-24  7:03                 ` Ævar Arnfjörð Bjarmason
  0 siblings, 2 replies; 138+ messages in thread
From: brian m. carlson @ 2021-05-23 23:27 UTC (permalink / raw)
  To: Felipe Contreras
  Cc: Johannes Schindelin, Derrick Stolee, Bagas Sanjaya,
	Lénaïc Huard, git, Junio C Hamano, Derrick Stolee,
	Eric Sunshine, Đoàn Trần Công Danh,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason

[-- Attachment #1: Type: text/plain, Size: 5457 bytes --]

On 2021-05-23 at 18:36:10, Felipe Contreras wrote:
> Johannes Schindelin wrote:
> > On Fri, 21 May 2021, Derrick Stolee wrote:
> > 
> > > On 5/21/2021 5:59 AM, Bagas Sanjaya wrote:
> > > > On 21/05/21 05.13, Lénaïc Huard wrote:
> > > >> The existing mechanism for scheduling background maintenance is done
> > > >> through cron. On Linux systems managed by systemd, systemd provides an
> > > >> alternative to schedule recurring tasks: systemd timers.
> > > >>
> > > >> The main motivations to implement systemd timers in addition to cron
> > > >> are:
> > > >> * cron is optional and Linux systems running systemd might not have it
> > > >>    installed.
> > > >> * The execution of `crontab -l` can tell us if cron is installed but not
> > > >>    if the daemon is actually running.
> > > >> * With systemd, each service is run in its own cgroup and its logs are
> > > >>    tagged by the service inside journald. With cron, all scheduled tasks
> > > >>    are running in the cron daemon cgroup and all the logs of the
> > > >>    user-scheduled tasks are pretended to belong to the system cron
> > > >>    service.
> > > >>    Concretely, a user that doesn’t have access to the system logs won’t
> > > >>    have access to the log of its own tasks scheduled by cron whereas he
> > > >>    will have access to the log of its own tasks scheduled by systemd
> > > >>    timer.
> > > >
> > > > For gender neutrality, we can use he/she instead.
> > >
> > > Singular "they" is better. Fully accurate and less awkward.
> > 
> > I agree.
> 
> I disagree.

I'm fully in support of singular "they".  It provides a useful pronoun
to use in this context, is widely understood and used, is less awkward
than "he/she," and is less sexist than the indefinite "he."

> > If the singular they was good enough for Shakespeare,
> 
> Shakespeare:
> 
>  1. Did not know gammar
>  2. Invented words as we went along
>  3. Was no writing prose

The point is that it has been used by native English speakers as part of
the language for over half a millennium and is widely used and
understood.  This usage is specified in Merriam Webster[0]:

  The use of they, their, them, and themselves as pronouns of indefinite
  gender and indefinite number is well established in speech and
  writing, even in literary and formal contexts.

Wiktionary notes[1] (references omitted):

  Usage of they as a singular pronoun began in the 1300s and has been
  common ever since, despite attempts by some grammarians, beginning in
  1795, to condemn it as a violation of traditional (Latinate)
  agreement rules.  Some other grammarians have countered that criticism
  since at least 1896.  Fowler's Modern English Usage (third edition)
  notes that it "is being left unaltered by copy editors" and is "not
  widely felt to lie in a prohibited zone."  Some authors compare use of
  singular they to widespread use of singular you instead of thou.

> The singular they is a controversial topic[1][2], even among linguists.
> This is a software project, we must not make decrees about proper use of
> English language, especially when linguists themselves have not yet
> fully decided.

Linguists fit roughly into two camps: prescriptive and descriptive.  The
former specify rules for people to use, and the latter document language
as it is actually used without forming a judgment.  While I am not a
linguist, I have a B.A. in English, and my views fit firmly into the
descriptivist camp.

Some prescriptivists think it is acceptable, and some do not.  But
descriptivists will rightly note that it is and has been commonly used
in English across countries, cultures, and contexts for an extended
period of time and is therefore generally accepted by most English
speakers as a normal part of the language.  Since we are writing text
for an English language audience who are mostly not linguists, we should
probably consider using the language that most people will use in this
context.

> If you want to use "they", go ahead, other people want to use "he/she".
> The git project should steer cleer of value judgements that one is
> _better_ than the other.

Any time we provide a suggestion in a code review, we are proposing that
we have an idea that may be better than the existing one.  It may or may
not actually be so, but we are proposing it in an effort to make the
code and documentation better.  No good-faith contributor would propose
a suggestion to make the project _worse_.

It's completely fine for a contributor to propose that they think an
idea is better provided that they do so in a respectful and considerate
way, which I think happened here.  As with all matters of opinion,
whether a thing is truly better or not is unknowable, and the best we
can do is to adopt an approach that seems to be the most widely accepted
and most provident.

In this case, given the fact that singular they is accepted in a wide
variety of contexts, including many literary and formal contexts, and
even the relatively stalwart Chicago "recognizes that such usage is
gaining [further] acceptance," I think it should be fine to use singular
they here.  You are, of course, free to feel differently.

[0] https://www.merriam-webster.com/dictionary/they
[1] https://en.wiktionary.org/wiki/they
-- 
brian m. carlson (he/him or they/them)
Houston, Texas, US

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux
  2021-05-23 23:27               ` brian m. carlson
@ 2021-05-24  1:18                 ` Felipe Contreras
  2021-05-24  7:03                 ` Ævar Arnfjörð Bjarmason
  1 sibling, 0 replies; 138+ messages in thread
From: Felipe Contreras @ 2021-05-24  1:18 UTC (permalink / raw)
  To: brian m. carlson, Felipe Contreras
  Cc: Johannes Schindelin, Derrick Stolee, Bagas Sanjaya,
	Lénaïc Huard, git, Junio C Hamano, Derrick Stolee,
	Eric Sunshine, Đoàn Trần Công Danh,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason

brian m. carlson wrote:
> On 2021-05-23 at 18:36:10, Felipe Contreras wrote:
> > Johannes Schindelin wrote:
> > > On Fri, 21 May 2021, Derrick Stolee wrote:
> > > > On 5/21/2021 5:59 AM, Bagas Sanjaya wrote:
> > > > > On 21/05/21 05.13, Lénaïc Huard wrote:
> > > > >> The existing mechanism for scheduling background maintenance is done
> > > > >> through cron. On Linux systems managed by systemd, systemd provides an
> > > > >> alternative to schedule recurring tasks: systemd timers.
> > > > >>
> > > > >> The main motivations to implement systemd timers in addition to cron
> > > > >> are:
> > > > >> * cron is optional and Linux systems running systemd might not have it
> > > > >>    installed.
> > > > >> * The execution of `crontab -l` can tell us if cron is installed but not
> > > > >>    if the daemon is actually running.
> > > > >> * With systemd, each service is run in its own cgroup and its logs are
> > > > >>    tagged by the service inside journald. With cron, all scheduled tasks
> > > > >>    are running in the cron daemon cgroup and all the logs of the
> > > > >>    user-scheduled tasks are pretended to belong to the system cron
> > > > >>    service.
> > > > >>    Concretely, a user that doesn’t have access to the system logs won’t
> > > > >>    have access to the log of its own tasks scheduled by cron whereas he
> > > > >>    will have access to the log of its own tasks scheduled by systemd
> > > > >>    timer.
> > > > >
> > > > > For gender neutrality, we can use he/she instead.
> > > >
> > > > Singular "they" is better. Fully accurate and less awkward.
> > > 
> > > I agree.
> > 
> > I disagree.
> 
> I'm fully in support of singular "they".

To each her/his own.

> It provides a useful pronoun to use in this context, is widely
> understood and used, is less awkward than "he/she," and is less sexist
> than the indefinite "he."

I disagree. But I wouldn't presume to dictate how other people speak.

If you like it, use it.

> > > If the singular they was good enough for Shakespeare,
> > 
> > Shakespeare:
> > 
> >  1. Did not know gammar
> >  2. Invented words as we went along
> >  3. Was no writing prose
> 
> The point is that it has been used by native English speakers as part of
> the language for over half a millennium and is widely used and
> understood.

A similar thing happens with the word "nucular" [1]; it is used by
native English speakers as part of the language for many decades and is
widely used and understood.

Does that mean it's a valid word? Maybe.

But does that make "nuclear" invalid? No.

You can use "nucular" if you want (many people do). I will use
"nuclear".

Both can be valid.

> This usage is specified in Merriam Webster[0]:

Merriam-Webster is not infallible.

But fine: let's say they are correct.

Where does it say "they" is *better*? Or worse: where does it say
"his/her" is discouraged?

> > The singular they is a controversial topic[1][2], even among linguists.
> > This is a software project, we must not make decrees about proper use of
> > English language, especially when linguists themselves have not yet
> > fully decided.
> 
> Linguists fit roughly into two camps: prescriptive and descriptive.

I am perfectly aware of the two camps, and I have written about it.

> While I am not a linguist, I have a B.A. in English, and my views fit
> firmly into the descriptivist camp.

If you are a descriptivist (as am I), then you must acknowledge that
people use "his/her" *today*.

As a descriptivist you shouldn't dare to _prescribe_ how people use words.

Therefore, if I use "his/her" (as many people do), you should not
prescribe otherwise.

> Since we are writing text for an English language audience who are
> mostly not linguists, we should probably consider using the language
> that most people will use in this context.

Consider, yes. Dictate? No.

And you don't know how *most* most people speak. Your guess is as good
as mine.

> It's completely fine for a contributor to propose that they think an
> idea is better provided that they do so in a respectful and considerate
> way, which I think happened here.

If you say you think **in your personal opinion** that A is better than
B, that's fine.

What was stated here was "A is better". So that's not what happened.

> I think it should be fine to use singular they here.  You are, of
> course, free to feel differently.

I do feel differently, but I would not presume to dictate that B is
better; I argue both A and B are fine.

 A. he/she
 B. they


To be perfectly clear; I'm not saying you shouldn't use B; I'm saying A
is fine.

Cheers.

[1] https://en.wikipedia.org/wiki/Nucular

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux
  2021-05-23 23:27               ` brian m. carlson
  2021-05-24  1:18                 ` Felipe Contreras
@ 2021-05-24  7:03                 ` Ævar Arnfjörð Bjarmason
  2021-05-24 15:51                   ` Junio C Hamano
  2021-05-24 17:52                   ` [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux Felipe Contreras
  1 sibling, 2 replies; 138+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-24  7:03 UTC (permalink / raw)
  To: brian m. carlson
  Cc: Felipe Contreras, Johannes Schindelin, Derrick Stolee,
	Bagas Sanjaya, Lénaïc Huard, git, Junio C Hamano,
	Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Phillip Wood,
	Martin Ågren


On Sun, May 23 2021, brian m. carlson wrote:

> [[PGP Signed Part:Undecided]]
> On 2021-05-23 at 18:36:10, Felipe Contreras wrote:
>> Johannes Schindelin wrote:
>> > On Fri, 21 May 2021, Derrick Stolee wrote:
>> > 
>> > > On 5/21/2021 5:59 AM, Bagas Sanjaya wrote:
>> > > > On 21/05/21 05.13, Lénaïc Huard wrote:
>> > > >> The existing mechanism for scheduling background maintenance is done
>> > > >> through cron. On Linux systems managed by systemd, systemd provides an
>> > > >> alternative to schedule recurring tasks: systemd timers.
>> > > >>
>> > > >> The main motivations to implement systemd timers in addition to cron
>> > > >> are:
>> > > >> * cron is optional and Linux systems running systemd might not have it
>> > > >>    installed.
>> > > >> * The execution of `crontab -l` can tell us if cron is installed but not
>> > > >>    if the daemon is actually running.
>> > > >> * With systemd, each service is run in its own cgroup and its logs are
>> > > >>    tagged by the service inside journald. With cron, all scheduled tasks
>> > > >>    are running in the cron daemon cgroup and all the logs of the
>> > > >>    user-scheduled tasks are pretended to belong to the system cron
>> > > >>    service.
>> > > >>    Concretely, a user that doesn’t have access to the system logs won’t
>> > > >>    have access to the log of its own tasks scheduled by cron whereas he
>> > > >>    will have access to the log of its own tasks scheduled by systemd
>> > > >>    timer.
>> > > >
>> > > > For gender neutrality, we can use he/she instead.
>> > >
>> > > Singular "they" is better. Fully accurate and less awkward.
>> > 
>> > I agree.
>> 
>> I disagree.
>
> I'm fully in support of singular "they".  It provides a useful pronoun
> to use in this context, is widely understood and used, is less awkward
> than "he/she," and is less sexist than the indefinite "he."

I think we should be the most concerned about the lack of inclusivity
and chilling effect in us being overly picky about the minute details of
commit message wording, long past the point of practical utility.

In this particular case the context is a discussion about "a user" on a
*nix system and what they do and don't have access to.

I think it's a particularly misguided distraction to argue about
he/she/they here, since the most accurate thing would really be "it".

At least on my *nix systems most users are system users, and don't map
to any particular human being, but I digress.

I would like to encourage people in this thread who are calling for a
change in wording here to consider whether this sort of discussion is a
good use of the ML's time, and the chilling effect of being overly picky
when many contributors are working in their second, third etc. language.

Personally I don't care whether someone submits a patch where their
commit message discusses an example of "he", "she", "they", "it" or
whatever. It's just meant as an example, and not some statement about
what the gender (or lack thereof) of such a user *should* be.

It's immediately obvious what the author meant in this case, and that
the particular wording is arbitrary. For the purposes of discussing the
contribution it matters whether it's unclear or ambiguous, which it's
not.

^ permalink raw reply	[flat|nested] 138+ messages in thread

* [PATCH v4 0/4] add support for systemd timers on Linux
  2021-05-20 22:13   ` [PATCH v3 0/4] " Lénaïc Huard
                       ` (3 preceding siblings ...)
  2021-05-20 22:13     ` [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux Lénaïc Huard
@ 2021-05-24  7:15     ` Lénaïc Huard
  2021-05-24  7:15       ` [PATCH v4 1/4] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
                         ` (5 more replies)
  4 siblings, 6 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-05-24  7:15 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Lénaïc Huard

Hello,

The main difference between this v4 patchset and the previous v3
concerns the first patch, the one introducing a new function to use
$XDG_CONFIG_HOME.
Instead of re-purposing the existing `xdg_config_home(filename)`
function, I’ve created a new generic one named
`xdg_config_home_for(prog, filename)` to address the potential
conflict issue raised by Danh.

I’ve also reworded the commit message of the last patch to address the
review comments.
I replaced the “he” by a “they”.
To be honest, I’m not an English native speaker, so I don’t feel
comfortable debating which usage is common in informal speech or in
formal writtings.
So, I would be happy to change it to “he or she” if it is more
consensual and looks less “artificial” than “he/she”.

Lénaïc Huard (4):
  cache.h: Introduce a generic "xdg_config_home_for(…)" function
  maintenance: introduce ENABLE/DISABLE for code clarity
  maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  maintenance: add support for systemd timers on Linux

 Documentation/git-maintenance.txt |  60 ++++
 builtin/gc.c                      | 548 ++++++++++++++++++++++++++----
 cache.h                           |   7 +
 path.c                            |  13 +-
 t/t7900-maintenance.sh            | 110 +++++-
 5 files changed, 657 insertions(+), 81 deletions(-)

-- 
2.31.1


^ permalink raw reply	[flat|nested] 138+ messages in thread

* [PATCH v4 1/4] cache.h: Introduce a generic "xdg_config_home_for(…)" function
  2021-05-24  7:15     ` [PATCH v4 0/4] add support for " Lénaïc Huard
@ 2021-05-24  7:15       ` Lénaïc Huard
  2021-05-24  9:33         ` Phillip Wood
  2021-05-24  7:15       ` [PATCH v4 2/4] maintenance: introduce ENABLE/DISABLE for code clarity Lénaïc Huard
                         ` (4 subsequent siblings)
  5 siblings, 1 reply; 138+ messages in thread
From: Lénaïc Huard @ 2021-05-24  7:15 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Lénaïc Huard

Current implementation of `xdg_config_home(filename)` returns
`$XDG_CONFIG_HOME/git/$filename`, with the `git` subdirectory inserted
between the `XDG_CONFIG_HOME` environment variable and the parameter.

This patch introduces a `xdg_config_home_for(prog, filename)` function
which is more generic. It only concatenates "$XDG_CONFIG_HOME", or
"$HOME/.config" if the former isn’t defined, with the parameters,
without adding `git` in between.

`xdg_config_home(filename)` is now implemented by calling
`xdg_config_home_for("git", filename)` but this new generic function can
be used to compute the configuration directory of other programs.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 cache.h |  7 +++++++
 path.c  | 13 ++++++++++---
 2 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/cache.h b/cache.h
index 148d9ab5f1..8a2969414a 100644
--- a/cache.h
+++ b/cache.h
@@ -1263,6 +1263,13 @@ int is_ntfs_dotgitattributes(const char *name);
  */
 int looks_like_command_line_option(const char *str);
 
+/**
+ * Return a newly allocated string with the evaluation of
+ * "$XDG_CONFIG_HOME/$prog/$filename" if $XDG_CONFIG_HOME is non-empty, otherwise
+ * "$HOME/.config/$prog/$filename". Return NULL upon error.
+ */
+char *xdg_config_home_for(const char *prog, const char *filename);
+
 /**
  * Return a newly allocated string with the evaluation of
  * "$XDG_CONFIG_HOME/git/$filename" if $XDG_CONFIG_HOME is non-empty, otherwise
diff --git a/path.c b/path.c
index 7b385e5eb2..3641d4c456 100644
--- a/path.c
+++ b/path.c
@@ -1498,21 +1498,28 @@ int looks_like_command_line_option(const char *str)
 	return str && str[0] == '-';
 }
 
-char *xdg_config_home(const char *filename)
+char *xdg_config_home_for(const char *prog, const char *filename)
 {
 	const char *home, *config_home;
 
+	assert(prog);
 	assert(filename);
 	config_home = getenv("XDG_CONFIG_HOME");
 	if (config_home && *config_home)
-		return mkpathdup("%s/git/%s", config_home, filename);
+		return mkpathdup("%s/%s/%s", config_home, prog, filename);
 
 	home = getenv("HOME");
 	if (home)
-		return mkpathdup("%s/.config/git/%s", home, filename);
+		return mkpathdup("%s/.config/%s/%s", home, prog, filename);
+
 	return NULL;
 }
 
+char *xdg_config_home(const char *filename)
+{
+	return xdg_config_home_for("git", filename);
+}
+
 char *xdg_cache_home(const char *filename)
 {
 	const char *home, *cache_home;
-- 
2.31.1


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* [PATCH v4 2/4] maintenance: introduce ENABLE/DISABLE for code clarity
  2021-05-24  7:15     ` [PATCH v4 0/4] add support for " Lénaïc Huard
  2021-05-24  7:15       ` [PATCH v4 1/4] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
@ 2021-05-24  7:15       ` Lénaïc Huard
  2021-05-24  9:41         ` Phillip Wood
  2021-05-24  9:47         ` Ævar Arnfjörð Bjarmason
  2021-05-24  7:15       ` [PATCH v4 3/4] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
                         ` (3 subsequent siblings)
  5 siblings, 2 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-05-24  7:15 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Lénaïc Huard

The first parameter of `XXX_update_schedule` and alike functions is a
boolean specifying if the tasks should be scheduled or unscheduled.

Using an `enum` with `ENABLE` and `DISABLE` values can make the code
clearer.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 builtin/gc.c | 49 +++++++++++++++++++++++++++++++------------------
 1 file changed, 31 insertions(+), 18 deletions(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index ef7226d7bc..0caf8d45c4 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1570,19 +1570,21 @@ static char *launchctl_get_uid(void)
 	return xstrfmt("gui/%d", getuid());
 }
 
-static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
+enum enable_or_disable {
+	DISABLE,
+	ENABLE
+};
+
+static int launchctl_boot_plist(enum enable_or_disable enable,
+				const char *filename, const char *cmd)
 {
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
 	char *uid = launchctl_get_uid();
 
 	strvec_split(&child.args, cmd);
-	if (enable)
-		strvec_push(&child.args, "bootstrap");
-	else
-		strvec_push(&child.args, "bootout");
-	strvec_push(&child.args, uid);
-	strvec_push(&child.args, filename);
+	strvec_pushl(&child.args, enable == ENABLE ? "bootstrap" : "bootout",
+		     uid, filename, NULL);
 
 	child.no_stderr = 1;
 	child.no_stdout = 1;
@@ -1601,7 +1603,7 @@ static int launchctl_remove_plist(enum schedule_priority schedule, const char *c
 	const char *frequency = get_frequency(schedule);
 	char *name = launchctl_service_name(frequency);
 	char *filename = launchctl_service_filename(name);
-	int result = launchctl_boot_plist(0, filename, cmd);
+	int result = launchctl_boot_plist(DISABLE, filename, cmd);
 	unlink(filename);
 	free(filename);
 	free(name);
@@ -1684,8 +1686,8 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 	fclose(plist);
 
 	/* bootout might fail if not already running, so ignore */
-	launchctl_boot_plist(0, filename, cmd);
-	if (launchctl_boot_plist(1, filename, cmd))
+	launchctl_boot_plist(DISABLE, filename, cmd);
+	if (launchctl_boot_plist(ENABLE, filename, cmd))
 		die(_("failed to bootstrap service %s"), filename);
 
 	free(filename);
@@ -1702,12 +1704,17 @@ static int launchctl_add_plists(const char *cmd)
 		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
 }
 
-static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int launchctl_update_schedule(enum enable_or_disable run_maintenance,
+				     int fd, const char *cmd)
 {
-	if (run_maintenance)
+	switch (run_maintenance) {
+	case ENABLE:
 		return launchctl_add_plists(cmd);
-	else
+	case DISABLE:
 		return launchctl_remove_plists(cmd);
+	default:
+		BUG("invalid enable_or_disable value");
+	}
 }
 
 static char *schtasks_task_name(const char *frequency)
@@ -1864,18 +1871,24 @@ static int schtasks_schedule_tasks(const char *cmd)
 		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
 }
 
-static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int schtasks_update_schedule(enum enable_or_disable run_maintenance,
+				    int fd, const char *cmd)
 {
-	if (run_maintenance)
+	switch (run_maintenance) {
+	case ENABLE:
 		return schtasks_schedule_tasks(cmd);
-	else
+	case DISABLE:
 		return schtasks_remove_tasks(cmd);
+	default:
+		BUG("invalid enable_or_disable value");
+	}
 }
 
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
-static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int crontab_update_schedule(enum enable_or_disable run_maintenance,
+				   int fd, const char *cmd)
 {
 	int result = 0;
 	int in_old_region = 0;
@@ -1925,7 +1938,7 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 			fprintf(cron_in, "%s\n", line.buf);
 	}
 
-	if (run_maintenance) {
+	if (run_maintenance == ENABLE) {
 		struct strbuf line_format = STRBUF_INIT;
 		const char *exec_path = git_exec_path();
 
-- 
2.31.1


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* [PATCH v4 3/4] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-05-24  7:15     ` [PATCH v4 0/4] add support for " Lénaïc Huard
  2021-05-24  7:15       ` [PATCH v4 1/4] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
  2021-05-24  7:15       ` [PATCH v4 2/4] maintenance: introduce ENABLE/DISABLE for code clarity Lénaïc Huard
@ 2021-05-24  7:15       ` Lénaïc Huard
  2021-05-24 10:12         ` Phillip Wood
  2021-05-24  7:15       ` [PATCH v4 4/4] maintenance: add support for systemd timers on Linux Lénaïc Huard
                         ` (2 subsequent siblings)
  5 siblings, 1 reply; 138+ messages in thread
From: Lénaïc Huard @ 2021-05-24  7:15 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Lénaïc Huard

Depending on the system, different schedulers can be used to schedule
the hourly, daily and weekly executions of `git maintenance run`:
* `launchctl` for MacOS,
* `schtasks` for Windows and
* `crontab` for everything else.

`git maintenance run` now has an option to let the end-user explicitly
choose which scheduler he wants to use:
`--scheduler=auto|crontab|launchctl|schtasks`.

When `git maintenance start --scheduler=XXX` is run, it not only
registers `git maintenance run` tasks in the scheduler XXX, it also
removes the `git maintenance run` tasks from all the other schedulers to
ensure we cannot have two schedulers launching concurrent identical
tasks.

The default value is `auto` which chooses a suitable scheduler for the
system.

`git maintenance stop` doesn't have any `--scheduler` parameter because
this command will try to remove the `git maintenance run` tasks from all
the available schedulers.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 Documentation/git-maintenance.txt |  11 +
 builtin/gc.c                      | 333 ++++++++++++++++++++++++------
 t/t7900-maintenance.sh            |  56 ++++-
 3 files changed, 333 insertions(+), 67 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 80ddd33ceb..7c4bb38a2f 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -181,6 +181,17 @@ OPTIONS
 	`maintenance.<task>.enabled` configured as `true` are considered.
 	See the 'TASKS' section for the list of accepted `<task>` values.
 
+--scheduler=auto|crontab|launchctl|schtasks::
+	When combined with the `start` subcommand, specify the scheduler
+	to use to run the hourly, daily and weekly executions of
+	`git maintenance run`.
+	The possible values for `<scheduler>` depend on the system: `crontab`
+	is available on POSIX systems, `launchctl` is available on
+	MacOS and `schtasks` is available on Windows.
+	By default or when `auto` is specified, a suitable scheduler for
+	the system is used. On MacOS, `launchctl` is used. On Windows,
+	`schtasks` is used. On all other systems, `crontab` is used.
+
 
 TROUBLESHOOTING
 ---------------
diff --git a/builtin/gc.c b/builtin/gc.c
index 0caf8d45c4..bf21cec059 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1544,6 +1544,60 @@ static const char *get_frequency(enum schedule_priority schedule)
 	}
 }
 
+static int get_schedule_cmd(const char **cmd, int *is_available)
+{
+	char *item;
+	static char test_cmd[32];
+	char *testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
+
+	if (!testing)
+		return 0;
+
+	if (is_available)
+		*is_available = 0;
+
+	for(item = testing;;) {
+		char *sep;
+		char *end_item = strchr(item, ',');
+		if (end_item)
+			*end_item = '\0';
+
+		sep = strchr(item, ':');
+		if (!sep)
+			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
+		*sep = '\0';
+
+		if (!strcmp(*cmd, item)) {
+			strlcpy(test_cmd, sep+1, ARRAY_SIZE(test_cmd));
+			*cmd = test_cmd;
+			if (is_available)
+				*is_available = 1;
+			break;
+		}
+
+		if (!end_item)
+			break;
+		item = end_item + 1;
+	}
+
+	free(testing);
+	return 1;
+}
+
+static int is_launchctl_available(void)
+{
+	const char *cmd = "launchctl";
+	int is_available;
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+#ifdef __APPLE__
+	return 1;
+#else
+	return 0;
+#endif
+}
+
 static char *launchctl_service_name(const char *frequency)
 {
 	struct strbuf label = STRBUF_INIT;
@@ -1576,12 +1630,14 @@ enum enable_or_disable {
 };
 
 static int launchctl_boot_plist(enum enable_or_disable enable,
-				const char *filename, const char *cmd)
+				const char *filename)
 {
+	const char *cmd = "launchctl";
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
 	char *uid = launchctl_get_uid();
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&child.args, cmd);
 	strvec_pushl(&child.args, enable == ENABLE ? "bootstrap" : "bootout",
 		     uid, filename, NULL);
@@ -1598,26 +1654,26 @@ static int launchctl_boot_plist(enum enable_or_disable enable,
 	return result;
 }
 
-static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)
+static int launchctl_remove_plist(enum schedule_priority schedule)
 {
 	const char *frequency = get_frequency(schedule);
 	char *name = launchctl_service_name(frequency);
 	char *filename = launchctl_service_filename(name);
-	int result = launchctl_boot_plist(DISABLE, filename, cmd);
+	int result = launchctl_boot_plist(DISABLE, filename);
 	unlink(filename);
 	free(filename);
 	free(name);
 	return result;
 }
 
-static int launchctl_remove_plists(const char *cmd)
+static int launchctl_remove_plists(void)
 {
-	return launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||
-		launchctl_remove_plist(SCHEDULE_DAILY, cmd) ||
-		launchctl_remove_plist(SCHEDULE_WEEKLY, cmd);
+	return launchctl_remove_plist(SCHEDULE_HOURLY) ||
+		launchctl_remove_plist(SCHEDULE_DAILY) ||
+		launchctl_remove_plist(SCHEDULE_WEEKLY);
 }
 
-static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule)
 {
 	FILE *plist;
 	int i;
@@ -1686,8 +1742,8 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 	fclose(plist);
 
 	/* bootout might fail if not already running, so ignore */
-	launchctl_boot_plist(DISABLE, filename, cmd);
-	if (launchctl_boot_plist(ENABLE, filename, cmd))
+	launchctl_boot_plist(DISABLE, filename);
+	if (launchctl_boot_plist(ENABLE, filename))
 		die(_("failed to bootstrap service %s"), filename);
 
 	free(filename);
@@ -1695,28 +1751,42 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 	return 0;
 }
 
-static int launchctl_add_plists(const char *cmd)
+static int launchctl_add_plists(void)
 {
 	const char *exec_path = git_exec_path();
 
-	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||
-		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||
-		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
+	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY) ||
+		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY) ||
+		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY);
 }
 
 static int launchctl_update_schedule(enum enable_or_disable run_maintenance,
-				     int fd, const char *cmd)
+				     int fd)
 {
 	switch (run_maintenance) {
 	case ENABLE:
-		return launchctl_add_plists(cmd);
+		return launchctl_add_plists();
 	case DISABLE:
-		return launchctl_remove_plists(cmd);
+		return launchctl_remove_plists();
 	default:
 		BUG("invalid enable_or_disable value");
 	}
 }
 
+static int is_schtasks_available(void)
+{
+	const char *cmd = "schtasks";
+	int is_available;
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+#ifdef GIT_WINDOWS_NATIVE
+	return 1;
+#else
+	return 0;
+#endif
+}
+
 static char *schtasks_task_name(const char *frequency)
 {
 	struct strbuf label = STRBUF_INIT;
@@ -1724,13 +1794,15 @@ static char *schtasks_task_name(const char *frequency)
 	return strbuf_detach(&label, NULL);
 }
 
-static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)
+static int schtasks_remove_task(enum schedule_priority schedule)
 {
+	const char *cmd = "schtasks";
 	int result;
 	struct strvec args = STRVEC_INIT;
 	const char *frequency = get_frequency(schedule);
 	char *name = schtasks_task_name(frequency);
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&args, cmd);
 	strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
 
@@ -1741,15 +1813,16 @@ static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd
 	return result;
 }
 
-static int schtasks_remove_tasks(const char *cmd)
+static int schtasks_remove_tasks(void)
 {
-	return schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||
-		schtasks_remove_task(SCHEDULE_DAILY, cmd) ||
-		schtasks_remove_task(SCHEDULE_WEEKLY, cmd);
+	return schtasks_remove_task(SCHEDULE_HOURLY) ||
+		schtasks_remove_task(SCHEDULE_DAILY) ||
+		schtasks_remove_task(SCHEDULE_WEEKLY);
 }
 
-static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule)
 {
+	const char *cmd = "schtasks";
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
 	const char *xml;
@@ -1758,6 +1831,8 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
 	char *name = schtasks_task_name(frequency);
 	struct strbuf tfilename = STRBUF_INIT;
 
+	get_schedule_cmd(&cmd, NULL);
+
 	strbuf_addf(&tfilename, "%s/schedule_%s_XXXXXX",
 		    get_git_common_dir(), frequency);
 	tfile = xmks_tempfile(tfilename.buf);
@@ -1862,34 +1937,65 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
 	return result;
 }
 
-static int schtasks_schedule_tasks(const char *cmd)
+static int schtasks_schedule_tasks(void)
 {
 	const char *exec_path = git_exec_path();
 
-	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||
-		schtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||
-		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
+	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY) ||
+		schtasks_schedule_task(exec_path, SCHEDULE_DAILY) ||
+		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY);
 }
 
 static int schtasks_update_schedule(enum enable_or_disable run_maintenance,
-				    int fd, const char *cmd)
+				    int fd)
 {
 	switch (run_maintenance) {
 	case ENABLE:
-		return schtasks_schedule_tasks(cmd);
+		return schtasks_schedule_tasks();
 	case DISABLE:
-		return schtasks_remove_tasks(cmd);
+		return schtasks_remove_tasks();
 	default:
 		BUG("invalid enable_or_disable value");
 	}
 }
 
+static int is_crontab_available(void)
+{
+	const char *cmd = "crontab";
+	int is_available;
+	static int cached_result = -1;
+	struct child_process child = CHILD_PROCESS_INIT;
+
+	if (cached_result != -1)
+		return cached_result;
+
+	if (get_schedule_cmd(&cmd, &is_available) && !is_available)
+		return 0;
+
+	strvec_split(&child.args, cmd);
+	strvec_push(&child.args, "-l");
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child)) {
+		cached_result = 0;
+		return cached_result;
+	}
+	/* Ignore exit code, as an empty crontab will return error. */
+	finish_command(&child);
+	cached_result = 1;
+	return cached_result;
+}
+
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
 static int crontab_update_schedule(enum enable_or_disable run_maintenance,
-				   int fd, const char *cmd)
+				   int fd)
 {
+	const char *cmd = "crontab";
 	int result = 0;
 	int in_old_region = 0;
 	struct child_process crontab_list = CHILD_PROCESS_INIT;
@@ -1897,6 +2003,7 @@ static int crontab_update_schedule(enum enable_or_disable run_maintenance,
 	FILE *cron_list, *cron_in;
 	struct strbuf line = STRBUF_INIT;
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&crontab_list.args, cmd);
 	strvec_push(&crontab_list.args, "-l");
 	crontab_list.in = -1;
@@ -1972,61 +2079,161 @@ static int crontab_update_schedule(enum enable_or_disable run_maintenance,
 	return result;
 }
 
+enum scheduler {
+	SCHEDULER_INVALID = -1,
+	SCHEDULER_AUTO,
+	SCHEDULER_CRON,
+	SCHEDULER_LAUNCHCTL,
+	SCHEDULER_SCHTASKS,
+};
+
+static const struct {
+	const char *name;
+	int (*is_available)(void);
+	int (*update_schedule)(enum enable_or_disable run_maintenance, int fd);
+} scheduler_fn[] = {
+	[SCHEDULER_CRON] = {
+		.name = "crontab",
+		.is_available = is_crontab_available,
+		.update_schedule = crontab_update_schedule,
+	},
+	[SCHEDULER_LAUNCHCTL] = {
+		.name = "launchctl",
+		.is_available = is_launchctl_available,
+		.update_schedule = launchctl_update_schedule,
+	},
+	[SCHEDULER_SCHTASKS] = {
+		.name = "schtasks",
+		.is_available = is_schtasks_available,
+		.update_schedule = schtasks_update_schedule,
+	},
+};
+
+static enum scheduler parse_scheduler(const char *value)
+{
+	if (!value)
+		return SCHEDULER_INVALID;
+	else if (!strcasecmp(value, "auto"))
+		return SCHEDULER_AUTO;
+	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
+		return SCHEDULER_CRON;
+	else if (!strcasecmp(value, "launchctl"))
+		return SCHEDULER_LAUNCHCTL;
+	else if (!strcasecmp(value, "schtasks"))
+		return SCHEDULER_SCHTASKS;
+	else
+		return SCHEDULER_INVALID;
+}
+
+static int maintenance_opt_scheduler(const struct option *opt, const char *arg,
+				     int unset)
+{
+	enum scheduler *scheduler = opt->value;
+
+	BUG_ON_OPT_NEG(unset);
+
+	*scheduler = parse_scheduler(arg);
+	if (*scheduler == SCHEDULER_INVALID)
+		return error(_("unrecognized --scheduler argument '%s'"), arg);
+	return 0;
+}
+
+struct maintenance_start_opts {
+	enum scheduler scheduler;
+};
+
+static void resolve_auto_scheduler(enum scheduler *scheduler)
+{
+	if (*scheduler != SCHEDULER_AUTO)
+		return;
+
 #if defined(__APPLE__)
-static const char platform_scheduler[] = "launchctl";
+	*scheduler = SCHEDULER_LAUNCHCTL;
+	return;
+
 #elif defined(GIT_WINDOWS_NATIVE)
-static const char platform_scheduler[] = "schtasks";
+	*scheduler = SCHEDULER_SCHTASKS;
+	return;
+
 #else
-static const char platform_scheduler[] = "crontab";
+	*scheduler = SCHEDULER_CRON;
+	return;
 #endif
+}
 
-static int update_background_schedule(int enable)
+static void validate_scheduler(enum scheduler scheduler)
 {
-	int result;
-	const char *scheduler = platform_scheduler;
-	const char *cmd = scheduler;
-	char *testing;
+	if (scheduler == SCHEDULER_INVALID)
+		BUG("invalid scheduler");
+	if (scheduler == SCHEDULER_AUTO)
+		BUG("resolve_auto_scheduler should have been called before");
+
+	if (!scheduler_fn[scheduler].is_available())
+		die(_("%s scheduler is not available"),
+		    scheduler_fn[scheduler].name);
+}
+
+static int update_background_schedule(const struct maintenance_start_opts *opts,
+				      enum enable_or_disable enable)
+{
+	unsigned int i;
+	int res, result = 0;
 	struct lock_file lk;
 	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
 
-	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
-	if (testing) {
-		char *sep = strchr(testing, ':');
-		if (!sep)
-			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
-		*sep = '\0';
-		scheduler = testing;
-		cmd = sep + 1;
-	}
-
 	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
 		return error(_("another process is scheduling background maintenance"));
 
-	if (!strcmp(scheduler, "launchctl"))
-		result = launchctl_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else if (!strcmp(scheduler, "schtasks"))
-		result = schtasks_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else if (!strcmp(scheduler, "crontab"))
-		result = crontab_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else
-		die("unknown background scheduler: %s", scheduler);
+	for (i = 1; i < ARRAY_SIZE(scheduler_fn); i++) {
+		enum enable_or_disable enable_scheduler =
+			(enable == ENABLE && (opts->scheduler == i)) ?
+			ENABLE : DISABLE;
+		if (!scheduler_fn[i].is_available())
+			continue;
+		res = scheduler_fn[i].update_schedule(
+			enable_scheduler, get_lock_file_fd(&lk));
+		if (enable_scheduler)
+			result = res;
+	}
 
 	rollback_lock_file(&lk);
-	free(testing);
 	return result;
 }
 
-static int maintenance_start(void)
+static const char *const builtin_maintenance_start_usage[] = {
+	N_("git maintenance start [--scheduler=<scheduler>]"), NULL
+};
+
+static int maintenance_start(int argc, const char **argv, const char *prefix)
 {
+	struct maintenance_start_opts opts;
+	struct option builtin_maintenance_start_options[] = {
+		OPT_CALLBACK_F(
+			0, "scheduler", &opts.scheduler, N_("scheduler"),
+			N_("scheduler to use to trigger git maintenance run"),
+			PARSE_OPT_NONEG, maintenance_opt_scheduler),
+		OPT_END()
+	};
+	memset(&opts, 0, sizeof(opts));
+
+	argc = parse_options(argc, argv, prefix,
+			     builtin_maintenance_start_options,
+			     builtin_maintenance_start_usage, 0);
+	if (argc)
+		usage_with_options(builtin_maintenance_start_usage,
+				   builtin_maintenance_start_options);
+
+	resolve_auto_scheduler(&opts.scheduler);
+	validate_scheduler(opts.scheduler);
+
 	if (maintenance_register())
 		warning(_("failed to add repo to global config"));
-
-	return update_background_schedule(1);
+	return update_background_schedule(&opts, 1);
 }
 
 static int maintenance_stop(void)
 {
-	return update_background_schedule(0);
+	return update_background_schedule(NULL, 0);
 }
 
 static const char builtin_maintenance_usage[] =	N_("git maintenance <subcommand> [<options>]");
@@ -2040,7 +2247,7 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
 	if (!strcmp(argv[1], "run"))
 		return maintenance_run(argc - 1, argv + 1, prefix);
 	if (!strcmp(argv[1], "start"))
-		return maintenance_start();
+		return maintenance_start(argc - 1, argv + 1, prefix);
 	if (!strcmp(argv[1], "stop"))
 		return maintenance_stop();
 	if (!strcmp(argv[1], "register"))
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 2412d8c5c0..9eac260307 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -488,8 +488,21 @@ test_expect_success !MINGW 'register and unregister with regex metacharacters' '
 		maintenance.repo "$(pwd)/$META"
 '
 
+test_expect_success 'start --scheduler=<scheduler>' '
+	test_expect_code 129 git maintenance start --scheduler=foo 2>err &&
+	test_i18ngrep "unrecognized --scheduler argument" err &&
+
+	test_expect_code 129 git maintenance start --no-scheduler 2>err &&
+	test_i18ngrep "unknown option" err &&
+
+	test_expect_code 128 \
+		env GIT_TEST_MAINT_SCHEDULER="launchctl:true,schtasks:true" \
+		git maintenance start --scheduler=crontab 2>err &&
+	test_i18ngrep "fatal: crontab scheduler is not available" err
+'
+
 test_expect_success 'start from empty cron table' '
-	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -512,7 +525,7 @@ test_expect_success 'stop from existing schedule' '
 
 test_expect_success 'start preserves existing schedule' '
 	echo "Important information!" >cron.txt &&
-	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
 	grep "Important information!" cron.txt
 '
 
@@ -541,7 +554,7 @@ test_expect_success 'start and stop macOS maintenance' '
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start --scheduler=launchctl &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -592,7 +605,7 @@ test_expect_success 'start and stop Windows maintenance' '
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start --scheduler=schtasks &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -615,6 +628,41 @@ test_expect_success 'start and stop Windows maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success 'start and stop when several schedulers are available' '
+	write_script print-args <<-\EOF &&
+	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
+	EOF
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
+	rm -f expect &&
+	for frequency in hourly daily weekly
+	do
+		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
+		echo "launchctl bootout gui/[UID] $PLIST" >>expect &&
+		echo "launchctl bootstrap gui/[UID] $PLIST" >>expect || return 1
+	done &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
2.31.1


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* [PATCH v4 4/4] maintenance: add support for systemd timers on Linux
  2021-05-24  7:15     ` [PATCH v4 0/4] add support for " Lénaïc Huard
                         ` (2 preceding siblings ...)
  2021-05-24  7:15       ` [PATCH v4 3/4] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
@ 2021-05-24  7:15       ` Lénaïc Huard
  2021-05-24  9:55         ` Ævar Arnfjörð Bjarmason
                           ` (2 more replies)
  2021-05-24  9:04       ` [PATCH v4 0/4] " Junio C Hamano
  2021-06-08 13:39       ` [PATCH v5 0/3] " Lénaïc Huard
  5 siblings, 3 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-05-24  7:15 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Lénaïc Huard

The existing mechanism for scheduling background maintenance is done
through cron. On Linux systems managed by systemd, systemd provides an
alternative to schedule recurring tasks: systemd timers.

The main motivations to implement systemd timers in addition to cron
are:
* cron is optional and Linux systems running systemd might not have it
  installed.
* The execution of `crontab -l` can tell us if cron is installed but not
  if the daemon is actually running.
* With systemd, each service is run in its own cgroup and its logs are
  tagged by the service inside journald. With cron, all scheduled tasks
  are running in the cron daemon cgroup and all the logs of the
  user-scheduled tasks are pretended to belong to the system cron
  service.
  Concretely, a user that doesn’t have access to the system logs won’t
  have access to the log of their own tasks scheduled by cron whereas
  they will have access to the log of their own tasks scheduled by
  systemd timer.
  Although `cron` attempts to send email, that email may go unseen by
  the user because these days, local mailboxes are not heavily used
  anymore.

In order to schedule git maintenance, we need two unit template files:
* ~/.config/systemd/user/git-maintenance@.service
  to define the command to be started by systemd and
* ~/.config/systemd/user/git-maintenance@.timer
  to define the schedule at which the command should be run.

Those units are templates that are parameterized by the frequency.

Based on those templates, 3 timers are started:
* git-maintenance@hourly.timer
* git-maintenance@daily.timer
* git-maintenance@weekly.timer

The command launched by those three timers are the same as with the
other scheduling methods:

/path/to/git for-each-repo --exec-path=/path/to
--config=maintenance.repo maintenance run --schedule=%i

with the full path for git to ensure that the version of git launched
for the scheduled maintenance is the same as the one used to run
`maintenance start`.

The timer unit contains `Persistent=true` so that, if the computer is
powered down when a maintenance task should run, the task will be run
when the computer is back powered on.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 Documentation/git-maintenance.txt |  57 +++++++++-
 builtin/gc.c                      | 180 ++++++++++++++++++++++++++++++
 t/t7900-maintenance.sh            |  66 ++++++++++-
 3 files changed, 293 insertions(+), 10 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 7c4bb38a2f..50179e010f 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -181,16 +181,19 @@ OPTIONS
 	`maintenance.<task>.enabled` configured as `true` are considered.
 	See the 'TASKS' section for the list of accepted `<task>` values.
 
---scheduler=auto|crontab|launchctl|schtasks::
+--scheduler=auto|crontab|systemd-timer|launchctl|schtasks::
 	When combined with the `start` subcommand, specify the scheduler
 	to use to run the hourly, daily and weekly executions of
 	`git maintenance run`.
 	The possible values for `<scheduler>` depend on the system: `crontab`
-	is available on POSIX systems, `launchctl` is available on
-	MacOS and `schtasks` is available on Windows.
+	is available on POSIX systems, `systemd-timer` is available on Linux
+	systems, `launchctl` is available on MacOS and `schtasks` is available
+	on Windows.
 	By default or when `auto` is specified, a suitable scheduler for
 	the system is used. On MacOS, `launchctl` is used. On Windows,
-	`schtasks` is used. On all other systems, `crontab` is used.
+	`schtasks` is used. On Linux, `systemd-timer` is used if user systemd
+	timers are available, otherwise, `crontab` is used. On all other systems,
+	`crontab` is used.
 
 
 TROUBLESHOOTING
@@ -290,6 +293,52 @@ schedule to ensure you are executing the correct binaries in your
 schedule.
 
 
+BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMS
+-----------------------------------------------
+
+While Linux supports `cron`, depending on the distribution, `cron` may
+be an optional package not necessarily installed. On modern Linux
+distributions, systemd timers are superseding it.
+
+If user systemd timers are available, they will be used as a replacement
+of `cron`.
+
+In this case, `git maintenance start` will create user systemd timer units
+and start the timers. The current list of user-scheduled tasks can be found
+by running `systemctl --user list-timers`. The timers written by `git
+maintenance start` are similar to this:
+
+-----------------------------------------------------------------------
+$ systemctl --user list-timers
+NEXT                         LEFT          LAST                         PASSED     UNIT                         ACTIVATES
+Thu 2021-04-29 19:00:00 CEST 42min left    Thu 2021-04-29 18:00:11 CEST 17min ago  git-maintenance@hourly.timer git-maintenance@hourly.service
+Fri 2021-04-30 00:00:00 CEST 5h 42min left Thu 2021-04-29 00:00:11 CEST 18h ago    git-maintenance@daily.timer  git-maintenance@daily.service
+Mon 2021-05-03 00:00:00 CEST 3 days left   Mon 2021-04-26 00:00:11 CEST 3 days ago git-maintenance@weekly.timer git-maintenance@weekly.service
+-----------------------------------------------------------------------
+
+One timer is registered for each `--schedule=<frequency>` option.
+
+The definition of the systemd units can be inspected in the following files:
+
+-----------------------------------------------------------------------
+~/.config/systemd/user/git-maintenance@.timer
+~/.config/systemd/user/git-maintenance@.service
+~/.config/systemd/user/timers.target.wants/git-maintenance@hourly.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@daily.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@weekly.timer
+-----------------------------------------------------------------------
+
+`git maintenance start` will overwrite these files and start the timer
+again with `systemctl --user`, so any customization should be done by
+creating a drop-in file, i.e. a `.conf` suffixed file in the
+`~/.config/systemd/user/git-maintenance@.service.d` directory.
+
+`git maintenance stop` will stop the user systemd timers and delete
+the above mentioned files.
+
+For more details, see `systemd.timer(5)`.
+
+
 BACKGROUND MAINTENANCE ON MACOS SYSTEMS
 ---------------------------------------
 
diff --git a/builtin/gc.c b/builtin/gc.c
index bf21cec059..3eca1e5e6a 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -2079,10 +2079,173 @@ static int crontab_update_schedule(enum enable_or_disable run_maintenance,
 	return result;
 }
 
+static int is_systemd_timer_available(void)
+{
+	const char *cmd = "systemctl";
+	int is_available;
+	static int cached_result = -1;
+#ifdef __linux__
+	struct child_process child = CHILD_PROCESS_INIT;
+#endif
+
+	if (cached_result != -1)
+		return cached_result;
+
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+#ifdef __linux__
+	strvec_split(&child.args, cmd);
+	strvec_pushl(&child.args, "--user", "list-timers", NULL);
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child)) {
+		cached_result = 0;
+		return cached_result;
+	}
+	if (finish_command(&child)) {
+		cached_result = 0;
+		return cached_result;
+	}
+	cached_result = 1;
+	return cached_result;
+#else
+	return 0;
+#endif
+}
+
+static char *xdg_config_home_systemd(const char *filename)
+{
+	return xdg_config_home_for("systemd/user", filename);
+}
+
+static int systemd_timer_enable_unit(int enable,
+				     enum schedule_priority schedule)
+{
+	const char *cmd = "systemctl";
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *frequency = get_frequency(schedule);
+
+	get_schedule_cmd(&cmd, NULL);
+	strvec_split(&child.args, cmd);
+	strvec_pushl(&child.args, "--user", enable ? "enable" : "disable",
+		     "--now", NULL);
+	strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency);
+
+	if (start_command(&child))
+		die(_("failed to run systemctl"));
+	return finish_command(&child);
+}
+
+static int systemd_timer_delete_unit_templates(void)
+{
+	char *filename = xdg_config_home_systemd("git-maintenance@.timer");
+	unlink(filename);
+	free(filename);
+
+	filename = xdg_config_home_systemd("git-maintenance@.service");
+	unlink(filename);
+	free(filename);
+
+	return 0;
+}
+
+static int systemd_timer_delete_units(void)
+{
+	return systemd_timer_enable_unit(DISABLE, SCHEDULE_HOURLY) ||
+	       systemd_timer_enable_unit(DISABLE, SCHEDULE_DAILY) ||
+	       systemd_timer_enable_unit(DISABLE, SCHEDULE_WEEKLY) ||
+	       systemd_timer_delete_unit_templates();
+}
+
+static int systemd_timer_write_unit_templates(const char *exec_path)
+{
+	char *filename;
+	FILE *file;
+	const char *unit;
+
+	filename = xdg_config_home_systemd("git-maintenance@.timer");
+	if (safe_create_leading_directories(filename))
+		die(_("failed to create directories for '%s'"), filename);
+	file = xfopen(filename, "w");
+	FREE_AND_NULL(filename);
+
+	unit = "# This file was created and is maintained by Git.\n"
+	       "# Any edits made in this file might be replaced in the future\n"
+	       "# by a Git command.\n"
+	       "\n"
+	       "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Timer]\n"
+	       "OnCalendar=%i\n"
+	       "Persistent=true\n"
+	       "\n"
+	       "[Install]\n"
+	       "WantedBy=timers.target\n";
+	fputs(unit, file);
+	fclose(file);
+
+	filename = xdg_config_home_systemd("git-maintenance@.service");
+	file = xfopen(filename, "w");
+	free(filename);
+
+	unit = "# This file was created and is maintained by Git.\n"
+	       "# Any edits made in this file might be replaced in the future\n"
+	       "# by a Git command.\n"
+	       "\n"
+	       "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Service]\n"
+	       "Type=oneshot\n"
+	       "ExecStart=\"%s/git\" --exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
+	       "LockPersonality=yes\n"
+	       "MemoryDenyWriteExecute=yes\n"
+	       "NoNewPrivileges=yes\n"
+	       "RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\n"
+	       "RestrictNamespaces=yes\n"
+	       "RestrictRealtime=yes\n"
+	       "RestrictSUIDSGID=yes\n"
+	       "SystemCallArchitectures=native\n"
+	       "SystemCallFilter=@system-service\n";
+	fprintf(file, unit, exec_path, exec_path);
+	fclose(file);
+
+	return 0;
+}
+
+static int systemd_timer_setup_units(void)
+{
+	const char *exec_path = git_exec_path();
+
+	return systemd_timer_write_unit_templates(exec_path) ||
+	       systemd_timer_enable_unit(ENABLE, SCHEDULE_HOURLY) ||
+	       systemd_timer_enable_unit(ENABLE, SCHEDULE_DAILY) ||
+	       systemd_timer_enable_unit(ENABLE, SCHEDULE_WEEKLY);
+}
+
+static int systemd_timer_update_schedule(enum enable_or_disable run_maintenance,
+					 int fd)
+{
+	switch (run_maintenance) {
+	case ENABLE:
+		return systemd_timer_setup_units();
+	case DISABLE:
+		return systemd_timer_delete_units();
+	default:
+		BUG("invalid enable_or_disable value");
+	}
+}
+
 enum scheduler {
 	SCHEDULER_INVALID = -1,
 	SCHEDULER_AUTO,
 	SCHEDULER_CRON,
+	SCHEDULER_SYSTEMD,
 	SCHEDULER_LAUNCHCTL,
 	SCHEDULER_SCHTASKS,
 };
@@ -2097,6 +2260,11 @@ static const struct {
 		.is_available = is_crontab_available,
 		.update_schedule = crontab_update_schedule,
 	},
+	[SCHEDULER_SYSTEMD] = {
+		.name = "systemctl",
+		.is_available = is_systemd_timer_available,
+		.update_schedule = systemd_timer_update_schedule,
+	},
 	[SCHEDULER_LAUNCHCTL] = {
 		.name = "launchctl",
 		.is_available = is_launchctl_available,
@@ -2117,6 +2285,9 @@ static enum scheduler parse_scheduler(const char *value)
 		return SCHEDULER_AUTO;
 	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
 		return SCHEDULER_CRON;
+	else if (!strcasecmp(value, "systemd") ||
+		 !strcasecmp(value, "systemd-timer"))
+		return SCHEDULER_SYSTEMD;
 	else if (!strcasecmp(value, "launchctl"))
 		return SCHEDULER_LAUNCHCTL;
 	else if (!strcasecmp(value, "schtasks"))
@@ -2155,6 +2326,15 @@ static void resolve_auto_scheduler(enum scheduler *scheduler)
 	*scheduler = SCHEDULER_SCHTASKS;
 	return;
 
+#elif defined(__linux__)
+	if (is_systemd_timer_available())
+		*scheduler = SCHEDULER_SYSTEMD;
+	else if (is_crontab_available())
+		*scheduler = SCHEDULER_CRON;
+	else
+		die(_("neither systemd timers nor crontab are available"));
+	return;
+
 #else
 	*scheduler = SCHEDULER_CRON;
 	return;
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 9eac260307..c8a6f19ebc 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -20,6 +20,18 @@ test_xmllint () {
 	fi
 }
 
+test_lazy_prereq SYSTEMD_ANALYZE '
+	systemd-analyze --help >out &&
+	grep verify out
+'
+
+test_systemd_analyze_verify () {
+	if test_have_prereq SYSTEMD_ANALYZE
+	then
+		systemd-analyze verify "$@"
+	fi
+}
+
 test_expect_success 'help text' '
 	test_expect_code 129 git maintenance -h 2>err &&
 	test_i18ngrep "usage: git maintenance <subcommand>" err &&
@@ -628,14 +640,54 @@ test_expect_success 'start and stop Windows maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success 'start and stop Linux/systemd maintenance' '
+	write_script print-args <<-\EOF &&
+	printf "%s\n" "$*" >>args
+	EOF
+
+	XDG_CONFIG_HOME="$PWD" &&
+	export XDG_CONFIG_HOME &&
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance start --scheduler=systemd-timer &&
+
+	# start registers the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_systemd_analyze_verify "systemd/user/git-maintenance@.service" &&
+
+	printf -- "--user enable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_path_is_missing "systemd/user/git-maintenance@.timer" &&
+	test_path_is_missing "systemd/user/git-maintenance@.service" &&
+
+	printf -- "--user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'start and stop when several schedulers are available' '
 	write_script print-args <<-\EOF &&
 	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
-	rm -f expect &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=systemd-timer &&
+	printf -- "systemctl --user enable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >>expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
 	for frequency in hourly daily weekly
 	do
 		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
@@ -647,17 +699,19 @@ test_expect_success 'start and stop when several schedulers are available' '
 	test_cmp expect args &&
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
 	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
-		hourly daily weekly >expect &&
+		hourly daily weekly >>expect &&
 	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
 		hourly daily weekly >>expect &&
 	test_cmp expect args &&
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
 	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
-		hourly daily weekly >expect &&
+		hourly daily weekly >>expect &&
 	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
 		hourly daily weekly >>expect &&
 	test_cmp expect args
-- 
2.31.1


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* Re: [PATCH v4 0/4] add support for systemd timers on Linux
  2021-05-24  7:15     ` [PATCH v4 0/4] add support for " Lénaïc Huard
                         ` (3 preceding siblings ...)
  2021-05-24  7:15       ` [PATCH v4 4/4] maintenance: add support for systemd timers on Linux Lénaïc Huard
@ 2021-05-24  9:04       ` Junio C Hamano
  2021-06-08 13:39       ` [PATCH v5 0/3] " Lénaïc Huard
  5 siblings, 0 replies; 138+ messages in thread
From: Junio C Hamano @ 2021-05-24  9:04 UTC (permalink / raw)
  To: Lénaïc Huard
  Cc: git, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin

Lénaïc Huard <lenaic@lhuard.fr> writes:

> Hello,
>
> The main difference between this v4 patchset and the previous v3
> concerns ...

Ehh, before that, please be helpful to those who weren't involved in
the discussion of earlier rounds and haven't seen the paches v3 and
below.  What is this topic about in the bigger picture, why readers
should be interested in reading them, on top of what commit is the
series designed to apply, etc. etc.

Thanks.

> the first patch, the one introducing a new function to use
> $XDG_CONFIG_HOME.
> Instead of re-purposing the existing `xdg_config_home(filename)`
> function, I’ve created a new generic one named
> `xdg_config_home_for(prog, filename)` to address the potential
> conflict issue raised by Danh.
>
> I’ve also reworded the commit message of the last patch to address the
> review comments.
> I replaced the “he” by a “they”.
> To be honest, I’m not an English native speaker, so I don’t feel
> comfortable debating which usage is common in informal speech or in
> formal writtings.
> So, I would be happy to change it to “he or she” if it is more
> consensual and looks less “artificial” than “he/she”.
>
> Lénaïc Huard (4):
>   cache.h: Introduce a generic "xdg_config_home_for(…)" function
>   maintenance: introduce ENABLE/DISABLE for code clarity
>   maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
>   maintenance: add support for systemd timers on Linux
>
>  Documentation/git-maintenance.txt |  60 ++++
>  builtin/gc.c                      | 548 ++++++++++++++++++++++++++----
>  cache.h                           |   7 +
>  path.c                            |  13 +-
>  t/t7900-maintenance.sh            | 110 +++++-
>  5 files changed, 657 insertions(+), 81 deletions(-)

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v4 1/4] cache.h: Introduce a generic "xdg_config_home_for(…)" function
  2021-05-24  7:15       ` [PATCH v4 1/4] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
@ 2021-05-24  9:33         ` Phillip Wood
  2021-05-24 12:23           ` Đoàn Trần Công Danh
  0 siblings, 1 reply; 138+ messages in thread
From: Phillip Wood @ 2021-05-24  9:33 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Martin Ågren, Ævar Arnfjörð Bjarmason,
	Bagas Sanjaya, brian m . carlson, Johannes Schindelin

Hi Lénaïc

This looks fine to me. I'm not 100% sold on calling the parameter prog 
as our program name later in the series ends up being "systemd/user" so 
something like "subdir" might have been better but that is not worth 
rerolling for.

Best Wishes

Phillip

On 24/05/2021 08:15, Lénaïc Huard wrote:
> Current implementation of `xdg_config_home(filename)` returns
> `$XDG_CONFIG_HOME/git/$filename`, with the `git` subdirectory inserted
> between the `XDG_CONFIG_HOME` environment variable and the parameter.
> 
> This patch introduces a `xdg_config_home_for(prog, filename)` function
> which is more generic. It only concatenates "$XDG_CONFIG_HOME", or
> "$HOME/.config" if the former isn’t defined, with the parameters,
> without adding `git` in between.
> 
> `xdg_config_home(filename)` is now implemented by calling
> `xdg_config_home_for("git", filename)` but this new generic function can
> be used to compute the configuration directory of other programs.
> 
> Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
> ---
>   cache.h |  7 +++++++
>   path.c  | 13 ++++++++++---
>   2 files changed, 17 insertions(+), 3 deletions(-)
> 
> diff --git a/cache.h b/cache.h
> index 148d9ab5f1..8a2969414a 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -1263,6 +1263,13 @@ int is_ntfs_dotgitattributes(const char *name);
>    */
>   int looks_like_command_line_option(const char *str);
>   
> +/**
> + * Return a newly allocated string with the evaluation of
> + * "$XDG_CONFIG_HOME/$prog/$filename" if $XDG_CONFIG_HOME is non-empty, otherwise
> + * "$HOME/.config/$prog/$filename". Return NULL upon error.
> + */
> +char *xdg_config_home_for(const char *prog, const char *filename);
> +
>   /**
>    * Return a newly allocated string with the evaluation of
>    * "$XDG_CONFIG_HOME/git/$filename" if $XDG_CONFIG_HOME is non-empty, otherwise
> diff --git a/path.c b/path.c
> index 7b385e5eb2..3641d4c456 100644
> --- a/path.c
> +++ b/path.c
> @@ -1498,21 +1498,28 @@ int looks_like_command_line_option(const char *str)
>   	return str && str[0] == '-';
>   }
>   
> -char *xdg_config_home(const char *filename)
> +char *xdg_config_home_for(const char *prog, const char *filename)
>   {
>   	const char *home, *config_home;
>   
> +	assert(prog);
>   	assert(filename);
>   	config_home = getenv("XDG_CONFIG_HOME");
>   	if (config_home && *config_home)
> -		return mkpathdup("%s/git/%s", config_home, filename);
> +		return mkpathdup("%s/%s/%s", config_home, prog, filename);
>   
>   	home = getenv("HOME");
>   	if (home)
> -		return mkpathdup("%s/.config/git/%s", home, filename);
> +		return mkpathdup("%s/.config/%s/%s", home, prog, filename);
> +
>   	return NULL;
>   }
>   
> +char *xdg_config_home(const char *filename)
> +{
> +	return xdg_config_home_for("git", filename);
> +}
> +
>   char *xdg_cache_home(const char *filename)
>   {
>   	const char *home, *cache_home;
> 


^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v4 2/4] maintenance: introduce ENABLE/DISABLE for code clarity
  2021-05-24  7:15       ` [PATCH v4 2/4] maintenance: introduce ENABLE/DISABLE for code clarity Lénaïc Huard
@ 2021-05-24  9:41         ` Phillip Wood
  2021-05-24 12:36           ` Đoàn Trần Công Danh
  2021-05-24  9:47         ` Ævar Arnfjörð Bjarmason
  1 sibling, 1 reply; 138+ messages in thread
From: Phillip Wood @ 2021-05-24  9:41 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Martin Ågren, Ævar Arnfjörð Bjarmason,
	Bagas Sanjaya, brian m . carlson, Johannes Schindelin

Hi Lénaïc

On 24/05/2021 08:15, Lénaïc Huard wrote:
> The first parameter of `XXX_update_schedule` and alike functions is a
> boolean specifying if the tasks should be scheduled or unscheduled.
> 
> Using an `enum` with `ENABLE` and `DISABLE` values can make the code
> clearer.

I'm sorry to say that I'm not sure this does make the code clearer 
overall - I wish I'd spoken up when Danh suggested it.
While
	launchctl_boot_plist(DISABLE, filename, cmd)
is arguably clearer than
	launchctl_boot_plist(0, filename, cmd)
we end up with bizarre tests like
  	if (enabled == ENABLED)
rather than
	if (enabled)
and in the next patch we have
	(enable == ENABLE && (opts->scheduler == i)) ?
			ENABLE : DISABLE;
rather than
	enable && opts->scheduler == i

Also looking at the next patch it seems as this one is missing some 
conversions in maintenance_start() as it is still calling 
update_background_schedule() with an integer rather than the new enum.

I'd be happy to see this being dropped I'm afraid

Best Wishes

Phillip

> Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
> ---
>   builtin/gc.c | 49 +++++++++++++++++++++++++++++++------------------
>   1 file changed, 31 insertions(+), 18 deletions(-)
> 
> diff --git a/builtin/gc.c b/builtin/gc.c
> index ef7226d7bc..0caf8d45c4 100644
> --- a/builtin/gc.c
> +++ b/builtin/gc.c
> @@ -1570,19 +1570,21 @@ static char *launchctl_get_uid(void)
>   	return xstrfmt("gui/%d", getuid());
>   }
>   
> -static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
> +enum enable_or_disable {
> +	DISABLE,
> +	ENABLE
> +};
> +
> +static int launchctl_boot_plist(enum enable_or_disable enable,
> +				const char *filename, const char *cmd)
>   {
>   	int result;
>   	struct child_process child = CHILD_PROCESS_INIT;
>   	char *uid = launchctl_get_uid();
>   
>   	strvec_split(&child.args, cmd);
> -	if (enable)
> -		strvec_push(&child.args, "bootstrap");
> -	else
> -		strvec_push(&child.args, "bootout");
> -	strvec_push(&child.args, uid);
> -	strvec_push(&child.args, filename);
> +	strvec_pushl(&child.args, enable == ENABLE ? "bootstrap" : "bootout",
> +		     uid, filename, NULL);
>   
>   	child.no_stderr = 1;
>   	child.no_stdout = 1;
> @@ -1601,7 +1603,7 @@ static int launchctl_remove_plist(enum schedule_priority schedule, const char *c
>   	const char *frequency = get_frequency(schedule);
>   	char *name = launchctl_service_name(frequency);
>   	char *filename = launchctl_service_filename(name);
> -	int result = launchctl_boot_plist(0, filename, cmd);
> +	int result = launchctl_boot_plist(DISABLE, filename, cmd);
>   	unlink(filename);
>   	free(filename);
>   	free(name);
> @@ -1684,8 +1686,8 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
>   	fclose(plist);
>   
>   	/* bootout might fail if not already running, so ignore */
> -	launchctl_boot_plist(0, filename, cmd);
> -	if (launchctl_boot_plist(1, filename, cmd))
> +	launchctl_boot_plist(DISABLE, filename, cmd);
> +	if (launchctl_boot_plist(ENABLE, filename, cmd))
>   		die(_("failed to bootstrap service %s"), filename);
>   
>   	free(filename);
> @@ -1702,12 +1704,17 @@ static int launchctl_add_plists(const char *cmd)
>   		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
>   }
>   
> -static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)
> +static int launchctl_update_schedule(enum enable_or_disable run_maintenance,
> +				     int fd, const char *cmd)
>   {
> -	if (run_maintenance)
> +	switch (run_maintenance) {
> +	case ENABLE:
>   		return launchctl_add_plists(cmd);
> -	else
> +	case DISABLE:
>   		return launchctl_remove_plists(cmd);
> +	default:
> +		BUG("invalid enable_or_disable value");
> +	}
>   }
>   
>   static char *schtasks_task_name(const char *frequency)
> @@ -1864,18 +1871,24 @@ static int schtasks_schedule_tasks(const char *cmd)
>   		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
>   }
>   
> -static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)
> +static int schtasks_update_schedule(enum enable_or_disable run_maintenance,
> +				    int fd, const char *cmd)
>   {
> -	if (run_maintenance)
> +	switch (run_maintenance) {
> +	case ENABLE:
>   		return schtasks_schedule_tasks(cmd);
> -	else
> +	case DISABLE:
>   		return schtasks_remove_tasks(cmd);
> +	default:
> +		BUG("invalid enable_or_disable value");
> +	}
>   }
>   
>   #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
>   #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
>   
> -static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
> +static int crontab_update_schedule(enum enable_or_disable run_maintenance,
> +				   int fd, const char *cmd)
>   {
>   	int result = 0;
>   	int in_old_region = 0;
> @@ -1925,7 +1938,7 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
>   			fprintf(cron_in, "%s\n", line.buf);
>   	}
>   
> -	if (run_maintenance) {
> +	if (run_maintenance == ENABLE) {
>   		struct strbuf line_format = STRBUF_INIT;
>   		const char *exec_path = git_exec_path();
>   
> 


^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v4 2/4] maintenance: introduce ENABLE/DISABLE for code clarity
  2021-05-24  7:15       ` [PATCH v4 2/4] maintenance: introduce ENABLE/DISABLE for code clarity Lénaïc Huard
  2021-05-24  9:41         ` Phillip Wood
@ 2021-05-24  9:47         ` Ævar Arnfjörð Bjarmason
  1 sibling, 0 replies; 138+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-24  9:47 UTC (permalink / raw)
  To: Lénaïc Huard
  Cc: git, Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin


On Mon, May 24 2021, Lénaïc Huard wrote:

> The first parameter of `XXX_update_schedule` and alike functions is a
> boolean specifying if the tasks should be scheduled or unscheduled.
>
> Using an `enum` with `ENABLE` and `DISABLE` values can make the code
> clearer.

I'm a fan of enums in general for N values, but I think for this sort of
boolean case it's stepping into the territory of just making things less
readable. There's nothing unreadable about 0/1 as an on/off.

> -static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
> +enum enable_or_disable {
> +	DISABLE,
> +	ENABLE
> +};

So here we have ENABLE/DISABLE...

> +static int launchctl_boot_plist(enum enable_or_disable enable,
> +				const char *filename, const char *cmd)
>  {
>  	int result;
>  	struct child_process child = CHILD_PROCESS_INIT;
>  	char *uid = launchctl_get_uid();
>  
>  	strvec_split(&child.args, cmd);
> -	if (enable)
> -		strvec_push(&child.args, "bootstrap");
> -	else
> -		strvec_push(&child.args, "bootout");
> -	strvec_push(&child.args, uid);
> -	strvec_push(&child.args, filename);
> +	strvec_pushl(&child.args, enable == ENABLE ? "bootstrap" : "bootout",
> +		     uid, filename, NULL);

..And here we just check ENABLE, and assume !ENABLE == DISABLE...
[...]

>  {
> -	if (run_maintenance)
> +	switch (run_maintenance) {
> +	case ENABLE:
>  		return launchctl_add_plists(cmd);
> -	else
> +	case DISABLE:
>  		return launchctl_remove_plists(cmd);
> +	default:
> +		BUG("invalid enable_or_disable value");
> +	}
>  }

And here we use a switch, but also a "default". It's actually better if
you're going to use an enum like this to leave out the "default", the
compiler will catch non-enumerated values for us.

>  
>  static char *schtasks_task_name(const char *frequency)
> @@ -1864,18 +1871,24 @@ static int schtasks_schedule_tasks(const char *cmd)
>  		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
>  }
>  
> -static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)
> +static int schtasks_update_schedule(enum enable_or_disable run_maintenance,
> +				    int fd, const char *cmd)
>  {
> -	if (run_maintenance)
> +	switch (run_maintenance) {
> +	case ENABLE:
>  		return schtasks_schedule_tasks(cmd);
> -	else
> +	case DISABLE:
>  		return schtasks_remove_tasks(cmd);
> +	default:
> +		BUG("invalid enable_or_disable value");
> +	}
>  }

As an aside (I haven't read much/all the context) I wonder why we have
these wrapper functions, can't the caller just pass an "enable" flag?

>  #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
>  #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
>  
> -static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
> +static int crontab_update_schedule(enum enable_or_disable run_maintenance,
> +				   int fd, const char *cmd)
>  {
>  	int result = 0;
>  	int in_old_region = 0;
> @@ -1925,7 +1938,7 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
>  			fprintf(cron_in, "%s\n", line.buf);
>  	}
>  
> -	if (run_maintenance) {
> +	if (run_maintenance == ENABLE) {
>  		struct strbuf line_format = STRBUF_INIT;
>  		const char *exec_path = git_exec_path();

Same !ENABLE == DISABLE assumption?


^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v4 4/4] maintenance: add support for systemd timers on Linux
  2021-05-24  7:15       ` [PATCH v4 4/4] maintenance: add support for systemd timers on Linux Lénaïc Huard
@ 2021-05-24  9:55         ` Ævar Arnfjörð Bjarmason
  2021-05-24 16:39           ` Eric Sunshine
  2021-05-24 18:08         ` Felipe Contreras
  2021-05-26 10:26         ` Phillip Wood
  2 siblings, 1 reply; 138+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-24  9:55 UTC (permalink / raw)
  To: Lénaïc Huard
  Cc: git, Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin


On Mon, May 24 2021, Lénaïc Huard wrote:

If we're going to use ifdefs then... 

> +static int is_systemd_timer_available(void)
> +{
> +	const char *cmd = "systemctl";
> +	int is_available;
> +	static int cached_result = -1;
> +#ifdef __linux__
> +	struct child_process child = CHILD_PROCESS_INIT;
> +#endif
> +
> +	if (cached_result != -1)
> +		return cached_result;
> +
> +	if (get_schedule_cmd(&cmd, &is_available))
> +		return is_available;
> +
> +#ifdef __linux__
> +	strvec_split(&child.args, cmd);
> +	strvec_pushl(&child.args, "--user", "list-timers", NULL);
> +	child.no_stdin = 1;
> +	child.no_stdout = 1;
> +	child.no_stderr = 1;
> +	child.silent_exec_failure = 1;
> +
> +	if (start_command(&child)) {
> +		cached_result = 0;
> +		return cached_result;
> +	}
> +	if (finish_command(&child)) {
> +		cached_result = 0;
> +		return cached_result;
> +	}
> +	cached_result = 1;
> +	return cached_result;
> +#else
> +	return 0;
> +#endif
> +}

This sort of function would, I think, be clearer if we just had two
different functions in an ifdef/else, e.g. here cached_result" is
checked under !__linux__, but is never set in that case.

But aside from that, why we do we need all this is_available caching,
isn't this all called as a one-off? If it's not, it seems much better to
push that cache one level up the callstack. Have whatevere's keeping
track of the struct cache is_available itself, or cache it in another
(or the same, then tristate) field in the same struct.

> [...]
> +test_expect_success 'start and stop Linux/systemd maintenance' '
> +	write_script print-args <<-\EOF &&
> +	printf "%s\n" "$*" >>args
> +	EOF
> +
> +	XDG_CONFIG_HOME="$PWD" &&
> +	export XDG_CONFIG_HOME &&
> +	rm -f args &&

If you're going to care about cleanup here, and personally I wouldn't,
just call that "args" by the name "expect" instead (as is convention)
and clobber it every time...

Anyway, a better way to do the cleanup is:

    test_when_finished "rm args" &&
    echo this is the first time you write the file >args
    [the rest of the test code]

Then you don't need to re-rm it.

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v4 3/4] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-05-24  7:15       ` [PATCH v4 3/4] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
@ 2021-05-24 10:12         ` Phillip Wood
  2021-05-30  6:39           ` Lénaïc Huard
  0 siblings, 1 reply; 138+ messages in thread
From: Phillip Wood @ 2021-05-24 10:12 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Martin Ågren, Ævar Arnfjörð Bjarmason,
	Bagas Sanjaya, brian m . carlson, Johannes Schindelin

Hi Lénaïc

On 24/05/2021 08:15, Lénaïc Huard wrote:
> Depending on the system, different schedulers can be used to schedule
> the hourly, daily and weekly executions of `git maintenance run`:
> * `launchctl` for MacOS,
> * `schtasks` for Windows and
> * `crontab` for everything else.
> 
> `git maintenance run` now has an option to let the end-user explicitly
> choose which scheduler he wants to use:
> `--scheduler=auto|crontab|launchctl|schtasks`.
> 
> When `git maintenance start --scheduler=XXX` is run, it not only
> registers `git maintenance run` tasks in the scheduler XXX, it also
> removes the `git maintenance run` tasks from all the other schedulers to
> ensure we cannot have two schedulers launching concurrent identical
> tasks.
> 
> The default value is `auto` which chooses a suitable scheduler for the
> system.
> 
> `git maintenance stop` doesn't have any `--scheduler` parameter because
> this command will try to remove the `git maintenance run` tasks from all
> the available schedulers.

I like this change, it makes the test infrastructure less intrusive.

> Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
> ---
>   Documentation/git-maintenance.txt |  11 +
>   builtin/gc.c                      | 333 ++++++++++++++++++++++++------
>   t/t7900-maintenance.sh            |  56 ++++-
>   3 files changed, 333 insertions(+), 67 deletions(-)
> 
> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
> index 80ddd33ceb..7c4bb38a2f 100644
> --- a/Documentation/git-maintenance.txt
> +++ b/Documentation/git-maintenance.txt
> @@ -181,6 +181,17 @@ OPTIONS
>   	`maintenance.<task>.enabled` configured as `true` are considered.
>   	See the 'TASKS' section for the list of accepted `<task>` values.
>   
> +--scheduler=auto|crontab|launchctl|schtasks::
> +	When combined with the `start` subcommand, specify the scheduler
> +	to use to run the hourly, daily and weekly executions of
> +	`git maintenance run`.
> +	The possible values for `<scheduler>` depend on the system: `crontab`
> +	is available on POSIX systems, `launchctl` is available on
> +	MacOS and `schtasks` is available on Windows.
> +	By default or when `auto` is specified, a suitable scheduler for
> +	the system is used. On MacOS, `launchctl` is used. On Windows,
> +	`schtasks` is used. On all other systems, `crontab` is used.
> +
>   
>   TROUBLESHOOTING
>   ---------------
> diff --git a/builtin/gc.c b/builtin/gc.c
> index 0caf8d45c4..bf21cec059 100644
> --- a/builtin/gc.c
> +++ b/builtin/gc.c
> @@ -1544,6 +1544,60 @@ static const char *get_frequency(enum schedule_priority schedule)
>   	}
>   }
>   
> +static int get_schedule_cmd(const char **cmd, int *is_available)
> +{
> +	char *item;
> +	static char test_cmd[32];
> +	char *testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
> +
> +	if (!testing)
> +		return 0;
> +
> +	if (is_available)
> +		*is_available = 0;
> +
> +	for(item = testing;;) {
> +		char *sep;
> +		char *end_item = strchr(item, ',');
> +		if (end_item)
> +			*end_item = '\0';
> +
> +		sep = strchr(item, ':');
> +		if (!sep)
> +			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
> +		*sep = '\0';
> +
> +		if (!strcmp(*cmd, item)) {
> +			strlcpy(test_cmd, sep+1, ARRAY_SIZE(test_cmd));

This falls into the trap of assuming strlcpy() is safe and does not 
check the return value. It will silently truncate the command if it is 
too long. As this is for testing I'd be happy just to return 'sep + 1' 
in *cmd and leak it. We could mark 'testing' with UNLEAK() to keep asan 
happy.

> +			*cmd = test_cmd;
> +			if (is_available)
> +				*is_available = 1;
> +			break;
> +		}
> +
> +		if (!end_item)
> +			break;
> +		item = end_item + 1;
> +	}
> +
> +	free(testing);
> +	return 1;
> +}
> +
> +static int is_launchctl_available(void)
> +{
> +	const char *cmd = "launchctl";
> +	int is_available;
> +	if (get_schedule_cmd(&cmd, &is_available))
> +		return is_available;
> +
> +#ifdef __APPLE__
> +	return 1;
> +#else
> +	return 0;
> +#endif
> +}
> +
>   static char *launchctl_service_name(const char *frequency)
>   {
>   	struct strbuf label = STRBUF_INIT;
> @@ -1576,12 +1630,14 @@ enum enable_or_disable {
>   };
>   
>   static int launchctl_boot_plist(enum enable_or_disable enable,
> -				const char *filename, const char *cmd)
> +				const char *filename)

I'm so pleased to see all these 'cmd' arguments disappearing!

>   {
> +	const char *cmd = "launchctl";
>   	int result;
>   	struct child_process child = CHILD_PROCESS_INIT;
>   	char *uid = launchctl_get_uid();
>   
> +	get_schedule_cmd(&cmd, NULL);
>   	strvec_split(&child.args, cmd);

It's a shame we still have to have this strvec_split() call just to 
handle testing but your changes are an improvement.

>   	strvec_pushl(&child.args, enable == ENABLE ? "bootstrap" : "bootout",
>   		     uid, filename, NULL);
> @@ -1598,26 +1654,26 @@ static int launchctl_boot_plist(enum enable_or_disable enable,
>   	return result;
>   }
>   
> -static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)
> +static int launchctl_remove_plist(enum schedule_priority schedule)
>   {
>   	const char *frequency = get_frequency(schedule);
>   	char *name = launchctl_service_name(frequency);
>   	char *filename = launchctl_service_filename(name);
> -	int result = launchctl_boot_plist(DISABLE, filename, cmd);
> +	int result = launchctl_boot_plist(DISABLE, filename);
>   	unlink(filename);
>   	free(filename);
>   	free(name);
>   	return result;
>   }
>   
> -static int launchctl_remove_plists(const char *cmd)
> +static int launchctl_remove_plists(void)
>   {
> -	return launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||
> -		launchctl_remove_plist(SCHEDULE_DAILY, cmd) ||
> -		launchctl_remove_plist(SCHEDULE_WEEKLY, cmd);
> +	return launchctl_remove_plist(SCHEDULE_HOURLY) ||
> +		launchctl_remove_plist(SCHEDULE_DAILY) ||
> +		launchctl_remove_plist(SCHEDULE_WEEKLY);
>   }
>   
> -static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)
> +static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule)
>   {
>   	FILE *plist;
>   	int i;
> @@ -1686,8 +1742,8 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
>   	fclose(plist);
>   
>   	/* bootout might fail if not already running, so ignore */
> -	launchctl_boot_plist(DISABLE, filename, cmd);
> -	if (launchctl_boot_plist(ENABLE, filename, cmd))
> +	launchctl_boot_plist(DISABLE, filename);
> +	if (launchctl_boot_plist(ENABLE, filename))
>   		die(_("failed to bootstrap service %s"), filename);
>   
>   	free(filename);
> @@ -1695,28 +1751,42 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
>   	return 0;
>   }
>   
> -static int launchctl_add_plists(const char *cmd)
> +static int launchctl_add_plists(void)
>   {
>   	const char *exec_path = git_exec_path();
>   
> -	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||
> -		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||
> -		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
> +	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY) ||
> +		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY) ||
> +		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY);
>   }
>   
>   static int launchctl_update_schedule(enum enable_or_disable run_maintenance,
> -				     int fd, const char *cmd)
> +				     int fd)
>   {
>   	switch (run_maintenance) {
>   	case ENABLE:
> -		return launchctl_add_plists(cmd);
> +		return launchctl_add_plists();
>   	case DISABLE:
> -		return launchctl_remove_plists(cmd);
> +		return launchctl_remove_plists();
>   	default:
>   		BUG("invalid enable_or_disable value");
>   	}
>   }
>   
> +static int is_schtasks_available(void)
> +{
> +	const char *cmd = "schtasks";
> +	int is_available;
> +	if (get_schedule_cmd(&cmd, &is_available))
> +		return is_available;
> +
> +#ifdef GIT_WINDOWS_NATIVE
> +	return 1;
> +#else
> +	return 0;
> +#endif
> +}
> +
>   static char *schtasks_task_name(const char *frequency)
>   {
>   	struct strbuf label = STRBUF_INIT;
> @@ -1724,13 +1794,15 @@ static char *schtasks_task_name(const char *frequency)
>   	return strbuf_detach(&label, NULL);
>   }
>   
> -static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)
> +static int schtasks_remove_task(enum schedule_priority schedule)
>   {
> +	const char *cmd = "schtasks";
>   	int result;
>   	struct strvec args = STRVEC_INIT;
>   	const char *frequency = get_frequency(schedule);
>   	char *name = schtasks_task_name(frequency);
>   
> +	get_schedule_cmd(&cmd, NULL);
>   	strvec_split(&args, cmd);
>   	strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
>   
> @@ -1741,15 +1813,16 @@ static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd
>   	return result;
>   }
>   
> -static int schtasks_remove_tasks(const char *cmd)
> +static int schtasks_remove_tasks(void)
>   {
> -	return schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||
> -		schtasks_remove_task(SCHEDULE_DAILY, cmd) ||
> -		schtasks_remove_task(SCHEDULE_WEEKLY, cmd);
> +	return schtasks_remove_task(SCHEDULE_HOURLY) ||
> +		schtasks_remove_task(SCHEDULE_DAILY) ||
> +		schtasks_remove_task(SCHEDULE_WEEKLY);
>   }
>   
> -static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
> +static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule)
>   {
> +	const char *cmd = "schtasks";
>   	int result;
>   	struct child_process child = CHILD_PROCESS_INIT;
>   	const char *xml;
> @@ -1758,6 +1831,8 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
>   	char *name = schtasks_task_name(frequency);
>   	struct strbuf tfilename = STRBUF_INIT;
>   
> +	get_schedule_cmd(&cmd, NULL);
> +
>   	strbuf_addf(&tfilename, "%s/schedule_%s_XXXXXX",
>   		    get_git_common_dir(), frequency);
>   	tfile = xmks_tempfile(tfilename.buf);
> @@ -1862,34 +1937,65 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
>   	return result;
>   }
>   
> -static int schtasks_schedule_tasks(const char *cmd)
> +static int schtasks_schedule_tasks(void)
>   {
>   	const char *exec_path = git_exec_path();
>   
> -	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||
> -		schtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||
> -		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
> +	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY) ||
> +		schtasks_schedule_task(exec_path, SCHEDULE_DAILY) ||
> +		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY);
>   }
>   
>   static int schtasks_update_schedule(enum enable_or_disable run_maintenance,
> -				    int fd, const char *cmd)
> +				    int fd)
>   {
>   	switch (run_maintenance) {
>   	case ENABLE:
> -		return schtasks_schedule_tasks(cmd);
> +		return schtasks_schedule_tasks();
>   	case DISABLE:
> -		return schtasks_remove_tasks(cmd);
> +		return schtasks_remove_tasks();
>   	default:
>   		BUG("invalid enable_or_disable value");
>   	}
>   }
>   
> +static int is_crontab_available(void)
> +{
> +	const char *cmd = "crontab";
> +	int is_available;
> +	static int cached_result = -1;
> +	struct child_process child = CHILD_PROCESS_INIT;
> +
> +	if (cached_result != -1)
> +		return cached_result;
> +
> +	if (get_schedule_cmd(&cmd, &is_available) && !is_available)
> +		return 0;
> +
> +	strvec_split(&child.args, cmd);
> +	strvec_push(&child.args, "-l");
> +	child.no_stdin = 1;
> +	child.no_stdout = 1;
> +	child.no_stderr = 1;
> +	child.silent_exec_failure = 1;
> +
> +	if (start_command(&child)) {
> +		cached_result = 0;
> +		return cached_result;
> +	}
> +	/* Ignore exit code, as an empty crontab will return error. */
> +	finish_command(&child);
> +	cached_result = 1;
> +	return cached_result;
> +}
> +
>   #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
>   #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
>   
>   static int crontab_update_schedule(enum enable_or_disable run_maintenance,
> -				   int fd, const char *cmd)
> +				   int fd)
>   {
> +	const char *cmd = "crontab";
>   	int result = 0;
>   	int in_old_region = 0;
>   	struct child_process crontab_list = CHILD_PROCESS_INIT;
> @@ -1897,6 +2003,7 @@ static int crontab_update_schedule(enum enable_or_disable run_maintenance,
>   	FILE *cron_list, *cron_in;
>   	struct strbuf line = STRBUF_INIT;
>   
> +	get_schedule_cmd(&cmd, NULL);
>   	strvec_split(&crontab_list.args, cmd);
>   	strvec_push(&crontab_list.args, "-l");
>   	crontab_list.in = -1;
> @@ -1972,61 +2079,161 @@ static int crontab_update_schedule(enum enable_or_disable run_maintenance,
>   	return result;
>   }
>   
> +enum scheduler {
> +	SCHEDULER_INVALID = -1,
> +	SCHEDULER_AUTO,
> +	SCHEDULER_CRON,
> +	SCHEDULER_LAUNCHCTL,
> +	SCHEDULER_SCHTASKS,
> +};
> +
> +static const struct {
> +	const char *name;
> +	int (*is_available)(void);
> +	int (*update_schedule)(enum enable_or_disable run_maintenance, int fd);
> +} scheduler_fn[] = {
> +	[SCHEDULER_CRON] = {
> +		.name = "crontab",
> +		.is_available = is_crontab_available,
> +		.update_schedule = crontab_update_schedule,
> +	},
> +	[SCHEDULER_LAUNCHCTL] = {
> +		.name = "launchctl",
> +		.is_available = is_launchctl_available,
> +		.update_schedule = launchctl_update_schedule,
> +	},
> +	[SCHEDULER_SCHTASKS] = {
> +		.name = "schtasks",
> +		.is_available = is_schtasks_available,
> +		.update_schedule = schtasks_update_schedule,
> +	},
> +};
> +
> +static enum scheduler parse_scheduler(const char *value)
> +{
> +	if (!value)
> +		return SCHEDULER_INVALID;
> +	else if (!strcasecmp(value, "auto"))
> +		return SCHEDULER_AUTO;
> +	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
> +		return SCHEDULER_CRON;
> +	else if (!strcasecmp(value, "launchctl"))
> +		return SCHEDULER_LAUNCHCTL;
> +	else if (!strcasecmp(value, "schtasks"))
> +		return SCHEDULER_SCHTASKS;
> +	else
> +		return SCHEDULER_INVALID;
> +}
> +
> +static int maintenance_opt_scheduler(const struct option *opt, const char *arg,
> +				     int unset)
> +{
> +	enum scheduler *scheduler = opt->value;
> +
> +	BUG_ON_OPT_NEG(unset);
> +
> +	*scheduler = parse_scheduler(arg);
> +	if (*scheduler == SCHEDULER_INVALID)
> +		return error(_("unrecognized --scheduler argument '%s'"), arg);
> +	return 0;
> +}
> +
> +struct maintenance_start_opts {
> +	enum scheduler scheduler;
> +};
> +
> +static void resolve_auto_scheduler(enum scheduler *scheduler)
> +{
> +	if (*scheduler != SCHEDULER_AUTO)
> +		return;
> +
>   #if defined(__APPLE__)
> -static const char platform_scheduler[] = "launchctl";
> +	*scheduler = SCHEDULER_LAUNCHCTL;
> +	return;
> +
>   #elif defined(GIT_WINDOWS_NATIVE)
> -static const char platform_scheduler[] = "schtasks";
> +	*scheduler = SCHEDULER_SCHTASKS;
> +	return;
> +
>   #else
> -static const char platform_scheduler[] = "crontab";
> +	*scheduler = SCHEDULER_CRON;
> +	return;
>   #endif
> +}
>   
> -static int update_background_schedule(int enable)
> +static void validate_scheduler(enum scheduler scheduler)
>   {
> -	int result;
> -	const char *scheduler = platform_scheduler;
> -	const char *cmd = scheduler;
> -	char *testing;
> +	if (scheduler == SCHEDULER_INVALID)
> +		BUG("invalid scheduler");
> +	if (scheduler == SCHEDULER_AUTO)
> +		BUG("resolve_auto_scheduler should have been called before");
> +
> +	if (!scheduler_fn[scheduler].is_available())
> +		die(_("%s scheduler is not available"),
> +		    scheduler_fn[scheduler].name);
> +}
> +
> +static int update_background_schedule(const struct maintenance_start_opts *opts,
> +				      enum enable_or_disable enable)
> +{
> +	unsigned int i;
> +	int res, result = 0;
>   	struct lock_file lk;
>   	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
>   
> -	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
> -	if (testing) {
> -		char *sep = strchr(testing, ':');
> -		if (!sep)
> -			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
> -		*sep = '\0';
> -		scheduler = testing;
> -		cmd = sep + 1;
> -	}
> -
>   	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
>   		return error(_("another process is scheduling background maintenance"));
>   
> -	if (!strcmp(scheduler, "launchctl"))
> -		result = launchctl_update_schedule(enable, get_lock_file_fd(&lk), cmd);
> -	else if (!strcmp(scheduler, "schtasks"))
> -		result = schtasks_update_schedule(enable, get_lock_file_fd(&lk), cmd);
> -	else if (!strcmp(scheduler, "crontab"))
> -		result = crontab_update_schedule(enable, get_lock_file_fd(&lk), cmd);
> -	else
> -		die("unknown background scheduler: %s", scheduler);
> +	for (i = 1; i < ARRAY_SIZE(scheduler_fn); i++) {
> +		enum enable_or_disable enable_scheduler =
> +			(enable == ENABLE && (opts->scheduler == i)) ?

Strictly speaking none of the parenthesis are required, I'd certainly 
drop the inner ones.

> +			ENABLE : DISABLE;
> +		if (!scheduler_fn[i].is_available())
> +			continue;
> +		res = scheduler_fn[i].update_schedule(
> +			enable_scheduler, get_lock_file_fd(&lk));

It does not matter for this patch as we're not really disabling anything 
with the existing range of schedulers but in the next patch we can end 
up enabling the new scheduler before disabling the old one leading to a 
race where they can both start 'git maintenance' on the same repo. As I 
said in my previous review it would be clearer to disable all the 
schedulers first before enabling the new one.

> +		if (enable_scheduler)
> +			result = res;
> +	}
>   
>   	rollback_lock_file(&lk);
> -	free(testing);
>   	return result;
>   }
>   
> -static int maintenance_start(void)
> +static const char *const builtin_maintenance_start_usage[] = {
> +	N_("git maintenance start [--scheduler=<scheduler>]"), NULL
> +};

I'm not sure what the { } and NULL are doing here, it should just be 
assigning a string. You could put it inside maintenance_start() and just 
call the variable "usage"

> +static int maintenance_start(int argc, const char **argv, const char *prefix)
>   {
> +	struct maintenance_start_opts opts;
> +	struct option builtin_maintenance_start_options[] = {

As this variable is local to the function you could call it something 
shorter like options.

> +		OPT_CALLBACK_F(
> +			0, "scheduler", &opts.scheduler, N_("scheduler"),
> +			N_("scheduler to use to trigger git maintenance run"),
> +			PARSE_OPT_NONEG, maintenance_opt_scheduler),
> +		OPT_END()
> +	};
> +	memset(&opts, 0, sizeof(opts));
> +
> +	argc = parse_options(argc, argv, prefix,
> +			     builtin_maintenance_start_options,
> +			     builtin_maintenance_start_usage, 0);
> +	if (argc)
> +		usage_with_options(builtin_maintenance_start_usage,
> +				   builtin_maintenance_start_options);
> +
> +	resolve_auto_scheduler(&opts.scheduler);
> +	validate_scheduler(opts.scheduler);
> +
>   	if (maintenance_register())
>   		warning(_("failed to add repo to global config"));
> -
> -	return update_background_schedule(1);
> +	return update_background_schedule(&opts, 1);

This conversion got missed in the last patch (that is if we end up 
wanting to use an enum).

>   }
>   
>   static int maintenance_stop(void)
>   {
> -	return update_background_schedule(0);
> +	return update_background_schedule(NULL, 0);
>   }
>   
>   static const char builtin_maintenance_usage[] =	N_("git maintenance <subcommand> [<options>]");
> @@ -2040,7 +2247,7 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
>   	if (!strcmp(argv[1], "run"))
>   		return maintenance_run(argc - 1, argv + 1, prefix);
>   	if (!strcmp(argv[1], "start"))
> -		return maintenance_start();
> +		return maintenance_start(argc - 1, argv + 1, prefix);
>   	if (!strcmp(argv[1], "stop"))
>   		return maintenance_stop();
>   	if (!strcmp(argv[1], "register"))
> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
> index 2412d8c5c0..9eac260307 100755
> --- a/t/t7900-maintenance.sh
> +++ b/t/t7900-maintenance.sh
> @@ -488,8 +488,21 @@ test_expect_success !MINGW 'register and unregister with regex metacharacters' '
>   		maintenance.repo "$(pwd)/$META"
>   '
>   
> +test_expect_success 'start --scheduler=<scheduler>' '
> +	test_expect_code 129 git maintenance start --scheduler=foo 2>err &&
> +	test_i18ngrep "unrecognized --scheduler argument" err &&
 >
> +	test_expect_code 129 git maintenance start --no-scheduler 2>err &&
> +	test_i18ngrep "unknown option" err &&
> +
> +	test_expect_code 128 \
> +		env GIT_TEST_MAINT_SCHEDULER="launchctl:true,schtasks:true" \
> +		git maintenance start --scheduler=crontab 2>err &&
> +	test_i18ngrep "fatal: crontab scheduler is not available" err
> +'
> +
>   test_expect_success 'start from empty cron table' '
> -	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
> +	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
>   
>   	# start registers the repo
>   	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
> @@ -512,7 +525,7 @@ test_expect_success 'stop from existing schedule' '
>   
>   test_expect_success 'start preserves existing schedule' '
>   	echo "Important information!" >cron.txt &&
> -	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
> +	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
>   	grep "Important information!" cron.txt
>   '
>   
> @@ -541,7 +554,7 @@ test_expect_success 'start and stop macOS maintenance' '
>   	EOF
>   
>   	rm -f args &&
> -	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&
> +	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start --scheduler=launchctl &&
>   
>   	# start registers the repo
>   	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
> @@ -592,7 +605,7 @@ test_expect_success 'start and stop Windows maintenance' '
>   	EOF
>   
>   	rm -f args &&
> -	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start &&
> +	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start --scheduler=schtasks &&
>   
>   	# start registers the repo
>   	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
> @@ -615,6 +628,41 @@ test_expect_success 'start and stop Windows maintenance' '
>   	test_cmp expect args
>   '
>   
> +test_expect_success 'start and stop when several schedulers are available' '
> +	write_script print-args <<-\EOF &&
> +	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
> +	EOF
> +
> +	rm -f args &&
> +	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
> +	rm -f expect &&
> +	for frequency in hourly daily weekly
> +	do
> +		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
> +		echo "launchctl bootout gui/[UID] $PLIST" >>expect &&
> +		echo "launchctl bootstrap gui/[UID] $PLIST" >>expect || return 1
> +	done &&
> +	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
> +		hourly daily weekly >>expect &&
> +	test_cmp expect args &&
> +
> +	rm -f args &&
> +	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
> +	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
> +		hourly daily weekly >expect &&
> +	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
> +		hourly daily weekly >>expect &&
> +	test_cmp expect args &&
> +
> +	rm -f args &&
> +	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
> +	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
> +		hourly daily weekly >expect &&
> +	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
> +		hourly daily weekly >>expect &&
> +	test_cmp expect args
> +'
> +

Thanks for adding tests for the new option and converting the old ones 
to use it. I think that with a couple of small changes this patch will 
be ready. I've run out of time for now but I'll try and look at the 
fourth patch in the next couple of days.
Best Wishes

Phillip

>   test_expect_success 'register preserves existing strategy' '
>   	git config maintenance.strategy none &&
>   	git maintenance register &&
> 


^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v4 1/4] cache.h: Introduce a generic "xdg_config_home_for(…)" function
  2021-05-24  9:33         ` Phillip Wood
@ 2021-05-24 12:23           ` Đoàn Trần Công Danh
  0 siblings, 0 replies; 138+ messages in thread
From: Đoàn Trần Công Danh @ 2021-05-24 12:23 UTC (permalink / raw)
  To: Phillip Wood
  Cc: Lénaïc Huard, git, Junio C Hamano, Derrick Stolee,
	Eric Sunshine, Felipe Contreras, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin

On 2021-05-24 10:33:30+0100, Phillip Wood <phillip.wood123@gmail.com> wrote:
> Hi Lénaïc
> 
> This looks fine to me. I'm not 100% sold on calling the parameter prog as
> our program name later in the series ends up being "systemd/user" so
> something like "subdir" might have been better but that is not worth
> rerolling for.

I'll take the blame for that "prog".
I didn't think very hard at the time of writing :(

Yes, "subdir" is definitely better.
And it's aligned with the XDG Base Directory specifications:

	A user-specific version of the configuration file may be
	created in $XDG_CONFIG_HOME/subdir/filename

> On 24/05/2021 08:15, Lénaïc Huard wrote:
> > Current implementation of `xdg_config_home(filename)` returns
> > `$XDG_CONFIG_HOME/git/$filename`, with the `git` subdirectory inserted
> > between the `XDG_CONFIG_HOME` environment variable and the parameter.
> > 
> > This patch introduces a `xdg_config_home_for(prog, filename)` function
> > which is more generic. It only concatenates "$XDG_CONFIG_HOME", or
> > "$HOME/.config" if the former isn’t defined, with the parameters,
> > without adding `git` in between.
> > 
> > `xdg_config_home(filename)` is now implemented by calling
> > `xdg_config_home_for("git", filename)` but this new generic function can
> > be used to compute the configuration directory of other programs.

-- 
Danh

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v4 2/4] maintenance: introduce ENABLE/DISABLE for code clarity
  2021-05-24  9:41         ` Phillip Wood
@ 2021-05-24 12:36           ` Đoàn Trần Công Danh
  2021-05-25  7:18             ` Lénaïc Huard
  0 siblings, 1 reply; 138+ messages in thread
From: Đoàn Trần Công Danh @ 2021-05-24 12:36 UTC (permalink / raw)
  To: Phillip Wood
  Cc: Lénaïc Huard, git, Junio C Hamano, Derrick Stolee,
	Eric Sunshine, Felipe Contreras, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin

On 2021-05-24 10:41:18+0100, Phillip Wood <phillip.wood123@gmail.com> wrote:
> Hi Lénaïc
> 
> On 24/05/2021 08:15, Lénaïc Huard wrote:
> > The first parameter of `XXX_update_schedule` and alike functions is a
> > boolean specifying if the tasks should be scheduled or unscheduled.
> > 
> > Using an `enum` with `ENABLE` and `DISABLE` values can make the code
> > clearer.
> 
> I'm sorry to say that I'm not sure this does make the code clearer overall -
> I wish I'd spoken up when Danh suggested it.
> While
> 	launchctl_boot_plist(DISABLE, filename, cmd)
> is arguably clearer than
> 	launchctl_boot_plist(0, filename, cmd)
> we end up with bizarre tests like
>  	if (enabled == ENABLED)
> rather than
> 	if (enabled)
> and in the next patch we have
> 	(enable == ENABLE && (opts->scheduler == i)) ?
> 			ENABLE : DISABLE;
> rather than
> 	enable && opts->scheduler == i
> 
> Also looking at the next patch it seems as this one is missing some
> conversions in maintenance_start() as it is still calling
> update_background_schedule() with an integer rather than the new enum.

Yes, in this form, I also think the change looks bizarre.
And, it's entirely my fault.

I also agree with Ævar that 0 and 1 is meant well for off/on.

However, I still think

 	launchctl_boot_plist(0, filename, cmd)

would require some degree on code navigation to figure out what would
that LoC does.

I'm thinking about rename the function. But, it would trigger a forever
bikeshedding, which shouldn't be a blocker for this series.

> I'd be happy to see this being dropped I'm afraid

So, let's drop this patch and start a new conversation when the dust
settled.

-- 
Danh

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux
  2021-05-24  7:03                 ` Ævar Arnfjörð Bjarmason
@ 2021-05-24 15:51                   ` Junio C Hamano
  2021-05-25  1:50                     ` Johannes Schindelin
  2021-05-24 17:52                   ` [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux Felipe Contreras
  1 sibling, 1 reply; 138+ messages in thread
From: Junio C Hamano @ 2021-05-24 15:51 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: brian m. carlson, Felipe Contreras, Johannes Schindelin,
	Derrick Stolee, Bagas Sanjaya, Lénaïc Huard, git,
	Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Phillip Wood,
	Martin Ågren

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> Personally I don't care whether someone submits a patch where their
> commit message discusses an example of "he", "she", "they", "it" or
> whatever. It's just meant as an example, and not some statement about
> what the gender (or lack thereof) of such a user *should* be.
>
> It's immediately obvious what the author meant in this case, and that
> the particular wording is arbitrary. For the purposes of discussing the
> contribution it matters whether it's unclear or ambiguous, which it's
> not.

Nicely put.  Thanks.

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v4 4/4] maintenance: add support for systemd timers on Linux
  2021-05-24  9:55         ` Ævar Arnfjörð Bjarmason
@ 2021-05-24 16:39           ` Eric Sunshine
  0 siblings, 0 replies; 138+ messages in thread
From: Eric Sunshine @ 2021-05-24 16:39 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Lénaïc Huard, Git List, Junio C Hamano, Derrick Stolee,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin

On Mon, May 24, 2021 at 6:03 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> On Mon, May 24 2021, Lénaïc Huard wrote:
> > +test_expect_success 'start and stop Linux/systemd maintenance' '
> > +     write_script print-args <<-\EOF &&
> > +     printf "%s\n" "$*" >>args
> > +     EOF
> > +
> > +     XDG_CONFIG_HOME="$PWD" &&
> > +     export XDG_CONFIG_HOME &&
> > +     rm -f args &&
>
> If you're going to care about cleanup here, and personally I wouldn't,
> just call that "args" by the name "expect" instead (as is convention)
> and clobber it every time...
>
> Anyway, a better way to do the cleanup is:
>
>     test_when_finished "rm args" &&
>     echo this is the first time you write the file >args
>     [the rest of the test code]
>
> Then you don't need to re-rm it.

A few comments:

This is following an already-established pattern in this test script,
so it would be unfortunate and out of place to change only these
newly-added tests, thus it's probably better to follow the existing
idiom as is done here. If someone wants to rework all this, it can be
done later script-wide and need not be part of this series nor done by
Lénaïc.

The name `args` is reflective of what is being captured here;
specifically, it captures the arguments passed to the system-specific
scheduler command. It's also the name used by all the other tests, so
it's probably fine as-is.

The git-maintenance command is invoked multiple times in a single test
and `args` needs to be removed before each invocation since its
content is accumulated via `>>` within the `print-args` script, which
is necessary since that script may be run multiple times by a single
git-maintenance command. So, `rm -f args` is not mere cleanup here;
it's an integral part of the test, thus test_when_finished() would be
incorrect.

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux
  2021-05-24  7:03                 ` Ævar Arnfjörð Bjarmason
  2021-05-24 15:51                   ` Junio C Hamano
@ 2021-05-24 17:52                   ` Felipe Contreras
  1 sibling, 0 replies; 138+ messages in thread
From: Felipe Contreras @ 2021-05-24 17:52 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason, brian m. carlson
  Cc: Felipe Contreras, Johannes Schindelin, Derrick Stolee,
	Bagas Sanjaya, Lénaïc Huard, git, Junio C Hamano,
	Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Phillip Wood,
	Martin Ågren

Ævar Arnfjörð Bjarmason wrote:
> I would like to encourage people in this thread who are calling for a
> change in wording here to consider whether this sort of discussion is a
> good use of the ML's time, and the chilling effect of being overly picky
> when many contributors are working in their second, third etc. language.

I completely agree. However, there's a difference between calling for a
change, and expressing one's personal taste. The later can be useful
feedback, the former tramples the progress of a patch series.

> Personally I don't care whether someone submits a patch where their
> commit message discusses an example of "he", "she", "they", "it" or
> whatever. It's just meant as an example, and not some statement about
> what the gender (or lack thereof) of such a user *should* be.

Agreed. My only concern is readability, nonetheless a poorly readable
commit message is not a road-blocker.


What should not be debatable is that this is a software project, it
should not concern itself with linguistic debates. The proper use of
language is a matter for others to decide.

If a member of the community expresses her/his personal taste that Y is a
better word than X, that's fine. But to *demand* a word change is
something else.

Cheers.

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 138+ messages in thread

* RE: [PATCH v4 4/4] maintenance: add support for systemd timers on Linux
  2021-05-24  7:15       ` [PATCH v4 4/4] maintenance: add support for systemd timers on Linux Lénaïc Huard
  2021-05-24  9:55         ` Ævar Arnfjörð Bjarmason
@ 2021-05-24 18:08         ` Felipe Contreras
  2021-05-26 10:26         ` Phillip Wood
  2 siblings, 0 replies; 138+ messages in thread
From: Felipe Contreras @ 2021-05-24 18:08 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Lénaïc Huard

Lénaïc Huard wrote:
>   Concretely, a user that doesn’t have access to the system logs won’t
>   have access to the log of their own tasks scheduled by cron whereas
>   they will have access to the log of their own tasks scheduled by
>   systemd timer.

A user is not a person:

  a user that doesn't have access to the system logs won't have access
  to the log of its own tasks scheduled by cron, whereas it will have
  access to the log of its own tasks scheduled by systemd timer.

Cheers.

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux
  2021-05-24 15:51                   ` Junio C Hamano
@ 2021-05-25  1:50                     ` Johannes Schindelin
  2021-05-25 11:13                       ` Felipe Contreras
  2021-05-26 10:29                       ` CoC, inclusivity etc. (was "Re: [...] systemd timers on Linux") Ævar Arnfjörð Bjarmason
  0 siblings, 2 replies; 138+ messages in thread
From: Johannes Schindelin @ 2021-05-25  1:50 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Ævar Arnfjörð Bjarmason, brian m. carlson,
	Felipe Contreras, Derrick Stolee, Bagas Sanjaya,
	Lénaïc Huard, git, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Phillip Wood,
	Martin Ågren

[-- Attachment #1: Type: text/plain, Size: 2558 bytes --]

Hi,

On Tue, 25 May 2021, Junio C Hamano wrote:

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
> > Personally I don't care whether someone submits a patch where their
> > commit message discusses an example of "he", "she", "they", "it" or
> > whatever. It's just meant as an example, and not some statement about
> > what the gender (or lack thereof) of such a user *should* be.
> >
> > It's immediately obvious what the author meant in this case, and that
> > the particular wording is arbitrary. For the purposes of discussing the
> > contribution it matters whether it's unclear or ambiguous, which it's
> > not.
>
> Nicely put.  Thanks.

_Personally_ I don't care either. Because I am exactly in that group of
young, Caucasian male developers that are so highly overrepresented in our
community already. I will never have a problem thinking that I don't
belong here. I never had that problem.

And I believe that you, Ævar, are very much in the same boat. You will
never feel as if you don't belong in tech. You're Caucasian, male, and
like me, come with an abundance of confidence.

Now, let's go for a little thought experiment. Let's pretend for a moment
that we lacked that confidence. That we were trying to enter a community
with almost no male members at all. Where an email thread was going on
about inviting us to join, using no male pronouns, or putting them last.
And remember: no confidence, no representation. Now, would you dare
chiming in, offering myself as a target? I know I wouldn't. Yet that's
exactly the atmosphere we're fostering here.

What you say, matters. _How_ you say it, matters.

In other words, I think that the _personal_ opinions of everybody who
spoke up in this mail thread (you might have noticed that all of us are
male, you could even call it a "male thread") are not the problem we are
discussing. Personal opinions are kind of missing the point here. By a
mile. And then some.

The actual point is that we want to avoid giving the impression that
only people who feel included by the pronoun "he" are invited. That we
only really care about male users and developers, and pay only scant
tribute to the rest.

And yes, even "he/she", or "(s)he" would give that impression, by
emphasizing a priority order. And "they" simply would not do that. And if
it makes a few male readers slightly uncomfortable, it might present a
fine opportunity to exercise some empathy with those who feel
uncomfortable and excluded all the time.

Thank you,
Johannes

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v4 2/4] maintenance: introduce ENABLE/DISABLE for code clarity
  2021-05-24 12:36           ` Đoàn Trần Công Danh
@ 2021-05-25  7:18             ` Lénaïc Huard
  2021-05-25  8:02               ` Junio C Hamano
  0 siblings, 1 reply; 138+ messages in thread
From: Lénaïc Huard @ 2021-05-25  7:18 UTC (permalink / raw)
  To: Phillip Wood, Đoàn Trần Công Danh
  Cc: git, Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Felipe Contreras, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin

Le lundi 24 mai 2021, 14:36:14 CEST Đoàn Trần Công Danh a écrit :
> On 2021-05-24 10:41:18+0100, Phillip Wood <phillip.wood123@gmail.com> wrote:
> > Hi Lénaïc
> > 
> > On 24/05/2021 08:15, Lénaïc Huard wrote:
> > > The first parameter of `XXX_update_schedule` and alike functions is a
> > > boolean specifying if the tasks should be scheduled or unscheduled.
> > > 
> > > Using an `enum` with `ENABLE` and `DISABLE` values can make the code
> > > clearer.
> > 
> > I'm sorry to say that I'm not sure this does make the code clearer overall
> > - I wish I'd spoken up when Danh suggested it.
> > While
> > 
> > 	launchctl_boot_plist(DISABLE, filename, cmd)
> > 
> > is arguably clearer than
> > 
> > 	launchctl_boot_plist(0, filename, cmd)
> > 
> > we end up with bizarre tests like
> > 
> >  	if (enabled == ENABLED)
> > 
> > rather than
> > 
> > 	if (enabled)
> > 
> > and in the next patch we have
> > 
> > 	(enable == ENABLE && (opts->scheduler == i)) ?
> > 	
> > 			ENABLE : DISABLE;
> > 
> > rather than
> > 
> > 	enable && opts->scheduler == i
> > 
> > Also looking at the next patch it seems as this one is missing some
> > conversions in maintenance_start() as it is still calling
> > update_background_schedule() with an integer rather than the new enum.
> 
> Yes, in this form, I also think the change looks bizarre.
> And, it's entirely my fault.
> 
> I also agree with Ævar that 0 and 1 is meant well for off/on.
> 
> However, I still think
> 
>  	launchctl_boot_plist(0, filename, cmd)
> 
> would require some degree on code navigation to figure out what would
> that LoC does.
> 
> I'm thinking about rename the function. But, it would trigger a forever
> bikeshedding, which shouldn't be a blocker for this series.
> 
> > I'd be happy to see this being dropped I'm afraid
> 
> So, let's drop this patch and start a new conversation when the dust
> settled.

Hi,

I think the reason why the code looks worse is because I used an enum and I 
didn’t want to make any assumption about how the enum members would be 
evaluated in a boolean context.

Do you think it would make sense to drop the enum type, to revert all logic 
changes (Use `if (enabled)` back instead of `switch`, etc.), and to define the 
following constants :

static const int DISABLE = 0;
static const int ENABLE = 1;

so that we can keep function invocation in the form of 
`launchctl_boot_plist(DISABLE, filename, cmd)` ?



^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v4 2/4] maintenance: introduce ENABLE/DISABLE for code clarity
  2021-05-25  7:18             ` Lénaïc Huard
@ 2021-05-25  8:02               ` Junio C Hamano
  0 siblings, 0 replies; 138+ messages in thread
From: Junio C Hamano @ 2021-05-25  8:02 UTC (permalink / raw)
  To: Lénaïc Huard
  Cc: Phillip Wood, Đoàn Trần Công Danh, git,
	Derrick Stolee, Eric Sunshine, Felipe Contreras,
	Martin Ågren, Ævar Arnfjörð Bjarmason,
	Bagas Sanjaya, brian m . carlson, Johannes Schindelin

Lénaïc Huard <lenaic@lhuard.fr> writes:

> I think the reason why the code looks worse is because I used an enum and I 
> didn’t want to make any assumption about how the enum members would be 
> evaluated in a boolean context.
>
> Do you think it would make sense to drop the enum type, to revert all logic 
> changes (Use `if (enabled)` back instead of `switch`, etc.), and to define the 
> following constants :
>
> static const int DISABLE = 0;
> static const int ENABLE = 1;
>
> so that we can keep function invocation in the form of 
> `launchctl_boot_plist(DISABLE, filename, cmd)` ?

I think the code is much better off without DISABLE/ENABLE at all.

As has already been pointed out, you cannot read and write _without_
being aware of the fact that DISABLE is 0 if you want to write
readable code, i.e. instead of "if (able == ENABLE) do this;", you
would want to say "if (able) do this;".


^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux
  2021-05-25  1:50                     ` Johannes Schindelin
@ 2021-05-25 11:13                       ` Felipe Contreras
  2021-05-26 10:29                       ` CoC, inclusivity etc. (was "Re: [...] systemd timers on Linux") Ævar Arnfjörð Bjarmason
  1 sibling, 0 replies; 138+ messages in thread
From: Felipe Contreras @ 2021-05-25 11:13 UTC (permalink / raw)
  To: Johannes Schindelin, Junio C Hamano
  Cc: Ævar Arnfjörð Bjarmason, brian m. carlson,
	Felipe Contreras, Derrick Stolee, Bagas Sanjaya,
	Lénaïc Huard, git, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Phillip Wood,
	Martin Ågren

Johannes Schindelin wrote:

> Where an email thread was going on about inviting us to join, using no
> male pronouns, or putting them last.

This is totally and completely missing the point. Nobody objected to
using a female pronoun, nor using she/he.

I for one would have no problem using a female pronoun, and in fact I
often do [1].

This has absolutely nothing to do the issue at hand.

> I know I wouldn't. Yet that's exactly the atmosphere we're fostering
> here.

Except this is disconnected from reality.

 1. I know female colleagues who don't want to be treated more
    delicately than their male counterparts. And they certainly don't
    want unsolicited help, especially regarding hypothetical feelings
    they might have towards the use of a pronoun.

 2. On the other hand I do know people that lurk the mailing list and
    are afraid of expressing their opinions because of fear of reprisals
    in this PC-culture climate.

If anybody needs consideration I would side with the real, rather than
hypothetical, people.

[1] https://lore.kernel.org/git/60a66b11d6ffd_2448320885@natae.notmuch/

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v4 4/4] maintenance: add support for systemd timers on Linux
  2021-05-24  7:15       ` [PATCH v4 4/4] maintenance: add support for systemd timers on Linux Lénaïc Huard
  2021-05-24  9:55         ` Ævar Arnfjörð Bjarmason
  2021-05-24 18:08         ` Felipe Contreras
@ 2021-05-26 10:26         ` Phillip Wood
  2 siblings, 0 replies; 138+ messages in thread
From: Phillip Wood @ 2021-05-26 10:26 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Martin Ågren, Ævar Arnfjörð Bjarmason,
	Bagas Sanjaya, brian m . carlson, Johannes Schindelin

Lénaïc

On 24/05/2021 08:15, Lénaïc Huard wrote:
> The existing mechanism for scheduling background maintenance is done
> through cron. On Linux systems managed by systemd, systemd provides an
> alternative to schedule recurring tasks: systemd timers.
> 
> The main motivations to implement systemd timers in addition to cron
> are:
> * cron is optional and Linux systems running systemd might not have it
>    installed.
> * The execution of `crontab -l` can tell us if cron is installed but not
>    if the daemon is actually running.
> * With systemd, each service is run in its own cgroup and its logs are
>    tagged by the service inside journald. With cron, all scheduled tasks
>    are running in the cron daemon cgroup and all the logs of the
>    user-scheduled tasks are pretended to belong to the system cron
>    service.
>    Concretely, a user that doesn’t have access to the system logs won’t
>    have access to the log of their own tasks scheduled by cron whereas
>    they will have access to the log of their own tasks scheduled by
>    systemd timer.
>    Although `cron` attempts to send email, that email may go unseen by
>    the user because these days, local mailboxes are not heavily used
>    anymore.
> 
> In order to schedule git maintenance, we need two unit template files:
> * ~/.config/systemd/user/git-maintenance@.service
>    to define the command to be started by systemd and
> * ~/.config/systemd/user/git-maintenance@.timer
>    to define the schedule at which the command should be run.
> 
> Those units are templates that are parameterized by the frequency.
> 
> Based on those templates, 3 timers are started:
> * git-maintenance@hourly.timer
> * git-maintenance@daily.timer
> * git-maintenance@weekly.timer
> 
> The command launched by those three timers are the same as with the
> other scheduling methods:
> 
> /path/to/git for-each-repo --exec-path=/path/to
> --config=maintenance.repo maintenance run --schedule=%i
> 
> with the full path for git to ensure that the version of git launched
> for the scheduled maintenance is the same as the one used to run
> `maintenance start`.
> 
> The timer unit contains `Persistent=true` so that, if the computer is
> powered down when a maintenance task should run, the task will be run
> when the computer is back powered on.

The commit message is comprehensive and does a good job of explaining 
the patch I do not think there is any need to reword it - I find using 
'they' rather than 'it' to refer to a user is more readable.

> Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
> ---
>   Documentation/git-maintenance.txt |  57 +++++++++-
>   builtin/gc.c                      | 180 ++++++++++++++++++++++++++++++
>   t/t7900-maintenance.sh            |  66 ++++++++++-
>   3 files changed, 293 insertions(+), 10 deletions(-)
> 
> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
> index 7c4bb38a2f..50179e010f 100644
> --- a/Documentation/git-maintenance.txt
> +++ b/Documentation/git-maintenance.txt
> @@ -181,16 +181,19 @@ OPTIONS
>   	`maintenance.<task>.enabled` configured as `true` are considered.
>   	See the 'TASKS' section for the list of accepted `<task>` values.
>   
> ---scheduler=auto|crontab|launchctl|schtasks::
> +--scheduler=auto|crontab|systemd-timer|launchctl|schtasks::
>   	When combined with the `start` subcommand, specify the scheduler
>   	to use to run the hourly, daily and weekly executions of
>   	`git maintenance run`.
>   	The possible values for `<scheduler>` depend on the system: `crontab`
> -	is available on POSIX systems, `launchctl` is available on
> -	MacOS and `schtasks` is available on Windows.
> +	is available on POSIX systems, `systemd-timer` is available on Linux
> +	systems, `launchctl` is available on MacOS and `schtasks` is available
> +	on Windows.
>   	By default or when `auto` is specified, a suitable scheduler for
>   	the system is used. On MacOS, `launchctl` is used. On Windows,
> -	`schtasks` is used. On all other systems, `crontab` is used.
> +	`schtasks` is used. On Linux, `systemd-timer` is used if user systemd
> +	timers are available, otherwise, `crontab` is used. On all other systems,
> +	`crontab` is used.
>   
>   
>   TROUBLESHOOTING
> @@ -290,6 +293,52 @@ schedule to ensure you are executing the correct binaries in your
>   schedule.
>   
>   
> +BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMS
> +-----------------------------------------------
> +
> +While Linux supports `cron`, depending on the distribution, `cron` may
> +be an optional package not necessarily installed. On modern Linux
> +distributions, systemd timers are superseding it.
> +
> +If user systemd timers are available, they will be used as a replacement
> +of `cron`.
> +
> +In this case, `git maintenance start` will create user systemd timer units
> +and start the timers. The current list of user-scheduled tasks can be found
> +by running `systemctl --user list-timers`. The timers written by `git
> +maintenance start` are similar to this:
> +
> +-----------------------------------------------------------------------
> +$ systemctl --user list-timers
> +NEXT                         LEFT          LAST                         PASSED     UNIT                         ACTIVATES
> +Thu 2021-04-29 19:00:00 CEST 42min left    Thu 2021-04-29 18:00:11 CEST 17min ago  git-maintenance@hourly.timer git-maintenance@hourly.service
> +Fri 2021-04-30 00:00:00 CEST 5h 42min left Thu 2021-04-29 00:00:11 CEST 18h ago    git-maintenance@daily.timer  git-maintenance@daily.service
> +Mon 2021-05-03 00:00:00 CEST 3 days left   Mon 2021-04-26 00:00:11 CEST 3 days ago git-maintenance@weekly.timer git-maintenance@weekly.service
> +-----------------------------------------------------------------------
> +
> +One timer is registered for each `--schedule=<frequency>` option.
> +
> +The definition of the systemd units can be inspected in the following files:
> +
> +-----------------------------------------------------------------------
> +~/.config/systemd/user/git-maintenance@.timer
> +~/.config/systemd/user/git-maintenance@.service
> +~/.config/systemd/user/timers.target.wants/git-maintenance@hourly.timer
> +~/.config/systemd/user/timers.target.wants/git-maintenance@daily.timer
> +~/.config/systemd/user/timers.target.wants/git-maintenance@weekly.timer
> +-----------------------------------------------------------------------
> +
> +`git maintenance start` will overwrite these files and start the timer
> +again with `systemctl --user`, so any customization should be done by
> +creating a drop-in file, i.e. a `.conf` suffixed file in the
> +`~/.config/systemd/user/git-maintenance@.service.d` directory.
> +
> +`git maintenance stop` will stop the user systemd timers and delete
> +the above mentioned files.
> +
> +For more details, see `systemd.timer(5)`.
> +
> +
>   BACKGROUND MAINTENANCE ON MACOS SYSTEMS
>   ---------------------------------------
>   
> diff --git a/builtin/gc.c b/builtin/gc.c
> index bf21cec059..3eca1e5e6a 100644
> --- a/builtin/gc.c
> +++ b/builtin/gc.c
> @@ -2079,10 +2079,173 @@ static int crontab_update_schedule(enum enable_or_disable run_maintenance,
>   	return result;
>   }
>   
> +static int is_systemd_timer_available(void)
> +{
> +	const char *cmd = "systemctl";
> +	int is_available;
> +	static int cached_result = -1;
> +#ifdef __linux__
> +	struct child_process child = CHILD_PROCESS_INIT;
> +#endif
> +
> +	if (cached_result != -1)
> +		return cached_result;
> +
> +	if (get_schedule_cmd(&cmd, &is_available))
> +		return is_available;
> +
> +#ifdef __linux__
> +	strvec_split(&child.args, cmd);
> +	strvec_pushl(&child.args, "--user", "list-timers", NULL);
> +	child.no_stdin = 1;
> +	child.no_stdout = 1;
> +	child.no_stderr = 1;
> +	child.silent_exec_failure = 1;
> +
> +	if (start_command(&child)) {
> +		cached_result = 0;
> +		return cached_result;
> +	}
> +	if (finish_command(&child)) {
> +		cached_result = 0;
> +		return cached_result;
> +	}
> +	cached_result = 1;
> +	return cached_result;
> +#else
> +	return 0;
> +#endif
> +}

I agree with Ævar that this would be more readable with two separate 
definitions rather chosen by a single #ifdef. I'm dubious that we need 
the caching as well.

> +static char *xdg_config_home_systemd(const char *filename)
> +{
> +	return xdg_config_home_for("systemd/user", filename);
> +}
> +
> +static int systemd_timer_enable_unit(int enable,
> +				     enum schedule_priority schedule)
> +{
> +	const char *cmd = "systemctl";
> +	struct child_process child = CHILD_PROCESS_INIT;
> +	const char *frequency = get_frequency(schedule);
> +
> +	get_schedule_cmd(&cmd, NULL);
> +	strvec_split(&child.args, cmd);
> +	strvec_pushl(&child.args, "--user", enable ? "enable" : "disable",
> +		     "--now", NULL);
> +	strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency);
> +
> +	if (start_command(&child))
> +		die(_("failed to run systemctl"));
> +	return finish_command(&child);

I left some comments about the combination of dying and returning an 
error when reading v2[1]. If you disagree with a reviewers comments 
that's fine but please try to comment on the reviewers suggestions so 
that they know their suggestions weren't just forgotten by accident. 
Looking through the patch this applies to a number of comments I left 
about the error handling (or lack of it) and cleanup. I don't think I 
have anything new to add.

Best Wishes

Phillip

[1] 
https://lore.kernel.org/git/3fd17223-8667-24be-2e65-f1970d411bdf@gmail.com/

> +}
> +
> +static int systemd_timer_delete_unit_templates(void)
> +{
> +	char *filename = xdg_config_home_systemd("git-maintenance@.timer");
> +	unlink(filename);
> +	free(filename);
> +
> +	filename = xdg_config_home_systemd("git-maintenance@.service");
> +	unlink(filename);
> +	free(filename);
> +
> +	return 0;
> +}
> +
> +static int systemd_timer_delete_units(void)
> +{
> +	return systemd_timer_enable_unit(DISABLE, SCHEDULE_HOURLY) ||
> +	       systemd_timer_enable_unit(DISABLE, SCHEDULE_DAILY) ||
> +	       systemd_timer_enable_unit(DISABLE, SCHEDULE_WEEKLY) ||
> +	       systemd_timer_delete_unit_templates();
> +}
> +
> +static int systemd_timer_write_unit_templates(const char *exec_path)
> +{
> +	char *filename;
> +	FILE *file;
> +	const char *unit;
> +
> +	filename = xdg_config_home_systemd("git-maintenance@.timer");
> +	if (safe_create_leading_directories(filename))
> +		die(_("failed to create directories for '%s'"), filename);
> +	file = xfopen(filename, "w");
> +	FREE_AND_NULL(filename);
> +
> +	unit = "# This file was created and is maintained by Git.\n"
> +	       "# Any edits made in this file might be replaced in the future\n"
> +	       "# by a Git command.\n"
> +	       "\n"
> +	       "[Unit]\n"
> +	       "Description=Optimize Git repositories data\n"
> +	       "\n"
> +	       "[Timer]\n"
> +	       "OnCalendar=%i\n"
> +	       "Persistent=true\n"
> +	       "\n"
> +	       "[Install]\n"
> +	       "WantedBy=timers.target\n";
> +	fputs(unit, file);
> +	fclose(file);
> +
> +	filename = xdg_config_home_systemd("git-maintenance@.service");
> +	file = xfopen(filename, "w");
> +	free(filename);
> +
> +	unit = "# This file was created and is maintained by Git.\n"
> +	       "# Any edits made in this file might be replaced in the future\n"
> +	       "# by a Git command.\n"
> +	       "\n"
> +	       "[Unit]\n"
> +	       "Description=Optimize Git repositories data\n"
> +	       "\n"
> +	       "[Service]\n"
> +	       "Type=oneshot\n"
> +	       "ExecStart=\"%s/git\" --exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
> +	       "LockPersonality=yes\n"
> +	       "MemoryDenyWriteExecute=yes\n"
> +	       "NoNewPrivileges=yes\n"
> +	       "RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\n"
> +	       "RestrictNamespaces=yes\n"
> +	       "RestrictRealtime=yes\n"
> +	       "RestrictSUIDSGID=yes\n"
> +	       "SystemCallArchitectures=native\n"
> +	       "SystemCallFilter=@system-service\n";
> +	fprintf(file, unit, exec_path, exec_path);
> +	fclose(file);
> +
> +	return 0;
> +}
> +
> +static int systemd_timer_setup_units(void)
> +{
> +	const char *exec_path = git_exec_path();
> +
> +	return systemd_timer_write_unit_templates(exec_path) ||
> +	       systemd_timer_enable_unit(ENABLE, SCHEDULE_HOURLY) ||
> +	       systemd_timer_enable_unit(ENABLE, SCHEDULE_DAILY) ||
> +	       systemd_timer_enable_unit(ENABLE, SCHEDULE_WEEKLY);
> +}
> +
> +static int systemd_timer_update_schedule(enum enable_or_disable run_maintenance,
> +					 int fd)
> +{
> +	switch (run_maintenance) {
> +	case ENABLE:
> +		return systemd_timer_setup_units();
> +	case DISABLE:
> +		return systemd_timer_delete_units();
> +	default:
> +		BUG("invalid enable_or_disable value");
> +	}
> +}
> +
>   enum scheduler {
>   	SCHEDULER_INVALID = -1,
>   	SCHEDULER_AUTO,
>   	SCHEDULER_CRON,
> +	SCHEDULER_SYSTEMD,
>   	SCHEDULER_LAUNCHCTL,
>   	SCHEDULER_SCHTASKS,
>   };
> @@ -2097,6 +2260,11 @@ static const struct {
>   		.is_available = is_crontab_available,
>   		.update_schedule = crontab_update_schedule,
>   	},
> +	[SCHEDULER_SYSTEMD] = {
> +		.name = "systemctl",
> +		.is_available = is_systemd_timer_available,
> +		.update_schedule = systemd_timer_update_schedule,
> +	},
>   	[SCHEDULER_LAUNCHCTL] = {
>   		.name = "launchctl",
>   		.is_available = is_launchctl_available,
> @@ -2117,6 +2285,9 @@ static enum scheduler parse_scheduler(const char *value)
>   		return SCHEDULER_AUTO;
>   	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
>   		return SCHEDULER_CRON;
> +	else if (!strcasecmp(value, "systemd") ||
> +		 !strcasecmp(value, "systemd-timer"))
> +		return SCHEDULER_SYSTEMD;
>   	else if (!strcasecmp(value, "launchctl"))
>   		return SCHEDULER_LAUNCHCTL;
>   	else if (!strcasecmp(value, "schtasks"))
> @@ -2155,6 +2326,15 @@ static void resolve_auto_scheduler(enum scheduler *scheduler)
>   	*scheduler = SCHEDULER_SCHTASKS;
>   	return;
>   
> +#elif defined(__linux__)
> +	if (is_systemd_timer_available())
> +		*scheduler = SCHEDULER_SYSTEMD;
> +	else if (is_crontab_available())
> +		*scheduler = SCHEDULER_CRON;
> +	else
> +		die(_("neither systemd timers nor crontab are available"));
> +	return;
> +
>   #else
>   	*scheduler = SCHEDULER_CRON;
>   	return;
> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
> index 9eac260307..c8a6f19ebc 100755
> --- a/t/t7900-maintenance.sh
> +++ b/t/t7900-maintenance.sh
> @@ -20,6 +20,18 @@ test_xmllint () {
>   	fi
>   }
>   
> +test_lazy_prereq SYSTEMD_ANALYZE '
> +	systemd-analyze --help >out &&
> +	grep verify out
> +'
> +
> +test_systemd_analyze_verify () {
> +	if test_have_prereq SYSTEMD_ANALYZE
> +	then
> +		systemd-analyze verify "$@"
> +	fi
> +}
> +
>   test_expect_success 'help text' '
>   	test_expect_code 129 git maintenance -h 2>err &&
>   	test_i18ngrep "usage: git maintenance <subcommand>" err &&
> @@ -628,14 +640,54 @@ test_expect_success 'start and stop Windows maintenance' '
>   	test_cmp expect args
>   '
>   
> +test_expect_success 'start and stop Linux/systemd maintenance' '
> +	write_script print-args <<-\EOF &&
> +	printf "%s\n" "$*" >>args
> +	EOF
> +
> +	XDG_CONFIG_HOME="$PWD" &&
> +	export XDG_CONFIG_HOME &&
> +	rm -f args &&
> +	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance start --scheduler=systemd-timer &&
> +
> +	# start registers the repo
> +	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
> +
> +	test_systemd_analyze_verify "systemd/user/git-maintenance@.service" &&
> +
> +	printf -- "--user enable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
> +	test_cmp expect args &&
> +
> +	rm -f args &&
> +	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance stop &&
> +
> +	# stop does not unregister the repo
> +	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
> +
> +	test_path_is_missing "systemd/user/git-maintenance@.timer" &&
> +	test_path_is_missing "systemd/user/git-maintenance@.service" &&
> +
> +	printf -- "--user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
> +	test_cmp expect args
> +'
> +
>   test_expect_success 'start and stop when several schedulers are available' '
>   	write_script print-args <<-\EOF &&
>   	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
>   	EOF
>   
>   	rm -f args &&
> -	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
> -	rm -f expect &&
> +	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=systemd-timer &&
> +	printf -- "systemctl --user enable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
> +	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
> +		hourly daily weekly >>expect &&
> +	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
> +		hourly daily weekly >>expect &&
> +	test_cmp expect args &&
> +
> +	rm -f args &&
> +	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
> +	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
>   	for frequency in hourly daily weekly
>   	do
>   		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
> @@ -647,17 +699,19 @@ test_expect_success 'start and stop when several schedulers are available' '
>   	test_cmp expect args &&
>   
>   	rm -f args &&
> -	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
> +	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
> +	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
>   	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
> -		hourly daily weekly >expect &&
> +		hourly daily weekly >>expect &&
>   	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
>   		hourly daily weekly >>expect &&
>   	test_cmp expect args &&
>   
>   	rm -f args &&
> -	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
> +	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
> +	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
>   	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
> -		hourly daily weekly >expect &&
> +		hourly daily weekly >>expect &&
>   	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
>   		hourly daily weekly >>expect &&
>   	test_cmp expect args
> 


^ permalink raw reply	[flat|nested] 138+ messages in thread

* CoC, inclusivity etc. (was "Re: [...] systemd timers on Linux")
  2021-05-25  1:50                     ` Johannes Schindelin
  2021-05-25 11:13                       ` Felipe Contreras
@ 2021-05-26 10:29                       ` Ævar Arnfjörð Bjarmason
  2021-05-26 16:05                         ` Felipe Contreras
  2021-05-27 14:24                         ` Jeff King
  1 sibling, 2 replies; 138+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-05-26 10:29 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Junio C Hamano, brian m. carlson, Felipe Contreras,
	Derrick Stolee, Bagas Sanjaya, Lénaïc Huard, git,
	Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Phillip Wood,
	Martin Ågren


On Tue, May 25 2021, Johannes Schindelin wrote:

> Hi,
>
> On Tue, 25 May 2021, Junio C Hamano wrote:
>
>> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>>
>> > Personally I don't care whether someone submits a patch where their
>> > commit message discusses an example of "he", "she", "they", "it" or
>> > whatever. It's just meant as an example, and not some statement about
>> > what the gender (or lack thereof) of such a user *should* be.
>> >
>> > It's immediately obvious what the author meant in this case, and that
>> > the particular wording is arbitrary. For the purposes of discussing the
>> > contribution it matters whether it's unclear or ambiguous, which it's
>> > not.
>>
>> Nicely put.  Thanks.
>
> _Personally_ I don't care either. Because I am exactly in that group of
> young, Caucasian male developers that are so highly overrepresented in our
> community already. I will never have a problem thinking that I don't
> belong here. I never had that problem.
>
> And I believe that you, Ævar, are very much in the same boat. You will
> never feel as if you don't belong in tech. You're Caucasian, male, and
> like me, come with an abundance of confidence.
>
> Now, let's go for a little thought experiment. Let's pretend for a moment
> that we lacked that confidence. That we were trying to enter a community
> with almost no male members at all. Where an email thread was going on
> about inviting us to join, using no male pronouns, or putting them last.
> And remember: no confidence, no representation. Now, would you dare
> chiming in, offering myself as a target? I know I wouldn't. Yet that's
> exactly the atmosphere we're fostering here.
>
> What you say, matters. _How_ you say it, matters.
>
> In other words, I think that the _personal_ opinions of everybody who
> spoke up in this mail thread (you might have noticed that all of us are
> male, you could even call it a "male thread") are not the problem we are
> discussing. Personal opinions are kind of missing the point here. By a
> mile. And then some.
>
> The actual point is that we want to avoid giving the impression that
> only people who feel included by the pronoun "he" are invited. That we
> only really care about male users and developers, and pay only scant
> tribute to the rest.
>
> And yes, even "he/she", or "(s)he" would give that impression, by
> emphasizing a priority order. And "they" simply would not do that. And if
> it makes a few male readers slightly uncomfortable, it might present a
> fine opportunity to exercise some empathy with those who feel
> uncomfortable and excluded all the time.
>
> Thank you,

I don't think it's helpful or in the spirit of our CoC to characterize
discussion in terms of the genders or nationalities of its participants.

I'm not accusing you of some CoC violation, just suggesting that it
makes more better conversation to take participants at their word and
assume good faith.

You can't say based on some superficial piece of information what a
project participant might be dealing with, e.g. perhaps they're anxious
about screwing up some trivial thing on a public ML, feel that they're
inexperienced and/or have some "impostor syndrome" etc.

The reason I chimed in on this thread was that I thought concern over
one such topic had started to negatively impact another. We've got a lot
of people trying to contribute who aren't comfortable contributing in
English, or whose proficiency doesn't extend to the latest linguistic
trends.

I'm suggesting that it's more helpful to leave certain things be than to
pounce on contributors about things that are ultimately not integral to
their work, and which can be readily understood.

E.g. if someone's clearly a speaker of a Slavic language and it shows in
their grammar I don't think we should be aiming to have them massage
their patch until it's indistinguishable from that of a native speaker
of English.

We should make sure we understand what's being said[1], but I think
anything past that point is starting to bring us negative value. At some
point it's good enough that we can understand the intent, and by
assuming good faith we're better off letting things like "he" v.s. "she"
v.s. "they" slide.

To not do so is to needlessly create a barrier to participation in this
project. I think the side discussion about Merriam Webster[2] somewhat
misses the mark here.

The question isn't whether what's being suggested would be a valid
sentence according to a dictionary, but whether e.g. someone who's
learned English from a commonly available textbook (I daresay most
"Beginner English" textbooks have more "he" and "she" examples than
"they") is going to inadvertently run afoul of other people's
preferences.

If they do it's more work for them (partially because they're a
non-native speaker), and ultimately more things we need to cover in
Documentation/SubmittingPatches etc. All of that's more barriers of
entry to project participation.

1. https://lore.kernel.org/git/87wns6u8pc.fsf@evledraar.gmail.com/
2. https://lore.kernel.org/git/YKrk4dEjEm6+48ji@camp.crustytoothpaste.net/

^ permalink raw reply	[flat|nested] 138+ messages in thread

* RE: CoC, inclusivity etc. (was "Re: [...] systemd timers on Linux")
  2021-05-26 10:29                       ` CoC, inclusivity etc. (was "Re: [...] systemd timers on Linux") Ævar Arnfjörð Bjarmason
@ 2021-05-26 16:05                         ` Felipe Contreras
  2021-05-27 14:24                         ` Jeff King
  1 sibling, 0 replies; 138+ messages in thread
From: Felipe Contreras @ 2021-05-26 16:05 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason, Johannes Schindelin
  Cc: Junio C Hamano, brian m. carlson, Felipe Contreras,
	Derrick Stolee, Bagas Sanjaya, Lénaïc Huard, git,
	Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Phillip Wood,
	Martin Ågren

Ævar Arnfjörð Bjarmason wrote:
> On Tue, May 25 2021, Johannes Schindelin wrote:

> > _Personally_ I don't care either. Because I am exactly in that group of
> > young, Caucasian male developers that are so highly overrepresented in our
> > community already. I will never have a problem thinking that I don't
> > belong here. I never had that problem.
> >
> > And I believe that you, Ævar, are very much in the same boat. You will
> > never feel as if you don't belong in tech. You're Caucasian, male, and
> > like me, come with an abundance of confidence.
> 
> I don't think it's helpful or in the spirit of our CoC to characterize
> discussion in terms of the genders or nationalities of its participants.

Not to mention that it's worryingly close to implying "if you are a straight
white cis male, your opinion doesn't matter".

I don't think the Git project should be assigning weight to the opinions
of community members based on gender, or race.

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: CoC, inclusivity etc. (was "Re: [...] systemd timers on Linux")
  2021-05-26 10:29                       ` CoC, inclusivity etc. (was "Re: [...] systemd timers on Linux") Ævar Arnfjörð Bjarmason
  2021-05-26 16:05                         ` Felipe Contreras
@ 2021-05-27 14:24                         ` Jeff King
  2021-05-27 17:30                           ` Felipe Contreras
                                             ` (2 more replies)
  1 sibling, 3 replies; 138+ messages in thread
From: Jeff King @ 2021-05-27 14:24 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Johannes Schindelin, Junio C Hamano, brian m. carlson,
	Felipe Contreras, Derrick Stolee, Bagas Sanjaya,
	Lénaïc Huard, git, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Phillip Wood,
	Martin Ågren

On Wed, May 26, 2021 at 12:29:01PM +0200, Ævar Arnfjörð Bjarmason wrote:

> The reason I chimed in on this thread was that I thought concern over
> one such topic had started to negatively impact another. We've got a lot
> of people trying to contribute who aren't comfortable contributing in
> English, or whose proficiency doesn't extend to the latest linguistic
> trends.
> 
> I'm suggesting that it's more helpful to leave certain things be than to
> pounce on contributors about things that are ultimately not integral to
> their work, and which can be readily understood.

Yes, I want to express my support of this point, and not just for this
particular issue.

If you're a new contributor (or even an old one), it can be frustrating
to spend a lot of time working on and polishing up an improvement to the
software, to be met with feedback that mostly consists of "there's a
typo in your commit message". Whether that's true or not, or whether it
improves the commit message or not, it can feel like reviewers are
missing the point of the patch, which will discourage contributors.

As reviewers, I think we can do a few things to encourage people,
especially new contributors:

  - let little errors slide if they're not important. I think sometimes
    we get into a mentality that the commit message is baked into
    history, and thus needs to be revised and perfected. But commit
    messages are also just a conversation that's happening and being
    recorded. There will be hiccups, and polishing them forever has
    diminishing returns.

    (Of course this requires judgement; some commit messages really are
    just hard to follow, and I think you made that distinction with the
    phrase "make sure understand what's being said").

  - temper small corrections with positive feedback. Especially for new
    contributors, being told explicitly "yes, what you're trying to do
    here overall is welcome, and it all looks good except for this..."
    is much more encouraging than "this part is wrong". In the latter,
    they're left to guess if anybody even values the rest of the work at
    all.

  - likewise, I think it helps to give feedback on expectations for the
    process. Saying explicitly "this looks good; I think with this style
    change, it would be ready to get picked up" helps them understand
    that the fix will get them across the finish line (as opposed to
    just getting another round of fix requests).

I would even extend some of those into the code itself. Obviously we
don't want to lower the bar and take incorrect code, or even typos in
error messages. But I think we could stand to relax sometimes on issues
of style or "I would do it like this" (and at the very least, the
"temper small corrections" advice may apply).

I'm not really targeting anybody in particular in this thread (and
Lénaïc seems to have taken it all in stride in this case). It's more
just a reminder that it's easy to forget to do these kinds of things,
and keep this kind of perspective. I know I have not always done it
perfectly at times.

-Peff

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: CoC, inclusivity etc. (was "Re: [...] systemd timers on Linux")
  2021-05-27 14:24                         ` Jeff King
@ 2021-05-27 17:30                           ` Felipe Contreras
  2021-05-27 23:58                           ` Junio C Hamano
  2021-05-28 14:44                           ` Phillip Susi
  2 siblings, 0 replies; 138+ messages in thread
From: Felipe Contreras @ 2021-05-27 17:30 UTC (permalink / raw)
  To: Jeff King, Ævar Arnfjörð Bjarmason
  Cc: Johannes Schindelin, Junio C Hamano, brian m. carlson,
	Felipe Contreras, Derrick Stolee, Bagas Sanjaya,
	Lénaïc Huard, git, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Phillip Wood,
	Martin Ågren

Jeff King wrote:
> On Wed, May 26, 2021 at 12:29:01PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> > The reason I chimed in on this thread was that I thought concern over
> > one such topic had started to negatively impact another. We've got a lot
> > of people trying to contribute who aren't comfortable contributing in
> > English, or whose proficiency doesn't extend to the latest linguistic
> > trends.
> > 
> > I'm suggesting that it's more helpful to leave certain things be than to
> > pounce on contributors about things that are ultimately not integral to
> > their work, and which can be readily understood.
> 
> Yes, I want to express my support of this point, and not just for this
> particular issue.
> 
> If you're a new contributor (or even an old one), it can be frustrating
> to spend a lot of time working on and polishing up an improvement to the
> software, to be met with feedback that mostly consists of "there's a
> typo in your commit message". Whether that's true or not, or whether it
> improves the commit message or not, it can feel like reviewers are
> missing the point of the patch, which will discourage contributors.
> 
> As reviewers, I think we can do a few things to encourage people,
> especially new contributors:
> 
>   - let little errors slide if they're not important. I think sometimes
>     we get into a mentality that the commit message is baked into
>     history, and thus needs to be revised and perfected. But commit
>     messages are also just a conversation that's happening and being
>     recorded. There will be hiccups, and polishing them forever has
>     diminishing returns.

Yes, polishing forever has diminishing returns, but that's not really
the hard problem here; the problem is when two people (or more) can't
agree on what "polishing" means. That's a more eternal forever.

Fortunately we do have a dictator, and he can make determinations to end
the eternal debate.

He can simply fix the commit message--or any other minor
details--himself.

Sure, that doesn't scale, and ideally we would want the patch
contributors to do these tasks themselves, and not burden the
maintainer. But if the task of herding the cats is taken forever, why
keep insisting? Just fix the commit message and move on.

> I would even extend some of those into the code itself. Obviously we
> don't want to lower the bar and take incorrect code, or even typos in
> error messages. But I think we could stand to relax sometimes on issues
> of style or "I would do it like this" (and at the very least, the
> "temper small corrections" advice may apply).

Plus code can be fixed later. If as a reviewer you really dislike one
aspect of a patch, you don't necessarily need to convince the author to
change it. You can propose a separate patch later on.


Sometimes good is good enough. Don't let perfect be the enemy of good.

Cheers.

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: CoC, inclusivity etc. (was "Re: [...] systemd timers on Linux")
  2021-05-27 14:24                         ` Jeff King
  2021-05-27 17:30                           ` Felipe Contreras
@ 2021-05-27 23:58                           ` Junio C Hamano
  2021-05-28 14:44                           ` Phillip Susi
  2 siblings, 0 replies; 138+ messages in thread
From: Junio C Hamano @ 2021-05-27 23:58 UTC (permalink / raw)
  To: Jeff King
  Cc: Ævar Arnfjörð Bjarmason, Johannes Schindelin,
	brian m. carlson, Felipe Contreras, Derrick Stolee,
	Bagas Sanjaya, Lénaïc Huard, git, Derrick Stolee,
	Eric Sunshine, Đoàn Trần Công Danh,
	Phillip Wood, Martin Ågren

Jeff King <peff@peff.net> writes:

> I'm not really targeting anybody in particular in this thread (and
> Lénaïc seems to have taken it all in stride in this case). It's more
> just a reminder that it's easy to forget to do these kinds of things,
> and keep this kind of perspective. I know I have not always done it
> perfectly at times.

Thanks.

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: CoC, inclusivity etc. (was "Re: [...] systemd timers on Linux")
  2021-05-27 14:24                         ` Jeff King
  2021-05-27 17:30                           ` Felipe Contreras
  2021-05-27 23:58                           ` Junio C Hamano
@ 2021-05-28 14:44                           ` Phillip Susi
  2021-05-30 21:58                             ` Jeff King
  2 siblings, 1 reply; 138+ messages in thread
From: Phillip Susi @ 2021-05-28 14:44 UTC (permalink / raw)
  To: Jeff King
  Cc: Ævar Arnfjörð Bjarmason, Johannes Schindelin,
	Junio C Hamano, brian m. carlson, Felipe Contreras,
	Derrick Stolee, Bagas Sanjaya, Lénaïc Huard, git,
	Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Phillip Wood,
	Martin Ågren


Jeff King writes:

>   - temper small corrections with positive feedback. Especially for new
>     contributors, being told explicitly "yes, what you're trying to do
>     here overall is welcome, and it all looks good except for this..."
>     is much more encouraging than "this part is wrong". In the latter,
>     they're left to guess if anybody even values the rest of the work at
>     all.

When I see only a minor nit like that I assume that by default, that
means there are no more serious issues, fix the typo, and resubmit.  If
a new contributor thinks that means they aren't welcome then I think
they have an expectation mismatch.

>   - likewise, I think it helps to give feedback on expectations for the
>     process. Saying explicitly "this looks good; I think with this style
>     change, it would be ready to get picked up" helps them understand
>     that the fix will get them across the finish line (as opposed to
>     just getting another round of fix requests).

That would be nice, but such comments can really only come from a
maintainer that plans on pushing the patch.  Most comments come from
bystanders and so nessesarily only consist of pointing out flaws, and
don't really need to be bloated with a bunch of fluff.  I prefer short,
and to the point communication.

> I would even extend some of those into the code itself. Obviously we
> don't want to lower the bar and take incorrect code, or even typos in
> error messages. But I think we could stand to relax sometimes on issues
> of style or "I would do it like this" (and at the very least, the
> "temper small corrections" advice may apply).

Isn't saying "I would do it like this" already a tempering statement?  I
take that as meaning there isn't anything neccesarily wrong with what
you did, but you might consider this advice.


^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v4 3/4] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-05-24 10:12         ` Phillip Wood
@ 2021-05-30  6:39           ` Lénaïc Huard
  2021-05-30 10:16             ` Phillip Wood
  0 siblings, 1 reply; 138+ messages in thread
From: Lénaïc Huard @ 2021-05-30  6:39 UTC (permalink / raw)
  To: git, Phillip Wood
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Martin Ågren, Ævar Arnfjörð Bjarmason,
	Bagas Sanjaya, brian m . carlson, Johannes Schindelin

Hello Phillip,

I’m working on the next iteration of this patch, but I would have a question 
about one comment of the review.

Le lundi 24 mai 2021, 12:12:10 CEST Phillip Wood a écrit :
> > -static int maintenance_start(void)
> > +static const char *const builtin_maintenance_start_usage[] = {
> > +	N_("git maintenance start [--scheduler=<scheduler>]"), NULL
> > +};
> 
> I'm not sure what the { } and NULL are doing here, it should just be
> assigning a string. You could put it inside maintenance_start() and just
> call the variable "usage"

I did this because `parse_options(…, usage, …)` expects a NULL-terminated 
array of strings and not a single string.

While I agree that this variable doesn’t need to be global as it is used only 
from inside a single function, I followed the pattern that I saw elsewhere.

For ex. `builting_gc_usage` or `builtin_maintenance_run_usage` in `builtin/
gc.c`.

In fact, we’re even told to do so in `Documentation/technical/api-parse-
options.txt` which says:

    . define a NULL-terminated
      `static const char * const builtin_foo_usage[]` array
      containing alternative usage strings

In the files that I looked at, the command usage was always defined as a global 
long-named variable even if it was used in a single function.

> > +static int maintenance_start(int argc, const char **argv, const char
> > *prefix)> 
> >   {
> > 
> > +	struct maintenance_start_opts opts;
> > +	struct option builtin_maintenance_start_options[] = {
> 
> As this variable is local to the function you could call it something
> shorter like options.

I agree.
I also followed the pattern that I saw elsewhere.
For ex., still in `builtin/gc.c`, there are `builtin_gc_options` and 
`builtin_maintenance_run_options` which are local variables, but still defined 
with an explicit and unique long name.

So, I’m wondering if I should follow the existing pattern or if I should 
shorten the name of the local variable.
I thought the existing convention could be useful when grepping for option or 
usage of a command in the code ?



^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v4 3/4] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-05-30  6:39           ` Lénaïc Huard
@ 2021-05-30 10:16             ` Phillip Wood
  0 siblings, 0 replies; 138+ messages in thread
From: Phillip Wood @ 2021-05-30 10:16 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Martin Ågren, Ævar Arnfjörð Bjarmason,
	Bagas Sanjaya, brian m . carlson, Johannes Schindelin

Hi Lénaïc

On 30/05/2021 07:39, Lénaïc Huard wrote:
> Hello Phillip,
> 
> I’m working on the next iteration of this patch, but I would have a question
> about one comment of the review.
> 
> Le lundi 24 mai 2021, 12:12:10 CEST Phillip Wood a écrit :
>>> -static int maintenance_start(void)
>>> +static const char *const builtin_maintenance_start_usage[] = {
>>> +	N_("git maintenance start [--scheduler=<scheduler>]"), NULL
>>> +};
>>
>> I'm not sure what the { } and NULL are doing here, it should just be
>> assigning a string. You could put it inside maintenance_start() and just
>> call the variable "usage"
> 
> I did this because `parse_options(…, usage, …)` expects a NULL-terminated
> array of strings and not a single string.

Oh sorry I got confused by the definition of builtin_maintenance_usage 
which is just a string but cmd_maintenance() uses usage() rather than 
parse_options() which has a different api.

> While I agree that this variable doesn’t need to be global as it is used only
> from inside a single function, I followed the pattern that I saw elsewhere.
> 
> For ex. `builting_gc_usage` or `builtin_maintenance_run_usage` in `builtin/
> gc.c`.
> 
> In fact, we’re even told to do so in `Documentation/technical/api-parse-
> options.txt` which says:
> 
>      . define a NULL-terminated
>        `static const char * const builtin_foo_usage[]` array
>        containing alternative usage strings
> 
> In the files that I looked at, the command usage was always defined as a global
> long-named variable even if it was used in a single function.

Yeah, I'm not sure why that convention has built up. If there is a 
single command in a file then defining the usage string at the top of 
the file arguably acts as some form of documentation but when there is 
more than one command in a file I'm not so sure. Anyway feel free to 
leave it as it is.

>>> +static int maintenance_start(int argc, const char **argv, const char
>>> *prefix)>
>>>    {
>>>
>>> +	struct maintenance_start_opts opts;
>>> +	struct option builtin_maintenance_start_options[] = {
>>
>> As this variable is local to the function you could call it something
>> shorter like options.
> 
> I agree.
> I also followed the pattern that I saw elsewhere.
> For ex., still in `builtin/gc.c`, there are `builtin_gc_options` and
> `builtin_maintenance_run_options` which are local variables, but still defined
> with an explicit and unique long name.
> 
> So, I’m wondering if I should follow the existing pattern or if I should
> shorten the name of the local variable.
> I thought the existing convention could be useful when grepping for option or
> usage of a command in the code ?

It's up to you, personally I'd lean towards a shorter name defined in 
the function where it is used but if you're following a existing pattern 
then that should be fine too and would mean that you don't need to spend 
time changing what you've got already.

Best Wishes

Phillip

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: CoC, inclusivity etc. (was "Re: [...] systemd timers on Linux")
  2021-05-28 14:44                           ` Phillip Susi
@ 2021-05-30 21:58                             ` Jeff King
  0 siblings, 0 replies; 138+ messages in thread
From: Jeff King @ 2021-05-30 21:58 UTC (permalink / raw)
  To: Phillip Susi
  Cc: Ævar Arnfjörð Bjarmason, Johannes Schindelin,
	Junio C Hamano, brian m. carlson, Felipe Contreras,
	Derrick Stolee, Bagas Sanjaya, Lénaïc Huard, git,
	Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Phillip Wood,
	Martin Ågren

On Fri, May 28, 2021 at 10:44:25AM -0400, Phillip Susi wrote:

> >   - temper small corrections with positive feedback. Especially for new
> >     contributors, being told explicitly "yes, what you're trying to do
> >     here overall is welcome, and it all looks good except for this..."
> >     is much more encouraging than "this part is wrong". In the latter,
> >     they're left to guess if anybody even values the rest of the work at
> >     all.
> 
> When I see only a minor nit like that I assume that by default, that
> means there are no more serious issues, fix the typo, and resubmit.  If
> a new contributor thinks that means they aren't welcome then I think
> they have an expectation mismatch.

Sure, that may be your intent as a reviewer. My point was that the
recipient of the review does not always know that. Helping them
understand those expectations is part of welcoming them into the
community.

And even as somebody who has been part of the community for a long time
and understands that, it is still comforting to get actual positive
feedback, rather than an assumed "if I did not complain, it is OK".

> >   - likewise, I think it helps to give feedback on expectations for the
> >     process. Saying explicitly "this looks good; I think with this style
> >     change, it would be ready to get picked up" helps them understand
> >     that the fix will get them across the finish line (as opposed to
> >     just getting another round of fix requests).
> 
> That would be nice, but such comments can really only come from a
> maintainer that plans on pushing the patch.  Most comments come from
> bystanders and so nessesarily only consist of pointing out flaws, and
> don't really need to be bloated with a bunch of fluff.  I prefer short,
> and to the point communication.

Yes, I hesitated a little bit on this advice for that reason: it may be
even worse to mislead people about the state of a patch series, if you
the reviewer and the maintainer do not agree. IMHO it is still good for
reviewers to try to help manage newcomers through the process, but it
does make sense for them to be careful not to over-promise.

> > I would even extend some of those into the code itself. Obviously we
> > don't want to lower the bar and take incorrect code, or even typos in
> > error messages. But I think we could stand to relax sometimes on issues
> > of style or "I would do it like this" (and at the very least, the
> > "temper small corrections" advice may apply).
> 
> Isn't saying "I would do it like this" already a tempering statement?  I
> take that as meaning there isn't anything neccesarily wrong with what
> you did, but you might consider this advice.

Here I more meant cases where the two approaches have about the same
value. If your "I would do it like this" can be backed up with reasons
why it might be better (more efficient, more maintainable, and so on),
then that's probably helpful review to give. If it can't, then I'd
question whether it is worth the time to even bring up.

There's a big gray area, of course. Saying "it would be more readable
like this..." is sometimes a nuisance, and sometimes great advice. One
extra complication is that new contributors are often unsure how strong
the request is (e.g., if they disagree, do they _need_ to change it for
the patch to be accepted, or is it OK). I'll often qualify comments with
an explicit "I'm OK with doing it this way, but in case you really like this
other direction, I thought I'd mention it...".

Another complication with all of this advice is that sometimes new
contributors are in a mentoring relationship with one or more reviewers
(e.g., GSoC, Outreachy, or just people who have asked for help). And
there the cost/benefit tradeoff is different between frustrating a new
contributor and teaching them our style, norms, etc.

-Peff

^ permalink raw reply	[flat|nested] 138+ messages in thread

* [PATCH v5 0/3] add support for systemd timers on Linux
  2021-05-24  7:15     ` [PATCH v4 0/4] add support for " Lénaïc Huard
                         ` (4 preceding siblings ...)
  2021-05-24  9:04       ` [PATCH v4 0/4] " Junio C Hamano
@ 2021-06-08 13:39       ` Lénaïc Huard
  2021-06-08 13:39         ` [PATCH v5 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
                           ` (6 more replies)
  5 siblings, 7 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-06-08 13:39 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Lénaïc Huard

Hello,

I’ve reworked this submission based on the valuable feedback I’ve received.
Thanks again for it!

The patchset contains now the following patches:

* cache.h: Introduce a generic "xdg_config_home_for(…)" function

  This patch introduces a function to compute configuration files
  paths inside $XDG_CONFIG_HOME or ~/.config for other programs than
  git itself.
  It is used in the latest patch of this series to compute systemd
  unit files location.

  The only change in this patch compared to its previous version is
  the renaming of the first parameter of the `xdg_config_home_for(…)`
  function from `prog` to `subdir`.

* maintenance: introduce ENABLE/DISABLE for code clarity

  I just completely dropped this patch as it turned out that replacing
  some 0/1 values by `ENABLE`/`DISABLE` enum values wasn’t making the
  code look nicer as initially expected.

* maintenance: `git maintenance run` learned `--scheduler=<scheduler>`

  This patch contains all the code that is related to the addition of
  the new `--scheduler` parameter of the `git maintenance start`
  command, independently of the systemd timers.

  The main changes in this patch compared to its previous version are:

    * Revert all the changes that were previously introduced by the
      `ENABLE`/`DISABLE` enum values.

    * Remove the `strlcpy` in the testing framework inside the
      `get_schedule_cmd` function.

    * `update_background_schedule` loops over all the available
      schedulers, disables all of them except the one which is
      enabled.
      In this new version of the patch, it is now ensured that all the
      schedulers deactivation are done before the activation.
      The goal of this change is avoid a potential race condition
      where two schedulers could be enabled at the same time.
      This behaviour change has been reflected in the tests.

    * The local variable `builtin_maintenance_start_options` has been
      shortened.

* maintenance: add support for systemd timers on Linux

  This patch implements the support of systemd timers on top of
  crontab scheduler on Linux systems.

  The main changes in this patch compared to its previous version are:

    * The caching logic of `is_systemd_timer_available` has been
      dropped.
      I initially wanted to cache the outcome of forking and executing
      an external command to avoid doing it several times as
      `is_systemd_timer_available` is invoked from several places
      (`resolve_auto_scheduler`, `validate_scheduler` and
      `update_background_scheduler`).
      But it’s true they’re not always all called.
      In the case of `maintenance stop`, `resolve_auto_scheduler` and
      `validate_scheduler` are not called.
      In the case of `maintenance start`, the `if (enable &&
      opts->scheduler == i)` statement inside
      `update_background_schedule` skips the execution of
      `is_systemd_timer_available`.

    * The `is_systemd_timer_available` has been split in two parts:
      * `is_systemd_timer_available` is the entry point and holds the
        platform agnostic testing framework logic.
      * `real_is_systemd_timer_available` contains the platform
        specific logic.

    * The error management of `systemd_timer_write_unit_templates` has
      been reviewed.
      The return code of `fopen`, `fputs`, `fclose`, etc. are now
      checked.
      If this function manages to write one file, but fails at writing
      the second one, it will attempt to delete the first one to not
      leave the system in an inconsistent state.

    * The error management of `systemd_timer_delete_unit_templates`
      has also been reviewed. The error code of `unlink` is now
      checked.

I hope I’ve addressed all your valuable feedback. Do not hesitate to
let me know if I’ve forgotten anything.

Lénaïc Huard (3):
  cache.h: Introduce a generic "xdg_config_home_for(…)" function
  maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  maintenance: add support for systemd timers on Linux

 Documentation/git-maintenance.txt |  60 ++++
 builtin/gc.c                      | 564 ++++++++++++++++++++++++++----
 cache.h                           |   7 +
 path.c                            |  13 +-
 t/t7900-maintenance.sh            | 110 +++++-
 5 files changed, 676 insertions(+), 78 deletions(-)

-- 
2.32.0


^ permalink raw reply	[flat|nested] 138+ messages in thread

* [PATCH v5 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function
  2021-06-08 13:39       ` [PATCH v5 0/3] " Lénaïc Huard
@ 2021-06-08 13:39         ` Lénaïc Huard
  2021-06-08 13:39         ` [PATCH v5 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
                           ` (5 subsequent siblings)
  6 siblings, 0 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-06-08 13:39 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Lénaïc Huard

Current implementation of `xdg_config_home(filename)` returns
`$XDG_CONFIG_HOME/git/$filename`, with the `git` subdirectory inserted
between the `XDG_CONFIG_HOME` environment variable and the parameter.

This patch introduces a `xdg_config_home_for(subdir, filename)` function
which is more generic. It only concatenates "$XDG_CONFIG_HOME", or
"$HOME/.config" if the former isn’t defined, with the parameters,
without adding `git` in between.

`xdg_config_home(filename)` is now implemented by calling
`xdg_config_home_for("git", filename)` but this new generic function can
be used to compute the configuration directory of other programs.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 cache.h |  7 +++++++
 path.c  | 13 ++++++++++---
 2 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/cache.h b/cache.h
index ba04ff8bd3..2a0fb3e4ba 100644
--- a/cache.h
+++ b/cache.h
@@ -1286,6 +1286,13 @@ int is_ntfs_dotmailmap(const char *name);
  */
 int looks_like_command_line_option(const char *str);
 
+/**
+ * Return a newly allocated string with the evaluation of
+ * "$XDG_CONFIG_HOME/$subdir/$filename" if $XDG_CONFIG_HOME is non-empty, otherwise
+ * "$HOME/.config/$subdir/$filename". Return NULL upon error.
+ */
+char *xdg_config_home_for(const char *subdir, const char *filename);
+
 /**
  * Return a newly allocated string with the evaluation of
  * "$XDG_CONFIG_HOME/git/$filename" if $XDG_CONFIG_HOME is non-empty, otherwise
diff --git a/path.c b/path.c
index 7bccd830e9..1b1de3be09 100644
--- a/path.c
+++ b/path.c
@@ -1503,21 +1503,28 @@ int looks_like_command_line_option(const char *str)
 	return str && str[0] == '-';
 }
 
-char *xdg_config_home(const char *filename)
+char *xdg_config_home_for(const char *subdir, const char *filename)
 {
 	const char *home, *config_home;
 
+	assert(subdir);
 	assert(filename);
 	config_home = getenv("XDG_CONFIG_HOME");
 	if (config_home && *config_home)
-		return mkpathdup("%s/git/%s", config_home, filename);
+		return mkpathdup("%s/%s/%s", config_home, subdir, filename);
 
 	home = getenv("HOME");
 	if (home)
-		return mkpathdup("%s/.config/git/%s", home, filename);
+		return mkpathdup("%s/.config/%s/%s", home, subdir, filename);
+
 	return NULL;
 }
 
+char *xdg_config_home(const char *filename)
+{
+	return xdg_config_home_for("git", filename);
+}
+
 char *xdg_cache_home(const char *filename)
 {
 	const char *home, *cache_home;
-- 
2.32.0


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* [PATCH v5 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-06-08 13:39       ` [PATCH v5 0/3] " Lénaïc Huard
  2021-06-08 13:39         ` [PATCH v5 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
@ 2021-06-08 13:39         ` Lénaïc Huard
  2021-06-08 13:40         ` [PATCH v5 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
                           ` (4 subsequent siblings)
  6 siblings, 0 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-06-08 13:39 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Lénaïc Huard

Depending on the system, different schedulers can be used to schedule
the hourly, daily and weekly executions of `git maintenance run`:
* `launchctl` for MacOS,
* `schtasks` for Windows and
* `crontab` for everything else.

`git maintenance run` now has an option to let the end-user explicitly
choose which scheduler he wants to use:
`--scheduler=auto|crontab|launchctl|schtasks`.

When `git maintenance start --scheduler=XXX` is run, it not only
registers `git maintenance run` tasks in the scheduler XXX, it also
removes the `git maintenance run` tasks from all the other schedulers to
ensure we cannot have two schedulers launching concurrent identical
tasks.

The default value is `auto` which chooses a suitable scheduler for the
system.

`git maintenance stop` doesn't have any `--scheduler` parameter because
this command will try to remove the `git maintenance run` tasks from all
the available schedulers.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 Documentation/git-maintenance.txt |  11 +
 builtin/gc.c                      | 336 +++++++++++++++++++++++-------
 t/t7900-maintenance.sh            |  55 ++++-
 3 files changed, 327 insertions(+), 75 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 1e738ad398..07065ed4f3 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -179,6 +179,17 @@ OPTIONS
 	`maintenance.<task>.enabled` configured as `true` are considered.
 	See the 'TASKS' section for the list of accepted `<task>` values.
 
+--scheduler=auto|crontab|launchctl|schtasks::
+	When combined with the `start` subcommand, specify the scheduler
+	to use to run the hourly, daily and weekly executions of
+	`git maintenance run`.
+	The possible values for `<scheduler>` depend on the system: `crontab`
+	is available on POSIX systems, `launchctl` is available on
+	MacOS and `schtasks` is available on Windows.
+	By default or when `auto` is specified, a suitable scheduler for
+	the system is used. On MacOS, `launchctl` is used. On Windows,
+	`schtasks` is used. On all other systems, `crontab` is used.
+
 
 TROUBLESHOOTING
 ---------------
diff --git a/builtin/gc.c b/builtin/gc.c
index f05d2f0a1a..f2a81ecb44 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1529,6 +1529,59 @@ static const char *get_frequency(enum schedule_priority schedule)
 	}
 }
 
+static int get_schedule_cmd(const char **cmd, int *is_available)
+{
+	char *item;
+	char *testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
+
+	if (!testing)
+		return 0;
+
+	if (is_available)
+		*is_available = 0;
+
+	for(item = testing;;) {
+		char *sep;
+		char *end_item = strchr(item, ',');
+		if (end_item)
+			*end_item = '\0';
+
+		sep = strchr(item, ':');
+		if (!sep)
+			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
+		*sep = '\0';
+
+		if (!strcmp(*cmd, item)) {
+			*cmd = sep+1;
+			if (is_available)
+				*is_available = 1;
+			UNLEAK(testing);
+			return 1;
+		}
+
+		if (!end_item)
+			break;
+		item = end_item + 1;
+	}
+
+	free(testing);
+	return 1;
+}
+
+static int is_launchctl_available(void)
+{
+	const char *cmd = "launchctl";
+	int is_available;
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+#ifdef __APPLE__
+	return 1;
+#else
+	return 0;
+#endif
+}
+
 static char *launchctl_service_name(const char *frequency)
 {
 	struct strbuf label = STRBUF_INIT;
@@ -1555,19 +1608,17 @@ static char *launchctl_get_uid(void)
 	return xstrfmt("gui/%d", getuid());
 }
 
-static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
+static int launchctl_boot_plist(int enable, const char *filename)
 {
+	const char *cmd = "launchctl";
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
 	char *uid = launchctl_get_uid();
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&child.args, cmd);
-	if (enable)
-		strvec_push(&child.args, "bootstrap");
-	else
-		strvec_push(&child.args, "bootout");
-	strvec_push(&child.args, uid);
-	strvec_push(&child.args, filename);
+	strvec_pushl(&child.args, enable ? "bootstrap" : "bootout",
+		     uid, filename, NULL);
 
 	child.no_stderr = 1;
 	child.no_stdout = 1;
@@ -1581,26 +1632,26 @@ static int launchctl_boot_plist(int enable, const char *filename, const char *cm
 	return result;
 }
 
-static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)
+static int launchctl_remove_plist(enum schedule_priority schedule)
 {
 	const char *frequency = get_frequency(schedule);
 	char *name = launchctl_service_name(frequency);
 	char *filename = launchctl_service_filename(name);
-	int result = launchctl_boot_plist(0, filename, cmd);
+	int result = launchctl_boot_plist(0, filename);
 	unlink(filename);
 	free(filename);
 	free(name);
 	return result;
 }
 
-static int launchctl_remove_plists(const char *cmd)
+static int launchctl_remove_plists(void)
 {
-	return launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||
-		launchctl_remove_plist(SCHEDULE_DAILY, cmd) ||
-		launchctl_remove_plist(SCHEDULE_WEEKLY, cmd);
+	return launchctl_remove_plist(SCHEDULE_HOURLY) ||
+		launchctl_remove_plist(SCHEDULE_DAILY) ||
+		launchctl_remove_plist(SCHEDULE_WEEKLY);
 }
 
-static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule)
 {
 	FILE *plist;
 	int i;
@@ -1669,8 +1720,8 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 	fclose(plist);
 
 	/* bootout might fail if not already running, so ignore */
-	launchctl_boot_plist(0, filename, cmd);
-	if (launchctl_boot_plist(1, filename, cmd))
+	launchctl_boot_plist(0, filename);
+	if (launchctl_boot_plist(1, filename))
 		die(_("failed to bootstrap service %s"), filename);
 
 	free(filename);
@@ -1678,21 +1729,35 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 	return 0;
 }
 
-static int launchctl_add_plists(const char *cmd)
+static int launchctl_add_plists(void)
 {
 	const char *exec_path = git_exec_path();
 
-	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||
-		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||
-		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
+	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY) ||
+		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY) ||
+		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY);
 }
 
-static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int launchctl_update_schedule(int run_maintenance, int fd)
 {
 	if (run_maintenance)
-		return launchctl_add_plists(cmd);
+		return launchctl_add_plists();
 	else
-		return launchctl_remove_plists(cmd);
+		return launchctl_remove_plists();
+}
+
+static int is_schtasks_available(void)
+{
+	const char *cmd = "schtasks";
+	int is_available;
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+#ifdef GIT_WINDOWS_NATIVE
+	return 1;
+#else
+	return 0;
+#endif
 }
 
 static char *schtasks_task_name(const char *frequency)
@@ -1702,13 +1767,15 @@ static char *schtasks_task_name(const char *frequency)
 	return strbuf_detach(&label, NULL);
 }
 
-static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)
+static int schtasks_remove_task(enum schedule_priority schedule)
 {
+	const char *cmd = "schtasks";
 	int result;
 	struct strvec args = STRVEC_INIT;
 	const char *frequency = get_frequency(schedule);
 	char *name = schtasks_task_name(frequency);
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&args, cmd);
 	strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
 
@@ -1719,15 +1786,16 @@ static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd
 	return result;
 }
 
-static int schtasks_remove_tasks(const char *cmd)
+static int schtasks_remove_tasks(void)
 {
-	return schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||
-		schtasks_remove_task(SCHEDULE_DAILY, cmd) ||
-		schtasks_remove_task(SCHEDULE_WEEKLY, cmd);
+	return schtasks_remove_task(SCHEDULE_HOURLY) ||
+		schtasks_remove_task(SCHEDULE_DAILY) ||
+		schtasks_remove_task(SCHEDULE_WEEKLY);
 }
 
-static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule)
 {
+	const char *cmd = "schtasks";
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
 	const char *xml;
@@ -1736,6 +1804,8 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
 	char *name = schtasks_task_name(frequency);
 	struct strbuf tfilename = STRBUF_INIT;
 
+	get_schedule_cmd(&cmd, NULL);
+
 	strbuf_addf(&tfilename, "%s/schedule_%s_XXXXXX",
 		    get_git_common_dir(), frequency);
 	tfile = xmks_tempfile(tfilename.buf);
@@ -1840,28 +1910,52 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
 	return result;
 }
 
-static int schtasks_schedule_tasks(const char *cmd)
+static int schtasks_schedule_tasks(void)
 {
 	const char *exec_path = git_exec_path();
 
-	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||
-		schtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||
-		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
+	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY) ||
+		schtasks_schedule_task(exec_path, SCHEDULE_DAILY) ||
+		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY);
 }
 
-static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int schtasks_update_schedule(int run_maintenance, int fd)
 {
 	if (run_maintenance)
-		return schtasks_schedule_tasks(cmd);
+		return schtasks_schedule_tasks();
 	else
-		return schtasks_remove_tasks(cmd);
+		return schtasks_remove_tasks();
+}
+
+static int is_crontab_available(void)
+{
+	const char *cmd = "crontab";
+	int is_available;
+	struct child_process child = CHILD_PROCESS_INIT;
+
+	if (get_schedule_cmd(&cmd, &is_available) && !is_available)
+		return 0;
+
+	strvec_split(&child.args, cmd);
+	strvec_push(&child.args, "-l");
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child))
+		return 0;
+	/* Ignore exit code, as an empty crontab will return error. */
+	finish_command(&child);
+	return 1;
 }
 
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
-static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int crontab_update_schedule(int run_maintenance, int fd)
 {
+	const char *cmd = "crontab";
 	int result = 0;
 	int in_old_region = 0;
 	struct child_process crontab_list = CHILD_PROCESS_INIT;
@@ -1869,6 +1963,7 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 	FILE *cron_list, *cron_in;
 	struct strbuf line = STRBUF_INIT;
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&crontab_list.args, cmd);
 	strvec_push(&crontab_list.args, "-l");
 	crontab_list.in = -1;
@@ -1945,66 +2040,165 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 	return result;
 }
 
+enum scheduler {
+	SCHEDULER_INVALID = -1,
+	SCHEDULER_AUTO,
+	SCHEDULER_CRON,
+	SCHEDULER_LAUNCHCTL,
+	SCHEDULER_SCHTASKS,
+};
+
+static const struct {
+	const char *name;
+	int (*is_available)(void);
+	int (*update_schedule)(int run_maintenance, int fd);
+} scheduler_fn[] = {
+	[SCHEDULER_CRON] = {
+		.name = "crontab",
+		.is_available = is_crontab_available,
+		.update_schedule = crontab_update_schedule,
+	},
+	[SCHEDULER_LAUNCHCTL] = {
+		.name = "launchctl",
+		.is_available = is_launchctl_available,
+		.update_schedule = launchctl_update_schedule,
+	},
+	[SCHEDULER_SCHTASKS] = {
+		.name = "schtasks",
+		.is_available = is_schtasks_available,
+		.update_schedule = schtasks_update_schedule,
+	},
+};
+
+static enum scheduler parse_scheduler(const char *value)
+{
+	if (!value)
+		return SCHEDULER_INVALID;
+	else if (!strcasecmp(value, "auto"))
+		return SCHEDULER_AUTO;
+	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
+		return SCHEDULER_CRON;
+	else if (!strcasecmp(value, "launchctl"))
+		return SCHEDULER_LAUNCHCTL;
+	else if (!strcasecmp(value, "schtasks"))
+		return SCHEDULER_SCHTASKS;
+	else
+		return SCHEDULER_INVALID;
+}
+
+static int maintenance_opt_scheduler(const struct option *opt, const char *arg,
+				     int unset)
+{
+	enum scheduler *scheduler = opt->value;
+
+	BUG_ON_OPT_NEG(unset);
+
+	*scheduler = parse_scheduler(arg);
+	if (*scheduler == SCHEDULER_INVALID)
+		return error(_("unrecognized --scheduler argument '%s'"), arg);
+	return 0;
+}
+
+struct maintenance_start_opts {
+	enum scheduler scheduler;
+};
+
+static void resolve_auto_scheduler(enum scheduler *scheduler)
+{
+	if (*scheduler != SCHEDULER_AUTO)
+		return;
+
 #if defined(__APPLE__)
-static const char platform_scheduler[] = "launchctl";
+	*scheduler = SCHEDULER_LAUNCHCTL;
+	return;
+
 #elif defined(GIT_WINDOWS_NATIVE)
-static const char platform_scheduler[] = "schtasks";
+	*scheduler = SCHEDULER_SCHTASKS;
+	return;
+
 #else
-static const char platform_scheduler[] = "crontab";
+	*scheduler = SCHEDULER_CRON;
+	return;
 #endif
+}
 
-static int update_background_schedule(int enable)
+static void validate_scheduler(enum scheduler scheduler)
 {
-	int result;
-	const char *scheduler = platform_scheduler;
-	const char *cmd = scheduler;
-	char *testing;
+	if (scheduler == SCHEDULER_INVALID)
+		BUG("invalid scheduler");
+	if (scheduler == SCHEDULER_AUTO)
+		BUG("resolve_auto_scheduler should have been called before");
+
+	if (!scheduler_fn[scheduler].is_available())
+		die(_("%s scheduler is not available"),
+		    scheduler_fn[scheduler].name);
+}
+
+static int update_background_schedule(const struct maintenance_start_opts *opts,
+				      int enable)
+{
+	unsigned int i;
+	int result = 0;
 	struct lock_file lk;
 	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
 
-	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
-	if (testing) {
-		char *sep = strchr(testing, ':');
-		if (!sep)
-			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
-		*sep = '\0';
-		scheduler = testing;
-		cmd = sep + 1;
+	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {
+		free(lock_path);
+		return error(_("another process is scheduling background maintenance"));
 	}
 
-	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {
-		result = error(_("another process is scheduling background maintenance"));
-		goto cleanup;
+	for (i = 1; i < ARRAY_SIZE(scheduler_fn); i++) {
+		if (enable && opts->scheduler == i)
+			continue;
+		if (!scheduler_fn[i].is_available())
+			continue;
+		scheduler_fn[i].update_schedule(
+			0, get_lock_file_fd(&lk));
 	}
 
-	if (!strcmp(scheduler, "launchctl"))
-		result = launchctl_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else if (!strcmp(scheduler, "schtasks"))
-		result = schtasks_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else if (!strcmp(scheduler, "crontab"))
-		result = crontab_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else
-		die("unknown background scheduler: %s", scheduler);
+	if (enable)
+		result = scheduler_fn[opts->scheduler].update_schedule(
+			1, get_lock_file_fd(&lk));
 
 	rollback_lock_file(&lk);
 
-cleanup:
 	free(lock_path);
-	free(testing);
 	return result;
 }
 
-static int maintenance_start(void)
+static const char *const builtin_maintenance_start_usage[] = {
+	N_("git maintenance start [--scheduler=<scheduler>]"),
+	NULL
+};
+
+static int maintenance_start(int argc, const char **argv, const char *prefix)
 {
+	struct maintenance_start_opts opts = { 0 };
+	struct option options[] = {
+		OPT_CALLBACK_F(
+			0, "scheduler", &opts.scheduler, N_("scheduler"),
+			N_("scheduler to use to trigger git maintenance run"),
+			PARSE_OPT_NONEG, maintenance_opt_scheduler),
+		OPT_END()
+	};
+
+	argc = parse_options(argc, argv, prefix, options,
+			     builtin_maintenance_start_usage, 0);
+	if (argc)
+		usage_with_options(builtin_maintenance_start_usage,
+				   options);
+
+	resolve_auto_scheduler(&opts.scheduler);
+	validate_scheduler(opts.scheduler);
+
 	if (maintenance_register())
 		warning(_("failed to add repo to global config"));
-
-	return update_background_schedule(1);
+	return update_background_schedule(&opts, 1);
 }
 
 static int maintenance_stop(void)
 {
-	return update_background_schedule(0);
+	return update_background_schedule(NULL, 0);
 }
 
 static const char builtin_maintenance_usage[] =	N_("git maintenance <subcommand> [<options>]");
@@ -2018,7 +2212,7 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
 	if (!strcmp(argv[1], "run"))
 		return maintenance_run(argc - 1, argv + 1, prefix);
 	if (!strcmp(argv[1], "start"))
-		return maintenance_start();
+		return maintenance_start(argc - 1, argv + 1, prefix);
 	if (!strcmp(argv[1], "stop"))
 		return maintenance_stop();
 	if (!strcmp(argv[1], "register"))
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index b93ae014ee..b36b7f5fb0 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -494,8 +494,21 @@ test_expect_success !MINGW 'register and unregister with regex metacharacters' '
 		maintenance.repo "$(pwd)/$META"
 '
 
+test_expect_success 'start --scheduler=<scheduler>' '
+	test_expect_code 129 git maintenance start --scheduler=foo 2>err &&
+	test_i18ngrep "unrecognized --scheduler argument" err &&
+
+	test_expect_code 129 git maintenance start --no-scheduler 2>err &&
+	test_i18ngrep "unknown option" err &&
+
+	test_expect_code 128 \
+		env GIT_TEST_MAINT_SCHEDULER="launchctl:true,schtasks:true" \
+		git maintenance start --scheduler=crontab 2>err &&
+	test_i18ngrep "fatal: crontab scheduler is not available" err
+'
+
 test_expect_success 'start from empty cron table' '
-	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -518,7 +531,7 @@ test_expect_success 'stop from existing schedule' '
 
 test_expect_success 'start preserves existing schedule' '
 	echo "Important information!" >cron.txt &&
-	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
 	grep "Important information!" cron.txt
 '
 
@@ -547,7 +560,7 @@ test_expect_success 'start and stop macOS maintenance' '
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start --scheduler=launchctl &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -598,7 +611,7 @@ test_expect_success 'start and stop Windows maintenance' '
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start --scheduler=schtasks &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -621,6 +634,40 @@ test_expect_success 'start and stop Windows maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success 'start and stop when several schedulers are available' '
+	write_script print-args <<-\EOF &&
+	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
+	EOF
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >expect &&
+	for frequency in hourly daily weekly
+	do
+		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
+		echo "launchctl bootout gui/[UID] $PLIST" >>expect &&
+		echo "launchctl bootstrap gui/[UID] $PLIST" >>expect || return 1
+	done &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
2.32.0


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* [PATCH v5 3/3] maintenance: add support for systemd timers on Linux
  2021-06-08 13:39       ` [PATCH v5 0/3] " Lénaïc Huard
  2021-06-08 13:39         ` [PATCH v5 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
  2021-06-08 13:39         ` [PATCH v5 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
@ 2021-06-08 13:40         ` Lénaïc Huard
  2021-06-09  9:34           ` Jeff King
  2021-06-09 15:01           ` Phillip Wood
  2021-06-09  0:21         ` [PATCH v5 0/3] " Junio C Hamano
                           ` (3 subsequent siblings)
  6 siblings, 2 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-06-08 13:40 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Lénaïc Huard

The existing mechanism for scheduling background maintenance is done
through cron. On Linux systems managed by systemd, systemd provides an
alternative to schedule recurring tasks: systemd timers.

The main motivations to implement systemd timers in addition to cron
are:
* cron is optional and Linux systems running systemd might not have it
  installed.
* The execution of `crontab -l` can tell us if cron is installed but not
  if the daemon is actually running.
* With systemd, each service is run in its own cgroup and its logs are
  tagged by the service inside journald. With cron, all scheduled tasks
  are running in the cron daemon cgroup and all the logs of the
  user-scheduled tasks are pretended to belong to the system cron
  service.
  Concretely, a user that doesn’t have access to the system logs won’t
  have access to the log of their own tasks scheduled by cron whereas
  they will have access to the log of their own tasks scheduled by
  systemd timer.
  Although `cron` attempts to send email, that email may go unseen by
  the user because these days, local mailboxes are not heavily used
  anymore.

In order to schedule git maintenance, we need two unit template files:
* ~/.config/systemd/user/git-maintenance@.service
  to define the command to be started by systemd and
* ~/.config/systemd/user/git-maintenance@.timer
  to define the schedule at which the command should be run.

Those units are templates that are parameterized by the frequency.

Based on those templates, 3 timers are started:
* git-maintenance@hourly.timer
* git-maintenance@daily.timer
* git-maintenance@weekly.timer

The command launched by those three timers are the same as with the
other scheduling methods:

/path/to/git for-each-repo --exec-path=/path/to
--config=maintenance.repo maintenance run --schedule=%i

with the full path for git to ensure that the version of git launched
for the scheduled maintenance is the same as the one used to run
`maintenance start`.

The timer unit contains `Persistent=true` so that, if the computer is
powered down when a maintenance task should run, the task will be run
when the computer is back powered on.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 Documentation/git-maintenance.txt |  57 +++++++-
 builtin/gc.c                      | 228 ++++++++++++++++++++++++++++++
 t/t7900-maintenance.sh            |  67 ++++++++-
 3 files changed, 342 insertions(+), 10 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 07065ed4f3..7b7dbbbca9 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -179,16 +179,19 @@ OPTIONS
 	`maintenance.<task>.enabled` configured as `true` are considered.
 	See the 'TASKS' section for the list of accepted `<task>` values.
 
---scheduler=auto|crontab|launchctl|schtasks::
+--scheduler=auto|crontab|systemd-timer|launchctl|schtasks::
 	When combined with the `start` subcommand, specify the scheduler
 	to use to run the hourly, daily and weekly executions of
 	`git maintenance run`.
 	The possible values for `<scheduler>` depend on the system: `crontab`
-	is available on POSIX systems, `launchctl` is available on
-	MacOS and `schtasks` is available on Windows.
+	is available on POSIX systems, `systemd-timer` is available on Linux
+	systems, `launchctl` is available on MacOS and `schtasks` is available
+	on Windows.
 	By default or when `auto` is specified, a suitable scheduler for
 	the system is used. On MacOS, `launchctl` is used. On Windows,
-	`schtasks` is used. On all other systems, `crontab` is used.
+	`schtasks` is used. On Linux, `systemd-timer` is used if user systemd
+	timers are available, otherwise, `crontab` is used. On all other systems,
+	`crontab` is used.
 
 
 TROUBLESHOOTING
@@ -288,6 +291,52 @@ schedule to ensure you are executing the correct binaries in your
 schedule.
 
 
+BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMS
+-----------------------------------------------
+
+While Linux supports `cron`, depending on the distribution, `cron` may
+be an optional package not necessarily installed. On modern Linux
+distributions, systemd timers are superseding it.
+
+If user systemd timers are available, they will be used as a replacement
+of `cron`.
+
+In this case, `git maintenance start` will create user systemd timer units
+and start the timers. The current list of user-scheduled tasks can be found
+by running `systemctl --user list-timers`. The timers written by `git
+maintenance start` are similar to this:
+
+-----------------------------------------------------------------------
+$ systemctl --user list-timers
+NEXT                         LEFT          LAST                         PASSED     UNIT                         ACTIVATES
+Thu 2021-04-29 19:00:00 CEST 42min left    Thu 2021-04-29 18:00:11 CEST 17min ago  git-maintenance@hourly.timer git-maintenance@hourly.service
+Fri 2021-04-30 00:00:00 CEST 5h 42min left Thu 2021-04-29 00:00:11 CEST 18h ago    git-maintenance@daily.timer  git-maintenance@daily.service
+Mon 2021-05-03 00:00:00 CEST 3 days left   Mon 2021-04-26 00:00:11 CEST 3 days ago git-maintenance@weekly.timer git-maintenance@weekly.service
+-----------------------------------------------------------------------
+
+One timer is registered for each `--schedule=<frequency>` option.
+
+The definition of the systemd units can be inspected in the following files:
+
+-----------------------------------------------------------------------
+~/.config/systemd/user/git-maintenance@.timer
+~/.config/systemd/user/git-maintenance@.service
+~/.config/systemd/user/timers.target.wants/git-maintenance@hourly.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@daily.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@weekly.timer
+-----------------------------------------------------------------------
+
+`git maintenance start` will overwrite these files and start the timer
+again with `systemctl --user`, so any customization should be done by
+creating a drop-in file, i.e. a `.conf` suffixed file in the
+`~/.config/systemd/user/git-maintenance@.service.d` directory.
+
+`git maintenance stop` will stop the user systemd timers and delete
+the above mentioned files.
+
+For more details, see `systemd.timer(5)`.
+
+
 BACKGROUND MAINTENANCE ON MACOS SYSTEMS
 ---------------------------------------
 
diff --git a/builtin/gc.c b/builtin/gc.c
index f2a81ecb44..5fe416c903 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -2040,10 +2040,221 @@ static int crontab_update_schedule(int run_maintenance, int fd)
 	return result;
 }
 
+#ifdef __linux__
+
+static int real_is_systemd_timer_available(void)
+{
+	struct child_process child = CHILD_PROCESS_INIT;
+
+	strvec_pushl(&child.args, "systemctl", "--user", "list-timers", NULL);
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child))
+		return 0;
+	if (finish_command(&child))
+		return 0;
+	return 1;
+}
+
+#else
+
+static int real_is_systemd_timer_available(void)
+{
+	return 0;
+}
+
+#endif
+
+static int is_systemd_timer_available(void)
+{
+	const char *cmd = "systemctl";
+	int is_available;
+
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+	return real_is_systemd_timer_available();
+}
+
+static char *xdg_config_home_systemd(const char *filename)
+{
+	return xdg_config_home_for("systemd/user", filename);
+}
+
+static int systemd_timer_enable_unit(int enable,
+				     enum schedule_priority schedule)
+{
+	const char *cmd = "systemctl";
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *frequency = get_frequency(schedule);
+
+	/*
+	 * Disabling the systemd unit while it is already disabled makes
+	 * systemctl print an error.
+	 * Let's ignore it since it means we already are in the expected state:
+	 * the unit is disabled.
+	 *
+	 * On the other hand, enabling a systemd unit which is already enabled
+	 * produces no error.
+	 */
+	if (!enable)
+		child.no_stderr = 1;
+
+	get_schedule_cmd(&cmd, NULL);
+	strvec_split(&child.args, cmd);
+	strvec_pushl(&child.args, "--user", enable ? "enable" : "disable",
+		     "--now", NULL);
+	strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency);
+
+	if (start_command(&child))
+		return error(_("failed to start systemctl"));
+	if (finish_command(&child))
+		/*
+		 * Disabling an already disabled systemd unit makes
+		 * systemctl fail.
+		 * Let's ignore this failure.
+		 *
+		 * Enabling an enabled systemd unit doesn't fail.
+		 */
+		if (enable)
+			return error(_("failed to run systemctl"));
+	return 0;
+}
+
+static int systemd_timer_delete_unit_templates(void)
+{
+	int ret = 0;
+	char *filename = xdg_config_home_systemd("git-maintenance@.timer");
+	if(unlink(filename) && !is_missing_file_error(errno))
+		ret = error_errno(_("failed to delete '%s'"), filename);
+	FREE_AND_NULL(filename);
+
+	filename = xdg_config_home_systemd("git-maintenance@.service");
+	if(unlink(filename) && !is_missing_file_error(errno))
+		ret = error_errno(_("failed to delete '%s'"), filename);
+
+	free(filename);
+	return ret;
+}
+
+static int systemd_timer_delete_units(void)
+{
+	return systemd_timer_enable_unit(0, SCHEDULE_HOURLY) ||
+	       systemd_timer_enable_unit(0, SCHEDULE_DAILY) ||
+	       systemd_timer_enable_unit(0, SCHEDULE_WEEKLY) ||
+	       systemd_timer_delete_unit_templates();
+}
+
+static int systemd_timer_write_unit_templates(const char *exec_path)
+{
+	char *filename;
+	FILE *file;
+	const char *unit;
+
+	filename = xdg_config_home_systemd("git-maintenance@.timer");
+	if (safe_create_leading_directories(filename)) {
+		error(_("failed to create directories for '%s'"), filename);
+		goto error;
+	}
+	file = fopen_or_warn(filename, "w");
+	if (file == NULL)
+		goto error;
+	FREE_AND_NULL(filename);
+
+	unit = "# This file was created and is maintained by Git.\n"
+	       "# Any edits made in this file might be replaced in the future\n"
+	       "# by a Git command.\n"
+	       "\n"
+	       "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Timer]\n"
+	       "OnCalendar=%i\n"
+	       "Persistent=true\n"
+	       "\n"
+	       "[Install]\n"
+	       "WantedBy=timers.target\n";
+	if (fputs(unit, file) == EOF) {
+		error(_("failed to write to '%s'"), filename);
+		fclose(file);
+		goto error;
+	}
+	if (fclose(file) == EOF) {
+		error_errno(_("failed to flush '%s'"), filename);
+		goto error;
+	}
+
+	filename = xdg_config_home_systemd("git-maintenance@.service");
+	file = fopen_or_warn(filename, "w");
+	if (file == NULL)
+		goto error;
+	FREE_AND_NULL(filename);
+
+	unit = "# This file was created and is maintained by Git.\n"
+	       "# Any edits made in this file might be replaced in the future\n"
+	       "# by a Git command.\n"
+	       "\n"
+	       "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Service]\n"
+	       "Type=oneshot\n"
+	       "ExecStart=\"%s/git\" --exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
+	       "LockPersonality=yes\n"
+	       "MemoryDenyWriteExecute=yes\n"
+	       "NoNewPrivileges=yes\n"
+	       "RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\n"
+	       "RestrictNamespaces=yes\n"
+	       "RestrictRealtime=yes\n"
+	       "RestrictSUIDSGID=yes\n"
+	       "SystemCallArchitectures=native\n"
+	       "SystemCallFilter=@system-service\n";
+	if (fprintf(file, unit, exec_path, exec_path) < 0) {
+		error(_("failed to write to '%s'"), filename);
+		fclose(file);
+		goto error;
+	}
+	if (fclose(file) == EOF) {
+		error_errno(_("failed to flush '%s'"), filename);
+		goto error;
+	}
+	return 0;
+
+error:
+	free(filename);
+	systemd_timer_delete_unit_templates();
+	return -1;
+}
+
+static int systemd_timer_setup_units(void)
+{
+	const char *exec_path = git_exec_path();
+
+	int ret = systemd_timer_write_unit_templates(exec_path) ||
+	          systemd_timer_enable_unit(1, SCHEDULE_HOURLY) ||
+	          systemd_timer_enable_unit(1, SCHEDULE_DAILY) ||
+	          systemd_timer_enable_unit(1, SCHEDULE_WEEKLY);
+	if (ret)
+		systemd_timer_delete_units();
+	return ret;
+}
+
+static int systemd_timer_update_schedule(int run_maintenance, int fd)
+{
+	if (run_maintenance)
+		return systemd_timer_setup_units();
+	else
+		return systemd_timer_delete_units();
+}
+
 enum scheduler {
 	SCHEDULER_INVALID = -1,
 	SCHEDULER_AUTO,
 	SCHEDULER_CRON,
+	SCHEDULER_SYSTEMD,
 	SCHEDULER_LAUNCHCTL,
 	SCHEDULER_SCHTASKS,
 };
@@ -2058,6 +2269,11 @@ static const struct {
 		.is_available = is_crontab_available,
 		.update_schedule = crontab_update_schedule,
 	},
+	[SCHEDULER_SYSTEMD] = {
+		.name = "systemctl",
+		.is_available = is_systemd_timer_available,
+		.update_schedule = systemd_timer_update_schedule,
+	},
 	[SCHEDULER_LAUNCHCTL] = {
 		.name = "launchctl",
 		.is_available = is_launchctl_available,
@@ -2078,6 +2294,9 @@ static enum scheduler parse_scheduler(const char *value)
 		return SCHEDULER_AUTO;
 	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
 		return SCHEDULER_CRON;
+	else if (!strcasecmp(value, "systemd") ||
+		 !strcasecmp(value, "systemd-timer"))
+		return SCHEDULER_SYSTEMD;
 	else if (!strcasecmp(value, "launchctl"))
 		return SCHEDULER_LAUNCHCTL;
 	else if (!strcasecmp(value, "schtasks"))
@@ -2116,6 +2335,15 @@ static void resolve_auto_scheduler(enum scheduler *scheduler)
 	*scheduler = SCHEDULER_SCHTASKS;
 	return;
 
+#elif defined(__linux__)
+	if (is_systemd_timer_available())
+		*scheduler = SCHEDULER_SYSTEMD;
+	else if (is_crontab_available())
+		*scheduler = SCHEDULER_CRON;
+	else
+		die(_("neither systemd timers nor crontab are available"));
+	return;
+
 #else
 	*scheduler = SCHEDULER_CRON;
 	return;
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index b36b7f5fb0..b289cae6b9 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -20,6 +20,18 @@ test_xmllint () {
 	fi
 }
 
+test_lazy_prereq SYSTEMD_ANALYZE '
+	systemd-analyze --help >out &&
+	grep verify out
+'
+
+test_systemd_analyze_verify () {
+	if test_have_prereq SYSTEMD_ANALYZE
+	then
+		systemd-analyze verify "$@"
+	fi
+}
+
 test_expect_success 'help text' '
 	test_expect_code 129 git maintenance -h 2>err &&
 	test_i18ngrep "usage: git maintenance <subcommand>" err &&
@@ -634,15 +646,56 @@ test_expect_success 'start and stop Windows maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success 'start and stop Linux/systemd maintenance' '
+	write_script print-args <<-\EOF &&
+	printf "%s\n" "$*" >>args
+	EOF
+
+	XDG_CONFIG_HOME="$PWD" &&
+	export XDG_CONFIG_HOME &&
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance start --scheduler=systemd-timer &&
+
+	# start registers the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_systemd_analyze_verify "systemd/user/git-maintenance@.service" &&
+
+	printf -- "--user enable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_path_is_missing "systemd/user/git-maintenance@.timer" &&
+	test_path_is_missing "systemd/user/git-maintenance@.service" &&
+
+	printf -- "--user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'start and stop when several schedulers are available' '
 	write_script print-args <<-\EOF &&
 	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
-	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=systemd-timer &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
 		hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
+	printf -- "systemctl --user enable --now git-maintenance@%s.timer\n" hourly daily weekly >>expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
 	for frequency in hourly daily weekly
 	do
 		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
@@ -652,17 +705,19 @@ test_expect_success 'start and stop when several schedulers are available' '
 	test_cmp expect args &&
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
 	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
-		hourly daily weekly >expect &&
+		hourly daily weekly >>expect &&
 	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
 		hourly daily weekly >>expect &&
 	test_cmp expect args &&
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
 	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
-		hourly daily weekly >expect &&
+		hourly daily weekly >>expect &&
 	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
 		hourly daily weekly >>expect &&
 	test_cmp expect args
-- 
2.32.0


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* Re: [PATCH v2 1/1] maintenance: use systemd timers on Linux
  2021-05-10 18:03     ` Phillip Wood
  2021-05-10 18:25       ` Eric Sunshine
@ 2021-06-08 14:55       ` Lénaïc Huard
  1 sibling, 0 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-06-08 14:55 UTC (permalink / raw)
  To: git, phillip.wood
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine, brian m . carlson,
	Bagas Sanjaya, Đoàn Trần Công Danh,
	Ævar Arnfjörð Bjarmason

Hello Phillip,

Le lundi 10 mai 2021, 20:03:58 CEST Phillip Wood a écrit :
> > +	unit = "# This file was created and is maintained by Git.\n"
> > +	       "# Any edits made in this file might be replaced in the 
future\n"
> > +	       "# by a Git command.\n"
> > +	       "\n"
> > +	       "[Unit]\n"
> > +	       "Description=Optimize Git repositories data\n"
> > +	       "\n"
> > +	       "[Service]\n"
> > +	       "Type=oneshot\n"
> > +	       "ExecStart=\"%1$s/git\" --exec-path=\"%1$s\" for-each-repo
> > --config=maintenance.repo maintenance run --schedule=%%i\n" +	      
> > "LockPersonality=yes\n"
> > +	       "MemoryDenyWriteExecute=yes\n"
> > +	       "NoNewPrivileges=yes\n"
> > +	       "RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\n"
> > +	       "RestrictNamespaces=yes\n"
> > +	       "RestrictRealtime=yes\n"
> > +	       "RestrictSUIDSGID=yes\n"
> 
> After a quick read of the systemd.exec man page it is unclear to me if
> these Restrict... lines are needed as we already have
> NoNewPrivileges=yes - maybe they have some effect if `git maintence` is
> run as root?

I think that the only thing that `NoNewPrivileges=yes` do is to set the no new 
privileges flag described in [1] on the process.

The `Restrict…` options are enabling some other sandboxing features by 
blocking some syscalls through a seccomp profile.

My understanding of the systemd.exec man page is the other way round, i.e.: as 
soon as there’s a `Restrict…` option, the `NoNewPrivileges=yes` is implied.

So, I would say that strictly speaking `NoNewPrivileges=yes` isn’t needed.
But `NoNewPrivileges=yes` doesn’t imply the `Restrict…` options.

But I thought that, from a security point of view, it’s better to set as many 
sandboxing options as possible and be as explicit as possible.

[1] https://www.kernel.org/doc/html/latest/userspace-api/no_new_privs.html




^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v5 0/3] add support for systemd timers on Linux
  2021-06-08 13:39       ` [PATCH v5 0/3] " Lénaïc Huard
                           ` (2 preceding siblings ...)
  2021-06-08 13:40         ` [PATCH v5 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
@ 2021-06-09  0:21         ` Junio C Hamano
  2021-06-09 14:54         ` Phillip Wood
                           ` (2 subsequent siblings)
  6 siblings, 0 replies; 138+ messages in thread
From: Junio C Hamano @ 2021-06-09  0:21 UTC (permalink / raw)
  To: Lénaïc Huard
  Cc: git, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin

Lénaïc Huard <lenaic@lhuard.fr> writes:

> Hello,
>
> I’ve reworked this submission based on the valuable feedback I’ve received.
> Thanks again for it!
>
> The patchset contains now the following patches:
> ...

A summary very well written.  I wish all the cover letters were
written like this one.

> I hope I’ve addressed all your valuable feedback. Do not hesitate to
> let me know if I’ve forgotten anything.
>
> Lénaïc Huard (3):
>   cache.h: Introduce a generic "xdg_config_home_for(…)" function
>   maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
>   maintenance: add support for systemd timers on Linux

Thanks.

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v5 3/3] maintenance: add support for systemd timers on Linux
  2021-06-08 13:40         ` [PATCH v5 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
@ 2021-06-09  9:34           ` Jeff King
  2021-06-09 15:01           ` Phillip Wood
  1 sibling, 0 replies; 138+ messages in thread
From: Jeff King @ 2021-06-09  9:34 UTC (permalink / raw)
  To: Lénaïc Huard
  Cc: git, Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin

On Tue, Jun 08, 2021 at 03:40:00PM +0200, Lénaïc Huard wrote:

> +static int systemd_timer_write_unit_templates(const char *exec_path)
> +{
> +	char *filename;
> +	FILE *file;
> +	const char *unit;
> +
> +	filename = xdg_config_home_systemd("git-maintenance@.timer");
> +	if (safe_create_leading_directories(filename)) {
> +		error(_("failed to create directories for '%s'"), filename);
> +		goto error;
> +	}
> +	file = fopen_or_warn(filename, "w");
> +	if (file == NULL)
> +		goto error;
> +	FREE_AND_NULL(filename);

Here we free the filename variable. But later...

> +	unit = "# This file was created and is maintained by Git.\n"
> +	       "# Any edits made in this file might be replaced in the future\n"
> +	       "# by a Git command.\n"
> +	       "\n"
> +	       "[Unit]\n"
> +	       "Description=Optimize Git repositories data\n"
> +	       "\n"
> +	       "[Timer]\n"
> +	       "OnCalendar=%i\n"
> +	       "Persistent=true\n"
> +	       "\n"
> +	       "[Install]\n"
> +	       "WantedBy=timers.target\n";
> +	if (fputs(unit, file) == EOF) {
> +		error(_("failed to write to '%s'"), filename);
> +		fclose(file);
> +		goto error;
> +	}
> +	if (fclose(file) == EOF) {
> +		error_errno(_("failed to flush '%s'"), filename);
> +		goto error;
> +	}

If we see an error we'll try to use it as part of the message. I think
the FREE_AND_NULL() can just be moved down here. And really just be
free(), since we then immediately reassign it:

> +	filename = xdg_config_home_systemd("git-maintenance@.service");
> +	file = fopen_or_warn(filename, "w");
> +	if (file == NULL)
> +		goto error;
> +	FREE_AND_NULL(filename);

And then this one has the same problem. We free here, but...

> +	unit = "# This file was created and is maintained by Git.\n"
> +	       "# Any edits made in this file might be replaced in the future\n"
> +	       "# by a Git command.\n"
> +	       "\n"
> +	       "[Unit]\n"
> +	       "Description=Optimize Git repositories data\n"
> +	       "\n"
> +	       "[Service]\n"
> +	       "Type=oneshot\n"
> +	       "ExecStart=\"%s/git\" --exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
> +	       "LockPersonality=yes\n"
> +	       "MemoryDenyWriteExecute=yes\n"
> +	       "NoNewPrivileges=yes\n"
> +	       "RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\n"
> +	       "RestrictNamespaces=yes\n"
> +	       "RestrictRealtime=yes\n"
> +	       "RestrictSUIDSGID=yes\n"
> +	       "SystemCallArchitectures=native\n"
> +	       "SystemCallFilter=@system-service\n";
> +	if (fprintf(file, unit, exec_path, exec_path) < 0) {
> +		error(_("failed to write to '%s'"), filename);
> +		fclose(file);
> +		goto error;
> +	}
> +	if (fclose(file) == EOF) {
> +		error_errno(_("failed to flush '%s'"), filename);
> +		goto error;
> +	}

...use it in the error messages. This one could also just be free()
before we return:

> +	return 0;
> +
> +error:
> +	free(filename);
> +	systemd_timer_delete_unit_templates();
> +	return -1;
> +}

And all of the jumps to the error label are fine, since it frees
the filename (and we don't have to worry about FREE_AND_NULL, since it
would always be valid during those jumps).

-Peff

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v5 0/3] add support for systemd timers on Linux
  2021-06-08 13:39       ` [PATCH v5 0/3] " Lénaïc Huard
                           ` (3 preceding siblings ...)
  2021-06-09  0:21         ` [PATCH v5 0/3] " Junio C Hamano
@ 2021-06-09 14:54         ` Phillip Wood
  2021-06-12 16:50         ` [PATCH v6 0/3] maintenance: " Lénaïc Huard
  2021-08-17 17:22         ` [PATCH v5 0/3] " Derrick Stolee
  6 siblings, 0 replies; 138+ messages in thread
From: Phillip Wood @ 2021-06-09 14:54 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Martin Ågren, Ævar Arnfjörð Bjarmason,
	Bagas Sanjaya, brian m . carlson, Johannes Schindelin

Hi Lénaïc

Thanks for the excellent cover letter, I found it very useful while 
reviewing these patches. I think the changes address all of my previous 
concerns, the error handling in the last patch looks good. Having read 
through the patches I don't have anything to add to Peff's comments - 
with those small memory management fixed I think this will be a good shape.

Thanks for your work on this

Phillip

On 08/06/2021 14:39, Lénaïc Huard wrote:
>[...] > The patchset contains now the following patches:
> 
> * cache.h: Introduce a generic "xdg_config_home_for(…)" function
> 
>    This patch introduces a function to compute configuration files
>    paths inside $XDG_CONFIG_HOME or ~/.config for other programs than
>    git itself.
>    It is used in the latest patch of this series to compute systemd
>    unit files location.
> 
>    The only change in this patch compared to its previous version is
>    the renaming of the first parameter of the `xdg_config_home_for(…)`
>    function from `prog` to `subdir`.
> 
> * maintenance: introduce ENABLE/DISABLE for code clarity
> 
>    I just completely dropped this patch as it turned out that replacing
>    some 0/1 values by `ENABLE`/`DISABLE` enum values wasn’t making the
>    code look nicer as initially expected.
> 
> * maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
> 
>    This patch contains all the code that is related to the addition of
>    the new `--scheduler` parameter of the `git maintenance start`
>    command, independently of the systemd timers.
> 
>    The main changes in this patch compared to its previous version are:
> 
>      * Revert all the changes that were previously introduced by the
>        `ENABLE`/`DISABLE` enum values.
> 
>      * Remove the `strlcpy` in the testing framework inside the
>        `get_schedule_cmd` function.
> 
>      * `update_background_schedule` loops over all the available
>        schedulers, disables all of them except the one which is
>        enabled.
>        In this new version of the patch, it is now ensured that all the
>        schedulers deactivation are done before the activation.
>        The goal of this change is avoid a potential race condition
>        where two schedulers could be enabled at the same time.
>        This behaviour change has been reflected in the tests.
> 
>      * The local variable `builtin_maintenance_start_options` has been
>        shortened.
> 
> * maintenance: add support for systemd timers on Linux
> 
>    This patch implements the support of systemd timers on top of
>    crontab scheduler on Linux systems.
> 
>    The main changes in this patch compared to its previous version are:
> 
>      * The caching logic of `is_systemd_timer_available` has been
>        dropped.
>        I initially wanted to cache the outcome of forking and executing
>        an external command to avoid doing it several times as
>        `is_systemd_timer_available` is invoked from several places
>        (`resolve_auto_scheduler`, `validate_scheduler` and
>        `update_background_scheduler`).
>        But it’s true they’re not always all called.
>        In the case of `maintenance stop`, `resolve_auto_scheduler` and
>        `validate_scheduler` are not called.
>        In the case of `maintenance start`, the `if (enable &&
>        opts->scheduler == i)` statement inside
>        `update_background_schedule` skips the execution of
>        `is_systemd_timer_available`.
> 
>      * The `is_systemd_timer_available` has been split in two parts:
>        * `is_systemd_timer_available` is the entry point and holds the
>          platform agnostic testing framework logic.
>        * `real_is_systemd_timer_available` contains the platform
>          specific logic.
> 
>      * The error management of `systemd_timer_write_unit_templates` has
>        been reviewed.
>        The return code of `fopen`, `fputs`, `fclose`, etc. are now
>        checked.
>        If this function manages to write one file, but fails at writing
>        the second one, it will attempt to delete the first one to not
>        leave the system in an inconsistent state.
> 
>      * The error management of `systemd_timer_delete_unit_templates`
>        has also been reviewed. The error code of `unlink` is now
>        checked.
> 
> I hope I’ve addressed all your valuable feedback. Do not hesitate to
> let me know if I’ve forgotten anything.
> 
> Lénaïc Huard (3):
>    cache.h: Introduce a generic "xdg_config_home_for(…)" function
>    maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
>    maintenance: add support for systemd timers on Linux
> 
>   Documentation/git-maintenance.txt |  60 ++++
>   builtin/gc.c                      | 564 ++++++++++++++++++++++++++----
>   cache.h                           |   7 +
>   path.c                            |  13 +-
>   t/t7900-maintenance.sh            | 110 +++++-
>   5 files changed, 676 insertions(+), 78 deletions(-)
> 

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v5 3/3] maintenance: add support for systemd timers on Linux
  2021-06-08 13:40         ` [PATCH v5 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
  2021-06-09  9:34           ` Jeff King
@ 2021-06-09 15:01           ` Phillip Wood
  1 sibling, 0 replies; 138+ messages in thread
From: Phillip Wood @ 2021-06-09 15:01 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Martin Ågren, Ævar Arnfjörð Bjarmason,
	Bagas Sanjaya, brian m . carlson, Johannes Schindelin



On 08/06/2021 14:40, Lénaïc Huard wrote:
> The existing mechanism for scheduling background maintenance is done
> through cron. On Linux systems managed by systemd, systemd provides an
> alternative to schedule recurring tasks: systemd timers.
> 
> The main motivations to implement systemd timers in addition to cron
> are:
> * cron is optional and Linux systems running systemd might not have it
>    installed.
> * The execution of `crontab -l` can tell us if cron is installed but not
>    if the daemon is actually running.
> * With systemd, each service is run in its own cgroup and its logs are
>    tagged by the service inside journald. With cron, all scheduled tasks
>    are running in the cron daemon cgroup and all the logs of the
>    user-scheduled tasks are pretended to belong to the system cron
>    service.
>    Concretely, a user that doesn’t have access to the system logs won’t
>    have access to the log of their own tasks scheduled by cron whereas
>    they will have access to the log of their own tasks scheduled by
>    systemd timer.
>    Although `cron` attempts to send email, that email may go unseen by
>    the user because these days, local mailboxes are not heavily used
>    anymore.
> 
> In order to schedule git maintenance, we need two unit template files:
> * ~/.config/systemd/user/git-maintenance@.service
>    to define the command to be started by systemd and
> * ~/.config/systemd/user/git-maintenance@.timer
>    to define the schedule at which the command should be run.
> 
> Those units are templates that are parameterized by the frequency.
> 
> Based on those templates, 3 timers are started:
> * git-maintenance@hourly.timer
> * git-maintenance@daily.timer
> * git-maintenance@weekly.timer
> 
> The command launched by those three timers are the same as with the
> other scheduling methods:
> 
> /path/to/git for-each-repo --exec-path=/path/to
> --config=maintenance.repo maintenance run --schedule=%i
> 
> with the full path for git to ensure that the version of git launched
> for the scheduled maintenance is the same as the one used to run
> `maintenance start`.
> 
> The timer unit contains `Persistent=true` so that, if the computer is
> powered down when a maintenance task should run, the task will be run
> when the computer is back powered on.
> 
> Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
> ---
>   Documentation/git-maintenance.txt |  57 +++++++-
>   builtin/gc.c                      | 228 ++++++++++++++++++++++++++++++
>   t/t7900-maintenance.sh            |  67 ++++++++-
>   3 files changed, 342 insertions(+), 10 deletions(-)
> 
> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
> index 07065ed4f3..7b7dbbbca9 100644
> --- a/Documentation/git-maintenance.txt
> +++ b/Documentation/git-maintenance.txt
> @@ -179,16 +179,19 @@ OPTIONS
>   	`maintenance.<task>.enabled` configured as `true` are considered.
>   	See the 'TASKS' section for the list of accepted `<task>` values.
>   
> ---scheduler=auto|crontab|launchctl|schtasks::
> +--scheduler=auto|crontab|systemd-timer|launchctl|schtasks::
>   	When combined with the `start` subcommand, specify the scheduler
>   	to use to run the hourly, daily and weekly executions of
>   	`git maintenance run`.
>   	The possible values for `<scheduler>` depend on the system: `crontab`
> -	is available on POSIX systems, `launchctl` is available on
> -	MacOS and `schtasks` is available on Windows.
> +	is available on POSIX systems, `systemd-timer` is available on Linux
> +	systems, `launchctl` is available on MacOS and `schtasks` is available
> +	on Windows.
>   	By default or when `auto` is specified, a suitable scheduler for
>   	the system is used. On MacOS, `launchctl` is used. On Windows,
> -	`schtasks` is used. On all other systems, `crontab` is used.
> +	`schtasks` is used. On Linux, `systemd-timer` is used if user systemd
> +	timers are available, otherwise, `crontab` is used. On all other systems,
> +	`crontab` is used.
>   
>   
>   TROUBLESHOOTING
> @@ -288,6 +291,52 @@ schedule to ensure you are executing the correct binaries in your
>   schedule.
>   
>   
> +BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMS
> +-----------------------------------------------
> +
> +While Linux supports `cron`, depending on the distribution, `cron` may
> +be an optional package not necessarily installed. On modern Linux
> +distributions, systemd timers are superseding it.
> +
> +If user systemd timers are available, they will be used as a replacement
> +of `cron`.
> +
> +In this case, `git maintenance start` will create user systemd timer units
> +and start the timers. The current list of user-scheduled tasks can be found
> +by running `systemctl --user list-timers`. The timers written by `git
> +maintenance start` are similar to this:
> +
> +-----------------------------------------------------------------------
> +$ systemctl --user list-timers
> +NEXT                         LEFT          LAST                         PASSED     UNIT                         ACTIVATES
> +Thu 2021-04-29 19:00:00 CEST 42min left    Thu 2021-04-29 18:00:11 CEST 17min ago  git-maintenance@hourly.timer git-maintenance@hourly.service
> +Fri 2021-04-30 00:00:00 CEST 5h 42min left Thu 2021-04-29 00:00:11 CEST 18h ago    git-maintenance@daily.timer  git-maintenance@daily.service
> +Mon 2021-05-03 00:00:00 CEST 3 days left   Mon 2021-04-26 00:00:11 CEST 3 days ago git-maintenance@weekly.timer git-maintenance@weekly.service
> +-----------------------------------------------------------------------
> +
> +One timer is registered for each `--schedule=<frequency>` option.
> +
> +The definition of the systemd units can be inspected in the following files:
> +
> +-----------------------------------------------------------------------
> +~/.config/systemd/user/git-maintenance@.timer
> +~/.config/systemd/user/git-maintenance@.service
> +~/.config/systemd/user/timers.target.wants/git-maintenance@hourly.timer
> +~/.config/systemd/user/timers.target.wants/git-maintenance@daily.timer
> +~/.config/systemd/user/timers.target.wants/git-maintenance@weekly.timer
> +-----------------------------------------------------------------------
> +
> +`git maintenance start` will overwrite these files and start the timer
> +again with `systemctl --user`, so any customization should be done by
> +creating a drop-in file, i.e. a `.conf` suffixed file in the
> +`~/.config/systemd/user/git-maintenance@.service.d` directory.
> +
> +`git maintenance stop` will stop the user systemd timers and delete
> +the above mentioned files.
> +
> +For more details, see `systemd.timer(5)`.
> +
> +
>   BACKGROUND MAINTENANCE ON MACOS SYSTEMS
>   ---------------------------------------
>   
> diff --git a/builtin/gc.c b/builtin/gc.c
> index f2a81ecb44..5fe416c903 100644
> --- a/builtin/gc.c
> +++ b/builtin/gc.c
> @@ -2040,10 +2040,221 @@ static int crontab_update_schedule(int run_maintenance, int fd)
>   	return result;
>   }
>   
> +#ifdef __linux__
> +
> +static int real_is_systemd_timer_available(void)
> +{
> +	struct child_process child = CHILD_PROCESS_INIT;
> +
> +	strvec_pushl(&child.args, "systemctl", "--user", "list-timers", NULL);
> +	child.no_stdin = 1;
> +	child.no_stdout = 1;
> +	child.no_stderr = 1;
> +	child.silent_exec_failure = 1;
> +
> +	if (start_command(&child))
> +		return 0;
> +	if (finish_command(&child))
> +		return 0;
> +	return 1;
> +}
> +
> +#else
> +
> +static int real_is_systemd_timer_available(void)
> +{
> +	return 0;
> +}
> +
> +#endif
> +
> +static int is_systemd_timer_available(void)
> +{
> +	const char *cmd = "systemctl";
> +	int is_available;
> +
> +	if (get_schedule_cmd(&cmd, &is_available))
> +		return is_available;
> +
> +	return real_is_systemd_timer_available();
> +}
> +
> +static char *xdg_config_home_systemd(const char *filename)
> +{
> +	return xdg_config_home_for("systemd/user", filename);
> +}
> +
> +static int systemd_timer_enable_unit(int enable,
> +				     enum schedule_priority schedule)
> +{
> +	const char *cmd = "systemctl";
> +	struct child_process child = CHILD_PROCESS_INIT;
> +	const char *frequency = get_frequency(schedule);
> +
> +	/*
> +	 * Disabling the systemd unit while it is already disabled makes
> +	 * systemctl print an error.
> +	 * Let's ignore it since it means we already are in the expected state:
> +	 * the unit is disabled.
> +	 *
> +	 * On the other hand, enabling a systemd unit which is already enabled
> +	 * produces no error.
> +	 */
> +	if (!enable)
> +		child.no_stderr = 1;
> +
> +	get_schedule_cmd(&cmd, NULL);
> +	strvec_split(&child.args, cmd);
> +	strvec_pushl(&child.args, "--user", enable ? "enable" : "disable",
> +		     "--now", NULL);
> +	strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency);
> +
> +	if (start_command(&child))
> +		return error(_("failed to start systemctl"));
> +	if (finish_command(&child))
> +		/*
> +		 * Disabling an already disabled systemd unit makes
> +		 * systemctl fail.
> +		 * Let's ignore this failure.
> +		 *
> +		 * Enabling an enabled systemd unit doesn't fail.
> +		 */
> +		if (enable)
> +			return error(_("failed to run systemctl"));
> +	return 0;
> +}
> +
> +static int systemd_timer_delete_unit_templates(void)
> +{
> +	int ret = 0;
> +	char *filename = xdg_config_home_systemd("git-maintenance@.timer");
> +	if(unlink(filename) && !is_missing_file_error(errno))
> +		ret = error_errno(_("failed to delete '%s'"), filename);
> +	FREE_AND_NULL(filename);
> +
> +	filename = xdg_config_home_systemd("git-maintenance@.service");
> +	if(unlink(filename) && !is_missing_file_error(errno))
> +		ret = error_errno(_("failed to delete '%s'"), filename);
> +
> +	free(filename);
> +	return ret;
> +}
> +
> +static int systemd_timer_delete_units(void)
> +{
> +	return systemd_timer_enable_unit(0, SCHEDULE_HOURLY) ||
> +	       systemd_timer_enable_unit(0, SCHEDULE_DAILY) ||
> +	       systemd_timer_enable_unit(0, SCHEDULE_WEEKLY) ||
> +	       systemd_timer_delete_unit_templates();
> +}
> +
> +static int systemd_timer_write_unit_templates(const char *exec_path)
> +{
> +	char *filename;
> +	FILE *file;
> +	const char *unit;
> +
> +	filename = xdg_config_home_systemd("git-maintenance@.timer");
> +	if (safe_create_leading_directories(filename)) {
> +		error(_("failed to create directories for '%s'"), filename);
> +		goto error;
> +	}
> +	file = fopen_or_warn(filename, "w");
> +	if (file == NULL)
> +		goto error;
> +	FREE_AND_NULL(filename);
> +
> +	unit = "# This file was created and is maintained by Git.\n"
> +	       "# Any edits made in this file might be replaced in the future\n"
> +	       "# by a Git command.\n"
> +	       "\n"
> +	       "[Unit]\n"
> +	       "Description=Optimize Git repositories data\n"
> +	       "\n"
> +	       "[Timer]\n"
> +	       "OnCalendar=%i\n"
> +	       "Persistent=true\n"
> +	       "\n"
> +	       "[Install]\n"
> +	       "WantedBy=timers.target\n";
> +	if (fputs(unit, file) == EOF) {
> +		error(_("failed to write to '%s'"), filename);
> +		fclose(file);
> +		goto error;
> +	}
> +	if (fclose(file) == EOF) {
> +		error_errno(_("failed to flush '%s'"), filename);
> +		goto error;
> +	}
> +
> +	filename = xdg_config_home_systemd("git-maintenance@.service");
> +	file = fopen_or_warn(filename, "w");
> +	if (file == NULL)
> +		goto error;
> +	FREE_AND_NULL(filename);
> +
> +	unit = "# This file was created and is maintained by Git.\n"
> +	       "# Any edits made in this file might be replaced in the future\n"
> +	       "# by a Git command.\n"
> +	       "\n"
> +	       "[Unit]\n"
> +	       "Description=Optimize Git repositories data\n"
> +	       "\n"
> +	       "[Service]\n"
> +	       "Type=oneshot\n"
> +	       "ExecStart=\"%s/git\" --exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
> +	       "LockPersonality=yes\n"
> +	       "MemoryDenyWriteExecute=yes\n"
> +	       "NoNewPrivileges=yes\n"
> +	       "RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\n"
> +	       "RestrictNamespaces=yes\n"
> +	       "RestrictRealtime=yes\n"
> +	       "RestrictSUIDSGID=yes\n"
> +	       "SystemCallArchitectures=native\n"
> +	       "SystemCallFilter=@system-service\n";
> +	if (fprintf(file, unit, exec_path, exec_path) < 0) {
> +		error(_("failed to write to '%s'"), filename);
> +		fclose(file);
> +		goto error;
> +	}
> +	if (fclose(file) == EOF) {
> +		error_errno(_("failed to flush '%s'"), filename);
> +		goto error;
> +	}
> +	return 0;
> +
> +error:
> +	free(filename);
> +	systemd_timer_delete_unit_templates();

One small comment just to show I have read the patches - I think 
systemd_timer_delete_unit_templates() ends up being called twice if we 
fail to write one of the template files as systemd_timer_setup_units() 
calls it if this function fails. I don't think this matters and there is 
no need to change it - from a future maintenance perspective it is 
probably safer to leave it as is.

Best Wishes

Phillip

> +	return -1;
> +}
> +
> +static int systemd_timer_setup_units(void)
> +{
> +	const char *exec_path = git_exec_path();
> +
> +	int ret = systemd_timer_write_unit_templates(exec_path) ||
> +	          systemd_timer_enable_unit(1, SCHEDULE_HOURLY) ||
> +	          systemd_timer_enable_unit(1, SCHEDULE_DAILY) ||
> +	          systemd_timer_enable_unit(1, SCHEDULE_WEEKLY);
> +	if (ret)
> +		systemd_timer_delete_units();
> +	return ret;
> +}
> +
> +static int systemd_timer_update_schedule(int run_maintenance, int fd)
> +{
> +	if (run_maintenance)
> +		return systemd_timer_setup_units();
> +	else
> +		return systemd_timer_delete_units();
> +}
> +
>   enum scheduler {
>   	SCHEDULER_INVALID = -1,
>   	SCHEDULER_AUTO,
>   	SCHEDULER_CRON,
> +	SCHEDULER_SYSTEMD,
>   	SCHEDULER_LAUNCHCTL,
>   	SCHEDULER_SCHTASKS,
>   };
> @@ -2058,6 +2269,11 @@ static const struct {
>   		.is_available = is_crontab_available,
>   		.update_schedule = crontab_update_schedule,
>   	},
> +	[SCHEDULER_SYSTEMD] = {
> +		.name = "systemctl",
> +		.is_available = is_systemd_timer_available,
> +		.update_schedule = systemd_timer_update_schedule,
> +	},
>   	[SCHEDULER_LAUNCHCTL] = {
>   		.name = "launchctl",
>   		.is_available = is_launchctl_available,
> @@ -2078,6 +2294,9 @@ static enum scheduler parse_scheduler(const char *value)
>   		return SCHEDULER_AUTO;
>   	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
>   		return SCHEDULER_CRON;
> +	else if (!strcasecmp(value, "systemd") ||
> +		 !strcasecmp(value, "systemd-timer"))
> +		return SCHEDULER_SYSTEMD;
>   	else if (!strcasecmp(value, "launchctl"))
>   		return SCHEDULER_LAUNCHCTL;
>   	else if (!strcasecmp(value, "schtasks"))
> @@ -2116,6 +2335,15 @@ static void resolve_auto_scheduler(enum scheduler *scheduler)
>   	*scheduler = SCHEDULER_SCHTASKS;
>   	return;
>   
> +#elif defined(__linux__)
> +	if (is_systemd_timer_available())
> +		*scheduler = SCHEDULER_SYSTEMD;
> +	else if (is_crontab_available())
> +		*scheduler = SCHEDULER_CRON;
> +	else
> +		die(_("neither systemd timers nor crontab are available"));
> +	return;
> +
>   #else
>   	*scheduler = SCHEDULER_CRON;
>   	return;
> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
> index b36b7f5fb0..b289cae6b9 100755
> --- a/t/t7900-maintenance.sh
> +++ b/t/t7900-maintenance.sh
> @@ -20,6 +20,18 @@ test_xmllint () {
>   	fi
>   }
>   
> +test_lazy_prereq SYSTEMD_ANALYZE '
> +	systemd-analyze --help >out &&
> +	grep verify out
> +'
> +
> +test_systemd_analyze_verify () {
> +	if test_have_prereq SYSTEMD_ANALYZE
> +	then
> +		systemd-analyze verify "$@"
> +	fi
> +}
> +
>   test_expect_success 'help text' '
>   	test_expect_code 129 git maintenance -h 2>err &&
>   	test_i18ngrep "usage: git maintenance <subcommand>" err &&
> @@ -634,15 +646,56 @@ test_expect_success 'start and stop Windows maintenance' '
>   	test_cmp expect args
>   '
>   
> +test_expect_success 'start and stop Linux/systemd maintenance' '
> +	write_script print-args <<-\EOF &&
> +	printf "%s\n" "$*" >>args
> +	EOF
> +
> +	XDG_CONFIG_HOME="$PWD" &&
> +	export XDG_CONFIG_HOME &&
> +	rm -f args &&
> +	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance start --scheduler=systemd-timer &&
> +
> +	# start registers the repo
> +	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
> +
> +	test_systemd_analyze_verify "systemd/user/git-maintenance@.service" &&
> +
> +	printf -- "--user enable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
> +	test_cmp expect args &&
> +
> +	rm -f args &&
> +	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance stop &&
> +
> +	# stop does not unregister the repo
> +	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
> +
> +	test_path_is_missing "systemd/user/git-maintenance@.timer" &&
> +	test_path_is_missing "systemd/user/git-maintenance@.service" &&
> +
> +	printf -- "--user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
> +	test_cmp expect args
> +'
> +
>   test_expect_success 'start and stop when several schedulers are available' '
>   	write_script print-args <<-\EOF &&
>   	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
>   	EOF
>   
>   	rm -f args &&
> -	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
> -	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
> +	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=systemd-timer &&
> +	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
>   		hourly daily weekly >expect &&
> +	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
> +		hourly daily weekly >>expect &&
> +	printf -- "systemctl --user enable --now git-maintenance@%s.timer\n" hourly daily weekly >>expect &&
> +	test_cmp expect args &&
> +
> +	rm -f args &&
> +	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
> +	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
> +	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
> +		hourly daily weekly >>expect &&
>   	for frequency in hourly daily weekly
>   	do
>   		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
> @@ -652,17 +705,19 @@ test_expect_success 'start and stop when several schedulers are available' '
>   	test_cmp expect args &&
>   
>   	rm -f args &&
> -	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
> +	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
> +	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
>   	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
> -		hourly daily weekly >expect &&
> +		hourly daily weekly >>expect &&
>   	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
>   		hourly daily weekly >>expect &&
>   	test_cmp expect args &&
>   
>   	rm -f args &&
> -	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
> +	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
> +	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
>   	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
> -		hourly daily weekly >expect &&
> +		hourly daily weekly >>expect &&
>   	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
>   		hourly daily weekly >>expect &&
>   	test_cmp expect args
> 

^ permalink raw reply	[flat|nested] 138+ messages in thread

* [PATCH v6 0/3] maintenance: add support for systemd timers on Linux
  2021-06-08 13:39       ` [PATCH v5 0/3] " Lénaïc Huard
                           ` (4 preceding siblings ...)
  2021-06-09 14:54         ` Phillip Wood
@ 2021-06-12 16:50         ` Lénaïc Huard
  2021-06-12 16:50           ` [PATCH v6 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
                             ` (3 more replies)
  2021-08-17 17:22         ` [PATCH v5 0/3] " Derrick Stolee
  6 siblings, 4 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-06-12 16:50 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King,
	Lénaïc Huard

Hello,

Please find hereafter my fixed patchset to add support for systemd
timers on Linux for the `git maintenance start` command.

There are very few changes compared to the previous version.
The main change is the fix for the use-after-free bug that Jeff
pointed out.
Except from that, I’ve done some minor style fixes based on the `git
clang-format --diff …` recommendations.

The patches are:

* cache.h: Introduce a generic "xdg_config_home_for(…)" function

  This patch introduces a function to compute configuration files
  paths inside $XDG_CONFIG_HOME.

  This patch is unchanged compared to its previous version.

* maintenance: `git maintenance run` learned `--scheduler=<scheduler>`

  This patch adds a new parameter to the `git maintenance run` to let
  the user choose a scheduler.

  The only changes in this patch compared to its previous version are
  some code alignment changes that were suggested by `git clang-format
  --diff …`

* maintenance: add support for systemd timers on Linux

  This patch implements the support of systemd timers on top of
  crontab scheduler on Linux systems.

  The only change in this patch compared to its previous version is
  the fix of the use-after-free bug that Jeff pointed out.
  I’ve moved the `free(filename)` after the last usage of `filename`.
  I left the `free(filename)` below the `error` label as the
  `filename` will always be allocated at each `goto error`.

Best wishes,
Lénaïc.

Lénaïc Huard (3):
  cache.h: Introduce a generic "xdg_config_home_for(…)" function
  maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  maintenance: add support for systemd timers on Linux

 Documentation/git-maintenance.txt |  60 ++++
 builtin/gc.c                      | 562 ++++++++++++++++++++++++++----
 cache.h                           |   7 +
 path.c                            |  13 +-
 t/t7900-maintenance.sh            | 110 +++++-
 5 files changed, 674 insertions(+), 78 deletions(-)

-- 
2.32.0


^ permalink raw reply	[flat|nested] 138+ messages in thread

* [PATCH v6 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function
  2021-06-12 16:50         ` [PATCH v6 0/3] maintenance: " Lénaïc Huard
@ 2021-06-12 16:50           ` Lénaïc Huard
  2021-06-12 16:50           ` [PATCH v6 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
                             ` (2 subsequent siblings)
  3 siblings, 0 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-06-12 16:50 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King,
	Lénaïc Huard

Current implementation of `xdg_config_home(filename)` returns
`$XDG_CONFIG_HOME/git/$filename`, with the `git` subdirectory inserted
between the `XDG_CONFIG_HOME` environment variable and the parameter.

This patch introduces a `xdg_config_home_for(subdir, filename)` function
which is more generic. It only concatenates "$XDG_CONFIG_HOME", or
"$HOME/.config" if the former isn’t defined, with the parameters,
without adding `git` in between.

`xdg_config_home(filename)` is now implemented by calling
`xdg_config_home_for("git", filename)` but this new generic function can
be used to compute the configuration directory of other programs.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 cache.h |  7 +++++++
 path.c  | 13 ++++++++++---
 2 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/cache.h b/cache.h
index ba04ff8bd3..2a0fb3e4ba 100644
--- a/cache.h
+++ b/cache.h
@@ -1286,6 +1286,13 @@ int is_ntfs_dotmailmap(const char *name);
  */
 int looks_like_command_line_option(const char *str);
 
+/**
+ * Return a newly allocated string with the evaluation of
+ * "$XDG_CONFIG_HOME/$subdir/$filename" if $XDG_CONFIG_HOME is non-empty, otherwise
+ * "$HOME/.config/$subdir/$filename". Return NULL upon error.
+ */
+char *xdg_config_home_for(const char *subdir, const char *filename);
+
 /**
  * Return a newly allocated string with the evaluation of
  * "$XDG_CONFIG_HOME/git/$filename" if $XDG_CONFIG_HOME is non-empty, otherwise
diff --git a/path.c b/path.c
index 7bccd830e9..1b1de3be09 100644
--- a/path.c
+++ b/path.c
@@ -1503,21 +1503,28 @@ int looks_like_command_line_option(const char *str)
 	return str && str[0] == '-';
 }
 
-char *xdg_config_home(const char *filename)
+char *xdg_config_home_for(const char *subdir, const char *filename)
 {
 	const char *home, *config_home;
 
+	assert(subdir);
 	assert(filename);
 	config_home = getenv("XDG_CONFIG_HOME");
 	if (config_home && *config_home)
-		return mkpathdup("%s/git/%s", config_home, filename);
+		return mkpathdup("%s/%s/%s", config_home, subdir, filename);
 
 	home = getenv("HOME");
 	if (home)
-		return mkpathdup("%s/.config/git/%s", home, filename);
+		return mkpathdup("%s/.config/%s/%s", home, subdir, filename);
+
 	return NULL;
 }
 
+char *xdg_config_home(const char *filename)
+{
+	return xdg_config_home_for("git", filename);
+}
+
 char *xdg_cache_home(const char *filename)
 {
 	const char *home, *cache_home;
-- 
2.32.0


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* [PATCH v6 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-06-12 16:50         ` [PATCH v6 0/3] maintenance: " Lénaïc Huard
  2021-06-12 16:50           ` [PATCH v6 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
@ 2021-06-12 16:50           ` Lénaïc Huard
  2021-06-14  4:36             ` Eric Sunshine
  2021-06-12 16:50           ` [PATCH v6 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
  2021-07-02 14:25           ` [PATCH v7 0/3] " Lénaïc Huard
  3 siblings, 1 reply; 138+ messages in thread
From: Lénaïc Huard @ 2021-06-12 16:50 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King,
	Lénaïc Huard

Depending on the system, different schedulers can be used to schedule
the hourly, daily and weekly executions of `git maintenance run`:
* `launchctl` for MacOS,
* `schtasks` for Windows and
* `crontab` for everything else.

`git maintenance run` now has an option to let the end-user explicitly
choose which scheduler he wants to use:
`--scheduler=auto|crontab|launchctl|schtasks`.

When `git maintenance start --scheduler=XXX` is run, it not only
registers `git maintenance run` tasks in the scheduler XXX, it also
removes the `git maintenance run` tasks from all the other schedulers to
ensure we cannot have two schedulers launching concurrent identical
tasks.

The default value is `auto` which chooses a suitable scheduler for the
system.

`git maintenance stop` doesn't have any `--scheduler` parameter because
this command will try to remove the `git maintenance run` tasks from all
the available schedulers.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 Documentation/git-maintenance.txt |  11 +
 builtin/gc.c                      | 334 +++++++++++++++++++++++-------
 t/t7900-maintenance.sh            |  55 ++++-
 3 files changed, 325 insertions(+), 75 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 1e738ad398..07065ed4f3 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -179,6 +179,17 @@ OPTIONS
 	`maintenance.<task>.enabled` configured as `true` are considered.
 	See the 'TASKS' section for the list of accepted `<task>` values.
 
+--scheduler=auto|crontab|launchctl|schtasks::
+	When combined with the `start` subcommand, specify the scheduler
+	to use to run the hourly, daily and weekly executions of
+	`git maintenance run`.
+	The possible values for `<scheduler>` depend on the system: `crontab`
+	is available on POSIX systems, `launchctl` is available on
+	MacOS and `schtasks` is available on Windows.
+	By default or when `auto` is specified, a suitable scheduler for
+	the system is used. On MacOS, `launchctl` is used. On Windows,
+	`schtasks` is used. On all other systems, `crontab` is used.
+
 
 TROUBLESHOOTING
 ---------------
diff --git a/builtin/gc.c b/builtin/gc.c
index f05d2f0a1a..4a5ed7cb6f 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1529,6 +1529,59 @@ static const char *get_frequency(enum schedule_priority schedule)
 	}
 }
 
+static int get_schedule_cmd(const char **cmd, int *is_available)
+{
+	char *item;
+	char *testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
+
+	if (!testing)
+		return 0;
+
+	if (is_available)
+		*is_available = 0;
+
+	for (item = testing;;) {
+		char *sep;
+		char *end_item = strchr(item, ',');
+		if (end_item)
+			*end_item = '\0';
+
+		sep = strchr(item, ':');
+		if (!sep)
+			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
+		*sep = '\0';
+
+		if (!strcmp(*cmd, item)) {
+			*cmd = sep + 1;
+			if (is_available)
+				*is_available = 1;
+			UNLEAK(testing);
+			return 1;
+		}
+
+		if (!end_item)
+			break;
+		item = end_item + 1;
+	}
+
+	free(testing);
+	return 1;
+}
+
+static int is_launchctl_available(void)
+{
+	const char *cmd = "launchctl";
+	int is_available;
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+#ifdef __APPLE__
+	return 1;
+#else
+	return 0;
+#endif
+}
+
 static char *launchctl_service_name(const char *frequency)
 {
 	struct strbuf label = STRBUF_INIT;
@@ -1555,19 +1608,17 @@ static char *launchctl_get_uid(void)
 	return xstrfmt("gui/%d", getuid());
 }
 
-static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
+static int launchctl_boot_plist(int enable, const char *filename)
 {
+	const char *cmd = "launchctl";
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
 	char *uid = launchctl_get_uid();
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&child.args, cmd);
-	if (enable)
-		strvec_push(&child.args, "bootstrap");
-	else
-		strvec_push(&child.args, "bootout");
-	strvec_push(&child.args, uid);
-	strvec_push(&child.args, filename);
+	strvec_pushl(&child.args, enable ? "bootstrap" : "bootout", uid,
+		     filename, NULL);
 
 	child.no_stderr = 1;
 	child.no_stdout = 1;
@@ -1581,26 +1632,26 @@ static int launchctl_boot_plist(int enable, const char *filename, const char *cm
 	return result;
 }
 
-static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)
+static int launchctl_remove_plist(enum schedule_priority schedule)
 {
 	const char *frequency = get_frequency(schedule);
 	char *name = launchctl_service_name(frequency);
 	char *filename = launchctl_service_filename(name);
-	int result = launchctl_boot_plist(0, filename, cmd);
+	int result = launchctl_boot_plist(0, filename);
 	unlink(filename);
 	free(filename);
 	free(name);
 	return result;
 }
 
-static int launchctl_remove_plists(const char *cmd)
+static int launchctl_remove_plists(void)
 {
-	return launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||
-		launchctl_remove_plist(SCHEDULE_DAILY, cmd) ||
-		launchctl_remove_plist(SCHEDULE_WEEKLY, cmd);
+	return launchctl_remove_plist(SCHEDULE_HOURLY) ||
+	       launchctl_remove_plist(SCHEDULE_DAILY) ||
+	       launchctl_remove_plist(SCHEDULE_WEEKLY);
 }
 
-static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule)
 {
 	FILE *plist;
 	int i;
@@ -1669,8 +1720,8 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 	fclose(plist);
 
 	/* bootout might fail if not already running, so ignore */
-	launchctl_boot_plist(0, filename, cmd);
-	if (launchctl_boot_plist(1, filename, cmd))
+	launchctl_boot_plist(0, filename);
+	if (launchctl_boot_plist(1, filename))
 		die(_("failed to bootstrap service %s"), filename);
 
 	free(filename);
@@ -1678,21 +1729,35 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 	return 0;
 }
 
-static int launchctl_add_plists(const char *cmd)
+static int launchctl_add_plists(void)
 {
 	const char *exec_path = git_exec_path();
 
-	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||
-		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||
-		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
+	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY) ||
+	       launchctl_schedule_plist(exec_path, SCHEDULE_DAILY) ||
+	       launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY);
 }
 
-static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int launchctl_update_schedule(int run_maintenance, int fd)
 {
 	if (run_maintenance)
-		return launchctl_add_plists(cmd);
+		return launchctl_add_plists();
 	else
-		return launchctl_remove_plists(cmd);
+		return launchctl_remove_plists();
+}
+
+static int is_schtasks_available(void)
+{
+	const char *cmd = "schtasks";
+	int is_available;
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+#ifdef GIT_WINDOWS_NATIVE
+	return 1;
+#else
+	return 0;
+#endif
 }
 
 static char *schtasks_task_name(const char *frequency)
@@ -1702,13 +1767,15 @@ static char *schtasks_task_name(const char *frequency)
 	return strbuf_detach(&label, NULL);
 }
 
-static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)
+static int schtasks_remove_task(enum schedule_priority schedule)
 {
+	const char *cmd = "schtasks";
 	int result;
 	struct strvec args = STRVEC_INIT;
 	const char *frequency = get_frequency(schedule);
 	char *name = schtasks_task_name(frequency);
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&args, cmd);
 	strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
 
@@ -1719,15 +1786,16 @@ static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd
 	return result;
 }
 
-static int schtasks_remove_tasks(const char *cmd)
+static int schtasks_remove_tasks(void)
 {
-	return schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||
-		schtasks_remove_task(SCHEDULE_DAILY, cmd) ||
-		schtasks_remove_task(SCHEDULE_WEEKLY, cmd);
+	return schtasks_remove_task(SCHEDULE_HOURLY) ||
+	       schtasks_remove_task(SCHEDULE_DAILY) ||
+	       schtasks_remove_task(SCHEDULE_WEEKLY);
 }
 
-static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule)
 {
+	const char *cmd = "schtasks";
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
 	const char *xml;
@@ -1736,6 +1804,8 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
 	char *name = schtasks_task_name(frequency);
 	struct strbuf tfilename = STRBUF_INIT;
 
+	get_schedule_cmd(&cmd, NULL);
+
 	strbuf_addf(&tfilename, "%s/schedule_%s_XXXXXX",
 		    get_git_common_dir(), frequency);
 	tfile = xmks_tempfile(tfilename.buf);
@@ -1840,28 +1910,52 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
 	return result;
 }
 
-static int schtasks_schedule_tasks(const char *cmd)
+static int schtasks_schedule_tasks(void)
 {
 	const char *exec_path = git_exec_path();
 
-	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||
-		schtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||
-		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
+	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY) ||
+	       schtasks_schedule_task(exec_path, SCHEDULE_DAILY) ||
+	       schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY);
 }
 
-static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int schtasks_update_schedule(int run_maintenance, int fd)
 {
 	if (run_maintenance)
-		return schtasks_schedule_tasks(cmd);
+		return schtasks_schedule_tasks();
 	else
-		return schtasks_remove_tasks(cmd);
+		return schtasks_remove_tasks();
+}
+
+static int is_crontab_available(void)
+{
+	const char *cmd = "crontab";
+	int is_available;
+	struct child_process child = CHILD_PROCESS_INIT;
+
+	if (get_schedule_cmd(&cmd, &is_available) && !is_available)
+		return 0;
+
+	strvec_split(&child.args, cmd);
+	strvec_push(&child.args, "-l");
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child))
+		return 0;
+	/* Ignore exit code, as an empty crontab will return error. */
+	finish_command(&child);
+	return 1;
 }
 
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
-static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int crontab_update_schedule(int run_maintenance, int fd)
 {
+	const char *cmd = "crontab";
 	int result = 0;
 	int in_old_region = 0;
 	struct child_process crontab_list = CHILD_PROCESS_INIT;
@@ -1869,6 +1963,7 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 	FILE *cron_list, *cron_in;
 	struct strbuf line = STRBUF_INIT;
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&crontab_list.args, cmd);
 	strvec_push(&crontab_list.args, "-l");
 	crontab_list.in = -1;
@@ -1945,66 +2040,163 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 	return result;
 }
 
+enum scheduler {
+	SCHEDULER_INVALID = -1,
+	SCHEDULER_AUTO,
+	SCHEDULER_CRON,
+	SCHEDULER_LAUNCHCTL,
+	SCHEDULER_SCHTASKS,
+};
+
+static const struct {
+	const char *name;
+	int (*is_available)(void);
+	int (*update_schedule)(int run_maintenance, int fd);
+} scheduler_fn[] = {
+	[SCHEDULER_CRON] = {
+		.name = "crontab",
+		.is_available = is_crontab_available,
+		.update_schedule = crontab_update_schedule,
+	},
+	[SCHEDULER_LAUNCHCTL] = {
+		.name = "launchctl",
+		.is_available = is_launchctl_available,
+		.update_schedule = launchctl_update_schedule,
+	},
+	[SCHEDULER_SCHTASKS] = {
+		.name = "schtasks",
+		.is_available = is_schtasks_available,
+		.update_schedule = schtasks_update_schedule,
+	},
+};
+
+static enum scheduler parse_scheduler(const char *value)
+{
+	if (!value)
+		return SCHEDULER_INVALID;
+	else if (!strcasecmp(value, "auto"))
+		return SCHEDULER_AUTO;
+	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
+		return SCHEDULER_CRON;
+	else if (!strcasecmp(value, "launchctl"))
+		return SCHEDULER_LAUNCHCTL;
+	else if (!strcasecmp(value, "schtasks"))
+		return SCHEDULER_SCHTASKS;
+	else
+		return SCHEDULER_INVALID;
+}
+
+static int maintenance_opt_scheduler(const struct option *opt, const char *arg,
+				     int unset)
+{
+	enum scheduler *scheduler = opt->value;
+
+	BUG_ON_OPT_NEG(unset);
+
+	*scheduler = parse_scheduler(arg);
+	if (*scheduler == SCHEDULER_INVALID)
+		return error(_("unrecognized --scheduler argument '%s'"), arg);
+	return 0;
+}
+
+struct maintenance_start_opts {
+	enum scheduler scheduler;
+};
+
+static void resolve_auto_scheduler(enum scheduler *scheduler)
+{
+	if (*scheduler != SCHEDULER_AUTO)
+		return;
+
 #if defined(__APPLE__)
-static const char platform_scheduler[] = "launchctl";
+	*scheduler = SCHEDULER_LAUNCHCTL;
+	return;
+
 #elif defined(GIT_WINDOWS_NATIVE)
-static const char platform_scheduler[] = "schtasks";
+	*scheduler = SCHEDULER_SCHTASKS;
+	return;
+
 #else
-static const char platform_scheduler[] = "crontab";
+	*scheduler = SCHEDULER_CRON;
+	return;
 #endif
+}
 
-static int update_background_schedule(int enable)
+static void validate_scheduler(enum scheduler scheduler)
 {
-	int result;
-	const char *scheduler = platform_scheduler;
-	const char *cmd = scheduler;
-	char *testing;
+	if (scheduler == SCHEDULER_INVALID)
+		BUG("invalid scheduler");
+	if (scheduler == SCHEDULER_AUTO)
+		BUG("resolve_auto_scheduler should have been called before");
+
+	if (!scheduler_fn[scheduler].is_available())
+		die(_("%s scheduler is not available"),
+		    scheduler_fn[scheduler].name);
+}
+
+static int update_background_schedule(const struct maintenance_start_opts *opts,
+				      int enable)
+{
+	unsigned int i;
+	int result = 0;
 	struct lock_file lk;
 	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
 
-	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
-	if (testing) {
-		char *sep = strchr(testing, ':');
-		if (!sep)
-			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
-		*sep = '\0';
-		scheduler = testing;
-		cmd = sep + 1;
+	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {
+		free(lock_path);
+		return error(_("another process is scheduling background maintenance"));
 	}
 
-	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {
-		result = error(_("another process is scheduling background maintenance"));
-		goto cleanup;
+	for (i = 1; i < ARRAY_SIZE(scheduler_fn); i++) {
+		if (enable && opts->scheduler == i)
+			continue;
+		if (!scheduler_fn[i].is_available())
+			continue;
+		scheduler_fn[i].update_schedule(0, get_lock_file_fd(&lk));
 	}
 
-	if (!strcmp(scheduler, "launchctl"))
-		result = launchctl_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else if (!strcmp(scheduler, "schtasks"))
-		result = schtasks_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else if (!strcmp(scheduler, "crontab"))
-		result = crontab_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else
-		die("unknown background scheduler: %s", scheduler);
+	if (enable)
+		result = scheduler_fn[opts->scheduler].update_schedule(
+			1, get_lock_file_fd(&lk));
 
 	rollback_lock_file(&lk);
 
-cleanup:
 	free(lock_path);
-	free(testing);
 	return result;
 }
 
-static int maintenance_start(void)
+static const char *const builtin_maintenance_start_usage[] = {
+	N_("git maintenance start [--scheduler=<scheduler>]"),
+	NULL
+};
+
+static int maintenance_start(int argc, const char **argv, const char *prefix)
 {
+	struct maintenance_start_opts opts = { 0 };
+	struct option options[] = {
+		OPT_CALLBACK_F(
+			0, "scheduler", &opts.scheduler, N_("scheduler"),
+			N_("scheduler to use to trigger git maintenance run"),
+			PARSE_OPT_NONEG, maintenance_opt_scheduler),
+		OPT_END()
+	};
+
+	argc = parse_options(argc, argv, prefix, options,
+			     builtin_maintenance_start_usage, 0);
+	if (argc)
+		usage_with_options(builtin_maintenance_start_usage, options);
+
+	resolve_auto_scheduler(&opts.scheduler);
+	validate_scheduler(opts.scheduler);
+
 	if (maintenance_register())
 		warning(_("failed to add repo to global config"));
-
-	return update_background_schedule(1);
+	return update_background_schedule(&opts, 1);
 }
 
 static int maintenance_stop(void)
 {
-	return update_background_schedule(0);
+	return update_background_schedule(NULL, 0);
 }
 
 static const char builtin_maintenance_usage[] =	N_("git maintenance <subcommand> [<options>]");
@@ -2018,7 +2210,7 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
 	if (!strcmp(argv[1], "run"))
 		return maintenance_run(argc - 1, argv + 1, prefix);
 	if (!strcmp(argv[1], "start"))
-		return maintenance_start();
+		return maintenance_start(argc - 1, argv + 1, prefix);
 	if (!strcmp(argv[1], "stop"))
 		return maintenance_stop();
 	if (!strcmp(argv[1], "register"))
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index b93ae014ee..b36b7f5fb0 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -494,8 +494,21 @@ test_expect_success !MINGW 'register and unregister with regex metacharacters' '
 		maintenance.repo "$(pwd)/$META"
 '
 
+test_expect_success 'start --scheduler=<scheduler>' '
+	test_expect_code 129 git maintenance start --scheduler=foo 2>err &&
+	test_i18ngrep "unrecognized --scheduler argument" err &&
+
+	test_expect_code 129 git maintenance start --no-scheduler 2>err &&
+	test_i18ngrep "unknown option" err &&
+
+	test_expect_code 128 \
+		env GIT_TEST_MAINT_SCHEDULER="launchctl:true,schtasks:true" \
+		git maintenance start --scheduler=crontab 2>err &&
+	test_i18ngrep "fatal: crontab scheduler is not available" err
+'
+
 test_expect_success 'start from empty cron table' '
-	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -518,7 +531,7 @@ test_expect_success 'stop from existing schedule' '
 
 test_expect_success 'start preserves existing schedule' '
 	echo "Important information!" >cron.txt &&
-	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
 	grep "Important information!" cron.txt
 '
 
@@ -547,7 +560,7 @@ test_expect_success 'start and stop macOS maintenance' '
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start --scheduler=launchctl &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -598,7 +611,7 @@ test_expect_success 'start and stop Windows maintenance' '
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start --scheduler=schtasks &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -621,6 +634,40 @@ test_expect_success 'start and stop Windows maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success 'start and stop when several schedulers are available' '
+	write_script print-args <<-\EOF &&
+	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
+	EOF
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >expect &&
+	for frequency in hourly daily weekly
+	do
+		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
+		echo "launchctl bootout gui/[UID] $PLIST" >>expect &&
+		echo "launchctl bootstrap gui/[UID] $PLIST" >>expect || return 1
+	done &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
2.32.0


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* [PATCH v6 3/3] maintenance: add support for systemd timers on Linux
  2021-06-12 16:50         ` [PATCH v6 0/3] maintenance: " Lénaïc Huard
  2021-06-12 16:50           ` [PATCH v6 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
  2021-06-12 16:50           ` [PATCH v6 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
@ 2021-06-12 16:50           ` Lénaïc Huard
  2021-07-02 14:25           ` [PATCH v7 0/3] " Lénaïc Huard
  3 siblings, 0 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-06-12 16:50 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King,
	Lénaïc Huard

The existing mechanism for scheduling background maintenance is done
through cron. On Linux systems managed by systemd, systemd provides an
alternative to schedule recurring tasks: systemd timers.

The main motivations to implement systemd timers in addition to cron
are:
* cron is optional and Linux systems running systemd might not have it
  installed.
* The execution of `crontab -l` can tell us if cron is installed but not
  if the daemon is actually running.
* With systemd, each service is run in its own cgroup and its logs are
  tagged by the service inside journald. With cron, all scheduled tasks
  are running in the cron daemon cgroup and all the logs of the
  user-scheduled tasks are pretended to belong to the system cron
  service.
  Concretely, a user that doesn’t have access to the system logs won’t
  have access to the log of their own tasks scheduled by cron whereas
  they will have access to the log of their own tasks scheduled by
  systemd timer.
  Although `cron` attempts to send email, that email may go unseen by
  the user because these days, local mailboxes are not heavily used
  anymore.

In order to schedule git maintenance, we need two unit template files:
* ~/.config/systemd/user/git-maintenance@.service
  to define the command to be started by systemd and
* ~/.config/systemd/user/git-maintenance@.timer
  to define the schedule at which the command should be run.

Those units are templates that are parameterized by the frequency.

Based on those templates, 3 timers are started:
* git-maintenance@hourly.timer
* git-maintenance@daily.timer
* git-maintenance@weekly.timer

The command launched by those three timers are the same as with the
other scheduling methods:

/path/to/git for-each-repo --exec-path=/path/to
--config=maintenance.repo maintenance run --schedule=%i

with the full path for git to ensure that the version of git launched
for the scheduled maintenance is the same as the one used to run
`maintenance start`.

The timer unit contains `Persistent=true` so that, if the computer is
powered down when a maintenance task should run, the task will be run
when the computer is back powered on.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 Documentation/git-maintenance.txt |  57 +++++++-
 builtin/gc.c                      | 228 ++++++++++++++++++++++++++++++
 t/t7900-maintenance.sh            |  67 ++++++++-
 3 files changed, 342 insertions(+), 10 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 07065ed4f3..7b7dbbbca9 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -179,16 +179,19 @@ OPTIONS
 	`maintenance.<task>.enabled` configured as `true` are considered.
 	See the 'TASKS' section for the list of accepted `<task>` values.
 
---scheduler=auto|crontab|launchctl|schtasks::
+--scheduler=auto|crontab|systemd-timer|launchctl|schtasks::
 	When combined with the `start` subcommand, specify the scheduler
 	to use to run the hourly, daily and weekly executions of
 	`git maintenance run`.
 	The possible values for `<scheduler>` depend on the system: `crontab`
-	is available on POSIX systems, `launchctl` is available on
-	MacOS and `schtasks` is available on Windows.
+	is available on POSIX systems, `systemd-timer` is available on Linux
+	systems, `launchctl` is available on MacOS and `schtasks` is available
+	on Windows.
 	By default or when `auto` is specified, a suitable scheduler for
 	the system is used. On MacOS, `launchctl` is used. On Windows,
-	`schtasks` is used. On all other systems, `crontab` is used.
+	`schtasks` is used. On Linux, `systemd-timer` is used if user systemd
+	timers are available, otherwise, `crontab` is used. On all other systems,
+	`crontab` is used.
 
 
 TROUBLESHOOTING
@@ -288,6 +291,52 @@ schedule to ensure you are executing the correct binaries in your
 schedule.
 
 
+BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMS
+-----------------------------------------------
+
+While Linux supports `cron`, depending on the distribution, `cron` may
+be an optional package not necessarily installed. On modern Linux
+distributions, systemd timers are superseding it.
+
+If user systemd timers are available, they will be used as a replacement
+of `cron`.
+
+In this case, `git maintenance start` will create user systemd timer units
+and start the timers. The current list of user-scheduled tasks can be found
+by running `systemctl --user list-timers`. The timers written by `git
+maintenance start` are similar to this:
+
+-----------------------------------------------------------------------
+$ systemctl --user list-timers
+NEXT                         LEFT          LAST                         PASSED     UNIT                         ACTIVATES
+Thu 2021-04-29 19:00:00 CEST 42min left    Thu 2021-04-29 18:00:11 CEST 17min ago  git-maintenance@hourly.timer git-maintenance@hourly.service
+Fri 2021-04-30 00:00:00 CEST 5h 42min left Thu 2021-04-29 00:00:11 CEST 18h ago    git-maintenance@daily.timer  git-maintenance@daily.service
+Mon 2021-05-03 00:00:00 CEST 3 days left   Mon 2021-04-26 00:00:11 CEST 3 days ago git-maintenance@weekly.timer git-maintenance@weekly.service
+-----------------------------------------------------------------------
+
+One timer is registered for each `--schedule=<frequency>` option.
+
+The definition of the systemd units can be inspected in the following files:
+
+-----------------------------------------------------------------------
+~/.config/systemd/user/git-maintenance@.timer
+~/.config/systemd/user/git-maintenance@.service
+~/.config/systemd/user/timers.target.wants/git-maintenance@hourly.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@daily.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@weekly.timer
+-----------------------------------------------------------------------
+
+`git maintenance start` will overwrite these files and start the timer
+again with `systemctl --user`, so any customization should be done by
+creating a drop-in file, i.e. a `.conf` suffixed file in the
+`~/.config/systemd/user/git-maintenance@.service.d` directory.
+
+`git maintenance stop` will stop the user systemd timers and delete
+the above mentioned files.
+
+For more details, see `systemd.timer(5)`.
+
+
 BACKGROUND MAINTENANCE ON MACOS SYSTEMS
 ---------------------------------------
 
diff --git a/builtin/gc.c b/builtin/gc.c
index 4a5ed7cb6f..7fe0e03b0e 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -2040,10 +2040,221 @@ static int crontab_update_schedule(int run_maintenance, int fd)
 	return result;
 }
 
+#ifdef __linux__
+
+static int real_is_systemd_timer_available(void)
+{
+	struct child_process child = CHILD_PROCESS_INIT;
+
+	strvec_pushl(&child.args, "systemctl", "--user", "list-timers", NULL);
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child))
+		return 0;
+	if (finish_command(&child))
+		return 0;
+	return 1;
+}
+
+#else
+
+static int real_is_systemd_timer_available(void)
+{
+	return 0;
+}
+
+#endif
+
+static int is_systemd_timer_available(void)
+{
+	const char *cmd = "systemctl";
+	int is_available;
+
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+	return real_is_systemd_timer_available();
+}
+
+static char *xdg_config_home_systemd(const char *filename)
+{
+	return xdg_config_home_for("systemd/user", filename);
+}
+
+static int systemd_timer_enable_unit(int enable,
+				     enum schedule_priority schedule)
+{
+	const char *cmd = "systemctl";
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *frequency = get_frequency(schedule);
+
+	/*
+	 * Disabling the systemd unit while it is already disabled makes
+	 * systemctl print an error.
+	 * Let's ignore it since it means we already are in the expected state:
+	 * the unit is disabled.
+	 *
+	 * On the other hand, enabling a systemd unit which is already enabled
+	 * produces no error.
+	 */
+	if (!enable)
+		child.no_stderr = 1;
+
+	get_schedule_cmd(&cmd, NULL);
+	strvec_split(&child.args, cmd);
+	strvec_pushl(&child.args, "--user", enable ? "enable" : "disable",
+		     "--now", NULL);
+	strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency);
+
+	if (start_command(&child))
+		return error(_("failed to start systemctl"));
+	if (finish_command(&child))
+		/*
+		 * Disabling an already disabled systemd unit makes
+		 * systemctl fail.
+		 * Let's ignore this failure.
+		 *
+		 * Enabling an enabled systemd unit doesn't fail.
+		 */
+		if (enable)
+			return error(_("failed to run systemctl"));
+	return 0;
+}
+
+static int systemd_timer_delete_unit_templates(void)
+{
+	int ret = 0;
+	char *filename = xdg_config_home_systemd("git-maintenance@.timer");
+	if (unlink(filename) && !is_missing_file_error(errno))
+		ret = error_errno(_("failed to delete '%s'"), filename);
+	FREE_AND_NULL(filename);
+
+	filename = xdg_config_home_systemd("git-maintenance@.service");
+	if (unlink(filename) && !is_missing_file_error(errno))
+		ret = error_errno(_("failed to delete '%s'"), filename);
+
+	free(filename);
+	return ret;
+}
+
+static int systemd_timer_delete_units(void)
+{
+	return systemd_timer_enable_unit(0, SCHEDULE_HOURLY) ||
+	       systemd_timer_enable_unit(0, SCHEDULE_DAILY) ||
+	       systemd_timer_enable_unit(0, SCHEDULE_WEEKLY) ||
+	       systemd_timer_delete_unit_templates();
+}
+
+static int systemd_timer_write_unit_templates(const char *exec_path)
+{
+	char *filename;
+	FILE *file;
+	const char *unit;
+
+	filename = xdg_config_home_systemd("git-maintenance@.timer");
+	if (safe_create_leading_directories(filename)) {
+		error(_("failed to create directories for '%s'"), filename);
+		goto error;
+	}
+	file = fopen_or_warn(filename, "w");
+	if (file == NULL)
+		goto error;
+
+	unit = "# This file was created and is maintained by Git.\n"
+	       "# Any edits made in this file might be replaced in the future\n"
+	       "# by a Git command.\n"
+	       "\n"
+	       "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Timer]\n"
+	       "OnCalendar=%i\n"
+	       "Persistent=true\n"
+	       "\n"
+	       "[Install]\n"
+	       "WantedBy=timers.target\n";
+	if (fputs(unit, file) == EOF) {
+		error(_("failed to write to '%s'"), filename);
+		fclose(file);
+		goto error;
+	}
+	if (fclose(file) == EOF) {
+		error_errno(_("failed to flush '%s'"), filename);
+		goto error;
+	}
+	free(filename);
+
+	filename = xdg_config_home_systemd("git-maintenance@.service");
+	file = fopen_or_warn(filename, "w");
+	if (file == NULL)
+		goto error;
+
+	unit = "# This file was created and is maintained by Git.\n"
+	       "# Any edits made in this file might be replaced in the future\n"
+	       "# by a Git command.\n"
+	       "\n"
+	       "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Service]\n"
+	       "Type=oneshot\n"
+	       "ExecStart=\"%s/git\" --exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
+	       "LockPersonality=yes\n"
+	       "MemoryDenyWriteExecute=yes\n"
+	       "NoNewPrivileges=yes\n"
+	       "RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\n"
+	       "RestrictNamespaces=yes\n"
+	       "RestrictRealtime=yes\n"
+	       "RestrictSUIDSGID=yes\n"
+	       "SystemCallArchitectures=native\n"
+	       "SystemCallFilter=@system-service\n";
+	if (fprintf(file, unit, exec_path, exec_path) < 0) {
+		error(_("failed to write to '%s'"), filename);
+		fclose(file);
+		goto error;
+	}
+	if (fclose(file) == EOF) {
+		error_errno(_("failed to flush '%s'"), filename);
+		goto error;
+	}
+	free(filename);
+	return 0;
+
+error:
+	free(filename);
+	systemd_timer_delete_unit_templates();
+	return -1;
+}
+
+static int systemd_timer_setup_units(void)
+{
+	const char *exec_path = git_exec_path();
+
+	int ret = systemd_timer_write_unit_templates(exec_path) ||
+	          systemd_timer_enable_unit(1, SCHEDULE_HOURLY) ||
+	          systemd_timer_enable_unit(1, SCHEDULE_DAILY) ||
+	          systemd_timer_enable_unit(1, SCHEDULE_WEEKLY);
+	if (ret)
+		systemd_timer_delete_units();
+	return ret;
+}
+
+static int systemd_timer_update_schedule(int run_maintenance, int fd)
+{
+	if (run_maintenance)
+		return systemd_timer_setup_units();
+	else
+		return systemd_timer_delete_units();
+}
+
 enum scheduler {
 	SCHEDULER_INVALID = -1,
 	SCHEDULER_AUTO,
 	SCHEDULER_CRON,
+	SCHEDULER_SYSTEMD,
 	SCHEDULER_LAUNCHCTL,
 	SCHEDULER_SCHTASKS,
 };
@@ -2058,6 +2269,11 @@ static const struct {
 		.is_available = is_crontab_available,
 		.update_schedule = crontab_update_schedule,
 	},
+	[SCHEDULER_SYSTEMD] = {
+		.name = "systemctl",
+		.is_available = is_systemd_timer_available,
+		.update_schedule = systemd_timer_update_schedule,
+	},
 	[SCHEDULER_LAUNCHCTL] = {
 		.name = "launchctl",
 		.is_available = is_launchctl_available,
@@ -2078,6 +2294,9 @@ static enum scheduler parse_scheduler(const char *value)
 		return SCHEDULER_AUTO;
 	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
 		return SCHEDULER_CRON;
+	else if (!strcasecmp(value, "systemd") ||
+		 !strcasecmp(value, "systemd-timer"))
+		return SCHEDULER_SYSTEMD;
 	else if (!strcasecmp(value, "launchctl"))
 		return SCHEDULER_LAUNCHCTL;
 	else if (!strcasecmp(value, "schtasks"))
@@ -2116,6 +2335,15 @@ static void resolve_auto_scheduler(enum scheduler *scheduler)
 	*scheduler = SCHEDULER_SCHTASKS;
 	return;
 
+#elif defined(__linux__)
+	if (is_systemd_timer_available())
+		*scheduler = SCHEDULER_SYSTEMD;
+	else if (is_crontab_available())
+		*scheduler = SCHEDULER_CRON;
+	else
+		die(_("neither systemd timers nor crontab are available"));
+	return;
+
 #else
 	*scheduler = SCHEDULER_CRON;
 	return;
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index b36b7f5fb0..b289cae6b9 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -20,6 +20,18 @@ test_xmllint () {
 	fi
 }
 
+test_lazy_prereq SYSTEMD_ANALYZE '
+	systemd-analyze --help >out &&
+	grep verify out
+'
+
+test_systemd_analyze_verify () {
+	if test_have_prereq SYSTEMD_ANALYZE
+	then
+		systemd-analyze verify "$@"
+	fi
+}
+
 test_expect_success 'help text' '
 	test_expect_code 129 git maintenance -h 2>err &&
 	test_i18ngrep "usage: git maintenance <subcommand>" err &&
@@ -634,15 +646,56 @@ test_expect_success 'start and stop Windows maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success 'start and stop Linux/systemd maintenance' '
+	write_script print-args <<-\EOF &&
+	printf "%s\n" "$*" >>args
+	EOF
+
+	XDG_CONFIG_HOME="$PWD" &&
+	export XDG_CONFIG_HOME &&
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance start --scheduler=systemd-timer &&
+
+	# start registers the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_systemd_analyze_verify "systemd/user/git-maintenance@.service" &&
+
+	printf -- "--user enable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_path_is_missing "systemd/user/git-maintenance@.timer" &&
+	test_path_is_missing "systemd/user/git-maintenance@.service" &&
+
+	printf -- "--user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'start and stop when several schedulers are available' '
 	write_script print-args <<-\EOF &&
 	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
-	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=systemd-timer &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
 		hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
+	printf -- "systemctl --user enable --now git-maintenance@%s.timer\n" hourly daily weekly >>expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
 	for frequency in hourly daily weekly
 	do
 		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
@@ -652,17 +705,19 @@ test_expect_success 'start and stop when several schedulers are available' '
 	test_cmp expect args &&
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
 	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
-		hourly daily weekly >expect &&
+		hourly daily weekly >>expect &&
 	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
 		hourly daily weekly >>expect &&
 	test_cmp expect args &&
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
 	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
-		hourly daily weekly >expect &&
+		hourly daily weekly >>expect &&
 	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
 		hourly daily weekly >>expect &&
 	test_cmp expect args
-- 
2.32.0


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* Re: [PATCH v6 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-06-12 16:50           ` [PATCH v6 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
@ 2021-06-14  4:36             ` Eric Sunshine
  2021-06-16 18:12               ` Derrick Stolee
                                 ` (2 more replies)
  0 siblings, 3 replies; 138+ messages in thread
From: Eric Sunshine @ 2021-06-14  4:36 UTC (permalink / raw)
  To: Lénaïc Huard
  Cc: Git List, Junio C Hamano, Derrick Stolee,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King

On Sat, Jun 12, 2021 at 12:51 PM Lénaïc Huard <lenaic@lhuard.fr> wrote:
> Depending on the system, different schedulers can be used to schedule
> the hourly, daily and weekly executions of `git maintenance run`:
> * `launchctl` for MacOS,
> * `schtasks` for Windows and
> * `crontab` for everything else.
>
> `git maintenance run` now has an option to let the end-user explicitly
> choose which scheduler he wants to use:
> `--scheduler=auto|crontab|launchctl|schtasks`.
>
> When `git maintenance start --scheduler=XXX` is run, it not only
> registers `git maintenance run` tasks in the scheduler XXX, it also
> removes the `git maintenance run` tasks from all the other schedulers to
> ensure we cannot have two schedulers launching concurrent identical
> tasks.
>
> The default value is `auto` which chooses a suitable scheduler for the
> system.
>
> `git maintenance stop` doesn't have any `--scheduler` parameter because
> this command will try to remove the `git maintenance run` tasks from all
> the available schedulers.
>
> Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>

Thanks. Unfortunately, I haven't been following this series too
closely since I reviewed v1, so I set aside time to review v6, which I
have now done. The material in the cover letter and individual commit
messages was helpful in understanding the nuances of the changes, and
the series seems pretty well complete at this point. (If, however, you
do happen to re-roll for some reason, please consider using the
--range-diff option of git-format-patch as an aid to reviewers.)

I did leave a number of comments below regarding possible improvements
to the code and documentation, however, they're probably mostly
subjective and don't necessarily warrant a re-roll; I'd have no
problem seeing this accepted as-is without the suggestions applied.
(They can always be applied later on if someone considers them
important enough.)

I do, though, have one question (below) about is_crontab_available()
for which I could not figure out the answer.

> ---
> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
> @@ -179,6 +179,17 @@ OPTIONS
> +--scheduler=auto|crontab|launchctl|schtasks::
> +       When combined with the `start` subcommand, specify the scheduler
> +       to use to run the hourly, daily and weekly executions of
> +       `git maintenance run`.
> +       The possible values for `<scheduler>` depend on the system: `crontab`
> +       is available on POSIX systems, `launchctl` is available on
> +       MacOS and `schtasks` is available on Windows.
> +       By default or when `auto` is specified, a suitable scheduler for
> +       the system is used. On MacOS, `launchctl` is used. On Windows,
> +       `schtasks` is used. On all other systems, `crontab` is used.

The above description is somewhat redundant. Another way to write it
without the redundancy might be:

    Specify the scheduler -- in combination with subcommand `start` --
    for running the hourly, daily and weekly invocations of `git
    maintenance run`. Possible values for `<scheduler>` are `auto`,
    `crontab` (POSIX), `launchctl` (macOS), and `schtasks` (Windows).
    When `auto` is specified, the appropriate platform-specific
    scheduler is used. Default is `auto`.

> diff --git a/builtin/gc.c b/builtin/gc.c
> @@ -1529,6 +1529,59 @@ static const char *get_frequency(enum schedule_priority schedule)
> +static int get_schedule_cmd(const char **cmd, int *is_available)
> +{
> +       char *item;
> +       char *testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
> +
> +       if (!testing)
> +               return 0;
> +
> +       if (is_available)
> +               *is_available = 0;
> +
> +       for (item = testing;;) {
> +               char *sep;
> +               char *end_item = strchr(item, ',');
> +               if (end_item)
> +                       *end_item = '\0';
> +
> +               sep = strchr(item, ':');
> +               if (!sep)
> +                       die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
> +               *sep = '\0';
> +
> +               if (!strcmp(*cmd, item)) {
> +                       *cmd = sep + 1;
> +                       if (is_available)
> +                               *is_available = 1;
> +                       UNLEAK(testing);
> +                       return 1;
> +               }
> +
> +               if (!end_item)
> +                       break;
> +               item = end_item + 1;
> +       }
> +
> +       free(testing);
> +       return 1;
> +}

I ended up studying this implementation several times since I had to
come back to it repeatedly after reading calling code in order to (I
hope) fully understand all the different conditions represented by its
three distinct return values (the function return value, and the
values returned in **cmd and **is_available). That it required several
readings might warrant a comment block explaining what the function
does and what the various return conditions mean. As a bonus, an
explanation of the value of GIT_TEST_MAINT_SCHEDULER -- a
comma-separated list of colon-delimited tuples, and what those tuples
represent -- could be helpful.

> +static int is_launchctl_available(void)
> +{
> +       const char *cmd = "launchctl";
> +       int is_available;
> +       if (get_schedule_cmd(&cmd, &is_available))
> +               return is_available;
> +
> +#ifdef __APPLE__
> +       return 1;
> +#else
> +       return 0;
> +#endif
> +}

On this project, we usually frown upon #if conditionals within
functions since the code often can become unreadable. The usage in
this function doesn't suffer from that problem, however,
resolve_auto_scheduler() is somewhat ugly. An alternative would be to
set up these values outside of all functions, perhaps like this:

    #ifdef __APPLE__
    #define MAINT_SCHEDULER SCHEDULER_LAUNCHCTL
    #elif GIT_WINDOWS_NATIVE
    #define MAINT_SCHEDULER SCHEDULER_SCHTASKS
    #else
    #define MAINT_SCHEDULER SCHEDULER_CRON
    #endif

and then:

    static int is_launchctl_available(void)
    {
        if (get_schedule_cmd(...))
            return is_available;
        return MAINT_SCHEDULER == SCHEDULER_LAUNCHCTL;
    }

    static void resolve_auto_scheduler(enum scheduler *scheduler)
    {
        if (*scheduler == SCHEDULER_AUTO)
            *scheduler = MAINT_SCHEDULER;
    }

> +static int is_crontab_available(void)
> +{
> +       const char *cmd = "crontab";
> +       int is_available;
> +       struct child_process child = CHILD_PROCESS_INIT;
> +
> +       if (get_schedule_cmd(&cmd, &is_available) && !is_available)
> +               return 0;
> +
> +       strvec_split(&child.args, cmd);
> +       strvec_push(&child.args, "-l");
> +       child.no_stdin = 1;
> +       child.no_stdout = 1;
> +       child.no_stderr = 1;
> +       child.silent_exec_failure = 1;
> +
> +       if (start_command(&child))
> +               return 0;
> +       /* Ignore exit code, as an empty crontab will return error. */
> +       finish_command(&child);
> +       return 1;
>  }

If I understand get_schedule_cmd() correctly, it will always return
true if GIT_TEST_MAINT_SCHEDULER is present in the environment,
however, it will only set `is_available` to true if
GIT_TEST_MAINT_SCHEDULER contains a matching entry for `cmd` (which in
this case is "crontab"). Assuming this understanding is correct, then
I'm having trouble understanding why this:

    if (get_schedule_cmd(&cmd, &is_available) && !is_available)
        return 0;

isn't instead written like this:

    if (get_schedule_cmd(&cmd, &is_available))
        return is_available;

That is, why doesn't is_crontab_available() trust the result of
get_schedule_cmd(), instead going ahead and trying to invoke `crontab`
itself? Am I missing something which makes the `!is_available` case
special?

> +static void resolve_auto_scheduler(enum scheduler *scheduler)
> +{
> +       if (*scheduler != SCHEDULER_AUTO)
> +               return;
> +
>  #if defined(__APPLE__)
> +       *scheduler = SCHEDULER_LAUNCHCTL;
> +       return;
> +
>  #elif defined(GIT_WINDOWS_NATIVE)
> +       *scheduler = SCHEDULER_SCHTASKS;
> +       return;
> +
>  #else
> +       *scheduler = SCHEDULER_CRON;
> +       return;
>  #endif
> +}

(See above for a way to simplify this implementation.)

Is there a strong reason which I'm missing that this function alters
its argument rather than simply returning the resolved scheduler?

    static enum scheduler resolve_scheduler(enum scheduler x) {...}

Or is it just personal preference?

(Minor: I took the liberty of shortening the function name since it
doesn't feel like the longer name adds much value.)

> +static int maintenance_start(int argc, const char **argv, const char *prefix)
>  {
> +       struct maintenance_start_opts opts = { 0 };
> +       struct option options[] = {
> +               OPT_CALLBACK_F(
> +                       0, "scheduler", &opts.scheduler, N_("scheduler"),
> +                       N_("scheduler to use to trigger git maintenance run"),

Dropping "to use" would make this more concise without losing clarity:

    "scheduler to trigger git maintenance run"

> +                       PARSE_OPT_NONEG, maintenance_opt_scheduler),
> +               OPT_END()
> +       };
> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
> @@ -494,8 +494,21 @@ test_expect_success !MINGW 'register and unregister with regex metacharacters' '
> +test_expect_success 'start --scheduler=<scheduler>' '
> +       test_expect_code 129 git maintenance start --scheduler=foo 2>err &&
> +       test_i18ngrep "unrecognized --scheduler argument" err &&
> +
> +       test_expect_code 129 git maintenance start --no-scheduler 2>err &&
> +       test_i18ngrep "unknown option" err &&
> +
> +       test_expect_code 128 \
> +               env GIT_TEST_MAINT_SCHEDULER="launchctl:true,schtasks:true" \
> +               git maintenance start --scheduler=crontab 2>err &&
> +       test_i18ngrep "fatal: crontab scheduler is not available" err
> +'

Why does this test care about the exact exit codes rather than simply
using test_must_fail() as is typically done elsewhere in the test
suite, especially since we're also checking the error message itself?
Am I missing some non-obvious property of the error codes?

I don't see `auto` being tested anywhere. Do we want such a test? (It
seems like it should be doable, though perhaps the complexity is too
high -- I haven't thought it through fully.)

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v6 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-06-14  4:36             ` Eric Sunshine
@ 2021-06-16 18:12               ` Derrick Stolee
  2021-06-17  4:11                 ` Eric Sunshine
  2021-06-17 14:26               ` Phillip Wood
  2021-07-02 15:04               ` Lénaïc Huard
  2 siblings, 1 reply; 138+ messages in thread
From: Derrick Stolee @ 2021-06-16 18:12 UTC (permalink / raw)
  To: Eric Sunshine, Lénaïc Huard
  Cc: Git List, Junio C Hamano, Derrick Stolee,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King

On 6/14/2021 12:36 AM, Eric Sunshine wrote:
> On Sat, Jun 12, 2021 at 12:51 PM Lénaïc Huard <lenaic@lhuard.fr> wrote:
...
>> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
>> @@ -179,6 +179,17 @@ OPTIONS
>> +--scheduler=auto|crontab|launchctl|schtasks::
>> +       When combined with the `start` subcommand, specify the scheduler
>> +       to use to run the hourly, daily and weekly executions of
>> +       `git maintenance run`.
>> +       The possible values for `<scheduler>` depend on the system: `crontab`
>> +       is available on POSIX systems, `launchctl` is available on
>> +       MacOS and `schtasks` is available on Windows.
>> +       By default or when `auto` is specified, a suitable scheduler for
>> +       the system is used. On MacOS, `launchctl` is used. On Windows,
>> +       `schtasks` is used. On all other systems, `crontab` is used.
> 
> The above description is somewhat redundant. Another way to write it
> without the redundancy might be:
> 
>     Specify the scheduler -- in combination with subcommand `start` --

I think this change to the start is not contributing to the drop
in redundancy, but _does_ break from the pattern of previous
options. These start with "When combined with the `X` subcommand"
to clarify when they can be used.

I'm not against improving that pattern. I'm just saying that we
should do it across the whole file if we do it at all.

>     for running the hourly, daily and weekly invocations of `git
>     maintenance run`. Possible values for `<scheduler>` are `auto`,
>     `crontab` (POSIX), `launchctl` (macOS), and `schtasks` (Windows).
>     When `auto` is specified, the appropriate platform-specific
>     scheduler is used. Default is `auto`.

I find this to be a good way to reorganize the paragraph to be
very clear. How do you propose adding `systemd-timer` in the next
patch? Is it simply adding "`systemd-timer (Linux)`" or do we
need to be more careful using "(when available)"? Unlike the others,
the availability of that option is not as cut-and-dry.

...

> I don't see `auto` being tested anywhere. Do we want such a test? (It
> seems like it should be doable, though perhaps the complexity is too
> high -- I haven't thought it through fully.)

Unfortunately, `auto` seems to live in a world where we need to
actually run commands such as crontab and systemctl to determine
if they are available, but that happens only when
GIT_TEST_MAINT_SCHEDULER is unset. But, we don't want to actually
edit the timing information for the test runner, so some other
abstraction needs to be inserted at the proper layer.

It's worth a shot, but I expect it to be challenging to get right.

Thanks,
-Stolee

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v6 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-06-16 18:12               ` Derrick Stolee
@ 2021-06-17  4:11                 ` Eric Sunshine
  0 siblings, 0 replies; 138+ messages in thread
From: Eric Sunshine @ 2021-06-17  4:11 UTC (permalink / raw)
  To: Derrick Stolee
  Cc: Lénaïc Huard, Git List, Junio C Hamano, Derrick Stolee,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King

On Wed, Jun 16, 2021 at 8:48 PM Derrick Stolee <stolee@gmail.com> wrote:
> On 6/14/2021 12:36 AM, Eric Sunshine wrote:
> > On Sat, Jun 12, 2021 at 12:51 PM Lénaïc Huard <lenaic@lhuard.fr> wrote:
> > The above description is somewhat redundant. Another way to write it
> > without the redundancy might be:
> >
> >     Specify the scheduler -- in combination with subcommand `start` --
>
> I think this change to the start is not contributing to the drop
> in redundancy, but _does_ break from the pattern of previous
> options. These start with "When combined with the `X` subcommand"
> to clarify when they can be used.
>
> I'm not against improving that pattern. I'm just saying that we
> should do it across the whole file if we do it at all.

Indeed, it's not a big deal. As mentioned in my review, all my
comments were subjective, and none of them should prevent this from
being accepted as-is if everyone is happy with it.

> >     for running the hourly, daily and weekly invocations of `git
> >     maintenance run`. Possible values for `<scheduler>` are `auto`,
> >     `crontab` (POSIX), `launchctl` (macOS), and `schtasks` (Windows).
> >     When `auto` is specified, the appropriate platform-specific
> >     scheduler is used. Default is `auto`.
>
> I find this to be a good way to reorganize the paragraph to be
> very clear. How do you propose adding `systemd-timer` in the next
> patch? Is it simply adding "`systemd-timer (Linux)`" or do we
> need to be more careful using "(when available)"? Unlike the others,
> the availability of that option is not as cut-and-dry.

It should be easy enough. For instance:

    Possible values for `<scheduler>` are `auto`, `crontab` (POSIX),
    `systemd-timer` (Linux), `launchctl` (macOS), and `schtasks`
    (Windows). When `auto` is specified, the appropriate platform-
    specific scheduler is used; on Linux, `systemd-timer` is used if
    available, otherwise `crontab`. Default is `auto`.

But this sort of minor rewrite can always be done later.

> > I don't see `auto` being tested anywhere. Do we want such a test? (It
> > seems like it should be doable, though perhaps the complexity is too
> > high -- I haven't thought it through fully.)
>
> Unfortunately, `auto` seems to live in a world where we need to
> actually run commands such as crontab and systemctl to determine
> if they are available, but that happens only when
> GIT_TEST_MAINT_SCHEDULER is unset. But, we don't want to actually
> edit the timing information for the test runner, so some other
> abstraction needs to be inserted at the proper layer.
>
> It's worth a shot, but I expect it to be challenging to get right.

I imagine that it can be mocked up with GIT_TEST_MAINT_SCHEDULER in
some fashion, but it's not worth holding up this series or expecting a
re-roll for that one little question which popped out of my brain.

As I mentioned in my review, my comments were subjective, and I think
the series is in a state in which it can be accepted as-is without
additional re-rolls[1] if everyone is happy with it.

[1]: Except for the one question I asked about is_crontab_available();
I don't understand that one bit of logic, thus can't tell if the
behavior makes sense or not.

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v6 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-06-14  4:36             ` Eric Sunshine
  2021-06-16 18:12               ` Derrick Stolee
@ 2021-06-17 14:26               ` Phillip Wood
  2021-07-02 15:04               ` Lénaïc Huard
  2 siblings, 0 replies; 138+ messages in thread
From: Phillip Wood @ 2021-06-17 14:26 UTC (permalink / raw)
  To: Eric Sunshine, Lénaïc Huard
  Cc: Git List, Junio C Hamano, Derrick Stolee,
	Đoàn Trần Công Danh, Felipe Contreras,
	Martin Ågren, Ævar Arnfjörð Bjarmason,
	Bagas Sanjaya, brian m . carlson, Johannes Schindelin, Jeff King

On 14/06/2021 05:36, Eric Sunshine wrote:
> On Sat, Jun 12, 2021 at 12:51 PM Lénaïc Huard <lenaic@lhuard.fr> wrote:
>> Depending on the system, different schedulers can be used to schedule
>> the hourly, daily and weekly executions of `git maintenance run`:
>> * `launchctl` for MacOS,
>> * `schtasks` for Windows and
>> * `crontab` for everything else.
>>
>> `git maintenance run` now has an option to let the end-user explicitly
>> choose which scheduler he wants to use:
>> `--scheduler=auto|crontab|launchctl|schtasks`.
>>
>> When `git maintenance start --scheduler=XXX` is run, it not only
>> registers `git maintenance run` tasks in the scheduler XXX, it also
>> removes the `git maintenance run` tasks from all the other schedulers to
>> ensure we cannot have two schedulers launching concurrent identical
>> tasks.
>>
>> The default value is `auto` which chooses a suitable scheduler for the
>> system.
>>
>> `git maintenance stop` doesn't have any `--scheduler` parameter because
>> this command will try to remove the `git maintenance run` tasks from all
>> the available schedulers.
>>
>> Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
> 
> Thanks. Unfortunately, I haven't been following this series too
> closely since I reviewed v1, so I set aside time to review v6, which I
> have now done. The material in the cover letter and individual commit
> messages was helpful in understanding the nuances of the changes, and
> the series seems pretty well complete at this point. (If, however, you
> do happen to re-roll for some reason, please consider using the
> --range-diff option of git-format-patch as an aid to reviewers.)
> 
> I did leave a number of comments below regarding possible improvements
> to the code and documentation, however, they're probably mostly
> subjective and don't necessarily warrant a re-roll; I'd have no
> problem seeing this accepted as-is without the suggestions applied.
> (They can always be applied later on if someone considers them
> important enough.)
>
> I do, though, have one question (below) about is_crontab_available()
> for which I could not figure out the answer.

I think that is a bug

>> [...]
>> diff --git a/builtin/gc.c b/builtin/gc.c
>> @@ -1529,6 +1529,59 @@ static const char *get_frequency(enum schedule_priority schedule)
>> +static int get_schedule_cmd(const char **cmd, int *is_available)
>> +{
>> +       char *item;
>> +       char *testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
>> +
>> +       if (!testing)
>> +               return 0;
>> +
>> +       if (is_available)
>> +               *is_available = 0;
>> +
>> +       for (item = testing;;) {
>> +               char *sep;
>> +               char *end_item = strchr(item, ',');
>> +               if (end_item)
>> +                       *end_item = '\0';
>> +
>> +               sep = strchr(item, ':');
>> +               if (!sep)
>> +                       die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
>> +               *sep = '\0';
>> +
>> +               if (!strcmp(*cmd, item)) {
>> +                       *cmd = sep + 1;
>> +                       if (is_available)
>> +                               *is_available = 1;
>> +                       UNLEAK(testing);
>> +                       return 1;
>> +               }
>> +
>> +               if (!end_item)
>> +                       break;
>> +               item = end_item + 1;
>> +       }
>> +
>> +       free(testing);
>> +       return 1;
>> +}
> 
> I ended up studying this implementation several times since I had to
> come back to it repeatedly after reading calling code in order to (I
> hope) fully understand all the different conditions represented by its
> three distinct return values (the function return value, and the
> values returned in **cmd and **is_available). That it required several
> readings might warrant a comment block explaining what the function
> does and what the various return conditions mean. As a bonus, an
> explanation of the value of GIT_TEST_MAINT_SCHEDULER -- a
> comma-separated list of colon-delimited tuples, and what those tuples
> represent -- could be helpful.

I agree documenting GIT_TEST_MAINT_SCHEDULER would be useful

>> +static int is_launchctl_available(void)
>> +{
>> +       const char *cmd = "launchctl";
>> +       int is_available;
>> +       if (get_schedule_cmd(&cmd, &is_available))
>> +               return is_available;
>> +
>> +#ifdef __APPLE__
>> +       return 1;
>> +#else
>> +       return 0;
>> +#endif
>> +}
> 
> On this project, we usually frown upon #if conditionals within
> functions since the code often can become unreadable. The usage in
> this function doesn't suffer from that problem, however,
> resolve_auto_scheduler() is somewhat ugly. An alternative would be to
> set up these values outside of all functions, perhaps like this:
> 
>      #ifdef __APPLE__
>      #define MAINT_SCHEDULER SCHEDULER_LAUNCHCTL
>      #elif GIT_WINDOWS_NATIVE
>      #define MAINT_SCHEDULER SCHEDULER_SCHTASKS
>      #else
>      #define MAINT_SCHEDULER SCHEDULER_CRON
>      #endif
> 
> and then:
> 
>      static int is_launchctl_available(void)
>      {
>          if (get_schedule_cmd(...))
>              return is_available;
>          return MAINT_SCHEDULER == SCHEDULER_LAUNCHCTL;
>      }
> 
>      static void resolve_auto_scheduler(enum scheduler *scheduler)
>      {
>          if (*scheduler == SCHEDULER_AUTO)
>              *scheduler = MAINT_SCHEDULER;
>      }
> 
>> +static int is_crontab_available(void)
>> +{
>> +       const char *cmd = "crontab";
>> +       int is_available;
>> +       struct child_process child = CHILD_PROCESS_INIT;
>> +
>> +       if (get_schedule_cmd(&cmd, &is_available) && !is_available)
>> +               return 0;
>> +
>> +       strvec_split(&child.args, cmd);
>> +       strvec_push(&child.args, "-l");
>> +       child.no_stdin = 1;
>> +       child.no_stdout = 1;
>> +       child.no_stderr = 1;
>> +       child.silent_exec_failure = 1;
>> +
>> +       if (start_command(&child))
>> +               return 0;
>> +       /* Ignore exit code, as an empty crontab will return error. */
>> +       finish_command(&child);
>> +       return 1;
>>   }
> 
> If I understand get_schedule_cmd() correctly, it will always return
> true if GIT_TEST_MAINT_SCHEDULER is present in the environment,
> however, it will only set `is_available` to true if
> GIT_TEST_MAINT_SCHEDULER contains a matching entry for `cmd` (which in
> this case is "crontab"). Assuming this understanding is correct, then
> I'm having trouble understanding why this:
> 
>      if (get_schedule_cmd(&cmd, &is_available) && !is_available)
>          return 0;
> 
> isn't instead written like this:
> 
>      if (get_schedule_cmd(&cmd, &is_available))
>          return is_available;
> 
> That is, why doesn't is_crontab_available() trust the result of
> get_schedule_cmd(), instead going ahead and trying to invoke `crontab`
> itself? Am I missing something which makes the `!is_available` case
> special?

I agree, I think we should be returning is_available irrespective of its 
value if get_schedule_cmd() returns true. This is what 
is_systemd_timer_available() does in the next patch. Well spotted.

Best Wishes

Phillip

>> +static void resolve_auto_scheduler(enum scheduler *scheduler)
>> +{
>> +       if (*scheduler != SCHEDULER_AUTO)
>> +               return;
>> +
>>   #if defined(__APPLE__)
>> +       *scheduler = SCHEDULER_LAUNCHCTL;
>> +       return;
>> +
>>   #elif defined(GIT_WINDOWS_NATIVE)
>> +       *scheduler = SCHEDULER_SCHTASKS;
>> +       return;
>> +
>>   #else
>> +       *scheduler = SCHEDULER_CRON;
>> +       return;
>>   #endif
>> +}
> 
> (See above for a way to simplify this implementation.)
> 
> Is there a strong reason which I'm missing that this function alters
> its argument rather than simply returning the resolved scheduler?
> 
>      static enum scheduler resolve_scheduler(enum scheduler x) {...}
> 
> Or is it just personal preference?
>
> (Minor: I took the liberty of shortening the function name since it
> doesn't feel like the longer name adds much value.)
> 
>> +static int maintenance_start(int argc, const char **argv, const char *prefix)
>>   {
>> +       struct maintenance_start_opts opts = { 0 };
>> +       struct option options[] = {
>> +               OPT_CALLBACK_F(
>> +                       0, "scheduler", &opts.scheduler, N_("scheduler"),
>> +                       N_("scheduler to use to trigger git maintenance run"),
> 
> Dropping "to use" would make this more concise without losing clarity:
> 
>      "scheduler to trigger git maintenance run"
> 
>> +                       PARSE_OPT_NONEG, maintenance_opt_scheduler),
>> +               OPT_END()
>> +       };
>> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
>> @@ -494,8 +494,21 @@ test_expect_success !MINGW 'register and unregister with regex metacharacters' '
>> +test_expect_success 'start --scheduler=<scheduler>' '
>> +       test_expect_code 129 git maintenance start --scheduler=foo 2>err &&
>> +       test_i18ngrep "unrecognized --scheduler argument" err &&
>> +
>> +       test_expect_code 129 git maintenance start --no-scheduler 2>err &&
>> +       test_i18ngrep "unknown option" err &&
>> +
>> +       test_expect_code 128 \
>> +               env GIT_TEST_MAINT_SCHEDULER="launchctl:true,schtasks:true" \
>> +               git maintenance start --scheduler=crontab 2>err &&
>> +       test_i18ngrep "fatal: crontab scheduler is not available" err
>> +'
> 
> Why does this test care about the exact exit codes rather than simply
> using test_must_fail() as is typically done elsewhere in the test
> suite, especially since we're also checking the error message itself?
> Am I missing some non-obvious property of the error codes?
> 
> I don't see `auto` being tested anywhere. Do we want such a test? (It
> seems like it should be doable, though perhaps the complexity is too
> high -- I haven't thought it through fully.)
> 

^ permalink raw reply	[flat|nested] 138+ messages in thread

* [PATCH v7 0/3] maintenance: add support for systemd timers on Linux
  2021-06-12 16:50         ` [PATCH v6 0/3] maintenance: " Lénaïc Huard
                             ` (2 preceding siblings ...)
  2021-06-12 16:50           ` [PATCH v6 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
@ 2021-07-02 14:25           ` Lénaïc Huard
  2021-07-02 14:25             ` [PATCH v7 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
                               ` (5 more replies)
  3 siblings, 6 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-07-02 14:25 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King,
	Lénaïc Huard

Hello,

Please find hereafter a new reroll of my patchset to add support for
systemd timers on Linux for the `git maintenance start` command.

The changes compared to the previous version address the remarks
raised during the previous review.

The patches are:

* cache.h: Introduce a generic "xdg_config_home_for(…)" function

  This patch introduces a function to compute configuration files
  paths inside $XDG_CONFIG_HOME.
  It is used in the latest patch of this series to compute systemd
  unit files location.

  This patch is unchanged compared to its previous version.

* maintenance: `git maintenance run` learned `--scheduler=<scheduler>`

  This patch adds a new parameter to the `git maintenance run` to let
  the user choose a scheduler.

  This patch contains the following changes compared to its previous
  version:

  * `is_crontab_available` was not returning directly `is_available`
    under tests condition (when `GIT_TEST_MAINT_SCHEDULER` is set).
    This was indeed a bug as it means `cron` could be invoked by tests
    whereas it should be mocked.
    This is now fixed and `is_crontab_available` now has exactly the
    same behavior as the `is_systemd_timer_available` function that is
    introduced in the last patch.

  * The `get_schedule_cmd` function that centralizes the testing logic
    is now prefixed by a comment block describing its behavior and the
    expected values for the `GIT_TEST_MAINT_SCHEDULER` environment
    variable.

  * The help message for the `--scheduler` option of the `git
    maintenance start` command has been reworded following Eric’s
    suggestion.
    I’ve however kept the “When combined with the `start` subcommand…”
    opening to keep consistency with the other options documented on
    the same page.

  * `resolve_auto_scheduler` function has been renamed
    `resolve_scheduler` and it is now returning a value instead of
    altering its parameter.

* maintenance: add support for systemd timers on Linux

  This patch implements the support of systemd timers on top of
  crontab scheduler on Linux systems.

  The changes in this patch are only followups of changes mentioned in
  the previous patch:
  * `resolve_scheduler` is now returning a value instead of altering
  its parameter.

Best wishes,
Lénaïc.


Lénaïc Huard (3):
  cache.h: Introduce a generic "xdg_config_home_for(…)" function
  maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  maintenance: add support for systemd timers on Linux

 Documentation/git-maintenance.txt |  57 +++
 builtin/gc.c                      | 597 ++++++++++++++++++++++++++----
 cache.h                           |   7 +
 path.c                            |  13 +-
 t/t7900-maintenance.sh            | 110 +++++-
 5 files changed, 706 insertions(+), 78 deletions(-)

Diff-intervalle contre v6 :
-:  ---------- > 1:  899b11ed5b cache.h: Introduce a generic "xdg_config_home_for(…)" function
1:  604627f347 ! 2:  f3e2f0256b maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
    @@ Documentation/git-maintenance.txt: OPTIONS
      
     +--scheduler=auto|crontab|launchctl|schtasks::
     +	When combined with the `start` subcommand, specify the scheduler
    -+	to use to run the hourly, daily and weekly executions of
    ++	for running the hourly, daily and weekly executions of
     +	`git maintenance run`.
    -+	The possible values for `<scheduler>` depend on the system: `crontab`
    -+	is available on POSIX systems, `launchctl` is available on
    -+	MacOS and `schtasks` is available on Windows.
    -+	By default or when `auto` is specified, a suitable scheduler for
    -+	the system is used. On MacOS, `launchctl` is used. On Windows,
    -+	`schtasks` is used. On all other systems, `crontab` is used.
    ++	Possible values for `<scheduler>` are `auto`, `crontab` (POSIX),
    ++	`launchctl` (macOS), and `schtasks` (Windows).
    ++	When `auto` is specified, the appropriate platform-specific
    ++	scheduler is used. Default is `auto`.
     +
      
      TROUBLESHOOTING
    @@ builtin/gc.c: static const char *get_frequency(enum schedule_priority schedule)
      	}
      }
      
    ++/*
    ++ * get_schedule_cmd` reads the GIT_TEST_MAINT_SCHEDULER environment variable
    ++ * to mock the schedulers that `git maintenance start` rely on.
    ++ *
    ++ * For test purpose, GIT_TEST_MAINT_SCHEDULER can be set to a comma-separated
    ++ * list of colon-separated key/value pairs where each pair contains a scheduler
    ++ * and its corresponding mock.
    ++ *
    ++ * * If $GET_TEST_MAINT_SCHEDULER is not set, return false and leave the
    ++ *   arguments unmodified.
    ++ *
    ++ * * If $GET_TEST_MAINT_SCHEDULER is set, return true.
    ++ *   In this case, the *cmd value is read as input.
    ++ *
    ++ *   * if the input value *cmd is the key of one of the comma-separated list
    ++ *     item, then *is_available is set to true and *cmd is modified and becomes
    ++ *     the mock command.
    ++ *
    ++ *   * if the input value *cmd isn’t the key of any of the comma-separated list
    ++ *     item, then *is_available is set to false.
    ++ *
    ++ * Ex.:
    ++ *   GIT_TEST_MAINT_SCHEDULER not set
    ++ *     ┏━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
    ++ *     ┃ Input ┃                     Output                      ┃
    ++ *     ┃ *cmd  ┃ return code │       *cmd        │ *is_available ┃
    ++ *     ┣━━━━━━━╋━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━┫
    ++ *     ┃ "foo" ┃    false    │ "foo" (unchanged) │  (unchanged)  ┃
    ++ *     ┗━━━━━━━┻━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━┛
    ++ *
    ++ *   GIT_TEST_MAINT_SCHEDULER set to “foo:./mock_foo.sh,bar:./mock_bar.sh”
    ++ *     ┏━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
    ++ *     ┃ Input ┃                     Output                      ┃
    ++ *     ┃ *cmd  ┃ return code │       *cmd        │ *is_available ┃
    ++ *     ┣━━━━━━━╋━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━┫
    ++ *     ┃ "foo" ┃    true     │  "./mock.foo.sh"  │     true      ┃
    ++ *     ┃ "qux" ┃    true     │ "qux" (unchanged) │     false     ┃
    ++ *     ┗━━━━━━━┻━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━┛
    ++ */
     +static int get_schedule_cmd(const char **cmd, int *is_available)
     +{
     +	char *item;
    @@ builtin/gc.c: static int schtasks_schedule_task(const char *exec_path, enum sche
     +	int is_available;
     +	struct child_process child = CHILD_PROCESS_INIT;
     +
    -+	if (get_schedule_cmd(&cmd, &is_available) && !is_available)
    -+		return 0;
    ++	if (get_schedule_cmd(&cmd, &is_available))
    ++		return is_available;
     +
     +	strvec_split(&child.args, cmd);
     +	strvec_push(&child.args, "-l");
    @@ builtin/gc.c: static int crontab_update_schedule(int run_maintenance, int fd, co
     +	enum scheduler scheduler;
     +};
     +
    -+static void resolve_auto_scheduler(enum scheduler *scheduler)
    ++static enum scheduler resolve_scheduler(enum scheduler scheduler)
     +{
    -+	if (*scheduler != SCHEDULER_AUTO)
    -+		return;
    ++	if (scheduler != SCHEDULER_AUTO)
    ++		return scheduler;
     +
      #if defined(__APPLE__)
     -static const char platform_scheduler[] = "launchctl";
    -+	*scheduler = SCHEDULER_LAUNCHCTL;
    -+	return;
    ++	return SCHEDULER_LAUNCHCTL;
     +
      #elif defined(GIT_WINDOWS_NATIVE)
     -static const char platform_scheduler[] = "schtasks";
    -+	*scheduler = SCHEDULER_SCHTASKS;
    -+	return;
    ++	return SCHEDULER_SCHTASKS;
     +
      #else
     -static const char platform_scheduler[] = "crontab";
    -+	*scheduler = SCHEDULER_CRON;
    -+	return;
    ++	return SCHEDULER_CRON;
      #endif
     +}
      
    @@ builtin/gc.c: static int crontab_update_schedule(int run_maintenance, int fd, co
     +	if (scheduler == SCHEDULER_INVALID)
     +		BUG("invalid scheduler");
     +	if (scheduler == SCHEDULER_AUTO)
    -+		BUG("resolve_auto_scheduler should have been called before");
    ++		BUG("resolve_scheduler should have been called before");
     +
     +	if (!scheduler_fn[scheduler].is_available())
     +		die(_("%s scheduler is not available"),
    @@ builtin/gc.c: static int crontab_update_schedule(int run_maintenance, int fd, co
     +	struct option options[] = {
     +		OPT_CALLBACK_F(
     +			0, "scheduler", &opts.scheduler, N_("scheduler"),
    -+			N_("scheduler to use to trigger git maintenance run"),
    ++			N_("scheduler to trigger git maintenance run"),
     +			PARSE_OPT_NONEG, maintenance_opt_scheduler),
     +		OPT_END()
     +	};
    @@ builtin/gc.c: static int crontab_update_schedule(int run_maintenance, int fd, co
     +	if (argc)
     +		usage_with_options(builtin_maintenance_start_usage, options);
     +
    -+	resolve_auto_scheduler(&opts.scheduler);
    ++	opts.scheduler = resolve_scheduler(opts.scheduler);
     +	validate_scheduler(opts.scheduler);
     +
      	if (maintenance_register())
2:  29628b5a92 ! 3:  0ea5b2fc45 maintenance: add support for systemd timers on Linux
    @@ Documentation/git-maintenance.txt: OPTIONS
     ---scheduler=auto|crontab|launchctl|schtasks::
     +--scheduler=auto|crontab|systemd-timer|launchctl|schtasks::
      	When combined with the `start` subcommand, specify the scheduler
    - 	to use to run the hourly, daily and weekly executions of
    + 	for running the hourly, daily and weekly executions of
      	`git maintenance run`.
    - 	The possible values for `<scheduler>` depend on the system: `crontab`
    --	is available on POSIX systems, `launchctl` is available on
    --	MacOS and `schtasks` is available on Windows.
    -+	is available on POSIX systems, `systemd-timer` is available on Linux
    -+	systems, `launchctl` is available on MacOS and `schtasks` is available
    -+	on Windows.
    - 	By default or when `auto` is specified, a suitable scheduler for
    - 	the system is used. On MacOS, `launchctl` is used. On Windows,
    --	`schtasks` is used. On all other systems, `crontab` is used.
    -+	`schtasks` is used. On Linux, `systemd-timer` is used if user systemd
    -+	timers are available, otherwise, `crontab` is used. On all other systems,
    -+	`crontab` is used.
    +-	Possible values for `<scheduler>` are `auto`, `crontab` (POSIX),
    +-	`launchctl` (macOS), and `schtasks` (Windows).
    +-	When `auto` is specified, the appropriate platform-specific
    +-	scheduler is used. Default is `auto`.
    ++	Possible values for `<scheduler>` are `auto`, `crontab`
    ++	(POSIX), `systemd-timer` (Linux), `launchctl` (macOS), and
    ++	`schtasks` (Windows). When `auto` is specified, the
    ++	appropriate platform-specific scheduler is used; on Linux,
    ++	`systemd-timer` is used if available, otherwise
    ++	`crontab`. Default is `auto`.
      
      
      TROUBLESHOOTING
    @@ builtin/gc.c: static enum scheduler parse_scheduler(const char *value)
      	else if (!strcasecmp(value, "launchctl"))
      		return SCHEDULER_LAUNCHCTL;
      	else if (!strcasecmp(value, "schtasks"))
    -@@ builtin/gc.c: static void resolve_auto_scheduler(enum scheduler *scheduler)
    - 	*scheduler = SCHEDULER_SCHTASKS;
    - 	return;
    +@@ builtin/gc.c: static enum scheduler resolve_scheduler(enum scheduler scheduler)
    + #elif defined(GIT_WINDOWS_NATIVE)
    + 	return SCHEDULER_SCHTASKS;
      
     +#elif defined(__linux__)
     +	if (is_systemd_timer_available())
    -+		*scheduler = SCHEDULER_SYSTEMD;
    ++		return SCHEDULER_SYSTEMD;
     +	else if (is_crontab_available())
    -+		*scheduler = SCHEDULER_CRON;
    ++		return SCHEDULER_CRON;
     +	else
     +		die(_("neither systemd timers nor crontab are available"));
    -+	return;
     +
      #else
    - 	*scheduler = SCHEDULER_CRON;
    - 	return;
    + 	return SCHEDULER_CRON;
    + #endif
     
      ## t/t7900-maintenance.sh ##
     @@ t/t7900-maintenance.sh: test_xmllint () {
-- 
2.32.0


^ permalink raw reply	[flat|nested] 138+ messages in thread

* [PATCH v7 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function
  2021-07-02 14:25           ` [PATCH v7 0/3] " Lénaïc Huard
@ 2021-07-02 14:25             ` Lénaïc Huard
  2021-07-02 14:25             ` [PATCH v7 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
                               ` (4 subsequent siblings)
  5 siblings, 0 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-07-02 14:25 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King,
	Lénaïc Huard

Current implementation of `xdg_config_home(filename)` returns
`$XDG_CONFIG_HOME/git/$filename`, with the `git` subdirectory inserted
between the `XDG_CONFIG_HOME` environment variable and the parameter.

This patch introduces a `xdg_config_home_for(subdir, filename)` function
which is more generic. It only concatenates "$XDG_CONFIG_HOME", or
"$HOME/.config" if the former isn’t defined, with the parameters,
without adding `git` in between.

`xdg_config_home(filename)` is now implemented by calling
`xdg_config_home_for("git", filename)` but this new generic function can
be used to compute the configuration directory of other programs.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 cache.h |  7 +++++++
 path.c  | 13 ++++++++++---
 2 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/cache.h b/cache.h
index ba04ff8bd3..2a0fb3e4ba 100644
--- a/cache.h
+++ b/cache.h
@@ -1286,6 +1286,13 @@ int is_ntfs_dotmailmap(const char *name);
  */
 int looks_like_command_line_option(const char *str);
 
+/**
+ * Return a newly allocated string with the evaluation of
+ * "$XDG_CONFIG_HOME/$subdir/$filename" if $XDG_CONFIG_HOME is non-empty, otherwise
+ * "$HOME/.config/$subdir/$filename". Return NULL upon error.
+ */
+char *xdg_config_home_for(const char *subdir, const char *filename);
+
 /**
  * Return a newly allocated string with the evaluation of
  * "$XDG_CONFIG_HOME/git/$filename" if $XDG_CONFIG_HOME is non-empty, otherwise
diff --git a/path.c b/path.c
index 7bccd830e9..1b1de3be09 100644
--- a/path.c
+++ b/path.c
@@ -1503,21 +1503,28 @@ int looks_like_command_line_option(const char *str)
 	return str && str[0] == '-';
 }
 
-char *xdg_config_home(const char *filename)
+char *xdg_config_home_for(const char *subdir, const char *filename)
 {
 	const char *home, *config_home;
 
+	assert(subdir);
 	assert(filename);
 	config_home = getenv("XDG_CONFIG_HOME");
 	if (config_home && *config_home)
-		return mkpathdup("%s/git/%s", config_home, filename);
+		return mkpathdup("%s/%s/%s", config_home, subdir, filename);
 
 	home = getenv("HOME");
 	if (home)
-		return mkpathdup("%s/.config/git/%s", home, filename);
+		return mkpathdup("%s/.config/%s/%s", home, subdir, filename);
+
 	return NULL;
 }
 
+char *xdg_config_home(const char *filename)
+{
+	return xdg_config_home_for("git", filename);
+}
+
 char *xdg_cache_home(const char *filename)
 {
 	const char *home, *cache_home;
-- 
2.32.0


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* [PATCH v7 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-07-02 14:25           ` [PATCH v7 0/3] " Lénaïc Huard
  2021-07-02 14:25             ` [PATCH v7 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
@ 2021-07-02 14:25             ` Lénaïc Huard
  2021-07-06 19:56               ` Ævar Arnfjörð Bjarmason
  2021-07-02 14:25             ` [PATCH v7 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
                               ` (3 subsequent siblings)
  5 siblings, 1 reply; 138+ messages in thread
From: Lénaïc Huard @ 2021-07-02 14:25 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King,
	Lénaïc Huard

Depending on the system, different schedulers can be used to schedule
the hourly, daily and weekly executions of `git maintenance run`:
* `launchctl` for MacOS,
* `schtasks` for Windows and
* `crontab` for everything else.

`git maintenance run` now has an option to let the end-user explicitly
choose which scheduler he wants to use:
`--scheduler=auto|crontab|launchctl|schtasks`.

When `git maintenance start --scheduler=XXX` is run, it not only
registers `git maintenance run` tasks in the scheduler XXX, it also
removes the `git maintenance run` tasks from all the other schedulers to
ensure we cannot have two schedulers launching concurrent identical
tasks.

The default value is `auto` which chooses a suitable scheduler for the
system.

`git maintenance stop` doesn't have any `--scheduler` parameter because
this command will try to remove the `git maintenance run` tasks from all
the available schedulers.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 Documentation/git-maintenance.txt |   9 +
 builtin/gc.c                      | 370 ++++++++++++++++++++++++------
 t/t7900-maintenance.sh            |  55 ++++-
 3 files changed, 359 insertions(+), 75 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 1e738ad398..576290b5c6 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -179,6 +179,15 @@ OPTIONS
 	`maintenance.<task>.enabled` configured as `true` are considered.
 	See the 'TASKS' section for the list of accepted `<task>` values.
 
+--scheduler=auto|crontab|launchctl|schtasks::
+	When combined with the `start` subcommand, specify the scheduler
+	for running the hourly, daily and weekly executions of
+	`git maintenance run`.
+	Possible values for `<scheduler>` are `auto`, `crontab` (POSIX),
+	`launchctl` (macOS), and `schtasks` (Windows).
+	When `auto` is specified, the appropriate platform-specific
+	scheduler is used. Default is `auto`.
+
 
 TROUBLESHOOTING
 ---------------
diff --git a/builtin/gc.c b/builtin/gc.c
index f05d2f0a1a..96a43f99b4 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1529,6 +1529,98 @@ static const char *get_frequency(enum schedule_priority schedule)
 	}
 }
 
+/*
+ * get_schedule_cmd` reads the GIT_TEST_MAINT_SCHEDULER environment variable
+ * to mock the schedulers that `git maintenance start` rely on.
+ *
+ * For test purpose, GIT_TEST_MAINT_SCHEDULER can be set to a comma-separated
+ * list of colon-separated key/value pairs where each pair contains a scheduler
+ * and its corresponding mock.
+ *
+ * * If $GET_TEST_MAINT_SCHEDULER is not set, return false and leave the
+ *   arguments unmodified.
+ *
+ * * If $GET_TEST_MAINT_SCHEDULER is set, return true.
+ *   In this case, the *cmd value is read as input.
+ *
+ *   * if the input value *cmd is the key of one of the comma-separated list
+ *     item, then *is_available is set to true and *cmd is modified and becomes
+ *     the mock command.
+ *
+ *   * if the input value *cmd isn’t the key of any of the comma-separated list
+ *     item, then *is_available is set to false.
+ *
+ * Ex.:
+ *   GIT_TEST_MAINT_SCHEDULER not set
+ *     ┏━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
+ *     ┃ Input ┃                     Output                      ┃
+ *     ┃ *cmd  ┃ return code │       *cmd        │ *is_available ┃
+ *     ┣━━━━━━━╋━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━┫
+ *     ┃ "foo" ┃    false    │ "foo" (unchanged) │  (unchanged)  ┃
+ *     ┗━━━━━━━┻━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━┛
+ *
+ *   GIT_TEST_MAINT_SCHEDULER set to “foo:./mock_foo.sh,bar:./mock_bar.sh”
+ *     ┏━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
+ *     ┃ Input ┃                     Output                      ┃
+ *     ┃ *cmd  ┃ return code │       *cmd        │ *is_available ┃
+ *     ┣━━━━━━━╋━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━┫
+ *     ┃ "foo" ┃    true     │  "./mock.foo.sh"  │     true      ┃
+ *     ┃ "qux" ┃    true     │ "qux" (unchanged) │     false     ┃
+ *     ┗━━━━━━━┻━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━┛
+ */
+static int get_schedule_cmd(const char **cmd, int *is_available)
+{
+	char *item;
+	char *testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
+
+	if (!testing)
+		return 0;
+
+	if (is_available)
+		*is_available = 0;
+
+	for (item = testing;;) {
+		char *sep;
+		char *end_item = strchr(item, ',');
+		if (end_item)
+			*end_item = '\0';
+
+		sep = strchr(item, ':');
+		if (!sep)
+			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
+		*sep = '\0';
+
+		if (!strcmp(*cmd, item)) {
+			*cmd = sep + 1;
+			if (is_available)
+				*is_available = 1;
+			UNLEAK(testing);
+			return 1;
+		}
+
+		if (!end_item)
+			break;
+		item = end_item + 1;
+	}
+
+	free(testing);
+	return 1;
+}
+
+static int is_launchctl_available(void)
+{
+	const char *cmd = "launchctl";
+	int is_available;
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+#ifdef __APPLE__
+	return 1;
+#else
+	return 0;
+#endif
+}
+
 static char *launchctl_service_name(const char *frequency)
 {
 	struct strbuf label = STRBUF_INIT;
@@ -1555,19 +1647,17 @@ static char *launchctl_get_uid(void)
 	return xstrfmt("gui/%d", getuid());
 }
 
-static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
+static int launchctl_boot_plist(int enable, const char *filename)
 {
+	const char *cmd = "launchctl";
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
 	char *uid = launchctl_get_uid();
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&child.args, cmd);
-	if (enable)
-		strvec_push(&child.args, "bootstrap");
-	else
-		strvec_push(&child.args, "bootout");
-	strvec_push(&child.args, uid);
-	strvec_push(&child.args, filename);
+	strvec_pushl(&child.args, enable ? "bootstrap" : "bootout", uid,
+		     filename, NULL);
 
 	child.no_stderr = 1;
 	child.no_stdout = 1;
@@ -1581,26 +1671,26 @@ static int launchctl_boot_plist(int enable, const char *filename, const char *cm
 	return result;
 }
 
-static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)
+static int launchctl_remove_plist(enum schedule_priority schedule)
 {
 	const char *frequency = get_frequency(schedule);
 	char *name = launchctl_service_name(frequency);
 	char *filename = launchctl_service_filename(name);
-	int result = launchctl_boot_plist(0, filename, cmd);
+	int result = launchctl_boot_plist(0, filename);
 	unlink(filename);
 	free(filename);
 	free(name);
 	return result;
 }
 
-static int launchctl_remove_plists(const char *cmd)
+static int launchctl_remove_plists(void)
 {
-	return launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||
-		launchctl_remove_plist(SCHEDULE_DAILY, cmd) ||
-		launchctl_remove_plist(SCHEDULE_WEEKLY, cmd);
+	return launchctl_remove_plist(SCHEDULE_HOURLY) ||
+	       launchctl_remove_plist(SCHEDULE_DAILY) ||
+	       launchctl_remove_plist(SCHEDULE_WEEKLY);
 }
 
-static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule)
 {
 	FILE *plist;
 	int i;
@@ -1669,8 +1759,8 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 	fclose(plist);
 
 	/* bootout might fail if not already running, so ignore */
-	launchctl_boot_plist(0, filename, cmd);
-	if (launchctl_boot_plist(1, filename, cmd))
+	launchctl_boot_plist(0, filename);
+	if (launchctl_boot_plist(1, filename))
 		die(_("failed to bootstrap service %s"), filename);
 
 	free(filename);
@@ -1678,21 +1768,35 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 	return 0;
 }
 
-static int launchctl_add_plists(const char *cmd)
+static int launchctl_add_plists(void)
 {
 	const char *exec_path = git_exec_path();
 
-	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||
-		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||
-		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
+	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY) ||
+	       launchctl_schedule_plist(exec_path, SCHEDULE_DAILY) ||
+	       launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY);
 }
 
-static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int launchctl_update_schedule(int run_maintenance, int fd)
 {
 	if (run_maintenance)
-		return launchctl_add_plists(cmd);
+		return launchctl_add_plists();
 	else
-		return launchctl_remove_plists(cmd);
+		return launchctl_remove_plists();
+}
+
+static int is_schtasks_available(void)
+{
+	const char *cmd = "schtasks";
+	int is_available;
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+#ifdef GIT_WINDOWS_NATIVE
+	return 1;
+#else
+	return 0;
+#endif
 }
 
 static char *schtasks_task_name(const char *frequency)
@@ -1702,13 +1806,15 @@ static char *schtasks_task_name(const char *frequency)
 	return strbuf_detach(&label, NULL);
 }
 
-static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)
+static int schtasks_remove_task(enum schedule_priority schedule)
 {
+	const char *cmd = "schtasks";
 	int result;
 	struct strvec args = STRVEC_INIT;
 	const char *frequency = get_frequency(schedule);
 	char *name = schtasks_task_name(frequency);
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&args, cmd);
 	strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
 
@@ -1719,15 +1825,16 @@ static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd
 	return result;
 }
 
-static int schtasks_remove_tasks(const char *cmd)
+static int schtasks_remove_tasks(void)
 {
-	return schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||
-		schtasks_remove_task(SCHEDULE_DAILY, cmd) ||
-		schtasks_remove_task(SCHEDULE_WEEKLY, cmd);
+	return schtasks_remove_task(SCHEDULE_HOURLY) ||
+	       schtasks_remove_task(SCHEDULE_DAILY) ||
+	       schtasks_remove_task(SCHEDULE_WEEKLY);
 }
 
-static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule)
 {
+	const char *cmd = "schtasks";
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
 	const char *xml;
@@ -1736,6 +1843,8 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
 	char *name = schtasks_task_name(frequency);
 	struct strbuf tfilename = STRBUF_INIT;
 
+	get_schedule_cmd(&cmd, NULL);
+
 	strbuf_addf(&tfilename, "%s/schedule_%s_XXXXXX",
 		    get_git_common_dir(), frequency);
 	tfile = xmks_tempfile(tfilename.buf);
@@ -1840,28 +1949,52 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
 	return result;
 }
 
-static int schtasks_schedule_tasks(const char *cmd)
+static int schtasks_schedule_tasks(void)
 {
 	const char *exec_path = git_exec_path();
 
-	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||
-		schtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||
-		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
+	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY) ||
+	       schtasks_schedule_task(exec_path, SCHEDULE_DAILY) ||
+	       schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY);
 }
 
-static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int schtasks_update_schedule(int run_maintenance, int fd)
 {
 	if (run_maintenance)
-		return schtasks_schedule_tasks(cmd);
+		return schtasks_schedule_tasks();
 	else
-		return schtasks_remove_tasks(cmd);
+		return schtasks_remove_tasks();
+}
+
+static int is_crontab_available(void)
+{
+	const char *cmd = "crontab";
+	int is_available;
+	struct child_process child = CHILD_PROCESS_INIT;
+
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+	strvec_split(&child.args, cmd);
+	strvec_push(&child.args, "-l");
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child))
+		return 0;
+	/* Ignore exit code, as an empty crontab will return error. */
+	finish_command(&child);
+	return 1;
 }
 
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
-static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int crontab_update_schedule(int run_maintenance, int fd)
 {
+	const char *cmd = "crontab";
 	int result = 0;
 	int in_old_region = 0;
 	struct child_process crontab_list = CHILD_PROCESS_INIT;
@@ -1869,6 +2002,7 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 	FILE *cron_list, *cron_in;
 	struct strbuf line = STRBUF_INIT;
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&crontab_list.args, cmd);
 	strvec_push(&crontab_list.args, "-l");
 	crontab_list.in = -1;
@@ -1945,66 +2079,160 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 	return result;
 }
 
+enum scheduler {
+	SCHEDULER_INVALID = -1,
+	SCHEDULER_AUTO,
+	SCHEDULER_CRON,
+	SCHEDULER_LAUNCHCTL,
+	SCHEDULER_SCHTASKS,
+};
+
+static const struct {
+	const char *name;
+	int (*is_available)(void);
+	int (*update_schedule)(int run_maintenance, int fd);
+} scheduler_fn[] = {
+	[SCHEDULER_CRON] = {
+		.name = "crontab",
+		.is_available = is_crontab_available,
+		.update_schedule = crontab_update_schedule,
+	},
+	[SCHEDULER_LAUNCHCTL] = {
+		.name = "launchctl",
+		.is_available = is_launchctl_available,
+		.update_schedule = launchctl_update_schedule,
+	},
+	[SCHEDULER_SCHTASKS] = {
+		.name = "schtasks",
+		.is_available = is_schtasks_available,
+		.update_schedule = schtasks_update_schedule,
+	},
+};
+
+static enum scheduler parse_scheduler(const char *value)
+{
+	if (!value)
+		return SCHEDULER_INVALID;
+	else if (!strcasecmp(value, "auto"))
+		return SCHEDULER_AUTO;
+	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
+		return SCHEDULER_CRON;
+	else if (!strcasecmp(value, "launchctl"))
+		return SCHEDULER_LAUNCHCTL;
+	else if (!strcasecmp(value, "schtasks"))
+		return SCHEDULER_SCHTASKS;
+	else
+		return SCHEDULER_INVALID;
+}
+
+static int maintenance_opt_scheduler(const struct option *opt, const char *arg,
+				     int unset)
+{
+	enum scheduler *scheduler = opt->value;
+
+	BUG_ON_OPT_NEG(unset);
+
+	*scheduler = parse_scheduler(arg);
+	if (*scheduler == SCHEDULER_INVALID)
+		return error(_("unrecognized --scheduler argument '%s'"), arg);
+	return 0;
+}
+
+struct maintenance_start_opts {
+	enum scheduler scheduler;
+};
+
+static enum scheduler resolve_scheduler(enum scheduler scheduler)
+{
+	if (scheduler != SCHEDULER_AUTO)
+		return scheduler;
+
 #if defined(__APPLE__)
-static const char platform_scheduler[] = "launchctl";
+	return SCHEDULER_LAUNCHCTL;
+
 #elif defined(GIT_WINDOWS_NATIVE)
-static const char platform_scheduler[] = "schtasks";
+	return SCHEDULER_SCHTASKS;
+
 #else
-static const char platform_scheduler[] = "crontab";
+	return SCHEDULER_CRON;
 #endif
+}
 
-static int update_background_schedule(int enable)
+static void validate_scheduler(enum scheduler scheduler)
 {
-	int result;
-	const char *scheduler = platform_scheduler;
-	const char *cmd = scheduler;
-	char *testing;
+	if (scheduler == SCHEDULER_INVALID)
+		BUG("invalid scheduler");
+	if (scheduler == SCHEDULER_AUTO)
+		BUG("resolve_scheduler should have been called before");
+
+	if (!scheduler_fn[scheduler].is_available())
+		die(_("%s scheduler is not available"),
+		    scheduler_fn[scheduler].name);
+}
+
+static int update_background_schedule(const struct maintenance_start_opts *opts,
+				      int enable)
+{
+	unsigned int i;
+	int result = 0;
 	struct lock_file lk;
 	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
 
-	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
-	if (testing) {
-		char *sep = strchr(testing, ':');
-		if (!sep)
-			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
-		*sep = '\0';
-		scheduler = testing;
-		cmd = sep + 1;
+	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {
+		free(lock_path);
+		return error(_("another process is scheduling background maintenance"));
 	}
 
-	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {
-		result = error(_("another process is scheduling background maintenance"));
-		goto cleanup;
+	for (i = 1; i < ARRAY_SIZE(scheduler_fn); i++) {
+		if (enable && opts->scheduler == i)
+			continue;
+		if (!scheduler_fn[i].is_available())
+			continue;
+		scheduler_fn[i].update_schedule(0, get_lock_file_fd(&lk));
 	}
 
-	if (!strcmp(scheduler, "launchctl"))
-		result = launchctl_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else if (!strcmp(scheduler, "schtasks"))
-		result = schtasks_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else if (!strcmp(scheduler, "crontab"))
-		result = crontab_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else
-		die("unknown background scheduler: %s", scheduler);
+	if (enable)
+		result = scheduler_fn[opts->scheduler].update_schedule(
+			1, get_lock_file_fd(&lk));
 
 	rollback_lock_file(&lk);
 
-cleanup:
 	free(lock_path);
-	free(testing);
 	return result;
 }
 
-static int maintenance_start(void)
+static const char *const builtin_maintenance_start_usage[] = {
+	N_("git maintenance start [--scheduler=<scheduler>]"),
+	NULL
+};
+
+static int maintenance_start(int argc, const char **argv, const char *prefix)
 {
+	struct maintenance_start_opts opts = { 0 };
+	struct option options[] = {
+		OPT_CALLBACK_F(
+			0, "scheduler", &opts.scheduler, N_("scheduler"),
+			N_("scheduler to trigger git maintenance run"),
+			PARSE_OPT_NONEG, maintenance_opt_scheduler),
+		OPT_END()
+	};
+
+	argc = parse_options(argc, argv, prefix, options,
+			     builtin_maintenance_start_usage, 0);
+	if (argc)
+		usage_with_options(builtin_maintenance_start_usage, options);
+
+	opts.scheduler = resolve_scheduler(opts.scheduler);
+	validate_scheduler(opts.scheduler);
+
 	if (maintenance_register())
 		warning(_("failed to add repo to global config"));
-
-	return update_background_schedule(1);
+	return update_background_schedule(&opts, 1);
 }
 
 static int maintenance_stop(void)
 {
-	return update_background_schedule(0);
+	return update_background_schedule(NULL, 0);
 }
 
 static const char builtin_maintenance_usage[] =	N_("git maintenance <subcommand> [<options>]");
@@ -2018,7 +2246,7 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
 	if (!strcmp(argv[1], "run"))
 		return maintenance_run(argc - 1, argv + 1, prefix);
 	if (!strcmp(argv[1], "start"))
-		return maintenance_start();
+		return maintenance_start(argc - 1, argv + 1, prefix);
 	if (!strcmp(argv[1], "stop"))
 		return maintenance_stop();
 	if (!strcmp(argv[1], "register"))
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index b93ae014ee..b36b7f5fb0 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -494,8 +494,21 @@ test_expect_success !MINGW 'register and unregister with regex metacharacters' '
 		maintenance.repo "$(pwd)/$META"
 '
 
+test_expect_success 'start --scheduler=<scheduler>' '
+	test_expect_code 129 git maintenance start --scheduler=foo 2>err &&
+	test_i18ngrep "unrecognized --scheduler argument" err &&
+
+	test_expect_code 129 git maintenance start --no-scheduler 2>err &&
+	test_i18ngrep "unknown option" err &&
+
+	test_expect_code 128 \
+		env GIT_TEST_MAINT_SCHEDULER="launchctl:true,schtasks:true" \
+		git maintenance start --scheduler=crontab 2>err &&
+	test_i18ngrep "fatal: crontab scheduler is not available" err
+'
+
 test_expect_success 'start from empty cron table' '
-	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -518,7 +531,7 @@ test_expect_success 'stop from existing schedule' '
 
 test_expect_success 'start preserves existing schedule' '
 	echo "Important information!" >cron.txt &&
-	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
 	grep "Important information!" cron.txt
 '
 
@@ -547,7 +560,7 @@ test_expect_success 'start and stop macOS maintenance' '
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start --scheduler=launchctl &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -598,7 +611,7 @@ test_expect_success 'start and stop Windows maintenance' '
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start --scheduler=schtasks &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -621,6 +634,40 @@ test_expect_success 'start and stop Windows maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success 'start and stop when several schedulers are available' '
+	write_script print-args <<-\EOF &&
+	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
+	EOF
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >expect &&
+	for frequency in hourly daily weekly
+	do
+		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
+		echo "launchctl bootout gui/[UID] $PLIST" >>expect &&
+		echo "launchctl bootstrap gui/[UID] $PLIST" >>expect || return 1
+	done &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
2.32.0


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* [PATCH v7 3/3] maintenance: add support for systemd timers on Linux
  2021-07-02 14:25           ` [PATCH v7 0/3] " Lénaïc Huard
  2021-07-02 14:25             ` [PATCH v7 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
  2021-07-02 14:25             ` [PATCH v7 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
@ 2021-07-02 14:25             ` Lénaïc Huard
  2021-07-06 20:03               ` Ævar Arnfjörð Bjarmason
  2021-07-02 18:18             ` [PATCH v7 0/3] " Junio C Hamano
                               ` (2 subsequent siblings)
  5 siblings, 1 reply; 138+ messages in thread
From: Lénaïc Huard @ 2021-07-02 14:25 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King,
	Lénaïc Huard

The existing mechanism for scheduling background maintenance is done
through cron. On Linux systems managed by systemd, systemd provides an
alternative to schedule recurring tasks: systemd timers.

The main motivations to implement systemd timers in addition to cron
are:
* cron is optional and Linux systems running systemd might not have it
  installed.
* The execution of `crontab -l` can tell us if cron is installed but not
  if the daemon is actually running.
* With systemd, each service is run in its own cgroup and its logs are
  tagged by the service inside journald. With cron, all scheduled tasks
  are running in the cron daemon cgroup and all the logs of the
  user-scheduled tasks are pretended to belong to the system cron
  service.
  Concretely, a user that doesn’t have access to the system logs won’t
  have access to the log of their own tasks scheduled by cron whereas
  they will have access to the log of their own tasks scheduled by
  systemd timer.
  Although `cron` attempts to send email, that email may go unseen by
  the user because these days, local mailboxes are not heavily used
  anymore.

In order to schedule git maintenance, we need two unit template files:
* ~/.config/systemd/user/git-maintenance@.service
  to define the command to be started by systemd and
* ~/.config/systemd/user/git-maintenance@.timer
  to define the schedule at which the command should be run.

Those units are templates that are parameterized by the frequency.

Based on those templates, 3 timers are started:
* git-maintenance@hourly.timer
* git-maintenance@daily.timer
* git-maintenance@weekly.timer

The command launched by those three timers are the same as with the
other scheduling methods:

/path/to/git for-each-repo --exec-path=/path/to
--config=maintenance.repo maintenance run --schedule=%i

with the full path for git to ensure that the version of git launched
for the scheduled maintenance is the same as the one used to run
`maintenance start`.

The timer unit contains `Persistent=true` so that, if the computer is
powered down when a maintenance task should run, the task will be run
when the computer is back powered on.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 Documentation/git-maintenance.txt |  58 +++++++-
 builtin/gc.c                      | 227 ++++++++++++++++++++++++++++++
 t/t7900-maintenance.sh            |  67 ++++++++-
 3 files changed, 341 insertions(+), 11 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 576290b5c6..e2cfb68ab5 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -179,14 +179,16 @@ OPTIONS
 	`maintenance.<task>.enabled` configured as `true` are considered.
 	See the 'TASKS' section for the list of accepted `<task>` values.
 
---scheduler=auto|crontab|launchctl|schtasks::
+--scheduler=auto|crontab|systemd-timer|launchctl|schtasks::
 	When combined with the `start` subcommand, specify the scheduler
 	for running the hourly, daily and weekly executions of
 	`git maintenance run`.
-	Possible values for `<scheduler>` are `auto`, `crontab` (POSIX),
-	`launchctl` (macOS), and `schtasks` (Windows).
-	When `auto` is specified, the appropriate platform-specific
-	scheduler is used. Default is `auto`.
+	Possible values for `<scheduler>` are `auto`, `crontab`
+	(POSIX), `systemd-timer` (Linux), `launchctl` (macOS), and
+	`schtasks` (Windows). When `auto` is specified, the
+	appropriate platform-specific scheduler is used; on Linux,
+	`systemd-timer` is used if available, otherwise
+	`crontab`. Default is `auto`.
 
 
 TROUBLESHOOTING
@@ -286,6 +288,52 @@ schedule to ensure you are executing the correct binaries in your
 schedule.
 
 
+BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMS
+-----------------------------------------------
+
+While Linux supports `cron`, depending on the distribution, `cron` may
+be an optional package not necessarily installed. On modern Linux
+distributions, systemd timers are superseding it.
+
+If user systemd timers are available, they will be used as a replacement
+of `cron`.
+
+In this case, `git maintenance start` will create user systemd timer units
+and start the timers. The current list of user-scheduled tasks can be found
+by running `systemctl --user list-timers`. The timers written by `git
+maintenance start` are similar to this:
+
+-----------------------------------------------------------------------
+$ systemctl --user list-timers
+NEXT                         LEFT          LAST                         PASSED     UNIT                         ACTIVATES
+Thu 2021-04-29 19:00:00 CEST 42min left    Thu 2021-04-29 18:00:11 CEST 17min ago  git-maintenance@hourly.timer git-maintenance@hourly.service
+Fri 2021-04-30 00:00:00 CEST 5h 42min left Thu 2021-04-29 00:00:11 CEST 18h ago    git-maintenance@daily.timer  git-maintenance@daily.service
+Mon 2021-05-03 00:00:00 CEST 3 days left   Mon 2021-04-26 00:00:11 CEST 3 days ago git-maintenance@weekly.timer git-maintenance@weekly.service
+-----------------------------------------------------------------------
+
+One timer is registered for each `--schedule=<frequency>` option.
+
+The definition of the systemd units can be inspected in the following files:
+
+-----------------------------------------------------------------------
+~/.config/systemd/user/git-maintenance@.timer
+~/.config/systemd/user/git-maintenance@.service
+~/.config/systemd/user/timers.target.wants/git-maintenance@hourly.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@daily.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@weekly.timer
+-----------------------------------------------------------------------
+
+`git maintenance start` will overwrite these files and start the timer
+again with `systemctl --user`, so any customization should be done by
+creating a drop-in file, i.e. a `.conf` suffixed file in the
+`~/.config/systemd/user/git-maintenance@.service.d` directory.
+
+`git maintenance stop` will stop the user systemd timers and delete
+the above mentioned files.
+
+For more details, see `systemd.timer(5)`.
+
+
 BACKGROUND MAINTENANCE ON MACOS SYSTEMS
 ---------------------------------------
 
diff --git a/builtin/gc.c b/builtin/gc.c
index 96a43f99b4..0cc02fa79e 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -2079,10 +2079,221 @@ static int crontab_update_schedule(int run_maintenance, int fd)
 	return result;
 }
 
+#ifdef __linux__
+
+static int real_is_systemd_timer_available(void)
+{
+	struct child_process child = CHILD_PROCESS_INIT;
+
+	strvec_pushl(&child.args, "systemctl", "--user", "list-timers", NULL);
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child))
+		return 0;
+	if (finish_command(&child))
+		return 0;
+	return 1;
+}
+
+#else
+
+static int real_is_systemd_timer_available(void)
+{
+	return 0;
+}
+
+#endif
+
+static int is_systemd_timer_available(void)
+{
+	const char *cmd = "systemctl";
+	int is_available;
+
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+	return real_is_systemd_timer_available();
+}
+
+static char *xdg_config_home_systemd(const char *filename)
+{
+	return xdg_config_home_for("systemd/user", filename);
+}
+
+static int systemd_timer_enable_unit(int enable,
+				     enum schedule_priority schedule)
+{
+	const char *cmd = "systemctl";
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *frequency = get_frequency(schedule);
+
+	/*
+	 * Disabling the systemd unit while it is already disabled makes
+	 * systemctl print an error.
+	 * Let's ignore it since it means we already are in the expected state:
+	 * the unit is disabled.
+	 *
+	 * On the other hand, enabling a systemd unit which is already enabled
+	 * produces no error.
+	 */
+	if (!enable)
+		child.no_stderr = 1;
+
+	get_schedule_cmd(&cmd, NULL);
+	strvec_split(&child.args, cmd);
+	strvec_pushl(&child.args, "--user", enable ? "enable" : "disable",
+		     "--now", NULL);
+	strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency);
+
+	if (start_command(&child))
+		return error(_("failed to start systemctl"));
+	if (finish_command(&child))
+		/*
+		 * Disabling an already disabled systemd unit makes
+		 * systemctl fail.
+		 * Let's ignore this failure.
+		 *
+		 * Enabling an enabled systemd unit doesn't fail.
+		 */
+		if (enable)
+			return error(_("failed to run systemctl"));
+	return 0;
+}
+
+static int systemd_timer_delete_unit_templates(void)
+{
+	int ret = 0;
+	char *filename = xdg_config_home_systemd("git-maintenance@.timer");
+	if (unlink(filename) && !is_missing_file_error(errno))
+		ret = error_errno(_("failed to delete '%s'"), filename);
+	FREE_AND_NULL(filename);
+
+	filename = xdg_config_home_systemd("git-maintenance@.service");
+	if (unlink(filename) && !is_missing_file_error(errno))
+		ret = error_errno(_("failed to delete '%s'"), filename);
+
+	free(filename);
+	return ret;
+}
+
+static int systemd_timer_delete_units(void)
+{
+	return systemd_timer_enable_unit(0, SCHEDULE_HOURLY) ||
+	       systemd_timer_enable_unit(0, SCHEDULE_DAILY) ||
+	       systemd_timer_enable_unit(0, SCHEDULE_WEEKLY) ||
+	       systemd_timer_delete_unit_templates();
+}
+
+static int systemd_timer_write_unit_templates(const char *exec_path)
+{
+	char *filename;
+	FILE *file;
+	const char *unit;
+
+	filename = xdg_config_home_systemd("git-maintenance@.timer");
+	if (safe_create_leading_directories(filename)) {
+		error(_("failed to create directories for '%s'"), filename);
+		goto error;
+	}
+	file = fopen_or_warn(filename, "w");
+	if (file == NULL)
+		goto error;
+
+	unit = "# This file was created and is maintained by Git.\n"
+	       "# Any edits made in this file might be replaced in the future\n"
+	       "# by a Git command.\n"
+	       "\n"
+	       "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Timer]\n"
+	       "OnCalendar=%i\n"
+	       "Persistent=true\n"
+	       "\n"
+	       "[Install]\n"
+	       "WantedBy=timers.target\n";
+	if (fputs(unit, file) == EOF) {
+		error(_("failed to write to '%s'"), filename);
+		fclose(file);
+		goto error;
+	}
+	if (fclose(file) == EOF) {
+		error_errno(_("failed to flush '%s'"), filename);
+		goto error;
+	}
+	free(filename);
+
+	filename = xdg_config_home_systemd("git-maintenance@.service");
+	file = fopen_or_warn(filename, "w");
+	if (file == NULL)
+		goto error;
+
+	unit = "# This file was created and is maintained by Git.\n"
+	       "# Any edits made in this file might be replaced in the future\n"
+	       "# by a Git command.\n"
+	       "\n"
+	       "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Service]\n"
+	       "Type=oneshot\n"
+	       "ExecStart=\"%s/git\" --exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
+	       "LockPersonality=yes\n"
+	       "MemoryDenyWriteExecute=yes\n"
+	       "NoNewPrivileges=yes\n"
+	       "RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\n"
+	       "RestrictNamespaces=yes\n"
+	       "RestrictRealtime=yes\n"
+	       "RestrictSUIDSGID=yes\n"
+	       "SystemCallArchitectures=native\n"
+	       "SystemCallFilter=@system-service\n";
+	if (fprintf(file, unit, exec_path, exec_path) < 0) {
+		error(_("failed to write to '%s'"), filename);
+		fclose(file);
+		goto error;
+	}
+	if (fclose(file) == EOF) {
+		error_errno(_("failed to flush '%s'"), filename);
+		goto error;
+	}
+	free(filename);
+	return 0;
+
+error:
+	free(filename);
+	systemd_timer_delete_unit_templates();
+	return -1;
+}
+
+static int systemd_timer_setup_units(void)
+{
+	const char *exec_path = git_exec_path();
+
+	int ret = systemd_timer_write_unit_templates(exec_path) ||
+	          systemd_timer_enable_unit(1, SCHEDULE_HOURLY) ||
+	          systemd_timer_enable_unit(1, SCHEDULE_DAILY) ||
+	          systemd_timer_enable_unit(1, SCHEDULE_WEEKLY);
+	if (ret)
+		systemd_timer_delete_units();
+	return ret;
+}
+
+static int systemd_timer_update_schedule(int run_maintenance, int fd)
+{
+	if (run_maintenance)
+		return systemd_timer_setup_units();
+	else
+		return systemd_timer_delete_units();
+}
+
 enum scheduler {
 	SCHEDULER_INVALID = -1,
 	SCHEDULER_AUTO,
 	SCHEDULER_CRON,
+	SCHEDULER_SYSTEMD,
 	SCHEDULER_LAUNCHCTL,
 	SCHEDULER_SCHTASKS,
 };
@@ -2097,6 +2308,11 @@ static const struct {
 		.is_available = is_crontab_available,
 		.update_schedule = crontab_update_schedule,
 	},
+	[SCHEDULER_SYSTEMD] = {
+		.name = "systemctl",
+		.is_available = is_systemd_timer_available,
+		.update_schedule = systemd_timer_update_schedule,
+	},
 	[SCHEDULER_LAUNCHCTL] = {
 		.name = "launchctl",
 		.is_available = is_launchctl_available,
@@ -2117,6 +2333,9 @@ static enum scheduler parse_scheduler(const char *value)
 		return SCHEDULER_AUTO;
 	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
 		return SCHEDULER_CRON;
+	else if (!strcasecmp(value, "systemd") ||
+		 !strcasecmp(value, "systemd-timer"))
+		return SCHEDULER_SYSTEMD;
 	else if (!strcasecmp(value, "launchctl"))
 		return SCHEDULER_LAUNCHCTL;
 	else if (!strcasecmp(value, "schtasks"))
@@ -2153,6 +2372,14 @@ static enum scheduler resolve_scheduler(enum scheduler scheduler)
 #elif defined(GIT_WINDOWS_NATIVE)
 	return SCHEDULER_SCHTASKS;
 
+#elif defined(__linux__)
+	if (is_systemd_timer_available())
+		return SCHEDULER_SYSTEMD;
+	else if (is_crontab_available())
+		return SCHEDULER_CRON;
+	else
+		die(_("neither systemd timers nor crontab are available"));
+
 #else
 	return SCHEDULER_CRON;
 #endif
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index b36b7f5fb0..b289cae6b9 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -20,6 +20,18 @@ test_xmllint () {
 	fi
 }
 
+test_lazy_prereq SYSTEMD_ANALYZE '
+	systemd-analyze --help >out &&
+	grep verify out
+'
+
+test_systemd_analyze_verify () {
+	if test_have_prereq SYSTEMD_ANALYZE
+	then
+		systemd-analyze verify "$@"
+	fi
+}
+
 test_expect_success 'help text' '
 	test_expect_code 129 git maintenance -h 2>err &&
 	test_i18ngrep "usage: git maintenance <subcommand>" err &&
@@ -634,15 +646,56 @@ test_expect_success 'start and stop Windows maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success 'start and stop Linux/systemd maintenance' '
+	write_script print-args <<-\EOF &&
+	printf "%s\n" "$*" >>args
+	EOF
+
+	XDG_CONFIG_HOME="$PWD" &&
+	export XDG_CONFIG_HOME &&
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance start --scheduler=systemd-timer &&
+
+	# start registers the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_systemd_analyze_verify "systemd/user/git-maintenance@.service" &&
+
+	printf -- "--user enable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_path_is_missing "systemd/user/git-maintenance@.timer" &&
+	test_path_is_missing "systemd/user/git-maintenance@.service" &&
+
+	printf -- "--user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'start and stop when several schedulers are available' '
 	write_script print-args <<-\EOF &&
 	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
-	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=systemd-timer &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
 		hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
+	printf -- "systemctl --user enable --now git-maintenance@%s.timer\n" hourly daily weekly >>expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
 	for frequency in hourly daily weekly
 	do
 		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
@@ -652,17 +705,19 @@ test_expect_success 'start and stop when several schedulers are available' '
 	test_cmp expect args &&
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
 	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
-		hourly daily weekly >expect &&
+		hourly daily weekly >>expect &&
 	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
 		hourly daily weekly >>expect &&
 	test_cmp expect args &&
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
 	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
-		hourly daily weekly >expect &&
+		hourly daily weekly >>expect &&
 	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
 		hourly daily weekly >>expect &&
 	test_cmp expect args
-- 
2.32.0


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* Re: [PATCH v6 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-06-14  4:36             ` Eric Sunshine
  2021-06-16 18:12               ` Derrick Stolee
  2021-06-17 14:26               ` Phillip Wood
@ 2021-07-02 15:04               ` Lénaïc Huard
  2 siblings, 0 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-07-02 15:04 UTC (permalink / raw)
  To: Eric Sunshine
  Cc: Git List, Junio C Hamano, Derrick Stolee,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King

Le lundi 14 juin 2021, 06:36:09 CEST Eric Sunshine a écrit :

> Thanks. Unfortunately, I haven't been following this series too
> closely since I reviewed v1, so I set aside time to review v6, which I
> have now done.
> …
> I do, though, have one question (below) about is_crontab_available()
> for which I could not figure out the answer.

Thank you very much for your review.
I’ve just submitted a new re-roll that should address the points you raised 
and in particular the `is_crontab_available` unexpected behavior.

…
> > +static int is_launchctl_available(void)
> > +{
> > +       const char *cmd = "launchctl";
> > +       int is_available;
> > +       if (get_schedule_cmd(&cmd, &is_available))
> > +               return is_available;
> > +
> > +#ifdef __APPLE__
> > +       return 1;
> > +#else
> > +       return 0;
> > +#endif
> > +}
> 
> On this project, we usually frown upon #if conditionals within
> functions since the code often can become unreadable. The usage in
> this function doesn't suffer from that problem, however,
> resolve_auto_scheduler() is somewhat ugly. An alternative would be to
> set up these values outside of all functions, perhaps like this:
> 
>     #ifdef __APPLE__
>     #define MAINT_SCHEDULER SCHEDULER_LAUNCHCTL
>     #elif GIT_WINDOWS_NATIVE
>     #define MAINT_SCHEDULER SCHEDULER_SCHTASKS
>     #else
>     #define MAINT_SCHEDULER SCHEDULER_CRON
>     #endif
> 
> and then:
> 
>     static int is_launchctl_available(void)
>     {
>         if (get_schedule_cmd(...))
>             return is_available;
>         return MAINT_SCHEDULER == SCHEDULER_LAUNCHCTL;
>     }
> 
>     static void resolve_auto_scheduler(enum scheduler *scheduler)
>     {
>         if (*scheduler == SCHEDULER_AUTO)
>             *scheduler = MAINT_SCHEDULER;
>     }
> 

This approach would unfortunately work only for the second patch of this 
series where a single scheduler is available on each platform.
With the third patch of this series, `resolve_auto_scheduler` doesn’t return a 
value that is fully determined at compilation time anymore.
On Linux, both `crontab` and `systemd-timers` are susceptible to be available 
and this is checked at runtime.
So, with the third patch of this series, it wouldn’t be possible anymore to 
define a single value for `MAINT_SCHEDULER` and to base 
`resolve_auto_scheduler` on it.

…
> > +                       PARSE_OPT_NONEG, maintenance_opt_scheduler),
> > +               OPT_END()
> > +       };
> > diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
> > @@ -494,8 +494,21 @@ test_expect_success !MINGW 'register and unregister
> > with regex metacharacters' ' +test_expect_success 'start
> > --scheduler=<scheduler>' '
> > +       test_expect_code 129 git maintenance start --scheduler=foo 2>err
> > &&
> > +       test_i18ngrep "unrecognized --scheduler argument" err &&
> > +
> > +       test_expect_code 129 git maintenance start --no-scheduler 2>err &&
> > +       test_i18ngrep "unknown option" err &&
> > +
> > +       test_expect_code 128 \
> > +               env
> > GIT_TEST_MAINT_SCHEDULER="launchctl:true,schtasks:true" \ +              
> > git maintenance start --scheduler=crontab 2>err && +       test_i18ngrep
> > "fatal: crontab scheduler is not available" err +'
> 
> Why does this test care about the exact exit codes rather than simply
> using test_must_fail() as is typically done elsewhere in the test
> suite, especially since we're also checking the error message itself?
> Am I missing some non-obvious property of the error codes?

I have no strong opinion on this.
I only mimicked the `help text` test that is at the top of the `t7900-
maintenance.sh` file as it was also testing invalid commands and checking the 
resulting error message.

> I don't see `auto` being tested anywhere. Do we want such a test? (It
> seems like it should be doable, though perhaps the complexity is too
> high -- I haven't thought it through fully.)

My main problem with the `auto` is that a big part of its logic is determined 
at compilation time with `#if` statements based on the platform.
And it seems that the tests are designed so far to test the same things on all 
platforms.
A solution could be to completely get rid of the platform `#if` statements and 
to turn all the detection logic as runtime tests.
But it would mean that, for ex., the git binary for Linux would systematically 
check if the MacOS and Windows specific schedulers are available.

Cheers,
Lénaïc.



^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v7 0/3] maintenance: add support for systemd timers on Linux
  2021-07-02 14:25           ` [PATCH v7 0/3] " Lénaïc Huard
                               ` (2 preceding siblings ...)
  2021-07-02 14:25             ` [PATCH v7 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
@ 2021-07-02 18:18             ` Junio C Hamano
  2021-07-06 13:18             ` Phillip Wood
  2021-08-23 20:40             ` [PATCH v8 " Lénaïc Huard
  5 siblings, 0 replies; 138+ messages in thread
From: Junio C Hamano @ 2021-07-02 18:18 UTC (permalink / raw)
  To: Lénaïc Huard
  Cc: git, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King

Lénaïc Huard <lenaic@lhuard.fr> writes:

> Hello,
>
> Please find hereafter a new reroll of my patchset to add support for
> systemd timers on Linux for the `git maintenance start` command.
>
> The changes compared to the previous version address the remarks
> raised during the previous review.

which are...?

>
> The patches are:
>
> * cache.h: Introduce a generic "xdg_config_home_for(…)" function
> * maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
> * maintenance: add support for systemd timers on Linux

Thanks, will replace.

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v7 0/3] maintenance: add support for systemd timers on Linux
  2021-07-02 14:25           ` [PATCH v7 0/3] " Lénaïc Huard
                               ` (3 preceding siblings ...)
  2021-07-02 18:18             ` [PATCH v7 0/3] " Junio C Hamano
@ 2021-07-06 13:18             ` Phillip Wood
  2021-08-23 20:40             ` [PATCH v8 " Lénaïc Huard
  5 siblings, 0 replies; 138+ messages in thread
From: Phillip Wood @ 2021-07-06 13:18 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Martin Ågren, Ævar Arnfjörð Bjarmason,
	Bagas Sanjaya, brian m . carlson, Johannes Schindelin, Jeff King

Hi  Lénaïc

On 02/07/2021 15:25, Lénaïc Huard wrote:
> Hello,
> 
> Please find hereafter a new reroll of my patchset to add support for
> systemd timers on Linux for the `git maintenance start` command.
> 
> The changes compared to the previous version address the remarks
> raised during the previous review.
> ... 
> Diff-intervalle contre v6 :
> -:  ---------- > 1:  899b11ed5b cache.h: Introduce a generic "xdg_config_home_for(…)" function
> 1:  604627f347 ! 2:  f3e2f0256b maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
>      @@ Documentation/git-maintenance.txt: OPTIONS
>        
>       +--scheduler=auto|crontab|launchctl|schtasks::
>       +	When combined with the `start` subcommand, specify the scheduler
>      -+	to use to run the hourly, daily and weekly executions of
>      ++	for running the hourly, daily and weekly executions of
>       +	`git maintenance run`.
>      -+	The possible values for `<scheduler>` depend on the system: `crontab`
>      -+	is available on POSIX systems, `launchctl` is available on
>      -+	MacOS and `schtasks` is available on Windows.
>      -+	By default or when `auto` is specified, a suitable scheduler for
>      -+	the system is used. On MacOS, `launchctl` is used. On Windows,
>      -+	`schtasks` is used. On all other systems, `crontab` is used.
>      ++	Possible values for `<scheduler>` are `auto`, `crontab` (POSIX),
>      ++	`launchctl` (macOS), and `schtasks` (Windows).
>      ++	When `auto` is specified, the appropriate platform-specific
>      ++	scheduler is used. Default is `auto`.
>       +
>        
>        TROUBLESHOOTING
>      @@ builtin/gc.c: static const char *get_frequency(enum schedule_priority schedule)
>        	}
>        }
>        
>      ++/*
>      ++ * get_schedule_cmd` reads the GIT_TEST_MAINT_SCHEDULER environment variable
>      ++ * to mock the schedulers that `git maintenance start` rely on.
>      ++ *
>      ++ * For test purpose, GIT_TEST_MAINT_SCHEDULER can be set to a comma-separated
>      ++ * list of colon-separated key/value pairs where each pair contains a scheduler
>      ++ * and its corresponding mock.
>      ++ *
>      ++ * * If $GET_TEST_MAINT_SCHEDULER is not set, return false and leave the
>      ++ *   arguments unmodified.
>      ++ *
>      ++ * * If $GET_TEST_MAINT_SCHEDULER is set, return true.
>      ++ *   In this case, the *cmd value is read as input.
>      ++ *
>      ++ *   * if the input value *cmd is the key of one of the comma-separated list
>      ++ *     item, then *is_available is set to true and *cmd is modified and becomes
>      ++ *     the mock command.
>      ++ *
>      ++ *   * if the input value *cmd isn’t the key of any of the comma-separated list
>      ++ *     item, then *is_available is set to false.
>      ++ *
>      ++ * Ex.:
>      ++ *   GIT_TEST_MAINT_SCHEDULER not set
>      ++ *     ┏━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
>      ++ *     ┃ Input ┃                     Output                      ┃
>      ++ *     ┃ *cmd  ┃ return code │       *cmd        │ *is_available ┃
>      ++ *     ┣━━━━━━━╋━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━┫
>      ++ *     ┃ "foo" ┃    false    │ "foo" (unchanged) │  (unchanged)  ┃
>      ++ *     ┗━━━━━━━┻━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━┛
>      ++ *
>      ++ *   GIT_TEST_MAINT_SCHEDULER set to “foo:./mock_foo.sh,bar:./mock_bar.sh”
>      ++ *     ┏━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
>      ++ *     ┃ Input ┃                     Output                      ┃
>      ++ *     ┃ *cmd  ┃ return code │       *cmd        │ *is_available ┃
>      ++ *     ┣━━━━━━━╋━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━┫
>      ++ *     ┃ "foo" ┃    true     │  "./mock.foo.sh"  │     true      ┃
>      ++ *     ┃ "qux" ┃    true     │ "qux" (unchanged) │     false     ┃
>      ++ *     ┗━━━━━━━┻━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━┛
>      ++ */
>       +static int get_schedule_cmd(const char **cmd, int *is_available)
>       +{
>       +	char *item;
>      @@ builtin/gc.c: static int schtasks_schedule_task(const char *exec_path, enum sche
>       +	int is_available;
>       +	struct child_process child = CHILD_PROCESS_INIT;
>       +
>      -+	if (get_schedule_cmd(&cmd, &is_available) && !is_available)
>      -+		return 0;
>      ++	if (get_schedule_cmd(&cmd, &is_available))
>      ++		return is_available;

This fixes the bug that Eric found with the last version - excellent.

>       +
>       +	strvec_split(&child.args, cmd);
>       +	strvec_push(&child.args, "-l");
>      @@ builtin/gc.c: static int crontab_update_schedule(int run_maintenance, int fd, co
>       +	enum scheduler scheduler;
>       +};
>       +
>      -+static void resolve_auto_scheduler(enum scheduler *scheduler)
>      ++static enum scheduler resolve_scheduler(enum scheduler scheduler)
>       +{
>      -+	if (*scheduler != SCHEDULER_AUTO)
>      -+		return;
>      ++	if (scheduler != SCHEDULER_AUTO)
>      ++		return scheduler;
>       +
>        #if defined(__APPLE__)
>       -static const char platform_scheduler[] = "launchctl";
>      -+	*scheduler = SCHEDULER_LAUNCHCTL;
>      -+	return;
>      ++	return SCHEDULER_LAUNCHCTL;
>       +
>        #elif defined(GIT_WINDOWS_NATIVE)
>       -static const char platform_scheduler[] = "schtasks";
>      -+	*scheduler = SCHEDULER_SCHTASKS;
>      -+	return;
>      ++	return SCHEDULER_SCHTASKS;
>       +
>        #else
>       -static const char platform_scheduler[] = "crontab";
>      -+	*scheduler = SCHEDULER_CRON;
>      -+	return;
>      ++	return SCHEDULER_CRON;
>        #endif
>       +}

This is one of the changes that Eric suggested, I agree it improves the 
code.

Thanks for your work on these patches, I've scanned the rest of the 
range-diff and I'd be happy to see this version merged

Best Wishes

Phillip

>      @@ builtin/gc.c: static int crontab_update_schedule(int run_maintenance, int fd, co
>       +	if (scheduler == SCHEDULER_INVALID)
>       +		BUG("invalid scheduler");
>       +	if (scheduler == SCHEDULER_AUTO)
>      -+		BUG("resolve_auto_scheduler should have been called before");
>      ++		BUG("resolve_scheduler should have been called before");
>       +
>       +	if (!scheduler_fn[scheduler].is_available())
>       +		die(_("%s scheduler is not available"),
>      @@ builtin/gc.c: static int crontab_update_schedule(int run_maintenance, int fd, co
>       +	struct option options[] = {
>       +		OPT_CALLBACK_F(
>       +			0, "scheduler", &opts.scheduler, N_("scheduler"),
>      -+			N_("scheduler to use to trigger git maintenance run"),
>      ++			N_("scheduler to trigger git maintenance run"),
>       +			PARSE_OPT_NONEG, maintenance_opt_scheduler),
>       +		OPT_END()
>       +	};
>      @@ builtin/gc.c: static int crontab_update_schedule(int run_maintenance, int fd, co
>       +	if (argc)
>       +		usage_with_options(builtin_maintenance_start_usage, options);
>       +
>      -+	resolve_auto_scheduler(&opts.scheduler);
>      ++	opts.scheduler = resolve_scheduler(opts.scheduler);
>       +	validate_scheduler(opts.scheduler);
>       +
>        	if (maintenance_register())
> 2:  29628b5a92 ! 3:  0ea5b2fc45 maintenance: add support for systemd timers on Linux
>      @@ Documentation/git-maintenance.txt: OPTIONS
>       ---scheduler=auto|crontab|launchctl|schtasks::
>       +--scheduler=auto|crontab|systemd-timer|launchctl|schtasks::
>        	When combined with the `start` subcommand, specify the scheduler
>      - 	to use to run the hourly, daily and weekly executions of
>      + 	for running the hourly, daily and weekly executions of
>        	`git maintenance run`.
>      - 	The possible values for `<scheduler>` depend on the system: `crontab`
>      --	is available on POSIX systems, `launchctl` is available on
>      --	MacOS and `schtasks` is available on Windows.
>      -+	is available on POSIX systems, `systemd-timer` is available on Linux
>      -+	systems, `launchctl` is available on MacOS and `schtasks` is available
>      -+	on Windows.
>      - 	By default or when `auto` is specified, a suitable scheduler for
>      - 	the system is used. On MacOS, `launchctl` is used. On Windows,
>      --	`schtasks` is used. On all other systems, `crontab` is used.
>      -+	`schtasks` is used. On Linux, `systemd-timer` is used if user systemd
>      -+	timers are available, otherwise, `crontab` is used. On all other systems,
>      -+	`crontab` is used.
>      +-	Possible values for `<scheduler>` are `auto`, `crontab` (POSIX),
>      +-	`launchctl` (macOS), and `schtasks` (Windows).
>      +-	When `auto` is specified, the appropriate platform-specific
>      +-	scheduler is used. Default is `auto`.
>      ++	Possible values for `<scheduler>` are `auto`, `crontab`
>      ++	(POSIX), `systemd-timer` (Linux), `launchctl` (macOS), and
>      ++	`schtasks` (Windows). When `auto` is specified, the
>      ++	appropriate platform-specific scheduler is used; on Linux,
>      ++	`systemd-timer` is used if available, otherwise
>      ++	`crontab`. Default is `auto`.
>        
>        
>        TROUBLESHOOTING
>      @@ builtin/gc.c: static enum scheduler parse_scheduler(const char *value)
>        	else if (!strcasecmp(value, "launchctl"))
>        		return SCHEDULER_LAUNCHCTL;
>        	else if (!strcasecmp(value, "schtasks"))
>      -@@ builtin/gc.c: static void resolve_auto_scheduler(enum scheduler *scheduler)
>      - 	*scheduler = SCHEDULER_SCHTASKS;
>      - 	return;
>      +@@ builtin/gc.c: static enum scheduler resolve_scheduler(enum scheduler scheduler)
>      + #elif defined(GIT_WINDOWS_NATIVE)
>      + 	return SCHEDULER_SCHTASKS;
>        
>       +#elif defined(__linux__)
>       +	if (is_systemd_timer_available())
>      -+		*scheduler = SCHEDULER_SYSTEMD;
>      ++		return SCHEDULER_SYSTEMD;
>       +	else if (is_crontab_available())
>      -+		*scheduler = SCHEDULER_CRON;
>      ++		return SCHEDULER_CRON;
>       +	else
>       +		die(_("neither systemd timers nor crontab are available"));
>      -+	return;
>       +
>        #else
>      - 	*scheduler = SCHEDULER_CRON;
>      - 	return;
>      + 	return SCHEDULER_CRON;
>      + #endif
>       
>        ## t/t7900-maintenance.sh ##
>       @@ t/t7900-maintenance.sh: test_xmllint () {
> 


^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v7 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-07-02 14:25             ` [PATCH v7 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
@ 2021-07-06 19:56               ` Ævar Arnfjörð Bjarmason
  2021-07-06 20:52                 ` Junio C Hamano
                                   ` (2 more replies)
  0 siblings, 3 replies; 138+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-06 19:56 UTC (permalink / raw)
  To: Lénaïc Huard
  Cc: git, Junio C Hamano, Derrick Stolee, Derrick Stolee,
	Eric Sunshine, Đoàn Trần Công Danh,
	Felipe Contreras, Phillip Wood, Martin Ågren, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King


On Fri, Jul 02 2021, Lénaïc Huard wrote:

> + *     ┏━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
> + *     ┃ Input ┃                     Output                      ┃
> + *     ┃ *cmd  ┃ return code │       *cmd        │ *is_available ┃
> + *     ┣━━━━━━━╋━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━┫
> + *     ┃ "foo" ┃    false    │ "foo" (unchanged) │  (unchanged)  ┃
> + *     ┗━━━━━━━┻━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━┛

I wonder if we have developers for whom the non-ASCII here is an issue.

> +static int get_schedule_cmd(const char **cmd, int *is_available)
> +{
> +	char *item;
> +	char *testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
> +
> +	if (!testing)
> +		return 0;
> +
> +	if (is_available)
> +		*is_available = 0;
> +
> +	for (item = testing;;) {
> +		char *sep;
> +		char *end_item = strchr(item, ',');
> +		if (end_item)
> +			*end_item = '\0';
> +
> +		sep = strchr(item, ':');
> +		if (!sep)
> +			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
> +		*sep = '\0';
> +
> +		if (!strcmp(*cmd, item)) {
> +			*cmd = sep + 1;
> +			if (is_available)
> +				*is_available = 1;
> +			UNLEAK(testing);
> +			return 1;
> +		}

This sort of code is much more pleseant to read and work with if you use
strbuf_split_buf(). This isn't performance sensitive, so a few more
allocations is fine.

> +#ifdef __APPLE__
> +	return 1;
> +#else
> +	return 0;
> +#endif
> +}

I see this is partially a pre-existing thing in the file, but we have an
__APPLE__ already in cache.h. Perhaps define a iLAUNCHCTL_AVAILABLE
there. See e.g. 62e5ee81a39 (read-cache.c: remove #ifdef NO_PTHREADS,
2018-11-03).

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v7 3/3] maintenance: add support for systemd timers on Linux
  2021-07-02 14:25             ` [PATCH v7 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
@ 2021-07-06 20:03               ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 138+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-07-06 20:03 UTC (permalink / raw)
  To: Lénaïc Huard
  Cc: git, Junio C Hamano, Derrick Stolee, Derrick Stolee,
	Eric Sunshine, Đoàn Trần Công Danh,
	Felipe Contreras, Phillip Wood, Martin Ågren, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King


On Fri, Jul 02 2021, Lénaïc Huard wrote:

[nits]

> +#ifdef __linux__
> +
> +static int real_is_systemd_timer_available(void)
> +{
> +	struct child_process child = CHILD_PROCESS_INIT;
> +
> +	strvec_pushl(&child.args, "systemctl", "--user", "list-timers", NULL);
> +	child.no_stdin = 1;
> +	child.no_stdout = 1;
> +	child.no_stderr = 1;
> +	child.silent_exec_failure = 1;
> +
> +	if (start_command(&child))
> +		return 0;
> +	if (finish_command(&child))
> +		return 0;
> +	return 1;
> +}
> +
> +#else
> +
> +static int real_is_systemd_timer_available(void)
> +{
> +	return 0;
> +}
> +
> +#endif

Ditto the Apple macro, i.e. if most/all of the code complies better to
just compile it on all platforms.

In 2/3 we have:
	
	+struct maintenance_start_opts {
	+	enum scheduler scheduler;
	+};
	
It's not mentioned (perhaps discussed in previous rounds) but I assumed
the struct would grow in 3/3, but it didn't. Why not pass "enum
scheduler " around directly?

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v7 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-07-06 19:56               ` Ævar Arnfjörð Bjarmason
@ 2021-07-06 20:52                 ` Junio C Hamano
  2021-07-13  0:15                   ` Jeff King
  2021-07-06 21:18                 ` Felipe Contreras
  2021-08-23 20:06                 ` Lénaïc Huard
  2 siblings, 1 reply; 138+ messages in thread
From: Junio C Hamano @ 2021-07-06 20:52 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Lénaïc Huard, git, Derrick Stolee, Derrick Stolee,
	Eric Sunshine, Đoàn Trần Công Danh,
	Felipe Contreras, Phillip Wood, Martin Ågren, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> On Fri, Jul 02 2021, Lénaïc Huard wrote:
>
>> + *     ┏━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
>> + *     ┃ Input ┃                     Output                      ┃
>> + *     ┃ *cmd  ┃ return code │       *cmd        │ *is_available ┃
>> + *     ┣━━━━━━━╋━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━┫
>> + *     ┃ "foo" ┃    false    │ "foo" (unchanged) │  (unchanged)  ┃
>> + *     ┗━━━━━━━┻━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━┛
>
> I wonder if we have developers for whom the non-ASCII here is an issue.

I do have an issue myself ;-) but I can survive.  I do not know
about others.

> This sort of code is much more pleseant to read and work with if you use
> strbuf_split_buf(). This isn't performance sensitive, so a few more
> allocations is fine.

Please do not encourage use of strbuf_split_buf().  It is a
misdesigned API as it rarely is justifyable to have an array, each
element of which can be independently tweaked by being strbuf.  We
are not implementing a text editor after all ;-)

A helper function that takes a string and returns a strvec would be
a good fit, though.

>> +#ifdef __APPLE__
>> +	return 1;
>> +#else
>> +	return 0;
>> +#endif
>> +}
>
> I see this is partially a pre-existing thing in the file, but we have an
> __APPLE__ already in cache.h. Perhaps define a iLAUNCHCTL_AVAILABLE
> there. See e.g. 62e5ee81a39 (read-cache.c: remove #ifdef NO_PTHREADS,
> 2018-11-03).

Excellent suggestion.

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v7 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-07-06 19:56               ` Ævar Arnfjörð Bjarmason
  2021-07-06 20:52                 ` Junio C Hamano
@ 2021-07-06 21:18                 ` Felipe Contreras
  2021-08-23 20:06                 ` Lénaïc Huard
  2 siblings, 0 replies; 138+ messages in thread
From: Felipe Contreras @ 2021-07-06 21:18 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason, Lénaïc Huard
  Cc: git, Junio C Hamano, Derrick Stolee, Derrick Stolee,
	Eric Sunshine, Đoàn Trần Công Danh,
	Felipe Contreras, Phillip Wood, Martin Ågren, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King

Ævar Arnfjörð Bjarmason wrote:
> 
> On Fri, Jul 02 2021, Lénaïc Huard wrote:
> 
> > + *     ┏━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
> > + *     ┃ Input ┃                     Output                      ┃
> > + *     ┃ *cmd  ┃ return code │       *cmd        │ *is_available ┃
> > + *     ┣━━━━━━━╋━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━┫
> > + *     ┃ "foo" ┃    false    │ "foo" (unchanged) │  (unchanged)  ┃
> > + *     ┗━━━━━━━┻━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━┛
> 
> I wonder if we have developers for whom the non-ASCII here is an issue.

It's not an issue per se for me, it just looks weird to my eyes, like
seeing an emoji in the middle of a text file.

It's not wrong, I'm just not used to it. Really not used to it.

-- 
Felipe Contreras

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v7 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-07-06 20:52                 ` Junio C Hamano
@ 2021-07-13  0:15                   ` Jeff King
  2021-07-13  2:22                     ` Eric Sunshine
  0 siblings, 1 reply; 138+ messages in thread
From: Jeff King @ 2021-07-13  0:15 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Ævar Arnfjörð Bjarmason, Lénaïc Huard,
	git, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin

On Tue, Jul 06, 2021 at 01:52:12PM -0700, Junio C Hamano wrote:

> > This sort of code is much more pleseant to read and work with if you use
> > strbuf_split_buf(). This isn't performance sensitive, so a few more
> > allocations is fine.
> 
> Please do not encourage use of strbuf_split_buf().  It is a
> misdesigned API as it rarely is justifyable to have an array, each
> element of which can be independently tweaked by being strbuf.  We
> are not implementing a text editor after all ;-)

Very much agreed on avoiding strbuf_split_buf(). My usual go-to is
string_list_split(), which I think would work here for splitting on ":".

> A helper function that takes a string and returns a strvec would be
> a good fit, though.

I was going to second that, but I see we already have one. :) Dscho
introduced it in c5aa6db64f (argv_array: offer to split a string by
whitespace, 2018-04-25), and then it later became strvec_split().

And indeed, Lénaïc's patches use it elsewhere. I think it doesn't work
in this instance because it can't take an arbitrary delimiter. But I
wouldn't at all mind seeing it grow that feature (and I suspect it could
even share some code with string_list_split(), but didn't look).

-Peff

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v7 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-07-13  0:15                   ` Jeff King
@ 2021-07-13  2:22                     ` Eric Sunshine
  2021-07-13  3:56                       ` Jeff King
  2021-07-13  7:04                       ` Bagas Sanjaya
  0 siblings, 2 replies; 138+ messages in thread
From: Eric Sunshine @ 2021-07-13  2:22 UTC (permalink / raw)
  To: Jeff King
  Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason,
	Lénaïc Huard, Git List, Derrick Stolee, Derrick Stolee,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin

On Mon, Jul 12, 2021 at 8:16 PM Jeff King <peff@peff.net> wrote:
> On Tue, Jul 06, 2021 at 01:52:12PM -0700, Junio C Hamano wrote:
> > A helper function that takes a string and returns a strvec would be
> > a good fit, though.
>
> I was going to second that, but I see we already have one. :) Dscho
> introduced it in c5aa6db64f (argv_array: offer to split a string by
> whitespace, 2018-04-25), and then it later became strvec_split().
>
> And indeed, Lénaïc's patches use it elsewhere. I think it doesn't work
> in this instance because it can't take an arbitrary delimiter. But I
> wouldn't at all mind seeing it grow that feature (and I suspect it could
> even share some code with string_list_split(), but didn't look).

Since Lénaïc is a relative newcomer to the project, can we, as
reviewers, be clear that we don't expect him to perform the task of
generalizing strvec_split() just to get this series -- which is
already at v7 -- landed? I gave the previous round a pretty thorough
going-over and -- aside from one minor test-time bug -- didn't find
any show-stoppers which should prevent it from landing. While it may
be the case that the series has a superficial wart here and there
(such as #ifdef's in function bodies, and non-ASCII fancy comment
boxes), the review comments on the latest round have pretty much all
been subjective; I haven't seen any outright actionable observations.
Extra polishing based upon the subjective review comments can always
be done later atop Lénaïc's series (if someone -- not necessarily
Lénaïc -- wants to do so) without asking him for endless re-rolls.

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v7 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-07-13  2:22                     ` Eric Sunshine
@ 2021-07-13  3:56                       ` Jeff King
  2021-07-13  5:17                         ` Eric Sunshine
  2021-07-13  7:04                       ` Bagas Sanjaya
  1 sibling, 1 reply; 138+ messages in thread
From: Jeff King @ 2021-07-13  3:56 UTC (permalink / raw)
  To: Eric Sunshine
  Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason,
	Lénaïc Huard, Git List, Derrick Stolee, Derrick Stolee,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin

On Mon, Jul 12, 2021 at 10:22:22PM -0400, Eric Sunshine wrote:

> On Mon, Jul 12, 2021 at 8:16 PM Jeff King <peff@peff.net> wrote:
> > On Tue, Jul 06, 2021 at 01:52:12PM -0700, Junio C Hamano wrote:
> > > A helper function that takes a string and returns a strvec would be
> > > a good fit, though.
> >
> > I was going to second that, but I see we already have one. :) Dscho
> > introduced it in c5aa6db64f (argv_array: offer to split a string by
> > whitespace, 2018-04-25), and then it later became strvec_split().
> >
> > And indeed, Lénaïc's patches use it elsewhere. I think it doesn't work
> > in this instance because it can't take an arbitrary delimiter. But I
> > wouldn't at all mind seeing it grow that feature (and I suspect it could
> > even share some code with string_list_split(), but didn't look).
> 
> Since Lénaïc is a relative newcomer to the project, can we, as
> reviewers, be clear that we don't expect him to perform the task of
> generalizing strvec_split() just to get this series -- which is
> already at v7 -- landed?

Yeah, sorry if I was unclear on that. That is absolutely not a
requirement for this series.

(I do not have an opinion on Ævar's original question about using a
split function rather than open-coding. _If_ we were to do that,
string_list_split() would be the sensible existing mechanism. But it is
also not worth derailing the series for).

-Peff

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v7 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-07-13  3:56                       ` Jeff King
@ 2021-07-13  5:17                         ` Eric Sunshine
  0 siblings, 0 replies; 138+ messages in thread
From: Eric Sunshine @ 2021-07-13  5:17 UTC (permalink / raw)
  To: Jeff King
  Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason,
	Lénaïc Huard, Git List, Derrick Stolee, Derrick Stolee,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin

On Mon, Jul 12, 2021 at 11:56 PM Jeff King <peff@peff.net> wrote:
> On Mon, Jul 12, 2021 at 10:22:22PM -0400, Eric Sunshine wrote:
> > On Mon, Jul 12, 2021 at 8:16 PM Jeff King <peff@peff.net> wrote:
> > > And indeed, Lénaïc's patches use it elsewhere. I think it doesn't work
> > > in this instance because it can't take an arbitrary delimiter. But I
> > > wouldn't at all mind seeing it grow that feature (and I suspect it could
> > > even share some code with string_list_split(), but didn't look).
> >
> > Since Lénaïc is a relative newcomer to the project, can we, as
> > reviewers, be clear that we don't expect him to perform the task of
> > generalizing strvec_split() just to get this series -- which is
> > already at v7 -- landed?
>
> Yeah, sorry if I was unclear on that. That is absolutely not a
> requirement for this series.

Thanks, and sorry if my response seemed to be aimed only at your
reply. It wasn't. It was a reaction to a number of recent reviews --
in general -- containing only subjective comments or a mix of
subjective comments and genuine actionable items without necessarily
making it clear which is which. Such ambiguity is hard on newcomers,
who may end up doing unnecessary work trying to get their patches
accepted, or might scare them away altogether.

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v7 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-07-13  2:22                     ` Eric Sunshine
  2021-07-13  3:56                       ` Jeff King
@ 2021-07-13  7:04                       ` Bagas Sanjaya
  1 sibling, 0 replies; 138+ messages in thread
From: Bagas Sanjaya @ 2021-07-13  7:04 UTC (permalink / raw)
  To: Eric Sunshine, Jeff King
  Cc: Junio C Hamano, Ævar Arnfjörð Bjarmason,
	Lénaïc Huard, Git List, Derrick Stolee, Derrick Stolee,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren, brian m . carlson,
	Johannes Schindelin

On 13/07/21 09.22, Eric Sunshine wrote:
> Since Lénaïc is a relative newcomer to the project, can we, as
> reviewers, be clear that we don't expect him to perform the task of
> generalizing strvec_split() just to get this series -- which is
> already at v7 -- landed? I gave the previous round a pretty thorough
> going-over and -- aside from one minor test-time bug -- didn't find
> any show-stoppers which should prevent it from landing. While it may
> be the case that the series has a superficial wart here and there
> (such as #ifdef's in function bodies, and non-ASCII fancy comment
> boxes), the review comments on the latest round have pretty much all
> been subjective; I haven't seen any outright actionable observations.
> Extra polishing based upon the subjective review comments can always
> be done later atop Lénaïc's series (if someone -- not necessarily
> Lénaïc -- wants to do so) without asking him for endless re-rolls.
> 

In such situation when there is endless re-roll of patch series due to 
subjective reviews, we can ask for final call from the maintainer, right?

-- 
An old man doll... just what I always wanted! - Clara

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v5 0/3] add support for systemd timers on Linux
  2021-06-08 13:39       ` [PATCH v5 0/3] " Lénaïc Huard
                           ` (5 preceding siblings ...)
  2021-06-12 16:50         ` [PATCH v6 0/3] maintenance: " Lénaïc Huard
@ 2021-08-17 17:22         ` Derrick Stolee
  2021-08-17 19:43           ` Phillip Wood
  2021-08-18  5:56           ` Lénaïc Huard
  6 siblings, 2 replies; 138+ messages in thread
From: Derrick Stolee @ 2021-08-17 17:22 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin

On 6/8/2021 9:39 AM, Lénaïc Huard wrote:
> Hello,
> 
> I’ve reworked this submission based on the valuable feedback I’ve received.
> Thanks again for it!

Hi Lénaïc!

I'm replying to your series because it appears you did not see our discussion
of this topic in the What's Cooking thread a couple weeks ago [1] and might
miss the discussion that began today [2].

[1] https://lore.kernel.org/git/4aed0293-6a48-d370-3b72-496b7c631cb5@gmail.com/
[2] https://lore.kernel.org/git/7a4b1238-5c3b-4c08-0e9d-511f857f9c38@gmail.com/

The proposal I give in [2] is that I can forward-fix the remaining comments
OR re-submit the patches with a new patch and some edits to your current patches.
(You would remain author of the patches you wrote.)

None of that is important if you plan to submit a v6 that responds to the
remaining feedback (summarized in [1]).

I'll hold off for a couple days to give you a chance to read and respond.

Thanks,
-Stolee

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v5 0/3] add support for systemd timers on Linux
  2021-08-17 17:22         ` [PATCH v5 0/3] " Derrick Stolee
@ 2021-08-17 19:43           ` Phillip Wood
  2021-08-17 20:29             ` Derrick Stolee
  2021-08-18  5:56           ` Lénaïc Huard
  1 sibling, 1 reply; 138+ messages in thread
From: Phillip Wood @ 2021-08-17 19:43 UTC (permalink / raw)
  To: Derrick Stolee, Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Martin Ågren, Ævar Arnfjörð Bjarmason,
	Bagas Sanjaya, brian m . carlson, Johannes Schindelin

On 17/08/2021 18:22, Derrick Stolee wrote:
> On 6/8/2021 9:39 AM, Lénaïc Huard wrote:
>> Hello,
>>
>> I’ve reworked this submission based on the valuable feedback I’ve received.
>> Thanks again for it!
> 
> Hi Lénaïc!
> 
> I'm replying to your series because it appears you did not see our discussion
> of this topic in the What's Cooking thread a couple weeks ago [1] and might
> miss the discussion that began today [2].
> 
> [1] https://lore.kernel.org/git/4aed0293-6a48-d370-3b72-496b7c631cb5@gmail.com/
> [2] https://lore.kernel.org/git/7a4b1238-5c3b-4c08-0e9d-511f857f9c38@gmail.com/
> 
> The proposal I give in [2] is that I can forward-fix the remaining comments
> OR re-submit the patches with a new patch and some edits to your current patches.
> (You would remain author of the patches you wrote.)
> 
> None of that is important if you plan to submit a v6 that responds to the
> remaining feedback (summarized in [1]).

I think you mean v8, v7[1] is what is in seen at the moment. There was a 
suggestion at the time that a v8 would not be needed[2]

Best Wishes

Phillip

[1] https://public-inbox.org/git/20210702142556.99864-1-lenaic@lhuard.fr/
[2] https://public-inbox.org/git/YO0O9JHtnYrk9qRm@coredump.intra.peff.net/

> I'll hold off for a couple days to give you a chance to read and respond.
> 
> Thanks,
> -Stolee
> 

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v5 0/3] add support for systemd timers on Linux
  2021-08-17 19:43           ` Phillip Wood
@ 2021-08-17 20:29             ` Derrick Stolee
  0 siblings, 0 replies; 138+ messages in thread
From: Derrick Stolee @ 2021-08-17 20:29 UTC (permalink / raw)
  To: phillip.wood, Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Martin Ågren, Ævar Arnfjörð Bjarmason,
	Bagas Sanjaya, brian m . carlson, Johannes Schindelin

On 8/17/2021 3:43 PM, Phillip Wood wrote:
> On 17/08/2021 18:22, Derrick Stolee wrote:
>> On 6/8/2021 9:39 AM, Lénaïc Huard wrote:
>>> Hello,
>>>
>>> I’ve reworked this submission based on the valuable feedback I’ve received.
>>> Thanks again for it!
>>
>> Hi Lénaïc!
>>
>> I'm replying to your series because it appears you did not see our discussion
>> of this topic in the What's Cooking thread a couple weeks ago [1] and might
>> miss the discussion that began today [2].
>>
>> [1] https://lore.kernel.org/git/4aed0293-6a48-d370-3b72-496b7c631cb5@gmail.com/
>> [2] https://lore.kernel.org/git/7a4b1238-5c3b-4c08-0e9d-511f857f9c38@gmail.com/
>>
>> The proposal I give in [2] is that I can forward-fix the remaining comments
>> OR re-submit the patches with a new patch and some edits to your current patches.
>> (You would remain author of the patches you wrote.)
>>
>> None of that is important if you plan to submit a v6 that responds to the
>> remaining feedback (summarized in [1]).
> 
> I think you mean v8, v7[1] is what is in seen at the moment. There was a suggestion at the time that a v8 would not be needed[2]
 
I do, thanks. Sorry for picking the wrong version to start
this discussion.

Thanks,
-Stolee

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v5 0/3] add support for systemd timers on Linux
  2021-08-17 17:22         ` [PATCH v5 0/3] " Derrick Stolee
  2021-08-17 19:43           ` Phillip Wood
@ 2021-08-18  5:56           ` Lénaïc Huard
  2021-08-18 13:28             ` Derrick Stolee
  1 sibling, 1 reply; 138+ messages in thread
From: Lénaïc Huard @ 2021-08-18  5:56 UTC (permalink / raw)
  To: git, Derrick Stolee
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin

Le mardi 17 août 2021, 19:22:05 CEST Derrick Stolee a écrit :
> On 6/8/2021 9:39 AM, Lénaïc Huard wrote:
> > Hello,
> > 
> > I’ve reworked this submission based on the valuable feedback I’ve
> > received.
> > Thanks again for it!
> 
> Hi Lénaïc!
> 
> I'm replying to your series because it appears you did not see our
> discussion of this topic in the What's Cooking thread a couple weeks ago
> [1] and might miss the discussion that began today [2].
> 
> [1]
> https://lore.kernel.org/git/4aed0293-6a48-d370-3b72-496b7c631cb5@gmail.com/
> [2]
> https://lore.kernel.org/git/7a4b1238-5c3b-4c08-0e9d-511f857f9c38@gmail.com/
> 
> The proposal I give in [2] is that I can forward-fix the remaining comments
> OR re-submit the patches with a new patch and some edits to your current
> patches. (You would remain author of the patches you wrote.)
> 
> None of that is important if you plan to submit a v6 that responds to the
> remaining feedback (summarized in [1]).
> 
> I'll hold off for a couple days to give you a chance to read and respond.
> 
> Thanks,
> -Stolee

Hello,

Sorry for the silence. I just happened to be in holiday for the past few weeks 
and did not have access to my mails.
I can catch up the discussions I missed and try to address the remaining 
concerns in a new re-roll.

Cheers,
Lénaïc.




^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v5 0/3] add support for systemd timers on Linux
  2021-08-18  5:56           ` Lénaïc Huard
@ 2021-08-18 13:28             ` Derrick Stolee
  2021-08-18 18:23               ` Junio C Hamano
  0 siblings, 1 reply; 138+ messages in thread
From: Derrick Stolee @ 2021-08-18 13:28 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin

On 8/18/2021 1:56 AM, Lénaïc Huard wrote:
> Le mardi 17 août 2021, 19:22:05 CEST Derrick Stolee a écrit :
>> None of that is important if you plan to submit a v6 that responds to the
>> remaining feedback (summarized in [1]).
>>
>> I'll hold off for a couple days to give you a chance to read and respond.
...
> Hello,
> 
> Sorry for the silence. I just happened to be in holiday for the past few weeks 
> and did not have access to my mails.
> I can catch up the discussions I missed and try to address the remaining 
> concerns in a new re-roll.

Perfect! Welcome back!

-Stolee

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v5 0/3] add support for systemd timers on Linux
  2021-08-18 13:28             ` Derrick Stolee
@ 2021-08-18 18:23               ` Junio C Hamano
  0 siblings, 0 replies; 138+ messages in thread
From: Junio C Hamano @ 2021-08-18 18:23 UTC (permalink / raw)
  To: Derrick Stolee
  Cc: Lénaïc Huard, git, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin

Derrick Stolee <stolee@gmail.com> writes:

> On 8/18/2021 1:56 AM, Lénaïc Huard wrote:
>> Le mardi 17 août 2021, 19:22:05 CEST Derrick Stolee a écrit :
>>> None of that is important if you plan to submit a v6 that responds to the
>>> remaining feedback (summarized in [1]).
>>>
>>> I'll hold off for a couple days to give you a chance to read and respond.
> ...
>> Hello,
>> 
>> Sorry for the silence. I just happened to be in holiday for the past few weeks 
>> and did not have access to my mails.
>> I can catch up the discussions I missed and try to address the remaining 
>> concerns in a new re-roll.
>
> Perfect! Welcome back!

Thanks, both.  Looking forward to see this topic reignited and see
its happy ending ;-)



^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v7 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-07-06 19:56               ` Ævar Arnfjörð Bjarmason
  2021-07-06 20:52                 ` Junio C Hamano
  2021-07-06 21:18                 ` Felipe Contreras
@ 2021-08-23 20:06                 ` Lénaïc Huard
  2021-08-23 22:30                   ` Junio C Hamano
  2 siblings, 1 reply; 138+ messages in thread
From: Lénaïc Huard @ 2021-08-23 20:06 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Derrick Stolee, Derrick Stolee,
	Eric Sunshine, Đoàn Trần Công Danh,
	Felipe Contreras, Phillip Wood, Martin Ågren, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King

Hello,

Sorry to come back after such a long time. I just resumed working on that 
series and would have a question about a review comment.

Le mardi 6 juillet 2021, 21:56:38 CEST Ævar Arnfjörð Bjarmason a écrit :
> On Fri, Jul 02 2021, Lénaïc Huard wrote:

> > +#ifdef __APPLE__
> > +	return 1;
> > +#else
> > +	return 0;
> > +#endif
> > +}
> 
> 
> I see this is partially a pre-existing thing in the file, but we have an
> __APPLE__ already in cache.h. Perhaps define a iLAUNCHCTL_AVAILABLE
> there. See e.g. 62e5ee81a39 (read-cache.c: remove #ifdef NO_PTHREADS,
> 2018-11-03).

Is the suggestion to replace

+#ifdef __APPLE__
+	return 1;
+#else
+	return 0;
+#endif
+}

by

+ return IS_LAUNCHCTL_AVAILABLE;

and to add

#ifdef __APPLE__
#define IS_LAUNCHCTL_AVAILABLE 1
#else
#define IS_LAUNCHCTL_AVAILABLE 0
#endif

somewhere else like at the top of builtin/gc.c ?

Also, do we agree this shouldn’t be defined in cache.h ? I’m a little bit 
confused.

Kind regards,
Lénaïc.





^ permalink raw reply	[flat|nested] 138+ messages in thread

* [PATCH v8 0/3] maintenance: add support for systemd timers on Linux
  2021-07-02 14:25           ` [PATCH v7 0/3] " Lénaïc Huard
                               ` (4 preceding siblings ...)
  2021-07-06 13:18             ` Phillip Wood
@ 2021-08-23 20:40             ` Lénaïc Huard
  2021-08-23 20:40               ` [PATCH v8 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
                                 ` (4 more replies)
  5 siblings, 5 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-08-23 20:40 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King,
	Lénaïc Huard

Hello,

I’ve just resumed working on my patchset to add support for systemd
timers on Linux for the `git maintenance start` command.

The patches are:

* cache.h: Introduce a generic "xdg_config_home_for(…)" function

  This patch introduces a function to compute configuration files
  paths inside $XDG_CONFIG_HOME.
  It is used in the latest patch of this series to compute systemd
  unit files location.

  This patch is unchanged compared to its previous version.

* maintenance: `git maintenance run` learned `--scheduler=<scheduler>`

  This patch adds a new parameter to the `git maintenance run` to let
  the user choose a scheduler.

  This patch contains the following changes compared to its previous
  version:

  * Remove some UTF-8 characters in a code comment and replace them by
    ASCII ones.

  * Leverage `string_list_split` in `get_schedule_cmd` to parse the
    comma-separated list of colon-separated pairs in
    GIT_TEST_MAINT_SCHEDULER environment variable.

* maintenance: add support for systemd timers on Linux

  This patch implements the support of systemd timers on top of
  crontab scheduler on Linux systems.

  This patch is unchanged compared to its previous version.


According to [1], there were 3 changes awaited in this v8:
* The two already mentionned above (utf-8 characters and
  `string_list_split` thing)
* An improvement around the #ifdef.

I must admit I haven’t touched anything around the #ifdef in this v8
because I’m not sure what to do. I’ve just asked for some more details
in [2].

[1] https://lore.kernel.org/git/4aed0293-6a48-d370-3b72-496b7c631cb5@gmail.com/
[2] https://lore.kernel.org/git/3218082.ccbTtk1zYS@coruscant.lhuard.fr/


Best wishes,
Lénaïc.


Lénaïc Huard (3):
  cache.h: Introduce a generic "xdg_config_home_for(…)" function
  maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  maintenance: add support for systemd timers on Linux

 Documentation/git-maintenance.txt |  57 +++
 builtin/gc.c                      | 592 ++++++++++++++++++++++++++----
 cache.h                           |   7 +
 path.c                            |  13 +-
 t/t7900-maintenance.sh            | 110 +++++-
 5 files changed, 701 insertions(+), 78 deletions(-)

Diff-intervalle contre v7 :
1:  899b11ed5b = 1:  1639bd151c cache.h: Introduce a generic "xdg_config_home_for(…)" function
2:  f3e2f0256b ! 2:  ea5568269c maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
    @@ builtin/gc.c: static const char *get_frequency(enum schedule_priority schedule)
     + *
     + * Ex.:
     + *   GIT_TEST_MAINT_SCHEDULER not set
    -+ *     ┏━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
    -+ *     ┃ Input ┃                     Output                      ┃
    -+ *     ┃ *cmd  ┃ return code │       *cmd        │ *is_available ┃
    -+ *     ┣━━━━━━━╋━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━┫
    -+ *     ┃ "foo" ┃    false    │ "foo" (unchanged) │  (unchanged)  ┃
    -+ *     ┗━━━━━━━┻━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━┛
    ++ *     +-------+-------------------------------------------------+
    ++ *     | Input |                     Output                      |
    ++ *     | *cmd  | return code |       *cmd        | *is_available |
    ++ *     +-------+-------------+-------------------+---------------+
    ++ *     | "foo" |    false    | "foo" (unchanged) |  (unchanged)  |
    ++ *     +-------+-------------+-------------------+---------------+
     + *
     + *   GIT_TEST_MAINT_SCHEDULER set to “foo:./mock_foo.sh,bar:./mock_bar.sh”
    -+ *     ┏━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
    -+ *     ┃ Input ┃                     Output                      ┃
    -+ *     ┃ *cmd  ┃ return code │       *cmd        │ *is_available ┃
    -+ *     ┣━━━━━━━╋━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━┫
    -+ *     ┃ "foo" ┃    true     │  "./mock.foo.sh"  │     true      ┃
    -+ *     ┃ "qux" ┃    true     │ "qux" (unchanged) │     false     ┃
    -+ *     ┗━━━━━━━┻━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━┛
    ++ *     +-------+-------------------------------------------------+
    ++ *     | Input |                     Output                      |
    ++ *     | *cmd  | return code |       *cmd        | *is_available |
    ++ *     +-------+-------------+-------------------+---------------+
    ++ *     | "foo" |    true     |  "./mock.foo.sh"  |     true      |
    ++ *     | "qux" |    true     | "qux" (unchanged) |     false     |
    ++ *     +-------+-------------+-------------------+---------------+
     + */
     +static int get_schedule_cmd(const char **cmd, int *is_available)
     +{
    -+	char *item;
     +	char *testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
    ++	struct string_list_item *item;
    ++	struct string_list list = STRING_LIST_INIT_NODUP;
     +
     +	if (!testing)
     +		return 0;
    @@ builtin/gc.c: static const char *get_frequency(enum schedule_priority schedule)
     +	if (is_available)
     +		*is_available = 0;
     +
    -+	for (item = testing;;) {
    -+		char *sep;
    -+		char *end_item = strchr(item, ',');
    -+		if (end_item)
    -+			*end_item = '\0';
    ++	string_list_split_in_place(&list, testing, ',', -1);
    ++	for_each_string_list_item(item, &list) {
    ++		struct string_list pair = STRING_LIST_INIT_NODUP;
     +
    -+		sep = strchr(item, ':');
    -+		if (!sep)
    -+			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
    -+		*sep = '\0';
    ++		if (string_list_split_in_place(&pair, item->string, ':', 2) != 2)
    ++			continue;
     +
    -+		if (!strcmp(*cmd, item)) {
    -+			*cmd = sep + 1;
    ++		if (!strcmp(*cmd, pair.items[0].string)) {
    ++			*cmd = pair.items[1].string;
     +			if (is_available)
     +				*is_available = 1;
    ++			string_list_clear(&list, 0);
     +			UNLEAK(testing);
     +			return 1;
     +		}
    -+
    -+		if (!end_item)
    -+			break;
    -+		item = end_item + 1;
     +	}
     +
    ++	string_list_clear(&list, 0);
     +	free(testing);
     +	return 1;
     +}
3:  0ea5b2fc45 = 3:  faf56c078f maintenance: add support for systemd timers on Linux
-- 
2.33.0


^ permalink raw reply	[flat|nested] 138+ messages in thread

* [PATCH v8 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function
  2021-08-23 20:40             ` [PATCH v8 " Lénaïc Huard
@ 2021-08-23 20:40               ` Lénaïc Huard
  2021-08-23 20:40               ` [PATCH v8 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
                                 ` (3 subsequent siblings)
  4 siblings, 0 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-08-23 20:40 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King,
	Lénaïc Huard

Current implementation of `xdg_config_home(filename)` returns
`$XDG_CONFIG_HOME/git/$filename`, with the `git` subdirectory inserted
between the `XDG_CONFIG_HOME` environment variable and the parameter.

This patch introduces a `xdg_config_home_for(subdir, filename)` function
which is more generic. It only concatenates "$XDG_CONFIG_HOME", or
"$HOME/.config" if the former isn’t defined, with the parameters,
without adding `git` in between.

`xdg_config_home(filename)` is now implemented by calling
`xdg_config_home_for("git", filename)` but this new generic function can
be used to compute the configuration directory of other programs.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 cache.h |  7 +++++++
 path.c  | 13 ++++++++++---
 2 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/cache.h b/cache.h
index bd4869beee..79c4e17ea4 100644
--- a/cache.h
+++ b/cache.h
@@ -1295,6 +1295,13 @@ int is_ntfs_dotmailmap(const char *name);
  */
 int looks_like_command_line_option(const char *str);
 
+/**
+ * Return a newly allocated string with the evaluation of
+ * "$XDG_CONFIG_HOME/$subdir/$filename" if $XDG_CONFIG_HOME is non-empty, otherwise
+ * "$HOME/.config/$subdir/$filename". Return NULL upon error.
+ */
+char *xdg_config_home_for(const char *subdir, const char *filename);
+
 /**
  * Return a newly allocated string with the evaluation of
  * "$XDG_CONFIG_HOME/git/$filename" if $XDG_CONFIG_HOME is non-empty, otherwise
diff --git a/path.c b/path.c
index 7bccd830e9..1b1de3be09 100644
--- a/path.c
+++ b/path.c
@@ -1503,21 +1503,28 @@ int looks_like_command_line_option(const char *str)
 	return str && str[0] == '-';
 }
 
-char *xdg_config_home(const char *filename)
+char *xdg_config_home_for(const char *subdir, const char *filename)
 {
 	const char *home, *config_home;
 
+	assert(subdir);
 	assert(filename);
 	config_home = getenv("XDG_CONFIG_HOME");
 	if (config_home && *config_home)
-		return mkpathdup("%s/git/%s", config_home, filename);
+		return mkpathdup("%s/%s/%s", config_home, subdir, filename);
 
 	home = getenv("HOME");
 	if (home)
-		return mkpathdup("%s/.config/git/%s", home, filename);
+		return mkpathdup("%s/.config/%s/%s", home, subdir, filename);
+
 	return NULL;
 }
 
+char *xdg_config_home(const char *filename)
+{
+	return xdg_config_home_for("git", filename);
+}
+
 char *xdg_cache_home(const char *filename)
 {
 	const char *home, *cache_home;
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* [PATCH v8 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-08-23 20:40             ` [PATCH v8 " Lénaïc Huard
  2021-08-23 20:40               ` [PATCH v8 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
@ 2021-08-23 20:40               ` Lénaïc Huard
  2021-08-24 17:45                 ` Derrick Stolee
  2021-08-23 20:40               ` [PATCH v8 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
                                 ` (2 subsequent siblings)
  4 siblings, 1 reply; 138+ messages in thread
From: Lénaïc Huard @ 2021-08-23 20:40 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King,
	Lénaïc Huard

Depending on the system, different schedulers can be used to schedule
the hourly, daily and weekly executions of `git maintenance run`:
* `launchctl` for MacOS,
* `schtasks` for Windows and
* `crontab` for everything else.

`git maintenance run` now has an option to let the end-user explicitly
choose which scheduler he wants to use:
`--scheduler=auto|crontab|launchctl|schtasks`.

When `git maintenance start --scheduler=XXX` is run, it not only
registers `git maintenance run` tasks in the scheduler XXX, it also
removes the `git maintenance run` tasks from all the other schedulers to
ensure we cannot have two schedulers launching concurrent identical
tasks.

The default value is `auto` which chooses a suitable scheduler for the
system.

`git maintenance stop` doesn't have any `--scheduler` parameter because
this command will try to remove the `git maintenance run` tasks from all
the available schedulers.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 Documentation/git-maintenance.txt |   9 +
 builtin/gc.c                      | 365 ++++++++++++++++++++++++------
 t/t7900-maintenance.sh            |  55 ++++-
 3 files changed, 354 insertions(+), 75 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 1e738ad398..576290b5c6 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -179,6 +179,15 @@ OPTIONS
 	`maintenance.<task>.enabled` configured as `true` are considered.
 	See the 'TASKS' section for the list of accepted `<task>` values.
 
+--scheduler=auto|crontab|launchctl|schtasks::
+	When combined with the `start` subcommand, specify the scheduler
+	for running the hourly, daily and weekly executions of
+	`git maintenance run`.
+	Possible values for `<scheduler>` are `auto`, `crontab` (POSIX),
+	`launchctl` (macOS), and `schtasks` (Windows).
+	When `auto` is specified, the appropriate platform-specific
+	scheduler is used. Default is `auto`.
+
 
 TROUBLESHOOTING
 ---------------
diff --git a/builtin/gc.c b/builtin/gc.c
index f05d2f0a1a..9e464d4a10 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1529,6 +1529,93 @@ static const char *get_frequency(enum schedule_priority schedule)
 	}
 }
 
+/*
+ * get_schedule_cmd` reads the GIT_TEST_MAINT_SCHEDULER environment variable
+ * to mock the schedulers that `git maintenance start` rely on.
+ *
+ * For test purpose, GIT_TEST_MAINT_SCHEDULER can be set to a comma-separated
+ * list of colon-separated key/value pairs where each pair contains a scheduler
+ * and its corresponding mock.
+ *
+ * * If $GET_TEST_MAINT_SCHEDULER is not set, return false and leave the
+ *   arguments unmodified.
+ *
+ * * If $GET_TEST_MAINT_SCHEDULER is set, return true.
+ *   In this case, the *cmd value is read as input.
+ *
+ *   * if the input value *cmd is the key of one of the comma-separated list
+ *     item, then *is_available is set to true and *cmd is modified and becomes
+ *     the mock command.
+ *
+ *   * if the input value *cmd isn’t the key of any of the comma-separated list
+ *     item, then *is_available is set to false.
+ *
+ * Ex.:
+ *   GIT_TEST_MAINT_SCHEDULER not set
+ *     +-------+-------------------------------------------------+
+ *     | Input |                     Output                      |
+ *     | *cmd  | return code |       *cmd        | *is_available |
+ *     +-------+-------------+-------------------+---------------+
+ *     | "foo" |    false    | "foo" (unchanged) |  (unchanged)  |
+ *     +-------+-------------+-------------------+---------------+
+ *
+ *   GIT_TEST_MAINT_SCHEDULER set to “foo:./mock_foo.sh,bar:./mock_bar.sh”
+ *     +-------+-------------------------------------------------+
+ *     | Input |                     Output                      |
+ *     | *cmd  | return code |       *cmd        | *is_available |
+ *     +-------+-------------+-------------------+---------------+
+ *     | "foo" |    true     |  "./mock.foo.sh"  |     true      |
+ *     | "qux" |    true     | "qux" (unchanged) |     false     |
+ *     +-------+-------------+-------------------+---------------+
+ */
+static int get_schedule_cmd(const char **cmd, int *is_available)
+{
+	char *testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
+	struct string_list_item *item;
+	struct string_list list = STRING_LIST_INIT_NODUP;
+
+	if (!testing)
+		return 0;
+
+	if (is_available)
+		*is_available = 0;
+
+	string_list_split_in_place(&list, testing, ',', -1);
+	for_each_string_list_item(item, &list) {
+		struct string_list pair = STRING_LIST_INIT_NODUP;
+
+		if (string_list_split_in_place(&pair, item->string, ':', 2) != 2)
+			continue;
+
+		if (!strcmp(*cmd, pair.items[0].string)) {
+			*cmd = pair.items[1].string;
+			if (is_available)
+				*is_available = 1;
+			string_list_clear(&list, 0);
+			UNLEAK(testing);
+			return 1;
+		}
+	}
+
+	string_list_clear(&list, 0);
+	free(testing);
+	return 1;
+}
+
+static int is_launchctl_available(void)
+{
+	const char *cmd = "launchctl";
+	int is_available;
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+#ifdef __APPLE__
+	return 1;
+#else
+	return 0;
+#endif
+}
+
 static char *launchctl_service_name(const char *frequency)
 {
 	struct strbuf label = STRBUF_INIT;
@@ -1555,19 +1642,17 @@ static char *launchctl_get_uid(void)
 	return xstrfmt("gui/%d", getuid());
 }
 
-static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
+static int launchctl_boot_plist(int enable, const char *filename)
 {
+	const char *cmd = "launchctl";
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
 	char *uid = launchctl_get_uid();
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&child.args, cmd);
-	if (enable)
-		strvec_push(&child.args, "bootstrap");
-	else
-		strvec_push(&child.args, "bootout");
-	strvec_push(&child.args, uid);
-	strvec_push(&child.args, filename);
+	strvec_pushl(&child.args, enable ? "bootstrap" : "bootout", uid,
+		     filename, NULL);
 
 	child.no_stderr = 1;
 	child.no_stdout = 1;
@@ -1581,26 +1666,26 @@ static int launchctl_boot_plist(int enable, const char *filename, const char *cm
 	return result;
 }
 
-static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)
+static int launchctl_remove_plist(enum schedule_priority schedule)
 {
 	const char *frequency = get_frequency(schedule);
 	char *name = launchctl_service_name(frequency);
 	char *filename = launchctl_service_filename(name);
-	int result = launchctl_boot_plist(0, filename, cmd);
+	int result = launchctl_boot_plist(0, filename);
 	unlink(filename);
 	free(filename);
 	free(name);
 	return result;
 }
 
-static int launchctl_remove_plists(const char *cmd)
+static int launchctl_remove_plists(void)
 {
-	return launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||
-		launchctl_remove_plist(SCHEDULE_DAILY, cmd) ||
-		launchctl_remove_plist(SCHEDULE_WEEKLY, cmd);
+	return launchctl_remove_plist(SCHEDULE_HOURLY) ||
+	       launchctl_remove_plist(SCHEDULE_DAILY) ||
+	       launchctl_remove_plist(SCHEDULE_WEEKLY);
 }
 
-static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule)
 {
 	FILE *plist;
 	int i;
@@ -1669,8 +1754,8 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 	fclose(plist);
 
 	/* bootout might fail if not already running, so ignore */
-	launchctl_boot_plist(0, filename, cmd);
-	if (launchctl_boot_plist(1, filename, cmd))
+	launchctl_boot_plist(0, filename);
+	if (launchctl_boot_plist(1, filename))
 		die(_("failed to bootstrap service %s"), filename);
 
 	free(filename);
@@ -1678,21 +1763,35 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 	return 0;
 }
 
-static int launchctl_add_plists(const char *cmd)
+static int launchctl_add_plists(void)
 {
 	const char *exec_path = git_exec_path();
 
-	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||
-		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||
-		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
+	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY) ||
+	       launchctl_schedule_plist(exec_path, SCHEDULE_DAILY) ||
+	       launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY);
 }
 
-static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int launchctl_update_schedule(int run_maintenance, int fd)
 {
 	if (run_maintenance)
-		return launchctl_add_plists(cmd);
+		return launchctl_add_plists();
 	else
-		return launchctl_remove_plists(cmd);
+		return launchctl_remove_plists();
+}
+
+static int is_schtasks_available(void)
+{
+	const char *cmd = "schtasks";
+	int is_available;
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+#ifdef GIT_WINDOWS_NATIVE
+	return 1;
+#else
+	return 0;
+#endif
 }
 
 static char *schtasks_task_name(const char *frequency)
@@ -1702,13 +1801,15 @@ static char *schtasks_task_name(const char *frequency)
 	return strbuf_detach(&label, NULL);
 }
 
-static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)
+static int schtasks_remove_task(enum schedule_priority schedule)
 {
+	const char *cmd = "schtasks";
 	int result;
 	struct strvec args = STRVEC_INIT;
 	const char *frequency = get_frequency(schedule);
 	char *name = schtasks_task_name(frequency);
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&args, cmd);
 	strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
 
@@ -1719,15 +1820,16 @@ static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd
 	return result;
 }
 
-static int schtasks_remove_tasks(const char *cmd)
+static int schtasks_remove_tasks(void)
 {
-	return schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||
-		schtasks_remove_task(SCHEDULE_DAILY, cmd) ||
-		schtasks_remove_task(SCHEDULE_WEEKLY, cmd);
+	return schtasks_remove_task(SCHEDULE_HOURLY) ||
+	       schtasks_remove_task(SCHEDULE_DAILY) ||
+	       schtasks_remove_task(SCHEDULE_WEEKLY);
 }
 
-static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule)
 {
+	const char *cmd = "schtasks";
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
 	const char *xml;
@@ -1736,6 +1838,8 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
 	char *name = schtasks_task_name(frequency);
 	struct strbuf tfilename = STRBUF_INIT;
 
+	get_schedule_cmd(&cmd, NULL);
+
 	strbuf_addf(&tfilename, "%s/schedule_%s_XXXXXX",
 		    get_git_common_dir(), frequency);
 	tfile = xmks_tempfile(tfilename.buf);
@@ -1840,28 +1944,52 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
 	return result;
 }
 
-static int schtasks_schedule_tasks(const char *cmd)
+static int schtasks_schedule_tasks(void)
 {
 	const char *exec_path = git_exec_path();
 
-	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||
-		schtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||
-		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
+	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY) ||
+	       schtasks_schedule_task(exec_path, SCHEDULE_DAILY) ||
+	       schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY);
 }
 
-static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int schtasks_update_schedule(int run_maintenance, int fd)
 {
 	if (run_maintenance)
-		return schtasks_schedule_tasks(cmd);
+		return schtasks_schedule_tasks();
 	else
-		return schtasks_remove_tasks(cmd);
+		return schtasks_remove_tasks();
+}
+
+static int is_crontab_available(void)
+{
+	const char *cmd = "crontab";
+	int is_available;
+	struct child_process child = CHILD_PROCESS_INIT;
+
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+	strvec_split(&child.args, cmd);
+	strvec_push(&child.args, "-l");
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child))
+		return 0;
+	/* Ignore exit code, as an empty crontab will return error. */
+	finish_command(&child);
+	return 1;
 }
 
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
-static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int crontab_update_schedule(int run_maintenance, int fd)
 {
+	const char *cmd = "crontab";
 	int result = 0;
 	int in_old_region = 0;
 	struct child_process crontab_list = CHILD_PROCESS_INIT;
@@ -1869,6 +1997,7 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 	FILE *cron_list, *cron_in;
 	struct strbuf line = STRBUF_INIT;
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&crontab_list.args, cmd);
 	strvec_push(&crontab_list.args, "-l");
 	crontab_list.in = -1;
@@ -1945,66 +2074,160 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 	return result;
 }
 
+enum scheduler {
+	SCHEDULER_INVALID = -1,
+	SCHEDULER_AUTO,
+	SCHEDULER_CRON,
+	SCHEDULER_LAUNCHCTL,
+	SCHEDULER_SCHTASKS,
+};
+
+static const struct {
+	const char *name;
+	int (*is_available)(void);
+	int (*update_schedule)(int run_maintenance, int fd);
+} scheduler_fn[] = {
+	[SCHEDULER_CRON] = {
+		.name = "crontab",
+		.is_available = is_crontab_available,
+		.update_schedule = crontab_update_schedule,
+	},
+	[SCHEDULER_LAUNCHCTL] = {
+		.name = "launchctl",
+		.is_available = is_launchctl_available,
+		.update_schedule = launchctl_update_schedule,
+	},
+	[SCHEDULER_SCHTASKS] = {
+		.name = "schtasks",
+		.is_available = is_schtasks_available,
+		.update_schedule = schtasks_update_schedule,
+	},
+};
+
+static enum scheduler parse_scheduler(const char *value)
+{
+	if (!value)
+		return SCHEDULER_INVALID;
+	else if (!strcasecmp(value, "auto"))
+		return SCHEDULER_AUTO;
+	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
+		return SCHEDULER_CRON;
+	else if (!strcasecmp(value, "launchctl"))
+		return SCHEDULER_LAUNCHCTL;
+	else if (!strcasecmp(value, "schtasks"))
+		return SCHEDULER_SCHTASKS;
+	else
+		return SCHEDULER_INVALID;
+}
+
+static int maintenance_opt_scheduler(const struct option *opt, const char *arg,
+				     int unset)
+{
+	enum scheduler *scheduler = opt->value;
+
+	BUG_ON_OPT_NEG(unset);
+
+	*scheduler = parse_scheduler(arg);
+	if (*scheduler == SCHEDULER_INVALID)
+		return error(_("unrecognized --scheduler argument '%s'"), arg);
+	return 0;
+}
+
+struct maintenance_start_opts {
+	enum scheduler scheduler;
+};
+
+static enum scheduler resolve_scheduler(enum scheduler scheduler)
+{
+	if (scheduler != SCHEDULER_AUTO)
+		return scheduler;
+
 #if defined(__APPLE__)
-static const char platform_scheduler[] = "launchctl";
+	return SCHEDULER_LAUNCHCTL;
+
 #elif defined(GIT_WINDOWS_NATIVE)
-static const char platform_scheduler[] = "schtasks";
+	return SCHEDULER_SCHTASKS;
+
 #else
-static const char platform_scheduler[] = "crontab";
+	return SCHEDULER_CRON;
 #endif
+}
 
-static int update_background_schedule(int enable)
+static void validate_scheduler(enum scheduler scheduler)
 {
-	int result;
-	const char *scheduler = platform_scheduler;
-	const char *cmd = scheduler;
-	char *testing;
+	if (scheduler == SCHEDULER_INVALID)
+		BUG("invalid scheduler");
+	if (scheduler == SCHEDULER_AUTO)
+		BUG("resolve_scheduler should have been called before");
+
+	if (!scheduler_fn[scheduler].is_available())
+		die(_("%s scheduler is not available"),
+		    scheduler_fn[scheduler].name);
+}
+
+static int update_background_schedule(const struct maintenance_start_opts *opts,
+				      int enable)
+{
+	unsigned int i;
+	int result = 0;
 	struct lock_file lk;
 	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
 
-	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
-	if (testing) {
-		char *sep = strchr(testing, ':');
-		if (!sep)
-			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
-		*sep = '\0';
-		scheduler = testing;
-		cmd = sep + 1;
+	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {
+		free(lock_path);
+		return error(_("another process is scheduling background maintenance"));
 	}
 
-	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {
-		result = error(_("another process is scheduling background maintenance"));
-		goto cleanup;
+	for (i = 1; i < ARRAY_SIZE(scheduler_fn); i++) {
+		if (enable && opts->scheduler == i)
+			continue;
+		if (!scheduler_fn[i].is_available())
+			continue;
+		scheduler_fn[i].update_schedule(0, get_lock_file_fd(&lk));
 	}
 
-	if (!strcmp(scheduler, "launchctl"))
-		result = launchctl_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else if (!strcmp(scheduler, "schtasks"))
-		result = schtasks_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else if (!strcmp(scheduler, "crontab"))
-		result = crontab_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else
-		die("unknown background scheduler: %s", scheduler);
+	if (enable)
+		result = scheduler_fn[opts->scheduler].update_schedule(
+			1, get_lock_file_fd(&lk));
 
 	rollback_lock_file(&lk);
 
-cleanup:
 	free(lock_path);
-	free(testing);
 	return result;
 }
 
-static int maintenance_start(void)
+static const char *const builtin_maintenance_start_usage[] = {
+	N_("git maintenance start [--scheduler=<scheduler>]"),
+	NULL
+};
+
+static int maintenance_start(int argc, const char **argv, const char *prefix)
 {
+	struct maintenance_start_opts opts = { 0 };
+	struct option options[] = {
+		OPT_CALLBACK_F(
+			0, "scheduler", &opts.scheduler, N_("scheduler"),
+			N_("scheduler to trigger git maintenance run"),
+			PARSE_OPT_NONEG, maintenance_opt_scheduler),
+		OPT_END()
+	};
+
+	argc = parse_options(argc, argv, prefix, options,
+			     builtin_maintenance_start_usage, 0);
+	if (argc)
+		usage_with_options(builtin_maintenance_start_usage, options);
+
+	opts.scheduler = resolve_scheduler(opts.scheduler);
+	validate_scheduler(opts.scheduler);
+
 	if (maintenance_register())
 		warning(_("failed to add repo to global config"));
-
-	return update_background_schedule(1);
+	return update_background_schedule(&opts, 1);
 }
 
 static int maintenance_stop(void)
 {
-	return update_background_schedule(0);
+	return update_background_schedule(NULL, 0);
 }
 
 static const char builtin_maintenance_usage[] =	N_("git maintenance <subcommand> [<options>]");
@@ -2018,7 +2241,7 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
 	if (!strcmp(argv[1], "run"))
 		return maintenance_run(argc - 1, argv + 1, prefix);
 	if (!strcmp(argv[1], "start"))
-		return maintenance_start();
+		return maintenance_start(argc - 1, argv + 1, prefix);
 	if (!strcmp(argv[1], "stop"))
 		return maintenance_stop();
 	if (!strcmp(argv[1], "register"))
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 58f46c77e6..27bce7992c 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -492,8 +492,21 @@ test_expect_success !MINGW 'register and unregister with regex metacharacters' '
 		maintenance.repo "$(pwd)/$META"
 '
 
+test_expect_success 'start --scheduler=<scheduler>' '
+	test_expect_code 129 git maintenance start --scheduler=foo 2>err &&
+	test_i18ngrep "unrecognized --scheduler argument" err &&
+
+	test_expect_code 129 git maintenance start --no-scheduler 2>err &&
+	test_i18ngrep "unknown option" err &&
+
+	test_expect_code 128 \
+		env GIT_TEST_MAINT_SCHEDULER="launchctl:true,schtasks:true" \
+		git maintenance start --scheduler=crontab 2>err &&
+	test_i18ngrep "fatal: crontab scheduler is not available" err
+'
+
 test_expect_success 'start from empty cron table' '
-	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -516,7 +529,7 @@ test_expect_success 'stop from existing schedule' '
 
 test_expect_success 'start preserves existing schedule' '
 	echo "Important information!" >cron.txt &&
-	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
 	grep "Important information!" cron.txt
 '
 
@@ -545,7 +558,7 @@ test_expect_success 'start and stop macOS maintenance' '
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start --scheduler=launchctl &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -596,7 +609,7 @@ test_expect_success 'start and stop Windows maintenance' '
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start --scheduler=schtasks &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -619,6 +632,40 @@ test_expect_success 'start and stop Windows maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success 'start and stop when several schedulers are available' '
+	write_script print-args <<-\EOF &&
+	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
+	EOF
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >expect &&
+	for frequency in hourly daily weekly
+	do
+		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
+		echo "launchctl bootout gui/[UID] $PLIST" >>expect &&
+		echo "launchctl bootstrap gui/[UID] $PLIST" >>expect || return 1
+	done &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* [PATCH v8 3/3] maintenance: add support for systemd timers on Linux
  2021-08-23 20:40             ` [PATCH v8 " Lénaïc Huard
  2021-08-23 20:40               ` [PATCH v8 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
  2021-08-23 20:40               ` [PATCH v8 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
@ 2021-08-23 20:40               ` Lénaïc Huard
  2021-08-24 17:45                 ` Derrick Stolee
  2021-08-24 17:47               ` [PATCH v8 0/3] " Derrick Stolee
  2021-08-27 21:02               ` [PATCH v9 " Lénaïc Huard
  4 siblings, 1 reply; 138+ messages in thread
From: Lénaïc Huard @ 2021-08-23 20:40 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King,
	Lénaïc Huard

The existing mechanism for scheduling background maintenance is done
through cron. On Linux systems managed by systemd, systemd provides an
alternative to schedule recurring tasks: systemd timers.

The main motivations to implement systemd timers in addition to cron
are:
* cron is optional and Linux systems running systemd might not have it
  installed.
* The execution of `crontab -l` can tell us if cron is installed but not
  if the daemon is actually running.
* With systemd, each service is run in its own cgroup and its logs are
  tagged by the service inside journald. With cron, all scheduled tasks
  are running in the cron daemon cgroup and all the logs of the
  user-scheduled tasks are pretended to belong to the system cron
  service.
  Concretely, a user that doesn’t have access to the system logs won’t
  have access to the log of their own tasks scheduled by cron whereas
  they will have access to the log of their own tasks scheduled by
  systemd timer.
  Although `cron` attempts to send email, that email may go unseen by
  the user because these days, local mailboxes are not heavily used
  anymore.

In order to schedule git maintenance, we need two unit template files:
* ~/.config/systemd/user/git-maintenance@.service
  to define the command to be started by systemd and
* ~/.config/systemd/user/git-maintenance@.timer
  to define the schedule at which the command should be run.

Those units are templates that are parameterized by the frequency.

Based on those templates, 3 timers are started:
* git-maintenance@hourly.timer
* git-maintenance@daily.timer
* git-maintenance@weekly.timer

The command launched by those three timers are the same as with the
other scheduling methods:

/path/to/git for-each-repo --exec-path=/path/to
--config=maintenance.repo maintenance run --schedule=%i

with the full path for git to ensure that the version of git launched
for the scheduled maintenance is the same as the one used to run
`maintenance start`.

The timer unit contains `Persistent=true` so that, if the computer is
powered down when a maintenance task should run, the task will be run
when the computer is back powered on.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 Documentation/git-maintenance.txt |  58 +++++++-
 builtin/gc.c                      | 227 ++++++++++++++++++++++++++++++
 t/t7900-maintenance.sh            |  67 ++++++++-
 3 files changed, 341 insertions(+), 11 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 576290b5c6..e2cfb68ab5 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -179,14 +179,16 @@ OPTIONS
 	`maintenance.<task>.enabled` configured as `true` are considered.
 	See the 'TASKS' section for the list of accepted `<task>` values.
 
---scheduler=auto|crontab|launchctl|schtasks::
+--scheduler=auto|crontab|systemd-timer|launchctl|schtasks::
 	When combined with the `start` subcommand, specify the scheduler
 	for running the hourly, daily and weekly executions of
 	`git maintenance run`.
-	Possible values for `<scheduler>` are `auto`, `crontab` (POSIX),
-	`launchctl` (macOS), and `schtasks` (Windows).
-	When `auto` is specified, the appropriate platform-specific
-	scheduler is used. Default is `auto`.
+	Possible values for `<scheduler>` are `auto`, `crontab`
+	(POSIX), `systemd-timer` (Linux), `launchctl` (macOS), and
+	`schtasks` (Windows). When `auto` is specified, the
+	appropriate platform-specific scheduler is used; on Linux,
+	`systemd-timer` is used if available, otherwise
+	`crontab`. Default is `auto`.
 
 
 TROUBLESHOOTING
@@ -286,6 +288,52 @@ schedule to ensure you are executing the correct binaries in your
 schedule.
 
 
+BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMS
+-----------------------------------------------
+
+While Linux supports `cron`, depending on the distribution, `cron` may
+be an optional package not necessarily installed. On modern Linux
+distributions, systemd timers are superseding it.
+
+If user systemd timers are available, they will be used as a replacement
+of `cron`.
+
+In this case, `git maintenance start` will create user systemd timer units
+and start the timers. The current list of user-scheduled tasks can be found
+by running `systemctl --user list-timers`. The timers written by `git
+maintenance start` are similar to this:
+
+-----------------------------------------------------------------------
+$ systemctl --user list-timers
+NEXT                         LEFT          LAST                         PASSED     UNIT                         ACTIVATES
+Thu 2021-04-29 19:00:00 CEST 42min left    Thu 2021-04-29 18:00:11 CEST 17min ago  git-maintenance@hourly.timer git-maintenance@hourly.service
+Fri 2021-04-30 00:00:00 CEST 5h 42min left Thu 2021-04-29 00:00:11 CEST 18h ago    git-maintenance@daily.timer  git-maintenance@daily.service
+Mon 2021-05-03 00:00:00 CEST 3 days left   Mon 2021-04-26 00:00:11 CEST 3 days ago git-maintenance@weekly.timer git-maintenance@weekly.service
+-----------------------------------------------------------------------
+
+One timer is registered for each `--schedule=<frequency>` option.
+
+The definition of the systemd units can be inspected in the following files:
+
+-----------------------------------------------------------------------
+~/.config/systemd/user/git-maintenance@.timer
+~/.config/systemd/user/git-maintenance@.service
+~/.config/systemd/user/timers.target.wants/git-maintenance@hourly.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@daily.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@weekly.timer
+-----------------------------------------------------------------------
+
+`git maintenance start` will overwrite these files and start the timer
+again with `systemctl --user`, so any customization should be done by
+creating a drop-in file, i.e. a `.conf` suffixed file in the
+`~/.config/systemd/user/git-maintenance@.service.d` directory.
+
+`git maintenance stop` will stop the user systemd timers and delete
+the above mentioned files.
+
+For more details, see `systemd.timer(5)`.
+
+
 BACKGROUND MAINTENANCE ON MACOS SYSTEMS
 ---------------------------------------
 
diff --git a/builtin/gc.c b/builtin/gc.c
index 9e464d4a10..a3b2fc55d2 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -2074,10 +2074,221 @@ static int crontab_update_schedule(int run_maintenance, int fd)
 	return result;
 }
 
+#ifdef __linux__
+
+static int real_is_systemd_timer_available(void)
+{
+	struct child_process child = CHILD_PROCESS_INIT;
+
+	strvec_pushl(&child.args, "systemctl", "--user", "list-timers", NULL);
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child))
+		return 0;
+	if (finish_command(&child))
+		return 0;
+	return 1;
+}
+
+#else
+
+static int real_is_systemd_timer_available(void)
+{
+	return 0;
+}
+
+#endif
+
+static int is_systemd_timer_available(void)
+{
+	const char *cmd = "systemctl";
+	int is_available;
+
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+	return real_is_systemd_timer_available();
+}
+
+static char *xdg_config_home_systemd(const char *filename)
+{
+	return xdg_config_home_for("systemd/user", filename);
+}
+
+static int systemd_timer_enable_unit(int enable,
+				     enum schedule_priority schedule)
+{
+	const char *cmd = "systemctl";
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *frequency = get_frequency(schedule);
+
+	/*
+	 * Disabling the systemd unit while it is already disabled makes
+	 * systemctl print an error.
+	 * Let's ignore it since it means we already are in the expected state:
+	 * the unit is disabled.
+	 *
+	 * On the other hand, enabling a systemd unit which is already enabled
+	 * produces no error.
+	 */
+	if (!enable)
+		child.no_stderr = 1;
+
+	get_schedule_cmd(&cmd, NULL);
+	strvec_split(&child.args, cmd);
+	strvec_pushl(&child.args, "--user", enable ? "enable" : "disable",
+		     "--now", NULL);
+	strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency);
+
+	if (start_command(&child))
+		return error(_("failed to start systemctl"));
+	if (finish_command(&child))
+		/*
+		 * Disabling an already disabled systemd unit makes
+		 * systemctl fail.
+		 * Let's ignore this failure.
+		 *
+		 * Enabling an enabled systemd unit doesn't fail.
+		 */
+		if (enable)
+			return error(_("failed to run systemctl"));
+	return 0;
+}
+
+static int systemd_timer_delete_unit_templates(void)
+{
+	int ret = 0;
+	char *filename = xdg_config_home_systemd("git-maintenance@.timer");
+	if (unlink(filename) && !is_missing_file_error(errno))
+		ret = error_errno(_("failed to delete '%s'"), filename);
+	FREE_AND_NULL(filename);
+
+	filename = xdg_config_home_systemd("git-maintenance@.service");
+	if (unlink(filename) && !is_missing_file_error(errno))
+		ret = error_errno(_("failed to delete '%s'"), filename);
+
+	free(filename);
+	return ret;
+}
+
+static int systemd_timer_delete_units(void)
+{
+	return systemd_timer_enable_unit(0, SCHEDULE_HOURLY) ||
+	       systemd_timer_enable_unit(0, SCHEDULE_DAILY) ||
+	       systemd_timer_enable_unit(0, SCHEDULE_WEEKLY) ||
+	       systemd_timer_delete_unit_templates();
+}
+
+static int systemd_timer_write_unit_templates(const char *exec_path)
+{
+	char *filename;
+	FILE *file;
+	const char *unit;
+
+	filename = xdg_config_home_systemd("git-maintenance@.timer");
+	if (safe_create_leading_directories(filename)) {
+		error(_("failed to create directories for '%s'"), filename);
+		goto error;
+	}
+	file = fopen_or_warn(filename, "w");
+	if (file == NULL)
+		goto error;
+
+	unit = "# This file was created and is maintained by Git.\n"
+	       "# Any edits made in this file might be replaced in the future\n"
+	       "# by a Git command.\n"
+	       "\n"
+	       "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Timer]\n"
+	       "OnCalendar=%i\n"
+	       "Persistent=true\n"
+	       "\n"
+	       "[Install]\n"
+	       "WantedBy=timers.target\n";
+	if (fputs(unit, file) == EOF) {
+		error(_("failed to write to '%s'"), filename);
+		fclose(file);
+		goto error;
+	}
+	if (fclose(file) == EOF) {
+		error_errno(_("failed to flush '%s'"), filename);
+		goto error;
+	}
+	free(filename);
+
+	filename = xdg_config_home_systemd("git-maintenance@.service");
+	file = fopen_or_warn(filename, "w");
+	if (file == NULL)
+		goto error;
+
+	unit = "# This file was created and is maintained by Git.\n"
+	       "# Any edits made in this file might be replaced in the future\n"
+	       "# by a Git command.\n"
+	       "\n"
+	       "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Service]\n"
+	       "Type=oneshot\n"
+	       "ExecStart=\"%s/git\" --exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
+	       "LockPersonality=yes\n"
+	       "MemoryDenyWriteExecute=yes\n"
+	       "NoNewPrivileges=yes\n"
+	       "RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\n"
+	       "RestrictNamespaces=yes\n"
+	       "RestrictRealtime=yes\n"
+	       "RestrictSUIDSGID=yes\n"
+	       "SystemCallArchitectures=native\n"
+	       "SystemCallFilter=@system-service\n";
+	if (fprintf(file, unit, exec_path, exec_path) < 0) {
+		error(_("failed to write to '%s'"), filename);
+		fclose(file);
+		goto error;
+	}
+	if (fclose(file) == EOF) {
+		error_errno(_("failed to flush '%s'"), filename);
+		goto error;
+	}
+	free(filename);
+	return 0;
+
+error:
+	free(filename);
+	systemd_timer_delete_unit_templates();
+	return -1;
+}
+
+static int systemd_timer_setup_units(void)
+{
+	const char *exec_path = git_exec_path();
+
+	int ret = systemd_timer_write_unit_templates(exec_path) ||
+	          systemd_timer_enable_unit(1, SCHEDULE_HOURLY) ||
+	          systemd_timer_enable_unit(1, SCHEDULE_DAILY) ||
+	          systemd_timer_enable_unit(1, SCHEDULE_WEEKLY);
+	if (ret)
+		systemd_timer_delete_units();
+	return ret;
+}
+
+static int systemd_timer_update_schedule(int run_maintenance, int fd)
+{
+	if (run_maintenance)
+		return systemd_timer_setup_units();
+	else
+		return systemd_timer_delete_units();
+}
+
 enum scheduler {
 	SCHEDULER_INVALID = -1,
 	SCHEDULER_AUTO,
 	SCHEDULER_CRON,
+	SCHEDULER_SYSTEMD,
 	SCHEDULER_LAUNCHCTL,
 	SCHEDULER_SCHTASKS,
 };
@@ -2092,6 +2303,11 @@ static const struct {
 		.is_available = is_crontab_available,
 		.update_schedule = crontab_update_schedule,
 	},
+	[SCHEDULER_SYSTEMD] = {
+		.name = "systemctl",
+		.is_available = is_systemd_timer_available,
+		.update_schedule = systemd_timer_update_schedule,
+	},
 	[SCHEDULER_LAUNCHCTL] = {
 		.name = "launchctl",
 		.is_available = is_launchctl_available,
@@ -2112,6 +2328,9 @@ static enum scheduler parse_scheduler(const char *value)
 		return SCHEDULER_AUTO;
 	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
 		return SCHEDULER_CRON;
+	else if (!strcasecmp(value, "systemd") ||
+		 !strcasecmp(value, "systemd-timer"))
+		return SCHEDULER_SYSTEMD;
 	else if (!strcasecmp(value, "launchctl"))
 		return SCHEDULER_LAUNCHCTL;
 	else if (!strcasecmp(value, "schtasks"))
@@ -2148,6 +2367,14 @@ static enum scheduler resolve_scheduler(enum scheduler scheduler)
 #elif defined(GIT_WINDOWS_NATIVE)
 	return SCHEDULER_SCHTASKS;
 
+#elif defined(__linux__)
+	if (is_systemd_timer_available())
+		return SCHEDULER_SYSTEMD;
+	else if (is_crontab_available())
+		return SCHEDULER_CRON;
+	else
+		die(_("neither systemd timers nor crontab are available"));
+
 #else
 	return SCHEDULER_CRON;
 #endif
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 27bce7992c..265f7793f5 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -20,6 +20,18 @@ test_xmllint () {
 	fi
 }
 
+test_lazy_prereq SYSTEMD_ANALYZE '
+	systemd-analyze --help >out &&
+	grep verify out
+'
+
+test_systemd_analyze_verify () {
+	if test_have_prereq SYSTEMD_ANALYZE
+	then
+		systemd-analyze verify "$@"
+	fi
+}
+
 test_expect_success 'help text' '
 	test_expect_code 129 git maintenance -h 2>err &&
 	test_i18ngrep "usage: git maintenance <subcommand>" err &&
@@ -632,15 +644,56 @@ test_expect_success 'start and stop Windows maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success 'start and stop Linux/systemd maintenance' '
+	write_script print-args <<-\EOF &&
+	printf "%s\n" "$*" >>args
+	EOF
+
+	XDG_CONFIG_HOME="$PWD" &&
+	export XDG_CONFIG_HOME &&
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance start --scheduler=systemd-timer &&
+
+	# start registers the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_systemd_analyze_verify "systemd/user/git-maintenance@.service" &&
+
+	printf -- "--user enable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_path_is_missing "systemd/user/git-maintenance@.timer" &&
+	test_path_is_missing "systemd/user/git-maintenance@.service" &&
+
+	printf -- "--user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'start and stop when several schedulers are available' '
 	write_script print-args <<-\EOF &&
 	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
-	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=systemd-timer &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
 		hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
+	printf -- "systemctl --user enable --now git-maintenance@%s.timer\n" hourly daily weekly >>expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
 	for frequency in hourly daily weekly
 	do
 		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
@@ -650,17 +703,19 @@ test_expect_success 'start and stop when several schedulers are available' '
 	test_cmp expect args &&
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
 	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
-		hourly daily weekly >expect &&
+		hourly daily weekly >>expect &&
 	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
 		hourly daily weekly >>expect &&
 	test_cmp expect args &&
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
 	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
-		hourly daily weekly >expect &&
+		hourly daily weekly >>expect &&
 	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
 		hourly daily weekly >>expect &&
 	test_cmp expect args
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* Re: [PATCH v7 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-08-23 20:06                 ` Lénaïc Huard
@ 2021-08-23 22:30                   ` Junio C Hamano
  0 siblings, 0 replies; 138+ messages in thread
From: Junio C Hamano @ 2021-08-23 22:30 UTC (permalink / raw)
  To: Lénaïc Huard
  Cc: Ævar Arnfjörð Bjarmason, git, Derrick Stolee,
	Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King

Lénaïc Huard <lenaic@lhuard.fr> writes:

> Is the suggestion to replace
>
> +#ifdef __APPLE__
> +	return 1;
> +#else
> +	return 0;
> +#endif
> +}
>
> by
>
> + return IS_LAUNCHCTL_AVAILABLE;
>
> and to add
>
> #ifdef __APPLE__
> #define IS_LAUNCHCTL_AVAILABLE 1
> #else
> #define IS_LAUNCHCTL_AVAILABLE 0
> #endif
>
> somewhere else like at the top of builtin/gc.c ?

I wasn't the one who suggested it, but the suggestion reads as such
to me.

> Also, do we agree this shouldn’t be defined in cache.h ? I’m a little bit 
> confused.

The audience of "cache.h" (or more precisely, "git-compat-util.h" is
where these come from by including system headers) is much wider
than those narrow users who care about launchtrl or cron, so
limiting it in builtin/gc.c would make more sense, I would think.

Thanks.




^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v8 3/3] maintenance: add support for systemd timers on Linux
  2021-08-23 20:40               ` [PATCH v8 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
@ 2021-08-24 17:45                 ` Derrick Stolee
  0 siblings, 0 replies; 138+ messages in thread
From: Derrick Stolee @ 2021-08-24 17:45 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King

On 8/23/2021 4:40 PM, Lénaïc Huard wrote:
> +#ifdef __linux__
> +
> +static int real_is_systemd_timer_available(void)
> +{
> +	struct child_process child = CHILD_PROCESS_INIT;
> +
> +	strvec_pushl(&child.args, "systemctl", "--user", "list-timers", NULL);
> +	child.no_stdin = 1;
> +	child.no_stdout = 1;
> +	child.no_stderr = 1;
> +	child.silent_exec_failure = 1;
> +
> +	if (start_command(&child))
> +		return 0;
> +	if (finish_command(&child))
> +		return 0;
> +	return 1;
> +}
> +
> +#else
> +
> +static int real_is_systemd_timer_available(void)
> +{
> +	return 0;
> +}
> +
> +#endif

This #ifdef option is one that could be changed. There is a lot
of code inside the #ifdef that would be nice to compile on all
platforms.

Technically, we could drop all conditionals here and rely on
the start_command() and finish_command() to tell us that systemd
is or is not installed. This would allow a potential future where
maybe macOS supports systemd (or users install a version
themselves).

Another option would be to compile in a conditional early return
inside real_is_systemd_timer_available() such as

static int real_is_systemd_timer_available(void)
{
	struct child_process child = CHILD_PROCESS_INIT;

#ifndef __linux__
	if (1)
		return 0;
#endif

	strvec_pushl(&child.args, "systemctl", "--user", "list-timers", NULL);
	child.no_stdin = 1;
	child.no_stdout = 1;
	child.no_stderr = 1;
	child.silent_exec_failure = 1;

	if (start_command(&child))
		return 0;
	if (finish_command(&child))
		return 0;
	return 1;
}

...but this also looks a bit awkward in order to avoid compilers
complaining about unreachable code (and some might still rightly
warn about unreachable code).

> +static int systemd_timer_setup_units(void)
> +{
> +	const char *exec_path = git_exec_path();
> +
> +	int ret = systemd_timer_write_unit_templates(exec_path) ||
> +	          systemd_timer_enable_unit(1, SCHEDULE_HOURLY) ||
> +	          systemd_timer_enable_unit(1, SCHEDULE_DAILY) ||
> +	          systemd_timer_enable_unit(1, SCHEDULE_WEEKLY);

These lines are incorrectly tabbed with spaces. Here is a corrected
version:

+	int ret = systemd_timer_write_unit_templates(exec_path) ||
+		  systemd_timer_enable_unit(1, SCHEDULE_HOURLY) ||
+		  systemd_timer_enable_unit(1, SCHEDULE_DAILY) ||
+		  systemd_timer_enable_unit(1, SCHEDULE_WEEKLY);

'git rebase --whitespace=fix <base>' will also fix these issues. You
can discover if they exist using 'git log --check'.

Thanks,
-Stolee

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v8 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-08-23 20:40               ` [PATCH v8 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
@ 2021-08-24 17:45                 ` Derrick Stolee
  0 siblings, 0 replies; 138+ messages in thread
From: Derrick Stolee @ 2021-08-24 17:45 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King

On 8/23/2021 4:40 PM, Lénaïc Huard wrote:> + * Ex.:
> + *   GIT_TEST_MAINT_SCHEDULER not set
> + *     +-------+-------------------------------------------------+
> + *     | Input |                     Output                      |
> + *     | *cmd  | return code |       *cmd        | *is_available |
> + *     +-------+-------------+-------------------+---------------+
> + *     | "foo" |    false    | "foo" (unchanged) |  (unchanged)  |
> + *     +-------+-------------+-------------------+---------------+
> + *
> + *   GIT_TEST_MAINT_SCHEDULER set to “foo:./mock_foo.sh,bar:./mock_bar.sh”
> + *     +-------+-------------------------------------------------+
> + *     | Input |                     Output                      |
> + *     | *cmd  | return code |       *cmd        | *is_available |
> + *     +-------+-------------+-------------------+---------------+
> + *     | "foo" |    true     |  "./mock.foo.sh"  |     true      |
> + *     | "qux" |    true     | "qux" (unchanged) |     false     |
> + *     +-------+-------------+-------------------+---------------+
> + */

Thank you for updating to this ASCII table. It has the same amount
of visual information without requiring special characters.

> +static int is_launchctl_available(void)
> +{
> +	const char *cmd = "launchctl";
> +	int is_available;
> +	if (get_schedule_cmd(&cmd, &is_available))
> +		return is_available;
> +
> +#ifdef __APPLE__
> +	return 1;
> +#else
> +	return 0;
> +#endif
> +}

I find this use of #ifdef to be perfectly fine. Adding a layer of
indirection into the compat layer through another macro is
unnecessary, in my opinion.

Thanks,
-Stolee

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v8 0/3] maintenance: add support for systemd timers on Linux
  2021-08-23 20:40             ` [PATCH v8 " Lénaïc Huard
                                 ` (2 preceding siblings ...)
  2021-08-23 20:40               ` [PATCH v8 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
@ 2021-08-24 17:47               ` Derrick Stolee
  2021-08-27 21:02               ` [PATCH v9 " Lénaïc Huard
  4 siblings, 0 replies; 138+ messages in thread
From: Derrick Stolee @ 2021-08-24 17:47 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King

On 8/23/2021 4:40 PM, Lénaïc Huard wrote:
> According to [1], there were 3 changes awaited in this v8:
> * The two already mentionned above (utf-8 characters and
>   `string_list_split` thing)
> * An improvement around the #ifdef.
> 
> I must admit I haven’t touched anything around the #ifdef in this v8
> because I’m not sure what to do. I’ve just asked for some more details
> in [2].

I commented on the relevant patches, but in summary:

1. I think your #ifdef __APPLE__ is fine in patch 2.

2. The #ifdef __linux__ in patch 3 could be removed
   entirely.

Then there is a nit about whitespace, but that is easily
rectified with 'git rebase --whitespace=fix'.

Thanks,
-Stolee

^ permalink raw reply	[flat|nested] 138+ messages in thread

* [PATCH v9 0/3] maintenance: add support for systemd timers on Linux
  2021-08-23 20:40             ` [PATCH v8 " Lénaïc Huard
                                 ` (3 preceding siblings ...)
  2021-08-24 17:47               ` [PATCH v8 0/3] " Derrick Stolee
@ 2021-08-27 21:02               ` Lénaïc Huard
  2021-08-27 21:02                 ` [PATCH v9 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
                                   ` (3 more replies)
  4 siblings, 4 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-08-27 21:02 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King,
	Lénaïc Huard

Hello,

Please find hereafter my updated patchset to add support for systemd
timers on Linux for the `git maintenance start` command.

The goal of this re-roll is to address Derrick’s remarks [1]
Namely:
* Get rid of the #ifdef around real_is_systemd_timer_available
* Fix the whitespace issue in systemd_timer_setup_units

[1] https://lore.kernel.org/git/44904983-a6a8-d72f-24db-50bf112c585b@gmail.com/

The patches are:

* cache.h: Introduce a generic "xdg_config_home_for(…)" function

  This patch introduces a function to compute configuration files
  paths inside $XDG_CONFIG_HOME.

  This patch is unchanged compared to its previous version.

* maintenance: `git maintenance run` learned `--scheduler=<scheduler>`

  This patch adds a new parameter to the `git maintenance run` to let
  the user choose a scheduler.

  This patch is unchanged compared to its previous version.

* maintenance: add support for systemd timers on Linux

  This patch implements the support of systemd timers on top of
  crontab scheduler on Linux systems.

  * The whitespace nit has been fixed with `git rebase
    --whitespace=fix`

  * The #ifdef around real_is_systemd_timer_available has been dropped
    so that its code is compiled on every platform.

    This means that:
    * The availability of `launchctl` and `schtasks` are completely
      determined at compilation time by the platform.
    * The availability of `cron` and `systemd timers` are fully
      determined at runtime even if today only Linux might have systemd.

Best wishes,
Lénaïc.


Lénaïc Huard (3):
  cache.h: Introduce a generic "xdg_config_home_for(…)" function
  maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  maintenance: add support for systemd timers on Linux

 Documentation/git-maintenance.txt |  57 +++
 builtin/gc.c                      | 581 ++++++++++++++++++++++++++----
 cache.h                           |   7 +
 path.c                            |  13 +-
 t/t7900-maintenance.sh            | 110 +++++-
 5 files changed, 690 insertions(+), 78 deletions(-)

Diff-intervalle contre v8 :
-:  ---------- > 1:  1639bd151c cache.h: Introduce a generic "xdg_config_home_for(…)" function
-:  ---------- > 2:  ea5568269c maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
1:  faf56c078f ! 3:  8c25ebd3be maintenance: add support for systemd timers on Linux
    @@ builtin/gc.c: static int crontab_update_schedule(int run_maintenance, int fd)
      	return result;
      }
      
    -+#ifdef __linux__
    -+
     +static int real_is_systemd_timer_available(void)
     +{
     +	struct child_process child = CHILD_PROCESS_INIT;
    @@ builtin/gc.c: static int crontab_update_schedule(int run_maintenance, int fd)
     +	return 1;
     +}
     +
    -+#else
    -+
    -+static int real_is_systemd_timer_available(void)
    -+{
    -+	return 0;
    -+}
    -+
    -+#endif
    -+
     +static int is_systemd_timer_available(void)
     +{
     +	const char *cmd = "systemctl";
    @@ builtin/gc.c: static int crontab_update_schedule(int run_maintenance, int fd)
     +	const char *exec_path = git_exec_path();
     +
     +	int ret = systemd_timer_write_unit_templates(exec_path) ||
    -+	          systemd_timer_enable_unit(1, SCHEDULE_HOURLY) ||
    -+	          systemd_timer_enable_unit(1, SCHEDULE_DAILY) ||
    -+	          systemd_timer_enable_unit(1, SCHEDULE_WEEKLY);
    ++		  systemd_timer_enable_unit(1, SCHEDULE_HOURLY) ||
    ++		  systemd_timer_enable_unit(1, SCHEDULE_DAILY) ||
    ++		  systemd_timer_enable_unit(1, SCHEDULE_WEEKLY);
     +	if (ret)
     +		systemd_timer_delete_units();
     +	return ret;
-- 
2.33.0


^ permalink raw reply	[flat|nested] 138+ messages in thread

* [PATCH v9 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function
  2021-08-27 21:02               ` [PATCH v9 " Lénaïc Huard
@ 2021-08-27 21:02                 ` Lénaïc Huard
  2021-08-27 21:02                 ` [PATCH v9 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
                                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-08-27 21:02 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King,
	Lénaïc Huard

Current implementation of `xdg_config_home(filename)` returns
`$XDG_CONFIG_HOME/git/$filename`, with the `git` subdirectory inserted
between the `XDG_CONFIG_HOME` environment variable and the parameter.

This patch introduces a `xdg_config_home_for(subdir, filename)` function
which is more generic. It only concatenates "$XDG_CONFIG_HOME", or
"$HOME/.config" if the former isn’t defined, with the parameters,
without adding `git` in between.

`xdg_config_home(filename)` is now implemented by calling
`xdg_config_home_for("git", filename)` but this new generic function can
be used to compute the configuration directory of other programs.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 cache.h |  7 +++++++
 path.c  | 13 ++++++++++---
 2 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/cache.h b/cache.h
index bd4869beee..79c4e17ea4 100644
--- a/cache.h
+++ b/cache.h
@@ -1295,6 +1295,13 @@ int is_ntfs_dotmailmap(const char *name);
  */
 int looks_like_command_line_option(const char *str);
 
+/**
+ * Return a newly allocated string with the evaluation of
+ * "$XDG_CONFIG_HOME/$subdir/$filename" if $XDG_CONFIG_HOME is non-empty, otherwise
+ * "$HOME/.config/$subdir/$filename". Return NULL upon error.
+ */
+char *xdg_config_home_for(const char *subdir, const char *filename);
+
 /**
  * Return a newly allocated string with the evaluation of
  * "$XDG_CONFIG_HOME/git/$filename" if $XDG_CONFIG_HOME is non-empty, otherwise
diff --git a/path.c b/path.c
index 7bccd830e9..1b1de3be09 100644
--- a/path.c
+++ b/path.c
@@ -1503,21 +1503,28 @@ int looks_like_command_line_option(const char *str)
 	return str && str[0] == '-';
 }
 
-char *xdg_config_home(const char *filename)
+char *xdg_config_home_for(const char *subdir, const char *filename)
 {
 	const char *home, *config_home;
 
+	assert(subdir);
 	assert(filename);
 	config_home = getenv("XDG_CONFIG_HOME");
 	if (config_home && *config_home)
-		return mkpathdup("%s/git/%s", config_home, filename);
+		return mkpathdup("%s/%s/%s", config_home, subdir, filename);
 
 	home = getenv("HOME");
 	if (home)
-		return mkpathdup("%s/.config/git/%s", home, filename);
+		return mkpathdup("%s/.config/%s/%s", home, subdir, filename);
+
 	return NULL;
 }
 
+char *xdg_config_home(const char *filename)
+{
+	return xdg_config_home_for("git", filename);
+}
+
 char *xdg_cache_home(const char *filename)
 {
 	const char *home, *cache_home;
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* [PATCH v9 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-08-27 21:02               ` [PATCH v9 " Lénaïc Huard
  2021-08-27 21:02                 ` [PATCH v9 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
@ 2021-08-27 21:02                 ` Lénaïc Huard
  2021-08-27 23:54                   ` Ramsay Jones
  2021-08-27 21:02                 ` [PATCH v9 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
  2021-09-04 20:54                 ` [PATCH v10 0/3] " Lénaïc Huard
  3 siblings, 1 reply; 138+ messages in thread
From: Lénaïc Huard @ 2021-08-27 21:02 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King,
	Lénaïc Huard

Depending on the system, different schedulers can be used to schedule
the hourly, daily and weekly executions of `git maintenance run`:
* `launchctl` for MacOS,
* `schtasks` for Windows and
* `crontab` for everything else.

`git maintenance run` now has an option to let the end-user explicitly
choose which scheduler he wants to use:
`--scheduler=auto|crontab|launchctl|schtasks`.

When `git maintenance start --scheduler=XXX` is run, it not only
registers `git maintenance run` tasks in the scheduler XXX, it also
removes the `git maintenance run` tasks from all the other schedulers to
ensure we cannot have two schedulers launching concurrent identical
tasks.

The default value is `auto` which chooses a suitable scheduler for the
system.

`git maintenance stop` doesn't have any `--scheduler` parameter because
this command will try to remove the `git maintenance run` tasks from all
the available schedulers.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 Documentation/git-maintenance.txt |   9 +
 builtin/gc.c                      | 365 ++++++++++++++++++++++++------
 t/t7900-maintenance.sh            |  55 ++++-
 3 files changed, 354 insertions(+), 75 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 1e738ad398..576290b5c6 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -179,6 +179,15 @@ OPTIONS
 	`maintenance.<task>.enabled` configured as `true` are considered.
 	See the 'TASKS' section for the list of accepted `<task>` values.
 
+--scheduler=auto|crontab|launchctl|schtasks::
+	When combined with the `start` subcommand, specify the scheduler
+	for running the hourly, daily and weekly executions of
+	`git maintenance run`.
+	Possible values for `<scheduler>` are `auto`, `crontab` (POSIX),
+	`launchctl` (macOS), and `schtasks` (Windows).
+	When `auto` is specified, the appropriate platform-specific
+	scheduler is used. Default is `auto`.
+
 
 TROUBLESHOOTING
 ---------------
diff --git a/builtin/gc.c b/builtin/gc.c
index f05d2f0a1a..9e464d4a10 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1529,6 +1529,93 @@ static const char *get_frequency(enum schedule_priority schedule)
 	}
 }
 
+/*
+ * get_schedule_cmd` reads the GIT_TEST_MAINT_SCHEDULER environment variable
+ * to mock the schedulers that `git maintenance start` rely on.
+ *
+ * For test purpose, GIT_TEST_MAINT_SCHEDULER can be set to a comma-separated
+ * list of colon-separated key/value pairs where each pair contains a scheduler
+ * and its corresponding mock.
+ *
+ * * If $GET_TEST_MAINT_SCHEDULER is not set, return false and leave the
+ *   arguments unmodified.
+ *
+ * * If $GET_TEST_MAINT_SCHEDULER is set, return true.
+ *   In this case, the *cmd value is read as input.
+ *
+ *   * if the input value *cmd is the key of one of the comma-separated list
+ *     item, then *is_available is set to true and *cmd is modified and becomes
+ *     the mock command.
+ *
+ *   * if the input value *cmd isn’t the key of any of the comma-separated list
+ *     item, then *is_available is set to false.
+ *
+ * Ex.:
+ *   GIT_TEST_MAINT_SCHEDULER not set
+ *     +-------+-------------------------------------------------+
+ *     | Input |                     Output                      |
+ *     | *cmd  | return code |       *cmd        | *is_available |
+ *     +-------+-------------+-------------------+---------------+
+ *     | "foo" |    false    | "foo" (unchanged) |  (unchanged)  |
+ *     +-------+-------------+-------------------+---------------+
+ *
+ *   GIT_TEST_MAINT_SCHEDULER set to “foo:./mock_foo.sh,bar:./mock_bar.sh”
+ *     +-------+-------------------------------------------------+
+ *     | Input |                     Output                      |
+ *     | *cmd  | return code |       *cmd        | *is_available |
+ *     +-------+-------------+-------------------+---------------+
+ *     | "foo" |    true     |  "./mock.foo.sh"  |     true      |
+ *     | "qux" |    true     | "qux" (unchanged) |     false     |
+ *     +-------+-------------+-------------------+---------------+
+ */
+static int get_schedule_cmd(const char **cmd, int *is_available)
+{
+	char *testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
+	struct string_list_item *item;
+	struct string_list list = STRING_LIST_INIT_NODUP;
+
+	if (!testing)
+		return 0;
+
+	if (is_available)
+		*is_available = 0;
+
+	string_list_split_in_place(&list, testing, ',', -1);
+	for_each_string_list_item(item, &list) {
+		struct string_list pair = STRING_LIST_INIT_NODUP;
+
+		if (string_list_split_in_place(&pair, item->string, ':', 2) != 2)
+			continue;
+
+		if (!strcmp(*cmd, pair.items[0].string)) {
+			*cmd = pair.items[1].string;
+			if (is_available)
+				*is_available = 1;
+			string_list_clear(&list, 0);
+			UNLEAK(testing);
+			return 1;
+		}
+	}
+
+	string_list_clear(&list, 0);
+	free(testing);
+	return 1;
+}
+
+static int is_launchctl_available(void)
+{
+	const char *cmd = "launchctl";
+	int is_available;
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+#ifdef __APPLE__
+	return 1;
+#else
+	return 0;
+#endif
+}
+
 static char *launchctl_service_name(const char *frequency)
 {
 	struct strbuf label = STRBUF_INIT;
@@ -1555,19 +1642,17 @@ static char *launchctl_get_uid(void)
 	return xstrfmt("gui/%d", getuid());
 }
 
-static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
+static int launchctl_boot_plist(int enable, const char *filename)
 {
+	const char *cmd = "launchctl";
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
 	char *uid = launchctl_get_uid();
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&child.args, cmd);
-	if (enable)
-		strvec_push(&child.args, "bootstrap");
-	else
-		strvec_push(&child.args, "bootout");
-	strvec_push(&child.args, uid);
-	strvec_push(&child.args, filename);
+	strvec_pushl(&child.args, enable ? "bootstrap" : "bootout", uid,
+		     filename, NULL);
 
 	child.no_stderr = 1;
 	child.no_stdout = 1;
@@ -1581,26 +1666,26 @@ static int launchctl_boot_plist(int enable, const char *filename, const char *cm
 	return result;
 }
 
-static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)
+static int launchctl_remove_plist(enum schedule_priority schedule)
 {
 	const char *frequency = get_frequency(schedule);
 	char *name = launchctl_service_name(frequency);
 	char *filename = launchctl_service_filename(name);
-	int result = launchctl_boot_plist(0, filename, cmd);
+	int result = launchctl_boot_plist(0, filename);
 	unlink(filename);
 	free(filename);
 	free(name);
 	return result;
 }
 
-static int launchctl_remove_plists(const char *cmd)
+static int launchctl_remove_plists(void)
 {
-	return launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||
-		launchctl_remove_plist(SCHEDULE_DAILY, cmd) ||
-		launchctl_remove_plist(SCHEDULE_WEEKLY, cmd);
+	return launchctl_remove_plist(SCHEDULE_HOURLY) ||
+	       launchctl_remove_plist(SCHEDULE_DAILY) ||
+	       launchctl_remove_plist(SCHEDULE_WEEKLY);
 }
 
-static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule)
 {
 	FILE *plist;
 	int i;
@@ -1669,8 +1754,8 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 	fclose(plist);
 
 	/* bootout might fail if not already running, so ignore */
-	launchctl_boot_plist(0, filename, cmd);
-	if (launchctl_boot_plist(1, filename, cmd))
+	launchctl_boot_plist(0, filename);
+	if (launchctl_boot_plist(1, filename))
 		die(_("failed to bootstrap service %s"), filename);
 
 	free(filename);
@@ -1678,21 +1763,35 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 	return 0;
 }
 
-static int launchctl_add_plists(const char *cmd)
+static int launchctl_add_plists(void)
 {
 	const char *exec_path = git_exec_path();
 
-	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||
-		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||
-		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
+	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY) ||
+	       launchctl_schedule_plist(exec_path, SCHEDULE_DAILY) ||
+	       launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY);
 }
 
-static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int launchctl_update_schedule(int run_maintenance, int fd)
 {
 	if (run_maintenance)
-		return launchctl_add_plists(cmd);
+		return launchctl_add_plists();
 	else
-		return launchctl_remove_plists(cmd);
+		return launchctl_remove_plists();
+}
+
+static int is_schtasks_available(void)
+{
+	const char *cmd = "schtasks";
+	int is_available;
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+#ifdef GIT_WINDOWS_NATIVE
+	return 1;
+#else
+	return 0;
+#endif
 }
 
 static char *schtasks_task_name(const char *frequency)
@@ -1702,13 +1801,15 @@ static char *schtasks_task_name(const char *frequency)
 	return strbuf_detach(&label, NULL);
 }
 
-static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)
+static int schtasks_remove_task(enum schedule_priority schedule)
 {
+	const char *cmd = "schtasks";
 	int result;
 	struct strvec args = STRVEC_INIT;
 	const char *frequency = get_frequency(schedule);
 	char *name = schtasks_task_name(frequency);
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&args, cmd);
 	strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
 
@@ -1719,15 +1820,16 @@ static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd
 	return result;
 }
 
-static int schtasks_remove_tasks(const char *cmd)
+static int schtasks_remove_tasks(void)
 {
-	return schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||
-		schtasks_remove_task(SCHEDULE_DAILY, cmd) ||
-		schtasks_remove_task(SCHEDULE_WEEKLY, cmd);
+	return schtasks_remove_task(SCHEDULE_HOURLY) ||
+	       schtasks_remove_task(SCHEDULE_DAILY) ||
+	       schtasks_remove_task(SCHEDULE_WEEKLY);
 }
 
-static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule)
 {
+	const char *cmd = "schtasks";
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
 	const char *xml;
@@ -1736,6 +1838,8 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
 	char *name = schtasks_task_name(frequency);
 	struct strbuf tfilename = STRBUF_INIT;
 
+	get_schedule_cmd(&cmd, NULL);
+
 	strbuf_addf(&tfilename, "%s/schedule_%s_XXXXXX",
 		    get_git_common_dir(), frequency);
 	tfile = xmks_tempfile(tfilename.buf);
@@ -1840,28 +1944,52 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
 	return result;
 }
 
-static int schtasks_schedule_tasks(const char *cmd)
+static int schtasks_schedule_tasks(void)
 {
 	const char *exec_path = git_exec_path();
 
-	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||
-		schtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||
-		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
+	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY) ||
+	       schtasks_schedule_task(exec_path, SCHEDULE_DAILY) ||
+	       schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY);
 }
 
-static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int schtasks_update_schedule(int run_maintenance, int fd)
 {
 	if (run_maintenance)
-		return schtasks_schedule_tasks(cmd);
+		return schtasks_schedule_tasks();
 	else
-		return schtasks_remove_tasks(cmd);
+		return schtasks_remove_tasks();
+}
+
+static int is_crontab_available(void)
+{
+	const char *cmd = "crontab";
+	int is_available;
+	struct child_process child = CHILD_PROCESS_INIT;
+
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+	strvec_split(&child.args, cmd);
+	strvec_push(&child.args, "-l");
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child))
+		return 0;
+	/* Ignore exit code, as an empty crontab will return error. */
+	finish_command(&child);
+	return 1;
 }
 
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
-static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int crontab_update_schedule(int run_maintenance, int fd)
 {
+	const char *cmd = "crontab";
 	int result = 0;
 	int in_old_region = 0;
 	struct child_process crontab_list = CHILD_PROCESS_INIT;
@@ -1869,6 +1997,7 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 	FILE *cron_list, *cron_in;
 	struct strbuf line = STRBUF_INIT;
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&crontab_list.args, cmd);
 	strvec_push(&crontab_list.args, "-l");
 	crontab_list.in = -1;
@@ -1945,66 +2074,160 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 	return result;
 }
 
+enum scheduler {
+	SCHEDULER_INVALID = -1,
+	SCHEDULER_AUTO,
+	SCHEDULER_CRON,
+	SCHEDULER_LAUNCHCTL,
+	SCHEDULER_SCHTASKS,
+};
+
+static const struct {
+	const char *name;
+	int (*is_available)(void);
+	int (*update_schedule)(int run_maintenance, int fd);
+} scheduler_fn[] = {
+	[SCHEDULER_CRON] = {
+		.name = "crontab",
+		.is_available = is_crontab_available,
+		.update_schedule = crontab_update_schedule,
+	},
+	[SCHEDULER_LAUNCHCTL] = {
+		.name = "launchctl",
+		.is_available = is_launchctl_available,
+		.update_schedule = launchctl_update_schedule,
+	},
+	[SCHEDULER_SCHTASKS] = {
+		.name = "schtasks",
+		.is_available = is_schtasks_available,
+		.update_schedule = schtasks_update_schedule,
+	},
+};
+
+static enum scheduler parse_scheduler(const char *value)
+{
+	if (!value)
+		return SCHEDULER_INVALID;
+	else if (!strcasecmp(value, "auto"))
+		return SCHEDULER_AUTO;
+	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
+		return SCHEDULER_CRON;
+	else if (!strcasecmp(value, "launchctl"))
+		return SCHEDULER_LAUNCHCTL;
+	else if (!strcasecmp(value, "schtasks"))
+		return SCHEDULER_SCHTASKS;
+	else
+		return SCHEDULER_INVALID;
+}
+
+static int maintenance_opt_scheduler(const struct option *opt, const char *arg,
+				     int unset)
+{
+	enum scheduler *scheduler = opt->value;
+
+	BUG_ON_OPT_NEG(unset);
+
+	*scheduler = parse_scheduler(arg);
+	if (*scheduler == SCHEDULER_INVALID)
+		return error(_("unrecognized --scheduler argument '%s'"), arg);
+	return 0;
+}
+
+struct maintenance_start_opts {
+	enum scheduler scheduler;
+};
+
+static enum scheduler resolve_scheduler(enum scheduler scheduler)
+{
+	if (scheduler != SCHEDULER_AUTO)
+		return scheduler;
+
 #if defined(__APPLE__)
-static const char platform_scheduler[] = "launchctl";
+	return SCHEDULER_LAUNCHCTL;
+
 #elif defined(GIT_WINDOWS_NATIVE)
-static const char platform_scheduler[] = "schtasks";
+	return SCHEDULER_SCHTASKS;
+
 #else
-static const char platform_scheduler[] = "crontab";
+	return SCHEDULER_CRON;
 #endif
+}
 
-static int update_background_schedule(int enable)
+static void validate_scheduler(enum scheduler scheduler)
 {
-	int result;
-	const char *scheduler = platform_scheduler;
-	const char *cmd = scheduler;
-	char *testing;
+	if (scheduler == SCHEDULER_INVALID)
+		BUG("invalid scheduler");
+	if (scheduler == SCHEDULER_AUTO)
+		BUG("resolve_scheduler should have been called before");
+
+	if (!scheduler_fn[scheduler].is_available())
+		die(_("%s scheduler is not available"),
+		    scheduler_fn[scheduler].name);
+}
+
+static int update_background_schedule(const struct maintenance_start_opts *opts,
+				      int enable)
+{
+	unsigned int i;
+	int result = 0;
 	struct lock_file lk;
 	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
 
-	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
-	if (testing) {
-		char *sep = strchr(testing, ':');
-		if (!sep)
-			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
-		*sep = '\0';
-		scheduler = testing;
-		cmd = sep + 1;
+	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {
+		free(lock_path);
+		return error(_("another process is scheduling background maintenance"));
 	}
 
-	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {
-		result = error(_("another process is scheduling background maintenance"));
-		goto cleanup;
+	for (i = 1; i < ARRAY_SIZE(scheduler_fn); i++) {
+		if (enable && opts->scheduler == i)
+			continue;
+		if (!scheduler_fn[i].is_available())
+			continue;
+		scheduler_fn[i].update_schedule(0, get_lock_file_fd(&lk));
 	}
 
-	if (!strcmp(scheduler, "launchctl"))
-		result = launchctl_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else if (!strcmp(scheduler, "schtasks"))
-		result = schtasks_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else if (!strcmp(scheduler, "crontab"))
-		result = crontab_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else
-		die("unknown background scheduler: %s", scheduler);
+	if (enable)
+		result = scheduler_fn[opts->scheduler].update_schedule(
+			1, get_lock_file_fd(&lk));
 
 	rollback_lock_file(&lk);
 
-cleanup:
 	free(lock_path);
-	free(testing);
 	return result;
 }
 
-static int maintenance_start(void)
+static const char *const builtin_maintenance_start_usage[] = {
+	N_("git maintenance start [--scheduler=<scheduler>]"),
+	NULL
+};
+
+static int maintenance_start(int argc, const char **argv, const char *prefix)
 {
+	struct maintenance_start_opts opts = { 0 };
+	struct option options[] = {
+		OPT_CALLBACK_F(
+			0, "scheduler", &opts.scheduler, N_("scheduler"),
+			N_("scheduler to trigger git maintenance run"),
+			PARSE_OPT_NONEG, maintenance_opt_scheduler),
+		OPT_END()
+	};
+
+	argc = parse_options(argc, argv, prefix, options,
+			     builtin_maintenance_start_usage, 0);
+	if (argc)
+		usage_with_options(builtin_maintenance_start_usage, options);
+
+	opts.scheduler = resolve_scheduler(opts.scheduler);
+	validate_scheduler(opts.scheduler);
+
 	if (maintenance_register())
 		warning(_("failed to add repo to global config"));
-
-	return update_background_schedule(1);
+	return update_background_schedule(&opts, 1);
 }
 
 static int maintenance_stop(void)
 {
-	return update_background_schedule(0);
+	return update_background_schedule(NULL, 0);
 }
 
 static const char builtin_maintenance_usage[] =	N_("git maintenance <subcommand> [<options>]");
@@ -2018,7 +2241,7 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
 	if (!strcmp(argv[1], "run"))
 		return maintenance_run(argc - 1, argv + 1, prefix);
 	if (!strcmp(argv[1], "start"))
-		return maintenance_start();
+		return maintenance_start(argc - 1, argv + 1, prefix);
 	if (!strcmp(argv[1], "stop"))
 		return maintenance_stop();
 	if (!strcmp(argv[1], "register"))
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 58f46c77e6..27bce7992c 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -492,8 +492,21 @@ test_expect_success !MINGW 'register and unregister with regex metacharacters' '
 		maintenance.repo "$(pwd)/$META"
 '
 
+test_expect_success 'start --scheduler=<scheduler>' '
+	test_expect_code 129 git maintenance start --scheduler=foo 2>err &&
+	test_i18ngrep "unrecognized --scheduler argument" err &&
+
+	test_expect_code 129 git maintenance start --no-scheduler 2>err &&
+	test_i18ngrep "unknown option" err &&
+
+	test_expect_code 128 \
+		env GIT_TEST_MAINT_SCHEDULER="launchctl:true,schtasks:true" \
+		git maintenance start --scheduler=crontab 2>err &&
+	test_i18ngrep "fatal: crontab scheduler is not available" err
+'
+
 test_expect_success 'start from empty cron table' '
-	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -516,7 +529,7 @@ test_expect_success 'stop from existing schedule' '
 
 test_expect_success 'start preserves existing schedule' '
 	echo "Important information!" >cron.txt &&
-	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
 	grep "Important information!" cron.txt
 '
 
@@ -545,7 +558,7 @@ test_expect_success 'start and stop macOS maintenance' '
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start --scheduler=launchctl &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -596,7 +609,7 @@ test_expect_success 'start and stop Windows maintenance' '
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start --scheduler=schtasks &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -619,6 +632,40 @@ test_expect_success 'start and stop Windows maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success 'start and stop when several schedulers are available' '
+	write_script print-args <<-\EOF &&
+	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
+	EOF
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >expect &&
+	for frequency in hourly daily weekly
+	do
+		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
+		echo "launchctl bootout gui/[UID] $PLIST" >>expect &&
+		echo "launchctl bootstrap gui/[UID] $PLIST" >>expect || return 1
+	done &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* [PATCH v9 3/3] maintenance: add support for systemd timers on Linux
  2021-08-27 21:02               ` [PATCH v9 " Lénaïc Huard
  2021-08-27 21:02                 ` [PATCH v9 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
  2021-08-27 21:02                 ` [PATCH v9 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
@ 2021-08-27 21:02                 ` Lénaïc Huard
  2021-09-04 20:54                 ` [PATCH v10 0/3] " Lénaïc Huard
  3 siblings, 0 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-08-27 21:02 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King,
	Lénaïc Huard

The existing mechanism for scheduling background maintenance is done
through cron. On Linux systems managed by systemd, systemd provides an
alternative to schedule recurring tasks: systemd timers.

The main motivations to implement systemd timers in addition to cron
are:
* cron is optional and Linux systems running systemd might not have it
  installed.
* The execution of `crontab -l` can tell us if cron is installed but not
  if the daemon is actually running.
* With systemd, each service is run in its own cgroup and its logs are
  tagged by the service inside journald. With cron, all scheduled tasks
  are running in the cron daemon cgroup and all the logs of the
  user-scheduled tasks are pretended to belong to the system cron
  service.
  Concretely, a user that doesn’t have access to the system logs won’t
  have access to the log of their own tasks scheduled by cron whereas
  they will have access to the log of their own tasks scheduled by
  systemd timer.
  Although `cron` attempts to send email, that email may go unseen by
  the user because these days, local mailboxes are not heavily used
  anymore.

In order to schedule git maintenance, we need two unit template files:
* ~/.config/systemd/user/git-maintenance@.service
  to define the command to be started by systemd and
* ~/.config/systemd/user/git-maintenance@.timer
  to define the schedule at which the command should be run.

Those units are templates that are parameterized by the frequency.

Based on those templates, 3 timers are started:
* git-maintenance@hourly.timer
* git-maintenance@daily.timer
* git-maintenance@weekly.timer

The command launched by those three timers are the same as with the
other scheduling methods:

/path/to/git for-each-repo --exec-path=/path/to
--config=maintenance.repo maintenance run --schedule=%i

with the full path for git to ensure that the version of git launched
for the scheduled maintenance is the same as the one used to run
`maintenance start`.

The timer unit contains `Persistent=true` so that, if the computer is
powered down when a maintenance task should run, the task will be run
when the computer is back powered on.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 Documentation/git-maintenance.txt |  58 +++++++-
 builtin/gc.c                      | 216 ++++++++++++++++++++++++++++++
 t/t7900-maintenance.sh            |  67 ++++++++-
 3 files changed, 330 insertions(+), 11 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 576290b5c6..e2cfb68ab5 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -179,14 +179,16 @@ OPTIONS
 	`maintenance.<task>.enabled` configured as `true` are considered.
 	See the 'TASKS' section for the list of accepted `<task>` values.
 
---scheduler=auto|crontab|launchctl|schtasks::
+--scheduler=auto|crontab|systemd-timer|launchctl|schtasks::
 	When combined with the `start` subcommand, specify the scheduler
 	for running the hourly, daily and weekly executions of
 	`git maintenance run`.
-	Possible values for `<scheduler>` are `auto`, `crontab` (POSIX),
-	`launchctl` (macOS), and `schtasks` (Windows).
-	When `auto` is specified, the appropriate platform-specific
-	scheduler is used. Default is `auto`.
+	Possible values for `<scheduler>` are `auto`, `crontab`
+	(POSIX), `systemd-timer` (Linux), `launchctl` (macOS), and
+	`schtasks` (Windows). When `auto` is specified, the
+	appropriate platform-specific scheduler is used; on Linux,
+	`systemd-timer` is used if available, otherwise
+	`crontab`. Default is `auto`.
 
 
 TROUBLESHOOTING
@@ -286,6 +288,52 @@ schedule to ensure you are executing the correct binaries in your
 schedule.
 
 
+BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMS
+-----------------------------------------------
+
+While Linux supports `cron`, depending on the distribution, `cron` may
+be an optional package not necessarily installed. On modern Linux
+distributions, systemd timers are superseding it.
+
+If user systemd timers are available, they will be used as a replacement
+of `cron`.
+
+In this case, `git maintenance start` will create user systemd timer units
+and start the timers. The current list of user-scheduled tasks can be found
+by running `systemctl --user list-timers`. The timers written by `git
+maintenance start` are similar to this:
+
+-----------------------------------------------------------------------
+$ systemctl --user list-timers
+NEXT                         LEFT          LAST                         PASSED     UNIT                         ACTIVATES
+Thu 2021-04-29 19:00:00 CEST 42min left    Thu 2021-04-29 18:00:11 CEST 17min ago  git-maintenance@hourly.timer git-maintenance@hourly.service
+Fri 2021-04-30 00:00:00 CEST 5h 42min left Thu 2021-04-29 00:00:11 CEST 18h ago    git-maintenance@daily.timer  git-maintenance@daily.service
+Mon 2021-05-03 00:00:00 CEST 3 days left   Mon 2021-04-26 00:00:11 CEST 3 days ago git-maintenance@weekly.timer git-maintenance@weekly.service
+-----------------------------------------------------------------------
+
+One timer is registered for each `--schedule=<frequency>` option.
+
+The definition of the systemd units can be inspected in the following files:
+
+-----------------------------------------------------------------------
+~/.config/systemd/user/git-maintenance@.timer
+~/.config/systemd/user/git-maintenance@.service
+~/.config/systemd/user/timers.target.wants/git-maintenance@hourly.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@daily.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@weekly.timer
+-----------------------------------------------------------------------
+
+`git maintenance start` will overwrite these files and start the timer
+again with `systemctl --user`, so any customization should be done by
+creating a drop-in file, i.e. a `.conf` suffixed file in the
+`~/.config/systemd/user/git-maintenance@.service.d` directory.
+
+`git maintenance stop` will stop the user systemd timers and delete
+the above mentioned files.
+
+For more details, see `systemd.timer(5)`.
+
+
 BACKGROUND MAINTENANCE ON MACOS SYSTEMS
 ---------------------------------------
 
diff --git a/builtin/gc.c b/builtin/gc.c
index 9e464d4a10..d97bfc44ae 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -2074,10 +2074,210 @@ static int crontab_update_schedule(int run_maintenance, int fd)
 	return result;
 }
 
+static int real_is_systemd_timer_available(void)
+{
+	struct child_process child = CHILD_PROCESS_INIT;
+
+	strvec_pushl(&child.args, "systemctl", "--user", "list-timers", NULL);
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child))
+		return 0;
+	if (finish_command(&child))
+		return 0;
+	return 1;
+}
+
+static int is_systemd_timer_available(void)
+{
+	const char *cmd = "systemctl";
+	int is_available;
+
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+	return real_is_systemd_timer_available();
+}
+
+static char *xdg_config_home_systemd(const char *filename)
+{
+	return xdg_config_home_for("systemd/user", filename);
+}
+
+static int systemd_timer_enable_unit(int enable,
+				     enum schedule_priority schedule)
+{
+	const char *cmd = "systemctl";
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *frequency = get_frequency(schedule);
+
+	/*
+	 * Disabling the systemd unit while it is already disabled makes
+	 * systemctl print an error.
+	 * Let's ignore it since it means we already are in the expected state:
+	 * the unit is disabled.
+	 *
+	 * On the other hand, enabling a systemd unit which is already enabled
+	 * produces no error.
+	 */
+	if (!enable)
+		child.no_stderr = 1;
+
+	get_schedule_cmd(&cmd, NULL);
+	strvec_split(&child.args, cmd);
+	strvec_pushl(&child.args, "--user", enable ? "enable" : "disable",
+		     "--now", NULL);
+	strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency);
+
+	if (start_command(&child))
+		return error(_("failed to start systemctl"));
+	if (finish_command(&child))
+		/*
+		 * Disabling an already disabled systemd unit makes
+		 * systemctl fail.
+		 * Let's ignore this failure.
+		 *
+		 * Enabling an enabled systemd unit doesn't fail.
+		 */
+		if (enable)
+			return error(_("failed to run systemctl"));
+	return 0;
+}
+
+static int systemd_timer_delete_unit_templates(void)
+{
+	int ret = 0;
+	char *filename = xdg_config_home_systemd("git-maintenance@.timer");
+	if (unlink(filename) && !is_missing_file_error(errno))
+		ret = error_errno(_("failed to delete '%s'"), filename);
+	FREE_AND_NULL(filename);
+
+	filename = xdg_config_home_systemd("git-maintenance@.service");
+	if (unlink(filename) && !is_missing_file_error(errno))
+		ret = error_errno(_("failed to delete '%s'"), filename);
+
+	free(filename);
+	return ret;
+}
+
+static int systemd_timer_delete_units(void)
+{
+	return systemd_timer_enable_unit(0, SCHEDULE_HOURLY) ||
+	       systemd_timer_enable_unit(0, SCHEDULE_DAILY) ||
+	       systemd_timer_enable_unit(0, SCHEDULE_WEEKLY) ||
+	       systemd_timer_delete_unit_templates();
+}
+
+static int systemd_timer_write_unit_templates(const char *exec_path)
+{
+	char *filename;
+	FILE *file;
+	const char *unit;
+
+	filename = xdg_config_home_systemd("git-maintenance@.timer");
+	if (safe_create_leading_directories(filename)) {
+		error(_("failed to create directories for '%s'"), filename);
+		goto error;
+	}
+	file = fopen_or_warn(filename, "w");
+	if (file == NULL)
+		goto error;
+
+	unit = "# This file was created and is maintained by Git.\n"
+	       "# Any edits made in this file might be replaced in the future\n"
+	       "# by a Git command.\n"
+	       "\n"
+	       "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Timer]\n"
+	       "OnCalendar=%i\n"
+	       "Persistent=true\n"
+	       "\n"
+	       "[Install]\n"
+	       "WantedBy=timers.target\n";
+	if (fputs(unit, file) == EOF) {
+		error(_("failed to write to '%s'"), filename);
+		fclose(file);
+		goto error;
+	}
+	if (fclose(file) == EOF) {
+		error_errno(_("failed to flush '%s'"), filename);
+		goto error;
+	}
+	free(filename);
+
+	filename = xdg_config_home_systemd("git-maintenance@.service");
+	file = fopen_or_warn(filename, "w");
+	if (file == NULL)
+		goto error;
+
+	unit = "# This file was created and is maintained by Git.\n"
+	       "# Any edits made in this file might be replaced in the future\n"
+	       "# by a Git command.\n"
+	       "\n"
+	       "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Service]\n"
+	       "Type=oneshot\n"
+	       "ExecStart=\"%s/git\" --exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
+	       "LockPersonality=yes\n"
+	       "MemoryDenyWriteExecute=yes\n"
+	       "NoNewPrivileges=yes\n"
+	       "RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\n"
+	       "RestrictNamespaces=yes\n"
+	       "RestrictRealtime=yes\n"
+	       "RestrictSUIDSGID=yes\n"
+	       "SystemCallArchitectures=native\n"
+	       "SystemCallFilter=@system-service\n";
+	if (fprintf(file, unit, exec_path, exec_path) < 0) {
+		error(_("failed to write to '%s'"), filename);
+		fclose(file);
+		goto error;
+	}
+	if (fclose(file) == EOF) {
+		error_errno(_("failed to flush '%s'"), filename);
+		goto error;
+	}
+	free(filename);
+	return 0;
+
+error:
+	free(filename);
+	systemd_timer_delete_unit_templates();
+	return -1;
+}
+
+static int systemd_timer_setup_units(void)
+{
+	const char *exec_path = git_exec_path();
+
+	int ret = systemd_timer_write_unit_templates(exec_path) ||
+		  systemd_timer_enable_unit(1, SCHEDULE_HOURLY) ||
+		  systemd_timer_enable_unit(1, SCHEDULE_DAILY) ||
+		  systemd_timer_enable_unit(1, SCHEDULE_WEEKLY);
+	if (ret)
+		systemd_timer_delete_units();
+	return ret;
+}
+
+static int systemd_timer_update_schedule(int run_maintenance, int fd)
+{
+	if (run_maintenance)
+		return systemd_timer_setup_units();
+	else
+		return systemd_timer_delete_units();
+}
+
 enum scheduler {
 	SCHEDULER_INVALID = -1,
 	SCHEDULER_AUTO,
 	SCHEDULER_CRON,
+	SCHEDULER_SYSTEMD,
 	SCHEDULER_LAUNCHCTL,
 	SCHEDULER_SCHTASKS,
 };
@@ -2092,6 +2292,11 @@ static const struct {
 		.is_available = is_crontab_available,
 		.update_schedule = crontab_update_schedule,
 	},
+	[SCHEDULER_SYSTEMD] = {
+		.name = "systemctl",
+		.is_available = is_systemd_timer_available,
+		.update_schedule = systemd_timer_update_schedule,
+	},
 	[SCHEDULER_LAUNCHCTL] = {
 		.name = "launchctl",
 		.is_available = is_launchctl_available,
@@ -2112,6 +2317,9 @@ static enum scheduler parse_scheduler(const char *value)
 		return SCHEDULER_AUTO;
 	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
 		return SCHEDULER_CRON;
+	else if (!strcasecmp(value, "systemd") ||
+		 !strcasecmp(value, "systemd-timer"))
+		return SCHEDULER_SYSTEMD;
 	else if (!strcasecmp(value, "launchctl"))
 		return SCHEDULER_LAUNCHCTL;
 	else if (!strcasecmp(value, "schtasks"))
@@ -2148,6 +2356,14 @@ static enum scheduler resolve_scheduler(enum scheduler scheduler)
 #elif defined(GIT_WINDOWS_NATIVE)
 	return SCHEDULER_SCHTASKS;
 
+#elif defined(__linux__)
+	if (is_systemd_timer_available())
+		return SCHEDULER_SYSTEMD;
+	else if (is_crontab_available())
+		return SCHEDULER_CRON;
+	else
+		die(_("neither systemd timers nor crontab are available"));
+
 #else
 	return SCHEDULER_CRON;
 #endif
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 27bce7992c..265f7793f5 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -20,6 +20,18 @@ test_xmllint () {
 	fi
 }
 
+test_lazy_prereq SYSTEMD_ANALYZE '
+	systemd-analyze --help >out &&
+	grep verify out
+'
+
+test_systemd_analyze_verify () {
+	if test_have_prereq SYSTEMD_ANALYZE
+	then
+		systemd-analyze verify "$@"
+	fi
+}
+
 test_expect_success 'help text' '
 	test_expect_code 129 git maintenance -h 2>err &&
 	test_i18ngrep "usage: git maintenance <subcommand>" err &&
@@ -632,15 +644,56 @@ test_expect_success 'start and stop Windows maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success 'start and stop Linux/systemd maintenance' '
+	write_script print-args <<-\EOF &&
+	printf "%s\n" "$*" >>args
+	EOF
+
+	XDG_CONFIG_HOME="$PWD" &&
+	export XDG_CONFIG_HOME &&
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance start --scheduler=systemd-timer &&
+
+	# start registers the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_systemd_analyze_verify "systemd/user/git-maintenance@.service" &&
+
+	printf -- "--user enable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_path_is_missing "systemd/user/git-maintenance@.timer" &&
+	test_path_is_missing "systemd/user/git-maintenance@.service" &&
+
+	printf -- "--user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'start and stop when several schedulers are available' '
 	write_script print-args <<-\EOF &&
 	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
-	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=systemd-timer &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
 		hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
+	printf -- "systemctl --user enable --now git-maintenance@%s.timer\n" hourly daily weekly >>expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
 	for frequency in hourly daily weekly
 	do
 		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
@@ -650,17 +703,19 @@ test_expect_success 'start and stop when several schedulers are available' '
 	test_cmp expect args &&
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
 	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
-		hourly daily weekly >expect &&
+		hourly daily weekly >>expect &&
 	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
 		hourly daily weekly >>expect &&
 	test_cmp expect args &&
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
 	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
-		hourly daily weekly >expect &&
+		hourly daily weekly >>expect &&
 	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
 		hourly daily weekly >>expect &&
 	test_cmp expect args
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* Re: [PATCH v9 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-08-27 21:02                 ` [PATCH v9 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
@ 2021-08-27 23:54                   ` Ramsay Jones
  0 siblings, 0 replies; 138+ messages in thread
From: Ramsay Jones @ 2021-08-27 23:54 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King

Hi Lénaïc,

I noticed a couple of typos while tracking down a test failure
on the previous version of this series:

On 27/08/2021 22:02, Lénaïc Huard wrote:
> Depending on the system, different schedulers can be used to schedule
> the hourly, daily and weekly executions of `git maintenance run`:
> * `launchctl` for MacOS,
> * `schtasks` for Windows and
> * `crontab` for everything else.
> 
> `git maintenance run` now has an option to let the end-user explicitly
> choose which scheduler he wants to use:
> `--scheduler=auto|crontab|launchctl|schtasks`.
> 
> When `git maintenance start --scheduler=XXX` is run, it not only
> registers `git maintenance run` tasks in the scheduler XXX, it also
> removes the `git maintenance run` tasks from all the other schedulers to
> ensure we cannot have two schedulers launching concurrent identical
> tasks.
> 
> The default value is `auto` which chooses a suitable scheduler for the
> system.
> 
> `git maintenance stop` doesn't have any `--scheduler` parameter because
> this command will try to remove the `git maintenance run` tasks from all
> the available schedulers.
> 
> Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
> ---
>  Documentation/git-maintenance.txt |   9 +
>  builtin/gc.c                      | 365 ++++++++++++++++++++++++------
>  t/t7900-maintenance.sh            |  55 ++++-
>  3 files changed, 354 insertions(+), 75 deletions(-)
> 
> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
> index 1e738ad398..576290b5c6 100644
> --- a/Documentation/git-maintenance.txt
> +++ b/Documentation/git-maintenance.txt
> @@ -179,6 +179,15 @@ OPTIONS
>  	`maintenance.<task>.enabled` configured as `true` are considered.
>  	See the 'TASKS' section for the list of accepted `<task>` values.
>  
> +--scheduler=auto|crontab|launchctl|schtasks::
> +	When combined with the `start` subcommand, specify the scheduler
> +	for running the hourly, daily and weekly executions of
> +	`git maintenance run`.
> +	Possible values for `<scheduler>` are `auto`, `crontab` (POSIX),
> +	`launchctl` (macOS), and `schtasks` (Windows).
> +	When `auto` is specified, the appropriate platform-specific
> +	scheduler is used. Default is `auto`.
> +
>  
>  TROUBLESHOOTING
>  ---------------
> diff --git a/builtin/gc.c b/builtin/gc.c
> index f05d2f0a1a..9e464d4a10 100644
> --- a/builtin/gc.c
> +++ b/builtin/gc.c
> @@ -1529,6 +1529,93 @@ static const char *get_frequency(enum schedule_priority schedule)
>  	}
>  }
>  
> +/*
> + * get_schedule_cmd` reads the GIT_TEST_MAINT_SCHEDULER environment variable
> + * to mock the schedulers that `git maintenance start` rely on.
> + *
> + * For test purpose, GIT_TEST_MAINT_SCHEDULER can be set to a comma-separated
> + * list of colon-separated key/value pairs where each pair contains a scheduler
> + * and its corresponding mock.
> + *
> + * * If $GET_TEST_MAINT_SCHEDULER is not set, return false and leave the

s/GET/GIT/

> + *   arguments unmodified.
> + *
> + * * If $GET_TEST_MAINT_SCHEDULER is set, return true.

s/GET/GIT/

ATB,
Ramsay Jones

> + *   In this case, the *cmd value is read as input.
> + *
> + *   * if the input value *cmd is the key of one of the comma-separated list
> + *     item, then *is_available is set to true and *cmd is modified and becomes
> + *     the mock command.
> + *
> + *   * if the input value *cmd isn’t the key of any of the comma-separated list
> + *     item, then *is_available is set to false.
> + *
> + * Ex.:
> + *   GIT_TEST_MAINT_SCHEDULER not set
> + *     +-------+-------------------------------------------------+
> + *     | Input |                     Output                      |
> + *     | *cmd  | return code |       *cmd        | *is_available |
> + *     +-------+-------------+-------------------+---------------+
> + *     | "foo" |    false    | "foo" (unchanged) |  (unchanged)  |
> + *     +-------+-------------+-------------------+---------------+
> + *
> + *   GIT_TEST_MAINT_SCHEDULER set to “foo:./mock_foo.sh,bar:./mock_bar.sh”
> + *     +-------+-------------------------------------------------+
> + *     | Input |                     Output                      |
> + *     | *cmd  | return code |       *cmd        | *is_available |
> + *     +-------+-------------+-------------------+---------------+
> + *     | "foo" |    true     |  "./mock.foo.sh"  |     true      |
> + *     | "qux" |    true     | "qux" (unchanged) |     false     |
> + *     +-------+-------------+-------------------+---------------+
> + */
> +static int get_schedule_cmd(const char **cmd, int *is_available)
> +{
> +	char *testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
> +	struct string_list_item *item;
> +	struct string_list list = STRING_LIST_INIT_NODUP;
> +
> +	if (!testing)
> +		return 0;
> +
> +	if (is_available)
> +		*is_available = 0;
> +
> +	string_list_split_in_place(&list, testing, ',', -1);
> +	for_each_string_list_item(item, &list) {
> +		struct string_list pair = STRING_LIST_INIT_NODUP;
> +
> +		if (string_list_split_in_place(&pair, item->string, ':', 2) != 2)
> +			continue;
> +
> +		if (!strcmp(*cmd, pair.items[0].string)) {
> +			*cmd = pair.items[1].string;
> +			if (is_available)
> +				*is_available = 1;
> +			string_list_clear(&list, 0);
> +			UNLEAK(testing);
> +			return 1;
> +		}
> +	}
> +
> +	string_list_clear(&list, 0);
> +	free(testing);
> +	return 1;
> +}
> +
> +static int is_launchctl_available(void)
> +{
> +	const char *cmd = "launchctl";
> +	int is_available;
> +	if (get_schedule_cmd(&cmd, &is_available))
> +		return is_available;
> +
> +#ifdef __APPLE__
> +	return 1;
> +#else
> +	return 0;
> +#endif
> +}
> +
>  static char *launchctl_service_name(const char *frequency)
>  {
>  	struct strbuf label = STRBUF_INIT;
> @@ -1555,19 +1642,17 @@ static char *launchctl_get_uid(void)
>  	return xstrfmt("gui/%d", getuid());
>  }
>  
> -static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
> +static int launchctl_boot_plist(int enable, const char *filename)
>  {
> +	const char *cmd = "launchctl";
>  	int result;
>  	struct child_process child = CHILD_PROCESS_INIT;
>  	char *uid = launchctl_get_uid();
>  
> +	get_schedule_cmd(&cmd, NULL);
>  	strvec_split(&child.args, cmd);
> -	if (enable)
> -		strvec_push(&child.args, "bootstrap");
> -	else
> -		strvec_push(&child.args, "bootout");
> -	strvec_push(&child.args, uid);
> -	strvec_push(&child.args, filename);
> +	strvec_pushl(&child.args, enable ? "bootstrap" : "bootout", uid,
> +		     filename, NULL);
>  
>  	child.no_stderr = 1;
>  	child.no_stdout = 1;
> @@ -1581,26 +1666,26 @@ static int launchctl_boot_plist(int enable, const char *filename, const char *cm
>  	return result;
>  }
>  
> -static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)
> +static int launchctl_remove_plist(enum schedule_priority schedule)
>  {
>  	const char *frequency = get_frequency(schedule);
>  	char *name = launchctl_service_name(frequency);
>  	char *filename = launchctl_service_filename(name);
> -	int result = launchctl_boot_plist(0, filename, cmd);
> +	int result = launchctl_boot_plist(0, filename);
>  	unlink(filename);
>  	free(filename);
>  	free(name);
>  	return result;
>  }
>  
> -static int launchctl_remove_plists(const char *cmd)
> +static int launchctl_remove_plists(void)
>  {
> -	return launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||
> -		launchctl_remove_plist(SCHEDULE_DAILY, cmd) ||
> -		launchctl_remove_plist(SCHEDULE_WEEKLY, cmd);
> +	return launchctl_remove_plist(SCHEDULE_HOURLY) ||
> +	       launchctl_remove_plist(SCHEDULE_DAILY) ||
> +	       launchctl_remove_plist(SCHEDULE_WEEKLY);
>  }
>  
> -static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)
> +static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule)
>  {
>  	FILE *plist;
>  	int i;
> @@ -1669,8 +1754,8 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
>  	fclose(plist);
>  
>  	/* bootout might fail if not already running, so ignore */
> -	launchctl_boot_plist(0, filename, cmd);
> -	if (launchctl_boot_plist(1, filename, cmd))
> +	launchctl_boot_plist(0, filename);
> +	if (launchctl_boot_plist(1, filename))
>  		die(_("failed to bootstrap service %s"), filename);
>  
>  	free(filename);
> @@ -1678,21 +1763,35 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
>  	return 0;
>  }
>  
> -static int launchctl_add_plists(const char *cmd)
> +static int launchctl_add_plists(void)
>  {
>  	const char *exec_path = git_exec_path();
>  
> -	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||
> -		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||
> -		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
> +	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY) ||
> +	       launchctl_schedule_plist(exec_path, SCHEDULE_DAILY) ||
> +	       launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY);
>  }
>  
> -static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)
> +static int launchctl_update_schedule(int run_maintenance, int fd)
>  {
>  	if (run_maintenance)
> -		return launchctl_add_plists(cmd);
> +		return launchctl_add_plists();
>  	else
> -		return launchctl_remove_plists(cmd);
> +		return launchctl_remove_plists();
> +}
> +
> +static int is_schtasks_available(void)
> +{
> +	const char *cmd = "schtasks";
> +	int is_available;
> +	if (get_schedule_cmd(&cmd, &is_available))
> +		return is_available;
> +
> +#ifdef GIT_WINDOWS_NATIVE
> +	return 1;
> +#else
> +	return 0;
> +#endif
>  }
>  
>  static char *schtasks_task_name(const char *frequency)
> @@ -1702,13 +1801,15 @@ static char *schtasks_task_name(const char *frequency)
>  	return strbuf_detach(&label, NULL);
>  }
>  
> -static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)
> +static int schtasks_remove_task(enum schedule_priority schedule)
>  {
> +	const char *cmd = "schtasks";
>  	int result;
>  	struct strvec args = STRVEC_INIT;
>  	const char *frequency = get_frequency(schedule);
>  	char *name = schtasks_task_name(frequency);
>  
> +	get_schedule_cmd(&cmd, NULL);
>  	strvec_split(&args, cmd);
>  	strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
>  
> @@ -1719,15 +1820,16 @@ static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd
>  	return result;
>  }
>  
> -static int schtasks_remove_tasks(const char *cmd)
> +static int schtasks_remove_tasks(void)
>  {
> -	return schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||
> -		schtasks_remove_task(SCHEDULE_DAILY, cmd) ||
> -		schtasks_remove_task(SCHEDULE_WEEKLY, cmd);
> +	return schtasks_remove_task(SCHEDULE_HOURLY) ||
> +	       schtasks_remove_task(SCHEDULE_DAILY) ||
> +	       schtasks_remove_task(SCHEDULE_WEEKLY);
>  }
>  
> -static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
> +static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule)
>  {
> +	const char *cmd = "schtasks";
>  	int result;
>  	struct child_process child = CHILD_PROCESS_INIT;
>  	const char *xml;
> @@ -1736,6 +1838,8 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
>  	char *name = schtasks_task_name(frequency);
>  	struct strbuf tfilename = STRBUF_INIT;
>  
> +	get_schedule_cmd(&cmd, NULL);
> +
>  	strbuf_addf(&tfilename, "%s/schedule_%s_XXXXXX",
>  		    get_git_common_dir(), frequency);
>  	tfile = xmks_tempfile(tfilename.buf);
> @@ -1840,28 +1944,52 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
>  	return result;
>  }
>  
> -static int schtasks_schedule_tasks(const char *cmd)
> +static int schtasks_schedule_tasks(void)
>  {
>  	const char *exec_path = git_exec_path();
>  
> -	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||
> -		schtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||
> -		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
> +	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY) ||
> +	       schtasks_schedule_task(exec_path, SCHEDULE_DAILY) ||
> +	       schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY);
>  }
>  
> -static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)
> +static int schtasks_update_schedule(int run_maintenance, int fd)
>  {
>  	if (run_maintenance)
> -		return schtasks_schedule_tasks(cmd);
> +		return schtasks_schedule_tasks();
>  	else
> -		return schtasks_remove_tasks(cmd);
> +		return schtasks_remove_tasks();
> +}
> +
> +static int is_crontab_available(void)
> +{
> +	const char *cmd = "crontab";
> +	int is_available;
> +	struct child_process child = CHILD_PROCESS_INIT;
> +
> +	if (get_schedule_cmd(&cmd, &is_available))
> +		return is_available;
> +
> +	strvec_split(&child.args, cmd);
> +	strvec_push(&child.args, "-l");
> +	child.no_stdin = 1;
> +	child.no_stdout = 1;
> +	child.no_stderr = 1;
> +	child.silent_exec_failure = 1;
> +
> +	if (start_command(&child))
> +		return 0;
> +	/* Ignore exit code, as an empty crontab will return error. */
> +	finish_command(&child);
> +	return 1;
>  }
>  
>  #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
>  #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
>  
> -static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
> +static int crontab_update_schedule(int run_maintenance, int fd)
>  {
> +	const char *cmd = "crontab";
>  	int result = 0;
>  	int in_old_region = 0;
>  	struct child_process crontab_list = CHILD_PROCESS_INIT;
> @@ -1869,6 +1997,7 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
>  	FILE *cron_list, *cron_in;
>  	struct strbuf line = STRBUF_INIT;
>  
> +	get_schedule_cmd(&cmd, NULL);
>  	strvec_split(&crontab_list.args, cmd);
>  	strvec_push(&crontab_list.args, "-l");
>  	crontab_list.in = -1;
> @@ -1945,66 +2074,160 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
>  	return result;
>  }
>  
> +enum scheduler {
> +	SCHEDULER_INVALID = -1,
> +	SCHEDULER_AUTO,
> +	SCHEDULER_CRON,
> +	SCHEDULER_LAUNCHCTL,
> +	SCHEDULER_SCHTASKS,
> +};
> +
> +static const struct {
> +	const char *name;
> +	int (*is_available)(void);
> +	int (*update_schedule)(int run_maintenance, int fd);
> +} scheduler_fn[] = {
> +	[SCHEDULER_CRON] = {
> +		.name = "crontab",
> +		.is_available = is_crontab_available,
> +		.update_schedule = crontab_update_schedule,
> +	},
> +	[SCHEDULER_LAUNCHCTL] = {
> +		.name = "launchctl",
> +		.is_available = is_launchctl_available,
> +		.update_schedule = launchctl_update_schedule,
> +	},
> +	[SCHEDULER_SCHTASKS] = {
> +		.name = "schtasks",
> +		.is_available = is_schtasks_available,
> +		.update_schedule = schtasks_update_schedule,
> +	},
> +};
> +
> +static enum scheduler parse_scheduler(const char *value)
> +{
> +	if (!value)
> +		return SCHEDULER_INVALID;
> +	else if (!strcasecmp(value, "auto"))
> +		return SCHEDULER_AUTO;
> +	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
> +		return SCHEDULER_CRON;
> +	else if (!strcasecmp(value, "launchctl"))
> +		return SCHEDULER_LAUNCHCTL;
> +	else if (!strcasecmp(value, "schtasks"))
> +		return SCHEDULER_SCHTASKS;
> +	else
> +		return SCHEDULER_INVALID;
> +}
> +
> +static int maintenance_opt_scheduler(const struct option *opt, const char *arg,
> +				     int unset)
> +{
> +	enum scheduler *scheduler = opt->value;
> +
> +	BUG_ON_OPT_NEG(unset);
> +
> +	*scheduler = parse_scheduler(arg);
> +	if (*scheduler == SCHEDULER_INVALID)
> +		return error(_("unrecognized --scheduler argument '%s'"), arg);
> +	return 0;
> +}
> +
> +struct maintenance_start_opts {
> +	enum scheduler scheduler;
> +};
> +
> +static enum scheduler resolve_scheduler(enum scheduler scheduler)
> +{
> +	if (scheduler != SCHEDULER_AUTO)
> +		return scheduler;
> +
>  #if defined(__APPLE__)
> -static const char platform_scheduler[] = "launchctl";
> +	return SCHEDULER_LAUNCHCTL;
> +
>  #elif defined(GIT_WINDOWS_NATIVE)
> -static const char platform_scheduler[] = "schtasks";
> +	return SCHEDULER_SCHTASKS;
> +
>  #else
> -static const char platform_scheduler[] = "crontab";
> +	return SCHEDULER_CRON;
>  #endif
> +}
>  
> -static int update_background_schedule(int enable)
> +static void validate_scheduler(enum scheduler scheduler)
>  {
> -	int result;
> -	const char *scheduler = platform_scheduler;
> -	const char *cmd = scheduler;
> -	char *testing;
> +	if (scheduler == SCHEDULER_INVALID)
> +		BUG("invalid scheduler");
> +	if (scheduler == SCHEDULER_AUTO)
> +		BUG("resolve_scheduler should have been called before");
> +
> +	if (!scheduler_fn[scheduler].is_available())
> +		die(_("%s scheduler is not available"),
> +		    scheduler_fn[scheduler].name);
> +}
> +
> +static int update_background_schedule(const struct maintenance_start_opts *opts,
> +				      int enable)
> +{
> +	unsigned int i;
> +	int result = 0;
>  	struct lock_file lk;
>  	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
>  
> -	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
> -	if (testing) {
> -		char *sep = strchr(testing, ':');
> -		if (!sep)
> -			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
> -		*sep = '\0';
> -		scheduler = testing;
> -		cmd = sep + 1;
> +	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {
> +		free(lock_path);
> +		return error(_("another process is scheduling background maintenance"));
>  	}
>  
> -	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {
> -		result = error(_("another process is scheduling background maintenance"));
> -		goto cleanup;
> +	for (i = 1; i < ARRAY_SIZE(scheduler_fn); i++) {
> +		if (enable && opts->scheduler == i)
> +			continue;
> +		if (!scheduler_fn[i].is_available())
> +			continue;
> +		scheduler_fn[i].update_schedule(0, get_lock_file_fd(&lk));
>  	}
>  
> -	if (!strcmp(scheduler, "launchctl"))
> -		result = launchctl_update_schedule(enable, get_lock_file_fd(&lk), cmd);
> -	else if (!strcmp(scheduler, "schtasks"))
> -		result = schtasks_update_schedule(enable, get_lock_file_fd(&lk), cmd);
> -	else if (!strcmp(scheduler, "crontab"))
> -		result = crontab_update_schedule(enable, get_lock_file_fd(&lk), cmd);
> -	else
> -		die("unknown background scheduler: %s", scheduler);
> +	if (enable)
> +		result = scheduler_fn[opts->scheduler].update_schedule(
> +			1, get_lock_file_fd(&lk));
>  
>  	rollback_lock_file(&lk);
>  
> -cleanup:
>  	free(lock_path);
> -	free(testing);
>  	return result;
>  }
>  
> -static int maintenance_start(void)
> +static const char *const builtin_maintenance_start_usage[] = {
> +	N_("git maintenance start [--scheduler=<scheduler>]"),
> +	NULL
> +};
> +
> +static int maintenance_start(int argc, const char **argv, const char *prefix)
>  {
> +	struct maintenance_start_opts opts = { 0 };
> +	struct option options[] = {
> +		OPT_CALLBACK_F(
> +			0, "scheduler", &opts.scheduler, N_("scheduler"),
> +			N_("scheduler to trigger git maintenance run"),
> +			PARSE_OPT_NONEG, maintenance_opt_scheduler),
> +		OPT_END()
> +	};
> +
> +	argc = parse_options(argc, argv, prefix, options,
> +			     builtin_maintenance_start_usage, 0);
> +	if (argc)
> +		usage_with_options(builtin_maintenance_start_usage, options);
> +
> +	opts.scheduler = resolve_scheduler(opts.scheduler);
> +	validate_scheduler(opts.scheduler);
> +
>  	if (maintenance_register())
>  		warning(_("failed to add repo to global config"));
> -
> -	return update_background_schedule(1);
> +	return update_background_schedule(&opts, 1);
>  }
>  
>  static int maintenance_stop(void)
>  {
> -	return update_background_schedule(0);
> +	return update_background_schedule(NULL, 0);
>  }
>  
>  static const char builtin_maintenance_usage[] =	N_("git maintenance <subcommand> [<options>]");
> @@ -2018,7 +2241,7 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
>  	if (!strcmp(argv[1], "run"))
>  		return maintenance_run(argc - 1, argv + 1, prefix);
>  	if (!strcmp(argv[1], "start"))
> -		return maintenance_start();
> +		return maintenance_start(argc - 1, argv + 1, prefix);
>  	if (!strcmp(argv[1], "stop"))
>  		return maintenance_stop();
>  	if (!strcmp(argv[1], "register"))
> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
> index 58f46c77e6..27bce7992c 100755
> --- a/t/t7900-maintenance.sh
> +++ b/t/t7900-maintenance.sh
> @@ -492,8 +492,21 @@ test_expect_success !MINGW 'register and unregister with regex metacharacters' '
>  		maintenance.repo "$(pwd)/$META"
>  '
>  
> +test_expect_success 'start --scheduler=<scheduler>' '
> +	test_expect_code 129 git maintenance start --scheduler=foo 2>err &&
> +	test_i18ngrep "unrecognized --scheduler argument" err &&
> +
> +	test_expect_code 129 git maintenance start --no-scheduler 2>err &&
> +	test_i18ngrep "unknown option" err &&
> +
> +	test_expect_code 128 \
> +		env GIT_TEST_MAINT_SCHEDULER="launchctl:true,schtasks:true" \
> +		git maintenance start --scheduler=crontab 2>err &&
> +	test_i18ngrep "fatal: crontab scheduler is not available" err
> +'
> +
>  test_expect_success 'start from empty cron table' '
> -	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
> +	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
>  
>  	# start registers the repo
>  	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
> @@ -516,7 +529,7 @@ test_expect_success 'stop from existing schedule' '
>  
>  test_expect_success 'start preserves existing schedule' '
>  	echo "Important information!" >cron.txt &&
> -	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
> +	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
>  	grep "Important information!" cron.txt
>  '
>  
> @@ -545,7 +558,7 @@ test_expect_success 'start and stop macOS maintenance' '
>  	EOF
>  
>  	rm -f args &&
> -	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&
> +	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start --scheduler=launchctl &&
>  
>  	# start registers the repo
>  	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
> @@ -596,7 +609,7 @@ test_expect_success 'start and stop Windows maintenance' '
>  	EOF
>  
>  	rm -f args &&
> -	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start &&
> +	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start --scheduler=schtasks &&
>  
>  	# start registers the repo
>  	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
> @@ -619,6 +632,40 @@ test_expect_success 'start and stop Windows maintenance' '
>  	test_cmp expect args
>  '
>  
> +test_expect_success 'start and stop when several schedulers are available' '
> +	write_script print-args <<-\EOF &&
> +	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
> +	EOF
> +
> +	rm -f args &&
> +	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
> +	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
> +		hourly daily weekly >expect &&
> +	for frequency in hourly daily weekly
> +	do
> +		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
> +		echo "launchctl bootout gui/[UID] $PLIST" >>expect &&
> +		echo "launchctl bootstrap gui/[UID] $PLIST" >>expect || return 1
> +	done &&
> +	test_cmp expect args &&
> +
> +	rm -f args &&
> +	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
> +	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
> +		hourly daily weekly >expect &&
> +	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
> +		hourly daily weekly >>expect &&
> +	test_cmp expect args &&
> +
> +	rm -f args &&
> +	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
> +	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
> +		hourly daily weekly >expect &&
> +	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
> +		hourly daily weekly >>expect &&
> +	test_cmp expect args
> +'
> +
>  test_expect_success 'register preserves existing strategy' '
>  	git config maintenance.strategy none &&
>  	git maintenance register &&
> 

^ permalink raw reply	[flat|nested] 138+ messages in thread

* [PATCH v10 0/3] maintenance: add support for systemd timers on Linux
  2021-08-27 21:02               ` [PATCH v9 " Lénaïc Huard
                                   ` (2 preceding siblings ...)
  2021-08-27 21:02                 ` [PATCH v9 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
@ 2021-09-04 20:54                 ` Lénaïc Huard
  2021-09-04 20:54                   ` [PATCH v10 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
                                     ` (3 more replies)
  3 siblings, 4 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-09-04 20:54 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King, Ramsay Jones,
	Lénaïc Huard

Hello,

Please find hereafter my updated patchset to add support for systemd
timers on Linux for the `git maintenance start` command.

The only changes compared to the previous version are fixes for the
two typos in a comment that Ramsay Jones pointed out [1]

[1] https://lore.kernel.org/git/51246c10-fe0b-b8e5-cdc3-54bdc6c8054e@ramsayjones.plus.com/


The patches are:

* cache.h: Introduce a generic "xdg_config_home_for(…)" function

  This patch introduces a function to compute configuration files
  paths inside $XDG_CONFIG_HOME.

* maintenance: `git maintenance run` learned `--scheduler=<scheduler>`

  This patch adds a new parameter to the `git maintenance run` to let
  the user choose a scheduler.

* maintenance: add support for systemd timers on Linux

  This patch implements the support of systemd timers on top of
  crontab scheduler on Linux systems.

Best wishes,
Lénaïc.

Lénaïc Huard (3):
  cache.h: Introduce a generic "xdg_config_home_for(…)" function
  maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  maintenance: add support for systemd timers on Linux

 Documentation/git-maintenance.txt |  57 +++
 builtin/gc.c                      | 581 ++++++++++++++++++++++++++----
 cache.h                           |   7 +
 path.c                            |  13 +-
 t/t7900-maintenance.sh            | 110 +++++-
 5 files changed, 690 insertions(+), 78 deletions(-)

Diff-intervalle contre v9 :
1:  1639bd151c = 1:  0c0362d4ec cache.h: Introduce a generic "xdg_config_home_for(…)" function
2:  ea5568269c ! 2:  5fb061ca9d maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
    @@ builtin/gc.c: static const char *get_frequency(enum schedule_priority schedule)
     + * list of colon-separated key/value pairs where each pair contains a scheduler
     + * and its corresponding mock.
     + *
    -+ * * If $GET_TEST_MAINT_SCHEDULER is not set, return false and leave the
    ++ * * If $GIT_TEST_MAINT_SCHEDULER is not set, return false and leave the
     + *   arguments unmodified.
     + *
    -+ * * If $GET_TEST_MAINT_SCHEDULER is set, return true.
    ++ * * If $GIT_TEST_MAINT_SCHEDULER is set, return true.
     + *   In this case, the *cmd value is read as input.
     + *
     + *   * if the input value *cmd is the key of one of the comma-separated list
3:  8c25ebd3be = 3:  b0f2f6df0e maintenance: add support for systemd timers on Linux
-- 
2.33.0


^ permalink raw reply	[flat|nested] 138+ messages in thread

* [PATCH v10 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function
  2021-09-04 20:54                 ` [PATCH v10 0/3] " Lénaïc Huard
@ 2021-09-04 20:54                   ` Lénaïc Huard
  2021-09-04 20:54                   ` [PATCH v10 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
                                     ` (2 subsequent siblings)
  3 siblings, 0 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-09-04 20:54 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King, Ramsay Jones,
	Lénaïc Huard

Current implementation of `xdg_config_home(filename)` returns
`$XDG_CONFIG_HOME/git/$filename`, with the `git` subdirectory inserted
between the `XDG_CONFIG_HOME` environment variable and the parameter.

This patch introduces a `xdg_config_home_for(subdir, filename)` function
which is more generic. It only concatenates "$XDG_CONFIG_HOME", or
"$HOME/.config" if the former isn’t defined, with the parameters,
without adding `git` in between.

`xdg_config_home(filename)` is now implemented by calling
`xdg_config_home_for("git", filename)` but this new generic function can
be used to compute the configuration directory of other programs.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 cache.h |  7 +++++++
 path.c  | 13 ++++++++++---
 2 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/cache.h b/cache.h
index d23de69368..72410b85d7 100644
--- a/cache.h
+++ b/cache.h
@@ -1297,6 +1297,13 @@ int is_ntfs_dotmailmap(const char *name);
  */
 int looks_like_command_line_option(const char *str);
 
+/**
+ * Return a newly allocated string with the evaluation of
+ * "$XDG_CONFIG_HOME/$subdir/$filename" if $XDG_CONFIG_HOME is non-empty, otherwise
+ * "$HOME/.config/$subdir/$filename". Return NULL upon error.
+ */
+char *xdg_config_home_for(const char *subdir, const char *filename);
+
 /**
  * Return a newly allocated string with the evaluation of
  * "$XDG_CONFIG_HOME/git/$filename" if $XDG_CONFIG_HOME is non-empty, otherwise
diff --git a/path.c b/path.c
index 0bc788ea40..2c895471d9 100644
--- a/path.c
+++ b/path.c
@@ -1510,21 +1510,28 @@ int looks_like_command_line_option(const char *str)
 	return str && str[0] == '-';
 }
 
-char *xdg_config_home(const char *filename)
+char *xdg_config_home_for(const char *subdir, const char *filename)
 {
 	const char *home, *config_home;
 
+	assert(subdir);
 	assert(filename);
 	config_home = getenv("XDG_CONFIG_HOME");
 	if (config_home && *config_home)
-		return mkpathdup("%s/git/%s", config_home, filename);
+		return mkpathdup("%s/%s/%s", config_home, subdir, filename);
 
 	home = getenv("HOME");
 	if (home)
-		return mkpathdup("%s/.config/git/%s", home, filename);
+		return mkpathdup("%s/.config/%s/%s", home, subdir, filename);
+
 	return NULL;
 }
 
+char *xdg_config_home(const char *filename)
+{
+	return xdg_config_home_for("git", filename);
+}
+
 char *xdg_cache_home(const char *filename)
 {
 	const char *home, *cache_home;
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* [PATCH v10 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>`
  2021-09-04 20:54                 ` [PATCH v10 0/3] " Lénaïc Huard
  2021-09-04 20:54                   ` [PATCH v10 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
@ 2021-09-04 20:54                   ` Lénaïc Huard
  2021-09-04 20:55                   ` [PATCH v10 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
  2021-09-07 16:48                   ` [PATCH v10 0/3] " Derrick Stolee
  3 siblings, 0 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-09-04 20:54 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King, Ramsay Jones,
	Lénaïc Huard

Depending on the system, different schedulers can be used to schedule
the hourly, daily and weekly executions of `git maintenance run`:
* `launchctl` for MacOS,
* `schtasks` for Windows and
* `crontab` for everything else.

`git maintenance run` now has an option to let the end-user explicitly
choose which scheduler he wants to use:
`--scheduler=auto|crontab|launchctl|schtasks`.

When `git maintenance start --scheduler=XXX` is run, it not only
registers `git maintenance run` tasks in the scheduler XXX, it also
removes the `git maintenance run` tasks from all the other schedulers to
ensure we cannot have two schedulers launching concurrent identical
tasks.

The default value is `auto` which chooses a suitable scheduler for the
system.

`git maintenance stop` doesn't have any `--scheduler` parameter because
this command will try to remove the `git maintenance run` tasks from all
the available schedulers.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 Documentation/git-maintenance.txt |   9 +
 builtin/gc.c                      | 365 ++++++++++++++++++++++++------
 t/t7900-maintenance.sh            |  55 ++++-
 3 files changed, 354 insertions(+), 75 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 1e738ad398..576290b5c6 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -179,6 +179,15 @@ OPTIONS
 	`maintenance.<task>.enabled` configured as `true` are considered.
 	See the 'TASKS' section for the list of accepted `<task>` values.
 
+--scheduler=auto|crontab|launchctl|schtasks::
+	When combined with the `start` subcommand, specify the scheduler
+	for running the hourly, daily and weekly executions of
+	`git maintenance run`.
+	Possible values for `<scheduler>` are `auto`, `crontab` (POSIX),
+	`launchctl` (macOS), and `schtasks` (Windows).
+	When `auto` is specified, the appropriate platform-specific
+	scheduler is used. Default is `auto`.
+
 
 TROUBLESHOOTING
 ---------------
diff --git a/builtin/gc.c b/builtin/gc.c
index 6ce5ca4512..4e3bee5c9a 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1529,6 +1529,93 @@ static const char *get_frequency(enum schedule_priority schedule)
 	}
 }
 
+/*
+ * get_schedule_cmd` reads the GIT_TEST_MAINT_SCHEDULER environment variable
+ * to mock the schedulers that `git maintenance start` rely on.
+ *
+ * For test purpose, GIT_TEST_MAINT_SCHEDULER can be set to a comma-separated
+ * list of colon-separated key/value pairs where each pair contains a scheduler
+ * and its corresponding mock.
+ *
+ * * If $GIT_TEST_MAINT_SCHEDULER is not set, return false and leave the
+ *   arguments unmodified.
+ *
+ * * If $GIT_TEST_MAINT_SCHEDULER is set, return true.
+ *   In this case, the *cmd value is read as input.
+ *
+ *   * if the input value *cmd is the key of one of the comma-separated list
+ *     item, then *is_available is set to true and *cmd is modified and becomes
+ *     the mock command.
+ *
+ *   * if the input value *cmd isn’t the key of any of the comma-separated list
+ *     item, then *is_available is set to false.
+ *
+ * Ex.:
+ *   GIT_TEST_MAINT_SCHEDULER not set
+ *     +-------+-------------------------------------------------+
+ *     | Input |                     Output                      |
+ *     | *cmd  | return code |       *cmd        | *is_available |
+ *     +-------+-------------+-------------------+---------------+
+ *     | "foo" |    false    | "foo" (unchanged) |  (unchanged)  |
+ *     +-------+-------------+-------------------+---------------+
+ *
+ *   GIT_TEST_MAINT_SCHEDULER set to “foo:./mock_foo.sh,bar:./mock_bar.sh”
+ *     +-------+-------------------------------------------------+
+ *     | Input |                     Output                      |
+ *     | *cmd  | return code |       *cmd        | *is_available |
+ *     +-------+-------------+-------------------+---------------+
+ *     | "foo" |    true     |  "./mock.foo.sh"  |     true      |
+ *     | "qux" |    true     | "qux" (unchanged) |     false     |
+ *     +-------+-------------+-------------------+---------------+
+ */
+static int get_schedule_cmd(const char **cmd, int *is_available)
+{
+	char *testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
+	struct string_list_item *item;
+	struct string_list list = STRING_LIST_INIT_NODUP;
+
+	if (!testing)
+		return 0;
+
+	if (is_available)
+		*is_available = 0;
+
+	string_list_split_in_place(&list, testing, ',', -1);
+	for_each_string_list_item(item, &list) {
+		struct string_list pair = STRING_LIST_INIT_NODUP;
+
+		if (string_list_split_in_place(&pair, item->string, ':', 2) != 2)
+			continue;
+
+		if (!strcmp(*cmd, pair.items[0].string)) {
+			*cmd = pair.items[1].string;
+			if (is_available)
+				*is_available = 1;
+			string_list_clear(&list, 0);
+			UNLEAK(testing);
+			return 1;
+		}
+	}
+
+	string_list_clear(&list, 0);
+	free(testing);
+	return 1;
+}
+
+static int is_launchctl_available(void)
+{
+	const char *cmd = "launchctl";
+	int is_available;
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+#ifdef __APPLE__
+	return 1;
+#else
+	return 0;
+#endif
+}
+
 static char *launchctl_service_name(const char *frequency)
 {
 	struct strbuf label = STRBUF_INIT;
@@ -1555,19 +1642,17 @@ static char *launchctl_get_uid(void)
 	return xstrfmt("gui/%d", getuid());
 }
 
-static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
+static int launchctl_boot_plist(int enable, const char *filename)
 {
+	const char *cmd = "launchctl";
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
 	char *uid = launchctl_get_uid();
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&child.args, cmd);
-	if (enable)
-		strvec_push(&child.args, "bootstrap");
-	else
-		strvec_push(&child.args, "bootout");
-	strvec_push(&child.args, uid);
-	strvec_push(&child.args, filename);
+	strvec_pushl(&child.args, enable ? "bootstrap" : "bootout", uid,
+		     filename, NULL);
 
 	child.no_stderr = 1;
 	child.no_stdout = 1;
@@ -1581,26 +1666,26 @@ static int launchctl_boot_plist(int enable, const char *filename, const char *cm
 	return result;
 }
 
-static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)
+static int launchctl_remove_plist(enum schedule_priority schedule)
 {
 	const char *frequency = get_frequency(schedule);
 	char *name = launchctl_service_name(frequency);
 	char *filename = launchctl_service_filename(name);
-	int result = launchctl_boot_plist(0, filename, cmd);
+	int result = launchctl_boot_plist(0, filename);
 	unlink(filename);
 	free(filename);
 	free(name);
 	return result;
 }
 
-static int launchctl_remove_plists(const char *cmd)
+static int launchctl_remove_plists(void)
 {
-	return launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||
-		launchctl_remove_plist(SCHEDULE_DAILY, cmd) ||
-		launchctl_remove_plist(SCHEDULE_WEEKLY, cmd);
+	return launchctl_remove_plist(SCHEDULE_HOURLY) ||
+	       launchctl_remove_plist(SCHEDULE_DAILY) ||
+	       launchctl_remove_plist(SCHEDULE_WEEKLY);
 }
 
-static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule)
 {
 	FILE *plist;
 	int i;
@@ -1669,8 +1754,8 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 	fclose(plist);
 
 	/* bootout might fail if not already running, so ignore */
-	launchctl_boot_plist(0, filename, cmd);
-	if (launchctl_boot_plist(1, filename, cmd))
+	launchctl_boot_plist(0, filename);
+	if (launchctl_boot_plist(1, filename))
 		die(_("failed to bootstrap service %s"), filename);
 
 	free(filename);
@@ -1678,21 +1763,35 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 	return 0;
 }
 
-static int launchctl_add_plists(const char *cmd)
+static int launchctl_add_plists(void)
 {
 	const char *exec_path = git_exec_path();
 
-	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||
-		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||
-		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
+	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY) ||
+	       launchctl_schedule_plist(exec_path, SCHEDULE_DAILY) ||
+	       launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY);
 }
 
-static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int launchctl_update_schedule(int run_maintenance, int fd)
 {
 	if (run_maintenance)
-		return launchctl_add_plists(cmd);
+		return launchctl_add_plists();
 	else
-		return launchctl_remove_plists(cmd);
+		return launchctl_remove_plists();
+}
+
+static int is_schtasks_available(void)
+{
+	const char *cmd = "schtasks";
+	int is_available;
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+#ifdef GIT_WINDOWS_NATIVE
+	return 1;
+#else
+	return 0;
+#endif
 }
 
 static char *schtasks_task_name(const char *frequency)
@@ -1702,13 +1801,15 @@ static char *schtasks_task_name(const char *frequency)
 	return strbuf_detach(&label, NULL);
 }
 
-static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)
+static int schtasks_remove_task(enum schedule_priority schedule)
 {
+	const char *cmd = "schtasks";
 	int result;
 	struct strvec args = STRVEC_INIT;
 	const char *frequency = get_frequency(schedule);
 	char *name = schtasks_task_name(frequency);
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&args, cmd);
 	strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
 
@@ -1719,15 +1820,16 @@ static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd
 	return result;
 }
 
-static int schtasks_remove_tasks(const char *cmd)
+static int schtasks_remove_tasks(void)
 {
-	return schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||
-		schtasks_remove_task(SCHEDULE_DAILY, cmd) ||
-		schtasks_remove_task(SCHEDULE_WEEKLY, cmd);
+	return schtasks_remove_task(SCHEDULE_HOURLY) ||
+	       schtasks_remove_task(SCHEDULE_DAILY) ||
+	       schtasks_remove_task(SCHEDULE_WEEKLY);
 }
 
-static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule)
 {
+	const char *cmd = "schtasks";
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
 	const char *xml;
@@ -1736,6 +1838,8 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
 	char *name = schtasks_task_name(frequency);
 	struct strbuf tfilename = STRBUF_INIT;
 
+	get_schedule_cmd(&cmd, NULL);
+
 	strbuf_addf(&tfilename, "%s/schedule_%s_XXXXXX",
 		    get_git_common_dir(), frequency);
 	tfile = xmks_tempfile(tfilename.buf);
@@ -1840,28 +1944,52 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
 	return result;
 }
 
-static int schtasks_schedule_tasks(const char *cmd)
+static int schtasks_schedule_tasks(void)
 {
 	const char *exec_path = git_exec_path();
 
-	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||
-		schtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||
-		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
+	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY) ||
+	       schtasks_schedule_task(exec_path, SCHEDULE_DAILY) ||
+	       schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY);
 }
 
-static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int schtasks_update_schedule(int run_maintenance, int fd)
 {
 	if (run_maintenance)
-		return schtasks_schedule_tasks(cmd);
+		return schtasks_schedule_tasks();
 	else
-		return schtasks_remove_tasks(cmd);
+		return schtasks_remove_tasks();
+}
+
+static int is_crontab_available(void)
+{
+	const char *cmd = "crontab";
+	int is_available;
+	struct child_process child = CHILD_PROCESS_INIT;
+
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+	strvec_split(&child.args, cmd);
+	strvec_push(&child.args, "-l");
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child))
+		return 0;
+	/* Ignore exit code, as an empty crontab will return error. */
+	finish_command(&child);
+	return 1;
 }
 
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
-static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
+static int crontab_update_schedule(int run_maintenance, int fd)
 {
+	const char *cmd = "crontab";
 	int result = 0;
 	int in_old_region = 0;
 	struct child_process crontab_list = CHILD_PROCESS_INIT;
@@ -1869,6 +1997,7 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 	FILE *cron_list, *cron_in;
 	struct strbuf line = STRBUF_INIT;
 
+	get_schedule_cmd(&cmd, NULL);
 	strvec_split(&crontab_list.args, cmd);
 	strvec_push(&crontab_list.args, "-l");
 	crontab_list.in = -1;
@@ -1945,66 +2074,160 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 	return result;
 }
 
+enum scheduler {
+	SCHEDULER_INVALID = -1,
+	SCHEDULER_AUTO,
+	SCHEDULER_CRON,
+	SCHEDULER_LAUNCHCTL,
+	SCHEDULER_SCHTASKS,
+};
+
+static const struct {
+	const char *name;
+	int (*is_available)(void);
+	int (*update_schedule)(int run_maintenance, int fd);
+} scheduler_fn[] = {
+	[SCHEDULER_CRON] = {
+		.name = "crontab",
+		.is_available = is_crontab_available,
+		.update_schedule = crontab_update_schedule,
+	},
+	[SCHEDULER_LAUNCHCTL] = {
+		.name = "launchctl",
+		.is_available = is_launchctl_available,
+		.update_schedule = launchctl_update_schedule,
+	},
+	[SCHEDULER_SCHTASKS] = {
+		.name = "schtasks",
+		.is_available = is_schtasks_available,
+		.update_schedule = schtasks_update_schedule,
+	},
+};
+
+static enum scheduler parse_scheduler(const char *value)
+{
+	if (!value)
+		return SCHEDULER_INVALID;
+	else if (!strcasecmp(value, "auto"))
+		return SCHEDULER_AUTO;
+	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
+		return SCHEDULER_CRON;
+	else if (!strcasecmp(value, "launchctl"))
+		return SCHEDULER_LAUNCHCTL;
+	else if (!strcasecmp(value, "schtasks"))
+		return SCHEDULER_SCHTASKS;
+	else
+		return SCHEDULER_INVALID;
+}
+
+static int maintenance_opt_scheduler(const struct option *opt, const char *arg,
+				     int unset)
+{
+	enum scheduler *scheduler = opt->value;
+
+	BUG_ON_OPT_NEG(unset);
+
+	*scheduler = parse_scheduler(arg);
+	if (*scheduler == SCHEDULER_INVALID)
+		return error(_("unrecognized --scheduler argument '%s'"), arg);
+	return 0;
+}
+
+struct maintenance_start_opts {
+	enum scheduler scheduler;
+};
+
+static enum scheduler resolve_scheduler(enum scheduler scheduler)
+{
+	if (scheduler != SCHEDULER_AUTO)
+		return scheduler;
+
 #if defined(__APPLE__)
-static const char platform_scheduler[] = "launchctl";
+	return SCHEDULER_LAUNCHCTL;
+
 #elif defined(GIT_WINDOWS_NATIVE)
-static const char platform_scheduler[] = "schtasks";
+	return SCHEDULER_SCHTASKS;
+
 #else
-static const char platform_scheduler[] = "crontab";
+	return SCHEDULER_CRON;
 #endif
+}
 
-static int update_background_schedule(int enable)
+static void validate_scheduler(enum scheduler scheduler)
 {
-	int result;
-	const char *scheduler = platform_scheduler;
-	const char *cmd = scheduler;
-	char *testing;
+	if (scheduler == SCHEDULER_INVALID)
+		BUG("invalid scheduler");
+	if (scheduler == SCHEDULER_AUTO)
+		BUG("resolve_scheduler should have been called before");
+
+	if (!scheduler_fn[scheduler].is_available())
+		die(_("%s scheduler is not available"),
+		    scheduler_fn[scheduler].name);
+}
+
+static int update_background_schedule(const struct maintenance_start_opts *opts,
+				      int enable)
+{
+	unsigned int i;
+	int result = 0;
 	struct lock_file lk;
 	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
 
-	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
-	if (testing) {
-		char *sep = strchr(testing, ':');
-		if (!sep)
-			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
-		*sep = '\0';
-		scheduler = testing;
-		cmd = sep + 1;
+	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {
+		free(lock_path);
+		return error(_("another process is scheduling background maintenance"));
 	}
 
-	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {
-		result = error(_("another process is scheduling background maintenance"));
-		goto cleanup;
+	for (i = 1; i < ARRAY_SIZE(scheduler_fn); i++) {
+		if (enable && opts->scheduler == i)
+			continue;
+		if (!scheduler_fn[i].is_available())
+			continue;
+		scheduler_fn[i].update_schedule(0, get_lock_file_fd(&lk));
 	}
 
-	if (!strcmp(scheduler, "launchctl"))
-		result = launchctl_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else if (!strcmp(scheduler, "schtasks"))
-		result = schtasks_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else if (!strcmp(scheduler, "crontab"))
-		result = crontab_update_schedule(enable, get_lock_file_fd(&lk), cmd);
-	else
-		die("unknown background scheduler: %s", scheduler);
+	if (enable)
+		result = scheduler_fn[opts->scheduler].update_schedule(
+			1, get_lock_file_fd(&lk));
 
 	rollback_lock_file(&lk);
 
-cleanup:
 	free(lock_path);
-	free(testing);
 	return result;
 }
 
-static int maintenance_start(void)
+static const char *const builtin_maintenance_start_usage[] = {
+	N_("git maintenance start [--scheduler=<scheduler>]"),
+	NULL
+};
+
+static int maintenance_start(int argc, const char **argv, const char *prefix)
 {
+	struct maintenance_start_opts opts = { 0 };
+	struct option options[] = {
+		OPT_CALLBACK_F(
+			0, "scheduler", &opts.scheduler, N_("scheduler"),
+			N_("scheduler to trigger git maintenance run"),
+			PARSE_OPT_NONEG, maintenance_opt_scheduler),
+		OPT_END()
+	};
+
+	argc = parse_options(argc, argv, prefix, options,
+			     builtin_maintenance_start_usage, 0);
+	if (argc)
+		usage_with_options(builtin_maintenance_start_usage, options);
+
+	opts.scheduler = resolve_scheduler(opts.scheduler);
+	validate_scheduler(opts.scheduler);
+
 	if (maintenance_register())
 		warning(_("failed to add repo to global config"));
-
-	return update_background_schedule(1);
+	return update_background_schedule(&opts, 1);
 }
 
 static int maintenance_stop(void)
 {
-	return update_background_schedule(0);
+	return update_background_schedule(NULL, 0);
 }
 
 static const char builtin_maintenance_usage[] =	N_("git maintenance <subcommand> [<options>]");
@@ -2018,7 +2241,7 @@ int cmd_maintenance(int argc, const char **argv, const char *prefix)
 	if (!strcmp(argv[1], "run"))
 		return maintenance_run(argc - 1, argv + 1, prefix);
 	if (!strcmp(argv[1], "start"))
-		return maintenance_start();
+		return maintenance_start(argc - 1, argv + 1, prefix);
 	if (!strcmp(argv[1], "stop"))
 		return maintenance_stop();
 	if (!strcmp(argv[1], "register"))
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 58f46c77e6..27bce7992c 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -492,8 +492,21 @@ test_expect_success !MINGW 'register and unregister with regex metacharacters' '
 		maintenance.repo "$(pwd)/$META"
 '
 
+test_expect_success 'start --scheduler=<scheduler>' '
+	test_expect_code 129 git maintenance start --scheduler=foo 2>err &&
+	test_i18ngrep "unrecognized --scheduler argument" err &&
+
+	test_expect_code 129 git maintenance start --no-scheduler 2>err &&
+	test_i18ngrep "unknown option" err &&
+
+	test_expect_code 128 \
+		env GIT_TEST_MAINT_SCHEDULER="launchctl:true,schtasks:true" \
+		git maintenance start --scheduler=crontab 2>err &&
+	test_i18ngrep "fatal: crontab scheduler is not available" err
+'
+
 test_expect_success 'start from empty cron table' '
-	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -516,7 +529,7 @@ test_expect_success 'stop from existing schedule' '
 
 test_expect_success 'start preserves existing schedule' '
 	echo "Important information!" >cron.txt &&
-	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start --scheduler=crontab &&
 	grep "Important information!" cron.txt
 '
 
@@ -545,7 +558,7 @@ test_expect_success 'start and stop macOS maintenance' '
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start --scheduler=launchctl &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -596,7 +609,7 @@ test_expect_success 'start and stop Windows maintenance' '
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start --scheduler=schtasks &&
 
 	# start registers the repo
 	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
@@ -619,6 +632,40 @@ test_expect_success 'start and stop Windows maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success 'start and stop when several schedulers are available' '
+	write_script print-args <<-\EOF &&
+	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
+	EOF
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >expect &&
+	for frequency in hourly daily weekly
+	do
+		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
+		echo "launchctl bootout gui/[UID] $PLIST" >>expect &&
+		echo "launchctl bootstrap gui/[UID] $PLIST" >>expect || return 1
+	done &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* [PATCH v10 3/3] maintenance: add support for systemd timers on Linux
  2021-09-04 20:54                 ` [PATCH v10 0/3] " Lénaïc Huard
  2021-09-04 20:54                   ` [PATCH v10 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
  2021-09-04 20:54                   ` [PATCH v10 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
@ 2021-09-04 20:55                   ` Lénaïc Huard
  2021-09-07 16:48                   ` [PATCH v10 0/3] " Derrick Stolee
  3 siblings, 0 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-09-04 20:55 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Derrick Stolee, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King, Ramsay Jones,
	Lénaïc Huard

The existing mechanism for scheduling background maintenance is done
through cron. On Linux systems managed by systemd, systemd provides an
alternative to schedule recurring tasks: systemd timers.

The main motivations to implement systemd timers in addition to cron
are:
* cron is optional and Linux systems running systemd might not have it
  installed.
* The execution of `crontab -l` can tell us if cron is installed but not
  if the daemon is actually running.
* With systemd, each service is run in its own cgroup and its logs are
  tagged by the service inside journald. With cron, all scheduled tasks
  are running in the cron daemon cgroup and all the logs of the
  user-scheduled tasks are pretended to belong to the system cron
  service.
  Concretely, a user that doesn’t have access to the system logs won’t
  have access to the log of their own tasks scheduled by cron whereas
  they will have access to the log of their own tasks scheduled by
  systemd timer.
  Although `cron` attempts to send email, that email may go unseen by
  the user because these days, local mailboxes are not heavily used
  anymore.

In order to schedule git maintenance, we need two unit template files:
* ~/.config/systemd/user/git-maintenance@.service
  to define the command to be started by systemd and
* ~/.config/systemd/user/git-maintenance@.timer
  to define the schedule at which the command should be run.

Those units are templates that are parameterized by the frequency.

Based on those templates, 3 timers are started:
* git-maintenance@hourly.timer
* git-maintenance@daily.timer
* git-maintenance@weekly.timer

The command launched by those three timers are the same as with the
other scheduling methods:

/path/to/git for-each-repo --exec-path=/path/to
--config=maintenance.repo maintenance run --schedule=%i

with the full path for git to ensure that the version of git launched
for the scheduled maintenance is the same as the one used to run
`maintenance start`.

The timer unit contains `Persistent=true` so that, if the computer is
powered down when a maintenance task should run, the task will be run
when the computer is back powered on.

Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr>
---
 Documentation/git-maintenance.txt |  58 +++++++-
 builtin/gc.c                      | 216 ++++++++++++++++++++++++++++++
 t/t7900-maintenance.sh            |  67 ++++++++-
 3 files changed, 330 insertions(+), 11 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 576290b5c6..e2cfb68ab5 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -179,14 +179,16 @@ OPTIONS
 	`maintenance.<task>.enabled` configured as `true` are considered.
 	See the 'TASKS' section for the list of accepted `<task>` values.
 
---scheduler=auto|crontab|launchctl|schtasks::
+--scheduler=auto|crontab|systemd-timer|launchctl|schtasks::
 	When combined with the `start` subcommand, specify the scheduler
 	for running the hourly, daily and weekly executions of
 	`git maintenance run`.
-	Possible values for `<scheduler>` are `auto`, `crontab` (POSIX),
-	`launchctl` (macOS), and `schtasks` (Windows).
-	When `auto` is specified, the appropriate platform-specific
-	scheduler is used. Default is `auto`.
+	Possible values for `<scheduler>` are `auto`, `crontab`
+	(POSIX), `systemd-timer` (Linux), `launchctl` (macOS), and
+	`schtasks` (Windows). When `auto` is specified, the
+	appropriate platform-specific scheduler is used; on Linux,
+	`systemd-timer` is used if available, otherwise
+	`crontab`. Default is `auto`.
 
 
 TROUBLESHOOTING
@@ -286,6 +288,52 @@ schedule to ensure you are executing the correct binaries in your
 schedule.
 
 
+BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMS
+-----------------------------------------------
+
+While Linux supports `cron`, depending on the distribution, `cron` may
+be an optional package not necessarily installed. On modern Linux
+distributions, systemd timers are superseding it.
+
+If user systemd timers are available, they will be used as a replacement
+of `cron`.
+
+In this case, `git maintenance start` will create user systemd timer units
+and start the timers. The current list of user-scheduled tasks can be found
+by running `systemctl --user list-timers`. The timers written by `git
+maintenance start` are similar to this:
+
+-----------------------------------------------------------------------
+$ systemctl --user list-timers
+NEXT                         LEFT          LAST                         PASSED     UNIT                         ACTIVATES
+Thu 2021-04-29 19:00:00 CEST 42min left    Thu 2021-04-29 18:00:11 CEST 17min ago  git-maintenance@hourly.timer git-maintenance@hourly.service
+Fri 2021-04-30 00:00:00 CEST 5h 42min left Thu 2021-04-29 00:00:11 CEST 18h ago    git-maintenance@daily.timer  git-maintenance@daily.service
+Mon 2021-05-03 00:00:00 CEST 3 days left   Mon 2021-04-26 00:00:11 CEST 3 days ago git-maintenance@weekly.timer git-maintenance@weekly.service
+-----------------------------------------------------------------------
+
+One timer is registered for each `--schedule=<frequency>` option.
+
+The definition of the systemd units can be inspected in the following files:
+
+-----------------------------------------------------------------------
+~/.config/systemd/user/git-maintenance@.timer
+~/.config/systemd/user/git-maintenance@.service
+~/.config/systemd/user/timers.target.wants/git-maintenance@hourly.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@daily.timer
+~/.config/systemd/user/timers.target.wants/git-maintenance@weekly.timer
+-----------------------------------------------------------------------
+
+`git maintenance start` will overwrite these files and start the timer
+again with `systemctl --user`, so any customization should be done by
+creating a drop-in file, i.e. a `.conf` suffixed file in the
+`~/.config/systemd/user/git-maintenance@.service.d` directory.
+
+`git maintenance stop` will stop the user systemd timers and delete
+the above mentioned files.
+
+For more details, see `systemd.timer(5)`.
+
+
 BACKGROUND MAINTENANCE ON MACOS SYSTEMS
 ---------------------------------------
 
diff --git a/builtin/gc.c b/builtin/gc.c
index 4e3bee5c9a..59b714f610 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -2074,10 +2074,210 @@ static int crontab_update_schedule(int run_maintenance, int fd)
 	return result;
 }
 
+static int real_is_systemd_timer_available(void)
+{
+	struct child_process child = CHILD_PROCESS_INIT;
+
+	strvec_pushl(&child.args, "systemctl", "--user", "list-timers", NULL);
+	child.no_stdin = 1;
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+	child.silent_exec_failure = 1;
+
+	if (start_command(&child))
+		return 0;
+	if (finish_command(&child))
+		return 0;
+	return 1;
+}
+
+static int is_systemd_timer_available(void)
+{
+	const char *cmd = "systemctl";
+	int is_available;
+
+	if (get_schedule_cmd(&cmd, &is_available))
+		return is_available;
+
+	return real_is_systemd_timer_available();
+}
+
+static char *xdg_config_home_systemd(const char *filename)
+{
+	return xdg_config_home_for("systemd/user", filename);
+}
+
+static int systemd_timer_enable_unit(int enable,
+				     enum schedule_priority schedule)
+{
+	const char *cmd = "systemctl";
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *frequency = get_frequency(schedule);
+
+	/*
+	 * Disabling the systemd unit while it is already disabled makes
+	 * systemctl print an error.
+	 * Let's ignore it since it means we already are in the expected state:
+	 * the unit is disabled.
+	 *
+	 * On the other hand, enabling a systemd unit which is already enabled
+	 * produces no error.
+	 */
+	if (!enable)
+		child.no_stderr = 1;
+
+	get_schedule_cmd(&cmd, NULL);
+	strvec_split(&child.args, cmd);
+	strvec_pushl(&child.args, "--user", enable ? "enable" : "disable",
+		     "--now", NULL);
+	strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency);
+
+	if (start_command(&child))
+		return error(_("failed to start systemctl"));
+	if (finish_command(&child))
+		/*
+		 * Disabling an already disabled systemd unit makes
+		 * systemctl fail.
+		 * Let's ignore this failure.
+		 *
+		 * Enabling an enabled systemd unit doesn't fail.
+		 */
+		if (enable)
+			return error(_("failed to run systemctl"));
+	return 0;
+}
+
+static int systemd_timer_delete_unit_templates(void)
+{
+	int ret = 0;
+	char *filename = xdg_config_home_systemd("git-maintenance@.timer");
+	if (unlink(filename) && !is_missing_file_error(errno))
+		ret = error_errno(_("failed to delete '%s'"), filename);
+	FREE_AND_NULL(filename);
+
+	filename = xdg_config_home_systemd("git-maintenance@.service");
+	if (unlink(filename) && !is_missing_file_error(errno))
+		ret = error_errno(_("failed to delete '%s'"), filename);
+
+	free(filename);
+	return ret;
+}
+
+static int systemd_timer_delete_units(void)
+{
+	return systemd_timer_enable_unit(0, SCHEDULE_HOURLY) ||
+	       systemd_timer_enable_unit(0, SCHEDULE_DAILY) ||
+	       systemd_timer_enable_unit(0, SCHEDULE_WEEKLY) ||
+	       systemd_timer_delete_unit_templates();
+}
+
+static int systemd_timer_write_unit_templates(const char *exec_path)
+{
+	char *filename;
+	FILE *file;
+	const char *unit;
+
+	filename = xdg_config_home_systemd("git-maintenance@.timer");
+	if (safe_create_leading_directories(filename)) {
+		error(_("failed to create directories for '%s'"), filename);
+		goto error;
+	}
+	file = fopen_or_warn(filename, "w");
+	if (file == NULL)
+		goto error;
+
+	unit = "# This file was created and is maintained by Git.\n"
+	       "# Any edits made in this file might be replaced in the future\n"
+	       "# by a Git command.\n"
+	       "\n"
+	       "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Timer]\n"
+	       "OnCalendar=%i\n"
+	       "Persistent=true\n"
+	       "\n"
+	       "[Install]\n"
+	       "WantedBy=timers.target\n";
+	if (fputs(unit, file) == EOF) {
+		error(_("failed to write to '%s'"), filename);
+		fclose(file);
+		goto error;
+	}
+	if (fclose(file) == EOF) {
+		error_errno(_("failed to flush '%s'"), filename);
+		goto error;
+	}
+	free(filename);
+
+	filename = xdg_config_home_systemd("git-maintenance@.service");
+	file = fopen_or_warn(filename, "w");
+	if (file == NULL)
+		goto error;
+
+	unit = "# This file was created and is maintained by Git.\n"
+	       "# Any edits made in this file might be replaced in the future\n"
+	       "# by a Git command.\n"
+	       "\n"
+	       "[Unit]\n"
+	       "Description=Optimize Git repositories data\n"
+	       "\n"
+	       "[Service]\n"
+	       "Type=oneshot\n"
+	       "ExecStart=\"%s/git\" --exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
+	       "LockPersonality=yes\n"
+	       "MemoryDenyWriteExecute=yes\n"
+	       "NoNewPrivileges=yes\n"
+	       "RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\n"
+	       "RestrictNamespaces=yes\n"
+	       "RestrictRealtime=yes\n"
+	       "RestrictSUIDSGID=yes\n"
+	       "SystemCallArchitectures=native\n"
+	       "SystemCallFilter=@system-service\n";
+	if (fprintf(file, unit, exec_path, exec_path) < 0) {
+		error(_("failed to write to '%s'"), filename);
+		fclose(file);
+		goto error;
+	}
+	if (fclose(file) == EOF) {
+		error_errno(_("failed to flush '%s'"), filename);
+		goto error;
+	}
+	free(filename);
+	return 0;
+
+error:
+	free(filename);
+	systemd_timer_delete_unit_templates();
+	return -1;
+}
+
+static int systemd_timer_setup_units(void)
+{
+	const char *exec_path = git_exec_path();
+
+	int ret = systemd_timer_write_unit_templates(exec_path) ||
+		  systemd_timer_enable_unit(1, SCHEDULE_HOURLY) ||
+		  systemd_timer_enable_unit(1, SCHEDULE_DAILY) ||
+		  systemd_timer_enable_unit(1, SCHEDULE_WEEKLY);
+	if (ret)
+		systemd_timer_delete_units();
+	return ret;
+}
+
+static int systemd_timer_update_schedule(int run_maintenance, int fd)
+{
+	if (run_maintenance)
+		return systemd_timer_setup_units();
+	else
+		return systemd_timer_delete_units();
+}
+
 enum scheduler {
 	SCHEDULER_INVALID = -1,
 	SCHEDULER_AUTO,
 	SCHEDULER_CRON,
+	SCHEDULER_SYSTEMD,
 	SCHEDULER_LAUNCHCTL,
 	SCHEDULER_SCHTASKS,
 };
@@ -2092,6 +2292,11 @@ static const struct {
 		.is_available = is_crontab_available,
 		.update_schedule = crontab_update_schedule,
 	},
+	[SCHEDULER_SYSTEMD] = {
+		.name = "systemctl",
+		.is_available = is_systemd_timer_available,
+		.update_schedule = systemd_timer_update_schedule,
+	},
 	[SCHEDULER_LAUNCHCTL] = {
 		.name = "launchctl",
 		.is_available = is_launchctl_available,
@@ -2112,6 +2317,9 @@ static enum scheduler parse_scheduler(const char *value)
 		return SCHEDULER_AUTO;
 	else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
 		return SCHEDULER_CRON;
+	else if (!strcasecmp(value, "systemd") ||
+		 !strcasecmp(value, "systemd-timer"))
+		return SCHEDULER_SYSTEMD;
 	else if (!strcasecmp(value, "launchctl"))
 		return SCHEDULER_LAUNCHCTL;
 	else if (!strcasecmp(value, "schtasks"))
@@ -2148,6 +2356,14 @@ static enum scheduler resolve_scheduler(enum scheduler scheduler)
 #elif defined(GIT_WINDOWS_NATIVE)
 	return SCHEDULER_SCHTASKS;
 
+#elif defined(__linux__)
+	if (is_systemd_timer_available())
+		return SCHEDULER_SYSTEMD;
+	else if (is_crontab_available())
+		return SCHEDULER_CRON;
+	else
+		die(_("neither systemd timers nor crontab are available"));
+
 #else
 	return SCHEDULER_CRON;
 #endif
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 27bce7992c..265f7793f5 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -20,6 +20,18 @@ test_xmllint () {
 	fi
 }
 
+test_lazy_prereq SYSTEMD_ANALYZE '
+	systemd-analyze --help >out &&
+	grep verify out
+'
+
+test_systemd_analyze_verify () {
+	if test_have_prereq SYSTEMD_ANALYZE
+	then
+		systemd-analyze verify "$@"
+	fi
+}
+
 test_expect_success 'help text' '
 	test_expect_code 129 git maintenance -h 2>err &&
 	test_i18ngrep "usage: git maintenance <subcommand>" err &&
@@ -632,15 +644,56 @@ test_expect_success 'start and stop Windows maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success 'start and stop Linux/systemd maintenance' '
+	write_script print-args <<-\EOF &&
+	printf "%s\n" "$*" >>args
+	EOF
+
+	XDG_CONFIG_HOME="$PWD" &&
+	export XDG_CONFIG_HOME &&
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance start --scheduler=systemd-timer &&
+
+	# start registers the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_systemd_analyze_verify "systemd/user/git-maintenance@.service" &&
+
+	printf -- "--user enable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
+
+	test_path_is_missing "systemd/user/git-maintenance@.timer" &&
+	test_path_is_missing "systemd/user/git-maintenance@.service" &&
+
+	printf -- "--user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'start and stop when several schedulers are available' '
 	write_script print-args <<-\EOF &&
 	printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
 	EOF
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
-	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=systemd-timer &&
+	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
 		hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
+	printf -- "systemctl --user enable --now git-maintenance@%s.timer\n" hourly daily weekly >>expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
+	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >>expect &&
 	for frequency in hourly daily weekly
 	do
 		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
@@ -650,17 +703,19 @@ test_expect_success 'start and stop when several schedulers are available' '
 	test_cmp expect args &&
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
 	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
-		hourly daily weekly >expect &&
+		hourly daily weekly >>expect &&
 	printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
 		hourly daily weekly >>expect &&
 	test_cmp expect args &&
 
 	rm -f args &&
-	GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
+	printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
 	printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
-		hourly daily weekly >expect &&
+		hourly daily weekly >>expect &&
 	printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
 		hourly daily weekly >>expect &&
 	test_cmp expect args
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 138+ messages in thread

* Re: [PATCH v10 0/3] maintenance: add support for systemd timers on Linux
  2021-09-04 20:54                 ` [PATCH v10 0/3] " Lénaïc Huard
                                     ` (2 preceding siblings ...)
  2021-09-04 20:55                   ` [PATCH v10 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
@ 2021-09-07 16:48                   ` Derrick Stolee
  2021-09-08 11:44                     ` Derrick Stolee
  3 siblings, 1 reply; 138+ messages in thread
From: Derrick Stolee @ 2021-09-07 16:48 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King, Ramsay Jones

On 9/4/2021 4:54 PM, Lénaïc Huard wrote:
> Hello,
> 
> Please find hereafter my updated patchset to add support for systemd
> timers on Linux for the `git maintenance start` command.
> 
> The only changes compared to the previous version are fixes for the
> two typos in a comment that Ramsay Jones pointed out [1]
> 
> [1] https://lore.kernel.org/git/51246c10-fe0b-b8e5-cdc3-54bdc6c8054e@ramsayjones.plus.com/

The changes in the most recent two versions look good to me.

Thank you for this contribution!
-Stolee

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v10 0/3] maintenance: add support for systemd timers on Linux
  2021-09-07 16:48                   ` [PATCH v10 0/3] " Derrick Stolee
@ 2021-09-08 11:44                     ` Derrick Stolee
  2021-09-09  5:52                       ` Lénaïc Huard
  0 siblings, 1 reply; 138+ messages in thread
From: Derrick Stolee @ 2021-09-08 11:44 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King, Ramsay Jones

On 9/7/2021 12:48 PM, Derrick Stolee wrote:
> On 9/4/2021 4:54 PM, Lénaïc Huard wrote:
>> Hello,
>>
>> Please find hereafter my updated patchset to add support for systemd
>> timers on Linux for the `git maintenance start` command.
>>
>> The only changes compared to the previous version are fixes for the
>> two typos in a comment that Ramsay Jones pointed out [1]
>>
>> [1] https://lore.kernel.org/git/51246c10-fe0b-b8e5-cdc3-54bdc6c8054e@ramsayjones.plus.com/
> 
> The changes in the most recent two versions look good to me.

I recently tested the 'seen' branch for an unrelated reason, but found
that the t7900-maintenance.sh test failed for me. It was during test 34,
'start and stop Linux/systemd maintenance' with the following issue:

  + systemd-analyze verify systemd/user/git-maintenance@.service
  Failed to create /user.slice/user-1000.slice/session-44.scope/init.scope control group: Permission denied
  Failed to initialize manager: Permission denied

Now, this test has the prereq SYSTEMD_ANALYZE, but for some reason this
later command fails for permission issues. I'm running Ubuntu, if that
helps.

Thanks,
-Stolee

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v10 0/3] maintenance: add support for systemd timers on Linux
  2021-09-08 11:44                     ` Derrick Stolee
@ 2021-09-09  5:52                       ` Lénaïc Huard
  2021-09-09 19:55                         ` Derrick Stolee
  0 siblings, 1 reply; 138+ messages in thread
From: Lénaïc Huard @ 2021-09-09  5:52 UTC (permalink / raw)
  To: git, Derrick Stolee
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King, Ramsay Jones

Le mercredi 8 septembre 2021, 13:44:26 CEST Derrick Stolee a écrit :
> On 9/7/2021 12:48 PM, Derrick Stolee wrote:
> > On 9/4/2021 4:54 PM, Lénaïc Huard wrote:
> >> Hello,
> >> 
> >> Please find hereafter my updated patchset to add support for systemd
> >> timers on Linux for the `git maintenance start` command.
> >> 
> >> The only changes compared to the previous version are fixes for the
> >> two typos in a comment that Ramsay Jones pointed out [1]
> >> 
> >> [1]
> >> https://lore.kernel.org/git/51246c10-fe0b-b8e5-cdc3-54bdc6c8054e@ramsayj
> >> ones.plus.com/> 
> > The changes in the most recent two versions look good to me.
> 
> I recently tested the 'seen' branch for an unrelated reason, but found
> that the t7900-maintenance.sh test failed for me. It was during test 34,
> 'start and stop Linux/systemd maintenance' with the following issue:
> 
>   + systemd-analyze verify systemd/user/git-maintenance@.service
>   Failed to create /user.slice/user-1000.slice/session-44.scope/init.scope
> control group: Permission denied Failed to initialize manager: Permission
> denied
> 
> Now, this test has the prereq SYSTEMD_ANALYZE, but for some reason this
> later command fails for permission issues. I'm running Ubuntu, if that
> helps.

Thank you for the feedback.

Could you please share which version of Ubuntu and which version of systemd 
you are using ?

I’ve just tried to start an Ubuntu Impish 21.10 which uses systemd 
248.3-1ubuntu3 and to test the `seen` git branch.

All tests of `t/t7900-maintenance.sh` passed including the one which is 
failing for you.

As `systemd-analyse verify` should only check a unit file validity [1], I 
wouldn’t expect it to fail on a cgroup manipulation.

[1] https://www.freedesktop.org/software/systemd/man/systemd-analyze.html#systemd-analyze%20verify%20FILE...

I tried to run 
systemd-analyze verify /etc/systemd/system/sshd.service
and it didn’t produce the error you mentioned but if I `strace` it, I can find:

mkdir("/sys/fs/cgroup/unified/user.slice/user-1000.slice/session-3.scope/
init.scope", 0755) = -1 EACCES (Permission denied)

This makes me think your version of systemd is wrongly considering this cgroup 
directory failure as fatal.
I’d like to know more precisely which versions are affected.

Kind regards,
Lénaïc.



^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v10 0/3] maintenance: add support for systemd timers on Linux
  2021-09-09  5:52                       ` Lénaïc Huard
@ 2021-09-09 19:55                         ` Derrick Stolee
  2021-09-27 12:50                           ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 138+ messages in thread
From: Derrick Stolee @ 2021-09-09 19:55 UTC (permalink / raw)
  To: Lénaïc Huard, git
  Cc: Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren,
	Ævar Arnfjörð Bjarmason, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King, Ramsay Jones

On 9/9/2021 1:52 AM, Lénaïc Huard wrote:
> Le mercredi 8 septembre 2021, 13:44:26 CEST Derrick Stolee a écrit :
>> On 9/7/2021 12:48 PM, Derrick Stolee wrote:
>>> On 9/4/2021 4:54 PM, Lénaïc Huard wrote:
>>>> Hello,
>>>>
>>>> Please find hereafter my updated patchset to add support for systemd
>>>> timers on Linux for the `git maintenance start` command.
>>>>
>>>> The only changes compared to the previous version are fixes for the
>>>> two typos in a comment that Ramsay Jones pointed out [1]
>>>>
>>>> [1]
>>>> https://lore.kernel.org/git/51246c10-fe0b-b8e5-cdc3-54bdc6c8054e@ramsayj
>>>> ones.plus.com/> 
>>> The changes in the most recent two versions look good to me.
>>
>> I recently tested the 'seen' branch for an unrelated reason, but found
>> that the t7900-maintenance.sh test failed for me. It was during test 34,
>> 'start and stop Linux/systemd maintenance' with the following issue:
>>
>>   + systemd-analyze verify systemd/user/git-maintenance@.service
>>   Failed to create /user.slice/user-1000.slice/session-44.scope/init.scope
>> control group: Permission denied Failed to initialize manager: Permission
>> denied
>>
>> Now, this test has the prereq SYSTEMD_ANALYZE, but for some reason this
>> later command fails for permission issues. I'm running Ubuntu, if that
>> helps.
> 
> Thank you for the feedback.
> 
> Could you please share which version of Ubuntu and which version of systemd 
> you are using ?
> 
> I’ve just tried to start an Ubuntu Impish 21.10 which uses systemd 
> 248.3-1ubuntu3 and to test the `seen` git branch.
> 
> All tests of `t/t7900-maintenance.sh` passed including the one which is 
> failing for you.
> 
> As `systemd-analyse verify` should only check a unit file validity [1], I 
> wouldn’t expect it to fail on a cgroup manipulation.
> 
> [1] https://www.freedesktop.org/software/systemd/man/systemd-analyze.html#systemd-analyze%20verify%20FILE...
> 
> I tried to run 
> systemd-analyze verify /etc/systemd/system/sshd.service
> and it didn’t produce the error you mentioned but if I `strace` it, I can find:
> 
> mkdir("/sys/fs/cgroup/unified/user.slice/user-1000.slice/session-3.scope/
> init.scope", 0755) = -1 EACCES (Permission denied)
> 
> This makes me think your version of systemd is wrongly considering this cgroup 
> directory failure as fatal.
> I’d like to know more precisely which versions are affected.
 I am on Ubuntu 18.04.

$ systemd --version
systemd 237
+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN -PCRE2 default-hierarchy=hybrid

I tried upgrading with apt-get, but that did not get me a new
version.

Thanks,
-Stolee

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v10 0/3] maintenance: add support for systemd timers on Linux
  2021-09-09 19:55                         ` Derrick Stolee
@ 2021-09-27 12:50                           ` Ævar Arnfjörð Bjarmason
  2021-09-27 21:44                             ` Lénaïc Huard
  0 siblings, 1 reply; 138+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-09-27 12:50 UTC (permalink / raw)
  To: Derrick Stolee
  Cc: Lénaïc Huard, git, Junio C Hamano, Derrick Stolee,
	Eric Sunshine, Đoàn Trần Công Danh,
	Felipe Contreras, Phillip Wood, Martin Ågren, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King, Ramsay Jones


On Thu, Sep 09 2021, Derrick Stolee wrote:

> On 9/9/2021 1:52 AM, Lénaïc Huard wrote:
>> Le mercredi 8 septembre 2021, 13:44:26 CEST Derrick Stolee a écrit :
>>> On 9/7/2021 12:48 PM, Derrick Stolee wrote:
>>>> On 9/4/2021 4:54 PM, Lénaïc Huard wrote:
>>>>> Hello,
>>>>>
>>>>> Please find hereafter my updated patchset to add support for systemd
>>>>> timers on Linux for the `git maintenance start` command.
>>>>>
>>>>> The only changes compared to the previous version are fixes for the
>>>>> two typos in a comment that Ramsay Jones pointed out [1]
>>>>>
>>>>> [1]
>>>>> https://lore.kernel.org/git/51246c10-fe0b-b8e5-cdc3-54bdc6c8054e@ramsayj
>>>>> ones.plus.com/> 
>>>> The changes in the most recent two versions look good to me.
>>>
>>> I recently tested the 'seen' branch for an unrelated reason, but found
>>> that the t7900-maintenance.sh test failed for me. It was during test 34,
>>> 'start and stop Linux/systemd maintenance' with the following issue:
>>>
>>>   + systemd-analyze verify systemd/user/git-maintenance@.service
>>>   Failed to create /user.slice/user-1000.slice/session-44.scope/init.scope
>>> control group: Permission denied Failed to initialize manager: Permission
>>> denied
>>>
>>> Now, this test has the prereq SYSTEMD_ANALYZE, but for some reason this
>>> later command fails for permission issues. I'm running Ubuntu, if that
>>> helps.
>> 
>> Thank you for the feedback.
>> 
>> Could you please share which version of Ubuntu and which version of systemd 
>> you are using ?
>> 
>> I’ve just tried to start an Ubuntu Impish 21.10 which uses systemd 
>> 248.3-1ubuntu3 and to test the `seen` git branch.
>> 
>> All tests of `t/t7900-maintenance.sh` passed including the one which is 
>> failing for you.
>> 
>> As `systemd-analyse verify` should only check a unit file validity [1], I 
>> wouldn’t expect it to fail on a cgroup manipulation.
>> 
>> [1] https://www.freedesktop.org/software/systemd/man/systemd-analyze.html#systemd-analyze%20verify%20FILE...
>> 
>> I tried to run 
>> systemd-analyze verify /etc/systemd/system/sshd.service
>> and it didn’t produce the error you mentioned but if I `strace` it, I can find:
>> 
>> mkdir("/sys/fs/cgroup/unified/user.slice/user-1000.slice/session-3.scope/
>> init.scope", 0755) = -1 EACCES (Permission denied)
>> 
>> This makes me think your version of systemd is wrongly considering this cgroup 
>> directory failure as fatal.
>> I’d like to know more precisely which versions are affected.
>  I am on Ubuntu 18.04.
>
> $ systemd --version
> systemd 237
> +PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP
> +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS
> +KMOD -IDN2 +IDN -PCRE2 default-hierarchy=hybrid
>
> I tried upgrading with apt-get, but that did not get me a new
> version.

It seems this discussion has gone stale, but this is still broken on
some systems. This is gcc135 on the GCC Farm, which passes the prereq
this commit adds:

    $ systemd-analyze verify systemd/user/git-maintenance@.service
    Failed to open /dev/tty0: Permission denied
    Failed to load systemd/user/git-maintenance@.service: Invalid argument

I don't know the systemd specifics involved, but this seems like a
rather straightforward problem of assuming permissions that aren't
universal. I.e. let's try to do that in the prereq instead?

OS details, if they matter:
    
    [avar@gcc135 t]$ systemd-analyze --version
    systemd 219
    +PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN
    [avar@gcc135 t]$ cat /etc/centos-release
    CentOS Linux release 7.9.2009 (AltArch)
    

^ permalink raw reply	[flat|nested] 138+ messages in thread

* Re: [PATCH v10 0/3] maintenance: add support for systemd timers on Linux
  2021-09-27 12:50                           ` Ævar Arnfjörð Bjarmason
@ 2021-09-27 21:44                             ` Lénaïc Huard
  0 siblings, 0 replies; 138+ messages in thread
From: Lénaïc Huard @ 2021-09-27 21:44 UTC (permalink / raw)
  To: Derrick Stolee, Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Derrick Stolee, Eric Sunshine,
	Đoàn Trần Công Danh, Felipe Contreras,
	Phillip Wood, Martin Ågren, Bagas Sanjaya,
	brian m . carlson, Johannes Schindelin, Jeff King, Ramsay Jones

Le lundi 27 septembre 2021, 14:50:28 CEST Ævar Arnfjörð Bjarmason a écrit :
> On Thu, Sep 09 2021, Derrick Stolee wrote:
> > On 9/9/2021 1:52 AM, Lénaïc Huard wrote:
> >> Le mercredi 8 septembre 2021, 13:44:26 CEST Derrick Stolee a écrit :
> >>> On 9/7/2021 12:48 PM, Derrick Stolee wrote:
> >>>> On 9/4/2021 4:54 PM, Lénaïc Huard wrote:
> >>>>> Hello,
> >>>>> 
> >>>>> Please find hereafter my updated patchset to add support for systemd
> >>>>> timers on Linux for the `git maintenance start` command.
> >>>>> 
> >>>>> The only changes compared to the previous version are fixes for the
> >>>>> two typos in a comment that Ramsay Jones pointed out [1]
> >>>>> 
> >>>>> [1]
> >>>>> https://lore.kernel.org/git/51246c10-fe0b-b8e5-cdc3-54bdc6c8054e@ramsa
> >>>>> yj
> >>>>> ones.plus.com/>
> >>>> 
> >>>> The changes in the most recent two versions look good to me.
> >>> 
> >>> I recently tested the 'seen' branch for an unrelated reason, but found
> >>> that the t7900-maintenance.sh test failed for me. It was during test 34,
> >>> 
> >>> 'start and stop Linux/systemd maintenance' with the following issue:
> >>>   + systemd-analyze verify systemd/user/git-maintenance@.service
> >>>   Failed to create
> >>>   /user.slice/user-1000.slice/session-44.scope/init.scope
> >>> 
> >>> control group: Permission denied Failed to initialize manager:
> >>> Permission
> >>> denied
> >>> 
> >>> Now, this test has the prereq SYSTEMD_ANALYZE, but for some reason this
> >>> later command fails for permission issues. I'm running Ubuntu, if that
> >>> helps.
> >> 
> >> Thank you for the feedback.
> >> 
> >> Could you please share which version of Ubuntu and which version of
> >> systemd
> >> you are using ?
> >> 
> >> I’ve just tried to start an Ubuntu Impish 21.10 which uses systemd
> >> 248.3-1ubuntu3 and to test the `seen` git branch.
> >> 
> >> All tests of `t/t7900-maintenance.sh` passed including the one which is
> >> failing for you.
> >> 
> >> As `systemd-analyse verify` should only check a unit file validity [1], I
> >> wouldn’t expect it to fail on a cgroup manipulation.
> >> 
> >> [1]
> >> https://www.freedesktop.org/software/systemd/man/systemd-analyze.html#sy
> >> stemd-analyze%20verify%20FILE...
> >> 
> >> I tried to run
> >> systemd-analyze verify /etc/systemd/system/sshd.service
> >> and it didn’t produce the error you mentioned but if I `strace` it, I can
> >> find:
> >> 
> >> mkdir("/sys/fs/cgroup/unified/user.slice/user-1000.slice/session-3.scope/
> >> init.scope", 0755) = -1 EACCES (Permission denied)
> >> 
> >> This makes me think your version of systemd is wrongly considering this
> >> cgroup directory failure as fatal.
> >> I’d like to know more precisely which versions are affected.
> >> 
> >  I am on Ubuntu 18.04.
> > 
> > $ systemd --version
> > systemd 237
> > +PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP
> > +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS
> > +KMOD -IDN2 +IDN -PCRE2 default-hierarchy=hybrid
> > 
> > I tried upgrading with apt-get, but that did not get me a new
> > version.
> 
> It seems this discussion has gone stale, but this is still broken on
> some systems. This is gcc135 on the GCC Farm, which passes the prereq
> this commit adds:
> 
>     $ systemd-analyze verify systemd/user/git-maintenance@.service
>     Failed to open /dev/tty0: Permission denied
>     Failed to load systemd/user/git-maintenance@.service: Invalid argument
> 
> I don't know the systemd specifics involved, but this seems like a
> rather straightforward problem of assuming permissions that aren't
> universal. I.e. let's try to do that in the prereq instead?
> 
> OS details, if they matter:
> 
>     [avar@gcc135 t]$ systemd-analyze --version
>     systemd 219
>     +PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP
> +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS
> +KMOD +IDN [avar@gcc135 t]$ cat /etc/centos-release
>     CentOS Linux release 7.9.2009 (AltArch)

Hello,

I’ve just submitted a new patch to change the way the prerequisite is 
implemented:

https://lore.kernel.org/git/20210927213016.21714-2-lenaic@lhuard.fr/

I’m not sure what was going wrong with old versions of systemd but with this 
new patch, the permission errors raised by `systemd-analyze` should now make 
the prerequisite fail instead of making the test fail.




^ permalink raw reply	[flat|nested] 138+ messages in thread

end of thread, other threads:[~2021-09-27 21:44 UTC | newest]

Thread overview: 138+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-05-01 14:52 [PATCH] maintenance: use systemd timers on Linux Lénaïc Huard
2021-05-01 20:02 ` brian m. carlson
2021-05-02  5:28 ` Bagas Sanjaya
2021-05-02  6:49   ` Eric Sunshine
2021-05-02  6:45 ` Eric Sunshine
2021-05-02 14:10   ` Phillip Wood
2021-05-05 12:19     ` Đoàn Trần Công Danh
2021-05-05 14:57       ` Phillip Wood
2021-05-05 12:01   ` Ævar Arnfjörð Bjarmason
2021-05-09 22:34     ` Lénaïc Huard
2021-05-10 13:03       ` Ævar Arnfjörð Bjarmason
2021-05-02 11:12 ` Bagas Sanjaya
2021-05-03 12:04 ` Derrick Stolee
2021-05-09 21:32 ` [PATCH v2 0/1] " Lénaïc Huard
2021-05-09 21:32   ` [PATCH v2 1/1] " Lénaïc Huard
2021-05-10  1:20     ` Đoàn Trần Công Danh
2021-05-10  2:48       ` Eric Sunshine
2021-05-10  6:25         ` Junio C Hamano
2021-05-12  0:29           ` Đoàn Trần Công Danh
2021-05-12  6:59             ` Felipe Contreras
2021-05-12 13:26               ` Phillip Wood
2021-05-12 13:38             ` Phillip Wood
2021-05-12 15:41               ` Đoàn Trần Công Danh
2021-05-10 18:03     ` Phillip Wood
2021-05-10 18:25       ` Eric Sunshine
2021-05-10 20:09         ` Phillip Wood
2021-05-10 20:52           ` Eric Sunshine
2021-06-08 14:55       ` Lénaïc Huard
2021-05-10 19:15     ` Martin Ågren
2021-05-11 14:50     ` Phillip Wood
2021-05-11 17:31     ` Derrick Stolee
2021-05-20 22:13   ` [PATCH v3 0/4] " Lénaïc Huard
2021-05-20 22:13     ` [PATCH v3 1/4] cache.h: rename "xdg_config_home" to "xdg_config_home_git" Lénaïc Huard
2021-05-20 23:44       ` Đoàn Trần Công Danh
2021-05-20 22:13     ` [PATCH v3 2/4] maintenance: introduce ENABLE/DISABLE for code clarity Lénaïc Huard
2021-05-20 22:13     ` [PATCH v3 3/4] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
2021-05-21  9:52       ` Bagas Sanjaya
2021-05-20 22:13     ` [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux Lénaïc Huard
2021-05-21  9:59       ` Bagas Sanjaya
2021-05-21 16:59         ` Derrick Stolee
2021-05-22  6:57           ` Johannes Schindelin
2021-05-23 18:36             ` Felipe Contreras
2021-05-23 23:27               ` brian m. carlson
2021-05-24  1:18                 ` Felipe Contreras
2021-05-24  7:03                 ` Ævar Arnfjörð Bjarmason
2021-05-24 15:51                   ` Junio C Hamano
2021-05-25  1:50                     ` Johannes Schindelin
2021-05-25 11:13                       ` Felipe Contreras
2021-05-26 10:29                       ` CoC, inclusivity etc. (was "Re: [...] systemd timers on Linux") Ævar Arnfjörð Bjarmason
2021-05-26 16:05                         ` Felipe Contreras
2021-05-27 14:24                         ` Jeff King
2021-05-27 17:30                           ` Felipe Contreras
2021-05-27 23:58                           ` Junio C Hamano
2021-05-28 14:44                           ` Phillip Susi
2021-05-30 21:58                             ` Jeff King
2021-05-24 17:52                   ` [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux Felipe Contreras
2021-05-24  7:15     ` [PATCH v4 0/4] add support for " Lénaïc Huard
2021-05-24  7:15       ` [PATCH v4 1/4] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
2021-05-24  9:33         ` Phillip Wood
2021-05-24 12:23           ` Đoàn Trần Công Danh
2021-05-24  7:15       ` [PATCH v4 2/4] maintenance: introduce ENABLE/DISABLE for code clarity Lénaïc Huard
2021-05-24  9:41         ` Phillip Wood
2021-05-24 12:36           ` Đoàn Trần Công Danh
2021-05-25  7:18             ` Lénaïc Huard
2021-05-25  8:02               ` Junio C Hamano
2021-05-24  9:47         ` Ævar Arnfjörð Bjarmason
2021-05-24  7:15       ` [PATCH v4 3/4] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
2021-05-24 10:12         ` Phillip Wood
2021-05-30  6:39           ` Lénaïc Huard
2021-05-30 10:16             ` Phillip Wood
2021-05-24  7:15       ` [PATCH v4 4/4] maintenance: add support for systemd timers on Linux Lénaïc Huard
2021-05-24  9:55         ` Ævar Arnfjörð Bjarmason
2021-05-24 16:39           ` Eric Sunshine
2021-05-24 18:08         ` Felipe Contreras
2021-05-26 10:26         ` Phillip Wood
2021-05-24  9:04       ` [PATCH v4 0/4] " Junio C Hamano
2021-06-08 13:39       ` [PATCH v5 0/3] " Lénaïc Huard
2021-06-08 13:39         ` [PATCH v5 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
2021-06-08 13:39         ` [PATCH v5 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
2021-06-08 13:40         ` [PATCH v5 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
2021-06-09  9:34           ` Jeff King
2021-06-09 15:01           ` Phillip Wood
2021-06-09  0:21         ` [PATCH v5 0/3] " Junio C Hamano
2021-06-09 14:54         ` Phillip Wood
2021-06-12 16:50         ` [PATCH v6 0/3] maintenance: " Lénaïc Huard
2021-06-12 16:50           ` [PATCH v6 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
2021-06-12 16:50           ` [PATCH v6 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
2021-06-14  4:36             ` Eric Sunshine
2021-06-16 18:12               ` Derrick Stolee
2021-06-17  4:11                 ` Eric Sunshine
2021-06-17 14:26               ` Phillip Wood
2021-07-02 15:04               ` Lénaïc Huard
2021-06-12 16:50           ` [PATCH v6 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
2021-07-02 14:25           ` [PATCH v7 0/3] " Lénaïc Huard
2021-07-02 14:25             ` [PATCH v7 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
2021-07-02 14:25             ` [PATCH v7 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
2021-07-06 19:56               ` Ævar Arnfjörð Bjarmason
2021-07-06 20:52                 ` Junio C Hamano
2021-07-13  0:15                   ` Jeff King
2021-07-13  2:22                     ` Eric Sunshine
2021-07-13  3:56                       ` Jeff King
2021-07-13  5:17                         ` Eric Sunshine
2021-07-13  7:04                       ` Bagas Sanjaya
2021-07-06 21:18                 ` Felipe Contreras
2021-08-23 20:06                 ` Lénaïc Huard
2021-08-23 22:30                   ` Junio C Hamano
2021-07-02 14:25             ` [PATCH v7 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
2021-07-06 20:03               ` Ævar Arnfjörð Bjarmason
2021-07-02 18:18             ` [PATCH v7 0/3] " Junio C Hamano
2021-07-06 13:18             ` Phillip Wood
2021-08-23 20:40             ` [PATCH v8 " Lénaïc Huard
2021-08-23 20:40               ` [PATCH v8 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
2021-08-23 20:40               ` [PATCH v8 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
2021-08-24 17:45                 ` Derrick Stolee
2021-08-23 20:40               ` [PATCH v8 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
2021-08-24 17:45                 ` Derrick Stolee
2021-08-24 17:47               ` [PATCH v8 0/3] " Derrick Stolee
2021-08-27 21:02               ` [PATCH v9 " Lénaïc Huard
2021-08-27 21:02                 ` [PATCH v9 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
2021-08-27 21:02                 ` [PATCH v9 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
2021-08-27 23:54                   ` Ramsay Jones
2021-08-27 21:02                 ` [PATCH v9 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
2021-09-04 20:54                 ` [PATCH v10 0/3] " Lénaïc Huard
2021-09-04 20:54                   ` [PATCH v10 1/3] cache.h: Introduce a generic "xdg_config_home_for(…)" function Lénaïc Huard
2021-09-04 20:54                   ` [PATCH v10 2/3] maintenance: `git maintenance run` learned `--scheduler=<scheduler>` Lénaïc Huard
2021-09-04 20:55                   ` [PATCH v10 3/3] maintenance: add support for systemd timers on Linux Lénaïc Huard
2021-09-07 16:48                   ` [PATCH v10 0/3] " Derrick Stolee
2021-09-08 11:44                     ` Derrick Stolee
2021-09-09  5:52                       ` Lénaïc Huard
2021-09-09 19:55                         ` Derrick Stolee
2021-09-27 12:50                           ` Ævar Arnfjörð Bjarmason
2021-09-27 21:44                             ` Lénaïc Huard
2021-08-17 17:22         ` [PATCH v5 0/3] " Derrick Stolee
2021-08-17 19:43           ` Phillip Wood
2021-08-17 20:29             ` Derrick Stolee
2021-08-18  5:56           ` Lénaïc Huard
2021-08-18 13:28             ` Derrick Stolee
2021-08-18 18:23               ` Junio C Hamano

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.