git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Junio C Hamano <gitster@pobox.com>
To: "Fabian Stelzer via GitGitGadget" <gitgitgadget@gmail.com>
Cc: git@vger.kernel.org, "Han-Wen Nienhuys" <hanwen@google.com>,
	"Fabian Stelzer" <fs@gigacodes.de>,
	"brian m. carlson" <sandals@crustytoothpaste.net>,
	"Randall S. Becker" <rsbecker@nexbridge.com>,
	"Bagas Sanjaya" <bagasdotme@gmail.com>,
	"Hans Jerry Illikainen" <hji@dyntopia.com>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
	"Felipe Contreras" <felipe.contreras@gmail.com>,
	"Eric Sunshine" <sunshine@sunshineco.com>,
	"Gwyneth Morgan" <gwymor@tilde.club>
Subject: Re: [PATCH v4 2/9] ssh signing: add ssh signature format and signing using ssh keys
Date: Mon, 19 Jul 2021 16:53:22 -0700	[thread overview]
Message-ID: <xmqqpmvdj1xp.fsf@gitster.g> (raw)
In-Reply-To: <2c75adee8e1d6147c5be1b3b0832cc90d44ba6df.1626701596.git.gitgitgadget@gmail.com> (Fabian Stelzer via GitGitGadget's message of "Mon, 19 Jul 2021 13:33:09 +0000")

"Fabian Stelzer via GitGitGadget" <gitgitgadget@gmail.com> writes:

> @@ -65,6 +73,14 @@ static struct gpg_format gpg_format[] = {
>  		.verify_signed_buffer = verify_gpg_signed_buffer,
>  		.sign_buffer = sign_buffer_gpg,
>  	},
> +	{
> +		.name = "ssh",
> +		.program = "ssh-keygen",
> +		.verify_args = ssh_verify_args,
> +		.sigs = ssh_sigs,
> +		.verify_signed_buffer = NULL, /* TODO */
> +		.sign_buffer = sign_buffer_ssh
> +	},
>  };

A payload a malicious person may feed this version of Git can have a
pattern that happens to match the ssh_sigs[] string, and the code
will blindly try to call .verify_signed_buffer==NULL and die, no?

That is not the end of the world; as long as we know that with the
above "TODO" comment it is probably OK.

> @@ -463,12 +482,26 @@ int sign_buffer(struct strbuf *buffer, struct strbuf *signature, const char *sig
>  	return use_format->sign_buffer(buffer, signature, signing_key);
>  }
>  
> +static void strbuf_trim_trailing_cr(struct strbuf *buffer, int offset)
> +{

This removes any and all CR, not just trimming the trailing ones, so
the function is misnamed. Call it remove_cr_after() perhaps?

Alternatively we could tighten the implementation and strip only the
CR that come immediately before a LF.  That would be a better longer
term thing to do, but because you are lifting an existing code from
the end of the gpg side of the thing, it may make sense to keep the
implementation as-is, but give it a name that is more faithful to
what it actually does.  When the dust settles, we may want to
revisit and fix this helper function to actually trim CRLF into LF
(and leave CR in the middle of lines intact), but I do not think it
is urgent.  Just leaving "NEEDSWORK: make it trim only CRs before LFs
and rename" comment would be OK.

Shouldn't the offset (aka bottom) be of type size_t?

I do not recommend giving the function a name that begins with
"strbuf_", as it would tempt unthinking person to suggest moving it
to strbuf.c, but the presense of the "offset" thing means it will be
klunky to reuse in other more generic contexts as a part of the
strbuf API.

> +	size_t i, j;
> +
> +	for (i = j = offset; i < buffer->len; i++) {
> +		if (buffer->buf[i] != '\r') {
> ...

Now it gets interesting ;-)

> +static int sign_buffer_ssh(struct strbuf *buffer, struct strbuf *signature,
> +			   const char *signing_key)
> +{
> +	struct child_process signer = CHILD_PROCESS_INIT;
> +	int ret = -1;
> +	size_t bottom;
> +	struct strbuf signer_stderr = STRBUF_INIT;
> +	struct tempfile *temp = NULL, *buffer_file = NULL;
> +	char *ssh_signing_key_file = NULL;
> +	struct strbuf ssh_signature_filename = STRBUF_INIT;
> +
> +	if (!signing_key || signing_key[0] == '\0')
> +		return error(
> +			_("user.signingkey needs to be set for ssh signing"));
> +
> +	if (istarts_with(signing_key, "ssh-")) {

Is it common in the ssh world to treat ssh- prefix as case
insensitive?  Not a strong objection but I tend to prefer to start
strict unless there is a good reason to be loose when we do not have
to, as loosening after the fact is much easier than tightening after
starting with a loose definition.

> +		/* A literal ssh key */
> +		temp = mks_tempfile_t(".git_signing_key_tmpXXXXXX");
> +		if (!temp)
> +			return error_errno(
> +				_("could not create temporary file"));
> +		if (write_in_full(temp->fd, signing_key, strlen(signing_key)) <
> +			    0 ||

"keylen = strlen(signing_key)" before that line, for example, could
have easily avoided the line-wrapping at such a place.  Wrapping at
places like after ||, i.e. after an operator with a low precedence,
would make the code easier to follow.

> +		    close_tempfile_gently(temp) < 0) {
> +			error_errno(_("failed writing ssh signing key to '%s'"),
> +				    temp->filename.buf);
> +			goto out;
> +		}
> +		ssh_signing_key_file = temp->filename.buf;

It is kind'a sad that we need a fresh temporary file every time, but
we can easily tell the user in the documentation that they can use a
file with a key in it to avoid it, so it's OK (actually, better than
OK, as without this, we may not consume temporary files but we won't
offer an ability to take a literal key string).

Is ".git_whatever file in the current directory" a good place to
have this temporary file?  I would have expected that we would use
either $GIT_DIR, $HOME, or $TMPDIR for a thing like this (with
different pros-and-cons discussion).  At least it is consistent with
how a temporary file for the payload to be sign-verified is created,
so let's leave it as-is.

> +	} else {
> +		/* We assume a file */
> +		ssh_signing_key_file = expand_user_path(signing_key, 1);
> +	}
> +
> +	buffer_file = mks_tempfile_t(".git_signing_buffer_tmpXXXXXX");
> +	if (!buffer_file) {
> +		error_errno(_("could not create temporary file"));
> +		goto out;
> +	}
> +
> +	if (write_in_full(buffer_file->fd, buffer->buf, buffer->len) < 0 ||
> +	    close_tempfile_gently(buffer_file) < 0) {
> +		error_errno(_("failed writing ssh signing key buffer to '%s'"),
> +			    buffer_file->filename.buf);
> +		goto out;
> +	}
> +
> +	strvec_pushl(&signer.args, use_format->program, "-Y", "sign", "-n",
> +		     "git", "-f", ssh_signing_key_file,

Wrap the line before "-n" to keep "-n" and "git" together, if "git"
is meant as an argument to the "-n" option.

> +		     buffer_file->filename.buf, NULL);
> +
> +	sigchain_push(SIGPIPE, SIG_IGN);
> +	ret = pipe_command(&signer, NULL, 0, NULL, 0, &signer_stderr, 0);
> +	sigchain_pop(SIGPIPE);
> +
> +	if (ret && strstr(signer_stderr.buf, "usage:")) {
> +		error(_("ssh-keygen -Y sign is needed for ssh signing (available in openssh version 8.2p1+)"));
> +		goto out;

This error message is important to give to the end users, but is it
enough?  That is, unless "usage:" does not appear, we show the whole
raw error message and that would help end users and those helping
them to diagnose the issue, but once the underlying program says
"usage:", no matter what else it says, it is hidden by this code,
since we assume it is a wrong version of openssh.

> +	}
> +
> +	if (ret) {
> +		error("%s", signer_stderr.buf);
> +		goto out;
> +	}

Also, prehaps

	if (ret) {
		if (strstr(..., "usage"))
			error(_("ssh-keygen -Y sign is needed..."));
		else
                        error("%s", signer_stderr.buf);
		goto out;
	}

would be easier to follow.

> +	bottom = signature->len;
> +
> +	strbuf_addbuf(&ssh_signature_filename, &buffer_file->filename);
> +	strbuf_addstr(&ssh_signature_filename, ".sig");
> +	if (strbuf_read_file(signature, ssh_signature_filename.buf, 2048) < 0) {

Is it likely that signature file is smaller than 2kB?  I am just
wondering how much we care to pick the default that is specific to
this codepath vs passing 0 to ask the strbuf API to use whatever
default it wants to.

> +		error_errno(
> +			_("failed reading ssh signing data buffer from '%s'"),
> +			ssh_signature_filename.buf);
> +	}
> +	unlink_or_warn(ssh_signature_filename.buf);

Wait a bit.  Even when pipe_command() tells us that we failed, we
read from ssh_signature_filename anyway?  What is going on?  And ...

> +	if (ret) {

... does this ever trigger?  I thought we would have hit one of the
two "goto out" when ret signals an error by being non-zero earlier,
and since then nobody touched the variable so far.

> +		error(_("ssh failed to sign the data"));
> +		goto out;
> +	}
> +
> +	/* Strip CR from the line endings, in case we are on Windows. */
> +	strbuf_trim_trailing_cr(signature, bottom);
> +
> +out:
> +	if (temp)
> +		delete_tempfile(&temp);
> +	if (buffer_file)
> +		delete_tempfile(&buffer_file);

It is clear that the latter one was holding the contents of the
buffer to be signed, but reminding the readers what "temp" was about
would be a good move.  Perhaps renaming the variable to "key_file"
or something may help?

> +	strbuf_release(&signer_stderr);
> +	strbuf_release(&ssh_signature_filename);
> +	return ret;
> +}

Looking good, except for the "when does 'ret' get updated?
shouldn't we refrain from reading the resulting buffer when it is
set?" question.

Thanks for a pleasant read.


  reply	other threads:[~2021-07-20  0:33 UTC|newest]

Thread overview: 153+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-07-06  8:19 [PATCH] Add commit & tag signing/verification via SSH keys using ssh-keygen Fabian Stelzer via GitGitGadget
2021-07-06 10:07 ` Han-Wen Nienhuys
2021-07-06 11:23   ` Fabian Stelzer
2021-07-06 14:44 ` brian m. carlson
2021-07-06 15:33   ` Fabian Stelzer
2021-07-06 15:04 ` Junio C Hamano
2021-07-06 15:45   ` Fabian Stelzer
2021-07-06 17:55     ` Junio C Hamano
2021-07-06 19:39     ` Randall S. Becker
2021-07-07  6:26 ` Bagas Sanjaya
2021-07-07  8:48   ` Fabian Stelzer
2021-07-12 12:19 ` [PATCH v2] Add commit, tag & push " Fabian Stelzer via GitGitGadget
2021-07-12 16:55   ` Ævar Arnfjörð Bjarmason
2021-07-12 20:35     ` Fabian Stelzer
2021-07-12 21:16       ` Felipe Contreras
2021-07-14 12:10   ` [PATCH v3 0/9] RFC: Add commit & tag " Fabian Stelzer via GitGitGadget
2021-07-14 12:10     ` [PATCH v3 1/9] Add commit, tag & push signing via SSH keys Fabian Stelzer via GitGitGadget
2021-07-14 18:19       ` Junio C Hamano
2021-07-14 23:57         ` Eric Sunshine
2021-07-15  8:20         ` Fabian Stelzer
2021-07-14 12:10     ` [PATCH v3 2/9] ssh signing: add documentation Fabian Stelzer via GitGitGadget
2021-07-14 20:07       ` Junio C Hamano
2021-07-15  8:48         ` Fabian Stelzer
2021-07-15 10:43           ` Bagas Sanjaya
2021-07-15 16:29           ` Junio C Hamano
2021-07-14 12:10     ` [PATCH v3 3/9] ssh signing: retrieve a default key from ssh-agent Fabian Stelzer via GitGitGadget
2021-07-14 20:20       ` Junio C Hamano
2021-07-15  7:49         ` Han-Wen Nienhuys
2021-07-15  8:06           ` Fabian Stelzer
2021-07-15  8:13         ` Fabian Stelzer
2021-07-14 12:10     ` [PATCH v3 4/9] ssh signing: sign using either gpg or ssh keys Fabian Stelzer via GitGitGadget
2021-07-14 20:32       ` Junio C Hamano
2021-07-15  8:28         ` Fabian Stelzer
2021-07-14 12:10     ` [PATCH v3 5/9] ssh signing: provide a textual representation of the signing key Fabian Stelzer via GitGitGadget
2021-07-14 12:10     ` [PATCH v3 6/9] ssh signing: parse ssh-keygen output and verify signatures Fabian Stelzer via GitGitGadget
2021-07-16  0:07       ` Gwyneth Morgan
2021-07-16  7:00         ` Fabian Stelzer
2021-07-14 12:10     ` [PATCH v3 7/9] ssh signing: add test prereqs Fabian Stelzer via GitGitGadget
2021-07-14 12:10     ` [PATCH v3 8/9] ssh signing: duplicate t7510 tests for commits Fabian Stelzer via GitGitGadget
2021-07-14 12:10     ` [PATCH v3 9/9] ssh signing: add more tests for logs, tags & push certs Fabian Stelzer via GitGitGadget
2021-07-19 13:33     ` [PATCH v4 0/9] ssh signing: Add commit & tag signing/verification via SSH keys using ssh-keygen Fabian Stelzer via GitGitGadget
2021-07-19 13:33       ` [PATCH v4 1/9] ssh signing: preliminary refactoring and clean-up Fabian Stelzer via GitGitGadget
2021-07-19 23:07         ` Junio C Hamano
2021-07-19 13:33       ` [PATCH v4 2/9] ssh signing: add ssh signature format and signing using ssh keys Fabian Stelzer via GitGitGadget
2021-07-19 23:53         ` Junio C Hamano [this message]
2021-07-20 12:26           ` Fabian Stelzer
2021-07-19 13:33       ` [PATCH v4 3/9] ssh signing: retrieve a default key from ssh-agent Fabian Stelzer via GitGitGadget
2021-07-19 13:33       ` [PATCH v4 4/9] ssh signing: provide a textual representation of the signing key Fabian Stelzer via GitGitGadget
2021-07-19 13:33       ` [PATCH v4 5/9] ssh signing: parse ssh-keygen output and verify signatures Fabian Stelzer via GitGitGadget
2021-07-19 13:33       ` [PATCH v4 6/9] ssh signing: add test prereqs Fabian Stelzer via GitGitGadget
2021-07-19 13:33       ` [PATCH v4 7/9] ssh signing: duplicate t7510 tests for commits Fabian Stelzer via GitGitGadget
2021-07-19 13:33       ` [PATCH v4 8/9] ssh signing: add more tests for logs, tags & push certs Fabian Stelzer via GitGitGadget
2021-07-19 13:33       ` [PATCH v4 9/9] ssh signing: add documentation Fabian Stelzer via GitGitGadget
2021-07-20  0:38       ` [PATCH v4 0/9] ssh signing: Add commit & tag signing/verification via SSH keys using ssh-keygen Junio C Hamano
2021-07-27 13:15       ` [PATCH v5 " Fabian Stelzer via GitGitGadget
2021-07-27 13:15         ` [PATCH v5 1/9] ssh signing: preliminary refactoring and clean-up Fabian Stelzer via GitGitGadget
2021-07-27 13:15         ` [PATCH v5 2/9] ssh signing: add ssh signature format and signing using ssh keys Fabian Stelzer via GitGitGadget
2021-07-27 13:15         ` [PATCH v5 3/9] ssh signing: retrieve a default key from ssh-agent Fabian Stelzer via GitGitGadget
2021-07-27 13:15         ` [PATCH v5 4/9] ssh signing: provide a textual representation of the signing key Fabian Stelzer via GitGitGadget
2021-07-27 13:15         ` [PATCH v5 5/9] ssh signing: parse ssh-keygen output and verify signatures Fabian Stelzer via GitGitGadget
2021-07-27 13:15         ` [PATCH v5 6/9] ssh signing: add test prereqs Fabian Stelzer via GitGitGadget
2021-07-27 13:15         ` [PATCH v5 7/9] ssh signing: duplicate t7510 tests for commits Fabian Stelzer via GitGitGadget
2021-07-27 13:15         ` [PATCH v5 8/9] ssh signing: add more tests for logs, tags & push certs Fabian Stelzer via GitGitGadget
2021-07-27 13:15         ` [PATCH v5 9/9] ssh signing: add documentation Fabian Stelzer via GitGitGadget
2021-07-28 19:36         ` [PATCH v6 0/9] ssh signing: Add commit & tag signing/verification via SSH keys using ssh-keygen Fabian Stelzer via GitGitGadget
2021-07-28 19:36           ` [PATCH v6 1/9] ssh signing: preliminary refactoring and clean-up Fabian Stelzer via GitGitGadget
2021-07-28 22:32             ` Jonathan Tan
2021-07-29  0:58               ` Junio C Hamano
2021-07-29  7:44                 ` Fabian Stelzer
2021-07-29  8:43               ` Fabian Stelzer
2021-07-28 19:36           ` [PATCH v6 2/9] ssh signing: add ssh signature format and signing using ssh keys Fabian Stelzer via GitGitGadget
2021-07-28 22:45             ` Jonathan Tan
2021-07-29  1:01               ` Junio C Hamano
2021-07-29 11:01               ` Fabian Stelzer
2021-07-29 19:09             ` Josh Steadmon
2021-07-29 21:25               ` Fabian Stelzer
2021-07-28 19:36           ` [PATCH v6 3/9] ssh signing: retrieve a default key from ssh-agent Fabian Stelzer via GitGitGadget
2021-07-28 21:29             ` Junio C Hamano
2021-07-28 22:48             ` Jonathan Tan
2021-07-29  8:59               ` Fabian Stelzer
2021-07-29 19:09                 ` Josh Steadmon
2021-07-29 19:56                   ` Junio C Hamano
2021-07-29 21:21                   ` Fabian Stelzer
2021-07-28 19:36           ` [PATCH v6 4/9] ssh signing: provide a textual representation of the signing key Fabian Stelzer via GitGitGadget
2021-07-28 21:34             ` Junio C Hamano
2021-07-29  8:21               ` Fabian Stelzer
2021-07-28 19:36           ` [PATCH v6 5/9] ssh signing: parse ssh-keygen output and verify signatures Fabian Stelzer via GitGitGadget
2021-07-28 21:55             ` Junio C Hamano
2021-07-29  9:12               ` Fabian Stelzer
2021-07-29 20:43                 ` Junio C Hamano
2021-07-28 23:04             ` Jonathan Tan
2021-07-29  9:48               ` Fabian Stelzer
2021-07-29 13:52                 ` Fabian Stelzer
2021-08-03  7:43                   ` Fabian Stelzer
2021-08-03  9:33                     ` Fabian Stelzer
2021-07-29 20:46                 ` Junio C Hamano
2021-07-29 21:01                   ` Randall S. Becker
2021-07-29 21:12                     ` Fabian Stelzer
2021-07-29 21:25                       ` Randall S. Becker
2021-07-29 21:28                         ` Fabian Stelzer
2021-07-29 22:28                           ` Randall S. Becker
2021-07-30  8:17                             ` Fabian Stelzer
2021-07-30 14:26                               ` Randall S. Becker
2021-07-30 14:32                                 ` Fabian Stelzer
2021-07-30 15:05                                   ` Randall S. Becker
2021-07-28 19:36           ` [PATCH v6 6/9] ssh signing: add test prereqs Fabian Stelzer via GitGitGadget
2021-07-29 19:09             ` Josh Steadmon
2021-07-29 19:57               ` Junio C Hamano
2021-07-30  7:32               ` Fabian Stelzer
2021-07-28 19:36           ` [PATCH v6 7/9] ssh signing: duplicate t7510 tests for commits Fabian Stelzer via GitGitGadget
2021-07-28 19:36           ` [PATCH v6 8/9] ssh signing: add more tests for logs, tags & push certs Fabian Stelzer via GitGitGadget
2021-07-28 19:36           ` [PATCH v6 9/9] ssh signing: add documentation Fabian Stelzer via GitGitGadget
2021-07-29  8:19           ` [PATCH v6 0/9] ssh signing: Add commit & tag signing/verification via SSH keys using ssh-keygen Bagas Sanjaya
2021-07-29 11:03             ` Fabian Stelzer
2021-08-03 13:45           ` [PATCH v7 " Fabian Stelzer via GitGitGadget
2021-08-03 13:45             ` [PATCH v7 1/9] ssh signing: preliminary refactoring and clean-up Fabian Stelzer via GitGitGadget
2021-08-03 13:45             ` [PATCH v7 2/9] ssh signing: add test prereqs Fabian Stelzer via GitGitGadget
2021-08-03 13:45             ` [PATCH v7 3/9] ssh signing: add ssh key format and signing code Fabian Stelzer via GitGitGadget
2021-08-03 13:45             ` [PATCH v7 4/9] ssh signing: retrieve a default key from ssh-agent Fabian Stelzer via GitGitGadget
2021-08-03 13:45             ` [PATCH v7 5/9] ssh signing: provide a textual signing_key_id Fabian Stelzer via GitGitGadget
2021-08-03 13:45             ` [PATCH v7 6/9] ssh signing: verify signatures using ssh-keygen Fabian Stelzer via GitGitGadget
2021-08-03 23:47               ` Junio C Hamano
2021-08-04  9:01                 ` Fabian Stelzer
2021-08-04 17:32                   ` Junio C Hamano
2021-08-03 13:45             ` [PATCH v7 7/9] ssh signing: duplicate t7510 tests for commits Fabian Stelzer via GitGitGadget
2021-08-03 13:45             ` [PATCH v7 8/9] ssh signing: tests for logs, tags & push certs Fabian Stelzer via GitGitGadget
2021-08-03 13:45             ` [PATCH v7 9/9] ssh signing: test that gpg fails for unkown keys Fabian Stelzer via GitGitGadget
2021-08-29 22:15             ` [PATCH v7 0/9] ssh signing: Add commit & tag signing/verification via SSH keys using ssh-keygen Junio C Hamano
2021-08-29 23:56               ` Gwyneth Morgan
2021-08-30 10:35               ` Fabian Stelzer
2021-09-07 17:35                 ` Junio C Hamano
2021-09-10  8:03                   ` Fabian Stelzer
2021-09-10 18:44                     ` Junio C Hamano
2021-09-10 19:49                       ` Fabian Stelzer
2021-09-10 20:20                         ` Carlo Arenas
2021-09-10 20:07             ` [PATCH v8 " Fabian Stelzer via GitGitGadget
2021-09-10 20:07               ` [PATCH v8 1/9] ssh signing: preliminary refactoring and clean-up Fabian Stelzer via GitGitGadget
2021-09-10 20:07               ` [PATCH v8 2/9] ssh signing: add test prereqs Fabian Stelzer via GitGitGadget
2021-09-10 20:07               ` [PATCH v8 3/9] ssh signing: add ssh key format and signing code Fabian Stelzer via GitGitGadget
2021-09-10 20:07               ` [PATCH v8 4/9] ssh signing: retrieve a default key from ssh-agent Fabian Stelzer via GitGitGadget
2021-09-10 20:07               ` [PATCH v8 5/9] ssh signing: provide a textual signing_key_id Fabian Stelzer via GitGitGadget
2021-09-10 20:07               ` [PATCH v8 6/9] ssh signing: verify signatures using ssh-keygen Fabian Stelzer via GitGitGadget
2021-09-10 20:07               ` [PATCH v8 7/9] ssh signing: duplicate t7510 tests for commits Fabian Stelzer via GitGitGadget
2021-09-10 20:07               ` [PATCH v8 8/9] ssh signing: tests for logs, tags & push certs Fabian Stelzer via GitGitGadget
2021-09-10 20:07               ` [PATCH v8 9/9] ssh signing: test that gpg fails for unknown keys Fabian Stelzer via GitGitGadget
2021-12-22  3:18                 ` t7510-signed-commit.sh hangs on old gpg, regression in 1bfb57f642d (was: [PATCH v8 9/9] ssh signing: test that gpg fails for unknown keys) Ævar Arnfjörð Bjarmason
2021-12-22 10:13                   ` Fabian Stelzer
2021-12-22 15:58                     ` brian m. carlson
2021-12-26 22:53                     ` Ævar Arnfjörð Bjarmason
2021-12-30 11:10                       ` Fabian Stelzer
2021-09-10 20:23               ` [PATCH v8 0/9] ssh signing: Add commit & tag signing/verification via SSH keys using ssh-keygen Junio C Hamano
2021-09-10 20:48                 ` Fabian Stelzer
2021-09-10 21:01                   ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=xmqqpmvdj1xp.fsf@gitster.g \
    --to=gitster@pobox.com \
    --cc=avarab@gmail.com \
    --cc=bagasdotme@gmail.com \
    --cc=felipe.contreras@gmail.com \
    --cc=fs@gigacodes.de \
    --cc=git@vger.kernel.org \
    --cc=gitgitgadget@gmail.com \
    --cc=gwymor@tilde.club \
    --cc=hanwen@google.com \
    --cc=hji@dyntopia.com \
    --cc=rsbecker@nexbridge.com \
    --cc=sandals@crustytoothpaste.net \
    --cc=sunshine@sunshineco.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).