From: Bernd Edlinger <bernd.edlinger@hotmail.de>
To: "Eric W. Biederman" <ebiederm@xmission.com>
Cc: Christian Brauner <christian.brauner@ubuntu.com>,
Kees Cook <keescook@chromium.org>, Jann Horn <jannh@google.com>,
Jonathan Corbet <corbet@lwn.net>,
Alexander Viro <viro@zeniv.linux.org.uk>,
Andrew Morton <akpm@linux-foundation.org>,
Alexey Dobriyan <adobriyan@gmail.com>,
Thomas Gleixner <tglx@linutronix.de>,
Oleg Nesterov <oleg@redhat.com>,
Frederic Weisbecker <frederic@kernel.org>,
Andrei Vagin <avagin@gmail.com>, Ingo Molnar <mingo@kernel.org>,
"Peter Zijlstra (Intel)" <peterz@infradead.org>,
Yuyang Du <duyuyang@gmail.com>,
David Hildenbrand <david@redhat.com>,
Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
Anshuman Khandual <anshuman.khandual@arm.com>,
David Howells <dhowells@redhat.com>,
James Morris <jamorris@linux.microsoft.com>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Shakeel Butt <shakeelb@google.com>,
Jason Gunthorpe <jgg@ziepe.ca>,
Christian Kellner <christian@kellner.me>,
Andrea Arcangeli <aarcange@redhat.com>,
Aleksa Sarai <cyphar@cyphar.com>,
"Dmitry V. Levin" <ldv@altlinux.org>,
"linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
"stable@vger.kernel.org" <stable@vger.kernel.org>,
"linux-api@vger.kernel.org" <linux-api@vger.kernel.org>
Subject: Re: [PATCHv5] exec: Fix a deadlock in ptrace
Date: Wed, 4 Mar 2020 21:49:33 +0000 [thread overview]
Message-ID: <AM6PR03MB51701E2F405F6DBF1EEB57CDE4E50@AM6PR03MB5170.eurprd03.prod.outlook.com> (raw)
In-Reply-To: <87r1y8dqqz.fsf@x220.int.ebiederm.org>
On 3/4/20 5:33 PM, Eric W. Biederman wrote:
> Bernd Edlinger <bernd.edlinger@hotmail.de> writes:
>
>> On 3/3/20 9:08 PM, Eric W. Biederman wrote:
>>> Bernd Edlinger <bernd.edlinger@hotmail.de> writes:
>>>
>>>> On 3/3/20 4:18 PM, Eric W. Biederman wrote:
>>>>> Bernd Edlinger <bernd.edlinger@hotmail.de> writes:
>>>>>> diff --git a/tools/testing/selftests/ptrace/vmaccess.c b/tools/testing/selftests/ptrace/vmaccess.c
>>>>>> new file mode 100644
>>>>>> index 0000000..6d8a048
>>>>>> --- /dev/null
>>>>>> +++ b/tools/testing/selftests/ptrace/vmaccess.c
>>>>>> @@ -0,0 +1,66 @@
>>>>>> +// SPDX-License-Identifier: GPL-2.0+
>>>>>> +/*
>>>>>> + * Copyright (c) 2020 Bernd Edlinger <bernd.edlinger@hotmail.de>
>>>>>> + * All rights reserved.
>>>>>> + *
>>>>>> + * Check whether /proc/$pid/mem can be accessed without causing deadlocks
>>>>>> + * when de_thread is blocked with ->cred_guard_mutex held.
>>>>>> + */
>>>>>> +
>>>>>> +#include "../kselftest_harness.h"
>>>>>> +#include <stdio.h>
>>>>>> +#include <fcntl.h>
>>>>>> +#include <pthread.h>
>>>>>> +#include <signal.h>
>>>>>> +#include <unistd.h>
>>>>>> +#include <sys/ptrace.h>
>>>>>> +
>>>>>> +static void *thread(void *arg)
>>>>>> +{
>>>>>> + ptrace(PTRACE_TRACEME, 0, 0L, 0L);
>>>>>> + return NULL;
>>>>>> +}
>>>>>> +
>>>>>> +TEST(vmaccess)
>>>>>> +{
>>>>>> + int f, pid = fork();
>>>>>> + char mm[64];
>>>>>> +
>>>>>> + if (!pid) {
>>>>>> + pthread_t pt;
>>>>>> +
>>>>>> + pthread_create(&pt, NULL, thread, NULL);
>>>>>> + pthread_join(pt, NULL);
>>>>>> + execlp("true", "true", NULL);
>>>>>> + }
>>>>>> +
>>>>>> + sleep(1);
>>>>>> + sprintf(mm, "/proc/%d/mem", pid);
>>>>>> + f = open(mm, O_RDONLY);
>>>>>> + ASSERT_LE(0, f);
>>>>>> + close(f);
>>>>>> + f = kill(pid, SIGCONT);
>>>>>> + ASSERT_EQ(0, f);
>>>>>> +}
>>>>>> +
>>>>>> +TEST(attach)
>>>>>> +{
>>>>>> + int f, pid = fork();
>>>>>> +
>>>>>> + if (!pid) {
>>>>>> + pthread_t pt;
>>>>>> +
>>>>>> + pthread_create(&pt, NULL, thread, NULL);
>>>>>> + pthread_join(pt, NULL);
>>>>>> + execlp("true", "true", NULL);
>>>>>> + }
>>>>>> +
>>>>>> + sleep(1);
>>>>>> + f = ptrace(PTRACE_ATTACH, pid, 0L, 0L);
>>>>>
>>>>> To be meaningful this code needs to learn to loop when
>>>>> ptrace returns -EAGAIN.
>>>>>
>>>>> Because that is pretty much what any self respecting user space
>>>>> process will do.
>>>>>
>>>>> At which point I am not certain we can say that the behavior has
>>>>> sufficiently improved not to be a deadlock.
>>>>>
>>>>
>>>> In this special dead-duck test it won't work, but it would
>>>> still be lots more transparent what is going on, since previously
>>>> you had two zombie process, and no way to even output debug
>>>> messages, which also all self respecting user space processes
>>>> should do.
>>>
>>> Agreed it is more transparent. So if you are going to deadlock
>>> it is better.
>>>
>>> My previous proposal (which I admit is more work to implement) would
>>> actually allow succeeding in this case and so it would not be subject to
>>> a dead lock (even via -EGAIN) at this point.
>>>
>>>> So yes, I can at least give a good example and re-try it several
>>>> times together with wait4 which a tracer is expected to do.
>>>
>>> Thank you,
>>>
>>> Eric
>>>
>>
>> Okay, I think it can be done with minimal API changes,
>> but it needs two mutexes, one that guards the execve,
>> and one that guards only the credentials.
>>
>> If no traced sibling thread exists, the mutexes are used this way:
>> lock(exec_guard_mutex)
>> cred_locked_in_execve = true;
>> de_thread()
>> lock(cred_guard_mutex)
>> unlock(cred_guard_mutex)
>> cred_locked_in_execve = false;
>> unlock(exec_guard_mutex)
>>
>> so effectively no API change at all.
>>
>> If a traced sibling thread exists, the mutexes are used differently:
>> lock(exec_guard_mutex)
>> cred_locked_in_execve = true;
>> unlock(exec_guard_mutex)
>> de_thread()
>> lock(cred_guard_mutex)
>> unlock(cred_guard_mutex)
>> lock(exec_guard_mutex)
>> cred_locked_in_execve = false;
>> unlock(exec_guard_mutex)
>>
>> Only the case changes that would deadlock anyway.
>
>
> Let me propose a slight alternative that I think sets us up for long
> term success.
>
> Leave cred_guard_mutex as is, but declare it undesirable. The
> cred_guard_mutex as designed really is something we should get rid of.
> As it it can sleep over several different userspace accesses. The
> copying of the exec arguments is technically as prone to deadlock as the
> ptrace case.
>
> Add a new mutex with a better name perhaps "exec_change_mutex" that is
> used to guard the changes that exec makes to a process.
>
> Then we gradually shift all the cred_guard_mutex users over to the new
> mutex. AKA one patch per user of cred_guard_mutex. At each patch that
> shifts things over we will have the opportunity to review the code to
> see that there no funny dependencies that were missed.
>
> I will sign up for working on the no_new_privs and ptrace_attach cases
> as I think I can make those happen. Especially no_new_privs.
>
> Getting the easier cases will resolve your issues and put things on a
> better footing.
>
> Eric
>
Okay, however I think we will need two mutexes in the long term.
So currently I have reduced the cred_guard_mutex to protect just
the loading of the executable code in the process vm, since that
is what works for vm_access, (one of the test cases).
And another mutex that protects the whole execve function, that
is need for ptrace, (and seccomp).
But I have only a test case for ptrace.
If I understand that right, I should not recycle cred_guard_mutex
but leave it as is, and create two additional mutexes which will
take over step by step.
Sounds reasonable, indeed.
I will send an update (v6) what I have right now,
but just for information, so you can see how my minimal API-Change
approach works.
Bernd.
next prev parent reply other threads:[~2020-03-04 21:49 UTC|newest]
Thread overview: 202+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-01 11:27 [PATCH] exec: Fix a deadlock in ptrace Bernd Edlinger
2020-03-01 15:13 ` Aleksa Sarai
2020-03-01 15:58 ` Christian Brauner
2020-03-01 17:46 ` Bernd Edlinger
2020-03-01 18:20 ` Christian Brauner
2020-03-01 17:24 ` Bernd Edlinger
2020-03-01 18:21 ` Jann Horn
2020-03-01 18:52 ` Christian Brauner
2020-03-01 19:00 ` Bernd Edlinger
2020-03-01 20:00 ` Jann Horn
2020-03-01 20:34 ` [PATCHv2] " Bernd Edlinger
2020-03-02 6:38 ` Eric W. Biederman
2020-03-02 15:43 ` Bernd Edlinger
2020-03-02 15:57 ` Eric W. Biederman
2020-03-02 16:02 ` Bernd Edlinger
2020-03-02 16:17 ` Eric W. Biederman
2020-03-02 16:43 ` Jann Horn
2020-03-02 17:01 ` Bernd Edlinger
2020-03-02 17:37 ` Jann Horn
2020-03-02 17:42 ` christian
2020-03-02 18:08 ` Jann Horn
2020-03-02 20:10 ` [PATCHv3] " Bernd Edlinger
2020-03-02 20:28 ` Bernd Edlinger
2020-03-03 20:13 ` [PATCHv2] " Eric W. Biederman
2020-03-02 17:13 ` Bernd Edlinger
2020-03-02 21:49 ` Eric W. Biederman
2020-03-02 22:00 ` Bernd Edlinger
2020-03-02 22:18 ` [PATCHv4] " Bernd Edlinger
2020-03-03 2:26 ` Kees Cook
2020-03-03 4:54 ` Bernd Edlinger
2020-03-03 5:29 ` Kees Cook
2020-03-03 8:08 ` Bernd Edlinger
2020-03-03 8:34 ` Christian Brauner
2020-03-03 8:43 ` Christian Brauner
2020-03-04 15:30 ` Christian Brauner
2020-03-03 8:58 ` Christian Brauner
2020-03-03 10:34 ` Bernd Edlinger
2020-03-03 11:23 ` Bernd Edlinger
2020-03-03 14:20 ` Christian Brauner
2020-03-03 13:02 ` [PATCHv5] " Bernd Edlinger
2020-03-03 15:18 ` Eric W. Biederman
2020-03-03 16:48 ` Bernd Edlinger
2020-03-03 17:01 ` Christian Brauner
2020-03-03 17:20 ` Christian Brauner
2020-03-03 20:08 ` Eric W. Biederman
2020-03-04 14:37 ` Bernd Edlinger
2020-03-04 16:33 ` Eric W. Biederman
2020-03-04 21:49 ` Bernd Edlinger [this message]
2020-03-04 21:56 ` [PATCHv6] " Bernd Edlinger
2020-03-05 18:36 ` Bernd Edlinger
2020-03-05 21:14 ` [PATCH 0/2] Infrastructure to allow fixing exec deadlocks Eric W. Biederman
2020-03-05 21:15 ` [PATCH 1/2] exec: Properly mark the point of no return Eric W. Biederman
2020-03-05 22:34 ` Bernd Edlinger
2020-03-06 5:19 ` Eric W. Biederman
2020-03-05 22:56 ` Bernd Edlinger
2020-03-06 5:09 ` Eric W. Biederman
2020-03-06 16:26 ` Bernd Edlinger
2020-03-06 17:16 ` Eric W. Biederman
2020-03-05 21:16 ` [PATCH 2/2] exec: Add a exec_update_mutex to replace cred_guard_mutex Eric W. Biederman
2020-03-05 21:51 ` Bernd Edlinger
2020-03-06 5:17 ` Eric W. Biederman
2020-03-06 11:46 ` Bernd Edlinger
2020-03-06 21:18 ` Eric W. Biederman
2020-03-06 19:16 ` Bernd Edlinger
2020-03-06 21:58 ` Eric W. Biederman
2020-03-06 22:29 ` Eric W. Biederman
2020-03-07 1:03 ` Eric W. Biederman
2020-03-08 12:58 ` [PATCH] exec: make de_thread alloc new signal struct earlier Bernd Edlinger
2020-03-08 18:12 ` Eric W. Biederman
2020-03-05 22:31 ` [PATCH 0/2] Infrastructure to allow fixing exec deadlocks Bernd Edlinger
2020-03-06 5:06 ` Eric W. Biederman
2020-03-08 21:34 ` [PATCH 0/5] " Eric W. Biederman
2020-03-08 21:35 ` [PATCH v2 1/5] exec: Only compute current once in flush_old_exec Eric W. Biederman
2020-03-09 13:56 ` Bernd Edlinger
2020-03-09 17:34 ` Eric W. Biederman
2020-03-09 17:56 ` Bernd Edlinger
2020-03-09 19:27 ` Bernd Edlinger
2020-03-10 20:17 ` Kees Cook
2020-03-10 21:12 ` Christian Brauner
2020-03-08 21:36 ` [PATCH v2 2/5] exec: Factor unshare_sighand out of de_thread and call it separately Eric W. Biederman
2020-03-09 19:28 ` Bernd Edlinger
2020-03-10 20:29 ` Kees Cook
2020-03-10 20:34 ` Bernd Edlinger
2020-03-10 20:57 ` Kees Cook
2020-03-10 21:21 ` Christian Brauner
2020-03-08 21:36 ` [PATCH v2 3/5] exec: Move cleanup of posix timers on exec out of de_thread Eric W. Biederman
2020-03-09 19:30 ` Bernd Edlinger
2020-03-09 19:59 ` Christian Brauner
2020-03-09 20:06 ` Eric W. Biederman
2020-03-09 20:17 ` Christian Brauner
2020-03-09 20:48 ` Eric W. Biederman
2020-03-10 8:55 ` Christian Brauner
2020-03-10 18:52 ` [PATCH] pidfd: Stop taking cred_guard_mutex Eric W. Biederman
2020-03-10 19:15 ` Christian Brauner
2020-03-10 19:16 ` Jann Horn
2020-03-10 19:27 ` Eric W. Biederman
2020-03-10 20:00 ` Jann Horn
2020-03-10 20:10 ` Jann Horn
2020-03-10 20:22 ` Bernd Edlinger
2020-03-11 6:11 ` Bernd Edlinger
2020-03-11 14:56 ` Jann Horn
2020-03-10 20:57 ` Eric W. Biederman
2020-03-10 21:29 ` Christian Brauner
2020-03-11 18:49 ` Kees Cook
2020-03-14 9:12 ` [PATCH] pidfd: Use new infrastructure to fix deadlocks in execve Bernd Edlinger
2020-03-10 20:16 ` [PATCH v2 3/5] exec: Move cleanup of posix timers on exec out of de_thread Kees Cook
2020-03-10 20:31 ` Kees Cook
2020-03-10 20:57 ` Jann Horn
2020-03-10 21:05 ` Eric W. Biederman
2020-03-10 21:22 ` Christian Brauner
2020-03-08 21:38 ` [PATCH v2 4/5] exec: Move exec_mmap right after de_thread in flush_old_exec Eric W. Biederman
2020-03-09 19:34 ` Bernd Edlinger
2020-03-09 19:45 ` Eric W. Biederman
2020-03-09 19:52 ` Bernd Edlinger
2020-03-09 19:58 ` Eric W. Biederman
2020-03-09 20:03 ` Bernd Edlinger
2020-03-09 20:35 ` Eric W. Biederman
2020-03-10 20:44 ` Kees Cook
2020-03-10 21:20 ` Eric W. Biederman
2020-03-10 20:47 ` Kees Cook
2020-03-10 21:09 ` Eric W. Biederman
2020-03-08 21:38 ` [PATCH v2 5/5] exec: Add a exec_update_mutex to replace cred_guard_mutex Eric W. Biederman
2020-03-09 13:45 ` Bernd Edlinger
2020-03-09 17:40 ` Eric W. Biederman
2020-03-09 18:01 ` Bernd Edlinger
2020-03-09 18:10 ` Eric W. Biederman
2020-03-09 18:24 ` Eric W. Biederman
2020-03-09 18:36 ` Eric W. Biederman
2020-03-09 18:47 ` Bernd Edlinger
2020-03-09 19:02 ` Eric W. Biederman
2020-03-09 19:24 ` Bernd Edlinger
2020-03-09 19:35 ` Eric W. Biederman
2020-03-09 19:39 ` Eric W. Biederman
2020-03-10 13:43 ` [PATCH 0/4] Use new infrastructure to fix deadlocks in execve Bernd Edlinger
2020-03-10 15:35 ` Eric W. Biederman
2020-03-10 17:44 ` [PATCH 0/4] Use new infrastructure in more simple cases Bernd Edlinger
2020-03-10 17:45 ` [PATCH 1/4] kernel/kcmp.c: Use new infrastructure to fix deadlocks in execve Bernd Edlinger
2020-03-10 19:01 ` Eric W. Biederman
2020-03-10 19:42 ` Bernd Edlinger
2020-03-10 17:45 ` [PATCH 2/4] proc: " Bernd Edlinger
2020-03-11 18:59 ` Kees Cook
2020-03-11 19:10 ` Kees Cook
2020-03-11 19:38 ` Bernd Edlinger
2020-03-10 17:45 ` [PATCH 3/4] proc: io_accounting: " Bernd Edlinger
2020-03-10 19:06 ` Eric W. Biederman
[not found] ` <AM6PR03MB5170A29F8D885147F1A3981EE4FF0@AM6PR03MB5170.eurprd03.prod.outlook.com>
2020-03-10 21:25 ` Eric W. Biederman
2020-03-11 19:08 ` Kees Cook
2020-03-11 19:48 ` Bernd Edlinger
2020-03-11 19:48 ` Eric W. Biederman
2020-03-10 17:45 ` [PATCH 4/4] perf: " Bernd Edlinger
2020-03-10 13:43 ` [PATCH 1/4] exec: Fix a deadlock in ptrace Bernd Edlinger
2020-03-10 15:13 ` Eric W. Biederman
2020-03-10 15:17 ` Bernd Edlinger
2020-03-10 21:00 ` Kees Cook
2020-03-10 13:44 ` [PATCH 2/4] selftests/ptrace: add test cases for dead-locks Bernd Edlinger
2020-03-10 21:36 ` Kees Cook
2020-03-10 22:41 ` Dmitry V. Levin
2020-03-10 13:44 ` [PATCH 3/4] mm: docs: Fix a comment in process_vm_rw_core Bernd Edlinger
2020-03-11 18:53 ` Kees Cook
2020-03-10 13:44 ` [PATCH 4/4] kernel: doc: remove outdated comment cred.c Bernd Edlinger
2020-03-11 18:54 ` Kees Cook
2020-03-09 19:33 ` [PATCH v2 5/5] exec: Add a exec_update_mutex to replace cred_guard_mutex Dmitry V. Levin
2020-03-09 19:42 ` Eric W. Biederman
2020-03-10 20:55 ` Kees Cook
2020-03-10 21:02 ` Eric W. Biederman
2020-03-10 21:21 ` Jann Horn
2020-03-10 21:30 ` Eric W. Biederman
2020-03-10 23:21 ` Jann Horn
2020-03-11 0:15 ` Eric W. Biederman
2020-03-11 6:33 ` Bernd Edlinger
2020-03-11 13:18 ` Qian Cai
2020-03-12 10:27 ` Kirill Tkhai
2020-03-12 12:24 ` Eric W. Biederman
2020-03-12 13:45 ` Kirill Tkhai
2020-03-12 14:38 ` Eric W. Biederman
2020-03-12 15:23 ` Kirill Tkhai
2020-03-13 1:05 ` Bernd Edlinger
2020-03-13 9:13 ` Kirill Tkhai
2020-03-14 9:11 ` [PATCH v3 " Bernd Edlinger
2020-03-17 8:56 ` Kirill Tkhai
2020-03-17 21:53 ` Bernd Edlinger
2020-03-18 12:22 ` Kirill Tkhai
2020-03-18 20:06 ` Bernd Edlinger
2020-03-19 7:13 ` Kirill Tkhai
2020-03-19 7:19 ` Bernd Edlinger
2020-03-19 9:11 ` [PATCH v4 3/5] " Bernd Edlinger
2020-03-19 9:13 ` Bernd Edlinger
2020-03-19 9:19 ` Greg Kroah-Hartman
2020-03-19 9:20 ` Bernd Edlinger
2020-03-21 22:53 ` Bernd Edlinger
2020-03-14 9:12 ` [PATCH 0/2] exec: Fix dead-lock in de_thread with ptrace_attach Bernd Edlinger
2020-03-14 9:12 ` [PATCH 1/2] " Bernd Edlinger
2020-03-14 9:13 ` [PATCH 2/2] doc: Update documentation of ->exec_*_mutex Bernd Edlinger
2020-03-14 9:57 ` [PATCH v2 5/5] exec: Add a exec_update_mutex to replace cred_guard_mutex Bernd Edlinger
2020-03-14 10:02 ` Bernd Edlinger
2020-03-17 8:58 ` Kirill Tkhai
2020-03-09 13:58 ` [PATCH 0/5] Infrastructure to allow fixing exec deadlocks Bernd Edlinger
2020-03-03 16:50 ` [PATCHv5] exec: Fix a deadlock in ptrace Christian Brauner
2020-03-02 12:28 ` [PATCHv2] " Oleg Nesterov
2020-03-02 15:56 ` Bernd Edlinger
2020-03-02 7:47 ` [PATCH] " Christian Brauner
2020-03-02 7:48 ` Christian Brauner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=AM6PR03MB51701E2F405F6DBF1EEB57CDE4E50@AM6PR03MB5170.eurprd03.prod.outlook.com \
--to=bernd.edlinger@hotmail.de \
--cc=aarcange@redhat.com \
--cc=adobriyan@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=anshuman.khandual@arm.com \
--cc=avagin@gmail.com \
--cc=bigeasy@linutronix.de \
--cc=christian.brauner@ubuntu.com \
--cc=christian@kellner.me \
--cc=corbet@lwn.net \
--cc=cyphar@cyphar.com \
--cc=david@redhat.com \
--cc=dhowells@redhat.com \
--cc=duyuyang@gmail.com \
--cc=ebiederm@xmission.com \
--cc=frederic@kernel.org \
--cc=gregkh@linuxfoundation.org \
--cc=jamorris@linux.microsoft.com \
--cc=jannh@google.com \
--cc=jgg@ziepe.ca \
--cc=keescook@chromium.org \
--cc=ldv@altlinux.org \
--cc=linux-api@vger.kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mingo@kernel.org \
--cc=oleg@redhat.com \
--cc=peterz@infradead.org \
--cc=shakeelb@google.com \
--cc=stable@vger.kernel.org \
--cc=tglx@linutronix.de \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).