From: Tom Talpey <tom@talpey.com>
To: Jerome Glisse <jglisse@redhat.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>,
Dan Williams <dan.j.williams@intel.com>, Jan Kara <jack@suse.cz>,
John Hubbard <jhubbard@nvidia.com>,
Matthew Wilcox <willy@infradead.org>,
John Hubbard <john.hubbard@gmail.com>,
Andrew Morton <akpm@linux-foundation.org>,
Linux MM <linux-mm@kvack.org>, Al Viro <viro@zeniv.linux.org.uk>,
benve@cisco.com, Christoph Hellwig <hch@infradead.org>,
Christopher Lameter <cl@linux.com>,
"Dalessandro, Dennis" <dennis.dalessandro@intel.com>,
Doug Ledford <dledford@redhat.com>,
Michal Hocko <mhocko@kernel.org>,
Mike Marciniszyn <mike.marciniszyn@intel.com>,
rcampbell@nvidia.com,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
linux-fsdevel <linux-fsdevel@vger.kernel.org>,
"Weiny, Ira" <ira.weiny@intel.com>
Subject: Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions
Date: Thu, 13 Dec 2018 13:12:06 -0500 [thread overview]
Message-ID: <a6c25182-9691-d9f9-b3b5-a42b2b5d6f7f@talpey.com> (raw)
In-Reply-To: <20181213151839.GC3186@redhat.com>
On 12/13/2018 10:18 AM, Jerome Glisse wrote:
> On Thu, Dec 13, 2018 at 09:51:18AM -0500, Tom Talpey wrote:
>> On 12/13/2018 9:18 AM, Jerome Glisse wrote:
>>> On Thu, Dec 13, 2018 at 08:40:49AM -0500, Tom Talpey wrote:
>>>> On 12/13/2018 7:43 AM, Jerome Glisse wrote:
>>>>> On Wed, Dec 12, 2018 at 08:20:43PM -0700, Jason Gunthorpe wrote:
>>>>>> On Wed, Dec 12, 2018 at 07:01:09PM -0500, Jerome Glisse wrote:
>>>>>>> On Wed, Dec 12, 2018 at 04:37:03PM -0700, Jason Gunthorpe wrote:
>>>>>>>> On Wed, Dec 12, 2018 at 04:53:49PM -0500, Jerome Glisse wrote:
>>>>>>>>>> Almost, we need some safety around assuming that DMA is complete the
>>>>>>>>>> page, so the notification would need to go all to way to userspace
>>>>>>>>>> with something like a file lease notification. It would also need to
>>>>>>>>>> be backstopped by an IOMMU in the case where the hardware does not /
>>>>>>>>>> can not stop in-flight DMA.
>>>>>>>>>
>>>>>>>>> You can always reprogram the hardware right away it will redirect
>>>>>>>>> any dma to the crappy page.
>>>>>>>>
>>>>>>>> That causes silent data corruption for RDMA users - we can't do that.
>>>>>>>>
>>>>>>>> The only way out for current hardware is to forcibly terminate the
>>>>>>>> RDMA activity somehow (and I'm not even sure this is possible, at
>>>>>>>> least it would be driver specific)
>>>>>>>>
>>>>>>>> Even the IOMMU idea probably doesn't work, I doubt all current
>>>>>>>> hardware can handle a PCI-E error TLP properly.
>>>>>>>
>>>>>>> What i saying is reprogram hardware to crappy page ie valid page
>>>>>>> dma map but that just has random content as a last resort to allow
>>>>>>> filesystem to reuse block. So their should be no PCIE error unless
>>>>>>> hardware freak out to see its page table reprogram randomly.
>>>>>>
>>>>>> No, that isn't an option. You can't silently provide corrupted data
>>>>>> for RDMA to transfer out onto the network, or silently discard data
>>>>>> coming in!!
>>>>>>
>>>>>> Think of the consequences of that - I have a fileserver process and
>>>>>> someone does ftruncate and now my clients receive corrupted data??
>>>>>
>>>>> This is what happens _today_ ie today someone do GUP on page file
>>>>> and then someone else do truncate the first GUP is effectively
>>>>> streaming _random_ data to network as the page does not correspond
>>>>> to anything anymore and once the RDMA MR goes aways and release
>>>>> the page the page content will be lost. So i am not changing anything
>>>>> here, what i proposed was to make it explicit to device driver at
>>>>> least that they were streaming random data. Right now this is all
>>>>> silent but this is what is happening wether you like it or not :)
>>>>>
>>>>> Note that i am saying do that only for truncate to allow to be
>>>>> nice to fs. But again i am fine with whatever solution but you can
>>>>> not please everyone here. Either block truncate and fs folks will
>>>>> hate you or make it clear to device driver that you are streaming
>>>>> random things and RDMA people hates you.
>>>>>
>>>>>
>>>>>> The only option is to prevent the RDMA transfer from ever happening,
>>>>>> and we just don't have hardware support (beyond destroy everything) to
>>>>>> do that.
>>>>>>
>>>>>>> The question is who do you want to punish ? RDMA user that pin stuff
>>>>>>> and expect thing to work forever without worrying for other fs
>>>>>>> activities ? Or filesystem to pin block forever :)
>>>>>>
>>>>>> I don't want to punish everyone, I want both sides to have complete
>>>>>> data integrity as the USER has deliberately decided to combine DAX and
>>>>>> RDMA. So either stop it at the front end (ie get_user_pages_longterm)
>>>>>> or make it work in a way that guarantees integrity for both.
>>>>>>
>>>>>>> S2: notify userspace program through device/sub-system
>>>>>>> specific API and delay ftruncate. After a while if there
>>>>>>> is no answer just be mean and force hardware to use
>>>>>>> crappy page as anyway this is what happens today
>>>>>>
>>>>>> I don't think this happens today (outside of DAX).. Does it?
>>>>>
>>>>> It does it is just silent, i don't remember anything in the code
>>>>> that would stop a truncate to happen because of elevated refcount.
>>>>> This does not happen with ODP mlx5 as it does abide by _all_ mmu
>>>>> notifier. This is for anything that does ODP without support for
>>>>> mmu notifier.
>>>>
>>>> Wait - is it expected that the MMU notifier upcall is handled
>>>> synchronously? That is, the page DMA mapping must be torn down
>>>> immediately, and before returning?
>>>
>>> Yes you must torn down mapping before returning from mmu notifier
>>> call back. Any time after is too late. You obviously need hardware
>>> that can support that. In the infiniband sub-system AFAIK only the
>>> mlx5 hardware can do that. In the GPU sub-system everyone is fine.
>>
>> I'm skeptical that MLX5 can actually make this guarantee. But we
>> can take that offline in linux-rdma.
>
> It does unless the code lies about what the hardware do :) See umem_odp.c
> in core and odp.c in mlx5 directories.
Ok, I did look and there are numerous error returns from these calls.
Some are related to resource shortages (including the rather ominous-
sounding "emergency_pages" in odp.c), others related to the generic
RDMA behaviors such as posting work requests and reaping their
completion status.
So I'd ask - what is the backup plan from the mmu notifier if the
unmap fails? Which it certainly will, in many real-world situations.
Tom.
>> I'm also skeptical that NVMe can do this.
>>
>>> Dunno about other sub-systems.
>>>
>>>
>>>> That's simply not possible, since the hardware needs to get control
>>>> to do this. Even if there were an IOMMU that could intercept the
>>>> DMA, reprogramming it will require a flush, which cannot be guaranteed
>>>> to occur "inline".
>>>
>>> If hardware can not do that then hardware should not use GUP, at
>>> least not on file back page. I advocated in favor of forbiding GUP
>>> for device that can not do that as right now this silently breaks
>>> in few cases (truncate, mremap, splice, reflink, ...). So the device
>>> in those cases can end up with GUPed pages that do not correspond
>>> to anything anymore ie they do not correspond to the memory backing
>>> the virtual address they were GUP against, nor they correspond to
>>> the file content at the given offset anymore. It is just random
>>> data as far as the kernel or filesystem is concern.
>>>
>>> Of course for this to happen you need an application that do stupid
>>> thing like create an MR in one thread on the mmap of a file and
>>> truncate that same file in another thread (or from the same thread).
>>>
>>> So this is unlikely to happen in sane program. It does not mean it
>>> will not happen.
>>
>> Completely agree. In other words, this is the responsibility of the
>> DAX (or g-u-p) consumer, which is nt necessarily the program itself,
>> it could be an upper layer.
>>
>> In SMB3 and NFSv4, which I've been focused on, we envision using the
>> existing protocol leases to protect this. When requesting a DAX mapping,
>> the server may requires an exclusive lease. If this mapping needs to
>> change, because of another conflicting mapping, the lease would be
>> recalled and the mapping dropped. This is a normal and well-established
>> filesystem requirement.
>>
>> The twist here is that the platform itself can initiate such an event.
>> It's my belief that this plumbing must flow to the *top* of the stack,
>> i.e. the entity that took the mapping (e.g. filesystem), and not
>> depend on the MMU notifier at the very bottom.
>
> So this patchset is about mm plumbings, what fs does before mm code
> gets call is a fs discussion. Note also that GUP is always against
> a virtual address of a process. GUP is use by few driver to allow
> the device direct access to some portion of process address space.
>
> The issues is that some of those device have designed an API that
> fully ignore things like munmap, splice, truncate, ... and as such
> they open the door for undefined behavior. I believe the original
> intention in all the cases was that the user would not do stupid
> thing like setup the device mapping through device specific API and
> then munmap or truncate or anything that would affect the range of
> virtual address.
>
> Thing is from kernel point of view we should not and can not assume
> that userspace will behave properly. So we have to brace for the
> worst. Which is what this patchset is trying to do ie fix some of
> issues and make the rest of them explicit to device driver so that
> they can decide what to do about it.
>
> My advice is for each of those sub-system/device to fail loudly
> when such thing happens so that the user knows he is doing something
> stupid or illegal.
>
>
>>> The second set of issue at to deals with set_page_dirty happening
>>> long time after page_release did happens and thus the fs dirty
>>> page callback will see page in bad state and will BUG() and you
>>> will have an oops and loose any data your device might have written
>>> to the page. This is highly filesystem dependend and also timing
>>> dependend and link to thing like memory pressure so it might not
>>> happen that often but again it can happen.
>>>
>>>
>>>>>> .. and the remedy here is to kill the process, not provide corrupt
>>>>>> data. Kill the process is likely to not go over well with any real
>>>>>> users that want this combination.
>>>>>>
>>>>>> Think Samba serving files over RDMA - you can't have random unpriv
>>>>>> users calling ftruncate and causing smbd to be killed or serve corrupt
>>>>>> data.
>>>>>
>>>>> So what i am saying is there is a choice and it would be better to
>>>>> decide something than let the existing status quo where we just keep
>>>>> streaming random data after truncate to a GUPed page.
>>>>
>>>> Let's also remember that any torn-down DMA mapping can't be recycled
>>>> until all uses of the old DMA addresses are destroyed. The whole
>>>> thing screams for reference counting all the way down, to me.
>>>
>>> I am not saying reuse the DMA address in the emergency_mean_callback
>>> the idea was:
>>>
>>> gup_page_emergency_revoke(device, page)
>>> {
>>> crapy_page = alloc_page();
>>> dma_addr = dma_map(crappy_page, device, ...);
>>> mydevice_page_table_update(device, crappy_page, dma_addr);
>>> mydevice_tlb_flush(device);
>>> mydevice_wait_pending_dma(device)
>>>
>>> // at this point the original GUPed page is not access by hw
>>>
>>> dma_unmap(page);
>>> put_user_page(page);
>>> }
>>
>> Ok, but my concern was also that the old DMA address then becomes
>> unused and may be grabbed by a new i/o. If the hardware still has
>> reads or writes in flight, and they arrive after the old address
>> becomes valid, well, oops.
>
> In above code at the comment point the driver garanty the hardware
> will never use the old dma address any more ie that all in flight
> dma are done. I know this can be done for some devices. It might
> very well not work for all and this is why this need to be a sub-
> system/device discussion.
>
> Cheers,
> Jérôme
>
>
next prev parent reply other threads:[~2018-12-13 18:12 UTC|newest]
Thread overview: 213+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-04 0:17 [PATCH 0/2] put_user_page*(): start converting the call sites john.hubbard
2018-12-04 0:17 ` [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions john.hubbard
2018-12-04 7:53 ` Mike Rapoport
2018-12-05 1:40 ` John Hubbard
2018-12-04 20:28 ` Dan Williams
2018-12-04 21:56 ` John Hubbard
2018-12-04 23:03 ` Dan Williams
2018-12-05 0:36 ` Jerome Glisse
2018-12-05 0:40 ` Dan Williams
2018-12-05 0:59 ` John Hubbard
2018-12-05 0:58 ` John Hubbard
2018-12-05 1:00 ` Dan Williams
2018-12-05 1:15 ` Matthew Wilcox
2018-12-05 1:44 ` Jerome Glisse
2018-12-05 1:57 ` John Hubbard
2018-12-07 2:45 ` John Hubbard
2018-12-07 19:16 ` Jerome Glisse
2018-12-07 19:26 ` Dan Williams
2018-12-07 19:40 ` Jerome Glisse
2018-12-08 0:52 ` John Hubbard
2018-12-08 2:24 ` Jerome Glisse
2018-12-10 10:28 ` Jan Kara
2018-12-12 15:03 ` Jerome Glisse
2018-12-12 16:27 ` Dan Williams
2018-12-12 17:02 ` Jerome Glisse
2018-12-12 17:49 ` Dan Williams
2018-12-12 19:07 ` John Hubbard
2018-12-12 21:30 ` Jerome Glisse
2018-12-12 21:40 ` Dan Williams
2018-12-12 21:53 ` Jerome Glisse
2018-12-12 22:11 ` Matthew Wilcox
2018-12-12 22:16 ` Jerome Glisse
2018-12-12 23:37 ` Jason Gunthorpe
2018-12-12 23:46 ` John Hubbard
2018-12-12 23:54 ` Dan Williams
2018-12-13 0:01 ` Jerome Glisse
2018-12-13 0:18 ` Dan Williams
2018-12-13 0:44 ` Jerome Glisse
2018-12-13 3:26 ` Jason Gunthorpe
2018-12-13 3:20 ` Jason Gunthorpe
2018-12-13 12:43 ` Jerome Glisse
2018-12-13 13:40 ` Tom Talpey
2018-12-13 14:18 ` Jerome Glisse
2018-12-13 14:51 ` Tom Talpey
2018-12-13 15:18 ` Jerome Glisse
2018-12-13 18:12 ` Tom Talpey [this message]
2018-12-13 19:18 ` Jerome Glisse
2018-12-14 10:41 ` Jan Kara
2018-12-14 15:25 ` Jerome Glisse
2018-12-12 21:56 ` John Hubbard
2018-12-12 22:04 ` Jerome Glisse
2018-12-12 22:11 ` John Hubbard
2018-12-12 22:14 ` Jerome Glisse
2018-12-12 22:17 ` John Hubbard
2018-12-12 21:46 ` Dave Chinner
2018-12-12 21:59 ` Jerome Glisse
2018-12-13 0:51 ` Dave Chinner
2018-12-13 2:02 ` Jerome Glisse
2018-12-13 15:56 ` Christopher Lameter
2018-12-13 16:02 ` Jerome Glisse
2018-12-14 6:00 ` Dave Chinner
2018-12-14 15:13 ` Jerome Glisse
2018-12-14 3:52 ` John Hubbard
2018-12-14 5:21 ` Dan Williams
2018-12-14 6:11 ` John Hubbard
2018-12-14 15:20 ` Jerome Glisse
2018-12-14 19:38 ` Dan Williams
2018-12-14 19:48 ` Matthew Wilcox
2018-12-14 19:53 ` Dave Hansen
2018-12-14 20:03 ` Matthew Wilcox
2018-12-14 20:17 ` Dan Williams
2018-12-14 20:29 ` Matthew Wilcox
2018-12-15 0:41 ` John Hubbard
2018-12-17 8:56 ` Jan Kara
2018-12-17 18:28 ` Dan Williams
2018-12-14 15:43 ` Jan Kara
2018-12-16 21:58 ` Dave Chinner
2018-12-17 18:11 ` Jerome Glisse
2018-12-17 18:34 ` Matthew Wilcox
2018-12-17 19:48 ` Jerome Glisse
2018-12-17 19:51 ` Matthew Wilcox
2018-12-17 19:54 ` Jerome Glisse
2018-12-17 19:59 ` Matthew Wilcox
2018-12-17 20:55 ` Jerome Glisse
2018-12-17 21:03 ` Matthew Wilcox
2018-12-17 21:15 ` Jerome Glisse
2018-12-18 1:09 ` Dave Chinner
2018-12-18 6:12 ` Darrick J. Wong
2018-12-18 9:30 ` Jan Kara
2018-12-18 23:29 ` John Hubbard
2018-12-19 2:07 ` Jerome Glisse
2018-12-19 11:08 ` Jan Kara
2018-12-20 10:54 ` John Hubbard
2018-12-20 16:50 ` Jerome Glisse
2018-12-20 16:57 ` Dan Williams
2018-12-20 16:49 ` Jerome Glisse
2019-01-03 1:55 ` Jerome Glisse
2019-01-03 3:27 ` John Hubbard
2019-01-03 14:57 ` Jerome Glisse
2019-01-03 9:26 ` Jan Kara
2019-01-03 14:44 ` Jerome Glisse
2019-01-11 2:59 ` John Hubbard
2019-01-11 2:59 ` John Hubbard
2019-01-11 16:51 ` Jerome Glisse
2019-01-11 16:51 ` Jerome Glisse
2019-01-12 1:04 ` John Hubbard
2019-01-12 1:04 ` John Hubbard
2019-01-12 2:02 ` Jerome Glisse
2019-01-12 2:02 ` Jerome Glisse
2019-01-12 2:38 ` John Hubbard
2019-01-12 2:38 ` John Hubbard
2019-01-12 2:46 ` Jerome Glisse
2019-01-12 2:46 ` Jerome Glisse
2019-01-12 3:06 ` John Hubbard
2019-01-12 3:06 ` John Hubbard
2019-01-12 3:25 ` Jerome Glisse
2019-01-12 3:25 ` Jerome Glisse
2019-01-12 20:46 ` John Hubbard
2019-01-12 20:46 ` John Hubbard
2019-01-14 14:54 ` Jan Kara
2019-01-14 14:54 ` Jan Kara
2019-01-14 17:21 ` Jerome Glisse
2019-01-14 17:21 ` Jerome Glisse
2019-01-14 19:09 ` John Hubbard
2019-01-14 19:09 ` John Hubbard
2019-01-15 8:34 ` Jan Kara
2019-01-15 8:34 ` Jan Kara
2019-01-15 21:39 ` John Hubbard
2019-01-15 21:39 ` John Hubbard
2019-01-15 8:07 ` Jan Kara
2019-01-15 8:07 ` Jan Kara
2019-01-15 17:15 ` Jerome Glisse
2019-01-15 17:15 ` Jerome Glisse
2019-01-15 21:56 ` John Hubbard
2019-01-15 21:56 ` John Hubbard
2019-01-15 22:12 ` Jerome Glisse
2019-01-15 22:12 ` Jerome Glisse
2019-01-16 0:44 ` John Hubbard
2019-01-16 0:44 ` John Hubbard
2019-01-16 1:56 ` Jerome Glisse
2019-01-16 1:56 ` Jerome Glisse
2019-01-16 2:01 ` Dan Williams
2019-01-16 2:01 ` Dan Williams
2019-01-16 2:23 ` Jerome Glisse
2019-01-16 2:23 ` Jerome Glisse
2019-01-16 4:34 ` Dave Chinner
2019-01-16 4:34 ` Dave Chinner
2019-01-16 14:50 ` Jerome Glisse
2019-01-16 14:50 ` Jerome Glisse
2019-01-16 22:51 ` Dave Chinner
2019-01-16 22:51 ` Dave Chinner
2019-01-16 11:38 ` Jan Kara
2019-01-16 11:38 ` Jan Kara
2019-01-16 13:08 ` Jerome Glisse
2019-01-16 13:08 ` Jerome Glisse
2019-01-17 5:42 ` John Hubbard
2019-01-17 5:42 ` John Hubbard
2019-01-17 15:21 ` Jerome Glisse
2019-01-17 15:21 ` Jerome Glisse
2019-01-18 0:16 ` Dave Chinner
2019-01-18 1:59 ` Jerome Glisse
2019-01-17 9:30 ` Jan Kara
2019-01-17 9:30 ` Jan Kara
2019-01-17 15:17 ` Jerome Glisse
2019-01-17 15:17 ` Jerome Glisse
2019-01-22 15:24 ` Jan Kara
2019-01-22 16:46 ` Jerome Glisse
2019-01-23 18:02 ` Jan Kara
2019-01-23 19:04 ` Jerome Glisse
2019-01-29 0:22 ` John Hubbard
2019-01-29 1:23 ` Jerome Glisse
2019-01-29 6:41 ` John Hubbard
2019-01-29 10:12 ` Jan Kara
2019-01-30 2:21 ` John Hubbard
2019-01-17 5:25 ` John Hubbard
2019-01-17 5:25 ` John Hubbard
2019-01-17 9:04 ` Jan Kara
2019-01-17 9:04 ` Jan Kara
2019-01-12 3:14 ` Jerome Glisse
2019-01-12 3:14 ` Jerome Glisse
2018-12-18 10:33 ` Jan Kara
2018-12-18 23:42 ` Dave Chinner
2018-12-19 3:03 ` Jason Gunthorpe
2018-12-19 5:26 ` Dan Williams
2018-12-19 11:19 ` Jan Kara
2018-12-19 10:28 ` Dave Chinner
2018-12-19 11:35 ` Jan Kara
2018-12-19 16:56 ` Jason Gunthorpe
2018-12-19 22:33 ` Dave Chinner
2018-12-20 9:07 ` Jan Kara
2018-12-20 16:54 ` Jerome Glisse
2018-12-19 13:24 ` Jan Kara
2018-12-08 5:18 ` Matthew Wilcox
2018-12-12 19:13 ` John Hubbard
2018-12-08 7:16 ` Dan Williams
2018-12-08 16:33 ` Jerome Glisse
2018-12-08 16:48 ` Christoph Hellwig
2018-12-08 17:47 ` Jerome Glisse
2018-12-08 18:26 ` Christoph Hellwig
2018-12-08 18:45 ` Jerome Glisse
2018-12-08 18:09 ` Dan Williams
2018-12-08 18:12 ` Christoph Hellwig
2018-12-11 6:18 ` Dave Chinner
2018-12-05 5:52 ` Dan Williams
2018-12-05 11:16 ` Jan Kara
2018-12-04 0:17 ` [PATCH 2/2] infiniband/mm: convert put_page() to put_user_page*() john.hubbard
2018-12-04 17:10 ` [PATCH 0/2] put_user_page*(): start converting the call sites David Laight
2018-12-05 1:05 ` John Hubbard
2018-12-05 14:08 ` David Laight
2018-12-28 8:37 ` Pavel Machek
2019-02-08 7:56 [PATCH 0/2] mm: put_user_page() call site conversion first john.hubbard
2019-02-08 7:56 ` [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions john.hubbard
2019-02-08 10:32 ` Mike Rapoport
2019-02-08 20:44 ` John Hubbard
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a6c25182-9691-d9f9-b3b5-a42b2b5d6f7f@talpey.com \
--to=tom@talpey.com \
--cc=akpm@linux-foundation.org \
--cc=benve@cisco.com \
--cc=cl@linux.com \
--cc=dan.j.williams@intel.com \
--cc=dennis.dalessandro@intel.com \
--cc=dledford@redhat.com \
--cc=hch@infradead.org \
--cc=ira.weiny@intel.com \
--cc=jack@suse.cz \
--cc=jgg@ziepe.ca \
--cc=jglisse@redhat.com \
--cc=jhubbard@nvidia.com \
--cc=john.hubbard@gmail.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@kernel.org \
--cc=mike.marciniszyn@intel.com \
--cc=rcampbell@nvidia.com \
--cc=viro@zeniv.linux.org.uk \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).