linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Tom Talpey <tom@talpey.com>
To: Jerome Glisse <jglisse@redhat.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>,
	Dan Williams <dan.j.williams@intel.com>, Jan Kara <jack@suse.cz>,
	John Hubbard <jhubbard@nvidia.com>,
	Matthew Wilcox <willy@infradead.org>,
	John Hubbard <john.hubbard@gmail.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Linux MM <linux-mm@kvack.org>, Al Viro <viro@zeniv.linux.org.uk>,
	benve@cisco.com, Christoph Hellwig <hch@infradead.org>,
	Christopher Lameter <cl@linux.com>,
	"Dalessandro, Dennis" <dennis.dalessandro@intel.com>,
	Doug Ledford <dledford@redhat.com>,
	Michal Hocko <mhocko@kernel.org>,
	Mike Marciniszyn <mike.marciniszyn@intel.com>,
	rcampbell@nvidia.com,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	"Weiny, Ira" <ira.weiny@intel.com>
Subject: Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions
Date: Thu, 13 Dec 2018 09:51:18 -0500	[thread overview]
Message-ID: <0b75a9a6-3907-ea88-6352-256bc2954f4a@talpey.com> (raw)
In-Reply-To: <20181213141838.GB3186@redhat.com>

On 12/13/2018 9:18 AM, Jerome Glisse wrote:
> On Thu, Dec 13, 2018 at 08:40:49AM -0500, Tom Talpey wrote:
>> On 12/13/2018 7:43 AM, Jerome Glisse wrote:
>>> On Wed, Dec 12, 2018 at 08:20:43PM -0700, Jason Gunthorpe wrote:
>>>> On Wed, Dec 12, 2018 at 07:01:09PM -0500, Jerome Glisse wrote:
>>>>> On Wed, Dec 12, 2018 at 04:37:03PM -0700, Jason Gunthorpe wrote:
>>>>>> On Wed, Dec 12, 2018 at 04:53:49PM -0500, Jerome Glisse wrote:
>>>>>>>> Almost, we need some safety around assuming that DMA is complete the
>>>>>>>> page, so the notification would need to go all to way to userspace
>>>>>>>> with something like a file lease notification. It would also need to
>>>>>>>> be backstopped by an IOMMU in the case where the hardware does not /
>>>>>>>> can not stop in-flight DMA.
>>>>>>>
>>>>>>> You can always reprogram the hardware right away it will redirect
>>>>>>> any dma to the crappy page.
>>>>>>
>>>>>> That causes silent data corruption for RDMA users - we can't do that.
>>>>>>
>>>>>> The only way out for current hardware is to forcibly terminate the
>>>>>> RDMA activity somehow (and I'm not even sure this is possible, at
>>>>>> least it would be driver specific)
>>>>>>
>>>>>> Even the IOMMU idea probably doesn't work, I doubt all current
>>>>>> hardware can handle a PCI-E error TLP properly.
>>>>>
>>>>> What i saying is reprogram hardware to crappy page ie valid page
>>>>> dma map but that just has random content as a last resort to allow
>>>>> filesystem to reuse block. So their should be no PCIE error unless
>>>>> hardware freak out to see its page table reprogram randomly.
>>>>
>>>> No, that isn't an option. You can't silently provide corrupted data
>>>> for RDMA to transfer out onto the network, or silently discard data
>>>> coming in!!
>>>>
>>>> Think of the consequences of that - I have a fileserver process and
>>>> someone does ftruncate and now my clients receive corrupted data??
>>>
>>> This is what happens _today_ ie today someone do GUP on page file
>>> and then someone else do truncate the first GUP is effectively
>>> streaming _random_ data to network as the page does not correspond
>>> to anything anymore and once the RDMA MR goes aways and release
>>> the page the page content will be lost. So i am not changing anything
>>> here, what i proposed was to make it explicit to device driver at
>>> least that they were streaming random data. Right now this is all
>>> silent but this is what is happening wether you like it or not :)
>>>
>>> Note that  i am saying do that only for truncate to allow to be
>>> nice to fs. But again i am fine with whatever solution but you can
>>> not please everyone here. Either block truncate and fs folks will
>>> hate you or make it clear to device driver that you are streaming
>>> random things and RDMA people hates you.
>>>
>>>
>>>> The only option is to prevent the RDMA transfer from ever happening,
>>>> and we just don't have hardware support (beyond destroy everything) to
>>>> do that.
>>>>
>>>>> The question is who do you want to punish ? RDMA user that pin stuff
>>>>> and expect thing to work forever without worrying for other fs
>>>>> activities ? Or filesystem to pin block forever :)
>>>>
>>>> I don't want to punish everyone, I want both sides to have complete
>>>> data integrity as the USER has deliberately decided to combine DAX and
>>>> RDMA. So either stop it at the front end (ie get_user_pages_longterm)
>>>> or make it work in a way that guarantees integrity for both.
>>>>
>>>>>       S2: notify userspace program through device/sub-system
>>>>>           specific API and delay ftruncate. After a while if there
>>>>>           is no answer just be mean and force hardware to use
>>>>>           crappy page as anyway this is what happens today
>>>>
>>>> I don't think this happens today (outside of DAX).. Does it?
>>>
>>> It does it is just silent, i don't remember anything in the code
>>> that would stop a truncate to happen because of elevated refcount.
>>> This does not happen with ODP mlx5 as it does abide by _all_ mmu
>>> notifier. This is for anything that does ODP without support for
>>> mmu notifier.
>>
>> Wait - is it expected that the MMU notifier upcall is handled
>> synchronously? That is, the page DMA mapping must be torn down
>> immediately, and before returning?
> 
> Yes you must torn down mapping before returning from mmu notifier
> call back. Any time after is too late. You obviously need hardware
> that can support that. In the infiniband sub-system AFAIK only the
> mlx5 hardware can do that. In the GPU sub-system everyone is fine.

I'm skeptical that MLX5 can actually make this guarantee. But we
can take that offline in linux-rdma.

I'm also skeptical that NVMe can do this.

> Dunno about other sub-systems.
> 
> 
>> That's simply not possible, since the hardware needs to get control
>> to do this. Even if there were an IOMMU that could intercept the
>> DMA, reprogramming it will require a flush, which cannot be guaranteed
>> to occur "inline".
> 
> If hardware can not do that then hardware should not use GUP, at
> least not on file back page. I advocated in favor of forbiding GUP
> for device that can not do that as right now this silently breaks
> in few cases (truncate, mremap, splice, reflink, ...). So the device
> in those cases can end up with GUPed pages that do not correspond
> to anything anymore ie they do not correspond to the memory backing
> the virtual address they were GUP against, nor they correspond to
> the file content at the given offset anymore. It is just random
> data as far as the kernel or filesystem is concern.
> 
> Of course for this to happen you need an application that do stupid
> thing like create an MR in one thread on the mmap of a file and
> truncate that same file in another thread (or from the same thread).
> 
> So this is unlikely to happen in sane program. It does not mean it
> will not happen.

Completely agree. In other words, this is the responsibility of the
DAX (or g-u-p) consumer, which is nt necessarily the program itself,
it could be an upper layer.

In SMB3 and NFSv4, which I've been focused on, we envision using the
existing protocol leases to protect this. When requesting a DAX mapping,
the server may requires an exclusive lease. If this mapping needs to
change, because of another conflicting mapping, the lease would be
recalled and the mapping dropped. This is a normal and well-established
filesystem requirement.

The twist here is that the platform itself can initiate such an event.
It's my belief that this plumbing must flow to the *top* of the stack,
i.e. the entity that took the mapping (e.g. filesystem), and not
depend on the MMU notifier at the very bottom.

> The second set of issue at to deals with set_page_dirty happening
> long time after page_release did happens and thus the fs dirty
> page callback will see page in bad state and will BUG() and you
> will have an oops and loose any data your device might have written
> to the page. This is highly filesystem dependend and also timing
> dependend and link to thing like memory pressure so it might not
> happen that often but again it can happen.
> 
> 
>>>> .. and the remedy here is to kill the process, not provide corrupt
>>>> data. Kill the process is likely to not go over well with any real
>>>> users that want this combination.
>>>>
>>>> Think Samba serving files over RDMA - you can't have random unpriv
>>>> users calling ftruncate and causing smbd to be killed or serve corrupt
>>>> data.
>>>
>>> So what i am saying is there is a choice and it would be better to
>>> decide something than let the existing status quo where we just keep
>>> streaming random data after truncate to a GUPed page.
>>
>> Let's also remember that any torn-down DMA mapping can't be recycled
>> until all uses of the old DMA addresses are destroyed. The whole
>> thing screams for reference counting all the way down, to me.
> 
> I am not saying reuse the DMA address in the emergency_mean_callback
> the idea was:
> 
>      gup_page_emergency_revoke(device, page)
>      {
>          crapy_page = alloc_page();
>          dma_addr = dma_map(crappy_page, device, ...);
>          mydevice_page_table_update(device, crappy_page, dma_addr);
>          mydevice_tlb_flush(device);
>          mydevice_wait_pending_dma(device)
> 
>          // at this point the original GUPed page is not access by hw
> 
>          dma_unmap(page);
>          put_user_page(page);
>      }

Ok, but my concern was also that the old DMA address then becomes
unused and may be grabbed by a new i/o. If the hardware still has
reads or writes in flight, and they arrive after the old address
becomes valid, well, oops.

Tom.

> I know that it is something we can do with GPU devices. So i assumed
> that other devices can do that to. But i understand this is highly
> device dependent. Not that if you have a command queue it is more like:
> 
>      gup_page_emergency_revoke(device, page)
>      {
>          crapy_page = alloc_page();
>          dma_addr = dma_map(crappy_page, device, ...);
> 
>          // below function update kernel side data structure that stores
>          // the pointer to the GUPed page and that are use to build cmds
>          // send to the hardware. It does not update the hardware, just
>          // the device driver internal data structure.
>          mydevice_replace_page_in_object(device, page, crappy_page, dma_addr);
> 
>          mydevice_queue_wait_pending_job(device);
> 
>          // at this point the original GUPed page is not access by hw and
>          // any new command will be using the crappy page not the GUPed
>          // page
> 
>          dma_unmap(page);
>          put_user_page(page);
>      }
> 
> Again if device can not do any of the above then it should really not
> be using GUP because they are corner case that are simply not solvable.
> We can avoid kernel OOPS but we can not pin the page as GUP user believe
> ie the virtual address the GUP happened against can point to a different
> page (true for both anonymous memory and file back memory).
> 
> 
> The put_user_page() patchset is about solving the OOPS and BUG() and
> also fixing the tiny race that exist for direct I/O. Fixing other user
> of GUP should happen sub-system by sub-system and each sub-system or
> device driver maintainer must choose their poison. This is what i am
> advocating for. If the emergency_revoke above is something that would
> work for device is something i can't say for certain, only for devices
> i know (which are GPU mostly).
> 
> Cheers,
> Jérôme
> 
> 

  reply	other threads:[~2018-12-13 14:51 UTC|newest]

Thread overview: 213+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-12-04  0:17 [PATCH 0/2] put_user_page*(): start converting the call sites john.hubbard
2018-12-04  0:17 ` [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions john.hubbard
2018-12-04  7:53   ` Mike Rapoport
2018-12-05  1:40     ` John Hubbard
2018-12-04 20:28   ` Dan Williams
2018-12-04 21:56     ` John Hubbard
2018-12-04 23:03       ` Dan Williams
2018-12-05  0:36         ` Jerome Glisse
2018-12-05  0:40           ` Dan Williams
2018-12-05  0:59             ` John Hubbard
2018-12-05  0:58         ` John Hubbard
2018-12-05  1:00           ` Dan Williams
2018-12-05  1:15           ` Matthew Wilcox
2018-12-05  1:44             ` Jerome Glisse
2018-12-05  1:57               ` John Hubbard
2018-12-07  2:45                 ` John Hubbard
2018-12-07 19:16                   ` Jerome Glisse
2018-12-07 19:26                     ` Dan Williams
2018-12-07 19:40                       ` Jerome Glisse
2018-12-08  0:52                     ` John Hubbard
2018-12-08  2:24                       ` Jerome Glisse
2018-12-10 10:28                         ` Jan Kara
2018-12-12 15:03                           ` Jerome Glisse
2018-12-12 16:27                             ` Dan Williams
2018-12-12 17:02                               ` Jerome Glisse
2018-12-12 17:49                                 ` Dan Williams
2018-12-12 19:07                                   ` John Hubbard
2018-12-12 21:30                               ` Jerome Glisse
2018-12-12 21:40                                 ` Dan Williams
2018-12-12 21:53                                   ` Jerome Glisse
2018-12-12 22:11                                     ` Matthew Wilcox
2018-12-12 22:16                                       ` Jerome Glisse
2018-12-12 23:37                                     ` Jason Gunthorpe
2018-12-12 23:46                                       ` John Hubbard
2018-12-12 23:54                                       ` Dan Williams
2018-12-13  0:01                                       ` Jerome Glisse
2018-12-13  0:18                                         ` Dan Williams
2018-12-13  0:44                                           ` Jerome Glisse
2018-12-13  3:26                                             ` Jason Gunthorpe
2018-12-13  3:20                                         ` Jason Gunthorpe
2018-12-13 12:43                                           ` Jerome Glisse
2018-12-13 13:40                                             ` Tom Talpey
2018-12-13 14:18                                               ` Jerome Glisse
2018-12-13 14:51                                                 ` Tom Talpey [this message]
2018-12-13 15:18                                                   ` Jerome Glisse
2018-12-13 18:12                                                     ` Tom Talpey
2018-12-13 19:18                                                       ` Jerome Glisse
2018-12-14 10:41                                             ` Jan Kara
2018-12-14 15:25                                               ` Jerome Glisse
2018-12-12 21:56                                 ` John Hubbard
2018-12-12 22:04                                   ` Jerome Glisse
2018-12-12 22:11                                     ` John Hubbard
2018-12-12 22:14                                       ` Jerome Glisse
2018-12-12 22:17                                         ` John Hubbard
2018-12-12 21:46                             ` Dave Chinner
2018-12-12 21:59                               ` Jerome Glisse
2018-12-13  0:51                                 ` Dave Chinner
2018-12-13  2:02                                   ` Jerome Glisse
2018-12-13 15:56                                     ` Christopher Lameter
2018-12-13 16:02                                       ` Jerome Glisse
2018-12-14  6:00                                     ` Dave Chinner
2018-12-14 15:13                                       ` Jerome Glisse
2018-12-14  3:52                                   ` John Hubbard
2018-12-14  5:21                                     ` Dan Williams
2018-12-14  6:11                                       ` John Hubbard
2018-12-14 15:20                                         ` Jerome Glisse
2018-12-14 19:38                                         ` Dan Williams
2018-12-14 19:48                                           ` Matthew Wilcox
2018-12-14 19:53                                             ` Dave Hansen
2018-12-14 20:03                                               ` Matthew Wilcox
2018-12-14 20:17                                                 ` Dan Williams
2018-12-14 20:29                                                   ` Matthew Wilcox
2018-12-15  0:41                                                 ` John Hubbard
2018-12-17  8:56                                           ` Jan Kara
2018-12-17 18:28                                             ` Dan Williams
2018-12-14 15:43                               ` Jan Kara
2018-12-16 21:58                                 ` Dave Chinner
2018-12-17 18:11                                   ` Jerome Glisse
2018-12-17 18:34                                     ` Matthew Wilcox
2018-12-17 19:48                                       ` Jerome Glisse
2018-12-17 19:51                                         ` Matthew Wilcox
2018-12-17 19:54                                           ` Jerome Glisse
2018-12-17 19:59                                             ` Matthew Wilcox
2018-12-17 20:55                                               ` Jerome Glisse
2018-12-17 21:03                                                 ` Matthew Wilcox
2018-12-17 21:15                                                   ` Jerome Glisse
2018-12-18  1:09                                       ` Dave Chinner
2018-12-18  6:12                                       ` Darrick J. Wong
2018-12-18  9:30                                       ` Jan Kara
2018-12-18 23:29                                         ` John Hubbard
2018-12-19  2:07                                           ` Jerome Glisse
2018-12-19 11:08                                             ` Jan Kara
2018-12-20 10:54                                               ` John Hubbard
2018-12-20 16:50                                                 ` Jerome Glisse
2018-12-20 16:57                                                   ` Dan Williams
2018-12-20 16:49                                               ` Jerome Glisse
2019-01-03  1:55                                               ` Jerome Glisse
2019-01-03  3:27                                                 ` John Hubbard
2019-01-03 14:57                                                   ` Jerome Glisse
2019-01-03  9:26                                                 ` Jan Kara
2019-01-03 14:44                                                   ` Jerome Glisse
2019-01-11  2:59                                                     ` John Hubbard
2019-01-11  2:59                                                       ` John Hubbard
2019-01-11 16:51                                                       ` Jerome Glisse
2019-01-11 16:51                                                         ` Jerome Glisse
2019-01-12  1:04                                                         ` John Hubbard
2019-01-12  1:04                                                           ` John Hubbard
2019-01-12  2:02                                                           ` Jerome Glisse
2019-01-12  2:02                                                             ` Jerome Glisse
2019-01-12  2:38                                                             ` John Hubbard
2019-01-12  2:38                                                               ` John Hubbard
2019-01-12  2:46                                                               ` Jerome Glisse
2019-01-12  2:46                                                                 ` Jerome Glisse
2019-01-12  3:06                                                                 ` John Hubbard
2019-01-12  3:06                                                                   ` John Hubbard
2019-01-12  3:25                                                                   ` Jerome Glisse
2019-01-12  3:25                                                                     ` Jerome Glisse
2019-01-12 20:46                                                                     ` John Hubbard
2019-01-12 20:46                                                                       ` John Hubbard
2019-01-14 14:54                                                                   ` Jan Kara
2019-01-14 14:54                                                                     ` Jan Kara
2019-01-14 17:21                                                                     ` Jerome Glisse
2019-01-14 17:21                                                                       ` Jerome Glisse
2019-01-14 19:09                                                                       ` John Hubbard
2019-01-14 19:09                                                                         ` John Hubbard
2019-01-15  8:34                                                                         ` Jan Kara
2019-01-15  8:34                                                                           ` Jan Kara
2019-01-15 21:39                                                                           ` John Hubbard
2019-01-15 21:39                                                                             ` John Hubbard
2019-01-15  8:07                                                                       ` Jan Kara
2019-01-15  8:07                                                                         ` Jan Kara
2019-01-15 17:15                                                                         ` Jerome Glisse
2019-01-15 17:15                                                                           ` Jerome Glisse
2019-01-15 21:56                                                                           ` John Hubbard
2019-01-15 21:56                                                                             ` John Hubbard
2019-01-15 22:12                                                                             ` Jerome Glisse
2019-01-15 22:12                                                                               ` Jerome Glisse
2019-01-16  0:44                                                                               ` John Hubbard
2019-01-16  0:44                                                                                 ` John Hubbard
2019-01-16  1:56                                                                                 ` Jerome Glisse
2019-01-16  1:56                                                                                   ` Jerome Glisse
2019-01-16  2:01                                                                                   ` Dan Williams
2019-01-16  2:01                                                                                     ` Dan Williams
2019-01-16  2:23                                                                                     ` Jerome Glisse
2019-01-16  2:23                                                                                       ` Jerome Glisse
2019-01-16  4:34                                                                                       ` Dave Chinner
2019-01-16  4:34                                                                                         ` Dave Chinner
2019-01-16 14:50                                                                                         ` Jerome Glisse
2019-01-16 14:50                                                                                           ` Jerome Glisse
2019-01-16 22:51                                                                                           ` Dave Chinner
2019-01-16 22:51                                                                                             ` Dave Chinner
2019-01-16 11:38                                                                         ` Jan Kara
2019-01-16 11:38                                                                           ` Jan Kara
2019-01-16 13:08                                                                           ` Jerome Glisse
2019-01-16 13:08                                                                             ` Jerome Glisse
2019-01-17  5:42                                                                             ` John Hubbard
2019-01-17  5:42                                                                               ` John Hubbard
2019-01-17 15:21                                                                               ` Jerome Glisse
2019-01-17 15:21                                                                                 ` Jerome Glisse
2019-01-18  0:16                                                                                 ` Dave Chinner
2019-01-18  1:59                                                                                   ` Jerome Glisse
2019-01-17  9:30                                                                             ` Jan Kara
2019-01-17  9:30                                                                               ` Jan Kara
2019-01-17 15:17                                                                               ` Jerome Glisse
2019-01-17 15:17                                                                                 ` Jerome Glisse
2019-01-22 15:24                                                                                 ` Jan Kara
2019-01-22 16:46                                                                                   ` Jerome Glisse
2019-01-23 18:02                                                                                     ` Jan Kara
2019-01-23 19:04                                                                                       ` Jerome Glisse
2019-01-29  0:22                                                                                         ` John Hubbard
2019-01-29  1:23                                                                                           ` Jerome Glisse
2019-01-29  6:41                                                                                             ` John Hubbard
2019-01-29 10:12                                                                                               ` Jan Kara
2019-01-30  2:21                                                                                                 ` John Hubbard
2019-01-17  5:25                                                                         ` John Hubbard
2019-01-17  5:25                                                                           ` John Hubbard
2019-01-17  9:04                                                                           ` Jan Kara
2019-01-17  9:04                                                                             ` Jan Kara
2019-01-12  3:14                                                               ` Jerome Glisse
2019-01-12  3:14                                                                 ` Jerome Glisse
2018-12-18 10:33                                   ` Jan Kara
2018-12-18 23:42                                     ` Dave Chinner
2018-12-19  3:03                                       ` Jason Gunthorpe
2018-12-19  5:26                                         ` Dan Williams
2018-12-19 11:19                                           ` Jan Kara
2018-12-19 10:28                                         ` Dave Chinner
2018-12-19 11:35                                           ` Jan Kara
2018-12-19 16:56                                             ` Jason Gunthorpe
2018-12-19 22:33                                             ` Dave Chinner
2018-12-20  9:07                                               ` Jan Kara
2018-12-20 16:54                                               ` Jerome Glisse
2018-12-19 13:24                                       ` Jan Kara
2018-12-08  5:18                       ` Matthew Wilcox
2018-12-12 19:13                         ` John Hubbard
2018-12-08  7:16                       ` Dan Williams
2018-12-08 16:33                         ` Jerome Glisse
2018-12-08 16:48                           ` Christoph Hellwig
2018-12-08 17:47                             ` Jerome Glisse
2018-12-08 18:26                               ` Christoph Hellwig
2018-12-08 18:45                                 ` Jerome Glisse
2018-12-08 18:09                             ` Dan Williams
2018-12-08 18:12                               ` Christoph Hellwig
2018-12-11  6:18                               ` Dave Chinner
2018-12-05  5:52             ` Dan Williams
2018-12-05 11:16       ` Jan Kara
2018-12-04  0:17 ` [PATCH 2/2] infiniband/mm: convert put_page() to put_user_page*() john.hubbard
2018-12-04 17:10 ` [PATCH 0/2] put_user_page*(): start converting the call sites David Laight
2018-12-05  1:05   ` John Hubbard
2018-12-05 14:08     ` David Laight
2018-12-28  8:37       ` Pavel Machek
2019-02-08  7:56 [PATCH 0/2] mm: put_user_page() call site conversion first john.hubbard
2019-02-08  7:56 ` [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions john.hubbard
2019-02-08 10:32   ` Mike Rapoport
2019-02-08 20:44     ` John Hubbard

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0b75a9a6-3907-ea88-6352-256bc2954f4a@talpey.com \
    --to=tom@talpey.com \
    --cc=akpm@linux-foundation.org \
    --cc=benve@cisco.com \
    --cc=cl@linux.com \
    --cc=dan.j.williams@intel.com \
    --cc=dennis.dalessandro@intel.com \
    --cc=dledford@redhat.com \
    --cc=hch@infradead.org \
    --cc=ira.weiny@intel.com \
    --cc=jack@suse.cz \
    --cc=jgg@ziepe.ca \
    --cc=jglisse@redhat.com \
    --cc=jhubbard@nvidia.com \
    --cc=john.hubbard@gmail.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@kernel.org \
    --cc=mike.marciniszyn@intel.com \
    --cc=rcampbell@nvidia.com \
    --cc=viro@zeniv.linux.org.uk \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).