All of lore.kernel.org
 help / color / mirror / Atom feed
From: Dan Williams <dan.j.williams@intel.com>
To: Ross Zwisler <ross.zwisler@linux.intel.com>,
	Jan Kara <jack@suse.cz>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	"linux-nvdimm@lists.01.org" <linux-nvdimm@lists.01.org>,
	Andy Lutomirski <luto@kernel.org>,
	linux-ext4 <linux-ext4@vger.kernel.org>,
	linux-xfs@vger.kernel.org, Christoph Hellwig <hch@infradead.org>,
	Dan Williams <dan.j.williams@intel.com>,
	Boaz Harrosh <boazh@netapp.com>
Subject: Re: [PATCH 10/13] mm: Wire up MAP_SYNC
Date: Tue, 22 Aug 2017 10:27:19 -0700	[thread overview]
Message-ID: <CAPcyv4jpM_4fY+f+0i6ysnQ=0i2W9eq1pUCO0axYTHKbOtaufg@mail.gmail.com> (raw)
In-Reply-To: <20170821215703.GA24473@linux.intel.com>

On Mon, Aug 21, 2017 at 2:57 PM, Ross Zwisler
<ross.zwisler@linux.intel.com> wrote:
> On Thu, Aug 17, 2017 at 06:08:12PM +0200, Jan Kara wrote:
>> Pretty crude for now...
>>
>> Signed-off-by: Jan Kara <jack@suse.cz>
>
> One other thing that should probably be wired up before this is all said and
> done is the VmFlag string in /proc/<pid>/smaps.  Right now when we set this
> flag it ends up as ??:
>
> 7f44e6cbd000-7f44e6dbd000 rw-s 00000000 103:00 12              /root/dax/data
> Size:               1024 kB
> Rss:                   0 kB
> Pss:                   0 kB
> Shared_Clean:          0 kB
> Shared_Dirty:          0 kB
> Private_Clean:         0 kB
> Private_Dirty:         0 kB
> Referenced:            0 kB
> Anonymous:             0 kB
> LazyFree:              0 kB
> AnonHugePages:         0 kB
> ShmemPmdMapped:        0 kB
> Shared_Hugetlb:        0 kB
> Private_Hugetlb:       0 kB
> Swap:                  0 kB
> SwapPss:               0 kB
> KernelPageSize:        4 kB
> MMUPageSize:           4 kB
> Locked:                0 kB
> VmFlags: rd wr sh mr mw me ms ?? mm hg
>
> The quick one-liner at the end of this patch changes that to:
>
> 7fe30e87f000-7fe30e97f000 rw-s 00000000 103:00 12               /root/dax/data
> Size:               1024 kB
> Rss:                   0 kB
> Pss:                   0 kB
> Shared_Clean:          0 kB
> Shared_Dirty:          0 kB
> Private_Clean:         0 kB
> Private_Dirty:         0 kB
> Referenced:            0 kB
> Anonymous:             0 kB
> LazyFree:              0 kB
> AnonHugePages:         0 kB
> ShmemPmdMapped:        0 kB
> Shared_Hugetlb:        0 kB
> Private_Hugetlb:       0 kB
> Swap:                  0 kB
> SwapPss:               0 kB
> KernelPageSize:        4 kB
> MMUPageSize:           4 kB
> Locked:                0 kB
> VmFlags: rd wr sh mr mw me ms sf mm hg
>
> I think that of software can rely on this flag for userspace flushing without
> worrying about any new TOCTOU races because there isn't a way to unset the
> VM_SYNC flag once it is set - it should be valid as long as the mmap() remains
> open and the mmap'd address is valid.
>
> --- 8< ---
>  fs/proc/task_mmu.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
> index b836fd6..a2a82ed 100644
> --- a/fs/proc/task_mmu.c
> +++ b/fs/proc/task_mmu.c
> @@ -650,6 +650,7 @@ static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma)
>                 [ilog2(VM_ACCOUNT)]     = "ac",
>                 [ilog2(VM_NORESERVE)]   = "nr",
>                 [ilog2(VM_HUGETLB)]     = "ht",
> +               [ilog2(VM_SYNC)]        = "sf",
>                 [ilog2(VM_ARCH_1)]      = "ar",
>                 [ilog2(VM_DONTDUMP)]    = "dd",
>  #ifdef CONFIG_MEM_SOFT_DIRTY

So, I'm not sure I agree with this. I'm explicitly *not* advertising
MAP_DIRECT in ->vm_flags in my patches because we've seen applications
try to use smaps as an API rather than a debug tool. The toctou race
is fundamentally unsolvable unless you trust the agent that setup the
mapping will not tear it down while you've observed the sync flag.
Otherwise, if you do trust that the mapping will not be torn down then
userspace can already just trust itself and not rely on the kernel to
communicate the flag state.

I'm not against adding it, but the reasoning should be for debug and
not an api guarantee that applications will rely on, and unfortunately
I think we've seen that applications will rely on smaps no matter how
we document it.
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

WARNING: multiple messages have this Message-ID (diff)
From: Dan Williams <dan.j.williams@intel.com>
To: Ross Zwisler <ross.zwisler@linux.intel.com>,
	Jan Kara <jack@suse.cz>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	"linux-nvdimm@lists.01.org" <linux-nvdimm@lists.01.org>,
	Andy Lutomirski <luto@kernel.org>,
	linux-ext4 <linux-ext4@vger.kernel.org>,
	linux-xfs@vger.kernel.org, Christoph Hellwig <hch@infradead.org>,
	Dan Williams <dan.j.williams@intel.com>,
	Boaz Harrosh <boazh@netapp.com>
Subject: Re: [PATCH 10/13] mm: Wire up MAP_SYNC
Date: Tue, 22 Aug 2017 10:27:19 -0700	[thread overview]
Message-ID: <CAPcyv4jpM_4fY+f+0i6ysnQ=0i2W9eq1pUCO0axYTHKbOtaufg@mail.gmail.com> (raw)
In-Reply-To: <20170821215703.GA24473@linux.intel.com>

On Mon, Aug 21, 2017 at 2:57 PM, Ross Zwisler
<ross.zwisler@linux.intel.com> wrote:
> On Thu, Aug 17, 2017 at 06:08:12PM +0200, Jan Kara wrote:
>> Pretty crude for now...
>>
>> Signed-off-by: Jan Kara <jack@suse.cz>
>
> One other thing that should probably be wired up before this is all said and
> done is the VmFlag string in /proc/<pid>/smaps.  Right now when we set this
> flag it ends up as ??:
>
> 7f44e6cbd000-7f44e6dbd000 rw-s 00000000 103:00 12              /root/dax/data
> Size:               1024 kB
> Rss:                   0 kB
> Pss:                   0 kB
> Shared_Clean:          0 kB
> Shared_Dirty:          0 kB
> Private_Clean:         0 kB
> Private_Dirty:         0 kB
> Referenced:            0 kB
> Anonymous:             0 kB
> LazyFree:              0 kB
> AnonHugePages:         0 kB
> ShmemPmdMapped:        0 kB
> Shared_Hugetlb:        0 kB
> Private_Hugetlb:       0 kB
> Swap:                  0 kB
> SwapPss:               0 kB
> KernelPageSize:        4 kB
> MMUPageSize:           4 kB
> Locked:                0 kB
> VmFlags: rd wr sh mr mw me ms ?? mm hg
>
> The quick one-liner at the end of this patch changes that to:
>
> 7fe30e87f000-7fe30e97f000 rw-s 00000000 103:00 12               /root/dax/data
> Size:               1024 kB
> Rss:                   0 kB
> Pss:                   0 kB
> Shared_Clean:          0 kB
> Shared_Dirty:          0 kB
> Private_Clean:         0 kB
> Private_Dirty:         0 kB
> Referenced:            0 kB
> Anonymous:             0 kB
> LazyFree:              0 kB
> AnonHugePages:         0 kB
> ShmemPmdMapped:        0 kB
> Shared_Hugetlb:        0 kB
> Private_Hugetlb:       0 kB
> Swap:                  0 kB
> SwapPss:               0 kB
> KernelPageSize:        4 kB
> MMUPageSize:           4 kB
> Locked:                0 kB
> VmFlags: rd wr sh mr mw me ms sf mm hg
>
> I think that of software can rely on this flag for userspace flushing without
> worrying about any new TOCTOU races because there isn't a way to unset the
> VM_SYNC flag once it is set - it should be valid as long as the mmap() remains
> open and the mmap'd address is valid.
>
> --- 8< ---
>  fs/proc/task_mmu.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
> index b836fd6..a2a82ed 100644
> --- a/fs/proc/task_mmu.c
> +++ b/fs/proc/task_mmu.c
> @@ -650,6 +650,7 @@ static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma)
>                 [ilog2(VM_ACCOUNT)]     = "ac",
>                 [ilog2(VM_NORESERVE)]   = "nr",
>                 [ilog2(VM_HUGETLB)]     = "ht",
> +               [ilog2(VM_SYNC)]        = "sf",
>                 [ilog2(VM_ARCH_1)]      = "ar",
>                 [ilog2(VM_DONTDUMP)]    = "dd",
>  #ifdef CONFIG_MEM_SOFT_DIRTY

So, I'm not sure I agree with this. I'm explicitly *not* advertising
MAP_DIRECT in ->vm_flags in my patches because we've seen applications
try to use smaps as an API rather than a debug tool. The toctou race
is fundamentally unsolvable unless you trust the agent that setup the
mapping will not tear it down while you've observed the sync flag.
Otherwise, if you do trust that the mapping will not be torn down then
userspace can already just trust itself and not rely on the kernel to
communicate the flag state.

I'm not against adding it, but the reasoning should be for debug and
not an api guarantee that applications will rely on, and unfortunately
I think we've seen that applications will rely on smaps no matter how
we document it.

WARNING: multiple messages have this Message-ID (diff)
From: Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
To: Ross Zwisler
	<ross.zwisler-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>,
	Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org>,
	linux-fsdevel
	<linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
	"linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org"
	<linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org>,
	Andy Lutomirski <luto-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	linux-ext4 <linux-ext4-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
	linux-xfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	Christoph Hellwig <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>,
	Dan Williams
	<dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>,
	Boaz Harrosh <boazh-HgOvQuBEEgTQT0dZR+AlfA@public.gmane.org>
Subject: Re: [PATCH 10/13] mm: Wire up MAP_SYNC
Date: Tue, 22 Aug 2017 10:27:19 -0700	[thread overview]
Message-ID: <CAPcyv4jpM_4fY+f+0i6ysnQ=0i2W9eq1pUCO0axYTHKbOtaufg@mail.gmail.com> (raw)
In-Reply-To: <20170821215703.GA24473-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>

On Mon, Aug 21, 2017 at 2:57 PM, Ross Zwisler
<ross.zwisler-VuQAYsv1563Yd54FQh9/CA@public.gmane.org> wrote:
> On Thu, Aug 17, 2017 at 06:08:12PM +0200, Jan Kara wrote:
>> Pretty crude for now...
>>
>> Signed-off-by: Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org>
>
> One other thing that should probably be wired up before this is all said and
> done is the VmFlag string in /proc/<pid>/smaps.  Right now when we set this
> flag it ends up as ??:
>
> 7f44e6cbd000-7f44e6dbd000 rw-s 00000000 103:00 12              /root/dax/data
> Size:               1024 kB
> Rss:                   0 kB
> Pss:                   0 kB
> Shared_Clean:          0 kB
> Shared_Dirty:          0 kB
> Private_Clean:         0 kB
> Private_Dirty:         0 kB
> Referenced:            0 kB
> Anonymous:             0 kB
> LazyFree:              0 kB
> AnonHugePages:         0 kB
> ShmemPmdMapped:        0 kB
> Shared_Hugetlb:        0 kB
> Private_Hugetlb:       0 kB
> Swap:                  0 kB
> SwapPss:               0 kB
> KernelPageSize:        4 kB
> MMUPageSize:           4 kB
> Locked:                0 kB
> VmFlags: rd wr sh mr mw me ms ?? mm hg
>
> The quick one-liner at the end of this patch changes that to:
>
> 7fe30e87f000-7fe30e97f000 rw-s 00000000 103:00 12               /root/dax/data
> Size:               1024 kB
> Rss:                   0 kB
> Pss:                   0 kB
> Shared_Clean:          0 kB
> Shared_Dirty:          0 kB
> Private_Clean:         0 kB
> Private_Dirty:         0 kB
> Referenced:            0 kB
> Anonymous:             0 kB
> LazyFree:              0 kB
> AnonHugePages:         0 kB
> ShmemPmdMapped:        0 kB
> Shared_Hugetlb:        0 kB
> Private_Hugetlb:       0 kB
> Swap:                  0 kB
> SwapPss:               0 kB
> KernelPageSize:        4 kB
> MMUPageSize:           4 kB
> Locked:                0 kB
> VmFlags: rd wr sh mr mw me ms sf mm hg
>
> I think that of software can rely on this flag for userspace flushing without
> worrying about any new TOCTOU races because there isn't a way to unset the
> VM_SYNC flag once it is set - it should be valid as long as the mmap() remains
> open and the mmap'd address is valid.
>
> --- 8< ---
>  fs/proc/task_mmu.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
> index b836fd6..a2a82ed 100644
> --- a/fs/proc/task_mmu.c
> +++ b/fs/proc/task_mmu.c
> @@ -650,6 +650,7 @@ static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma)
>                 [ilog2(VM_ACCOUNT)]     = "ac",
>                 [ilog2(VM_NORESERVE)]   = "nr",
>                 [ilog2(VM_HUGETLB)]     = "ht",
> +               [ilog2(VM_SYNC)]        = "sf",
>                 [ilog2(VM_ARCH_1)]      = "ar",
>                 [ilog2(VM_DONTDUMP)]    = "dd",
>  #ifdef CONFIG_MEM_SOFT_DIRTY

So, I'm not sure I agree with this. I'm explicitly *not* advertising
MAP_DIRECT in ->vm_flags in my patches because we've seen applications
try to use smaps as an API rather than a debug tool. The toctou race
is fundamentally unsolvable unless you trust the agent that setup the
mapping will not tear it down while you've observed the sync flag.
Otherwise, if you do trust that the mapping will not be torn down then
userspace can already just trust itself and not rely on the kernel to
communicate the flag state.

I'm not against adding it, but the reasoning should be for debug and
not an api guarantee that applications will rely on, and unfortunately
I think we've seen that applications will rely on smaps no matter how
we document it.

  parent reply	other threads:[~2017-08-22 17:24 UTC|newest]

Thread overview: 142+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-08-17 16:08 [RFC PATCH 0/13 v2] dax, ext4: Synchronous page faults Jan Kara
2017-08-17 16:08 ` Jan Kara
2017-08-17 16:08 ` Jan Kara
2017-08-17 16:08 ` Jan Kara
2017-08-17 16:08 ` [PATCH 01/13] mm: Remove VM_FAULT_HWPOISON_LARGE_MASK Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08 ` [PATCH 02/13] dax: Simplify arguments of dax_insert_mapping() Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08 ` [PATCH 03/13] dax: Factor out getting of pfn out of iomap Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-18 22:06   ` Ross Zwisler
2017-08-18 22:06     ` Ross Zwisler
2017-08-23 18:30   ` Christoph Hellwig
2017-08-23 18:30     ` Christoph Hellwig
2017-08-17 16:08 ` [PATCH 04/13] dax: Create local variable for VMA in dax_iomap_pte_fault() Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-18 22:08   ` Ross Zwisler
2017-08-18 22:08     ` Ross Zwisler
2017-08-23 18:30   ` Christoph Hellwig
2017-08-23 18:30     ` Christoph Hellwig
2017-08-17 16:08 ` [PATCH 05/13] dax: Create local variable for vmf->flags & FAULT_FLAG_WRITE test Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-18 22:08   ` Ross Zwisler
2017-08-18 22:08     ` Ross Zwisler
2017-08-18 22:08     ` Ross Zwisler
2017-08-23 18:31   ` Christoph Hellwig
2017-08-23 18:31     ` Christoph Hellwig
2017-08-23 18:31     ` Christoph Hellwig
2017-08-17 16:08 ` [PATCH 06/13] dax: Inline dax_insert_mapping() into the callsite Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-18 22:10   ` Ross Zwisler
2017-08-18 22:10     ` Ross Zwisler
2017-08-18 22:10     ` Ross Zwisler
2017-08-23 18:31   ` Christoph Hellwig
2017-08-23 18:31     ` Christoph Hellwig
2017-08-23 18:31     ` Christoph Hellwig
2017-08-17 16:08 ` [PATCH 07/13] dax: Inline dax_pmd_insert_mapping() " Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-18 22:12   ` Ross Zwisler
2017-08-18 22:12     ` Ross Zwisler
2017-08-18 22:12     ` Ross Zwisler
2017-08-23 18:32   ` Christoph Hellwig
2017-08-23 18:32     ` Christoph Hellwig
2017-08-17 16:08 ` [PATCH 08/13] dax: Fix comment describing dax_iomap_fault() Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-18 22:12   ` Ross Zwisler
2017-08-18 22:12     ` Ross Zwisler
2017-08-23 18:32   ` Christoph Hellwig
2017-08-23 18:32     ` Christoph Hellwig
2017-08-17 16:08 ` [PATCH 09/13] dax: Allow dax_iomap_fault() to return pfn Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-21 18:45   ` Ross Zwisler
2017-08-21 18:45     ` Ross Zwisler
2017-08-23 18:34   ` Christoph Hellwig
2017-08-23 18:34     ` Christoph Hellwig
2017-08-23 18:34     ` Christoph Hellwig
2017-08-24  7:26     ` Jan Kara
2017-08-24  7:26       ` Jan Kara
2017-08-17 16:08 ` [PATCH 10/13] mm: Wire up MAP_SYNC Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-21 21:37   ` Ross Zwisler
2017-08-21 21:37     ` Ross Zwisler
2017-08-22  9:36     ` Jan Kara
2017-08-22  9:36       ` Jan Kara
2017-08-21 21:57   ` Ross Zwisler
2017-08-21 21:57     ` Ross Zwisler
2017-08-21 21:57     ` Ross Zwisler
2017-08-22  9:34     ` Jan Kara
2017-08-22  9:34       ` Jan Kara
2017-08-22 17:27     ` Dan Williams [this message]
2017-08-22 17:27       ` Dan Williams
2017-08-22 17:27       ` Dan Williams
2017-08-23 18:43   ` Christoph Hellwig
2017-08-23 18:43     ` Christoph Hellwig
2017-08-23 18:43     ` Christoph Hellwig
2017-08-24  7:16     ` Jan Kara
2017-08-24  7:16       ` Jan Kara
2017-08-17 16:08 ` [PATCH 11/13] dax, iomap: Add support for synchronous faults Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-21 18:58   ` Ross Zwisler
2017-08-21 18:58     ` Ross Zwisler
2017-08-22  9:46     ` Jan Kara
2017-08-22  9:46       ` Jan Kara
2017-08-21 21:09   ` Ross Zwisler
2017-08-21 21:09     ` Ross Zwisler
2017-08-22 10:08     ` Jan Kara
2017-08-22 10:08       ` Jan Kara
2017-08-22 10:08       ` Jan Kara
2017-08-24 12:27   ` Christoph Hellwig
2017-08-24 12:27     ` Christoph Hellwig
2017-08-24 12:34     ` Jan Kara
2017-08-24 12:34       ` Jan Kara
2017-08-24 13:38       ` Christoph Hellwig
2017-08-24 13:38         ` Christoph Hellwig
2017-08-24 16:45         ` Jan Kara
2017-08-24 16:45           ` Jan Kara
2017-08-17 16:08 ` [PATCH 12/13] dax: Implement dax_insert_pfn_mkwrite() Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-21 19:01   ` Ross Zwisler
2017-08-21 19:01     ` Ross Zwisler
2017-08-17 16:08 ` [PATCH 13/13] ext4: Support for synchronous DAX faults Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-17 16:08   ` Jan Kara
2017-08-21 19:19   ` Ross Zwisler
2017-08-21 19:19     ` Ross Zwisler
2017-08-22 10:18     ` Jan Kara
2017-08-22 10:18       ` Jan Kara
2017-08-22 10:18       ` Jan Kara
2017-08-23 18:37   ` Christoph Hellwig
2017-08-23 18:37     ` Christoph Hellwig
2017-08-24  7:18     ` Jan Kara
2017-08-24  7:18       ` Jan Kara
2017-08-24 12:31   ` Christoph Hellwig
2017-08-24 12:31     ` Christoph Hellwig
2017-08-24 12:34     ` Christoph Hellwig
2017-08-24 12:34       ` Christoph Hellwig
2017-08-24 12:34       ` Christoph Hellwig
2017-08-24 12:36     ` Jan Kara
2017-08-24 12:36       ` Jan Kara

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAPcyv4jpM_4fY+f+0i6ysnQ=0i2W9eq1pUCO0axYTHKbOtaufg@mail.gmail.com' \
    --to=dan.j.williams@intel.com \
    --cc=boazh@netapp.com \
    --cc=hch@infradead.org \
    --cc=jack@suse.cz \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-nvdimm@lists.01.org \
    --cc=linux-xfs@vger.kernel.org \
    --cc=luto@kernel.org \
    --cc=ross.zwisler@linux.intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.