[PATCH v2] libnvdimm: re-enable deep flush for pmem devices

[PATCH v2] libnvdimm: re-enable deep flush for pmem devices

[Dave Jiang]

Re-enable deep flush so that users always have a way to be sure that a write
does make it all the way out to the NVDIMM. The PMEM driver writes always
make it "all the way to the NVDIMM", and it relies on the ADR mechanism to
flush the write buffers on power failure. Deep flush is there to explicitly
flush those write buffers to protect against (rare) ADR failure.
This change prevents a regression in deep flush behavior so that applications
can continue to depend on fsync() as a mechanism to trigger deep flush in the
filesystem-dax case.

Fixes: 06e8ccdab15f4 ("acpi: nfit: Add support for detect platform CPU cache flush on power loss")

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
---

v2: Updated commit patch header from Dan's comments.

 drivers/nvdimm/pmem.c |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c
index 10041ac4032c..06f8dcc52ca6 100644
--- a/drivers/nvdimm/pmem.c
+++ b/drivers/nvdimm/pmem.c
@@ -335,8 +335,7 @@ static int pmem_attach_disk(struct device *dev,
 		dev_warn(dev, "unable to guarantee persistence of writes\n");
 		fua = 0;
 	}
-	wbc = nvdimm_has_cache(nd_region) &&
-		!test_bit(ND_REGION_PERSIST_CACHE, &nd_region->flags);
+	wbc = nvdimm_has_cache(nd_region);
 
 	if (!devm_request_mem_region(dev, res->start, resource_size(res),
 				dev_name(&ndns->dev))) {

Re: [PATCH v2] libnvdimm: re-enable deep flush for pmem devices

[Dan Williams]

On Mon, Feb 12, 2018 at 1:46 PM, Dave Jiang <dave.jiang@intel.com> wrote:
> Re-enable deep flush so that users always have a way to be sure that a write
> does make it all the way out to the NVDIMM. The PMEM driver writes always
> make it "all the way to the NVDIMM", and it relies on the ADR mechanism to
> flush the write buffers on power failure. Deep flush is there to explicitly
> flush those write buffers to protect against (rare) ADR failure.
> This change prevents a regression in deep flush behavior so that applications
> can continue to depend on fsync() as a mechanism to trigger deep flush in the
> filesystem-dax case.
>
> Fixes: 06e8ccdab15f4 ("acpi: nfit: Add support for detect platform CPU cache flush on power loss")
>
> Signed-off-by: Dave Jiang <dave.jiang@intel.com>

Thanks Dave, applied.

Re: [PATCH v2] libnvdimm: re-enable deep flush for pmem devices

[Jeff Moyer]

Dave Jiang <dave.jiang@intel.com> writes:

> Re-enable deep flush so that users always have a way to be sure that a write
> does make it all the way out to the NVDIMM. The PMEM driver writes always
> make it "all the way to the NVDIMM", and it relies on the ADR mechanism to
> flush the write buffers on power failure. Deep flush is there to explicitly
> flush those write buffers to protect against (rare) ADR failure.
> This change prevents a regression in deep flush behavior so that applications
> can continue to depend on fsync() as a mechanism to trigger deep flush in the
> filesystem-dax case.

That's still very confusing text.  Specifically, the part where you say
that pmem driver writes always make it to the DIMM.  I think the
changelog could start with "Deep flush is there to explicitly flush
write buffers...."  Anyway, the fix looks right to me.

Reviewed-by: Jeff Moyer <jmoyer@redhat.com>

Re: [PATCH v2] libnvdimm: re-enable deep flush for pmem devices

[Dan Williams]

On Mon, Feb 12, 2018 at 2:53 PM, Jeff Moyer <jmoyer@redhat.com> wrote:
> Dave Jiang <dave.jiang@intel.com> writes:
>
>> Re-enable deep flush so that users always have a way to be sure that a write
>> does make it all the way out to the NVDIMM. The PMEM driver writes always
>> make it "all the way to the NVDIMM", and it relies on the ADR mechanism to
>> flush the write buffers on power failure. Deep flush is there to explicitly
>> flush those write buffers to protect against (rare) ADR failure.
>> This change prevents a regression in deep flush behavior so that applications
>> can continue to depend on fsync() as a mechanism to trigger deep flush in the
>> filesystem-dax case.
>
> That's still very confusing text.  Specifically, the part where you say
> that pmem driver writes always make it to the DIMM.  I think the
> changelog could start with "Deep flush is there to explicitly flush
> write buffers...."  Anyway, the fix looks right to me.

I ended up changing the commit message to this, let me know if it reads better:


    libnvdimm: re-enable deep flush for pmem devices via fsync()

    Re-enable deep flush so that users always have a way to be sure that a
    write makes it all the way out to media. The PMEM driver writes always
    arrive at the NVDIMM, and it relies on the ADR (Asynchronous DRAM
    Refresh) mechanism to flush the write buffers on power failure. Deep
    flush is there to explicitly flush those write buffers to protect
    against (rare) ADR failure.  This change prevents a regression in deep
    flush behavior so that applications can continue to depend on fsync() as
    a mechanism to trigger deep flush in the filesystem-DAX case.

    Fixes: 06e8ccdab15f4 ("acpi: nfit: Add support for detect platform
CPU cache...")
    Signed-off-by: Dave Jiang <dave.jiang@intel.com>
    Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Re: [PATCH v2] libnvdimm: re-enable deep flush for pmem devices

[Ross Zwisler]

On Mon, Feb 12, 2018 at 03:05:10PM -0800, Dan Williams wrote:
> On Mon, Feb 12, 2018 at 2:53 PM, Jeff Moyer <jmoyer@redhat.com> wrote:
> > Dave Jiang <dave.jiang@intel.com> writes:
> >
> >> Re-enable deep flush so that users always have a way to be sure that a write
> >> does make it all the way out to the NVDIMM. The PMEM driver writes always
> >> make it "all the way to the NVDIMM", and it relies on the ADR mechanism to
> >> flush the write buffers on power failure. Deep flush is there to explicitly
> >> flush those write buffers to protect against (rare) ADR failure.
> >> This change prevents a regression in deep flush behavior so that applications
> >> can continue to depend on fsync() as a mechanism to trigger deep flush in the
> >> filesystem-dax case.
> >
> > That's still very confusing text.  Specifically, the part where you say
> > that pmem driver writes always make it to the DIMM.  I think the
> > changelog could start with "Deep flush is there to explicitly flush
> > write buffers...."  Anyway, the fix looks right to me.
> 
> I ended up changing the commit message to this, let me know if it reads better:
> 
> 
>     libnvdimm: re-enable deep flush for pmem devices via fsync()
> 
>     Re-enable deep flush so that users always have a way to be sure that a
>     write makes it all the way out to media. The PMEM driver writes always
>     arrive at the NVDIMM, and it relies on the ADR (Asynchronous DRAM
>     Refresh) mechanism to flush the write buffers on power failure. Deep
>     flush is there to explicitly flush those write buffers to protect
>     against (rare) ADR failure.  This change prevents a regression in deep
>     flush behavior so that applications can continue to depend on fsync() as
>     a mechanism to trigger deep flush in the filesystem-DAX case.
> 
>     Fixes: 06e8ccdab15f4 ("acpi: nfit: Add support for detect platform
> CPU cache...")
>     Signed-off-by: Dave Jiang <dave.jiang@intel.com>
>     Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Plus Jeff's reviewed-by.

Re: [PATCH v2] libnvdimm: re-enable deep flush for pmem devices

[Jeff Moyer]

Dan Williams <dan.j.williams@intel.com> writes:

> On Mon, Feb 12, 2018 at 2:53 PM, Jeff Moyer <jmoyer@redhat.com> wrote:
>> Dave Jiang <dave.jiang@intel.com> writes:
>>
>>> Re-enable deep flush so that users always have a way to be sure that a write
>>> does make it all the way out to the NVDIMM. The PMEM driver writes always
>>> make it "all the way to the NVDIMM", and it relies on the ADR mechanism to
>>> flush the write buffers on power failure. Deep flush is there to explicitly
>>> flush those write buffers to protect against (rare) ADR failure.
>>> This change prevents a regression in deep flush behavior so that applications
>>> can continue to depend on fsync() as a mechanism to trigger deep flush in the
>>> filesystem-dax case.
>>
>> That's still very confusing text.  Specifically, the part where you say
>> that pmem driver writes always make it to the DIMM.  I think the
>> changelog could start with "Deep flush is there to explicitly flush
>> write buffers...."  Anyway, the fix looks right to me.
>
> I ended up changing the commit message to this, let me know if it reads better:

Thanks.  It's still unclear to me what the text, "The PMEM driver writes
always arrive at the NVDIMM" means.  However, it's good enough.

Thanks!
Jeff

>
>     libnvdimm: re-enable deep flush for pmem devices via fsync()
>
>     Re-enable deep flush so that users always have a way to be sure that a
>     write makes it all the way out to media. The PMEM driver writes always
>     arrive at the NVDIMM, and it relies on the ADR (Asynchronous DRAM
>     Refresh) mechanism to flush the write buffers on power failure. Deep
>     flush is there to explicitly flush those write buffers to protect
>     against (rare) ADR failure.  This change prevents a regression in deep
>     flush behavior so that applications can continue to depend on fsync() as
>     a mechanism to trigger deep flush in the filesystem-DAX case.
>
>     Fixes: 06e8ccdab15f4 ("acpi: nfit: Add support for detect platform
> CPU cache...")
>     Signed-off-by: Dave Jiang <dave.jiang@intel.com>
>     Signed-off-by: Dan Williams <dan.j.williams@intel.com>

Re: [PATCH v2] libnvdimm: re-enable deep flush for pmem devices

[Dan Williams]

On Tue, Feb 13, 2018 at 5:17 AM, Jeff Moyer <jmoyer@redhat.com> wrote:
> Dan Williams <dan.j.williams@intel.com> writes:
>
>> On Mon, Feb 12, 2018 at 2:53 PM, Jeff Moyer <jmoyer@redhat.com> wrote:
>>> Dave Jiang <dave.jiang@intel.com> writes:
>>>
>>>> Re-enable deep flush so that users always have a way to be sure that a write
>>>> does make it all the way out to the NVDIMM. The PMEM driver writes always
>>>> make it "all the way to the NVDIMM", and it relies on the ADR mechanism to
>>>> flush the write buffers on power failure. Deep flush is there to explicitly
>>>> flush those write buffers to protect against (rare) ADR failure.
>>>> This change prevents a regression in deep flush behavior so that applications
>>>> can continue to depend on fsync() as a mechanism to trigger deep flush in the
>>>> filesystem-dax case.
>>>
>>> That's still very confusing text.  Specifically, the part where you say
>>> that pmem driver writes always make it to the DIMM.  I think the
>>> changelog could start with "Deep flush is there to explicitly flush
>>> write buffers...."  Anyway, the fix looks right to me.
>>
>> I ended up changing the commit message to this, let me know if it reads better:
>
> Thanks.  It's still unclear to me what the text, "The PMEM driver writes
> always arrive at the NVDIMM" means.  However, it's good enough.

Yeah, Dave, had similar feedback. A better way of saying it is that
the writes always arrive at the persistence domain, but deep flush
pushes them to the smallest platform failure domain. On current
platforms that's to the ADR domain and past the ADR domain.