From: "Michal Suchánek" <msuchanek@suse.de>
To: Mikulas Patocka <mpatocka@redhat.com>
Cc: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>,
Jan Kara <jack@suse.cz>, linux-nvdimm <linux-nvdimm@lists.01.org>,
alistair@popple.id.au,
linuxppc-dev <linuxppc-dev@lists.ozlabs.org>
Subject: Re: [PATCH v2 3/5] libnvdimm/nvdimm/flush: Allow architecture to override the flush barrier
Date: Fri, 26 Jun 2020 12:20:16 +0200 [thread overview]
Message-ID: <20200626102016.GP21462@kitsune.suse.cz> (raw)
In-Reply-To: <alpine.LRH.2.02.2005220845200.17488@file01.intranet.prod.int.rdu2.redhat.com>
On Fri, May 22, 2020 at 09:01:17AM -0400, Mikulas Patocka wrote:
>
>
> On Fri, 22 May 2020, Aneesh Kumar K.V wrote:
>
> > On 5/22/20 3:01 PM, Michal Suchánek wrote:
> > > On Thu, May 21, 2020 at 02:52:30PM -0400, Mikulas Patocka wrote:
> > > >
> > > >
> > > > On Thu, 21 May 2020, Dan Williams wrote:
> > > >
> > > > > On Thu, May 21, 2020 at 10:03 AM Aneesh Kumar K.V
> > > > > <aneesh.kumar@linux.ibm.com> wrote:
> > > > > >
> > > > > > > Moving on to the patch itself--Aneesh, have you audited other
> > > > > > > persistent
> > > > > > > memory users in the kernel? For example, drivers/md/dm-writecache.c
> > > > > > > does
> > > > > > > this:
> > > > > > >
> > > > > > > static void writecache_commit_flushed(struct dm_writecache *wc, bool
> > > > > > > wait_for_ios)
> > > > > > > {
> > > > > > > if (WC_MODE_PMEM(wc))
> > > > > > > wmb(); <==========
> > > > > > > else
> > > > > > > ssd_commit_flushed(wc, wait_for_ios);
> > > > > > > }
> > > > > > >
> > > > > > > I believe you'll need to make modifications there.
> > > > > > >
> > > > > >
> > > > > > Correct. Thanks for catching that.
> > > > > >
> > > > > >
> > > > > > I don't understand dm much, wondering how this will work with
> > > > > > non-synchronous DAX device?
> > > > >
> > > > > That's a good point. DM-writecache needs to be cognizant of things
> > > > > like virtio-pmem that violate the rule that persisent memory writes
> > > > > can be flushed by CPU functions rather than calling back into the
> > > > > driver. It seems we need to always make the flush case a dax_operation
> > > > > callback to account for this.
> > > >
> > > > dm-writecache is normally sitting on the top of dm-linear, so it would
> > > > need to pass the wmb() call through the dm core and dm-linear target ...
> > > > that would slow it down ... I remember that you already did it this way
> > > > some times ago and then removed it.
> > > >
> > > > What's the exact problem with POWER? Could the POWER system have two types
> > > > of persistent memory that need two different ways of flushing?
> > >
> > > As far as I understand the discussion so far
> > >
> > > - on POWER $oldhardware uses $oldinstruction to ensure pmem consistency
> > > - on POWER $newhardware uses $newinstruction to ensure pmem consistency
> > > (compatible with $oldinstruction on $oldhardware)
> >
> > Correct.
> >
> > > - on some platforms instead of barrier instruction a callback into the
> > > driver is issued to ensure consistency
> >
> > This is virtio-pmem only at this point IIUC.
> >
> > -aneesh
>
> And does the virtio-pmem driver track which pages are dirty? Or does it
> need to specify the range of pages to flush in the flush function?
>
> > > None of this is reflected by the dm driver.
>
> We could make a new dax method:
> void *(dax_get_flush_function)(void);
>
> This would return a pointer to "wmb()" on x86 and something else on Power.
>
> The method "dax_get_flush_function" would be called only once when
> initializing the writecache driver (because the call would be slow because
> it would have to go through the DM stack) and then, the returned function
> would be called each time we need write ordering. The returned function
> would do just "sfence; ret".
Hello,
as far as I understand the code virtio_pmem has a fush function defined
which indeed can make use of the region properties, such as memory
range. If such function exists you need quivalent of sync() - call into
the device in question. If it does not calling arch_pmem_flush_barrier()
instead of wmb() should suffice.
I am not aware of an interface to determine if the flush function exists
for a particular region.
Thanks
Michal
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
next prev parent reply other threads:[~2020-06-26 10:20 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-13 3:47 [PATCH v2 1/5] powerpc/pmem: Add new instructions for persistent storage and sync Aneesh Kumar K.V
2020-05-13 3:47 ` [PATCH v2 2/5] powerpc/pmem: Add flush routines using new pmem store and sync instruction Aneesh Kumar K.V
2020-05-13 3:47 ` [PATCH v2 3/5] libnvdimm/nvdimm/flush: Allow architecture to override the flush barrier Aneesh Kumar K.V
2020-05-13 16:14 ` Dan Williams
2020-05-19 5:30 ` Aneesh Kumar K.V
2020-05-19 7:09 ` Dan Williams
2020-05-19 13:52 ` Aneesh Kumar K.V
2020-05-19 18:59 ` Dan Williams
2020-05-20 18:43 ` Aneesh Kumar K.V
2020-05-21 14:38 ` Jeff Moyer
2020-05-21 17:02 ` Aneesh Kumar K.V
2020-05-21 18:25 ` Dan Williams
2020-05-21 18:52 ` Mikulas Patocka
2020-05-22 9:31 ` Michal Suchánek
2020-05-22 10:08 ` Aneesh Kumar K.V
2020-05-22 13:01 ` Mikulas Patocka
2020-06-26 10:20 ` Michal Suchánek [this message]
2020-05-21 18:34 ` Dan Williams
2020-05-13 3:47 ` [PATCH v2 4/5] powerpc/pmem/of_pmem: Update of_pmem to use the new barrier instruction Aneesh Kumar K.V
2020-05-13 6:44 ` kbuild test robot
2020-05-13 3:47 ` [PATCH v2 5/5] powerpc/pmem: Avoid the barrier in flush routines Aneesh Kumar K.V
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200626102016.GP21462@kitsune.suse.cz \
--to=msuchanek@suse.de \
--cc=alistair@popple.id.au \
--cc=aneesh.kumar@linux.ibm.com \
--cc=jack@suse.cz \
--cc=linux-nvdimm@lists.01.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mpatocka@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).