From: Dan Williams <dan.j.williams@intel.com> To: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com> Cc: linuxppc-dev <linuxppc-dev@lists.ozlabs.org>, Michael Ellerman <mpe@ellerman.id.au>, linux-nvdimm <linux-nvdimm@lists.01.org>, alistair@popple.id.au Subject: Re: [PATCH v2 3/5] libnvdimm/nvdimm/flush: Allow architecture to override the flush barrier Date: Tue, 19 May 2020 00:09:11 -0700 [thread overview] Message-ID: <CAPcyv4g+oE305Q5bYWkNBKFifB9c0TZo6+hqFQnqiFqU5QFrhQ@mail.gmail.com> (raw) In-Reply-To: <87v9kspk3x.fsf@linux.ibm.com> On Mon, May 18, 2020 at 10:30 PM Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com> wrote: > > > Hi Dan, > > Apologies for the delay in response. I was waiting for feedback from > hardware team before responding to this email. > > > Dan Williams <dan.j.williams@intel.com> writes: > > > On Tue, May 12, 2020 at 8:47 PM Aneesh Kumar K.V > > <aneesh.kumar@linux.ibm.com> wrote: > >> > >> Architectures like ppc64 provide persistent memory specific barriers > >> that will ensure that all stores for which the modifications are > >> written to persistent storage by preceding dcbfps and dcbstps > >> instructions have updated persistent storage before any data > >> access or data transfer caused by subsequent instructions is initiated. > >> This is in addition to the ordering done by wmb() > >> > >> Update nvdimm core such that architecture can use barriers other than > >> wmb to ensure all previous writes are architecturally visible for > >> the platform buffer flush. > > > > This seems like an exceedingly bad idea, maybe I'm missing something. > > This implies that the deployed base of DAX applications using the old > > instruction sequence are going to regress on new hardware that > > requires the new instructions to be deployed. > > > pmdk support for ppc64 is still work in progress and there is pull > request to switch pmdk to use new instruction. Ok. > > https://github.com/tuliom/pmdk/commit/fix-flush > > All userspace applications will be switched to use the new > instructions. The new instructions are designed such that when running on P8 > and P9 they behave as 'dcbf' and 'hwsync'. Sure, makes sense. > Applications using new instructions will behave as expected when running > on P8 and P9. Only future hardware will differentiate between 'dcbf' and > 'dcbfps' Right, this is the problem. Applications using new instructions behave as expected, the kernel has been shipping of_pmem and papr_scm for several cycles now, you're saying that the DAX applications written against those platforms are going to be broken on P8 and P9? > > I'm thinking the kernel > > should go as far as to disable DAX operation by default on new > > hardware until userspace asserts that it is prepared to switch to the > > new implementation. Is there any other way to ensure the forward > > compatibility of deployed ppc64 DAX applications? > > AFAIU there is no released persistent memory hardware on ppc64 platform > and we need to make sure before applications get enabled to use these > persistent memory devices, they should switch to use the new > instruction? Right, I want the kernel to offer some level of safety here because everything you are describing sounds like a flag day conversion. Am I misreading? Is there some other gate that prevents existing users of of_pmem and papr_scm from having their expectations violated when running on P8 / P9 hardware? Maybe there's tighter ecosystem control that I'm just not familiar with, I'm only going off the fact that the kernel has shipped a non-zero number of NVDIMM drivers that build with ARCH=ppc64 for several cycles. _______________________________________________ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
WARNING: multiple messages have this Message-ID (diff)
From: Dan Williams <dan.j.williams@intel.com> To: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com> Cc: alistair@popple.id.au, linuxppc-dev <linuxppc-dev@lists.ozlabs.org>, linux-nvdimm <linux-nvdimm@lists.01.org> Subject: Re: [PATCH v2 3/5] libnvdimm/nvdimm/flush: Allow architecture to override the flush barrier Date: Tue, 19 May 2020 00:09:11 -0700 [thread overview] Message-ID: <CAPcyv4g+oE305Q5bYWkNBKFifB9c0TZo6+hqFQnqiFqU5QFrhQ@mail.gmail.com> (raw) In-Reply-To: <87v9kspk3x.fsf@linux.ibm.com> On Mon, May 18, 2020 at 10:30 PM Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com> wrote: > > > Hi Dan, > > Apologies for the delay in response. I was waiting for feedback from > hardware team before responding to this email. > > > Dan Williams <dan.j.williams@intel.com> writes: > > > On Tue, May 12, 2020 at 8:47 PM Aneesh Kumar K.V > > <aneesh.kumar@linux.ibm.com> wrote: > >> > >> Architectures like ppc64 provide persistent memory specific barriers > >> that will ensure that all stores for which the modifications are > >> written to persistent storage by preceding dcbfps and dcbstps > >> instructions have updated persistent storage before any data > >> access or data transfer caused by subsequent instructions is initiated. > >> This is in addition to the ordering done by wmb() > >> > >> Update nvdimm core such that architecture can use barriers other than > >> wmb to ensure all previous writes are architecturally visible for > >> the platform buffer flush. > > > > This seems like an exceedingly bad idea, maybe I'm missing something. > > This implies that the deployed base of DAX applications using the old > > instruction sequence are going to regress on new hardware that > > requires the new instructions to be deployed. > > > pmdk support for ppc64 is still work in progress and there is pull > request to switch pmdk to use new instruction. Ok. > > https://github.com/tuliom/pmdk/commit/fix-flush > > All userspace applications will be switched to use the new > instructions. The new instructions are designed such that when running on P8 > and P9 they behave as 'dcbf' and 'hwsync'. Sure, makes sense. > Applications using new instructions will behave as expected when running > on P8 and P9. Only future hardware will differentiate between 'dcbf' and > 'dcbfps' Right, this is the problem. Applications using new instructions behave as expected, the kernel has been shipping of_pmem and papr_scm for several cycles now, you're saying that the DAX applications written against those platforms are going to be broken on P8 and P9? > > I'm thinking the kernel > > should go as far as to disable DAX operation by default on new > > hardware until userspace asserts that it is prepared to switch to the > > new implementation. Is there any other way to ensure the forward > > compatibility of deployed ppc64 DAX applications? > > AFAIU there is no released persistent memory hardware on ppc64 platform > and we need to make sure before applications get enabled to use these > persistent memory devices, they should switch to use the new > instruction? Right, I want the kernel to offer some level of safety here because everything you are describing sounds like a flag day conversion. Am I misreading? Is there some other gate that prevents existing users of of_pmem and papr_scm from having their expectations violated when running on P8 / P9 hardware? Maybe there's tighter ecosystem control that I'm just not familiar with, I'm only going off the fact that the kernel has shipped a non-zero number of NVDIMM drivers that build with ARCH=ppc64 for several cycles.
next prev parent reply other threads:[~2020-05-19 7:09 UTC|newest] Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-05-13 3:47 [PATCH v2 1/5] powerpc/pmem: Add new instructions for persistent storage and sync Aneesh Kumar K.V 2020-05-13 3:47 ` Aneesh Kumar K.V 2020-05-13 3:47 ` [PATCH v2 2/5] powerpc/pmem: Add flush routines using new pmem store and sync instruction Aneesh Kumar K.V 2020-05-13 3:47 ` Aneesh Kumar K.V 2020-05-13 3:47 ` [PATCH v2 3/5] libnvdimm/nvdimm/flush: Allow architecture to override the flush barrier Aneesh Kumar K.V 2020-05-13 3:47 ` Aneesh Kumar K.V 2020-05-13 16:14 ` Dan Williams 2020-05-13 16:14 ` Dan Williams 2020-05-19 5:30 ` Aneesh Kumar K.V 2020-05-19 5:30 ` Aneesh Kumar K.V 2020-05-19 7:09 ` Dan Williams [this message] 2020-05-19 7:09 ` Dan Williams 2020-05-19 13:52 ` Aneesh Kumar K.V 2020-05-19 13:52 ` Aneesh Kumar K.V 2020-05-19 18:59 ` Dan Williams 2020-05-19 18:59 ` Dan Williams 2020-05-20 18:43 ` Aneesh Kumar K.V 2020-05-20 18:43 ` Aneesh Kumar K.V 2020-05-21 14:38 ` Jeff Moyer 2020-05-21 14:38 ` Jeff Moyer 2020-05-21 17:02 ` Aneesh Kumar K.V 2020-05-21 17:02 ` Aneesh Kumar K.V 2020-05-21 18:25 ` Dan Williams 2020-05-21 18:25 ` Dan Williams 2020-05-21 18:52 ` Mikulas Patocka 2020-05-21 18:52 ` Mikulas Patocka 2020-05-22 9:31 ` Michal Suchánek 2020-05-22 9:31 ` Michal Suchánek 2020-05-22 10:08 ` Aneesh Kumar K.V 2020-05-22 10:08 ` Aneesh Kumar K.V 2020-05-22 13:01 ` Mikulas Patocka 2020-05-22 13:01 ` Mikulas Patocka 2020-06-26 10:20 ` Michal Suchánek 2020-06-26 10:20 ` Michal Suchánek 2020-05-21 18:34 ` Dan Williams 2020-05-21 18:34 ` Dan Williams 2020-05-13 3:47 ` [PATCH v2 4/5] powerpc/pmem/of_pmem: Update of_pmem to use the new barrier instruction Aneesh Kumar K.V 2020-05-13 3:47 ` Aneesh Kumar K.V 2020-05-13 6:44 ` kbuild test robot 2020-05-13 6:44 ` kbuild test robot 2020-05-13 6:44 ` kbuild test robot 2020-05-13 3:47 ` [PATCH v2 5/5] powerpc/pmem: Avoid the barrier in flush routines Aneesh Kumar K.V 2020-05-13 3:47 ` Aneesh Kumar K.V
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=CAPcyv4g+oE305Q5bYWkNBKFifB9c0TZo6+hqFQnqiFqU5QFrhQ@mail.gmail.com \ --to=dan.j.williams@intel.com \ --cc=alistair@popple.id.au \ --cc=aneesh.kumar@linux.ibm.com \ --cc=linux-nvdimm@lists.01.org \ --cc=linuxppc-dev@lists.ozlabs.org \ --cc=mpe@ellerman.id.au \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.