From: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com> To: linuxppc-dev@lists.ozlabs.org, mpe@ellerman.id.au, linux-nvdimm@lists.01.org Cc: alistair@popple.id.au, "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com> Subject: [PATCH v2 5/5] powerpc/pmem: Avoid the barrier in flush routines Date: Wed, 13 May 2020 09:17:05 +0530 [thread overview] Message-ID: <20200513034705.172983-5-aneesh.kumar@linux.ibm.com> (raw) In-Reply-To: <20200513034705.172983-1-aneesh.kumar@linux.ibm.com> nvdimm expect the flush routines to just mark the cache clean. The barrier that mark the store globally visible is done in nvdimm_flush(). Update the papr_scm driver to a simplified nvdim_flush callback that do only the required barrier. Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com> --- arch/powerpc/lib/pmem.c | 34 +++++++++++++++++------ arch/powerpc/platforms/pseries/papr_scm.c | 13 +++++++++ 2 files changed, 39 insertions(+), 8 deletions(-) diff --git a/arch/powerpc/lib/pmem.c b/arch/powerpc/lib/pmem.c index 076d75efb57a..3ef15cfa925b 100644 --- a/arch/powerpc/lib/pmem.c +++ b/arch/powerpc/lib/pmem.c @@ -9,7 +9,7 @@ #include <asm/cacheflush.h> -static inline void clean_pmem_range_isa310(unsigned long start, unsigned long stop) +static inline void __clean_pmem_range(unsigned long start, unsigned long stop) { unsigned long shift = l1_dcache_shift(); unsigned long bytes = l1_dcache_bytes(); @@ -18,13 +18,22 @@ static inline void clean_pmem_range_isa310(unsigned long start, unsigned long st unsigned long i; for (i = 0; i < size >> shift; i++, addr += bytes) - asm volatile(PPC_DCBSTPS(%0, %1): :"i"(0), "r"(addr): "memory"); + dcbf(addr); +} +static inline void __flush_pmem_range(unsigned long start, unsigned long stop) +{ + unsigned long shift = l1_dcache_shift(); + unsigned long bytes = l1_dcache_bytes(); + void *addr = (void *)(start & ~(bytes - 1)); + unsigned long size = stop - (unsigned long)addr + (bytes - 1); + unsigned long i; - asm volatile(PPC_PHWSYNC ::: "memory"); + for (i = 0; i < size >> shift; i++, addr += bytes) + dcbf(addr); } -static inline void flush_pmem_range_isa310(unsigned long start, unsigned long stop) +static inline void clean_pmem_range_isa310(unsigned long start, unsigned long stop) { unsigned long shift = l1_dcache_shift(); unsigned long bytes = l1_dcache_bytes(); @@ -33,24 +42,33 @@ static inline void flush_pmem_range_isa310(unsigned long start, unsigned long st unsigned long i; for (i = 0; i < size >> shift; i++, addr += bytes) - asm volatile(PPC_DCBFPS(%0, %1): :"i"(0), "r"(addr): "memory"); + asm volatile(PPC_DCBSTPS(%0, %1): :"i"(0), "r"(addr): "memory"); +} +static inline void flush_pmem_range_isa310(unsigned long start, unsigned long stop) +{ + unsigned long shift = l1_dcache_shift(); + unsigned long bytes = l1_dcache_bytes(); + void *addr = (void *)(start & ~(bytes - 1)); + unsigned long size = stop - (unsigned long)addr + (bytes - 1); + unsigned long i; - asm volatile(PPC_PHWSYNC ::: "memory"); + for (i = 0; i < size >> shift; i++, addr += bytes) + asm volatile(PPC_DCBFPS(%0, %1): :"i"(0), "r"(addr): "memory"); } static inline void clean_pmem_range(unsigned long start, unsigned long stop) { if (cpu_has_feature(CPU_FTR_ARCH_31)) return clean_pmem_range_isa310(start, stop); - return flush_dcache_range(start, stop); + return __clean_pmem_range(start, stop); } static inline void flush_pmem_range(unsigned long start, unsigned long stop) { if (cpu_has_feature(CPU_FTR_ARCH_31)) return flush_pmem_range_isa310(start, stop); - return flush_dcache_range(start, stop); + return __flush_pmem_range(start, stop); } /* diff --git a/arch/powerpc/platforms/pseries/papr_scm.c b/arch/powerpc/platforms/pseries/papr_scm.c index f35592423380..ad506e7003c9 100644 --- a/arch/powerpc/platforms/pseries/papr_scm.c +++ b/arch/powerpc/platforms/pseries/papr_scm.c @@ -285,6 +285,18 @@ static int papr_scm_ndctl(struct nvdimm_bus_descriptor *nd_desc, return 0; } +/* + * We have made sure the pmem writes are done such that before calling this + * all the caches are flushed/clean. We use dcbf/dcbfps to ensure this. Here + * we just need to add the necessary barrier to make sure the above flushes + * are have updated persistent storage before any data access or data transfer + * caused by subsequent instructions is initiated. + */ +static int papr_scm_flush_sync(struct nd_region *nd_region, struct bio *bio) +{ + arch_pmem_flush_barrier(); + return 0; +} static int papr_scm_nvdimm_init(struct papr_scm_priv *p) { @@ -340,6 +352,7 @@ static int papr_scm_nvdimm_init(struct papr_scm_priv *p) ndr_desc.mapping = &mapping; ndr_desc.num_mappings = 1; ndr_desc.nd_set = &p->nd_set; + ndr_desc.flush = papr_scm_flush_sync; if (p->is_volatile) p->region = nvdimm_volatile_region_create(p->bus, &ndr_desc); -- 2.26.2 _______________________________________________ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
WARNING: multiple messages have this Message-ID (diff)
From: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com> To: linuxppc-dev@lists.ozlabs.org, mpe@ellerman.id.au, linux-nvdimm@lists.01.org Cc: alistair@popple.id.au, dan.j.williams@intel.com, oohall@gmail.com, "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com> Subject: [PATCH v2 5/5] powerpc/pmem: Avoid the barrier in flush routines Date: Wed, 13 May 2020 09:17:05 +0530 [thread overview] Message-ID: <20200513034705.172983-5-aneesh.kumar@linux.ibm.com> (raw) In-Reply-To: <20200513034705.172983-1-aneesh.kumar@linux.ibm.com> nvdimm expect the flush routines to just mark the cache clean. The barrier that mark the store globally visible is done in nvdimm_flush(). Update the papr_scm driver to a simplified nvdim_flush callback that do only the required barrier. Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com> --- arch/powerpc/lib/pmem.c | 34 +++++++++++++++++------ arch/powerpc/platforms/pseries/papr_scm.c | 13 +++++++++ 2 files changed, 39 insertions(+), 8 deletions(-) diff --git a/arch/powerpc/lib/pmem.c b/arch/powerpc/lib/pmem.c index 076d75efb57a..3ef15cfa925b 100644 --- a/arch/powerpc/lib/pmem.c +++ b/arch/powerpc/lib/pmem.c @@ -9,7 +9,7 @@ #include <asm/cacheflush.h> -static inline void clean_pmem_range_isa310(unsigned long start, unsigned long stop) +static inline void __clean_pmem_range(unsigned long start, unsigned long stop) { unsigned long shift = l1_dcache_shift(); unsigned long bytes = l1_dcache_bytes(); @@ -18,13 +18,22 @@ static inline void clean_pmem_range_isa310(unsigned long start, unsigned long st unsigned long i; for (i = 0; i < size >> shift; i++, addr += bytes) - asm volatile(PPC_DCBSTPS(%0, %1): :"i"(0), "r"(addr): "memory"); + dcbf(addr); +} +static inline void __flush_pmem_range(unsigned long start, unsigned long stop) +{ + unsigned long shift = l1_dcache_shift(); + unsigned long bytes = l1_dcache_bytes(); + void *addr = (void *)(start & ~(bytes - 1)); + unsigned long size = stop - (unsigned long)addr + (bytes - 1); + unsigned long i; - asm volatile(PPC_PHWSYNC ::: "memory"); + for (i = 0; i < size >> shift; i++, addr += bytes) + dcbf(addr); } -static inline void flush_pmem_range_isa310(unsigned long start, unsigned long stop) +static inline void clean_pmem_range_isa310(unsigned long start, unsigned long stop) { unsigned long shift = l1_dcache_shift(); unsigned long bytes = l1_dcache_bytes(); @@ -33,24 +42,33 @@ static inline void flush_pmem_range_isa310(unsigned long start, unsigned long st unsigned long i; for (i = 0; i < size >> shift; i++, addr += bytes) - asm volatile(PPC_DCBFPS(%0, %1): :"i"(0), "r"(addr): "memory"); + asm volatile(PPC_DCBSTPS(%0, %1): :"i"(0), "r"(addr): "memory"); +} +static inline void flush_pmem_range_isa310(unsigned long start, unsigned long stop) +{ + unsigned long shift = l1_dcache_shift(); + unsigned long bytes = l1_dcache_bytes(); + void *addr = (void *)(start & ~(bytes - 1)); + unsigned long size = stop - (unsigned long)addr + (bytes - 1); + unsigned long i; - asm volatile(PPC_PHWSYNC ::: "memory"); + for (i = 0; i < size >> shift; i++, addr += bytes) + asm volatile(PPC_DCBFPS(%0, %1): :"i"(0), "r"(addr): "memory"); } static inline void clean_pmem_range(unsigned long start, unsigned long stop) { if (cpu_has_feature(CPU_FTR_ARCH_31)) return clean_pmem_range_isa310(start, stop); - return flush_dcache_range(start, stop); + return __clean_pmem_range(start, stop); } static inline void flush_pmem_range(unsigned long start, unsigned long stop) { if (cpu_has_feature(CPU_FTR_ARCH_31)) return flush_pmem_range_isa310(start, stop); - return flush_dcache_range(start, stop); + return __flush_pmem_range(start, stop); } /* diff --git a/arch/powerpc/platforms/pseries/papr_scm.c b/arch/powerpc/platforms/pseries/papr_scm.c index f35592423380..ad506e7003c9 100644 --- a/arch/powerpc/platforms/pseries/papr_scm.c +++ b/arch/powerpc/platforms/pseries/papr_scm.c @@ -285,6 +285,18 @@ static int papr_scm_ndctl(struct nvdimm_bus_descriptor *nd_desc, return 0; } +/* + * We have made sure the pmem writes are done such that before calling this + * all the caches are flushed/clean. We use dcbf/dcbfps to ensure this. Here + * we just need to add the necessary barrier to make sure the above flushes + * are have updated persistent storage before any data access or data transfer + * caused by subsequent instructions is initiated. + */ +static int papr_scm_flush_sync(struct nd_region *nd_region, struct bio *bio) +{ + arch_pmem_flush_barrier(); + return 0; +} static int papr_scm_nvdimm_init(struct papr_scm_priv *p) { @@ -340,6 +352,7 @@ static int papr_scm_nvdimm_init(struct papr_scm_priv *p) ndr_desc.mapping = &mapping; ndr_desc.num_mappings = 1; ndr_desc.nd_set = &p->nd_set; + ndr_desc.flush = papr_scm_flush_sync; if (p->is_volatile) p->region = nvdimm_volatile_region_create(p->bus, &ndr_desc); -- 2.26.2
next prev parent reply other threads:[~2020-05-13 3:48 UTC|newest] Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-05-13 3:47 [PATCH v2 1/5] powerpc/pmem: Add new instructions for persistent storage and sync Aneesh Kumar K.V 2020-05-13 3:47 ` Aneesh Kumar K.V 2020-05-13 3:47 ` [PATCH v2 2/5] powerpc/pmem: Add flush routines using new pmem store and sync instruction Aneesh Kumar K.V 2020-05-13 3:47 ` Aneesh Kumar K.V 2020-05-13 3:47 ` [PATCH v2 3/5] libnvdimm/nvdimm/flush: Allow architecture to override the flush barrier Aneesh Kumar K.V 2020-05-13 3:47 ` Aneesh Kumar K.V 2020-05-13 16:14 ` Dan Williams 2020-05-13 16:14 ` Dan Williams 2020-05-19 5:30 ` Aneesh Kumar K.V 2020-05-19 5:30 ` Aneesh Kumar K.V 2020-05-19 7:09 ` Dan Williams 2020-05-19 7:09 ` Dan Williams 2020-05-19 13:52 ` Aneesh Kumar K.V 2020-05-19 13:52 ` Aneesh Kumar K.V 2020-05-19 18:59 ` Dan Williams 2020-05-19 18:59 ` Dan Williams 2020-05-20 18:43 ` Aneesh Kumar K.V 2020-05-20 18:43 ` Aneesh Kumar K.V 2020-05-21 14:38 ` Jeff Moyer 2020-05-21 14:38 ` Jeff Moyer 2020-05-21 17:02 ` Aneesh Kumar K.V 2020-05-21 17:02 ` Aneesh Kumar K.V 2020-05-21 18:25 ` Dan Williams 2020-05-21 18:25 ` Dan Williams 2020-05-21 18:52 ` Mikulas Patocka 2020-05-21 18:52 ` Mikulas Patocka 2020-05-22 9:31 ` Michal Suchánek 2020-05-22 9:31 ` Michal Suchánek 2020-05-22 10:08 ` Aneesh Kumar K.V 2020-05-22 10:08 ` Aneesh Kumar K.V 2020-05-22 13:01 ` Mikulas Patocka 2020-05-22 13:01 ` Mikulas Patocka 2020-06-26 10:20 ` Michal Suchánek 2020-06-26 10:20 ` Michal Suchánek 2020-05-21 18:34 ` Dan Williams 2020-05-21 18:34 ` Dan Williams 2020-05-13 3:47 ` [PATCH v2 4/5] powerpc/pmem/of_pmem: Update of_pmem to use the new barrier instruction Aneesh Kumar K.V 2020-05-13 3:47 ` Aneesh Kumar K.V 2020-05-13 6:44 ` kbuild test robot 2020-05-13 6:44 ` kbuild test robot 2020-05-13 6:44 ` kbuild test robot 2020-05-13 3:47 ` Aneesh Kumar K.V [this message] 2020-05-13 3:47 ` [PATCH v2 5/5] powerpc/pmem: Avoid the barrier in flush routines Aneesh Kumar K.V
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200513034705.172983-5-aneesh.kumar@linux.ibm.com \ --to=aneesh.kumar@linux.ibm.com \ --cc=alistair@popple.id.au \ --cc=linux-nvdimm@lists.01.org \ --cc=linuxppc-dev@lists.ozlabs.org \ --cc=mpe@ellerman.id.au \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.