From: Ross Zwisler <ross.zwisler@linux.intel.com> To: linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org, dan.j.williams@intel.com Cc: Ross Zwisler <ross.zwisler@linux.intel.com>, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com>, x86@kernel.org, Juergen Gross <jgross@suse.com>, Borislav Petkov <bp@suse.de>, Toshi Kani <toshi.kani@hp.com>, "Luis R. Rodriguez" <mcgrof@suse.com> Subject: [PATCH 3/6] x86: add clwb_cache_range() Date: Thu, 6 Aug 2015 11:43:17 -0600 [thread overview] Message-ID: <1438883000-9011-4-git-send-email-ross.zwisler@linux.intel.com> (raw) In-Reply-To: <1438883000-9011-1-git-send-email-ross.zwisler@linux.intel.com> Add support for writing back a cache range using CLWB instead of just flushing it using CLFLUSH or CLFLUSHOPT. This allows you to ensure that your data has become durable on your DIMM, but potentially leaves a clean version in the processor cache hierarchy for future loads. This will be used in DAX to write back stores to persistent memory. Details on CLWB can be found here: https://software.intel.com/sites/default/files/managed/0d/53/319433-022.pdf Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com> --- arch/x86/include/asm/cacheflush.h | 1 + arch/x86/mm/pageattr.c | 23 +++++++++++++++++++++++ 2 files changed, 24 insertions(+) diff --git a/arch/x86/include/asm/cacheflush.h b/arch/x86/include/asm/cacheflush.h index ae00766..490b3d6 100644 --- a/arch/x86/include/asm/cacheflush.h +++ b/arch/x86/include/asm/cacheflush.h @@ -88,6 +88,7 @@ int set_pages_rw(struct page *page, int numpages); void clflush_cache_range(void *addr, unsigned int size); +void clwb_cache_range(void *addr, size_t size); #ifdef CONFIG_DEBUG_RODATA void mark_rodata_ro(void); diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index 727158c..ce84d05 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -144,6 +144,29 @@ void clflush_cache_range(void *vaddr, unsigned int size) } EXPORT_SYMBOL_GPL(clflush_cache_range); +/** + * clwb_cache_range - write back a cache range with clwb + * @vaddr: virtual start address + * @size: number of bytes to write back + * + * clwb is an unordered instruction which needs fencing with mfence or sfence + * to avoid ordering issues. + */ +void clwb_cache_range(void *vaddr, size_t size) +{ + u16 x86_clflush_size = boot_cpu_data.x86_clflush_size; + unsigned long clflush_mask = x86_clflush_size - 1; + char *vend = (char *)vaddr + size; + char *p; + + for (p = (char *)((unsigned long)vaddr & ~clflush_mask); + p < vend; p += x86_clflush_size) + clwb(p); + + wmb(); +} +EXPORT_SYMBOL_GPL(clwb_cache_range); + static void __cpa_flush_all(void *arg) { unsigned long cache = (unsigned long)arg; -- 2.1.0
WARNING: multiple messages have this Message-ID (diff)
From: Ross Zwisler <ross.zwisler@linux.intel.com> To: linux-kernel@vger.kernel.org, linux-nvdimm@ml01.01.org, dan.j.williams@intel.com Cc: Ross Zwisler <ross.zwisler@linux.intel.com>, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com>, x86@kernel.org, Juergen Gross <jgross@suse.com>, Borislav Petkov <bp@suse.de>, Toshi Kani <toshi.kani@hp.com>, "Luis R. Rodriguez" <mcgrof@suse.com> Subject: [PATCH 3/6] x86: add clwb_cache_range() Date: Thu, 6 Aug 2015 11:43:17 -0600 [thread overview] Message-ID: <1438883000-9011-4-git-send-email-ross.zwisler@linux.intel.com> (raw) In-Reply-To: <1438883000-9011-1-git-send-email-ross.zwisler@linux.intel.com> Add support for writing back a cache range using CLWB instead of just flushing it using CLFLUSH or CLFLUSHOPT. This allows you to ensure that your data has become durable on your DIMM, but potentially leaves a clean version in the processor cache hierarchy for future loads. This will be used in DAX to write back stores to persistent memory. Details on CLWB can be found here: https://software.intel.com/sites/default/files/managed/0d/53/319433-022.pdf Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com> --- arch/x86/include/asm/cacheflush.h | 1 + arch/x86/mm/pageattr.c | 23 +++++++++++++++++++++++ 2 files changed, 24 insertions(+) diff --git a/arch/x86/include/asm/cacheflush.h b/arch/x86/include/asm/cacheflush.h index ae00766..490b3d6 100644 --- a/arch/x86/include/asm/cacheflush.h +++ b/arch/x86/include/asm/cacheflush.h @@ -88,6 +88,7 @@ int set_pages_rw(struct page *page, int numpages); void clflush_cache_range(void *addr, unsigned int size); +void clwb_cache_range(void *addr, size_t size); #ifdef CONFIG_DEBUG_RODATA void mark_rodata_ro(void); diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index 727158c..ce84d05 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -144,6 +144,29 @@ void clflush_cache_range(void *vaddr, unsigned int size) } EXPORT_SYMBOL_GPL(clflush_cache_range); +/** + * clwb_cache_range - write back a cache range with clwb + * @vaddr: virtual start address + * @size: number of bytes to write back + * + * clwb is an unordered instruction which needs fencing with mfence or sfence + * to avoid ordering issues. + */ +void clwb_cache_range(void *vaddr, size_t size) +{ + u16 x86_clflush_size = boot_cpu_data.x86_clflush_size; + unsigned long clflush_mask = x86_clflush_size - 1; + char *vend = (char *)vaddr + size; + char *p; + + for (p = (char *)((unsigned long)vaddr & ~clflush_mask); + p < vend; p += x86_clflush_size) + clwb(p); + + wmb(); +} +EXPORT_SYMBOL_GPL(clwb_cache_range); + static void __cpa_flush_all(void *arg) { unsigned long cache = (unsigned long)arg; -- 2.1.0
next prev parent reply other threads:[~2015-08-06 17:43 UTC|newest] Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top 2015-08-06 17:43 [PATCH 0/6] pmem, dax: I/O path enhancements Ross Zwisler 2015-08-06 17:43 ` Ross Zwisler 2015-08-06 17:43 ` [PATCH 1/6] pmem: remove indirection layer arch_has_pmem_api() Ross Zwisler 2015-08-06 17:43 ` Ross Zwisler 2015-08-07 6:38 ` Christoph Hellwig 2015-08-07 14:07 ` Ross Zwisler 2015-08-07 16:14 ` Dan Williams 2015-08-07 16:14 ` Dan Williams 2015-08-07 18:41 ` Ross Zwisler 2015-08-07 18:41 ` Ross Zwisler 2015-08-07 20:01 ` Dan Williams 2015-08-07 20:01 ` Dan Williams 2015-08-06 17:43 ` [PATCH 2/6] x86: clean up conditional pmem includes Ross Zwisler 2015-08-06 17:43 ` Ross Zwisler 2015-08-07 6:39 ` Christoph Hellwig 2015-08-07 14:08 ` Ross Zwisler 2015-08-06 17:43 ` Ross Zwisler [this message] 2015-08-06 17:43 ` [PATCH 3/6] x86: add clwb_cache_range() Ross Zwisler 2015-08-06 17:43 ` [PATCH 4/6] pmem: Add wb_cache_pmem() and flush_cache_pmem() Ross Zwisler 2015-08-06 17:43 ` Ross Zwisler 2015-08-06 17:43 ` [PATCH 5/6] nd_blk: add support for "read flush" DSM flag Ross Zwisler 2015-08-06 17:43 ` Ross Zwisler 2015-08-06 17:43 ` [PATCH 6/6] dax: update I/O path to do proper PMEM flushing Ross Zwisler 2015-08-06 17:43 ` Ross Zwisler 2015-08-06 21:04 ` Dave Chinner 2015-08-06 21:04 ` Dave Chinner 2015-08-07 19:08 ` Ross Zwisler 2015-08-07 19:08 ` Ross Zwisler 2015-08-06 21:26 ` Dan Williams 2015-08-06 21:26 ` Dan Williams 2015-08-07 16:47 ` [PATCH 0/6] pmem, dax: I/O path enhancements Dan Williams 2015-08-07 16:47 ` Dan Williams 2015-08-07 19:06 ` Ross Zwisler 2015-08-07 19:06 ` Ross Zwisler
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1438883000-9011-4-git-send-email-ross.zwisler@linux.intel.com \ --to=ross.zwisler@linux.intel.com \ --cc=bp@suse.de \ --cc=dan.j.williams@intel.com \ --cc=hpa@zytor.com \ --cc=jgross@suse.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-nvdimm@lists.01.org \ --cc=mcgrof@suse.com \ --cc=mingo@redhat.com \ --cc=tglx@linutronix.de \ --cc=toshi.kani@hp.com \ --cc=x86@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.