From: Tamas K Lengyel <tamas@tklengyel.com> To: xen-devel@lists.xenproject.org Cc: Tamas K Lengyel <tamas@tklengyel.com>, Wei Liu <wei.liu2@citrix.com>, George Dunlap <george.dunlap@eu.citrix.com>, Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich <jbeulich@suse.com>, Roger Pau Monne <roger.pau@citrix.com> Subject: [PATCH v5 2/4] x86/mem_sharing: copy a page_lock version to be internal to memshr Date: Thu, 16 May 2019 15:37:50 -0600 [thread overview] Message-ID: <20190516213752.1701-2-tamas@tklengyel.com> (raw) In-Reply-To: <20190516213752.1701-1-tamas@tklengyel.com> Patch cf4b30dca0a "Add debug code to detect illegal page_lock and put_page_type ordering" added extra sanity checking to page_lock/page_unlock for debug builds with the assumption that no hypervisor path ever locks two pages at once. This assumption doesn't hold during memory sharing so we copy a version of page_lock/unlock to be used exclusively in the memory sharing subsystem without the sanity checks. Signed-off-by: Tamas K Lengyel <tamas@tklengyel.com> Cc: George Dunlap <george.dunlap@eu.citrix.com> Cc: Jan Beulich <jbeulich@suse.com> Cc: Andrew Cooper <andrew.cooper3@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Roger Pau Monne <roger.pau@citrix.com> --- v5: comments and TODOs --- xen/arch/x86/mm/mem_sharing.c | 54 ++++++++++++++++++++++++++++++++--- xen/include/asm-x86/mm.h | 15 ++-------- 2 files changed, 53 insertions(+), 16 deletions(-) diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c index 13b2f009d4..f2354d2d6f 100644 --- a/xen/arch/x86/mm/mem_sharing.c +++ b/xen/arch/x86/mm/mem_sharing.c @@ -112,13 +112,59 @@ static inline void page_sharing_dispose(struct page_info *page) #endif /* MEM_SHARING_AUDIT */ -static inline int mem_sharing_page_lock(struct page_info *pg) +/* + * Private implementations of page_lock/unlock to bypass PV-only + * sanity checks not applicable to mem-sharing. + * + * _page_lock is used in memory sharing to protect addition (share) and removal + * (unshare) of (gfn,domain) tupples to a list of gfn's that the shared page is + * currently backing. + * Nesting may happen when sharing (and locking) two pages. + * Deadlock is avoided by locking pages in increasing order. + * All memory sharing code paths take the p2m lock of the affected gfn before + * taking the lock for the underlying page. We enforce ordering between page_lock + * and p2m_lock using an mm-locks.h construct. + * + * TODO: Investigate if PGT_validated is necessary. + */ +static inline bool _page_lock(struct page_info *page) { - int rc; + unsigned long x, nx; + + do { + while ( (x = page->u.inuse.type_info) & PGT_locked ) + cpu_relax(); + nx = x + (1 | PGT_locked); + if ( !(x & PGT_validated) || + !(x & PGT_count_mask) || + !(nx & PGT_count_mask) ) + return false; + } while ( cmpxchg(&page->u.inuse.type_info, x, nx) != x ); + + return true; +} + +static inline void _page_unlock(struct page_info *page) +{ + unsigned long x, nx, y = page->u.inuse.type_info; + + do { + x = y; + ASSERT((x & PGT_count_mask) && (x & PGT_locked)); + + nx = x - (1 | PGT_locked); + /* We must not drop the last reference here. */ + ASSERT(nx & PGT_count_mask); + } while ( (y = cmpxchg(&page->u.inuse.type_info, x, nx)) != x ); +} + +static inline bool mem_sharing_page_lock(struct page_info *pg) +{ + bool rc; pg_lock_data_t *pld = &(this_cpu(__pld)); page_sharing_mm_pre_lock(); - rc = page_lock(pg); + rc = _page_lock(pg); if ( rc ) { preempt_disable(); @@ -135,7 +181,7 @@ static inline void mem_sharing_page_unlock(struct page_info *pg) page_sharing_mm_unlock(pld->mm_unlock_level, &pld->recurse_count); preempt_enable(); - page_unlock(pg); + _page_unlock(pg); } static inline shr_handle_t get_next_handle(void) diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h index 6faa563167..24c4205ba7 100644 --- a/xen/include/asm-x86/mm.h +++ b/xen/include/asm-x86/mm.h @@ -356,24 +356,15 @@ struct platform_bad_page { const struct platform_bad_page *get_platform_badpages(unsigned int *array_size); /* Per page locks: - * page_lock() is used for two purposes: pte serialization, and memory sharing. + * page_lock() is used for pte serialization. * * All users of page lock for pte serialization live in mm.c, use it * to lock a page table page during pte updates, do not take other locks within * the critical section delimited by page_lock/unlock, and perform no * nesting. * - * All users of page lock for memory sharing live in mm/mem_sharing.c. Page_lock - * is used in memory sharing to protect addition (share) and removal (unshare) - * of (gfn,domain) tupples to a list of gfn's that the shared page is currently - * backing. Nesting may happen when sharing (and locking) two pages -- deadlock - * is avoided by locking pages in increasing order. - * All memory sharing code paths take the p2m lock of the affected gfn before - * taking the lock for the underlying page. We enforce ordering between page_lock - * and p2m_lock using an mm-locks.h construct. - * - * These two users (pte serialization and memory sharing) do not collide, since - * sharing is only supported for hvm guests, which do not perform pv pte updates. + * The use of PGT_locked in mem_sharing does not collide, since mem_sharing is + * only supported for hvm guests, which do not perform pv pte updates. */ int page_lock(struct page_info *page); void page_unlock(struct page_info *page); -- 2.20.1 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
WARNING: multiple messages have this Message-ID (diff)
From: Tamas K Lengyel <tamas@tklengyel.com> To: xen-devel@lists.xenproject.org Cc: Tamas K Lengyel <tamas@tklengyel.com>, Wei Liu <wei.liu2@citrix.com>, George Dunlap <george.dunlap@eu.citrix.com>, Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich <jbeulich@suse.com>, Roger Pau Monne <roger.pau@citrix.com> Subject: [Xen-devel] [PATCH v5 2/4] x86/mem_sharing: copy a page_lock version to be internal to memshr Date: Thu, 16 May 2019 15:37:50 -0600 [thread overview] Message-ID: <20190516213752.1701-2-tamas@tklengyel.com> (raw) Message-ID: <20190516213750.9r5uqlu8DEAIAIpMyACOhVGjUNw_bBpG01GjBR7K4q4@z> (raw) In-Reply-To: <20190516213752.1701-1-tamas@tklengyel.com> Patch cf4b30dca0a "Add debug code to detect illegal page_lock and put_page_type ordering" added extra sanity checking to page_lock/page_unlock for debug builds with the assumption that no hypervisor path ever locks two pages at once. This assumption doesn't hold during memory sharing so we copy a version of page_lock/unlock to be used exclusively in the memory sharing subsystem without the sanity checks. Signed-off-by: Tamas K Lengyel <tamas@tklengyel.com> Cc: George Dunlap <george.dunlap@eu.citrix.com> Cc: Jan Beulich <jbeulich@suse.com> Cc: Andrew Cooper <andrew.cooper3@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Roger Pau Monne <roger.pau@citrix.com> --- v5: comments and TODOs --- xen/arch/x86/mm/mem_sharing.c | 54 ++++++++++++++++++++++++++++++++--- xen/include/asm-x86/mm.h | 15 ++-------- 2 files changed, 53 insertions(+), 16 deletions(-) diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c index 13b2f009d4..f2354d2d6f 100644 --- a/xen/arch/x86/mm/mem_sharing.c +++ b/xen/arch/x86/mm/mem_sharing.c @@ -112,13 +112,59 @@ static inline void page_sharing_dispose(struct page_info *page) #endif /* MEM_SHARING_AUDIT */ -static inline int mem_sharing_page_lock(struct page_info *pg) +/* + * Private implementations of page_lock/unlock to bypass PV-only + * sanity checks not applicable to mem-sharing. + * + * _page_lock is used in memory sharing to protect addition (share) and removal + * (unshare) of (gfn,domain) tupples to a list of gfn's that the shared page is + * currently backing. + * Nesting may happen when sharing (and locking) two pages. + * Deadlock is avoided by locking pages in increasing order. + * All memory sharing code paths take the p2m lock of the affected gfn before + * taking the lock for the underlying page. We enforce ordering between page_lock + * and p2m_lock using an mm-locks.h construct. + * + * TODO: Investigate if PGT_validated is necessary. + */ +static inline bool _page_lock(struct page_info *page) { - int rc; + unsigned long x, nx; + + do { + while ( (x = page->u.inuse.type_info) & PGT_locked ) + cpu_relax(); + nx = x + (1 | PGT_locked); + if ( !(x & PGT_validated) || + !(x & PGT_count_mask) || + !(nx & PGT_count_mask) ) + return false; + } while ( cmpxchg(&page->u.inuse.type_info, x, nx) != x ); + + return true; +} + +static inline void _page_unlock(struct page_info *page) +{ + unsigned long x, nx, y = page->u.inuse.type_info; + + do { + x = y; + ASSERT((x & PGT_count_mask) && (x & PGT_locked)); + + nx = x - (1 | PGT_locked); + /* We must not drop the last reference here. */ + ASSERT(nx & PGT_count_mask); + } while ( (y = cmpxchg(&page->u.inuse.type_info, x, nx)) != x ); +} + +static inline bool mem_sharing_page_lock(struct page_info *pg) +{ + bool rc; pg_lock_data_t *pld = &(this_cpu(__pld)); page_sharing_mm_pre_lock(); - rc = page_lock(pg); + rc = _page_lock(pg); if ( rc ) { preempt_disable(); @@ -135,7 +181,7 @@ static inline void mem_sharing_page_unlock(struct page_info *pg) page_sharing_mm_unlock(pld->mm_unlock_level, &pld->recurse_count); preempt_enable(); - page_unlock(pg); + _page_unlock(pg); } static inline shr_handle_t get_next_handle(void) diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h index 6faa563167..24c4205ba7 100644 --- a/xen/include/asm-x86/mm.h +++ b/xen/include/asm-x86/mm.h @@ -356,24 +356,15 @@ struct platform_bad_page { const struct platform_bad_page *get_platform_badpages(unsigned int *array_size); /* Per page locks: - * page_lock() is used for two purposes: pte serialization, and memory sharing. + * page_lock() is used for pte serialization. * * All users of page lock for pte serialization live in mm.c, use it * to lock a page table page during pte updates, do not take other locks within * the critical section delimited by page_lock/unlock, and perform no * nesting. * - * All users of page lock for memory sharing live in mm/mem_sharing.c. Page_lock - * is used in memory sharing to protect addition (share) and removal (unshare) - * of (gfn,domain) tupples to a list of gfn's that the shared page is currently - * backing. Nesting may happen when sharing (and locking) two pages -- deadlock - * is avoided by locking pages in increasing order. - * All memory sharing code paths take the p2m lock of the affected gfn before - * taking the lock for the underlying page. We enforce ordering between page_lock - * and p2m_lock using an mm-locks.h construct. - * - * These two users (pte serialization and memory sharing) do not collide, since - * sharing is only supported for hvm guests, which do not perform pv pte updates. + * The use of PGT_locked in mem_sharing does not collide, since mem_sharing is + * only supported for hvm guests, which do not perform pv pte updates. */ int page_lock(struct page_info *page); void page_unlock(struct page_info *page); -- 2.20.1 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
next prev parent reply other threads:[~2019-05-16 21:38 UTC|newest] Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-05-16 21:37 [PATCH v5 1/4] x86/mem_sharing: reorder when pages are unlocked and released Tamas K Lengyel 2019-05-16 21:37 ` [Xen-devel] " Tamas K Lengyel 2019-05-16 21:37 ` Tamas K Lengyel [this message] 2019-05-16 21:37 ` [Xen-devel] [PATCH v5 2/4] x86/mem_sharing: copy a page_lock version to be internal to memshr Tamas K Lengyel 2019-05-17 7:21 ` Jan Beulich 2019-05-17 7:21 ` [Xen-devel] " Jan Beulich 2019-05-17 20:04 ` Tamas K Lengyel 2019-05-17 20:04 ` [Xen-devel] " Tamas K Lengyel 2019-06-17 12:21 ` Tamas K Lengyel 2019-05-16 21:37 ` [PATCH v5 3/4] x86/mem_sharing: enable mem_share audit mode only in debug builds Tamas K Lengyel 2019-05-16 21:37 ` [Xen-devel] " Tamas K Lengyel 2019-06-17 12:24 ` Tamas K Lengyel 2019-05-16 21:37 ` [PATCH v5 4/4] x86/mem_sharing: compile mem_sharing subsystem only when kconfig is enabled Tamas K Lengyel 2019-05-16 21:37 ` [Xen-devel] " Tamas K Lengyel 2019-05-17 7:23 ` Jan Beulich 2019-05-17 7:23 ` [Xen-devel] " Jan Beulich 2019-06-03 8:26 ` Jan Beulich 2019-06-03 8:26 ` [Xen-devel] " Jan Beulich 2019-06-03 16:38 ` Tamas K Lengyel 2019-06-03 16:38 ` [Xen-devel] " Tamas K Lengyel 2019-06-03 16:40 ` Julien Grall 2019-06-03 16:40 ` [Xen-devel] " Julien Grall 2019-06-03 16:55 ` Tamas K Lengyel 2019-06-03 16:55 ` [Xen-devel] " Tamas K Lengyel 2019-06-04 8:41 ` Razvan Cojocaru 2019-06-04 8:41 ` [Xen-devel] " Razvan Cojocaru 2019-06-04 14:36 ` Daniel De Graaf 2019-06-17 12:17 ` Tamas K Lengyel 2019-06-17 12:23 ` [Xen-devel] [PATCH v5 1/4] x86/mem_sharing: reorder when pages are unlocked and released Tamas K Lengyel 2019-06-17 13:46 ` Jan Beulich
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190516213752.1701-2-tamas@tklengyel.com \ --to=tamas@tklengyel.com \ --cc=andrew.cooper3@citrix.com \ --cc=george.dunlap@eu.citrix.com \ --cc=jbeulich@suse.com \ --cc=roger.pau@citrix.com \ --cc=wei.liu2@citrix.com \ --cc=xen-devel@lists.xenproject.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).