From: Byungchul Park <byungchul.park@lge.com> To: <peterz@infradead.org>, <mingo@kernel.org> Cc: <tglx@linutronix.de>, <walken@google.com>, <boqun.feng@gmail.com>, <kirill@shutemov.name>, <linux-kernel@vger.kernel.org>, <linux-mm@kvack.org>, <iamjoonsoo.kim@lge.com>, <akpm@linux-foundation.org>, <willy@infradead.org>, <npiggin@gmail.com>, <kernel-team@lge.com> Subject: [PATCH v6 06/15] lockdep: Handle non(or multi)-acquisition of a crosslock Date: Tue, 14 Mar 2017 17:18:53 +0900 [thread overview] Message-ID: <1489479542-27030-7-git-send-email-byungchul.park@lge.com> (raw) In-Reply-To: <1489479542-27030-1-git-send-email-byungchul.park@lge.com> No acquisition might be in progress on commit of a crosslock. Completion operations enabling crossrelease are the case like: CONTEXT X CONTEXT Y --------- --------- trigger completion context complete AX commit AX wait_for_complete AX acquire AX wait where AX is a crosslock. When no acquisition is in progress, we should not perform commit because the lock does not exist, which might cause incorrect memory access. So we have to track the number of acquisitions of a crosslock to handle it. Moreover, in case that more than one acquisition of a crosslock are overlapped like: CONTEXT W CONTEXT X CONTEXT Y CONTEXT Z --------- --------- --------- --------- acquire AX (gen_id: 1) acquire A acquire AX (gen_id: 10) acquire B commit AX acquire C commit AX where A, B and C are typical locks and AX is a crosslock. Current crossrelease code performs commits in Y and Z with gen_id = 10. However, we can use gen_id = 1 to do it, since not only 'acquire AX in X' but 'acquire AX in W' also depends on each acquisition in Y and Z until their commits. So make it use gen_id = 1 instead of 10 on their commits, which adds an additional dependency 'AX -> A' in the example above. Signed-off-by: Byungchul Park <byungchul.park@lge.com> --- include/linux/lockdep.h | 22 +++++++++++++++++++- kernel/locking/lockdep.c | 52 ++++++++++++++++++++++++++++++++++++++---------- 2 files changed, 62 insertions(+), 12 deletions(-) diff --git a/include/linux/lockdep.h b/include/linux/lockdep.h index 9902b2a..5356f71 100644 --- a/include/linux/lockdep.h +++ b/include/linux/lockdep.h @@ -312,6 +312,19 @@ struct hist_lock { */ struct cross_lock { /* + * When more than one acquisition of crosslocks are overlapped, + * we have to perform commit for them based on cross_gen_id of + * the first acquisition, which allows us to add more true + * dependencies. + * + * Moreover, when no acquisition of a crosslock is in progress, + * we should not perform commit because the lock might not exist + * any more, which might cause incorrect memory access. So we + * have to track the number of acquisitions of a crosslock. + */ + int nr_acquire; + + /* * Seperate hlock instance. This will be used at commit step. * * TODO: Use a smaller data structure containing only necessary @@ -510,9 +523,16 @@ extern void lockdep_init_map_crosslock(struct lockdep_map *lock, int subclass); extern void lock_commit_crosslock(struct lockdep_map *lock); +/* + * What we essencially have to initialize is 'nr_acquire'. Other members + * will be initialized in add_xlock(). + */ +#define STATIC_CROSS_LOCK_INIT() \ + { .nr_acquire = 0,} + #define STATIC_CROSS_LOCKDEP_MAP_INIT(_name, _key) \ { .map.name = (_name), .map.key = (void *)(_key), \ - .map.cross = 1, } + .map.cross = 1, .xlock = STATIC_CROSS_LOCK_INIT(), } /* * To initialize a lockdep_map statically use this macro. diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c index db15fce..ec4f6af 100644 --- a/kernel/locking/lockdep.c +++ b/kernel/locking/lockdep.c @@ -4780,11 +4780,28 @@ static int add_xlock(struct held_lock *hlock) xlock = &((struct lockdep_map_cross *)hlock->instance)->xlock; + /* + * When acquisitions for a crosslock are overlapped, we use + * nr_acquire to perform commit for them, based on cross_gen_id + * of the first acquisition, which allows to add additional + * dependencies. + * + * Moreover, when no acquisition of a crosslock is in progress, + * we should not perform commit because the lock might not exist + * any more, which might cause incorrect memory access. So we + * have to track the number of acquisitions of a crosslock. + * + * depend_after() is necessary to initialize only the first + * valid xlock so that the xlock can be used on its commit. + */ + if (xlock->nr_acquire++ && depend_after(&xlock->hlock)) + goto unlock; + gen_id = (unsigned int)atomic_inc_return(&cross_gen_id); xlock->hlock = *hlock; xlock->hlock.gen_id = gen_id; +unlock: graph_unlock(); - return 1; } @@ -4874,18 +4891,20 @@ static int commit_xhlocks(struct cross_lock *xlock) if (!graph_lock()) return 0; - for (i = cur - 1; !xhlock_same(i, cur); i--) { - struct hist_lock *xhlock = &xhlock(i); + if (xlock->nr_acquire) { + for (i = cur - 1; !xhlock_same(i, cur); i--) { + struct hist_lock *xhlock = &xhlock(i); - if (!xhlock_used(xhlock)) - break; + if (!xhlock_used(xhlock)) + break; - if (before(xhlock->hlock.gen_id, xlock->hlock.gen_id)) - break; + if (before(xhlock->hlock.gen_id, xlock->hlock.gen_id)) + break; - if (same_context_xhlock(xhlock) && - !commit_xhlock(xlock, xhlock)) - return 0; + if (same_context_xhlock(xhlock) && + !commit_xhlock(xlock, xhlock)) + return 0; + } } graph_unlock(); @@ -4923,16 +4942,27 @@ void lock_commit_crosslock(struct lockdep_map *lock) EXPORT_SYMBOL_GPL(lock_commit_crosslock); /* + * return 0: Stop. Failed to acquire graph_lock. * return 1: Done. No more release ops is needed. * return 2: Need to do normal release operation. */ static int lock_release_crosslock(struct lockdep_map *lock) { - return cross_lock(lock) ? 1 : 2; + if (cross_lock(lock)) { + if (!graph_lock()) + return 0; + ((struct lockdep_map_cross *)lock)->xlock.nr_acquire--; + graph_unlock(); + return 1; + } + return 2; } static void cross_init(struct lockdep_map *lock, int cross) { + if (cross) + ((struct lockdep_map_cross *)lock)->xlock.nr_acquire = 0; + lock->cross = cross; /* -- 1.9.1
WARNING: multiple messages have this Message-ID (diff)
From: Byungchul Park <byungchul.park@lge.com> To: peterz@infradead.org, mingo@kernel.org Cc: tglx@linutronix.de, walken@google.com, boqun.feng@gmail.com, kirill@shutemov.name, linux-kernel@vger.kernel.org, linux-mm@kvack.org, iamjoonsoo.kim@lge.com, akpm@linux-foundation.org, willy@infradead.org, npiggin@gmail.com, kernel-team@lge.com Subject: [PATCH v6 06/15] lockdep: Handle non(or multi)-acquisition of a crosslock Date: Tue, 14 Mar 2017 17:18:53 +0900 [thread overview] Message-ID: <1489479542-27030-7-git-send-email-byungchul.park@lge.com> (raw) In-Reply-To: <1489479542-27030-1-git-send-email-byungchul.park@lge.com> No acquisition might be in progress on commit of a crosslock. Completion operations enabling crossrelease are the case like: CONTEXT X CONTEXT Y --------- --------- trigger completion context complete AX commit AX wait_for_complete AX acquire AX wait where AX is a crosslock. When no acquisition is in progress, we should not perform commit because the lock does not exist, which might cause incorrect memory access. So we have to track the number of acquisitions of a crosslock to handle it. Moreover, in case that more than one acquisition of a crosslock are overlapped like: CONTEXT W CONTEXT X CONTEXT Y CONTEXT Z --------- --------- --------- --------- acquire AX (gen_id: 1) acquire A acquire AX (gen_id: 10) acquire B commit AX acquire C commit AX where A, B and C are typical locks and AX is a crosslock. Current crossrelease code performs commits in Y and Z with gen_id = 10. However, we can use gen_id = 1 to do it, since not only 'acquire AX in X' but 'acquire AX in W' also depends on each acquisition in Y and Z until their commits. So make it use gen_id = 1 instead of 10 on their commits, which adds an additional dependency 'AX -> A' in the example above. Signed-off-by: Byungchul Park <byungchul.park@lge.com> --- include/linux/lockdep.h | 22 +++++++++++++++++++- kernel/locking/lockdep.c | 52 ++++++++++++++++++++++++++++++++++++++---------- 2 files changed, 62 insertions(+), 12 deletions(-) diff --git a/include/linux/lockdep.h b/include/linux/lockdep.h index 9902b2a..5356f71 100644 --- a/include/linux/lockdep.h +++ b/include/linux/lockdep.h @@ -312,6 +312,19 @@ struct hist_lock { */ struct cross_lock { /* + * When more than one acquisition of crosslocks are overlapped, + * we have to perform commit for them based on cross_gen_id of + * the first acquisition, which allows us to add more true + * dependencies. + * + * Moreover, when no acquisition of a crosslock is in progress, + * we should not perform commit because the lock might not exist + * any more, which might cause incorrect memory access. So we + * have to track the number of acquisitions of a crosslock. + */ + int nr_acquire; + + /* * Seperate hlock instance. This will be used at commit step. * * TODO: Use a smaller data structure containing only necessary @@ -510,9 +523,16 @@ extern void lockdep_init_map_crosslock(struct lockdep_map *lock, int subclass); extern void lock_commit_crosslock(struct lockdep_map *lock); +/* + * What we essencially have to initialize is 'nr_acquire'. Other members + * will be initialized in add_xlock(). + */ +#define STATIC_CROSS_LOCK_INIT() \ + { .nr_acquire = 0,} + #define STATIC_CROSS_LOCKDEP_MAP_INIT(_name, _key) \ { .map.name = (_name), .map.key = (void *)(_key), \ - .map.cross = 1, } + .map.cross = 1, .xlock = STATIC_CROSS_LOCK_INIT(), } /* * To initialize a lockdep_map statically use this macro. diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c index db15fce..ec4f6af 100644 --- a/kernel/locking/lockdep.c +++ b/kernel/locking/lockdep.c @@ -4780,11 +4780,28 @@ static int add_xlock(struct held_lock *hlock) xlock = &((struct lockdep_map_cross *)hlock->instance)->xlock; + /* + * When acquisitions for a crosslock are overlapped, we use + * nr_acquire to perform commit for them, based on cross_gen_id + * of the first acquisition, which allows to add additional + * dependencies. + * + * Moreover, when no acquisition of a crosslock is in progress, + * we should not perform commit because the lock might not exist + * any more, which might cause incorrect memory access. So we + * have to track the number of acquisitions of a crosslock. + * + * depend_after() is necessary to initialize only the first + * valid xlock so that the xlock can be used on its commit. + */ + if (xlock->nr_acquire++ && depend_after(&xlock->hlock)) + goto unlock; + gen_id = (unsigned int)atomic_inc_return(&cross_gen_id); xlock->hlock = *hlock; xlock->hlock.gen_id = gen_id; +unlock: graph_unlock(); - return 1; } @@ -4874,18 +4891,20 @@ static int commit_xhlocks(struct cross_lock *xlock) if (!graph_lock()) return 0; - for (i = cur - 1; !xhlock_same(i, cur); i--) { - struct hist_lock *xhlock = &xhlock(i); + if (xlock->nr_acquire) { + for (i = cur - 1; !xhlock_same(i, cur); i--) { + struct hist_lock *xhlock = &xhlock(i); - if (!xhlock_used(xhlock)) - break; + if (!xhlock_used(xhlock)) + break; - if (before(xhlock->hlock.gen_id, xlock->hlock.gen_id)) - break; + if (before(xhlock->hlock.gen_id, xlock->hlock.gen_id)) + break; - if (same_context_xhlock(xhlock) && - !commit_xhlock(xlock, xhlock)) - return 0; + if (same_context_xhlock(xhlock) && + !commit_xhlock(xlock, xhlock)) + return 0; + } } graph_unlock(); @@ -4923,16 +4942,27 @@ void lock_commit_crosslock(struct lockdep_map *lock) EXPORT_SYMBOL_GPL(lock_commit_crosslock); /* + * return 0: Stop. Failed to acquire graph_lock. * return 1: Done. No more release ops is needed. * return 2: Need to do normal release operation. */ static int lock_release_crosslock(struct lockdep_map *lock) { - return cross_lock(lock) ? 1 : 2; + if (cross_lock(lock)) { + if (!graph_lock()) + return 0; + ((struct lockdep_map_cross *)lock)->xlock.nr_acquire--; + graph_unlock(); + return 1; + } + return 2; } static void cross_init(struct lockdep_map *lock, int cross) { + if (cross) + ((struct lockdep_map_cross *)lock)->xlock.nr_acquire = 0; + lock->cross = cross; /* -- 1.9.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2017-03-14 8:27 UTC|newest] Thread overview: 66+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-03-14 8:18 [PATCH v6 00/15] lockdep: Implement crossrelease feature Byungchul Park 2017-03-14 8:18 ` Byungchul Park 2017-03-14 8:18 ` [PATCH v6 01/15] lockdep: Refactor lookup_chain_cache() Byungchul Park 2017-03-14 8:18 ` Byungchul Park 2017-03-14 8:18 ` [PATCH v6 02/15] lockdep: Add a function building a chain between two classes Byungchul Park 2017-03-14 8:18 ` Byungchul Park 2017-03-14 8:18 ` [PATCH v6 03/15] lockdep: Change the meaning of check_prev_add()'s return value Byungchul Park 2017-03-14 8:18 ` Byungchul Park 2017-03-14 8:18 ` [PATCH v6 04/15] lockdep: Make check_prev_add() able to handle external stack_trace Byungchul Park 2017-03-14 8:18 ` Byungchul Park 2017-03-14 8:18 ` [PATCH v6 05/15] lockdep: Implement crossrelease feature Byungchul Park 2017-03-14 8:18 ` Byungchul Park 2017-04-19 14:25 ` Peter Zijlstra 2017-04-19 14:25 ` Peter Zijlstra 2017-04-24 5:11 ` Byungchul Park 2017-04-24 5:11 ` Byungchul Park 2017-04-24 10:17 ` Peter Zijlstra 2017-04-24 10:17 ` Peter Zijlstra 2017-04-25 5:40 ` Byungchul Park 2017-04-25 5:40 ` Byungchul Park 2017-05-16 14:18 ` Peter Zijlstra 2017-05-16 14:18 ` Peter Zijlstra 2017-05-18 6:22 ` Byungchul Park 2017-05-18 6:22 ` Byungchul Park 2017-04-19 15:08 ` Peter Zijlstra 2017-04-19 15:08 ` Peter Zijlstra 2017-04-24 4:36 ` Byungchul Park 2017-04-24 4:36 ` Byungchul Park 2017-04-19 17:19 ` Peter Zijlstra 2017-04-19 17:19 ` Peter Zijlstra 2017-04-24 3:04 ` Byungchul Park 2017-04-24 3:04 ` Byungchul Park 2017-04-24 9:30 ` Peter Zijlstra 2017-04-24 9:30 ` Peter Zijlstra 2017-04-25 6:59 ` Byungchul Park 2017-04-25 6:59 ` Byungchul Park 2017-04-19 17:20 ` Peter Zijlstra 2017-04-19 17:20 ` Peter Zijlstra 2017-04-24 3:13 ` Byungchul Park 2017-04-24 3:13 ` Byungchul Park 2017-05-19 8:07 ` Byungchul Park 2017-05-19 8:07 ` Byungchul Park 2017-05-19 10:30 ` Peter Zijlstra 2017-05-19 10:30 ` Peter Zijlstra 2017-05-19 10:56 ` Byungchul Park 2017-05-19 10:56 ` Byungchul Park 2017-03-14 8:18 ` Byungchul Park [this message] 2017-03-14 8:18 ` [PATCH v6 06/15] lockdep: Handle non(or multi)-acquisition of a crosslock Byungchul Park 2017-03-14 8:18 ` [PATCH v6 07/15] lockdep: Avoid adding redundant direct links of crosslocks Byungchul Park 2017-03-14 8:18 ` Byungchul Park 2017-03-14 8:18 ` [PATCH v6 08/15] lockdep: Fix incorrect condition to print bug msgs for MAX_LOCKDEP_CHAIN_HLOCKS Byungchul Park 2017-03-14 8:18 ` Byungchul Park 2017-03-14 8:18 ` [PATCH v6 09/15] lockdep: Make print_circular_bug() aware of crossrelease Byungchul Park 2017-03-14 8:18 ` Byungchul Park 2017-03-14 8:18 ` [PATCH v6 10/15] lockdep: Apply crossrelease to completions Byungchul Park 2017-03-14 8:18 ` Byungchul Park 2017-03-14 8:18 ` [PATCH v6 11/15] pagemap.h: Remove trailing white space Byungchul Park 2017-03-14 8:18 ` Byungchul Park 2017-03-14 8:18 ` [PATCH v6 12/15] lockdep: Apply crossrelease to PG_locked locks Byungchul Park 2017-03-14 8:18 ` Byungchul Park 2017-03-14 8:19 ` [PATCH v6 13/15] lockdep: Apply lock_acquire(release) on __Set(__Clear)PageLocked Byungchul Park 2017-03-14 8:19 ` Byungchul Park 2017-03-14 8:19 ` [PATCH v6 14/15] lockdep: Move data of CONFIG_LOCKDEP_PAGELOCK from page to page_ext Byungchul Park 2017-03-14 8:19 ` Byungchul Park 2017-03-14 8:19 ` [PATCH v6 15/15] lockdep: Crossrelease feature documentation Byungchul Park 2017-03-14 8:19 ` Byungchul Park
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1489479542-27030-7-git-send-email-byungchul.park@lge.com \ --to=byungchul.park@lge.com \ --cc=akpm@linux-foundation.org \ --cc=boqun.feng@gmail.com \ --cc=iamjoonsoo.kim@lge.com \ --cc=kernel-team@lge.com \ --cc=kirill@shutemov.name \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=mingo@kernel.org \ --cc=npiggin@gmail.com \ --cc=peterz@infradead.org \ --cc=tglx@linutronix.de \ --cc=walken@google.com \ --cc=willy@infradead.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.