From: Andrew Morton <akpm@linux-foundation.org>
To: 1vier1@web.de, akpm@linux-foundation.org, dave@stgolabs.net,
linux-mm@kvack.org, longman@redhat.com, manfred@colorfullife.com,
mm-commits@vger.kernel.org, peterz@infradead.org,
torvalds@linux-foundation.org, will.deacon@arm.com
Subject: [patch 18/67] ipc/sem.c: document and update memory barriers
Date: Mon, 03 Feb 2020 17:34:42 -0800 [thread overview]
Message-ID: <20200204013442.tPVtkj_dX%akpm@linux-foundation.org> (raw)
In-Reply-To: <20200203173311.6269a8be06a05e5a4aa08a93@linux-foundation.org>
From: Manfred Spraul <manfred@colorfullife.com>
Subject: ipc/sem.c: document and update memory barriers
Document and update the memory barriers in ipc/sem.c:
- Add smp_store_release() to wake_up_sem_queue_prepare() and
document why it is needed.
- Read q->status using READ_ONCE+smp_acquire__after_ctrl_dep().
as the pair for the barrier inside wake_up_sem_queue_prepare().
- Add comments to all barriers, and mention the rules in the block
regarding locking.
- Switch to using wake_q_add_safe().
Link: http://lkml.kernel.org/r/20191020123305.14715-6-manfred@colorfullife.com
Signed-off-by: Manfred Spraul <manfred@colorfullife.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: <1vier1@web.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
ipc/sem.c | 66 ++++++++++++++++++++++++++++++++--------------------
1 file changed, 41 insertions(+), 25 deletions(-)
--- a/ipc/sem.c~ipc-semc-document-and-update-memory-barriers
+++ a/ipc/sem.c
@@ -205,15 +205,38 @@ static int sysvipc_sem_proc_show(struct
*
* Memory ordering:
* Most ordering is enforced by using spin_lock() and spin_unlock().
- * The special case is use_global_lock:
+ *
+ * Exceptions:
+ * 1) use_global_lock: (SEM_BARRIER_1)
* Setting it from non-zero to 0 is a RELEASE, this is ensured by
- * using smp_store_release().
+ * using smp_store_release(): Immediately after setting it to 0,
+ * a simple op can start.
* Testing if it is non-zero is an ACQUIRE, this is ensured by using
* smp_load_acquire().
* Setting it from 0 to non-zero must be ordered with regards to
* this smp_load_acquire(), this is guaranteed because the smp_load_acquire()
* is inside a spin_lock() and after a write from 0 to non-zero a
* spin_lock()+spin_unlock() is done.
+ *
+ * 2) queue.status: (SEM_BARRIER_2)
+ * Initialization is done while holding sem_lock(), so no further barrier is
+ * required.
+ * Setting it to a result code is a RELEASE, this is ensured by both a
+ * smp_store_release() (for case a) and while holding sem_lock()
+ * (for case b).
+ * The AQUIRE when reading the result code without holding sem_lock() is
+ * achieved by using READ_ONCE() + smp_acquire__after_ctrl_dep().
+ * (case a above).
+ * Reading the result code while holding sem_lock() needs no further barriers,
+ * the locks inside sem_lock() enforce ordering (case b above)
+ *
+ * 3) current->state:
+ * current->state is set to TASK_INTERRUPTIBLE while holding sem_lock().
+ * The wakeup is handled using the wake_q infrastructure. wake_q wakeups may
+ * happen immediately after calling wake_q_add. As wake_q_add_safe() is called
+ * when holding sem_lock(), no further barriers are required.
+ *
+ * See also ipc/mqueue.c for more details on the covered races.
*/
#define sc_semmsl sem_ctls[0]
@@ -344,12 +367,8 @@ static void complexmode_tryleave(struct
return;
}
if (sma->use_global_lock == 1) {
- /*
- * Immediately after setting use_global_lock to 0,
- * a simple op can start. Thus: all memory writes
- * performed by the current operation must be visible
- * before we set use_global_lock to 0.
- */
+
+ /* See SEM_BARRIER_1 for purpose/pairing */
smp_store_release(&sma->use_global_lock, 0);
} else {
sma->use_global_lock--;
@@ -400,7 +419,7 @@ static inline int sem_lock(struct sem_ar
*/
spin_lock(&sem->lock);
- /* pairs with smp_store_release() */
+ /* see SEM_BARRIER_1 for purpose/pairing */
if (!smp_load_acquire(&sma->use_global_lock)) {
/* fast path successful! */
return sops->sem_num;
@@ -766,15 +785,12 @@ would_block:
static inline void wake_up_sem_queue_prepare(struct sem_queue *q, int error,
struct wake_q_head *wake_q)
{
- wake_q_add(wake_q, q->sleeper);
- /*
- * Rely on the above implicit barrier, such that we can
- * ensure that we hold reference to the task before setting
- * q->status. Otherwise we could race with do_exit if the
- * task is awoken by an external event before calling
- * wake_up_process().
- */
- WRITE_ONCE(q->status, error);
+ get_task_struct(q->sleeper);
+
+ /* see SEM_BARRIER_2 for purpuse/pairing */
+ smp_store_release(&q->status, error);
+
+ wake_q_add_safe(wake_q, q->sleeper);
}
static void unlink_queue(struct sem_array *sma, struct sem_queue *q)
@@ -2148,9 +2164,11 @@ static long do_semtimedop(int semid, str
}
do {
+ /* memory ordering ensured by the lock in sem_lock() */
WRITE_ONCE(queue.status, -EINTR);
queue.sleeper = current;
+ /* memory ordering is ensured by the lock in sem_lock() */
__set_current_state(TASK_INTERRUPTIBLE);
sem_unlock(sma, locknum);
rcu_read_unlock();
@@ -2173,13 +2191,8 @@ static long do_semtimedop(int semid, str
*/
error = READ_ONCE(queue.status);
if (error != -EINTR) {
- /*
- * User space could assume that semop() is a memory
- * barrier: Without the mb(), the cpu could
- * speculatively read in userspace stale data that was
- * overwritten by the previous owner of the semaphore.
- */
- smp_mb();
+ /* see SEM_BARRIER_2 for purpose/pairing */
+ smp_acquire__after_ctrl_dep();
goto out_free;
}
@@ -2189,6 +2202,9 @@ static long do_semtimedop(int semid, str
if (!ipc_valid_object(&sma->sem_perm))
goto out_unlock_free;
+ /*
+ * No necessity for any barrier: We are protect by sem_lock()
+ */
error = READ_ONCE(queue.status);
/*
_
next prev parent reply other threads:[~2020-02-04 1:34 UTC|newest]
Thread overview: 95+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-02-04 1:33 incoming Andrew Morton
2020-02-04 1:33 ` [patch 01/67] ocfs2: fix oops when writing cloned file Andrew Morton
2020-02-04 1:33 ` [patch 02/67] mm/page_alloc.c: fix uninitialized memmaps on a partially populated last section Andrew Morton
2020-02-04 1:33 ` [patch 03/67] fs/proc/page.c: allow inspection of last section and fix end detection Andrew Morton
2020-02-04 1:33 ` [patch 04/67] mm/page_alloc.c: initialize memmap of unavailable memory directly Andrew Morton
2020-02-04 1:33 ` [patch 05/67] mm/page_alloc: fix and rework pfn handling in memmap_init_zone() Andrew Morton
2020-02-04 2:45 ` Linus Torvalds
2020-02-04 1:34 ` [patch 06/67] mm: factor out next_present_section_nr() Andrew Morton
2020-02-04 3:04 ` Linus Torvalds
2020-02-04 4:29 ` Andrew Morton
2020-02-04 8:22 ` David Hildenbrand
2020-02-04 1:34 ` [patch 07/67] mm/memmap_init: update variable name in memmap_init_zone Andrew Morton
2020-02-04 1:34 ` [patch 08/67] mm/memory_hotplug: poison memmap in remove_pfn_range_from_zone() Andrew Morton
2020-02-04 1:34 ` [patch 09/67] mm/memory_hotplug: we always have a zone in find_(smallest|biggest)_section_pfn Andrew Morton
2020-02-04 1:34 ` [patch 10/67] mm/memory_hotplug: don't check for "all holes" in shrink_zone_span() Andrew Morton
2020-02-04 1:34 ` [patch 11/67] mm/memory_hotplug: drop local variables " Andrew Morton
2020-02-04 1:34 ` [patch 12/67] mm/memory_hotplug: cleanup __remove_pages() Andrew Morton
2020-02-04 1:34 ` [patch 13/67] mm/memory_hotplug: drop valid_start/valid_end from test_pages_in_a_zone() Andrew Morton
2020-02-04 1:34 ` [patch 14/67] smp_mb__{before,after}_atomic(): update Documentation Andrew Morton
2020-02-04 1:34 ` [patch 15/67] ipc/mqueue.c: remove duplicated code Andrew Morton
2020-02-04 1:34 ` [patch 16/67] ipc/mqueue.c: update/document memory barriers Andrew Morton
2020-02-04 1:34 ` [patch 17/67] ipc/msg.c: update and document " Andrew Morton
2020-02-04 1:34 ` Andrew Morton [this message]
2020-02-04 1:34 ` [patch 19/67] ipc/msg.c: consolidate all xxxctl_down() functions Andrew Morton
2020-02-04 1:34 ` [patch 20/67] drivers/block/null_blk_main.c: fix layout Andrew Morton
2020-02-04 1:34 ` [patch 21/67] drivers/block/null_blk_main.c: fix uninitialized var warnings Andrew Morton
2020-02-04 1:34 ` [patch 22/67] pinctrl: fix pxa2xx.c build warnings Andrew Morton
2020-02-04 1:34 ` [patch 23/67] mm: remove __krealloc Andrew Morton
2020-02-04 1:35 ` [patch 24/67] mm: add generic p?d_leaf() macros Andrew Morton
2020-02-04 1:35 ` [patch 25/67] arc: mm: add p?d_leaf() definitions Andrew Morton
2020-02-04 1:35 ` [patch 26/67] arm: " Andrew Morton
2020-02-04 1:35 ` [patch 27/67] arm64: " Andrew Morton
2020-02-04 1:35 ` [patch 28/67] mips: " Andrew Morton
2020-02-04 1:35 ` [patch 29/67] powerpc: " Andrew Morton
2020-02-04 1:35 ` [patch 30/67] riscv: " Andrew Morton
2020-02-04 1:35 ` [patch 31/67] s390: " Andrew Morton
2020-02-04 1:35 ` [patch 32/67] sparc: " Andrew Morton
2020-02-04 1:35 ` [patch 33/67] x86: " Andrew Morton
2020-02-04 1:35 ` [patch 34/67] mm: pagewalk: add p4d_entry() and pgd_entry() Andrew Morton
2020-02-04 1:35 ` [patch 35/67] mm: pagewalk: allow walking without vma Andrew Morton
2020-02-04 1:35 ` [patch 36/67] mm: pagewalk: don't lock PTEs for walk_page_range_novma() Andrew Morton
2020-02-04 1:35 ` [patch 37/67] mm: pagewalk: fix termination condition in walk_pte_range() Andrew Morton
2020-02-04 1:36 ` [patch 38/67] mm: pagewalk: add 'depth' parameter to pte_hole Andrew Morton
2020-02-04 1:36 ` [patch 39/67] x86: mm: point to struct seq_file from struct pg_state Andrew Morton
2020-02-04 1:36 ` [patch 40/67] x86: mm+efi: convert ptdump_walk_pgd_level() to take a mm_struct Andrew Morton
2020-02-04 1:36 ` [patch 41/67] x86: mm: convert ptdump_walk_pgd_level_debugfs() to take an mm_struct Andrew Morton
2020-02-04 1:36 ` [patch 42/67] mm: add generic ptdump Andrew Morton
2020-02-04 1:36 ` [patch 43/67] x86: mm: convert dump_pagetables to use walk_page_range Andrew Morton
2020-02-04 1:36 ` [patch 44/67] arm64: mm: convert mm/dump.c to use walk_page_range() Andrew Morton
2020-02-04 1:36 ` [patch 45/67] arm64: mm: display non-present entries in ptdump Andrew Morton
2020-02-04 1:36 ` [patch 46/67] mm: ptdump: reduce level numbers by 1 in note_page() Andrew Morton
2020-02-04 1:36 ` [patch 47/67] x86: mm: avoid allocating struct mm_struct on the stack Andrew Morton
2020-02-04 1:36 ` [patch 48/67] powerpc/mmu_gather: enable RCU_TABLE_FREE even for !SMP case Andrew Morton
2020-02-04 1:36 ` [patch 49/67] mm/mmu_gather: invalidate TLB correctly on batch allocation failure and flush Andrew Morton
2020-02-04 1:36 ` [patch 50/67] asm-generic/tlb: avoid potential double flush Andrew Morton
2020-02-04 1:36 ` [patch 51/67] asm-gemeric/tlb: remove stray function declarations Andrew Morton
2020-02-04 1:36 ` [patch 52/67] asm-generic/tlb: add missing CONFIG symbol Andrew Morton
2020-02-04 1:37 ` [patch 53/67] asm-generic/tlb: rename HAVE_RCU_TABLE_FREE Andrew Morton
2020-02-04 1:37 ` [patch 54/67] asm-generic/tlb: rename HAVE_MMU_GATHER_PAGE_SIZE Andrew Morton
2020-02-04 1:37 ` [patch 55/67] asm-generic/tlb: rename HAVE_MMU_GATHER_NO_GATHER Andrew Morton
2020-02-04 1:37 ` [patch 56/67] asm-generic/tlb: provide MMU_GATHER_TABLE_FREE Andrew Morton
2020-02-04 1:37 ` [patch 57/67] proc: decouple proc from VFS with "struct proc_ops" Andrew Morton
2020-02-04 1:37 ` [patch 58/67] proc: convert everything to " Andrew Morton
2020-02-04 1:37 ` [patch 59/67] lib/string: add strnchrnul() Andrew Morton
2020-02-04 1:37 ` [patch 60/67] bitops: more BITS_TO_* macros Andrew Morton
2020-02-04 1:37 ` [patch 61/67] lib: add test for bitmap_parse() Andrew Morton
2020-02-04 1:37 ` [patch 62/67] lib: make bitmap_parse_user a wrapper on bitmap_parse Andrew Morton
2020-02-04 1:37 ` [patch 63/67] lib: rework bitmap_parse() Andrew Morton
2020-02-04 1:37 ` [patch 64/67] lib: new testcases for bitmap_parse{_user} Andrew Morton
2020-02-04 1:37 ` [patch 65/67] include/linux/cpumask.h: don't calculate length of the input string Andrew Morton
2020-02-04 1:37 ` [patch 66/67] treewide: remove redundant IS_ERR() before error code check Andrew Morton
2020-02-04 1:37 ` [patch 67/67] ARM: dma-api: fix max_pfn off-by-one error in __dma_supported() Andrew Morton
2020-02-04 2:27 ` incoming Linus Torvalds
2020-02-04 2:46 ` incoming Andrew Morton
2020-02-04 3:11 ` incoming Linus Torvalds
2020-02-14 6:26 ` mmotm 2020-02-13-22-26 uploaded Andrew Morton
2020-02-14 16:29 ` mmotm 2020-02-13-22-26 uploaded (mm/hugetlb.c) Randy Dunlap
2020-02-14 17:18 ` Mike Kravetz
2020-02-14 20:51 ` Mina Almasry
[not found] ` <20200214204544.231482-1-almasrymina@google.com>
2020-02-14 21:00 ` [PATCH] hugetlb: fix CONFIG_CGROUP_HUGETLB ifdefs Mina Almasry
2020-02-15 1:17 ` Randy Dunlap
2020-02-15 1:56 ` Randy Dunlap
2020-02-16 20:40 ` Mina Almasry
2020-02-16 21:03 ` Mina Almasry
2020-02-17 2:48 ` Randy Dunlap
2020-02-17 2:57 ` [PATCH] hugetlb: fix <linux/hugetlb_cgroup.h> structs Randy Dunlap
2020-02-17 3:53 ` [PATCH] hugetlb: fix CONFIG_CGROUP_HUGETLB ifdefs Stephen Rothwell
2020-02-14 16:49 ` mmotm 2020-02-13-22-26 uploaded (mm/migrate.c, hugetlb_cgroup.h) Randy Dunlap
2020-02-25 3:53 ` mmotm 2020-02-24-19-53 uploaded Andrew Morton
2020-02-25 6:16 ` mmotm 2020-02-24-19-53 uploaded (init/main.c: initrd*) Randy Dunlap
2020-02-25 6:18 ` Randy Dunlap
2020-02-25 6:21 ` Randy Dunlap
2020-02-25 16:41 ` mmotm 2020-02-24-19-53 uploaded (drivers/platform/x86/intel_pmc_core.c) Randy Dunlap
2020-02-25 17:01 ` mmotm 2020-02-24-19-53 uploaded (objtool warning) Randy Dunlap
2020-02-27 21:52 ` Josh Poimboeuf
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200204013442.tPVtkj_dX%akpm@linux-foundation.org \
--to=akpm@linux-foundation.org \
--cc=1vier1@web.de \
--cc=dave@stgolabs.net \
--cc=linux-mm@kvack.org \
--cc=longman@redhat.com \
--cc=manfred@colorfullife.com \
--cc=mm-commits@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=torvalds@linux-foundation.org \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).