* [PATCH v3 0/4] membarrier fixes @ 2020-12-04 5:07 Andy Lutomirski 2020-12-04 5:07 ` [PATCH v3 1/4] x86/membarrier: Get rid of a dubious optimization Andy Lutomirski ` (3 more replies) 0 siblings, 4 replies; 15+ messages in thread From: Andy Lutomirski @ 2020-12-04 5:07 UTC (permalink / raw) To: x86, Mathieu Desnoyers Cc: LKML, Nicholas Piggin, Arnd Bergmann, Anton Blanchard, Andy Lutomirski Various membarrier fixes. Changes from v2: - Added reviewed-bys - Don't rseq_preempt the caller (Mathieu) - Fix single-thread short circuit (Mathieu) Changes from v1: - patch 1: comment fixes from Mathieu - patch 2: improved comments - patch 3: split out as a separate patch - patch 4: now has a proper explanation Mathieu, I think we have to make sync_core sync the caller. See patch 4. Andy Lutomirski (4): x86/membarrier: Get rid of a dubious optimization membarrier: Add an actual barrier before rseq_preempt() membarrier: Explicitly sync remote cores when SYNC_CORE is requested membarrier: Execute SYNC_CORE on the calling thread arch/x86/include/asm/sync_core.h | 9 ++-- arch/x86/mm/tlb.c | 10 ++++- kernel/sched/membarrier.c | 77 ++++++++++++++++++++++++-------- 3 files changed, 72 insertions(+), 24 deletions(-) -- 2.28.0 ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v3 1/4] x86/membarrier: Get rid of a dubious optimization 2020-12-04 5:07 [PATCH v3 0/4] membarrier fixes Andy Lutomirski @ 2020-12-04 5:07 ` Andy Lutomirski 2020-12-09 4:10 ` Andy Lutomirski 2020-12-09 8:42 ` [tip: x86/urgent] " tip-bot2 for Andy Lutomirski 2020-12-04 5:07 ` [PATCH v3 2/4] membarrier: Add an actual barrier before rseq_preempt() Andy Lutomirski ` (2 subsequent siblings) 3 siblings, 2 replies; 15+ messages in thread From: Andy Lutomirski @ 2020-12-04 5:07 UTC (permalink / raw) To: x86, Mathieu Desnoyers Cc: LKML, Nicholas Piggin, Arnd Bergmann, Anton Blanchard, Andy Lutomirski, stable sync_core_before_usermode() had an incorrect optimization. If we're in an IRQ, we can get to usermode without IRET -- we just have to schedule to a different task in the same mm and do SYSRET. Fortunately, there were no callers of sync_core_before_usermode() that could have had in_irq() or in_nmi() equal to true, because it's only ever called from the scheduler. While we're at it, clarify a related comment. Cc: stable@vger.kernel.org Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Andy Lutomirski <luto@kernel.org> --- arch/x86/include/asm/sync_core.h | 9 +++++---- arch/x86/mm/tlb.c | 10 ++++++++-- 2 files changed, 13 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/sync_core.h b/arch/x86/include/asm/sync_core.h index 0fd4a9dfb29c..ab7382f92aff 100644 --- a/arch/x86/include/asm/sync_core.h +++ b/arch/x86/include/asm/sync_core.h @@ -98,12 +98,13 @@ static inline void sync_core_before_usermode(void) /* With PTI, we unconditionally serialize before running user code. */ if (static_cpu_has(X86_FEATURE_PTI)) return; + /* - * Return from interrupt and NMI is done through iret, which is core - * serializing. + * Even if we're in an interrupt, we might reschedule before returning, + * in which case we could switch to a different thread in the same mm + * and return using SYSRET or SYSEXIT. Instead of trying to keep + * track of our need to sync the core, just sync right away. */ - if (in_irq() || in_nmi()) - return; sync_core(); } diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 11666ba19b62..569ac1d57f55 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -474,8 +474,14 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next, /* * The membarrier system call requires a full memory barrier and * core serialization before returning to user-space, after - * storing to rq->curr. Writing to CR3 provides that full - * memory barrier and core serializing instruction. + * storing to rq->curr, when changing mm. This is because + * membarrier() sends IPIs to all CPUs that are in the target mm + * to make them issue memory barriers. However, if another CPU + * switches to/from the target mm concurrently with + * membarrier(), it can cause that CPU not to receive an IPI + * when it really should issue a memory barrier. Writing to CR3 + * provides that full memory barrier and core serializing + * instruction. */ if (real_prev == next) { VM_WARN_ON(this_cpu_read(cpu_tlbstate.ctxs[prev_asid].ctx_id) != -- 2.28.0 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v3 1/4] x86/membarrier: Get rid of a dubious optimization 2020-12-04 5:07 ` [PATCH v3 1/4] x86/membarrier: Get rid of a dubious optimization Andy Lutomirski @ 2020-12-09 4:10 ` Andy Lutomirski 2020-12-09 8:42 ` [tip: x86/urgent] " tip-bot2 for Andy Lutomirski 1 sibling, 0 replies; 15+ messages in thread From: Andy Lutomirski @ 2020-12-09 4:10 UTC (permalink / raw) To: Andy Lutomirski Cc: X86 ML, Mathieu Desnoyers, LKML, Nicholas Piggin, Arnd Bergmann, Anton Blanchard, stable On Thu, Dec 3, 2020 at 9:07 PM Andy Lutomirski <luto@kernel.org> wrote: > > sync_core_before_usermode() had an incorrect optimization. If we're > in an IRQ, we can get to usermode without IRET -- we just have to > schedule to a different task in the same mm and do SYSRET. > Fortunately, there were no callers of sync_core_before_usermode() > that could have had in_irq() or in_nmi() equal to true, because it's > only ever called from the scheduler. > > While we're at it, clarify a related comment. > Fixes: ac1ab12a3e6e ("lockin/x86: Implement sync_core_before_usermode()") > Cc: stable@vger.kernel.org > Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> > Signed-off-by: Andy Lutomirski <luto@kernel.org> ^ permalink raw reply [flat|nested] 15+ messages in thread
* [tip: x86/urgent] x86/membarrier: Get rid of a dubious optimization 2020-12-04 5:07 ` [PATCH v3 1/4] x86/membarrier: Get rid of a dubious optimization Andy Lutomirski 2020-12-09 4:10 ` Andy Lutomirski @ 2020-12-09 8:42 ` tip-bot2 for Andy Lutomirski 1 sibling, 0 replies; 15+ messages in thread From: tip-bot2 for Andy Lutomirski @ 2020-12-09 8:42 UTC (permalink / raw) To: linux-tip-commits Cc: Andy Lutomirski, Thomas Gleixner, Mathieu Desnoyers, stable, x86, linux-kernel The following commit has been merged into the x86/urgent branch of tip: Commit-ID: a493d1ca1a03b532871f1da27f8dbda2b28b04c4 Gitweb: https://git.kernel.org/tip/a493d1ca1a03b532871f1da27f8dbda2b28b04c4 Author: Andy Lutomirski <luto@kernel.org> AuthorDate: Thu, 03 Dec 2020 21:07:03 -08:00 Committer: Thomas Gleixner <tglx@linutronix.de> CommitterDate: Wed, 09 Dec 2020 09:37:42 +01:00 x86/membarrier: Get rid of a dubious optimization sync_core_before_usermode() had an incorrect optimization. If the kernel returns from an interrupt, it can get to usermode without IRET. It just has to schedule to a different task in the same mm and do SYSRET. Fortunately, there were no callers of sync_core_before_usermode() that could have had in_irq() or in_nmi() equal to true, because it's only ever called from the scheduler. While at it, clarify a related comment. Fixes: 70216e18e519 ("membarrier: Provide core serializing command, *_SYNC_CORE") Signed-off-by: Andy Lutomirski <luto@kernel.org> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/5afc7632be1422f91eaf7611aaaa1b5b8580a086.1607058304.git.luto@kernel.org --- arch/x86/include/asm/sync_core.h | 9 +++++---- arch/x86/mm/tlb.c | 10 ++++++++-- 2 files changed, 13 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/sync_core.h b/arch/x86/include/asm/sync_core.h index 0fd4a9d..ab7382f 100644 --- a/arch/x86/include/asm/sync_core.h +++ b/arch/x86/include/asm/sync_core.h @@ -98,12 +98,13 @@ static inline void sync_core_before_usermode(void) /* With PTI, we unconditionally serialize before running user code. */ if (static_cpu_has(X86_FEATURE_PTI)) return; + /* - * Return from interrupt and NMI is done through iret, which is core - * serializing. + * Even if we're in an interrupt, we might reschedule before returning, + * in which case we could switch to a different thread in the same mm + * and return using SYSRET or SYSEXIT. Instead of trying to keep + * track of our need to sync the core, just sync right away. */ - if (in_irq() || in_nmi()) - return; sync_core(); } diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 11666ba..569ac1d 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -474,8 +474,14 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next, /* * The membarrier system call requires a full memory barrier and * core serialization before returning to user-space, after - * storing to rq->curr. Writing to CR3 provides that full - * memory barrier and core serializing instruction. + * storing to rq->curr, when changing mm. This is because + * membarrier() sends IPIs to all CPUs that are in the target mm + * to make them issue memory barriers. However, if another CPU + * switches to/from the target mm concurrently with + * membarrier(), it can cause that CPU not to receive an IPI + * when it really should issue a memory barrier. Writing to CR3 + * provides that full memory barrier and core serializing + * instruction. */ if (real_prev == next) { VM_WARN_ON(this_cpu_read(cpu_tlbstate.ctxs[prev_asid].ctx_id) != ^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH v3 2/4] membarrier: Add an actual barrier before rseq_preempt() 2020-12-04 5:07 [PATCH v3 0/4] membarrier fixes Andy Lutomirski 2020-12-04 5:07 ` [PATCH v3 1/4] x86/membarrier: Get rid of a dubious optimization Andy Lutomirski @ 2020-12-04 5:07 ` Andy Lutomirski 2020-12-09 4:12 ` Andy Lutomirski 2020-12-09 8:42 ` [tip: x86/urgent] " tip-bot2 for Andy Lutomirski 2020-12-04 5:07 ` [PATCH v3 3/4] membarrier: Explicitly sync remote cores when SYNC_CORE is requested Andy Lutomirski 2020-12-04 5:07 ` [PATCH v3 4/4] membarrier: Execute SYNC_CORE on the calling thread Andy Lutomirski 3 siblings, 2 replies; 15+ messages in thread From: Andy Lutomirski @ 2020-12-04 5:07 UTC (permalink / raw) To: x86, Mathieu Desnoyers Cc: LKML, Nicholas Piggin, Arnd Bergmann, Anton Blanchard, Andy Lutomirski, stable It seems to me that most RSEQ membarrier users will expect any stores done before the membarrier() syscall to be visible to the target task(s). While this is extremely likely to be true in practice, nothing actually guarantees it by a strict reading of the x86 manuals. Rather than providing this guarantee by accident and potentially causing a problem down the road, just add an explicit barrier. Cc: stable@vger.kernel.org Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Andy Lutomirski <luto@kernel.org> --- kernel/sched/membarrier.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/kernel/sched/membarrier.c b/kernel/sched/membarrier.c index 5a40b3828ff2..6251d3d12abe 100644 --- a/kernel/sched/membarrier.c +++ b/kernel/sched/membarrier.c @@ -168,6 +168,14 @@ static void ipi_mb(void *info) static void ipi_rseq(void *info) { + /* + * Ensure that all stores done by the calling thread are visible + * to the current task before the current task resumes. We could + * probably optimize this away on most architectures, but by the + * time we've already sent an IPI, the cost of the extra smp_mb() + * is negligible. + */ + smp_mb(); rseq_preempt(current); } -- 2.28.0 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v3 2/4] membarrier: Add an actual barrier before rseq_preempt() 2020-12-04 5:07 ` [PATCH v3 2/4] membarrier: Add an actual barrier before rseq_preempt() Andy Lutomirski @ 2020-12-09 4:12 ` Andy Lutomirski 2020-12-09 8:42 ` [tip: x86/urgent] " tip-bot2 for Andy Lutomirski 1 sibling, 0 replies; 15+ messages in thread From: Andy Lutomirski @ 2020-12-09 4:12 UTC (permalink / raw) To: Andy Lutomirski Cc: X86 ML, Mathieu Desnoyers, LKML, Nicholas Piggin, Arnd Bergmann, Anton Blanchard, stable On Thu, Dec 3, 2020 at 9:07 PM Andy Lutomirski <luto@kernel.org> wrote: > > It seems to me that most RSEQ membarrier users will expect any > stores done before the membarrier() syscall to be visible to the > target task(s). While this is extremely likely to be true in > practice, nothing actually guarantees it by a strict reading of the > x86 manuals. Rather than providing this guarantee by accident and > potentially causing a problem down the road, just add an explicit > barrier. > > Cc: stable@vger.kernel.org Fixes: 2a36ab717e8f ("rseq/membarrier: Add MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ") which is new in 5.10, so it doesn't need cc:stable if it makes 5.10. ^ permalink raw reply [flat|nested] 15+ messages in thread
* [tip: x86/urgent] membarrier: Add an actual barrier before rseq_preempt() 2020-12-04 5:07 ` [PATCH v3 2/4] membarrier: Add an actual barrier before rseq_preempt() Andy Lutomirski 2020-12-09 4:12 ` Andy Lutomirski @ 2020-12-09 8:42 ` tip-bot2 for Andy Lutomirski 2020-12-14 18:05 ` Andy Lutomirski 1 sibling, 1 reply; 15+ messages in thread From: tip-bot2 for Andy Lutomirski @ 2020-12-09 8:42 UTC (permalink / raw) To: linux-tip-commits Cc: Andy Lutomirski, Thomas Gleixner, Mathieu Desnoyers, stable, x86, linux-kernel The following commit has been merged into the x86/urgent branch of tip: Commit-ID: 2ecedd7569080fd05c1a457e8af2165afecfa29f Gitweb: https://git.kernel.org/tip/2ecedd7569080fd05c1a457e8af2165afecfa29f Author: Andy Lutomirski <luto@kernel.org> AuthorDate: Thu, 03 Dec 2020 21:07:04 -08:00 Committer: Thomas Gleixner <tglx@linutronix.de> CommitterDate: Wed, 09 Dec 2020 09:37:43 +01:00 membarrier: Add an actual barrier before rseq_preempt() It seems that most RSEQ membarrier users will expect any stores done before the membarrier() syscall to be visible to the target task(s). While this is extremely likely to be true in practice, nothing actually guarantees it by a strict reading of the x86 manuals. Rather than providing this guarantee by accident and potentially causing a problem down the road, just add an explicit barrier. Fixes: 70216e18e519 ("membarrier: Provide core serializing command, *_SYNC_CORE") Signed-off-by: Andy Lutomirski <luto@kernel.org> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/d3e7197e034fa4852afcf370ca49c30496e58e40.1607058304.git.luto@kernel.org --- kernel/sched/membarrier.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/kernel/sched/membarrier.c b/kernel/sched/membarrier.c index e23e74d..7d98ef5 100644 --- a/kernel/sched/membarrier.c +++ b/kernel/sched/membarrier.c @@ -40,6 +40,14 @@ static void ipi_mb(void *info) static void ipi_rseq(void *info) { + /* + * Ensure that all stores done by the calling thread are visible + * to the current task before the current task resumes. We could + * probably optimize this away on most architectures, but by the + * time we've already sent an IPI, the cost of the extra smp_mb() + * is negligible. + */ + smp_mb(); rseq_preempt(current); } ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [tip: x86/urgent] membarrier: Add an actual barrier before rseq_preempt() 2020-12-09 8:42 ` [tip: x86/urgent] " tip-bot2 for Andy Lutomirski @ 2020-12-14 18:05 ` Andy Lutomirski 0 siblings, 0 replies; 15+ messages in thread From: Andy Lutomirski @ 2020-12-14 18:05 UTC (permalink / raw) To: LKML, Greg KH, Sasha Levin Cc: linux-tip-commits, Andy Lutomirski, Thomas Gleixner, Mathieu Desnoyers, stable, X86 ML On Wed, Dec 9, 2020 at 12:42 AM tip-bot2 for Andy Lutomirski <tip-bot2@linutronix.de> wrote: > > The following commit has been merged into the x86/urgent branch of tip: > > Commit-ID: 2ecedd7569080fd05c1a457e8af2165afecfa29f > Gitweb: https://git.kernel.org/tip/2ecedd7569080fd05c1a457e8af2165afecfa29f > Author: Andy Lutomirski <luto@kernel.org> > AuthorDate: Thu, 03 Dec 2020 21:07:04 -08:00 > Committer: Thomas Gleixner <tglx@linutronix.de> > CommitterDate: Wed, 09 Dec 2020 09:37:43 +01:00 > > membarrier: Add an actual barrier before rseq_preempt() > > It seems that most RSEQ membarrier users will expect any stores done before > the membarrier() syscall to be visible to the target task(s). While this > is extremely likely to be true in practice, nothing actually guarantees it > by a strict reading of the x86 manuals. Rather than providing this > guarantee by accident and potentially causing a problem down the road, just > add an explicit barrier. > > Fixes: 70216e18e519 ("membarrier: Provide core serializing command, *_SYNC_CORE") Whoops, this got mangled on its way to tip. This should be: Fixes: 2a36ab717e8f ("rseq/membarrier: Add MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ") and this patch does not need to be backported. ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v3 3/4] membarrier: Explicitly sync remote cores when SYNC_CORE is requested 2020-12-04 5:07 [PATCH v3 0/4] membarrier fixes Andy Lutomirski 2020-12-04 5:07 ` [PATCH v3 1/4] x86/membarrier: Get rid of a dubious optimization Andy Lutomirski 2020-12-04 5:07 ` [PATCH v3 2/4] membarrier: Add an actual barrier before rseq_preempt() Andy Lutomirski @ 2020-12-04 5:07 ` Andy Lutomirski 2020-12-09 4:13 ` Andy Lutomirski 2020-12-09 8:42 ` [tip: x86/urgent] " tip-bot2 for Andy Lutomirski 2020-12-04 5:07 ` [PATCH v3 4/4] membarrier: Execute SYNC_CORE on the calling thread Andy Lutomirski 3 siblings, 2 replies; 15+ messages in thread From: Andy Lutomirski @ 2020-12-04 5:07 UTC (permalink / raw) To: x86, Mathieu Desnoyers Cc: LKML, Nicholas Piggin, Arnd Bergmann, Anton Blanchard, Andy Lutomirski, stable membarrier() does not explicitly sync_core() remote CPUs; instead, it relies on the assumption that an IPI will result in a core sync. On x86, I think this may be true in practice, but it's not architecturally reliable. In particular, the SDM and APM do not appear to guarantee that interrupt delivery is serializing. While IRET does serialize, IPI return can schedule, thereby switching to another task in the same mm that was sleeping in a syscall. The new task could then SYSRET back to usermode without ever executing IRET. Make this more robust by explicitly calling sync_core_before_usermode() on remote cores. (This also helps people who search the kernel tree for instances of sync_core() and sync_core_before_usermode() -- one might be surprised that the core membarrier code doesn't currently show up in a such a search.) Cc: stable@vger.kernel.org Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Signed-off-by: Andy Lutomirski <luto@kernel.org> --- kernel/sched/membarrier.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/kernel/sched/membarrier.c b/kernel/sched/membarrier.c index 6251d3d12abe..01538b31f27e 100644 --- a/kernel/sched/membarrier.c +++ b/kernel/sched/membarrier.c @@ -166,6 +166,23 @@ static void ipi_mb(void *info) smp_mb(); /* IPIs should be serializing but paranoid. */ } +static void ipi_sync_core(void *info) +{ + /* + * The smp_mb() in membarrier after all the IPIs is supposed to + * ensure that memory on remote CPUs that occur before the IPI + * become visible to membarrier()'s caller -- see scenario B in + * the big comment at the top of this file. + * + * A sync_core() would provide this guarantee, but + * sync_core_before_usermode() might end up being deferred until + * after membarrier()'s smp_mb(). + */ + smp_mb(); /* IPIs should be serializing but paranoid. */ + + sync_core_before_usermode(); +} + static void ipi_rseq(void *info) { /* @@ -301,6 +318,7 @@ static int membarrier_private_expedited(int flags, int cpu_id) if (!(atomic_read(&mm->membarrier_state) & MEMBARRIER_STATE_PRIVATE_EXPEDITED_SYNC_CORE_READY)) return -EPERM; + ipi_func = ipi_sync_core; } else if (flags == MEMBARRIER_FLAG_RSEQ) { if (!IS_ENABLED(CONFIG_RSEQ)) return -EINVAL; -- 2.28.0 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v3 3/4] membarrier: Explicitly sync remote cores when SYNC_CORE is requested 2020-12-04 5:07 ` [PATCH v3 3/4] membarrier: Explicitly sync remote cores when SYNC_CORE is requested Andy Lutomirski @ 2020-12-09 4:13 ` Andy Lutomirski 2020-12-09 8:42 ` [tip: x86/urgent] " tip-bot2 for Andy Lutomirski 1 sibling, 0 replies; 15+ messages in thread From: Andy Lutomirski @ 2020-12-09 4:13 UTC (permalink / raw) To: Andy Lutomirski Cc: X86 ML, Mathieu Desnoyers, LKML, Nicholas Piggin, Arnd Bergmann, Anton Blanchard, stable On Thu, Dec 3, 2020 at 9:07 PM Andy Lutomirski <luto@kernel.org> wrote: > > membarrier() does not explicitly sync_core() remote CPUs; instead, it > relies on the assumption that an IPI will result in a core sync. On > x86, I think this may be true in practice, but it's not architecturally > reliable. In particular, the SDM and APM do not appear to guarantee > that interrupt delivery is serializing. While IRET does serialize, IPI > return can schedule, thereby switching to another task in the same mm > that was sleeping in a syscall. The new task could then SYSRET back to > usermode without ever executing IRET. > > Make this more robust by explicitly calling sync_core_before_usermode() > on remote cores. (This also helps people who search the kernel tree for > instances of sync_core() and sync_core_before_usermode() -- one might be > surprised that the core membarrier code doesn't currently show up in a > such a search.) > Fixes: 70216e18e519 ("membarrier: Provide core serializing command, *_SYNC_CORE") > Cc: stable@vger.kernel.org > Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> > Signed-off-by: Andy Lutomirski <luto@kernel.org> ^ permalink raw reply [flat|nested] 15+ messages in thread
* [tip: x86/urgent] membarrier: Explicitly sync remote cores when SYNC_CORE is requested 2020-12-04 5:07 ` [PATCH v3 3/4] membarrier: Explicitly sync remote cores when SYNC_CORE is requested Andy Lutomirski 2020-12-09 4:13 ` Andy Lutomirski @ 2020-12-09 8:42 ` tip-bot2 for Andy Lutomirski 1 sibling, 0 replies; 15+ messages in thread From: tip-bot2 for Andy Lutomirski @ 2020-12-09 8:42 UTC (permalink / raw) To: linux-tip-commits Cc: Andy Lutomirski, Thomas Gleixner, Mathieu Desnoyers, stable, x86, linux-kernel The following commit has been merged into the x86/urgent branch of tip: Commit-ID: 758c9373d84168dc7d039cf85a0e920046b17b41 Gitweb: https://git.kernel.org/tip/758c9373d84168dc7d039cf85a0e920046b17b41 Author: Andy Lutomirski <luto@kernel.org> AuthorDate: Thu, 03 Dec 2020 21:07:05 -08:00 Committer: Thomas Gleixner <tglx@linutronix.de> CommitterDate: Wed, 09 Dec 2020 09:37:43 +01:00 membarrier: Explicitly sync remote cores when SYNC_CORE is requested membarrier() does not explicitly sync_core() remote CPUs; instead, it relies on the assumption that an IPI will result in a core sync. On x86, this may be true in practice, but it's not architecturally reliable. In particular, the SDM and APM do not appear to guarantee that interrupt delivery is serializing. While IRET does serialize, IPI return can schedule, thereby switching to another task in the same mm that was sleeping in a syscall. The new task could then SYSRET back to usermode without ever executing IRET. Make this more robust by explicitly calling sync_core_before_usermode() on remote cores. (This also helps people who search the kernel tree for instances of sync_core() and sync_core_before_usermode() -- one might be surprised that the core membarrier code doesn't currently show up in a such a search.) Fixes: 70216e18e519 ("membarrier: Provide core serializing command, *_SYNC_CORE") Signed-off-by: Andy Lutomirski <luto@kernel.org> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/776b448d5f7bd6b12690707f5ed67bcda7f1d427.1607058304.git.luto@kernel.org --- kernel/sched/membarrier.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/kernel/sched/membarrier.c b/kernel/sched/membarrier.c index 7d98ef5..1c278df 100644 --- a/kernel/sched/membarrier.c +++ b/kernel/sched/membarrier.c @@ -38,6 +38,23 @@ static void ipi_mb(void *info) smp_mb(); /* IPIs should be serializing but paranoid. */ } +static void ipi_sync_core(void *info) +{ + /* + * The smp_mb() in membarrier after all the IPIs is supposed to + * ensure that memory on remote CPUs that occur before the IPI + * become visible to membarrier()'s caller -- see scenario B in + * the big comment at the top of this file. + * + * A sync_core() would provide this guarantee, but + * sync_core_before_usermode() might end up being deferred until + * after membarrier()'s smp_mb(). + */ + smp_mb(); /* IPIs should be serializing but paranoid. */ + + sync_core_before_usermode(); +} + static void ipi_rseq(void *info) { /* @@ -162,6 +179,7 @@ static int membarrier_private_expedited(int flags, int cpu_id) if (!(atomic_read(&mm->membarrier_state) & MEMBARRIER_STATE_PRIVATE_EXPEDITED_SYNC_CORE_READY)) return -EPERM; + ipi_func = ipi_sync_core; } else if (flags == MEMBARRIER_FLAG_RSEQ) { if (!IS_ENABLED(CONFIG_RSEQ)) return -EINVAL; ^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH v3 4/4] membarrier: Execute SYNC_CORE on the calling thread 2020-12-04 5:07 [PATCH v3 0/4] membarrier fixes Andy Lutomirski ` (2 preceding siblings ...) 2020-12-04 5:07 ` [PATCH v3 3/4] membarrier: Explicitly sync remote cores when SYNC_CORE is requested Andy Lutomirski @ 2020-12-04 5:07 ` Andy Lutomirski 2020-12-04 19:35 ` Mathieu Desnoyers ` (2 more replies) 3 siblings, 3 replies; 15+ messages in thread From: Andy Lutomirski @ 2020-12-04 5:07 UTC (permalink / raw) To: x86, Mathieu Desnoyers Cc: LKML, Nicholas Piggin, Arnd Bergmann, Anton Blanchard, Andy Lutomirski, stable membarrier()'s MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE is documented as syncing the core on all sibling threads but not necessarily the calling thread. This behavior is fundamentally buggy and cannot be used safely. Suppose a user program has two threads. Thread A is on CPU 0 and thread B is on CPU 1. Thread A modifies some text and calls membarrier(MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE). Then thread B executes the modified code. If, at any point after membarrier() decides which CPUs to target, thread A could be preempted and replaced by thread B on CPU 0. This could even happen on exit from the membarrier() syscall. If this happens, thread B will end up running on CPU 0 without having synced. In principle, this could be fixed by arranging for the scheduler to sync_core_before_usermode() whenever switching between two threads in the same mm if there is any possibility of a concurrent membarrier() call, but this would have considerable overhead. Instead, make membarrier() sync the calling CPU as well. As an optimization, this avoids an extra smp_mb() in the default barrier-only mode. Cc: stable@vger.kernel.org Signed-off-by: Andy Lutomirski <luto@kernel.org> --- kernel/sched/membarrier.c | 51 +++++++++++++++++++++++++-------------- 1 file changed, 33 insertions(+), 18 deletions(-) diff --git a/kernel/sched/membarrier.c b/kernel/sched/membarrier.c index 01538b31f27e..57266ab32ef9 100644 --- a/kernel/sched/membarrier.c +++ b/kernel/sched/membarrier.c @@ -333,7 +333,8 @@ static int membarrier_private_expedited(int flags, int cpu_id) return -EPERM; } - if (atomic_read(&mm->mm_users) == 1 || num_online_cpus() == 1) + if (flags != MEMBARRIER_FLAG_SYNC_CORE && + (atomic_read(&mm->mm_users) == 1 || num_online_cpus() == 1)) return 0; /* @@ -352,8 +353,6 @@ static int membarrier_private_expedited(int flags, int cpu_id) if (cpu_id >= nr_cpu_ids || !cpu_online(cpu_id)) goto out; - if (cpu_id == raw_smp_processor_id()) - goto out; rcu_read_lock(); p = rcu_dereference(cpu_rq(cpu_id)->curr); if (!p || p->mm != mm) { @@ -368,16 +367,6 @@ static int membarrier_private_expedited(int flags, int cpu_id) for_each_online_cpu(cpu) { struct task_struct *p; - /* - * Skipping the current CPU is OK even through we can be - * migrated at any point. The current CPU, at the point - * where we read raw_smp_processor_id(), is ensured to - * be in program order with respect to the caller - * thread. Therefore, we can skip this CPU from the - * iteration. - */ - if (cpu == raw_smp_processor_id()) - continue; p = rcu_dereference(cpu_rq(cpu)->curr); if (p && p->mm == mm) __cpumask_set_cpu(cpu, tmpmask); @@ -385,12 +374,38 @@ static int membarrier_private_expedited(int flags, int cpu_id) rcu_read_unlock(); } - preempt_disable(); - if (cpu_id >= 0) + if (cpu_id >= 0) { + /* + * smp_call_function_single() will call ipi_func() if cpu_id + * is the calling CPU. + */ smp_call_function_single(cpu_id, ipi_func, NULL, 1); - else - smp_call_function_many(tmpmask, ipi_func, NULL, 1); - preempt_enable(); + } else { + /* + * For regular membarrier, we can save a few cycles by + * skipping the current cpu -- we're about to do smp_mb() + * below, and if we migrate to a different cpu, this cpu + * and the new cpu will execute a full barrier in the + * scheduler. + * + * For CORE_SYNC, we do need a barrier on the current cpu -- + * otherwise, if we are migrated and replaced by a different + * task in the same mm just before, during, or after + * membarrier, we will end up with some thread in the mm + * running without a core sync. + * + * For RSEQ, don't rseq_preempt() the caller. User code + * is not supposed to issue syscalls at all from inside an + * rseq critical section. + */ + if (flags != MEMBARRIER_FLAG_SYNC_CORE) { + preempt_disable(); + smp_call_function_many(tmpmask, ipi_func, NULL, true); + preempt_enable(); + } else { + on_each_cpu_mask(tmpmask, ipi_func, NULL, true); + } + } out: if (cpu_id < 0) -- 2.28.0 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v3 4/4] membarrier: Execute SYNC_CORE on the calling thread 2020-12-04 5:07 ` [PATCH v3 4/4] membarrier: Execute SYNC_CORE on the calling thread Andy Lutomirski @ 2020-12-04 19:35 ` Mathieu Desnoyers 2020-12-09 4:15 ` Andy Lutomirski 2020-12-09 8:42 ` [tip: x86/urgent] " tip-bot2 for Andy Lutomirski 2 siblings, 0 replies; 15+ messages in thread From: Mathieu Desnoyers @ 2020-12-04 19:35 UTC (permalink / raw) To: Andy Lutomirski Cc: x86, linux-kernel, Nicholas Piggin, Arnd Bergmann, Anton Blanchard, stable ----- On Dec 4, 2020, at 12:07 AM, Andy Lutomirski luto@kernel.org wrote: > membarrier()'s MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE is documented > as syncing the core on all sibling threads but not necessarily the > calling thread. This behavior is fundamentally buggy and cannot be used > safely. Suppose a user program has two threads. Thread A is on CPU 0 > and thread B is on CPU 1. Thread A modifies some text and calls > membarrier(MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE). Then thread B > executes the modified code. If, at any point after membarrier() decides > which CPUs to target, thread A could be preempted and replaced by thread > B on CPU 0. This could even happen on exit from the membarrier() > syscall. If this happens, thread B will end up running on CPU 0 without > having synced. > > In principle, this could be fixed by arranging for the scheduler to > sync_core_before_usermode() whenever switching between two threads in > the same mm if there is any possibility of a concurrent membarrier() > call, but this would have considerable overhead. Instead, make > membarrier() sync the calling CPU as well. > > As an optimization, this avoids an extra smp_mb() in the default > barrier-only mode. ^ we could also add to the commit message that it avoids doing rseq preempt on the caller as well. Other than that: Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Thanks! Mathieu > > Cc: stable@vger.kernel.org > Signed-off-by: Andy Lutomirski <luto@kernel.org> > --- > kernel/sched/membarrier.c | 51 +++++++++++++++++++++++++-------------- > 1 file changed, 33 insertions(+), 18 deletions(-) > > diff --git a/kernel/sched/membarrier.c b/kernel/sched/membarrier.c > index 01538b31f27e..57266ab32ef9 100644 > --- a/kernel/sched/membarrier.c > +++ b/kernel/sched/membarrier.c > @@ -333,7 +333,8 @@ static int membarrier_private_expedited(int flags, int > cpu_id) > return -EPERM; > } > > - if (atomic_read(&mm->mm_users) == 1 || num_online_cpus() == 1) > + if (flags != MEMBARRIER_FLAG_SYNC_CORE && > + (atomic_read(&mm->mm_users) == 1 || num_online_cpus() == 1)) > return 0; > > /* > @@ -352,8 +353,6 @@ static int membarrier_private_expedited(int flags, int > cpu_id) > > if (cpu_id >= nr_cpu_ids || !cpu_online(cpu_id)) > goto out; > - if (cpu_id == raw_smp_processor_id()) > - goto out; > rcu_read_lock(); > p = rcu_dereference(cpu_rq(cpu_id)->curr); > if (!p || p->mm != mm) { > @@ -368,16 +367,6 @@ static int membarrier_private_expedited(int flags, int > cpu_id) > for_each_online_cpu(cpu) { > struct task_struct *p; > > - /* > - * Skipping the current CPU is OK even through we can be > - * migrated at any point. The current CPU, at the point > - * where we read raw_smp_processor_id(), is ensured to > - * be in program order with respect to the caller > - * thread. Therefore, we can skip this CPU from the > - * iteration. > - */ > - if (cpu == raw_smp_processor_id()) > - continue; > p = rcu_dereference(cpu_rq(cpu)->curr); > if (p && p->mm == mm) > __cpumask_set_cpu(cpu, tmpmask); > @@ -385,12 +374,38 @@ static int membarrier_private_expedited(int flags, int > cpu_id) > rcu_read_unlock(); > } > > - preempt_disable(); > - if (cpu_id >= 0) > + if (cpu_id >= 0) { > + /* > + * smp_call_function_single() will call ipi_func() if cpu_id > + * is the calling CPU. > + */ > smp_call_function_single(cpu_id, ipi_func, NULL, 1); > - else > - smp_call_function_many(tmpmask, ipi_func, NULL, 1); > - preempt_enable(); > + } else { > + /* > + * For regular membarrier, we can save a few cycles by > + * skipping the current cpu -- we're about to do smp_mb() > + * below, and if we migrate to a different cpu, this cpu > + * and the new cpu will execute a full barrier in the > + * scheduler. > + * > + * For CORE_SYNC, we do need a barrier on the current cpu -- > + * otherwise, if we are migrated and replaced by a different > + * task in the same mm just before, during, or after > + * membarrier, we will end up with some thread in the mm > + * running without a core sync. > + * > + * For RSEQ, don't rseq_preempt() the caller. User code > + * is not supposed to issue syscalls at all from inside an > + * rseq critical section. > + */ > + if (flags != MEMBARRIER_FLAG_SYNC_CORE) { > + preempt_disable(); > + smp_call_function_many(tmpmask, ipi_func, NULL, true); > + preempt_enable(); > + } else { > + on_each_cpu_mask(tmpmask, ipi_func, NULL, true); > + } > + } > > out: > if (cpu_id < 0) > -- > 2.28.0 -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 4/4] membarrier: Execute SYNC_CORE on the calling thread 2020-12-04 5:07 ` [PATCH v3 4/4] membarrier: Execute SYNC_CORE on the calling thread Andy Lutomirski 2020-12-04 19:35 ` Mathieu Desnoyers @ 2020-12-09 4:15 ` Andy Lutomirski 2020-12-09 8:42 ` [tip: x86/urgent] " tip-bot2 for Andy Lutomirski 2 siblings, 0 replies; 15+ messages in thread From: Andy Lutomirski @ 2020-12-09 4:15 UTC (permalink / raw) To: Andy Lutomirski Cc: X86 ML, Mathieu Desnoyers, LKML, Nicholas Piggin, Arnd Bergmann, Anton Blanchard, stable On Thu, Dec 3, 2020 at 9:07 PM Andy Lutomirski <luto@kernel.org> wrote: > > membarrier()'s MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE is documented > as syncing the core on all sibling threads but not necessarily the > calling thread. This behavior is fundamentally buggy and cannot be used > safely. Suppose a user program has two threads. Thread A is on CPU 0 > and thread B is on CPU 1. Thread A modifies some text and calls > membarrier(MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE). Then thread B > executes the modified code. If, at any point after membarrier() decides > which CPUs to target, thread A could be preempted and replaced by thread > B on CPU 0. This could even happen on exit from the membarrier() > syscall. If this happens, thread B will end up running on CPU 0 without > having synced. > > In principle, this could be fixed by arranging for the scheduler to > sync_core_before_usermode() whenever switching between two threads in > the same mm if there is any possibility of a concurrent membarrier() > call, but this would have considerable overhead. Instead, make > membarrier() sync the calling CPU as well. > > As an optimization, this avoids an extra smp_mb() in the default > barrier-only mode. Fixes: 70216e18e519 ("membarrier: Provide core serializing command, *_SYNC_CORE") also: > + /* > + * For regular membarrier, we can save a few cycles by > + * skipping the current cpu -- we're about to do smp_mb() > + * below, and if we migrate to a different cpu, this cpu > + * and the new cpu will execute a full barrier in the > + * scheduler. > + * > + * For CORE_SYNC, we do need a barrier on the current cpu -- s/CORE_SYNC/SYNC_CORE/ --Andy ^ permalink raw reply [flat|nested] 15+ messages in thread
* [tip: x86/urgent] membarrier: Execute SYNC_CORE on the calling thread 2020-12-04 5:07 ` [PATCH v3 4/4] membarrier: Execute SYNC_CORE on the calling thread Andy Lutomirski 2020-12-04 19:35 ` Mathieu Desnoyers 2020-12-09 4:15 ` Andy Lutomirski @ 2020-12-09 8:42 ` tip-bot2 for Andy Lutomirski 2 siblings, 0 replies; 15+ messages in thread From: tip-bot2 for Andy Lutomirski @ 2020-12-09 8:42 UTC (permalink / raw) To: linux-tip-commits Cc: Andy Lutomirski, Thomas Gleixner, Mathieu Desnoyers, x86, linux-kernel The following commit has been merged into the x86/urgent branch of tip: Commit-ID: e45cdc71d1fa5ac3a57b23acc31eb959e4f60135 Gitweb: https://git.kernel.org/tip/e45cdc71d1fa5ac3a57b23acc31eb959e4f60135 Author: Andy Lutomirski <luto@kernel.org> AuthorDate: Thu, 03 Dec 2020 21:07:06 -08:00 Committer: Thomas Gleixner <tglx@linutronix.de> CommitterDate: Wed, 09 Dec 2020 09:37:43 +01:00 membarrier: Execute SYNC_CORE on the calling thread membarrier()'s MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE is documented as syncing the core on all sibling threads but not necessarily the calling thread. This behavior is fundamentally buggy and cannot be used safely. Suppose a user program has two threads. Thread A is on CPU 0 and thread B is on CPU 1. Thread A modifies some text and calls membarrier(MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE). Then thread B executes the modified code. If, at any point after membarrier() decides which CPUs to target, thread A could be preempted and replaced by thread B on CPU 0. This could even happen on exit from the membarrier() syscall. If this happens, thread B will end up running on CPU 0 without having synced. In principle, this could be fixed by arranging for the scheduler to issue sync_core_before_usermode() whenever switching between two threads in the same mm if there is any possibility of a concurrent membarrier() call, but this would have considerable overhead. Instead, make membarrier() sync the calling CPU as well. As an optimization, this avoids an extra smp_mb() in the default barrier-only mode and an extra rseq preempt on the caller. Fixes: 70216e18e519 ("membarrier: Provide core serializing command, *_SYNC_CORE") Signed-off-by: Andy Lutomirski <luto@kernel.org> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Link: https://lore.kernel.org/r/250ded637696d490c69bef1877148db86066881c.1607058304.git.luto@kernel.org --- kernel/sched/membarrier.c | 51 ++++++++++++++++++++++++-------------- 1 file changed, 33 insertions(+), 18 deletions(-) diff --git a/kernel/sched/membarrier.c b/kernel/sched/membarrier.c index 1c278df..9d8df34 100644 --- a/kernel/sched/membarrier.c +++ b/kernel/sched/membarrier.c @@ -194,7 +194,8 @@ static int membarrier_private_expedited(int flags, int cpu_id) return -EPERM; } - if (atomic_read(&mm->mm_users) == 1 || num_online_cpus() == 1) + if (flags != MEMBARRIER_FLAG_SYNC_CORE && + (atomic_read(&mm->mm_users) == 1 || num_online_cpus() == 1)) return 0; /* @@ -213,8 +214,6 @@ static int membarrier_private_expedited(int flags, int cpu_id) if (cpu_id >= nr_cpu_ids || !cpu_online(cpu_id)) goto out; - if (cpu_id == raw_smp_processor_id()) - goto out; rcu_read_lock(); p = rcu_dereference(cpu_rq(cpu_id)->curr); if (!p || p->mm != mm) { @@ -229,16 +228,6 @@ static int membarrier_private_expedited(int flags, int cpu_id) for_each_online_cpu(cpu) { struct task_struct *p; - /* - * Skipping the current CPU is OK even through we can be - * migrated at any point. The current CPU, at the point - * where we read raw_smp_processor_id(), is ensured to - * be in program order with respect to the caller - * thread. Therefore, we can skip this CPU from the - * iteration. - */ - if (cpu == raw_smp_processor_id()) - continue; p = rcu_dereference(cpu_rq(cpu)->curr); if (p && p->mm == mm) __cpumask_set_cpu(cpu, tmpmask); @@ -246,12 +235,38 @@ static int membarrier_private_expedited(int flags, int cpu_id) rcu_read_unlock(); } - preempt_disable(); - if (cpu_id >= 0) + if (cpu_id >= 0) { + /* + * smp_call_function_single() will call ipi_func() if cpu_id + * is the calling CPU. + */ smp_call_function_single(cpu_id, ipi_func, NULL, 1); - else - smp_call_function_many(tmpmask, ipi_func, NULL, 1); - preempt_enable(); + } else { + /* + * For regular membarrier, we can save a few cycles by + * skipping the current cpu -- we're about to do smp_mb() + * below, and if we migrate to a different cpu, this cpu + * and the new cpu will execute a full barrier in the + * scheduler. + * + * For SYNC_CORE, we do need a barrier on the current cpu -- + * otherwise, if we are migrated and replaced by a different + * task in the same mm just before, during, or after + * membarrier, we will end up with some thread in the mm + * running without a core sync. + * + * For RSEQ, don't rseq_preempt() the caller. User code + * is not supposed to issue syscalls at all from inside an + * rseq critical section. + */ + if (flags != MEMBARRIER_FLAG_SYNC_CORE) { + preempt_disable(); + smp_call_function_many(tmpmask, ipi_func, NULL, true); + preempt_enable(); + } else { + on_each_cpu_mask(tmpmask, ipi_func, NULL, true); + } + } out: if (cpu_id < 0) ^ permalink raw reply related [flat|nested] 15+ messages in thread
end of thread, other threads:[~2020-12-14 18:06 UTC | newest] Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-12-04 5:07 [PATCH v3 0/4] membarrier fixes Andy Lutomirski 2020-12-04 5:07 ` [PATCH v3 1/4] x86/membarrier: Get rid of a dubious optimization Andy Lutomirski 2020-12-09 4:10 ` Andy Lutomirski 2020-12-09 8:42 ` [tip: x86/urgent] " tip-bot2 for Andy Lutomirski 2020-12-04 5:07 ` [PATCH v3 2/4] membarrier: Add an actual barrier before rseq_preempt() Andy Lutomirski 2020-12-09 4:12 ` Andy Lutomirski 2020-12-09 8:42 ` [tip: x86/urgent] " tip-bot2 for Andy Lutomirski 2020-12-14 18:05 ` Andy Lutomirski 2020-12-04 5:07 ` [PATCH v3 3/4] membarrier: Explicitly sync remote cores when SYNC_CORE is requested Andy Lutomirski 2020-12-09 4:13 ` Andy Lutomirski 2020-12-09 8:42 ` [tip: x86/urgent] " tip-bot2 for Andy Lutomirski 2020-12-04 5:07 ` [PATCH v3 4/4] membarrier: Execute SYNC_CORE on the calling thread Andy Lutomirski 2020-12-04 19:35 ` Mathieu Desnoyers 2020-12-09 4:15 ` Andy Lutomirski 2020-12-09 8:42 ` [tip: x86/urgent] " tip-bot2 for Andy Lutomirski
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).