From: "tip-bot2 for Peter Zijlstra (Intel)" <tip-bot2@linutronix.de>
To: linux-tip-commits@vger.kernel.org
Cc: "Peter Zijlstra (Intel)" <peterz@infradead.org>,
Thomas Gleixner <tglx@linutronix.de>,
Alexandre Chartre <alexandre.chartre@oracle.com>,
x86 <x86@kernel.org>, LKML <linux-kernel@vger.kernel.org>
Subject: [tip: sched/core] sched: Clean up scheduler_ipi()
Date: Tue, 12 May 2020 15:13:27 -0000 [thread overview]
Message-ID: <158929640713.390.10558251425748115488.tip-bot2@tip-bot2> (raw)
In-Reply-To: <20200505134058.361859938@linutronix.de>
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 90b5363acd4739769c3f38c1aff16171bd133e8c
Gitweb: https://git.kernel.org/tip/90b5363acd4739769c3f38c1aff16171bd133e8c
Author: Peter Zijlstra (Intel) <peterz@infradead.org>
AuthorDate: Fri, 27 Mar 2020 11:44:56 +01:00
Committer: Thomas Gleixner <tglx@linutronix.de>
CommitterDate: Tue, 12 May 2020 17:10:48 +02:00
sched: Clean up scheduler_ipi()
The scheduler IPI has grown weird and wonderful over the years, time
for spring cleaning.
Move all the non-trivial stuff out of it and into a regular smp function
call IPI. This then reduces the schedule_ipi() to most of it's former NOP
glory and ensures to keep the interrupt vector lean and mean.
Aside of that avoiding the full irq_enter() in the x86 IPI implementation
is incorrect as scheduler_ipi() can be instrumented. To work around that
scheduler_ipi() had an irq_enter/exit() hack when heavy work was
pending. This is gone now.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Alexandre Chartre <alexandre.chartre@oracle.com>
Link: https://lkml.kernel.org/r/20200505134058.361859938@linutronix.de
---
kernel/sched/core.c | 64 ++++++++++++++++++++-----------------------
kernel/sched/fair.c | 5 +--
kernel/sched/sched.h | 6 ++--
3 files changed, 36 insertions(+), 39 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index b58efb1..cd2070d 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -219,6 +219,13 @@ void update_rq_clock(struct rq *rq)
update_rq_clock_task(rq, delta);
}
+static inline void
+rq_csd_init(struct rq *rq, call_single_data_t *csd, smp_call_func_t func)
+{
+ csd->flags = 0;
+ csd->func = func;
+ csd->info = rq;
+}
#ifdef CONFIG_SCHED_HRTICK
/*
@@ -314,16 +321,14 @@ void hrtick_start(struct rq *rq, u64 delay)
hrtimer_start(&rq->hrtick_timer, ns_to_ktime(delay),
HRTIMER_MODE_REL_PINNED_HARD);
}
+
#endif /* CONFIG_SMP */
static void hrtick_rq_init(struct rq *rq)
{
#ifdef CONFIG_SMP
- rq->hrtick_csd.flags = 0;
- rq->hrtick_csd.func = __hrtick_start;
- rq->hrtick_csd.info = rq;
+ rq_csd_init(rq, &rq->hrtick_csd, __hrtick_start);
#endif
-
hrtimer_init(&rq->hrtick_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL_HARD);
rq->hrtick_timer.function = hrtick;
}
@@ -650,6 +655,16 @@ static inline bool got_nohz_idle_kick(void)
return false;
}
+static void nohz_csd_func(void *info)
+{
+ struct rq *rq = info;
+
+ if (got_nohz_idle_kick()) {
+ rq->idle_balance = 1;
+ raise_softirq_irqoff(SCHED_SOFTIRQ);
+ }
+}
+
#else /* CONFIG_NO_HZ_COMMON */
static inline bool got_nohz_idle_kick(void)
@@ -2292,6 +2307,11 @@ void sched_ttwu_pending(void)
rq_unlock_irqrestore(rq, &rf);
}
+static void wake_csd_func(void *info)
+{
+ sched_ttwu_pending();
+}
+
void scheduler_ipi(void)
{
/*
@@ -2300,34 +2320,6 @@ void scheduler_ipi(void)
* this IPI.
*/
preempt_fold_need_resched();
-
- if (llist_empty(&this_rq()->wake_list) && !got_nohz_idle_kick())
- return;
-
- /*
- * Not all reschedule IPI handlers call irq_enter/irq_exit, since
- * traditionally all their work was done from the interrupt return
- * path. Now that we actually do some work, we need to make sure
- * we do call them.
- *
- * Some archs already do call them, luckily irq_enter/exit nest
- * properly.
- *
- * Arguably we should visit all archs and update all handlers,
- * however a fair share of IPIs are still resched only so this would
- * somewhat pessimize the simple resched case.
- */
- irq_enter();
- sched_ttwu_pending();
-
- /*
- * Check if someone kicked us for doing the nohz idle load balance.
- */
- if (unlikely(got_nohz_idle_kick())) {
- this_rq()->idle_balance = 1;
- raise_softirq_irqoff(SCHED_SOFTIRQ);
- }
- irq_exit();
}
static void ttwu_queue_remote(struct task_struct *p, int cpu, int wake_flags)
@@ -2336,9 +2328,9 @@ static void ttwu_queue_remote(struct task_struct *p, int cpu, int wake_flags)
p->sched_remote_wakeup = !!(wake_flags & WF_MIGRATED);
- if (llist_add(&p->wake_entry, &cpu_rq(cpu)->wake_list)) {
+ if (llist_add(&p->wake_entry, &rq->wake_list)) {
if (!set_nr_if_polling(rq->idle))
- smp_send_reschedule(cpu);
+ smp_call_function_single_async(cpu, &rq->wake_csd);
else
trace_sched_wake_idle_without_ipi(cpu);
}
@@ -6693,12 +6685,16 @@ void __init sched_init(void)
rq->avg_idle = 2*sysctl_sched_migration_cost;
rq->max_idle_balance_cost = sysctl_sched_migration_cost;
+ rq_csd_init(rq, &rq->wake_csd, wake_csd_func);
+
INIT_LIST_HEAD(&rq->cfs_tasks);
rq_attach_root(rq, &def_root_domain);
#ifdef CONFIG_NO_HZ_COMMON
rq->last_blocked_load_update_tick = jiffies;
atomic_set(&rq->nohz_flags, 0);
+
+ rq_csd_init(rq, &rq->nohz_csd, nohz_csd_func);
#endif
#endif /* CONFIG_SMP */
hrtick_rq_init(rq);
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 46b7bd4..6b7f147 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -10000,12 +10000,11 @@ static void kick_ilb(unsigned int flags)
return;
/*
- * Use smp_send_reschedule() instead of resched_cpu().
- * This way we generate a sched IPI on the target CPU which
+ * This way we generate an IPI on the target CPU which
* is idle. And the softirq performing nohz idle load balance
* will be run before returning from the IPI.
*/
- smp_send_reschedule(ilb_cpu);
+ smp_call_function_single_async(ilb_cpu, &cpu_rq(ilb_cpu)->nohz_csd);
}
/*
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 978c6fa..21416b3 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -889,9 +889,10 @@ struct rq {
#ifdef CONFIG_SMP
unsigned long last_blocked_load_update_tick;
unsigned int has_blocked_load;
+ call_single_data_t nohz_csd;
#endif /* CONFIG_SMP */
unsigned int nohz_tick_stopped;
- atomic_t nohz_flags;
+ atomic_t nohz_flags;
#endif /* CONFIG_NO_HZ_COMMON */
unsigned long nr_load_updates;
@@ -978,7 +979,7 @@ struct rq {
/* This is used to determine avg_idle's max value */
u64 max_idle_balance_cost;
-#endif
+#endif /* CONFIG_SMP */
#ifdef CONFIG_IRQ_TIME_ACCOUNTING
u64 prev_irq_time;
@@ -1020,6 +1021,7 @@ struct rq {
#endif
#ifdef CONFIG_SMP
+ call_single_data_t wake_csd;
struct llist_head wake_list;
#endif
next prev parent reply other threads:[~2020-05-12 15:13 UTC|newest]
Thread overview: 178+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-05 13:16 [patch V4 part 1 00/36] x86/entry: Entry/exception code rework, preparatory patches Thomas Gleixner
2020-05-05 13:16 ` [patch V4 part 1 01/36] rcu: Add comments marking transitions between RCU watching and not Thomas Gleixner
2020-05-05 13:16 ` [patch V4 part 1 02/36] x86/hw_breakpoint: Prevent data breakpoints on cpu_entry_area Thomas Gleixner
2020-05-06 8:14 ` Borislav Petkov
2020-05-06 12:11 ` Alexandre Chartre
2020-05-09 9:00 ` Lai Jiangshan
2020-05-09 9:23 ` Lai Jiangshan
2020-05-09 19:08 ` Andy Lutomirski
2020-05-19 19:58 ` [tip: x86/entry] " tip-bot2 for Andy Lutomirski
2020-05-05 13:16 ` [patch V4 part 1 03/36] sched: Clean up scheduler_ipi() Thomas Gleixner
2020-05-06 8:32 ` Thomas Gleixner
2020-05-06 8:40 ` Borislav Petkov
2020-05-06 9:12 ` Thomas Gleixner
2020-05-06 10:02 ` Borislav Petkov
2020-05-06 12:37 ` Alexandre Chartre
2020-05-06 15:03 ` Thomas Gleixner
2020-05-06 15:33 ` Peter Zijlstra
2020-05-06 18:28 ` Paul E. McKenney
2020-05-06 18:37 ` Peter Zijlstra
2020-05-06 18:46 ` Paul E. McKenney
2020-05-12 15:13 ` tip-bot2 for Peter Zijlstra (Intel) [this message]
2020-05-05 13:16 ` [patch V4 part 1 04/36] sched: Make scheduler_ipi inline Thomas Gleixner
2020-05-06 12:42 ` Alexandre Chartre
2020-05-12 15:13 ` [tip: sched/core] " tip-bot2 for Thomas Gleixner
2020-05-05 13:16 ` [patch V4 part 1 05/36] x86/entry: Flip _TIF_SIGPENDING and _TIF_NOTIFY_RESUME handling Thomas Gleixner
2020-05-06 11:53 ` Miroslav Benes
2020-05-06 12:06 ` Thomas Gleixner
2020-05-06 15:35 ` Peter Zijlstra
2020-05-06 13:06 ` Alexandre Chartre
2020-05-06 16:26 ` Borislav Petkov
2020-05-07 17:35 ` Andy Lutomirski
2020-05-13 20:56 ` Mathieu Desnoyers
2020-05-13 21:10 ` Steven Rostedt
2020-05-13 22:48 ` Mathieu Desnoyers
2020-05-14 0:12 ` Thomas Gleixner
2020-05-14 0:37 ` Steven Rostedt
2020-05-14 0:49 ` Thomas Gleixner
2020-05-14 1:22 ` Andy Lutomirski
2020-05-14 2:51 ` Mathieu Desnoyers
2020-05-14 9:19 ` Peter Zijlstra
2020-05-05 13:16 ` [patch V4 part 1 06/36] compiler: Simple READ/WRITE_ONCE() implementations Thomas Gleixner
2020-05-06 13:11 ` Alexandre Chartre
2020-05-06 13:33 ` Will Deacon
2020-05-06 15:36 ` Peter Zijlstra
2020-05-06 16:33 ` Borislav Petkov
2020-05-05 13:16 ` [patch V4 part 1 07/36] locking/atomics: Flip fallbacks and instrumentation Thomas Gleixner
2020-05-05 16:04 ` Mark Rutland
2020-05-07 23:41 ` Steven Rostedt
2020-05-08 8:40 ` Peter Zijlstra
2020-05-12 14:36 ` [tip: locking/kcsan] " tip-bot2 for Peter Zijlstra
2020-05-05 13:16 ` [patch V4 part 1 08/36] x86/doublefault: Remove memmove() call Thomas Gleixner
2020-05-06 13:47 ` Alexandre Chartre
2020-05-19 19:58 ` [tip: x86/entry] " tip-bot2 for Peter Zijlstra
2020-05-05 13:16 ` [patch V4 part 1 09/36] x86/entry/64: Avoid pointless code when CONTEXT_TRACKING=n Thomas Gleixner
2020-05-19 19:58 ` [tip: x86/entry] " tip-bot2 for Thomas Gleixner
2020-05-05 13:16 ` [patch V4 part 1 10/36] x86/entry: Remove the unused LOCKDEP_SYSEXIT cruft Thomas Gleixner
2020-05-06 13:52 ` Alexandre Chartre
2020-05-19 19:58 ` [tip: x86/entry] " tip-bot2 for Thomas Gleixner
2020-05-05 13:16 ` [patch V4 part 1 11/36] x86/kvm: Handle async page faults directly through do_page_fault() Thomas Gleixner
2020-05-06 7:00 ` Paolo Bonzini
2020-05-06 14:05 ` Alexandre Chartre
2020-05-19 19:58 ` [tip: x86/entry] " tip-bot2 for Andy Lutomirski
2020-05-05 13:16 ` [patch V4 part 1 12/36] x86/kvm: Sanitize kvm_async_pf_task_wait() Thomas Gleixner
2020-05-05 17:54 ` Paul E. McKenney
2020-05-05 21:50 ` Thomas Gleixner
2020-05-06 7:00 ` Paolo Bonzini
2020-05-06 12:53 ` Steven Rostedt
2020-05-06 15:13 ` Alexandre Chartre
2020-05-19 19:58 ` [tip: x86/entry] " tip-bot2 for Thomas Gleixner
2020-05-05 13:16 ` [patch V4 part 1 13/36] x86/kvm: Restrict ASYNC_PF to user space Thomas Gleixner
2020-05-06 7:00 ` Paolo Bonzini
2020-05-06 15:29 ` Alexandre Chartre
2020-05-19 19:58 ` [tip: x86/entry] " tip-bot2 for Thomas Gleixner
2020-05-05 13:16 ` [patch V4 part 1 14/36] x86/entry: Get rid of ist_begin/end_non_atomic() Thomas Gleixner
2020-05-06 15:34 ` Alexandre Chartre
2020-05-07 17:46 ` Andy Lutomirski
2020-05-13 22:57 ` Mathieu Desnoyers
2020-05-14 0:13 ` Steven Rostedt
2020-05-15 9:34 ` Thomas Gleixner
2020-05-15 13:11 ` Mathieu Desnoyers
2020-05-19 19:52 ` [tip: core/rcu] " tip-bot2 for Thomas Gleixner
2020-05-05 13:16 ` [patch V4 part 1 15/36] kprobes: Lock kprobe_mutex while showing kprobe_blacklist Thomas Gleixner
2020-05-06 15:38 ` Alexandre Chartre
2020-05-12 15:18 ` [tip: core/kprobes] " tip-bot2 for Masami Hiramatsu
2020-05-05 13:16 ` [patch V4 part 1 16/36] kprobes: Support __kprobes blacklist in modules Thomas Gleixner
2020-05-06 15:47 ` Alexandre Chartre
2020-05-12 15:18 ` [tip: core/kprobes] " tip-bot2 for Masami Hiramatsu
2020-05-05 13:16 ` [patch V4 part 1 17/36] kprobes: Support NOKPROBE_SYMBOL() " Thomas Gleixner
2020-05-06 15:54 ` Alexandre Chartre
2020-05-12 15:18 ` [tip: core/kprobes] " tip-bot2 for Masami Hiramatsu
2020-05-05 13:16 ` [patch V4 part 1 18/36] samples/kprobes: Add __kprobes and NOKPROBE_SYMBOL() for handlers Thomas Gleixner
2020-05-06 15:57 ` Alexandre Chartre
2020-05-12 15:18 ` [tip: core/kprobes] " tip-bot2 for Masami Hiramatsu
2020-05-05 13:16 ` [patch V4 part 1 19/36] x86/entry: Exclude low level entry code from sanitizing Thomas Gleixner
2020-05-05 20:39 ` Brian Gerst
2020-05-06 15:42 ` Peter Zijlstra
2020-05-06 16:03 ` Alexandre Chartre
2020-05-13 22:58 ` Mathieu Desnoyers
2020-05-19 19:58 ` [tip: x86/entry] " tip-bot2 for Peter Zijlstra
2020-05-05 13:16 ` [patch V4 part 1 20/36] vmlinux.lds.h: Create section for protection against instrumentation Thomas Gleixner
2020-05-06 16:08 ` Sean Christopherson
2020-05-06 16:28 ` Peter Zijlstra
2020-05-06 16:57 ` Thomas Gleixner
2020-05-19 19:52 ` [tip: core/rcu] " tip-bot2 for Thomas Gleixner
2020-05-05 13:16 ` [patch V4 part 1 21/36] kprobes: Prevent probes in .noinstr.text section Thomas Gleixner
2020-05-08 6:30 ` Masami Hiramatsu
2020-05-19 19:52 ` [tip: core/kprobes] " tip-bot2 for Thomas Gleixner
2020-05-05 13:16 ` [patch V4 part 1 22/36] tracing: Provide lockdep less trace_hardirqs_on/off() variants Thomas Gleixner
2020-05-07 17:55 ` Andy Lutomirski
2020-05-07 18:52 ` Thomas Gleixner
2020-05-19 19:58 ` [tip: x86/entry] " tip-bot2 for Thomas Gleixner
2020-05-05 13:16 ` [patch V4 part 1 23/36] bug: Annotate WARN/BUG/stackfail as noinstr safe Thomas Gleixner
2020-05-13 23:12 ` Mathieu Desnoyers
2020-05-19 19:58 ` [tip: x86/entry] " tip-bot2 for Thomas Gleixner
2020-05-05 13:16 ` [patch V4 part 1 24/36] lockdep: Prepare for noinstr sections Thomas Gleixner
2020-05-08 0:23 ` Steven Rostedt
2020-05-08 8:44 ` Peter Zijlstra
2020-05-19 19:58 ` [tip: x86/entry] " tip-bot2 for Peter Zijlstra
2020-05-05 13:16 ` [patch V4 part 1 25/36] rcu/tree: Mark the idle relevant functions noinstr Thomas Gleixner
2020-05-05 18:07 ` Paul E. McKenney
2020-05-19 19:48 ` Joel Fernandes
2020-05-19 19:52 ` [tip: core/rcu] " tip-bot2 for Thomas Gleixner
2020-09-28 22:22 ` Kim Phillips
2020-09-28 22:55 ` Paul E. McKenney
2020-09-29 7:25 ` Peter Zijlstra
2020-09-29 11:25 ` Peter Zijlstra
2020-09-29 14:34 ` Steven Rostedt
2020-09-29 14:52 ` Peter Zijlstra
2020-05-05 13:16 ` [patch V4 part 1 26/36] printk: Prepare for nested printk_nmi_enter() Thomas Gleixner
2020-05-19 19:52 ` [tip: core/rcu] " tip-bot2 for Petr Mladek
2020-05-05 13:16 ` [patch V4 part 1 27/36] arm64: Prepare arch_nmi_enter() for recursion Thomas Gleixner
2020-05-13 23:28 ` Mathieu Desnoyers
2020-05-15 14:04 ` Frederic Weisbecker
2020-05-15 15:45 ` Will Deacon
2020-05-15 16:01 ` Mathieu Desnoyers
2020-05-15 21:29 ` Thomas Gleixner
2020-05-15 21:31 ` Frederic Weisbecker
2020-05-19 19:52 ` [tip: core/rcu] " tip-bot2 for Frederic Weisbecker
2020-05-05 13:16 ` [patch V4 part 1 28/36] hardirq/nmi: Allow nested nmi_enter() Thomas Gleixner
2020-05-19 19:52 ` [tip: core/rcu] " tip-bot2 for Peter Zijlstra
2020-05-05 13:16 ` [patch V4 part 1 29/36] x86/mce: Send #MC singal from task work Thomas Gleixner
2020-05-07 18:02 ` Andy Lutomirski
2020-05-08 8:48 ` Peter Zijlstra
2020-05-08 21:30 ` Andy Lutomirski
2020-05-14 14:16 ` Borislav Petkov
2020-05-13 23:42 ` Mathieu Desnoyers
2020-05-14 17:38 ` Thomas Gleixner
2020-05-14 17:42 ` Mathieu Desnoyers
2020-05-14 14:17 ` Borislav Petkov
2020-05-14 16:03 ` Mathieu Desnoyers
2020-05-14 16:19 ` Andy Lutomirski
2020-05-14 16:39 ` Borislav Petkov
2020-05-14 17:05 ` Mathieu Desnoyers
2020-05-19 19:52 ` [tip: core/rcu] " tip-bot2 for Peter Zijlstra
2020-05-05 13:16 ` [patch V4 part 1 30/36] lockdep: Always inline lockdep_{off,on}() Thomas Gleixner
2020-05-13 23:46 ` Mathieu Desnoyers
2020-05-19 19:52 ` [tip: core/rcu] " tip-bot2 for Peter Zijlstra
2020-05-05 13:16 ` [patch V4 part 1 31/36] printk: Disallow instrumenting print_nmi_enter() Thomas Gleixner
2020-05-19 19:52 ` [tip: core/rcu] " tip-bot2 for Peter Zijlstra
2020-05-05 13:16 ` [patch V4 part 1 32/36] sh/ftrace: Move arch_ftrace_nmi_{enter,exit} into nmi exception Thomas Gleixner
2020-05-08 0:34 ` Steven Rostedt
2020-05-19 19:52 ` [tip: core/rcu] " tip-bot2 for Peter Zijlstra
2020-05-05 13:16 ` [patch V4 part 1 33/36] x86,tracing: Robustify ftrace_nmi_enter() Thomas Gleixner
2020-05-08 6:19 ` Masami Hiramatsu
2020-05-05 13:16 ` [patch V4 part 1 34/36] sched,rcu,tracing: Avoid tracing before in_nmi() is correct Thomas Gleixner
2020-05-19 19:52 ` [tip: core/rcu] " tip-bot2 for Peter Zijlstra
2020-05-05 13:16 ` [patch V4 part 1 35/36] x86: Replace ist_enter() with nmi_enter() Thomas Gleixner
2020-05-07 18:04 ` Andy Lutomirski
2020-05-07 18:17 ` Mathieu Desnoyers
2020-05-08 8:50 ` Peter Zijlstra
2020-05-08 17:12 ` Josh Poimboeuf
2020-05-14 0:12 ` Mathieu Desnoyers
2020-05-19 19:52 ` [tip: core/rcu] " tip-bot2 for Peter Zijlstra
2020-05-05 13:16 ` [patch V4 part 1 36/36] rcu: Make RCU IRQ enter/exit functions rely on in_nmi() Thomas Gleixner
2020-05-05 18:13 ` Paul E. McKenney
2020-05-06 17:09 ` Alexandre Chartre
2020-05-19 19:52 ` [tip: core/rcu] " tip-bot2 for Paul E. McKenney
2020-05-07 18:05 ` [patch V4 part 1 00/36] x86/entry: Entry/exception code rework, preparatory patches Andy Lutomirski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=158929640713.390.10558251425748115488.tip-bot2@tip-bot2 \
--to=tip-bot2@linutronix.de \
--cc=alexandre.chartre@oracle.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tip-commits@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).