* [PATCH 00/17] Dovetail integration - the finals
@ 2021-06-11 18:05 Jan Kiszka
2021-06-11 18:05 ` [PATCH 01/17] cobalt/thread: Rework kthread check for xnthread_signal Jan Kiszka
` (17 more replies)
0 siblings, 18 replies; 25+ messages in thread
From: Jan Kiszka @ 2021-06-11 18:05 UTC (permalink / raw)
To: xenomai
Now the final bits to git dovetail / 5.10 support into mainline. The
last missing piece was the avoidance xnmalloc in critical paths when
submitting inband work. I've used Hongzhan's patch for xnthread_signal
and also Philippe's patches for the remaining cases. The outcome seems
to work fine for both I-pipe and Dovetail now - but please review!
Not included are necessary RTDM extensions and the addition of latmus as
currently still queued in wip/dovetail. There were some open issues, but
those could not be easily addressed on top of next.
On top, I would also still like to understand - and ideally address
prior to releasing a new major version - if we cannot align the ARM
syscall ABI of Xenomai to the kernel /wrt argument passing. Now is a
good chance to do that.
Jan
CC: Philippe Gerum <rpm@xenomai.org>
Jan Kiszka (9):
cobalt/thread: Rework kthread check for xnthread_signal
cobalt/thread: Move xnthread_signal work off the stack
cobalt/thread: Privatize xnthread_map to map_kthread
cobalt/thread: Pull kthread inband work onto creator stack
cobalt/thread: Make sure relax inband work is on a stack outliving the
wakeup
lib/cobalt: Reorder low_init for having cobalt_use_legacy_tsc earlier
available
cobalt/tick: dovetail: Drop pointless inline specifier from
pipeline_must_force_program_tick
cobalt/syscall: Account for different syscall argument marshaling
ci: Add arm, arm64 and x86 dovetail targets for kernel 5.10
Philippe Gerum (8):
cobalt/thread: dovetail: keep hard irqs off on transition to in-band
cobalt/sched: dovetail: prevent early scheduling on uninit runqueues
cobalt/arm: dovetail: add architecture bits
lib/cobalt/arm: dovetail: skip detection of KUSER_TSC support
lib/cobalt/arm64: dovetail: skip detection of KUSER_TSC support
cobalt/tick: dovetail: improve accuracy of the host tick delay
cobalt/clock: dovetail: abstract interface to hardware TSC
cobalt/arm64: dovetail: add architecture bits
.gitlab-ci.yml | 26 ++
.../kernel/dovetail/pipeline/pipeline.h | 11 +
.../cobalt/kernel/dovetail/pipeline/sched.h | 12 +
.../cobalt/kernel/dovetail/pipeline/tick.h | 2 +-
.../cobalt/kernel/ipipe/pipeline/pipeline.h | 12 +
include/cobalt/kernel/ipipe/pipeline/sched.h | 20 +
include/cobalt/kernel/thread.h | 24 +-
include/cobalt/sys/cobalt.h | 2 +
include/copperplate/clockobj.h | 12 +-
kernel/cobalt/arch/arm/Kconfig | 4 +
kernel/cobalt/arch/arm/dovetail/Makefile | 5 +
.../include/asm/xenomai/calibration.h | 39 ++
.../dovetail/include/asm/xenomai/features.h | 30 ++
.../arm/dovetail/include/asm/xenomai/fptest.h | 52 +++
.../dovetail}/include/asm/xenomai/machine.h | 67 +--
.../dovetail/include/asm/xenomai/syscall.h | 101 +++++
.../dovetail/include/asm/xenomai/syscall32.h | 24 ++
.../arm/dovetail/include/asm/xenomai/thread.h | 32 ++
.../dovetail/include/asm/xenomai/wrappers.h | 27 ++
kernel/cobalt/arch/arm/dovetail/machine.c | 44 ++
kernel/cobalt/arch/arm64/dovetail/Makefile | 5 +
.../include/asm/xenomai/calibration.h | 24 ++
.../dovetail/include/asm/xenomai/features.h | 15 +
.../dovetail/include/asm/xenomai/fptest.h | 35 ++
.../dovetail/include/asm/xenomai/machine.h | 33 ++
.../dovetail/include/asm/xenomai/syscall.h | 63 +++
.../dovetail/include/asm/xenomai/syscall32.h | 12 +
.../dovetail/include/asm/xenomai/thread.h | 22 +
.../dovetail/include/asm/xenomai/wrappers.h | 15 +
kernel/cobalt/arch/arm64/dovetail/machine.c | 41 ++
.../arm64/ipipe/include/asm/xenomai/machine.h | 3 +
kernel/cobalt/arch/arm64/ipipe/machine.c | 3 +
.../dovetail/include/asm/xenomai/syscall.h | 5 -
kernel/cobalt/dovetail/kevents.c | 2 +-
kernel/cobalt/dovetail/sched.c | 1 +
kernel/cobalt/dovetail/tick.c | 14 +-
.../include/asm-generic/xenomai/syscall.h | 7 -
kernel/cobalt/ipipe/kevents.c | 2 +-
kernel/cobalt/posix/syscall.c | 14 +-
kernel/cobalt/sched.c | 10 +-
kernel/cobalt/synch.c | 4 +-
kernel/cobalt/thread.c | 402 ++++++++----------
lib/cobalt/arch/arm/features.c | 4 +
lib/cobalt/arch/arm/include/asm/xenomai/tsc.h | 2 +-
lib/cobalt/arch/arm64/features.c | 4 +
.../arch/arm64/include/asm/xenomai/tsc.h | 2 +-
.../arch/powerpc/include/asm/xenomai/tsc.h | 2 +-
lib/cobalt/arch/x86/include/asm/xenomai/tsc.h | 2 +-
lib/cobalt/clock.c | 6 +-
lib/cobalt/init.c | 2 +-
lib/cobalt/internal.c | 18 +-
lib/copperplate/clockobj.c | 12 +-
testsuite/smokey/tsc/tsc.c | 2 +-
53 files changed, 1030 insertions(+), 304 deletions(-)
create mode 100644 kernel/cobalt/arch/arm/dovetail/Makefile
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/calibration.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/features.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/fptest.h
copy kernel/cobalt/arch/{arm64/ipipe => arm/dovetail}/include/asm/xenomai/machine.h (59%)
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall32.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/thread.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/wrappers.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/machine.c
create mode 100644 kernel/cobalt/arch/arm64/dovetail/Makefile
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/calibration.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/features.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/fptest.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/machine.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall32.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/thread.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/wrappers.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/machine.c
--
2.26.2
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH 01/17] cobalt/thread: Rework kthread check for xnthread_signal
2021-06-11 18:05 [PATCH 00/17] Dovetail integration - the finals Jan Kiszka
@ 2021-06-11 18:05 ` Jan Kiszka
2021-06-11 18:05 ` [PATCH 02/17] cobalt/thread: Move xnthread_signal work off the stack Jan Kiszka
` (16 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Jan Kiszka @ 2021-06-11 18:05 UTC (permalink / raw)
To: xenomai
From: Jan Kiszka <jan.kiszka@siemens.com>
We only reach lostage_task_signal for a kthread if xnthread_signal has
sent us. So move the check to the caller, making debugging easier.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
kernel/cobalt/thread.c | 19 +++----------------
1 file changed, 3 insertions(+), 16 deletions(-)
diff --git a/kernel/cobalt/thread.c b/kernel/cobalt/thread.c
index d3a827eaa..6a3e3fa18 100644
--- a/kernel/cobalt/thread.c
+++ b/kernel/cobalt/thread.c
@@ -2089,19 +2089,9 @@ struct lostage_signal {
int signo, sigval;
};
-static inline void do_kthread_signal(struct task_struct *p,
- struct xnthread *thread,
- struct lostage_signal *rq)
-{
- printk(XENO_WARNING
- "kernel shadow %s received unhandled signal %d (action=0x%x)\n",
- thread->name, rq->signo, rq->sigval);
-}
-
static void lostage_task_signal(struct pipeline_inband_work *inband_work)
{
struct lostage_signal *rq;
- struct xnthread *thread;
struct task_struct *p;
kernel_siginfo_t si;
int signo;
@@ -2109,12 +2099,6 @@ static void lostage_task_signal(struct pipeline_inband_work *inband_work)
rq = container_of(inband_work, struct lostage_signal, inband_work);
p = rq->task;
- thread = xnthread_from_task(p);
- if (thread && !xnthread_test_state(thread, XNUSER)) {
- do_kthread_signal(p, thread, rq);
- return;
- }
-
signo = rq->signo;
trace_cobalt_lostage_signal(p, signo);
@@ -2298,6 +2282,9 @@ void xnthread_signal(struct xnthread *thread, int sig, int arg)
.sigval = sig == SIGDEBUG ? arg | sigdebug_marker : arg,
};
+ if (XENO_WARN_ON(COBALT, !xnthread_test_state(thread, XNUSER)))
+ return;
+
trace_cobalt_lostage_request("signal", sigwork.task);
pipeline_post_inband_work(&sigwork);
--
2.26.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 02/17] cobalt/thread: Move xnthread_signal work off the stack
2021-06-11 18:05 [PATCH 00/17] Dovetail integration - the finals Jan Kiszka
2021-06-11 18:05 ` [PATCH 01/17] cobalt/thread: Rework kthread check for xnthread_signal Jan Kiszka
@ 2021-06-11 18:05 ` Jan Kiszka
2021-06-11 18:05 ` [PATCH 03/17] cobalt/thread: Privatize xnthread_map to map_kthread Jan Kiszka
` (15 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Jan Kiszka @ 2021-06-11 18:05 UTC (permalink / raw)
To: xenomai
From: Jan Kiszka <jan.kiszka@siemens.com>
Unlike the I-pipe, Dovetail does not copy the work descriptor but
merely hands over the request to the common irq_work() mechanism. We
must guarantee that such descriptor lives in a portion of memory which
won't go stale until the handler has run, which by design can only
happen once the calling out-of-band context unwinds.
Therefore, we have to create signal slots per possible cause in the
thread's control block in order to overcome sigwork in xnthread_signal.
For SIGDEBUG, we are only interested in the very first event coming in,
so one slot is enough. All SIGSHADOW_* events need their own slot:
SIGSHADOW_ACTION_HARDEN and SIGSHADOW_ACTION_HOME can be raised by
remote threads asynchronously to the target thread.
SIGSHADOW_ACTION_BACKTRACE comes in addition to SIGDEBUG_MIGRATE_*. For
the latter reason, SIGSHADOW_ACTION_BACKTRACE cannot pile up though.
Including SIGTERM, we have totally 5 slots.
To ensure that multiple asynchronous signals on the same slot do not
overwrite each other (e.g. SIGSHADOW_ACTION_HOME or SIGDEBUG reasons),
synchronize the slot usage under nklock. Create a __xnthread_signal
variant that can be called from already locked contexts (the majority of
xnthread_signal users).
Based on original patch by Hongzhan Chen.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
include/cobalt/kernel/thread.h | 21 ++++++-
kernel/cobalt/dovetail/kevents.c | 2 +-
kernel/cobalt/ipipe/kevents.c | 2 +-
kernel/cobalt/synch.c | 4 +-
kernel/cobalt/thread.c | 103 +++++++++++++++++++++++--------
5 files changed, 101 insertions(+), 31 deletions(-)
diff --git a/include/cobalt/kernel/thread.h b/include/cobalt/kernel/thread.h
index c92037bfe..2d6ce9053 100644
--- a/include/cobalt/kernel/thread.h
+++ b/include/cobalt/kernel/thread.h
@@ -23,6 +23,7 @@
#include <linux/sched.h>
#include <linux/sched/rt.h>
#include <pipeline/thread.h>
+#include <pipeline/inband_work.h>
#include <cobalt/kernel/list.h>
#include <cobalt/kernel/stat.h>
#include <cobalt/kernel/timer.h>
@@ -42,6 +43,13 @@
#define XNTHREAD_BLOCK_BITS (XNSUSP|XNPEND|XNDELAY|XNDORMANT|XNRELAX|XNHELD|XNDBGSTOP)
#define XNTHREAD_MODE_BITS (XNRRB|XNWARN|XNTRAPLB)
+#define XNTHREAD_SIGDEBUG 0
+#define XNTHREAD_SIGSHADOW_HARDEN 1
+#define XNTHREAD_SIGSHADOW_BACKTRACE 2
+#define XNTHREAD_SIGSHADOW_HOME 3
+#define XNTHREAD_SIGTERM 4
+#define XNTHREAD_MAX_SIGNALS 5
+
struct xnthread;
struct xnsched;
struct xnselector;
@@ -50,6 +58,13 @@ struct xnsched_tpslot;
struct xnthread_personality;
struct completion;
+struct lostage_signal {
+ struct pipeline_inband_work inband_work; /* Must be first. */
+ struct task_struct *task;
+ int signo, sigval;
+ struct lostage_signal *self; /* Revisit: I-pipe requirement */
+};
+
struct xnthread_init_attr {
struct xnthread_personality *personality;
cpumask_t affinity;
@@ -199,6 +214,7 @@ struct xnthread {
const char *exe_path; /* Executable path */
u32 proghash; /* Hash value for exe_path */
#endif
+ struct lostage_signal sigarray[XNTHREAD_MAX_SIGNALS];
};
static inline int xnthread_get_state(const struct xnthread *thread)
@@ -492,8 +508,9 @@ void __xnthread_demote(struct xnthread *thread);
void xnthread_demote(struct xnthread *thread);
-void xnthread_signal(struct xnthread *thread,
- int sig, int arg);
+void __xnthread_signal(struct xnthread *thread, int sig, int arg);
+
+void xnthread_signal(struct xnthread *thread, int sig, int arg);
void xnthread_pin_initial(struct xnthread *thread);
diff --git a/kernel/cobalt/dovetail/kevents.c b/kernel/cobalt/dovetail/kevents.c
index 3987d119a..c07c08a6e 100644
--- a/kernel/cobalt/dovetail/kevents.c
+++ b/kernel/cobalt/dovetail/kevents.c
@@ -188,7 +188,7 @@ static int handle_setaffinity_event(struct dovetail_migration_data *d)
xnlock_get_irqsave(&nklock, s);
if (xnthread_test_state(thread, XNTHREAD_BLOCK_BITS & ~XNRELAX))
- xnthread_signal(thread, SIGSHADOW, SIGSHADOW_ACTION_HARDEN);
+ __xnthread_signal(thread, SIGSHADOW, SIGSHADOW_ACTION_HARDEN);
xnlock_put_irqrestore(&nklock, s);
diff --git a/kernel/cobalt/ipipe/kevents.c b/kernel/cobalt/ipipe/kevents.c
index 617e5cb6a..19970c8b8 100644
--- a/kernel/cobalt/ipipe/kevents.c
+++ b/kernel/cobalt/ipipe/kevents.c
@@ -261,7 +261,7 @@ static int handle_setaffinity_event(struct ipipe_cpu_migration_data *d)
xnlock_get_irqsave(&nklock, s);
if (xnthread_test_state(thread, XNTHREAD_BLOCK_BITS & ~XNRELAX))
- xnthread_signal(thread, SIGSHADOW, SIGSHADOW_ACTION_HARDEN);
+ __xnthread_signal(thread, SIGSHADOW, SIGSHADOW_ACTION_HARDEN);
xnlock_put_irqrestore(&nklock, s);
diff --git a/kernel/cobalt/synch.c b/kernel/cobalt/synch.c
index c65c07aac..911bc83f6 100644
--- a/kernel/cobalt/synch.c
+++ b/kernel/cobalt/synch.c
@@ -1147,7 +1147,7 @@ void xnsynch_detect_relaxed_owner(struct xnsynch *synch,
!xnthread_test_info(sleeper, XNPIALERT) &&
xnthread_test_state(synch->owner, XNRELAX)) {
xnthread_set_info(sleeper, XNPIALERT);
- xnthread_signal(sleeper, SIGDEBUG,
+ __xnthread_signal(sleeper, SIGDEBUG,
SIGDEBUG_MIGRATE_PRIOINV);
} else
xnthread_clear_info(sleeper, XNPIALERT);
@@ -1171,7 +1171,7 @@ void xnsynch_detect_boosted_relax(struct xnthread *owner)
xnsynch_for_each_sleeper(sleeper, synch) {
if (xnthread_test_state(sleeper, XNWARN)) {
xnthread_set_info(sleeper, XNPIALERT);
- xnthread_signal(sleeper, SIGDEBUG,
+ __xnthread_signal(sleeper, SIGDEBUG,
SIGDEBUG_MIGRATE_PRIOINV);
}
}
diff --git a/kernel/cobalt/thread.c b/kernel/cobalt/thread.c
index 6a3e3fa18..1f8e1119a 100644
--- a/kernel/cobalt/thread.c
+++ b/kernel/cobalt/thread.c
@@ -948,7 +948,7 @@ void xnthread_suspend(struct xnthread *thread, int mask,
*/
if (((oldstate & (XNTHREAD_BLOCK_BITS|XNUSER)) == (XNRELAX|XNUSER)) &&
(mask & (XNDELAY | XNSUSP | XNHELD)) != 0)
- xnthread_signal(thread, SIGSHADOW, SIGSHADOW_ACTION_HARDEN);
+ __xnthread_signal(thread, SIGSHADOW, SIGSHADOW_ACTION_HARDEN);
out:
xnlock_put_irqrestore(&nklock, s);
return;
@@ -959,7 +959,7 @@ lock_break:
!xnthread_test_localinfo(thread, XNLBALERT)) {
xnthread_set_info(thread, XNKICKED);
xnthread_set_localinfo(thread, XNLBALERT);
- xnthread_signal(thread, SIGDEBUG, SIGDEBUG_LOCK_BREAK);
+ __xnthread_signal(thread, SIGDEBUG, SIGDEBUG_LOCK_BREAK);
}
abort:
if (wchan) {
@@ -1492,7 +1492,7 @@ check_self_cancel:
*/
if (xnthread_test_state(thread, XNUSER)) {
__xnthread_demote(thread);
- xnthread_signal(thread, SIGTERM, 0);
+ __xnthread_signal(thread, SIGTERM, 0);
} else
__xnthread_kick(thread);
out:
@@ -1803,7 +1803,7 @@ int __xnthread_set_schedparam(struct xnthread *thread,
xnthread_set_info(thread, XNSCHEDP);
/* Ask the target thread to call back if relaxed. */
if (xnthread_test_state(thread, XNRELAX))
- xnthread_signal(thread, SIGSHADOW, SIGSHADOW_ACTION_HOME);
+ __xnthread_signal(thread, SIGSHADOW, SIGSHADOW_ACTION_HOME);
return ret;
}
@@ -2083,23 +2083,29 @@ void xnthread_relax(int notify, int reason)
}
EXPORT_SYMBOL_GPL(xnthread_relax);
-struct lostage_signal {
- struct pipeline_inband_work inband_work; /* Must be first. */
- struct task_struct *task;
- int signo, sigval;
-};
-
static void lostage_task_signal(struct pipeline_inband_work *inband_work)
{
struct lostage_signal *rq;
struct task_struct *p;
kernel_siginfo_t si;
- int signo;
+ int signo, sigval;
+ spl_t s;
rq = container_of(inband_work, struct lostage_signal, inband_work);
- p = rq->task;
+ /*
+ * Revisit: I-pipe requirement. It passes a copy of the original work
+ * struct, so retrieve the original one first in order to update is.
+ */
+ rq = rq->self;
+ xnlock_get_irqsave(&nklock, s);
+
+ p = rq->task;
signo = rq->signo;
+ sigval = rq->sigval;
+ rq->task = NULL;
+
+ xnlock_put_irqrestore(&nklock, s);
trace_cobalt_lostage_signal(p, signo);
@@ -2107,10 +2113,11 @@ static void lostage_task_signal(struct pipeline_inband_work *inband_work)
memset(&si, '\0', sizeof(si));
si.si_signo = signo;
si.si_code = SI_QUEUE;
- si.si_int = rq->sigval;
+ si.si_int = sigval;
send_sig_info(signo, &si, p);
- } else
+ } else {
send_sig(signo, p, 1);
+ }
}
static int force_wakeup(struct xnthread *thread) /* nklock locked, irqs off */
@@ -2272,22 +2279,68 @@ void xnthread_demote(struct xnthread *thread)
}
EXPORT_SYMBOL_GPL(xnthread_demote);
-void xnthread_signal(struct xnthread *thread, int sig, int arg)
+static int get_slot_index_from_sig(int sig, int arg)
{
- struct lostage_signal sigwork = {
- .inband_work = PIPELINE_INBAND_WORK_INITIALIZER(sigwork,
- lostage_task_signal),
- .task = xnthread_host_task(thread),
- .signo = sig,
- .sigval = sig == SIGDEBUG ? arg | sigdebug_marker : arg,
- };
+ int action;
+
+ switch (sig) {
+ case SIGDEBUG:
+ return XNTHREAD_SIGDEBUG;
+ case SIGSHADOW:
+ action = sigshadow_action(arg);
+ switch (action) {
+ case SIGSHADOW_ACTION_HARDEN:
+ return XNTHREAD_SIGSHADOW_HARDEN;
+ case SIGSHADOW_ACTION_BACKTRACE:
+ return XNTHREAD_SIGSHADOW_BACKTRACE;
+ case SIGSHADOW_ACTION_HOME:
+ return XNTHREAD_SIGSHADOW_HOME;
+ }
+ break;
+ case SIGTERM:
+ return XNTHREAD_SIGTERM;
+ }
+
+ return -1;
+}
+
+/* nklock locked, irqs off */
+void __xnthread_signal(struct xnthread *thread, int sig, int arg)
+{
+ struct lostage_signal *sigwork;
+ int slot;
if (XENO_WARN_ON(COBALT, !xnthread_test_state(thread, XNUSER)))
return;
- trace_cobalt_lostage_request("signal", sigwork.task);
+ slot = get_slot_index_from_sig(sig, arg);
+ if (WARN_ON_ONCE(slot < 0))
+ return;
+
+ sigwork = &thread->sigarray[slot];
+ if (sigwork->task)
+ return;
+
+ sigwork->inband_work = (struct pipeline_inband_work)
+ PIPELINE_INBAND_WORK_INITIALIZER(*sigwork,
+ lostage_task_signal);
+ sigwork->task = xnthread_host_task(thread);
+ sigwork->signo = sig;
+ sigwork->sigval = sig == SIGDEBUG ? arg | sigdebug_marker : arg;
+ sigwork->self = sigwork; /* Revisit: I-pipe requirement */
+
+ trace_cobalt_lostage_request("signal", sigwork->task);
+
+ pipeline_post_inband_work(sigwork);
+}
+
+void xnthread_signal(struct xnthread *thread, int sig, int arg)
+{
+ spl_t s;
- pipeline_post_inband_work(&sigwork);
+ xnlock_get_irqsave(&nklock, s);
+ __xnthread_signal(thread, sig, arg);
+ xnlock_put_irqrestore(&nklock, s);
}
EXPORT_SYMBOL_GPL(xnthread_signal);
@@ -2469,7 +2522,7 @@ void xnthread_call_mayday(struct xnthread *thread, int reason)
/* Mayday traps are available to userland threads only. */
XENO_BUG_ON(COBALT, !xnthread_test_state(thread, XNUSER));
xnthread_set_info(thread, XNKICKED);
- xnthread_signal(thread, SIGDEBUG, reason);
+ __xnthread_signal(thread, SIGDEBUG, reason);
pipeline_raise_mayday(p);
}
EXPORT_SYMBOL_GPL(xnthread_call_mayday);
--
2.26.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 03/17] cobalt/thread: Privatize xnthread_map to map_kthread
2021-06-11 18:05 [PATCH 00/17] Dovetail integration - the finals Jan Kiszka
2021-06-11 18:05 ` [PATCH 01/17] cobalt/thread: Rework kthread check for xnthread_signal Jan Kiszka
2021-06-11 18:05 ` [PATCH 02/17] cobalt/thread: Move xnthread_signal work off the stack Jan Kiszka
@ 2021-06-11 18:05 ` Jan Kiszka
2021-06-11 18:05 ` [PATCH 04/17] cobalt/thread: Pull kthread inband work onto creator stack Jan Kiszka
` (14 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Jan Kiszka @ 2021-06-11 18:05 UTC (permalink / raw)
To: xenomai
From: Jan Kiszka <jan.kiszka@siemens.com>
No user outside of the core, no need to export it. Move the code around
so that we can easily use it without forward declarations.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
include/cobalt/kernel/thread.h | 3 -
kernel/cobalt/thread.c | 239 ++++++++++++++-------------------
2 files changed, 98 insertions(+), 144 deletions(-)
diff --git a/include/cobalt/kernel/thread.h b/include/cobalt/kernel/thread.h
index 2d6ce9053..b79cb8429 100644
--- a/include/cobalt/kernel/thread.h
+++ b/include/cobalt/kernel/thread.h
@@ -514,9 +514,6 @@ void xnthread_signal(struct xnthread *thread, int sig, int arg);
void xnthread_pin_initial(struct xnthread *thread);
-int xnthread_map(struct xnthread *thread,
- struct completion *done);
-
void xnthread_call_mayday(struct xnthread *thread, int reason);
static inline void xnthread_get_resource(struct xnthread *curr)
diff --git a/kernel/cobalt/thread.c b/kernel/cobalt/thread.c
index 1f8e1119a..db68833c6 100644
--- a/kernel/cobalt/thread.c
+++ b/kernel/cobalt/thread.c
@@ -84,6 +84,103 @@ static inline void enlist_new_thread(struct xnthread *thread)
xnvfile_touch_tag(&nkthreadlist_tag);
}
+struct parent_wakeup_request {
+ struct pipeline_inband_work inband_work; /* Must be first. */
+ struct completion *done;
+};
+
+static void do_parent_wakeup(struct pipeline_inband_work *inband_work)
+{
+ struct parent_wakeup_request *rq;
+
+ rq = container_of(inband_work, struct parent_wakeup_request, inband_work);
+ complete(rq->done);
+}
+
+static inline void wakeup_parent(struct completion *done)
+{
+ struct parent_wakeup_request wakework = {
+ .inband_work = PIPELINE_INBAND_WORK_INITIALIZER(wakework,
+ do_parent_wakeup),
+ .done = done,
+ };
+
+ trace_cobalt_lostage_request("wakeup", current);
+
+ pipeline_post_inband_work(&wakework);
+}
+
+static inline void init_kthread_info(struct xnthread *thread)
+{
+ struct cobalt_threadinfo *p;
+
+ p = pipeline_current();
+ p->thread = thread;
+ p->process = NULL;
+}
+
+static int map_kthread(struct xnthread *thread, struct completion *done)
+{
+ int ret;
+ spl_t s;
+
+ if (xnthread_test_state(thread, XNUSER))
+ return -EINVAL;
+
+ if (xnthread_current() || xnthread_test_state(thread, XNMAPPED))
+ return -EBUSY;
+
+ thread->u_window = NULL;
+ xnthread_pin_initial(thread);
+
+ pipeline_init_shadow_tcb(thread);
+ xnthread_suspend(thread, XNRELAX, XN_INFINITE, XN_RELATIVE, NULL);
+ init_kthread_info(thread);
+ xnthread_set_state(thread, XNMAPPED);
+ xndebug_shadow_init(thread);
+ xnthread_run_handler(thread, map_thread);
+ pipeline_enable_kevents();
+
+ /*
+ * CAUTION: Soon after xnthread_init() has returned,
+ * xnthread_start() is commonly invoked from the root domain,
+ * therefore the call site may expect the started kernel
+ * shadow to preempt immediately. As a result of such
+ * assumption, start attributes (struct xnthread_start_attr)
+ * are often laid on the caller's stack.
+ *
+ * For this reason, we raise the completion signal to wake up
+ * the xnthread_init() caller only once the emerging thread is
+ * hardened, and __never__ before that point. Since we run
+ * over the Xenomai domain upon return from xnthread_harden(),
+ * we schedule a virtual interrupt handler in the root domain
+ * to signal the completion object.
+ */
+ xnthread_resume(thread, XNDORMANT);
+ ret = xnthread_harden();
+ wakeup_parent(done);
+
+ xnlock_get_irqsave(&nklock, s);
+
+ enlist_new_thread(thread);
+ /*
+ * Make sure xnthread_start() did not slip in from another CPU
+ * while we were back from wakeup_parent().
+ */
+ if (thread->entry == NULL)
+ xnthread_suspend(thread, XNDORMANT,
+ XN_INFINITE, XN_RELATIVE, NULL);
+
+ xnlock_put_irqrestore(&nklock, s);
+
+ xnthread_test_cancel();
+
+ xntrace_pid(xnthread_host_pid(thread),
+ xnthread_current_priority(thread));
+
+ return ret;
+}
+
struct kthread_arg {
struct xnthread *thread;
struct completion *done;
@@ -113,7 +210,7 @@ static int kthread_trampoline(void *arg)
param.sched_priority = prio;
sched_setscheduler(current, policy, ¶m);
- ret = xnthread_map(thread, ka->done);
+ ret = map_kthread(thread, ka->done);
if (ret) {
printk(XENO_WARNING "failed to create kernel shadow %s\n",
thread->name);
@@ -2374,146 +2471,6 @@ void xnthread_pin_initial(struct xnthread *thread)
xnlock_put_irqrestore(&nklock, s);
}
-struct parent_wakeup_request {
- struct pipeline_inband_work inband_work; /* Must be first. */
- struct completion *done;
-};
-
-static void do_parent_wakeup(struct pipeline_inband_work *inband_work)
-{
- struct parent_wakeup_request *rq;
-
- rq = container_of(inband_work, struct parent_wakeup_request, inband_work);
- complete(rq->done);
-}
-
-static inline void wakeup_parent(struct completion *done)
-{
- struct parent_wakeup_request wakework = {
- .inband_work = PIPELINE_INBAND_WORK_INITIALIZER(wakework,
- do_parent_wakeup),
- .done = done,
- };
-
- trace_cobalt_lostage_request("wakeup", current);
-
- pipeline_post_inband_work(&wakework);
-}
-
-static inline void init_kthread_info(struct xnthread *thread)
-{
- struct cobalt_threadinfo *p;
-
- p = pipeline_current();
- p->thread = thread;
- p->process = NULL;
-}
-
-/**
- * @fn int xnthread_map(struct xnthread *thread, struct completion *done)
- * @internal
- * @brief Create a shadow thread context over a kernel task.
- *
- * This call maps a Cobalt core thread to the "current" Linux task
- * running in kernel space. The priority and scheduling class of the
- * underlying Linux task are not affected; it is assumed that the
- * caller did set them appropriately before issuing the shadow mapping
- * request.
- *
- * This call immediately moves the calling kernel thread to the
- * Xenomai domain.
- *
- * @param thread The descriptor address of the new shadow thread to be
- * mapped to "current". This descriptor must have been previously
- * initialized by a call to xnthread_init().
- *
- * @param done A completion object to be signaled when @a thread is
- * fully mapped over the current Linux context, waiting for
- * xnthread_start().
- *
- * @return 0 is returned on success. Otherwise:
- *
- * - -ERESTARTSYS is returned if the current Linux task has received a
- * signal, thus preventing the final migration to the Xenomai domain
- * (i.e. in order to process the signal in the Linux domain). This
- * error should not be considered as fatal.
- *
- * - -EPERM is returned if the shadow thread has been killed before
- * the current task had a chance to return to the caller. In such a
- * case, the real-time mapping operation has failed globally, and no
- * Xenomai resource remains attached to it.
- *
- * - -EINVAL is returned if the thread control block bears the XNUSER
- * bit.
- *
- * - -EBUSY is returned if either the current Linux task or the
- * associated shadow thread is already involved in a shadow mapping.
- *
- * @coretags{secondary-only, might-switch}
- */
-int xnthread_map(struct xnthread *thread, struct completion *done)
-{
- int ret;
- spl_t s;
-
- if (xnthread_test_state(thread, XNUSER))
- return -EINVAL;
-
- if (xnthread_current() || xnthread_test_state(thread, XNMAPPED))
- return -EBUSY;
-
- thread->u_window = NULL;
- xnthread_pin_initial(thread);
-
- pipeline_init_shadow_tcb(thread);
- xnthread_suspend(thread, XNRELAX, XN_INFINITE, XN_RELATIVE, NULL);
- init_kthread_info(thread);
- xnthread_set_state(thread, XNMAPPED);
- xndebug_shadow_init(thread);
- xnthread_run_handler(thread, map_thread);
- pipeline_enable_kevents();
-
- /*
- * CAUTION: Soon after xnthread_init() has returned,
- * xnthread_start() is commonly invoked from the root domain,
- * therefore the call site may expect the started kernel
- * shadow to preempt immediately. As a result of such
- * assumption, start attributes (struct xnthread_start_attr)
- * are often laid on the caller's stack.
- *
- * For this reason, we raise the completion signal to wake up
- * the xnthread_init() caller only once the emerging thread is
- * hardened, and __never__ before that point. Since we run
- * over the Xenomai domain upon return from xnthread_harden(),
- * we schedule a virtual interrupt handler in the root domain
- * to signal the completion object.
- */
- xnthread_resume(thread, XNDORMANT);
- ret = xnthread_harden();
- wakeup_parent(done);
-
- xnlock_get_irqsave(&nklock, s);
-
- enlist_new_thread(thread);
- /*
- * Make sure xnthread_start() did not slip in from another CPU
- * while we were back from wakeup_parent().
- */
- if (thread->entry == NULL)
- xnthread_suspend(thread, XNDORMANT,
- XN_INFINITE, XN_RELATIVE, NULL);
-
- xnlock_put_irqrestore(&nklock, s);
-
- xnthread_test_cancel();
-
- xntrace_pid(xnthread_host_pid(thread),
- xnthread_current_priority(thread));
-
- return ret;
-}
-EXPORT_SYMBOL_GPL(xnthread_map);
-
/* nklock locked, irqs off */
void xnthread_call_mayday(struct xnthread *thread, int reason)
{
--
2.26.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 04/17] cobalt/thread: Pull kthread inband work onto creator stack
2021-06-11 18:05 [PATCH 00/17] Dovetail integration - the finals Jan Kiszka
` (2 preceding siblings ...)
2021-06-11 18:05 ` [PATCH 03/17] cobalt/thread: Privatize xnthread_map to map_kthread Jan Kiszka
@ 2021-06-11 18:05 ` Jan Kiszka
2021-06-11 18:05 ` [PATCH 05/17] cobalt/thread: Make sure relax inband work is on a stack outliving the wakeup Jan Kiszka
` (13 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Jan Kiszka @ 2021-06-11 18:05 UTC (permalink / raw)
To: xenomai
From: Jan Kiszka <jan.kiszka@siemens.com>
As dovetail does not copy the work passed to pipeline_post_inband_work,
we must ensure that the one used in xnthread_relax lives as long as the
wakeup takes. Therefore, embed the pipeline_inband_work struct needed
for signaling the kthread creator into kthread_arg struct. This lives on
the creator stack, as long as it takes to signal completion to it.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
kernel/cobalt/thread.c | 38 +++++++++++++-------------------------
1 file changed, 13 insertions(+), 25 deletions(-)
diff --git a/kernel/cobalt/thread.c b/kernel/cobalt/thread.c
index db68833c6..27f51bd58 100644
--- a/kernel/cobalt/thread.c
+++ b/kernel/cobalt/thread.c
@@ -84,30 +84,18 @@ static inline void enlist_new_thread(struct xnthread *thread)
xnvfile_touch_tag(&nkthreadlist_tag);
}
-struct parent_wakeup_request {
+struct kthread_arg {
struct pipeline_inband_work inband_work; /* Must be first. */
+ struct xnthread *thread;
struct completion *done;
};
static void do_parent_wakeup(struct pipeline_inband_work *inband_work)
{
- struct parent_wakeup_request *rq;
+ struct kthread_arg *ka;
- rq = container_of(inband_work, struct parent_wakeup_request, inband_work);
- complete(rq->done);
-}
-
-static inline void wakeup_parent(struct completion *done)
-{
- struct parent_wakeup_request wakework = {
- .inband_work = PIPELINE_INBAND_WORK_INITIALIZER(wakework,
- do_parent_wakeup),
- .done = done,
- };
-
- trace_cobalt_lostage_request("wakeup", current);
-
- pipeline_post_inband_work(&wakework);
+ ka = container_of(inband_work, struct kthread_arg, inband_work);
+ complete(ka->done);
}
static inline void init_kthread_info(struct xnthread *thread)
@@ -119,7 +107,7 @@ static inline void init_kthread_info(struct xnthread *thread)
p->process = NULL;
}
-static int map_kthread(struct xnthread *thread, struct completion *done)
+static int map_kthread(struct xnthread *thread, struct kthread_arg *ka)
{
int ret;
spl_t s;
@@ -158,7 +146,12 @@ static int map_kthread(struct xnthread *thread, struct completion *done)
*/
xnthread_resume(thread, XNDORMANT);
ret = xnthread_harden();
- wakeup_parent(done);
+
+ trace_cobalt_lostage_request("wakeup", current);
+
+ ka->inband_work = (struct pipeline_inband_work)
+ PIPELINE_INBAND_WORK_INITIALIZER(*ka, do_parent_wakeup);
+ pipeline_post_inband_work(ka);
xnlock_get_irqsave(&nklock, s);
@@ -181,11 +174,6 @@ static int map_kthread(struct xnthread *thread, struct completion *done)
return ret;
}
-struct kthread_arg {
- struct xnthread *thread;
- struct completion *done;
-};
-
static int kthread_trampoline(void *arg)
{
struct kthread_arg *ka = arg;
@@ -210,7 +198,7 @@ static int kthread_trampoline(void *arg)
param.sched_priority = prio;
sched_setscheduler(current, policy, ¶m);
- ret = map_kthread(thread, ka->done);
+ ret = map_kthread(thread, ka);
if (ret) {
printk(XENO_WARNING "failed to create kernel shadow %s\n",
thread->name);
--
2.26.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 05/17] cobalt/thread: Make sure relax inband work is on a stack outliving the wakeup
2021-06-11 18:05 [PATCH 00/17] Dovetail integration - the finals Jan Kiszka
` (3 preceding siblings ...)
2021-06-11 18:05 ` [PATCH 04/17] cobalt/thread: Pull kthread inband work onto creator stack Jan Kiszka
@ 2021-06-11 18:05 ` Jan Kiszka
2021-06-11 18:05 ` [PATCH 06/17] cobalt/thread: dovetail: keep hard irqs off on transition to in-band Jan Kiszka
` (12 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Jan Kiszka @ 2021-06-11 18:05 UTC (permalink / raw)
To: xenomai
From: Jan Kiszka <jan.kiszka@siemens.com>
As dovetail does not copy the work passed to pipeline_post_inband_work,
we must ensure that the one used in xnthread_relax lives as long as the
wakeup takes. Therefore, fold post_wakeup into the only caller so that
the stack frame of xnthread_relax is guaranteed to be used which
fulfills this requirement.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
kernel/cobalt/thread.c | 23 ++++++++---------------
1 file changed, 8 insertions(+), 15 deletions(-)
diff --git a/kernel/cobalt/thread.c b/kernel/cobalt/thread.c
index 27f51bd58..bc2eebc92 100644
--- a/kernel/cobalt/thread.c
+++ b/kernel/cobalt/thread.c
@@ -1993,19 +1993,6 @@ static void lostage_task_wakeup(struct pipeline_inband_work *inband_work)
wake_up_process(p);
}
-static void post_wakeup(struct task_struct *p)
-{
- struct lostage_wakeup wakework = {
- .inband_work = PIPELINE_INBAND_WORK_INITIALIZER(wakework,
- lostage_task_wakeup),
- .task = p,
- };
-
- trace_cobalt_lostage_request("wakeup", wakework.task);
-
- pipeline_post_inband_work(&wakework);
-}
-
void __xnthread_propagate_schedparam(struct xnthread *curr)
{
int kpolicy = SCHED_FIFO, kprio = curr->bprio, ret;
@@ -2065,9 +2052,14 @@ void __xnthread_propagate_schedparam(struct xnthread *curr)
*/
void xnthread_relax(int notify, int reason)
{
+ struct task_struct *p = current;
+ struct lostage_wakeup wakework = {
+ .inband_work = PIPELINE_INBAND_WORK_INITIALIZER(wakework,
+ lostage_task_wakeup),
+ .task = p,
+ };
struct xnthread *thread = xnthread_current();
int cpu __maybe_unused, suspension;
- struct task_struct *p = current;
kernel_siginfo_t si;
primary_mode_only();
@@ -2090,7 +2082,8 @@ void xnthread_relax(int notify, int reason)
* xnthread_suspend() has an interrupts-on section built in.
*/
splmax();
- post_wakeup(p);
+ trace_cobalt_lostage_request("wakeup", p);
+ pipeline_post_inband_work(&wakework);
/*
* Grab the nklock to synchronize the Linux task state
* manipulation with handle_sigwake_event. This lock will be
--
2.26.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 06/17] cobalt/thread: dovetail: keep hard irqs off on transition to in-band
2021-06-11 18:05 [PATCH 00/17] Dovetail integration - the finals Jan Kiszka
` (4 preceding siblings ...)
2021-06-11 18:05 ` [PATCH 05/17] cobalt/thread: Make sure relax inband work is on a stack outliving the wakeup Jan Kiszka
@ 2021-06-11 18:05 ` Jan Kiszka
2022-01-24 16:15 ` Jan Kiszka
2021-06-11 18:05 ` [PATCH 07/17] cobalt/sched: dovetail: prevent early scheduling on uninit runqueues Jan Kiszka
` (11 subsequent siblings)
17 siblings, 1 reply; 25+ messages in thread
From: Jan Kiszka @ 2021-06-11 18:05 UTC (permalink / raw)
To: xenomai
From: Philippe Gerum <rpm@xenomai.org>
Dovetail provides a fast service to escalate the caller to out-of-band
mode for executing a routine (run_oob_call()), which we use to enforce
primary mode in ___xnsched_run() to schedule out the relaxing thread.
Due to the way run_oob_call() works, enabling hardirqs during this
transition can trigger a subtle bug caused by the relaxing thread to
be switched out, as a result of taking an interrupt during the irqs on
section, before __xnsched_run() actually runs on behalf of
xnthread_relax() -> xnthread_suspend().
This may lead to a breakage of the inband interrupt state, revealed by
lockdep complaining about a HARDIRQ-IN-W -> HARDIRQ-ON-W situation,
when finalize_task_switch() runs for reconciling both the in-band and
Xenomai scheduler states.
Re-enabling hard irqs before switching out the relaxing thread was
throught as a mean to reduce the scope of the interrupt-free section
while relaxing a thread with the I-pipe, which unlike Dovetail
requires us to open code an escalation service based on triggering a
synthetic interrupt.
We differentiate the behavior between the I-pipe and Dovetail by
abstracting the related bits as pipeline_leave_oob_unlock(), keeping
the current implementation unchanged for the I-pipe.
Signed-off-by: Philippe Gerum <rpm@xenomai.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
.../cobalt/kernel/dovetail/pipeline/sched.h | 12 +++++++
include/cobalt/kernel/ipipe/pipeline/sched.h | 20 ++++++++++++
kernel/cobalt/thread.c | 32 +++++++------------
3 files changed, 43 insertions(+), 21 deletions(-)
diff --git a/include/cobalt/kernel/dovetail/pipeline/sched.h b/include/cobalt/kernel/dovetail/pipeline/sched.h
index b5d6c1773..45512b983 100644
--- a/include/cobalt/kernel/dovetail/pipeline/sched.h
+++ b/include/cobalt/kernel/dovetail/pipeline/sched.h
@@ -7,6 +7,8 @@
#ifndef _COBALT_KERNEL_DOVETAIL_SCHED_H
#define _COBALT_KERNEL_DOVETAIL_SCHED_H
+#include <cobalt/kernel/lock.h>
+
struct xnthread;
struct xnsched;
struct task_struct;
@@ -35,6 +37,16 @@ int pipeline_leave_inband(void);
int pipeline_leave_oob_prepare(void);
+static inline void pipeline_leave_oob_unlock(void)
+{
+ /*
+ * We may not re-enable hard irqs due to the specifics of
+ * stage escalation via run_oob_call(), to prevent breaking
+ * the (virtual) interrupt state.
+ */
+ xnlock_put(&nklock);
+}
+
void pipeline_leave_oob_finish(void);
static inline
diff --git a/include/cobalt/kernel/ipipe/pipeline/sched.h b/include/cobalt/kernel/ipipe/pipeline/sched.h
index 3fd5c4bea..9d7bf886b 100644
--- a/include/cobalt/kernel/ipipe/pipeline/sched.h
+++ b/include/cobalt/kernel/ipipe/pipeline/sched.h
@@ -7,6 +7,8 @@
#ifndef _COBALT_KERNEL_IPIPE_SCHED_H
#define _COBALT_KERNEL_IPIPE_SCHED_H
+#include <cobalt/kernel/lock.h>
+
struct xnthread;
struct xnsched;
struct task_struct;
@@ -27,6 +29,24 @@ int pipeline_leave_inband(void);
int pipeline_leave_oob_prepare(void);
+static inline void pipeline_leave_oob_unlock(void)
+{
+ /*
+ * Introduce an opportunity for interrupt delivery right
+ * before switching context, which shortens the
+ * uninterruptible code path.
+ *
+ * We have to shut irqs off before __xnsched_run() is called
+ * next though: if an interrupt could preempt us right after
+ * xnarch_escalate() is passed but before the nklock is
+ * grabbed, we would enter the critical section in
+ * ___xnsched_run() from the root domain, which would defeat
+ * the purpose of escalating the request.
+ */
+ xnlock_clear_irqon(&nklock);
+ splmax();
+}
+
void pipeline_leave_oob_finish(void);
void pipeline_finalize_thread(struct xnthread *thread);
diff --git a/kernel/cobalt/thread.c b/kernel/cobalt/thread.c
index bc2eebc92..d096a20d3 100644
--- a/kernel/cobalt/thread.c
+++ b/kernel/cobalt/thread.c
@@ -955,27 +955,17 @@ void xnthread_suspend(struct xnthread *thread, int mask,
if (wchan)
thread->wchan = wchan;
- /*
- * If the current thread is being relaxed, we must have been
- * called from xnthread_relax(), in which case we introduce an
- * opportunity for interrupt delivery right before switching
- * context, which shortens the uninterruptible code path.
- *
- * We have to shut irqs off before calling __xnsched_run()
- * though: if an interrupt could preempt us right after
- * xnarch_escalate() is passed but before the nklock is
- * grabbed, we would enter the critical section in
- * ___xnsched_run() from the root domain, which would defeat
- * the purpose of escalating the request.
- *
- * NOTE: using __xnsched_run() for rescheduling allows us to
- * break the scheduler lock temporarily.
- */
if (likely(thread == sched->curr)) {
xnsched_set_resched(sched);
+ /*
+ * Transition to secondary mode (XNRELAX) is a
+ * separate path which is only available to
+ * xnthread_relax(). Using __xnsched_run() there for
+ * rescheduling allows us to break the scheduler lock
+ * temporarily.
+ */
if (unlikely(mask & XNRELAX)) {
- xnlock_clear_irqon(&nklock);
- splmax();
+ pipeline_leave_oob_unlock();
__xnsched_run(sched);
return;
}
@@ -1683,7 +1673,7 @@ int xnthread_join(struct xnthread *thread, bool uninterruptible)
xnthread_set_state(thread, XNJOINED);
tpid = xnthread_host_pid(thread);
-
+
if (curr && !xnthread_test_state(curr, XNRELAX)) {
xnlock_put_irqrestore(&nklock, s);
xnthread_relax(0, 0);
@@ -1889,7 +1879,7 @@ int __xnthread_set_schedparam(struct xnthread *thread,
/* Ask the target thread to call back if relaxed. */
if (xnthread_test_state(thread, XNRELAX))
__xnthread_signal(thread, SIGSHADOW, SIGSHADOW_ACTION_HOME);
-
+
return ret;
}
@@ -2081,7 +2071,6 @@ void xnthread_relax(int notify, int reason)
* We disable interrupts during the migration sequence, but
* xnthread_suspend() has an interrupts-on section built in.
*/
- splmax();
trace_cobalt_lostage_request("wakeup", p);
pipeline_post_inband_work(&wakework);
/*
@@ -2089,6 +2078,7 @@ void xnthread_relax(int notify, int reason)
* manipulation with handle_sigwake_event. This lock will be
* dropped by xnthread_suspend().
*/
+ splmax();
xnlock_get(&nklock);
xnthread_run_handler_stack(thread, relax_thread);
suspension = pipeline_leave_oob_prepare();
--
2.26.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 07/17] cobalt/sched: dovetail: prevent early scheduling on uninit runqueues
2021-06-11 18:05 [PATCH 00/17] Dovetail integration - the finals Jan Kiszka
` (5 preceding siblings ...)
2021-06-11 18:05 ` [PATCH 06/17] cobalt/thread: dovetail: keep hard irqs off on transition to in-band Jan Kiszka
@ 2021-06-11 18:05 ` Jan Kiszka
2021-06-11 18:05 ` [PATCH 08/17] cobalt/arm: dovetail: add architecture bits Jan Kiszka
` (10 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Jan Kiszka @ 2021-06-11 18:05 UTC (permalink / raw)
To: xenomai
From: Philippe Gerum <rpm@xenomai.org>
Make sure not to raise XNSCHED when setting up the root thread, so
that we can't start rescheduling from irq_exit_pipeline() before all
CPUs have their runqueue fully built.
At this chance, add the missing tracepoint reporting a remote
scheduling request.
Signed-off-by: Philippe Gerum <rpm@xenomai.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
kernel/cobalt/dovetail/sched.c | 1 +
kernel/cobalt/sched.c | 10 +++++++++-
2 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/kernel/cobalt/dovetail/sched.c b/kernel/cobalt/dovetail/sched.c
index 074bc006e..01ea4422d 100644
--- a/kernel/cobalt/dovetail/sched.c
+++ b/kernel/cobalt/dovetail/sched.c
@@ -91,6 +91,7 @@ void pipeline_clear_mayday(void) /* May solely affect current. */
irqreturn_t pipeline_reschedule_ipi_handler(int irq, void *dev_id)
{
+ trace_cobalt_schedule_remote(xnsched_current());
/* Will reschedule from irq_exit_pipeline(). */
diff --git a/kernel/cobalt/sched.c b/kernel/cobalt/sched.c
index 7873fd652..222d0dd40 100644
--- a/kernel/cobalt/sched.c
+++ b/kernel/cobalt/sched.c
@@ -465,7 +465,15 @@ int xnsched_set_policy(struct xnthread *thread,
if (xnthread_test_state(thread, XNREADY))
xnsched_enqueue(thread);
- if (!xnthread_test_state(thread, XNDORMANT))
+ /*
+ * Make sure not to raise XNSCHED when setting up the root
+ * thread, so that we can't start rescheduling on interrupt
+ * exit before all CPUs have their runqueue fully
+ * built. Filtering on XNROOT here is correct because the root
+ * thread enters the idle class once as part of the runqueue
+ * setup process and never leaves it afterwards.
+ */
+ if (!xnthread_test_state(thread, XNDORMANT|XNROOT))
xnsched_set_resched(thread->sched);
return 0;
--
2.26.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 08/17] cobalt/arm: dovetail: add architecture bits
2021-06-11 18:05 [PATCH 00/17] Dovetail integration - the finals Jan Kiszka
` (6 preceding siblings ...)
2021-06-11 18:05 ` [PATCH 07/17] cobalt/sched: dovetail: prevent early scheduling on uninit runqueues Jan Kiszka
@ 2021-06-11 18:05 ` Jan Kiszka
2021-06-11 18:05 ` [PATCH 09/17] lib/cobalt: Reorder low_init for having cobalt_use_legacy_tsc earlier available Jan Kiszka
` (9 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Jan Kiszka @ 2021-06-11 18:05 UTC (permalink / raw)
To: xenomai
From: Philippe Gerum <rpm@xenomai.org>
The license notice for arch/arm/dovetail/machine.c is fixed in order
to match the remaining code, which is [1] for the prefaulting handler,
and myself for the rest.
[1] from Xenomai v2.5-rc1:
commit 8421cc62502db56b90d2912ea5f6049536177204
Author: Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org>
Date: Sat Nov 15 23:45:24 2008 +0000
Fault heaps memory on ARM
Signed-off-by: Philippe Gerum <rpm@xenomai.org>
[Jan: style fix]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
kernel/cobalt/arch/arm/Kconfig | 4 +
kernel/cobalt/arch/arm/dovetail/Makefile | 5 +
.../include/asm/xenomai/calibration.h | 39 ++++++++
.../dovetail/include/asm/xenomai/features.h | 30 ++++++
.../arm/dovetail/include/asm/xenomai/fptest.h | 52 ++++++++++
.../dovetail/include/asm/xenomai/machine.h | 72 ++++++++++++++
.../dovetail/include/asm/xenomai/syscall.h | 94 +++++++++++++++++++
.../dovetail/include/asm/xenomai/syscall32.h | 24 +++++
.../arm/dovetail/include/asm/xenomai/thread.h | 32 +++++++
.../dovetail/include/asm/xenomai/wrappers.h | 27 ++++++
kernel/cobalt/arch/arm/dovetail/machine.c | 44 +++++++++
11 files changed, 423 insertions(+)
create mode 100644 kernel/cobalt/arch/arm/dovetail/Makefile
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/calibration.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/features.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/fptest.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/machine.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall32.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/thread.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/wrappers.h
create mode 100644 kernel/cobalt/arch/arm/dovetail/machine.c
diff --git a/kernel/cobalt/arch/arm/Kconfig b/kernel/cobalt/arch/arm/Kconfig
index d81ff4c59..b0cbdc3b7 100644
--- a/kernel/cobalt/arch/arm/Kconfig
+++ b/kernel/cobalt/arch/arm/Kconfig
@@ -10,3 +10,7 @@ config XENO_ARCH_SYS3264
config XENO_ARCH_OUTOFLINE_XNLOCK
bool
default y
+
+config XENO_ARCH_IPIPE_COMPAT
+ def_bool DOVETAIL
+ select IPIPE_COMPAT
diff --git a/kernel/cobalt/arch/arm/dovetail/Makefile b/kernel/cobalt/arch/arm/dovetail/Makefile
new file mode 100644
index 000000000..13cbf84a9
--- /dev/null
+++ b/kernel/cobalt/arch/arm/dovetail/Makefile
@@ -0,0 +1,5 @@
+
+obj-$(CONFIG_XENOMAI) += xenomai.o
+xenomai-y := machine.o
+
+ccflags-y := -I$(srctree)/arch/arm/xenomai/include -I$(srctree)/include/xenomai
diff --git a/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/calibration.h b/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/calibration.h
new file mode 100644
index 000000000..c4724372d
--- /dev/null
+++ b/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/calibration.h
@@ -0,0 +1,39 @@
+/*
+ * Copyright (C) 2001-2021 Philippe Gerum <rpm@xenomai.org>.
+ *
+ * ARM port
+ * Copyright (C) 2005 Stelian Pop
+ *
+ * Xenomai is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * Xenomai is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with Xenomai; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
+ * 02111-1307, USA.
+ */
+#ifndef _COBALT_ARM_DOVETAIL_CALIBRATION_H
+#define _COBALT_ARM_DOVETAIL_CALIBRATION_H
+
+static inline void xnarch_get_latencies(struct xnclock_gravity *p)
+{
+ unsigned int sched_latency;
+
+#if CONFIG_XENO_OPT_TIMING_SCHEDLAT != 0
+ sched_latency = CONFIG_XENO_OPT_TIMING_SCHEDLAT;
+#else
+ sched_latency = 5000;
+#endif
+ p->user = sched_latency;
+ p->kernel = CONFIG_XENO_OPT_TIMING_KSCHEDLAT;
+ p->irq = CONFIG_XENO_OPT_TIMING_IRQLAT;
+}
+
+#endif /* !_COBALT_ARM_DOVETAIL_CALIBRATION_H */
diff --git a/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/features.h b/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/features.h
new file mode 100644
index 000000000..9c0af20e7
--- /dev/null
+++ b/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/features.h
@@ -0,0 +1,30 @@
+/*
+ * Copyright (C) 2005 Philippe Gerum <rpm@xenomai.org>.
+ *
+ * ARM port
+ * Copyright (C) 2005 Stelian Pop
+ *
+ * Xenomai is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published
+ * by the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * Xenomai is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with Xenomai; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
+ * 02111-1307, USA.
+ */
+#ifndef _COBALT_ARM_DOVETAIL_FEATURES_H
+#define _COBALT_ARM_DOVETAIL_FEATURES_H
+
+struct cobalt_featinfo;
+static inline void collect_arch_features(struct cobalt_featinfo *p) { }
+
+#include <asm/xenomai/uapi/features.h>
+
+#endif /* !_COBALT_ARM_DOVETAIL_FEATURES_H */
diff --git a/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/fptest.h b/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/fptest.h
new file mode 100644
index 000000000..ad7814cce
--- /dev/null
+++ b/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/fptest.h
@@ -0,0 +1,52 @@
+/*
+ * Copyright (C) 2006 Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org>.
+ *
+ * Xenomai is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published
+ * by the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * Xenomai is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with Xenomai; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
+ * 02111-1307, USA.
+ */
+#ifndef _COBALT_ARM_DOVETAIL_FPTEST_H
+#define _COBALT_ARM_DOVETAIL_FPTEST_H
+
+#include <linux/errno.h>
+#include <asm/hwcap.h>
+
+#ifdef CONFIG_VFP
+#define have_vfp (elf_hwcap & HWCAP_VFP)
+#else /* !CONFIG_VFP */
+#define have_vfp 0
+#endif /* !CONFIG_VFP */
+
+#include <asm/xenomai/uapi/fptest.h>
+
+static inline int fp_kernel_supported(void)
+{
+ return 0;
+}
+
+static inline int fp_linux_begin(void)
+{
+ return -ENOSYS;
+}
+
+static inline void fp_linux_end(void)
+{
+}
+
+static inline int fp_detect(void)
+{
+ return have_vfp ? __COBALT_HAVE_VFP : 0;
+}
+
+#endif /* _COBALT_ARM_DOVETAIL_FPTEST_H */
diff --git a/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/machine.h b/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/machine.h
new file mode 100644
index 000000000..a694a7891
--- /dev/null
+++ b/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/machine.h
@@ -0,0 +1,72 @@
+/**
+ * Copyright © 2002-2004 Philippe Gerum.
+ *
+ * ARM port
+ * Copyright (C) 2005 Stelian Pop
+ *
+ * Xenomai is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation, Inc., 675 Mass Ave,
+ * Cambridge MA 02139, USA; either version 2 of the License, or (at
+ * your option) any later version.
+ *
+ * Xenomai is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with Xenomai; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
+ * 02111-1307, USA.
+ */
+#ifndef _COBALT_ARM_DOVETAIL_MACHINE_H
+#define _COBALT_ARM_DOVETAIL_MACHINE_H
+
+#include <linux/version.h>
+#include <asm/byteorder.h>
+#include <asm/cacheflush.h>
+
+#define xnarch_cache_aliasing() cache_is_vivt()
+
+#if __LINUX_ARM_ARCH__ < 5
+static inline __attribute_const__ unsigned long ffnz(unsigned long x)
+{
+ int r = 0;
+
+ if (!x)
+ return 0;
+ if (!(x & 0xffff)) {
+ x >>= 16;
+ r += 16;
+ }
+ if (!(x & 0xff)) {
+ x >>= 8;
+ r += 8;
+ }
+ if (!(x & 0xf)) {
+ x >>= 4;
+ r += 4;
+ }
+ if (!(x & 3)) {
+ x >>= 2;
+ r += 2;
+ }
+ if (!(x & 1)) {
+ x >>= 1;
+ r += 1;
+ }
+ return r;
+}
+#else
+static inline __attribute_const__ unsigned long ffnz(unsigned long ul)
+{
+ int __r;
+ __asm__("clz\t%0, %1" : "=r" (__r) : "r"(ul & (-ul)) : "cc");
+ return 31 - __r;
+}
+#endif
+
+#include <asm-generic/xenomai/machine.h>
+
+#endif /* !_COBALT_ARM_DOVETAIL_MACHINE_H */
diff --git a/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall.h b/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall.h
new file mode 100644
index 000000000..eb4ec1bbe
--- /dev/null
+++ b/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall.h
@@ -0,0 +1,94 @@
+/*
+ * Copyright (C) 2001,2002,2003,2004 Philippe Gerum <rpm@xenomai.org>.
+ *
+ * ARM port
+ * Copyright (C) 2005 Stelian Pop
+ *
+ * Xenomai is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published
+ * by the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * Xenomai is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with Xenomai; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
+ * 02111-1307, USA.
+ */
+#ifndef _COBALT_ARM_DOVETAIL_SYSCALL_H
+#define _COBALT_ARM_DOVETAIL_SYSCALL_H
+
+#include <linux/errno.h>
+#include <linux/uaccess.h>
+#include <asm/unistd.h>
+#include <asm/ptrace.h>
+#include <asm-generic/xenomai/syscall.h>
+
+/*
+ * Cobalt syscall numbers can be fetched from ARM_ORIG_r0 with ARM_r7
+ * containing the Xenomai syscall marker, Linux syscalls directly from
+ * ARM_r7. Since we have to work with Dovetail whilst remaining binary
+ * compatible with applications built for the I-pipe, we retain the
+ * old syscall signature based on receiving XENO_ARM_SYSCALL in
+ * ARM_r7, possibly ORed with __COBALT_SYSCALL_BIT by Dovetail
+ * (IPIPE_COMPAT mode).
+ *
+ * FIXME: We also have __COBALT_SYSCALL_BIT (equal to
+ * __OOB_SYSCALL_BIT) present in the actual syscall number in r0,
+ * which is pretty much useless. Oh, well... When support for the
+ * I-pipe is dropped, we may switch back to the regular convention
+ * Dovetail abides by, with the actual syscall number into r7 ORed
+ * with __OOB_SYSCALL_BIT, freeing r0 for passing a call argument.
+ */
+#define __xn_reg_sys(__regs) ((__regs)->ARM_ORIG_r0)
+#define __xn_syscall_p(__regs) (((__regs)->ARM_r7 & ~__COBALT_SYSCALL_BIT) == XENO_ARM_SYSCALL)
+#define __xn_syscall(__regs) (__xn_reg_sys(__regs) & ~__COBALT_SYSCALL_BIT)
+
+/*
+ * Root syscall number with predicate (valid only if
+ * !__xn_syscall_p(__regs)).
+ */
+#define __xn_rootcall_p(__regs, __code) \
+ ({ \
+ *(__code) = (__regs)->ARM_r7; \
+ *(__code) < NR_syscalls || *(__code) >= __ARM_NR_BASE; \
+ })
+
+#define __xn_reg_rval(__regs) ((__regs)->ARM_r0)
+#define __xn_reg_arg1(__regs) ((__regs)->ARM_r1)
+#define __xn_reg_arg2(__regs) ((__regs)->ARM_r2)
+#define __xn_reg_arg3(__regs) ((__regs)->ARM_r3)
+#define __xn_reg_arg4(__regs) ((__regs)->ARM_r4)
+#define __xn_reg_arg5(__regs) ((__regs)->ARM_r5)
+#define __xn_reg_pc(__regs) ((__regs)->ARM_ip)
+#define __xn_reg_sp(__regs) ((__regs)->ARM_sp)
+
+static inline void __xn_error_return(struct pt_regs *regs, int v)
+{
+ __xn_reg_rval(regs) = v;
+}
+
+static inline void __xn_status_return(struct pt_regs *regs, long v)
+{
+ __xn_reg_rval(regs) = v;
+}
+
+static inline int __xn_interrupted_p(struct pt_regs *regs)
+{
+ return __xn_reg_rval(regs) == -EINTR;
+}
+
+static inline
+int xnarch_local_syscall(unsigned long a1, unsigned long a2,
+ unsigned long a3, unsigned long a4,
+ unsigned long a5)
+{
+ /* We need none of these with Dovetail. */
+ return -ENOSYS;
+}
+
+#endif /* !_COBALT_ARM_DOVETAIL_SYSCALL_H */
diff --git a/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall32.h b/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall32.h
new file mode 100644
index 000000000..95c5a1168
--- /dev/null
+++ b/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall32.h
@@ -0,0 +1,24 @@
+/*
+ * Copyright (C) 2014 Philippe Gerum <rpm@xenomai.org>.
+ *
+ * Xenomai is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published
+ * by the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * Xenomai is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with Xenomai; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
+ * 02111-1307, USA.
+ */
+#ifndef _COBALT_ARM_ASM_SYSCALL32_H
+#define _COBALT_ARM_ASM_SYSCALL32_H
+
+#include <asm-generic/xenomai/syscall32.h>
+
+#endif /* !_COBALT_ARM_ASM_SYSCALL32_H */
diff --git a/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/thread.h b/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/thread.h
new file mode 100644
index 000000000..792a3d26c
--- /dev/null
+++ b/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/thread.h
@@ -0,0 +1,32 @@
+/*
+ * Copyright (C) 2005 Stelian Pop
+ *
+ * Xenomai is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * Xenomai is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with Xenomai; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
+ * 02111-1307, USA.
+ */
+#ifndef _COBALT_ARM_DOVETAIL_THREAD_H
+#define _COBALT_ARM_DOVETAIL_THREAD_H
+
+#include <asm-generic/xenomai/dovetail/thread.h>
+#include <asm/traps.h>
+
+#define xnarch_fault_pc(__regs) ((__regs)->ARM_pc - (thumb_mode(__regs) ? 2 : 4))
+#define xnarch_fault_pf_p(__nr) ((__nr) == ARM_TRAP_ACCESS)
+#define xnarch_fault_bp_p(__nr) ((current->ptrace & PT_PTRACED) && \
+ ((__nr) == ARM_TRAP_BREAK || \
+ (__nr) == ARM_TRAP_UNDEFINSTR))
+#define xnarch_fault_notify(__nr) (!xnarch_fault_bp_p(__nr))
+
+#endif /* !_COBALT_ARM_DOVETAIL_THREAD_H */
diff --git a/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/wrappers.h b/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/wrappers.h
new file mode 100644
index 000000000..fe598966c
--- /dev/null
+++ b/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/wrappers.h
@@ -0,0 +1,27 @@
+/*
+ * Copyright (C) 2005 Philippe Gerum <rpm@xenomai.org>.
+ *
+ * Xenomai is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published
+ * by the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * Xenomai is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with Xenomai; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
+ * 02111-1307, USA.
+ */
+#ifndef _COBALT_ARM_ASM_WRAPPERS_H
+#define _COBALT_ARM_ASM_WRAPPERS_H
+
+#include <asm-generic/xenomai/wrappers.h> /* Read the generic portion. */
+
+#define __put_user_inatomic __put_user
+#define __get_user_inatomic __get_user
+
+#endif /* _COBALT_ARM_ASM_WRAPPERS_H */
diff --git a/kernel/cobalt/arch/arm/dovetail/machine.c b/kernel/cobalt/arch/arm/dovetail/machine.c
new file mode 100644
index 000000000..bc32f177c
--- /dev/null
+++ b/kernel/cobalt/arch/arm/dovetail/machine.c
@@ -0,0 +1,44 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0
+ *
+ * Copyright (C) 2008 Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org>
+ * Copyright (C) 2021 Philippe Gerum <rpm@xenomai.org>
+ */
+
+#include <linux/mm.h>
+#include <asm/xenomai/machine.h>
+
+static void mach_arm_prefault(struct vm_area_struct *vma)
+{
+ unsigned long addr;
+ unsigned int flags;
+
+ if ((vma->vm_flags & VM_MAYREAD)) {
+ flags = (vma->vm_flags & VM_MAYWRITE) ? FAULT_FLAG_WRITE : 0;
+ for (addr = vma->vm_start;
+ addr != vma->vm_end; addr += PAGE_SIZE)
+ handle_mm_fault(vma, addr, flags, NULL);
+ }
+}
+
+static const char *const fault_labels[] = {
+ [ARM_TRAP_ACCESS] = "Data or instruction access",
+ [ARM_TRAP_SECTION] = "Section fault",
+ [ARM_TRAP_DABT] = "Generic data abort",
+ [ARM_TRAP_PABT] = "Prefetch abort",
+ [ARM_TRAP_BREAK] = "Instruction breakpoint",
+ [ARM_TRAP_FPU] = "Floating point exception",
+ [ARM_TRAP_VFP] = "VFP Floating point exception",
+ [ARM_TRAP_UNDEFINSTR] = "Undefined instruction",
+ [ARM_TRAP_ALIGNMENT] = "Unaligned access exception",
+ [31] = NULL
+};
+
+struct cobalt_machine cobalt_machine = {
+ .name = "arm",
+ .init = NULL,
+ .late_init = NULL,
+ .cleanup = NULL,
+ .prefault = mach_arm_prefault,
+ .fault_labels = fault_labels,
+};
--
2.26.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 09/17] lib/cobalt: Reorder low_init for having cobalt_use_legacy_tsc earlier available
2021-06-11 18:05 [PATCH 00/17] Dovetail integration - the finals Jan Kiszka
` (7 preceding siblings ...)
2021-06-11 18:05 ` [PATCH 08/17] cobalt/arm: dovetail: add architecture bits Jan Kiszka
@ 2021-06-11 18:05 ` Jan Kiszka
2021-06-11 18:05 ` [PATCH 10/17] lib/cobalt/arm: dovetail: skip detection of KUSER_TSC support Jan Kiszka
` (8 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Jan Kiszka @ 2021-06-11 18:05 UTC (permalink / raw)
To: xenomai
From: Jan Kiszka <jan.kiszka@siemens.com>
This is needed as cobalt_arch_check_features() will make use of
__cobalt_tsc_clockfreq via cobalt_use_legacy_tsc().
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
lib/cobalt/init.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/cobalt/init.c b/lib/cobalt/init.c
index 20e28ccb0..dcd63cad8 100644
--- a/lib/cobalt/init.c
+++ b/lib/cobalt/init.c
@@ -177,11 +177,11 @@ static void low_init(void)
early_panic("mlockall: %s", strerror(errno));
trace_me("memory locked");
+ cobalt_ticks_init(f->clock_freq);
cobalt_features_init(f);
cobalt_init_umm(f->vdso_offset);
trace_me("memory heaps mapped");
cobalt_init_current_keys();
- cobalt_ticks_init(f->clock_freq);
}
static int cobalt_init_2(void);
--
2.26.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 10/17] lib/cobalt/arm: dovetail: skip detection of KUSER_TSC support
2021-06-11 18:05 [PATCH 00/17] Dovetail integration - the finals Jan Kiszka
` (8 preceding siblings ...)
2021-06-11 18:05 ` [PATCH 09/17] lib/cobalt: Reorder low_init for having cobalt_use_legacy_tsc earlier available Jan Kiszka
@ 2021-06-11 18:05 ` Jan Kiszka
2021-06-11 18:05 ` [PATCH 11/17] lib/cobalt/arm64: " Jan Kiszka
` (7 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Jan Kiszka @ 2021-06-11 18:05 UTC (permalink / raw)
To: xenomai
From: Philippe Gerum <rpm@xenomai.org>
When Dovetail is enabled in the kernel, we get timestamps for common
clocks (CLOCK_REALTIME, CLOCK_MONOTONIC) from the vDSO-based gettime()
routine as usual. Do not expect KUSER_TSC support to be there in such
a case, it is not.
Signed-off-by: Philippe Gerum <rpm@xenomai.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
lib/cobalt/arch/arm/features.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/lib/cobalt/arch/arm/features.c b/lib/cobalt/arch/arm/features.c
index d7b50f73f..b4eddaad4 100644
--- a/lib/cobalt/arch/arm/features.c
+++ b/lib/cobalt/arch/arm/features.c
@@ -25,6 +25,7 @@
#include <unistd.h>
#include <limits.h>
#include <cobalt/wrappers.h>
+#include <cobalt/ticks.h>
#include <asm/xenomai/syscall.h>
#include <asm/xenomai/tsc.h>
#include <asm/xenomai/features.h>
@@ -44,6 +45,9 @@ void cobalt_arch_check_features(struct cobalt_featinfo *finfo)
int err, fd;
void *addr;
+ if (!cobalt_use_legacy_tsc())
+ return;
+
if (__xn_tscinfo.kinfo.counter != NULL)
return;
--
2.26.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 11/17] lib/cobalt/arm64: dovetail: skip detection of KUSER_TSC support
2021-06-11 18:05 [PATCH 00/17] Dovetail integration - the finals Jan Kiszka
` (9 preceding siblings ...)
2021-06-11 18:05 ` [PATCH 10/17] lib/cobalt/arm: dovetail: skip detection of KUSER_TSC support Jan Kiszka
@ 2021-06-11 18:05 ` Jan Kiszka
2021-06-11 18:05 ` [PATCH 12/17] cobalt/tick: dovetail: improve accuracy of the host tick delay Jan Kiszka
` (6 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Jan Kiszka @ 2021-06-11 18:05 UTC (permalink / raw)
To: xenomai
From: Philippe Gerum <rpm@xenomai.org>
When Dovetail is enabled in the kernel, we get timestamps for common
clocks (CLOCK_REALTIME, CLOCK_MONOTONIC) from the vDSO-based gettime()
routine as usual. Do not expect KUSER_TSC support to be there in such
a case, it is not.
Signed-off-by: Philippe Gerum <rpm@xenomai.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
lib/cobalt/arch/arm64/features.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/lib/cobalt/arch/arm64/features.c b/lib/cobalt/arch/arm64/features.c
index 8b85a27ea..5baabfb69 100644
--- a/lib/cobalt/arch/arm64/features.c
+++ b/lib/cobalt/arch/arm64/features.c
@@ -25,6 +25,7 @@
#include <unistd.h>
#include <limits.h>
#include <cobalt/wrappers.h>
+#include <cobalt/ticks.h>
#include <asm/xenomai/syscall.h>
#include <asm/xenomai/tsc.h>
#include <asm/xenomai/features.h>
@@ -44,6 +45,9 @@ void cobalt_arch_check_features(struct cobalt_featinfo *finfo)
int err, fd;
void *addr;
+ if (!cobalt_use_legacy_tsc())
+ return;
+
if (__xn_tscinfo.kinfo.counter != NULL)
return;
--
2.26.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 12/17] cobalt/tick: dovetail: improve accuracy of the host tick delay
2021-06-11 18:05 [PATCH 00/17] Dovetail integration - the finals Jan Kiszka
` (10 preceding siblings ...)
2021-06-11 18:05 ` [PATCH 11/17] lib/cobalt/arm64: " Jan Kiszka
@ 2021-06-11 18:05 ` Jan Kiszka
2021-06-11 18:05 ` [PATCH 13/17] cobalt/tick: dovetail: Drop pointless inline specifier from pipeline_must_force_program_tick Jan Kiszka
` (5 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Jan Kiszka @ 2021-06-11 18:05 UTC (permalink / raw)
To: xenomai
From: Philippe Gerum <rpm@xenomai.org>
Expiration dates of in-band timers are based on the monotonic time
base read by ktime_get(), which is ok for us to use from non-NMI
context.
Signed-off-by: Philippe Gerum <rpm@xenomai.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
kernel/cobalt/dovetail/tick.c | 14 ++++++++------
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/kernel/cobalt/dovetail/tick.c b/kernel/cobalt/dovetail/tick.c
index fa041092e..ac343e4d3 100644
--- a/kernel/cobalt/dovetail/tick.c
+++ b/kernel/cobalt/dovetail/tick.c
@@ -55,19 +55,21 @@ void pipeline_set_timer_shot(unsigned long delay) /* ns */
}
static int proxy_set_next_ktime(ktime_t expires,
- struct clock_event_device *proxy_dev)
+ struct clock_event_device *proxy_dev) /* hard irqs on/off */
{
struct xnsched *sched;
- ktime_t delta;
unsigned long flags;
+ ktime_t delta;
int ret;
/*
- * When Negative delta have been observed, we set delta zero.
- * Or else exntimer_start() will return -ETIMEDOUT and do not
- * trigger shot
+ * Expiration dates of in-band timers are based on the common
+ * monotonic time base. If the timeout date has already
+ * elapsed, make sure xntimer_start() does not fail with
+ * -ETIMEDOUT but programs the hardware for ticking
+ * immediately instead.
*/
- delta = ktime_sub(expires, ktime_get_mono_fast_ns());
+ delta = ktime_sub(expires, ktime_get());
if (delta < 0)
delta = 0;
--
2.26.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 13/17] cobalt/tick: dovetail: Drop pointless inline specifier from pipeline_must_force_program_tick
2021-06-11 18:05 [PATCH 00/17] Dovetail integration - the finals Jan Kiszka
` (11 preceding siblings ...)
2021-06-11 18:05 ` [PATCH 12/17] cobalt/tick: dovetail: improve accuracy of the host tick delay Jan Kiszka
@ 2021-06-11 18:05 ` Jan Kiszka
2021-06-11 18:05 ` [PATCH 14/17] cobalt/clock: dovetail: abstract interface to hardware TSC Jan Kiszka
` (4 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Jan Kiszka @ 2021-06-11 18:05 UTC (permalink / raw)
To: xenomai
From: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
include/cobalt/kernel/dovetail/pipeline/tick.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/cobalt/kernel/dovetail/pipeline/tick.h b/include/cobalt/kernel/dovetail/pipeline/tick.h
index 8ac4760ec..372d8323f 100644
--- a/include/cobalt/kernel/dovetail/pipeline/tick.h
+++ b/include/cobalt/kernel/dovetail/pipeline/tick.h
@@ -11,6 +11,6 @@ void pipeline_uninstall_tick_proxy(void);
struct xnsched;
-inline bool pipeline_must_force_program_tick(struct xnsched *sched);
+bool pipeline_must_force_program_tick(struct xnsched *sched);
#endif /* !_COBALT_KERNEL_IPIPE_TICK_H */
--
2.26.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 14/17] cobalt/clock: dovetail: abstract interface to hardware TSC
2021-06-11 18:05 [PATCH 00/17] Dovetail integration - the finals Jan Kiszka
` (12 preceding siblings ...)
2021-06-11 18:05 ` [PATCH 13/17] cobalt/tick: dovetail: Drop pointless inline specifier from pipeline_must_force_program_tick Jan Kiszka
@ 2021-06-11 18:05 ` Jan Kiszka
2021-06-11 18:05 ` [PATCH 15/17] cobalt/arm64: dovetail: add architecture bits Jan Kiszka
` (3 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Jan Kiszka @ 2021-06-11 18:05 UTC (permalink / raw)
To: xenomai
From: Philippe Gerum <rpm@xenomai.org>
Unlike the I-pipe, Dovetail does not need to expose the underlying
timestamp counter to applications. The best equivalent to this counter
would be the common monotonic clock, expressing time as a count of
nanoseconds.
Fork the implementation accordingly between Dovetail and I-pipe
configurations, depending on which pipeline flavour was detected at
init.
Signed-off-by: Philippe Gerum <rpm@xenomai.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
include/cobalt/sys/cobalt.h | 2 ++
include/copperplate/clockobj.h | 12 ++++++++++--
lib/cobalt/arch/arm/include/asm/xenomai/tsc.h | 2 +-
.../arch/arm64/include/asm/xenomai/tsc.h | 2 +-
.../arch/powerpc/include/asm/xenomai/tsc.h | 2 +-
lib/cobalt/arch/x86/include/asm/xenomai/tsc.h | 2 +-
lib/cobalt/clock.c | 6 +++---
lib/cobalt/internal.c | 18 ++++++++++++++++--
lib/copperplate/clockobj.c | 12 +-----------
testsuite/smokey/tsc/tsc.c | 2 +-
10 files changed, 37 insertions(+), 23 deletions(-)
diff --git a/include/cobalt/sys/cobalt.h b/include/cobalt/sys/cobalt.h
index 1687e2be2..46096e880 100644
--- a/include/cobalt/sys/cobalt.h
+++ b/include/cobalt/sys/cobalt.h
@@ -134,6 +134,8 @@ void cobalt_register_tsd_hook(struct cobalt_tsd_hook *th);
void cobalt_assert_nrt(void);
+unsigned long long cobalt_read_tsc(void);
+
extern int __cobalt_control_bind;
#ifdef __cplusplus
diff --git a/include/copperplate/clockobj.h b/include/copperplate/clockobj.h
index 24c748557..58ec56652 100644
--- a/include/copperplate/clockobj.h
+++ b/include/copperplate/clockobj.h
@@ -70,8 +70,6 @@ void clockobj_get_distance(struct clockobj *clkobj,
const struct itimerspec *itm,
struct timespec *delta);
-ticks_t clockobj_get_tsc(void);
-
void clockobj_caltime_to_timeout(struct clockobj *clkobj, const struct tm *tm,
unsigned long rticks, struct timespec *ts);
@@ -112,6 +110,7 @@ void __clockobj_ticks_to_timespec(struct clockobj *clkobj,
#ifdef CONFIG_XENO_COBALT
#include <cobalt/ticks.h>
+#include <cobalt/sys/cobalt.h>
/*
* The Cobalt core exclusively deals with aperiodic timings, so a
@@ -121,6 +120,13 @@ void __clockobj_ticks_to_timespec(struct clockobj *clkobj,
* equivalent to Copperplate TSC units, and Copperplate ticks are
* periods of the reference clockobj which Cobalt does not know about.
*/
+
+static inline ticks_t clockobj_get_tsc(void)
+{
+ /* Guaranteed to be the source of CLOCK_COPPERPLATE. */
+ return cobalt_read_tsc();
+}
+
static inline sticks_t clockobj_ns_to_tsc(sticks_t ns)
{
return cobalt_ns_to_ticks(ns);
@@ -142,6 +148,8 @@ void clockobj_ns_to_timespec(ticks_t ns, struct timespec *ts)
#else /* CONFIG_XENO_MERCURY */
+ticks_t clockobj_get_tsc(void);
+
static inline sticks_t clockobj_ns_to_tsc(sticks_t ns)
{
return ns;
diff --git a/lib/cobalt/arch/arm/include/asm/xenomai/tsc.h b/lib/cobalt/arch/arm/include/asm/xenomai/tsc.h
index 594c4adac..2be4009d8 100644
--- a/lib/cobalt/arch/arm/include/asm/xenomai/tsc.h
+++ b/lib/cobalt/arch/arm/include/asm/xenomai/tsc.h
@@ -40,7 +40,7 @@ struct __xn_full_tscinfo {
extern struct __xn_full_tscinfo __xn_tscinfo;
static inline __attribute__((always_inline))
-unsigned long long cobalt_read_tsc(void)
+unsigned long long cobalt_read_legacy_tsc(void)
{
return __xn_tscinfo.kuser_tsc_get(__xn_tscinfo.kinfo.counter);
}
diff --git a/lib/cobalt/arch/arm64/include/asm/xenomai/tsc.h b/lib/cobalt/arch/arm64/include/asm/xenomai/tsc.h
index b145403c8..e664adb07 100644
--- a/lib/cobalt/arch/arm64/include/asm/xenomai/tsc.h
+++ b/lib/cobalt/arch/arm64/include/asm/xenomai/tsc.h
@@ -46,7 +46,7 @@ static inline uint64_t get_counter(void)
}
static inline __attribute__((always_inline))
-unsigned long long cobalt_read_tsc(void)
+unsigned long long cobalt_read_legacy_tsc(void)
{
return get_counter();
}
diff --git a/lib/cobalt/arch/powerpc/include/asm/xenomai/tsc.h b/lib/cobalt/arch/powerpc/include/asm/xenomai/tsc.h
index f6fc24af0..b4ff85218 100644
--- a/lib/cobalt/arch/powerpc/include/asm/xenomai/tsc.h
+++ b/lib/cobalt/arch/powerpc/include/asm/xenomai/tsc.h
@@ -18,7 +18,7 @@
#ifndef _LIB_COBALT_POWERPC_TSC_H
#define _LIB_COBALT_POWERPC_TSC_H
-static inline unsigned long long cobalt_read_tsc(void)
+static inline unsigned long long cobalt_read_legacy_tsc(void)
{
union {
unsigned long long t;
diff --git a/lib/cobalt/arch/x86/include/asm/xenomai/tsc.h b/lib/cobalt/arch/x86/include/asm/xenomai/tsc.h
index 6f3f5c45a..bf400e850 100644
--- a/lib/cobalt/arch/x86/include/asm/xenomai/tsc.h
+++ b/lib/cobalt/arch/x86/include/asm/xenomai/tsc.h
@@ -19,7 +19,7 @@
#ifndef _LIB_COBALT_X86_TSC_H
#define _LIB_COBALT_X86_TSC_H
-static inline unsigned long long cobalt_read_tsc(void)
+static inline unsigned long long cobalt_read_legacy_tsc(void)
{
#ifdef __i386__
unsigned long long t;
diff --git a/lib/cobalt/clock.c b/lib/cobalt/clock.c
index cacc19186..490426499 100644
--- a/lib/cobalt/clock.c
+++ b/lib/cobalt/clock.c
@@ -136,7 +136,7 @@ static int __do_clock_host_realtime(struct timespec *ts)
* mechanism in the kernel.
*/
unsynced_read_block(&tmp, &hostrt_data->lock) {
- now = cobalt_read_tsc();
+ now = cobalt_read_legacy_tsc();
base = hostrt_data->cycle_last;
mask = hostrt_data->mask;
mult = hostrt_data->mult;
@@ -175,12 +175,12 @@ static int gettime_via_tsc(clockid_t clock_id, struct timespec *tp)
break;
case CLOCK_MONOTONIC:
case CLOCK_MONOTONIC_RAW:
- ns = cobalt_ticks_to_ns(cobalt_read_tsc());
+ ns = cobalt_ticks_to_ns(cobalt_read_legacy_tsc());
tp->tv_sec = cobalt_divrem_billion(ns, &rem);
tp->tv_nsec = rem;
return 0;
case CLOCK_REALTIME:
- ns = cobalt_ticks_to_ns(cobalt_read_tsc());
+ ns = cobalt_ticks_to_ns(cobalt_read_legacy_tsc());
ns += cobalt_vdso->wallclock_offset;
tp->tv_sec = cobalt_divrem_billion(ns, &rem);
tp->tv_nsec = rem;
diff --git a/lib/cobalt/internal.c b/lib/cobalt/internal.c
index 303d86f99..42b60c35b 100644
--- a/lib/cobalt/internal.c
+++ b/lib/cobalt/internal.c
@@ -31,6 +31,8 @@
#include <stdarg.h>
#include <pthread.h>
#include <asm/xenomai/syscall.h>
+#include <asm/xenomai/tsc.h>
+#include <cobalt/ticks.h>
#include <cobalt/sys/cobalt.h>
#include "internal.h"
@@ -553,7 +555,7 @@ int cobalt_xlate_schedparam(int policy,
if (priority < 0)
priority = -priority;
-
+
memset(param, 0, sizeof(*param));
param->sched_priority = priority;
@@ -566,6 +568,18 @@ void cobalt_assert_nrt(void)
pthread_kill(pthread_self(), SIGDEBUG);
}
+unsigned long long cobalt_read_tsc(void)
+{
+ struct timespec ts;
+
+ if (cobalt_use_legacy_tsc())
+ return cobalt_read_legacy_tsc();
+
+ __cobalt_vdso_gettime(CLOCK_MONOTONIC, &ts);
+
+ return ts.tv_sec * 1000000000ULL + ts.tv_nsec;
+}
+
unsigned int cobalt_features;
void cobalt_features_init(struct cobalt_featinfo *f)
@@ -574,4 +588,4 @@ void cobalt_features_init(struct cobalt_featinfo *f)
/* Trigger arch specific feature initialization */
cobalt_arch_check_features(f);
-}
\ No newline at end of file
+}
diff --git a/lib/copperplate/clockobj.c b/lib/copperplate/clockobj.c
index 43928d753..b57bb46e7 100644
--- a/lib/copperplate/clockobj.c
+++ b/lib/copperplate/clockobj.c
@@ -229,22 +229,12 @@ int clockobj_set_resolution(struct clockobj *clkobj, unsigned int resolution_ns)
#ifdef CONFIG_XENO_COBALT
#include <cobalt/arith.h>
-#include <asm/xenomai/tsc.h>
+#include <cobalt/sys/cobalt.h>
#ifdef CONFIG_XENO_COPPERPLATE_CLOCK_RESTRICTED
#error "restricted CLOCK_COPPERPLATE not available"
#endif
-/*
- * NOTE: we can't inline this routine, as we don't want to expose
- * lib/cobalt/arch/.../include/asm/xenomai/tsc.h.
- */
-ticks_t clockobj_get_tsc(void)
-{
- /* Guaranteed to be the source of CLOCK_COPPERPLATE. */
- return cobalt_read_tsc();
-}
-
ticks_t clockobj_get_time(struct clockobj *clkobj)
{
ticks_t ns = cobalt_ticks_to_ns_rounded(cobalt_read_tsc());
diff --git a/testsuite/smokey/tsc/tsc.c b/testsuite/smokey/tsc/tsc.c
index 2865e338b..f9ae82fec 100644
--- a/testsuite/smokey/tsc/tsc.c
+++ b/testsuite/smokey/tsc/tsc.c
@@ -30,7 +30,7 @@
#include <smokey/smokey.h>
-#include <asm/xenomai/tsc.h>
+#include <cobalt/sys/cobalt.h>
#define DURATION 10000000
--
2.26.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 15/17] cobalt/arm64: dovetail: add architecture bits
2021-06-11 18:05 [PATCH 00/17] Dovetail integration - the finals Jan Kiszka
` (13 preceding siblings ...)
2021-06-11 18:05 ` [PATCH 14/17] cobalt/clock: dovetail: abstract interface to hardware TSC Jan Kiszka
@ 2021-06-11 18:05 ` Jan Kiszka
2021-06-11 18:05 ` [PATCH 16/17] cobalt/syscall: Account for different syscall argument marshaling Jan Kiszka
` (2 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Jan Kiszka @ 2021-06-11 18:05 UTC (permalink / raw)
To: xenomai
From: Philippe Gerum <rpm@xenomai.org>
At this chance, fix stale and missing copyright notices. Specifically,
Dmitriy Cherkasov <dmitriy@mperpetuo.com> is the original author of
the Xenomai/arm64 port, based on the ARM implementation; add credit
where it is due.
In the same move, introduce SPDX tags as replacement for GPL
boilerplate for the altered files.
Signed-off-by: Philippe Gerum <rpm@xenomai.org>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
kernel/cobalt/arch/arm64/dovetail/Makefile | 5 ++
.../include/asm/xenomai/calibration.h | 24 +++++++
.../dovetail/include/asm/xenomai/features.h | 15 ++++
.../dovetail/include/asm/xenomai/fptest.h | 35 ++++++++++
.../dovetail/include/asm/xenomai/machine.h | 33 +++++++++
.../dovetail/include/asm/xenomai/syscall.h | 68 +++++++++++++++++++
.../dovetail/include/asm/xenomai/syscall32.h | 12 ++++
.../dovetail/include/asm/xenomai/thread.h | 22 ++++++
.../dovetail/include/asm/xenomai/wrappers.h | 15 ++++
kernel/cobalt/arch/arm64/dovetail/machine.c | 41 +++++++++++
.../arm64/ipipe/include/asm/xenomai/machine.h | 3 +
kernel/cobalt/arch/arm64/ipipe/machine.c | 3 +
12 files changed, 276 insertions(+)
create mode 100644 kernel/cobalt/arch/arm64/dovetail/Makefile
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/calibration.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/features.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/fptest.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/machine.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall32.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/thread.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/wrappers.h
create mode 100644 kernel/cobalt/arch/arm64/dovetail/machine.c
diff --git a/kernel/cobalt/arch/arm64/dovetail/Makefile b/kernel/cobalt/arch/arm64/dovetail/Makefile
new file mode 100644
index 000000000..6c872fdb2
--- /dev/null
+++ b/kernel/cobalt/arch/arm64/dovetail/Makefile
@@ -0,0 +1,5 @@
+obj-$(CONFIG_XENOMAI) += xenomai.o
+
+xenomai-y := machine.o
+
+ccflags-y := -I$(srctree)/arch/arm64/xenomai/include -I$(srctree)/include/xenomai
diff --git a/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/calibration.h b/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/calibration.h
new file mode 100644
index 000000000..05cb6f3a5
--- /dev/null
+++ b/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/calibration.h
@@ -0,0 +1,24 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0
+ *
+ * Copyright (C) 2015 Philippe Gerum <rpm@xenomai.org>
+ */
+
+#ifndef _COBALT_ARM64_DOVETAIL_CALIBRATION_H
+#define _COBALT_ARM64_DOVETAIL_CALIBRATION_H
+
+static inline void xnarch_get_latencies(struct xnclock_gravity *p)
+{
+ unsigned int sched_latency;
+
+#if CONFIG_XENO_OPT_TIMING_SCHEDLAT != 0
+ sched_latency = CONFIG_XENO_OPT_TIMING_SCHEDLAT;
+#else
+ sched_latency = 5000;
+#endif
+ p->user = sched_latency;
+ p->kernel = CONFIG_XENO_OPT_TIMING_KSCHEDLAT;
+ p->irq = CONFIG_XENO_OPT_TIMING_IRQLAT;
+}
+
+#endif /* !_COBALT_ARM64_DOVETAIL_CALIBRATION_H */
diff --git a/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/features.h b/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/features.h
new file mode 100644
index 000000000..d5a438bab
--- /dev/null
+++ b/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/features.h
@@ -0,0 +1,15 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0
+ *
+ * Copyright (C) 2015 Philippe Gerum <rpm@xenomai.org>
+ */
+
+#ifndef _COBALT_ARM64_DOVETAIL_FEATURES_H
+#define _COBALT_ARM64_DOVETAIL_FEATURES_H
+
+struct cobalt_featinfo;
+static inline void collect_arch_features(struct cobalt_featinfo *p) { }
+
+#include <asm/xenomai/uapi/features.h>
+
+#endif /* !_COBALT_ARM64_DOVETAIL_FEATURES_H */
diff --git a/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/fptest.h b/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/fptest.h
new file mode 100644
index 000000000..5f3630dda
--- /dev/null
+++ b/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/fptest.h
@@ -0,0 +1,35 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0
+ *
+ * Copyright (C) 2006 Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org>
+ */
+
+#ifndef _COBALT_ARM64_DOVETAIL_FPTEST_H
+#define _COBALT_ARM64_DOVETAIL_FPTEST_H
+
+#include <linux/errno.h>
+#include <asm/xenomai/uapi/fptest.h>
+#include <asm/hwcap.h>
+
+#define have_fp (ELF_HWCAP & HWCAP_FP)
+
+static inline int fp_kernel_supported(void)
+{
+ return 0;
+}
+
+static inline int fp_linux_begin(void)
+{
+ return -ENOSYS;
+}
+
+static inline void fp_linux_end(void)
+{
+}
+
+static inline int fp_detect(void)
+{
+ return have_fp ? __COBALT_HAVE_FPU : 0;
+}
+
+#endif /* !_COBALT_ARM64_DOVETAIL_FPTEST_H */
diff --git a/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/machine.h b/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/machine.h
new file mode 100644
index 000000000..e71a5b7e3
--- /dev/null
+++ b/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/machine.h
@@ -0,0 +1,33 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0
+ *
+ * Copyright (C) 2015 Dmitriy Cherkasov <dmitriy@mperpetuo.com>
+ */
+
+#ifndef _COBALT_ARM64_DOVETAIL_MACHINE_H
+#define _COBALT_ARM64_DOVETAIL_MACHINE_H
+
+#include <linux/version.h>
+#include <asm/byteorder.h>
+#include <cobalt/kernel/assert.h>
+
+/* D-side always behaves as PIPT on AArch64 (see arch/arm64/include/asm/cachetype.h) */
+#define xnarch_cache_aliasing() 0
+
+static inline __attribute_const__ unsigned long ffnz(unsigned long ul)
+{
+ int __r;
+
+ /* zero input is not valid */
+ XENO_WARN_ON(COBALT, ul == 0);
+
+ __asm__ ("rbit\t%0, %1\n"
+ "clz\t%0, %0\n"
+ : "=r" (__r) : "r"(ul) : "cc");
+
+ return __r;
+}
+
+#include <asm-generic/xenomai/machine.h>
+
+#endif /* !_COBALT_ARM64_DOVETAIL_MACHINE_H */
diff --git a/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall.h b/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall.h
new file mode 100644
index 000000000..33b4a710f
--- /dev/null
+++ b/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall.h
@@ -0,0 +1,68 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0
+ *
+ * Copyright (C) 2015 Dmitriy Cherkasov <dmitriy@mperpetuo.com>
+ * Copyright (C) 2021 Philippe Gerum <rpm@xenomai.org>
+ */
+
+#ifndef _COBALT_ARM64_DOVETAIL_SYSCALL_H
+#define _COBALT_ARM64_DOVETAIL_SYSCALL_H
+
+#include <linux/errno.h>
+#include <linux/uaccess.h>
+#include <asm/unistd.h>
+#include <asm/ptrace.h>
+#include <asm-generic/xenomai/syscall.h>
+
+/*
+ * Cobalt and Linux syscall numbers can be fetched from syscallno,
+ * masking out the __COBALT_SYSCALL_BIT marker.
+ */
+#define __xn_reg_sys(__regs) ((unsigned long)(__regs)->syscallno)
+#define __xn_syscall_p(regs) ((__xn_reg_sys(regs) & __COBALT_SYSCALL_BIT) != 0)
+#define __xn_syscall(__regs) ((unsigned long)(__xn_reg_sys(__regs) & ~__COBALT_SYSCALL_BIT))
+
+#define __xn_reg_rval(__regs) ((__regs)->regs[0])
+#define __xn_reg_arg1(__regs) ((__regs)->regs[0])
+#define __xn_reg_arg2(__regs) ((__regs)->regs[1])
+#define __xn_reg_arg3(__regs) ((__regs)->regs[2])
+#define __xn_reg_arg4(__regs) ((__regs)->regs[3])
+#define __xn_reg_arg5(__regs) ((__regs)->regs[4])
+#define __xn_reg_pc(__regs) ((__regs)->pc)
+#define __xn_reg_sp(__regs) ((__regs)->sp)
+
+/*
+ * Root syscall number with predicate (valid only if
+ * !__xn_syscall_p(__regs)).
+ */
+#define __xn_rootcall_p(__regs, __code) \
+ ({ \
+ *(__code) = __xn_syscall(__regs); \
+ *(__code) < NR_syscalls; \
+ })
+
+static inline void __xn_error_return(struct pt_regs *regs, int v)
+{
+ __xn_reg_rval(regs) = v;
+}
+
+static inline void __xn_status_return(struct pt_regs *regs, long v)
+{
+ __xn_reg_rval(regs) = v;
+}
+
+static inline int __xn_interrupted_p(struct pt_regs *regs)
+{
+ return __xn_reg_rval(regs) == -EINTR;
+}
+
+static inline
+int xnarch_local_syscall(unsigned long a1, unsigned long a2,
+ unsigned long a3, unsigned long a4,
+ unsigned long a5)
+{
+ /* We need none of these with Dovetail. */
+ return -ENOSYS;
+}
+
+#endif /* !_COBALT_ARM64_DOVETAIL_SYSCALL_H */
diff --git a/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall32.h b/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall32.h
new file mode 100644
index 000000000..cd0f39245
--- /dev/null
+++ b/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall32.h
@@ -0,0 +1,12 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0
+ *
+ * Copyright (C) 2014 Philippe Gerum <rpm@xenomai.org>
+ */
+
+#ifndef _COBALT_ARM64_DOVETAIL_SYSCALL32_H
+#define _COBALT_ARM64_DOVETAIL_SYSCALL32_H
+
+#include <asm-generic/xenomai/syscall32.h>
+
+#endif /* !_COBALT_ARM64_DOVETAIL_SYSCALL32_H */
diff --git a/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/thread.h b/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/thread.h
new file mode 100644
index 000000000..5b60ff34a
--- /dev/null
+++ b/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/thread.h
@@ -0,0 +1,22 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0
+ *
+ * Copyright (C) 2015 Dmitriy Cherkasov <dmitriy@mperpetuo.com>
+ * Copyright (C) 2021 Philippe Gerum <rpm@xenomai.org>
+ */
+
+#ifndef _COBALT_ARM64_DOVETAIL_THREAD_H
+#define _COBALT_ARM64_DOVETAIL_THREAD_H
+
+#include <asm-generic/xenomai/dovetail/thread.h>
+#include <asm/dovetail.h>
+
+#define xnarch_fault_pc(__regs) ((unsigned long)((__regs)->pc - 4)) /* XXX ? */
+
+#define xnarch_fault_pf_p(__nr) ((__nr) == ARM64_TRAP_ACCESS)
+#define xnarch_fault_bp_p(__nr) ((current->ptrace & PT_PTRACED) && \
+ ((__nr) == ARM64_TRAP_DEBUG || (__nr) == ARM64_TRAP_UNDI))
+
+#define xnarch_fault_notify(__nr) (!xnarch_fault_bp_p(__nr))
+
+#endif /* !_COBALT_ARM64_DOVETAIL_THREAD_H */
diff --git a/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/wrappers.h b/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/wrappers.h
new file mode 100644
index 000000000..7a1122fd3
--- /dev/null
+++ b/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/wrappers.h
@@ -0,0 +1,15 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0
+ *
+ * Copyright (C) 2005 Philippe Gerum <rpm@xenomai.org>
+ */
+
+#ifndef _COBALT_ARM64_ASM_WRAPPERS_H
+#define _COBALT_ARM64_ASM_WRAPPERS_H
+
+#include <asm-generic/xenomai/wrappers.h> /* Read the generic portion. */
+
+#define __put_user_inatomic __put_user
+#define __get_user_inatomic __get_user
+
+#endif /* _COBALT_ARM64_ASM_WRAPPERS_H */
diff --git a/kernel/cobalt/arch/arm64/dovetail/machine.c b/kernel/cobalt/arch/arm64/dovetail/machine.c
new file mode 100644
index 000000000..e03d7b9fd
--- /dev/null
+++ b/kernel/cobalt/arch/arm64/dovetail/machine.c
@@ -0,0 +1,41 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2008 Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org>
+// Copyright (C) 2021 Philippe Gerum <rpm@xenomai.org>
+
+#include <linux/mm.h>
+#include <asm/xenomai/machine.h>
+
+static void mach_arm64_prefault(struct vm_area_struct *vma)
+{
+ unsigned long addr;
+ unsigned int flags;
+
+ if ((vma->vm_flags & VM_MAYREAD)) {
+ flags = (vma->vm_flags & VM_MAYWRITE) ? FAULT_FLAG_WRITE : 0;
+ for (addr = vma->vm_start;
+ addr != vma->vm_end; addr += PAGE_SIZE)
+ handle_mm_fault(vma, addr, flags, NULL);
+ }
+}
+
+static const char *const fault_labels[] = {
+ [ARM64_TRAP_ACCESS] = "Data or instruction abort",
+ [ARM64_TRAP_ALIGN] = "SP/PC alignment abort",
+ [ARM64_TRAP_SEA] = "Synchronous external abort",
+ [ARM64_TRAP_DEBUG] = "Debug trap",
+ [ARM64_TRAP_UNDI] = "Undefined instruction",
+ [ARM64_TRAP_UNDSE] = "Undefined synchronous exception",
+ [ARM64_TRAP_FPE] = "FPSIMD exception",
+ [ARM64_TRAP_SVE] = "SVE access trap",
+ [ARM64_TRAP_BTI] = "Branch target identification trap",
+ [31] = NULL
+};
+
+struct cobalt_machine cobalt_machine = {
+ .name = "arm64",
+ .init = NULL,
+ .late_init = NULL,
+ .cleanup = NULL,
+ .prefault = mach_arm64_prefault,
+ .fault_labels = fault_labels,
+};
diff --git a/kernel/cobalt/arch/arm64/ipipe/include/asm/xenomai/machine.h b/kernel/cobalt/arch/arm64/ipipe/include/asm/xenomai/machine.h
index 17be10339..c91c8f58e 100644
--- a/kernel/cobalt/arch/arm64/ipipe/include/asm/xenomai/machine.h
+++ b/kernel/cobalt/arch/arm64/ipipe/include/asm/xenomai/machine.h
@@ -4,6 +4,9 @@
* ARM port
* Copyright (C) 2005 Stelian Pop
*
+ * ARM64 port
+ * Copyright (C) 2015 Dmitriy Cherkasov <dmitriy@mperpetuo.com>
+ *
* Xenomai is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License as
* published by the Free Software Foundation, Inc., 675 Mass Ave,
diff --git a/kernel/cobalt/arch/arm64/ipipe/machine.c b/kernel/cobalt/arch/arm64/ipipe/machine.c
index a215e4caa..521b734da 100644
--- a/kernel/cobalt/arch/arm64/ipipe/machine.c
+++ b/kernel/cobalt/arch/arm64/ipipe/machine.c
@@ -1,6 +1,9 @@
/**
* Copyright (C) 2005 Stelian Pop
*
+ * ARM64 port
+ * Copyright (C) 2015 Dmitriy Cherkasov <dmitriy@mperpetuo.com>
+ *
* Xenomai is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License as
* published by the Free Software Foundation, Inc., 675 Mass Ave,
--
2.26.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 16/17] cobalt/syscall: Account for different syscall argument marshaling
2021-06-11 18:05 [PATCH 00/17] Dovetail integration - the finals Jan Kiszka
` (14 preceding siblings ...)
2021-06-11 18:05 ` [PATCH 15/17] cobalt/arm64: dovetail: add architecture bits Jan Kiszka
@ 2021-06-11 18:05 ` Jan Kiszka
2021-06-11 18:05 ` [PATCH 17/17] ci: Add arm, arm64 and x86 dovetail targets for kernel 5.10 Jan Kiszka
2021-06-13 17:07 ` [PATCH 00/17] Dovetail integration - the finals Philippe Gerum
17 siblings, 0 replies; 25+ messages in thread
From: Jan Kiszka @ 2021-06-11 18:05 UTC (permalink / raw)
To: xenomai
From: Jan Kiszka <jan.kiszka@siemens.com>
I-pipe makes sure that arguments of compat calls are ordered just like
native calls. Dovetail does not. But it is better to use the kernel's
syscall_get_arguments for retrieving the arguments anyway. Introduce
pipeline_get_syscall_args to abstract that difference.
Xenomai/ARM syscalls have an unconventional argument marshalling, with
ARM_ORIG_r0 containing the Xenomai syscall number (ORed with
__COBALT_SYSCALL_BIT), followed by the first actual argument to the
syscall in ARM_r1.
For this reason, using syscall_get_arguments() to collect the syscall
parameters won't yield the expected result, with regs[0] matching
__xn_reg_sys(regs) instead of __xn_reg_arg1(regs).
Allow Dovetail-enabled architectures to override the generic
implementation of pipeline_get_syscall_args(), provide a working one
for ARM.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
[Philippe: add special handling of ARM]
Signed-off-by: Philippe Gerum <rpm@xenomai.org>
---
.../cobalt/kernel/dovetail/pipeline/pipeline.h | 11 +++++++++++
include/cobalt/kernel/ipipe/pipeline/pipeline.h | 12 ++++++++++++
.../arm/dovetail/include/asm/xenomai/syscall.h | 17 ++++++++++++-----
.../dovetail/include/asm/xenomai/syscall.h | 5 -----
.../x86/dovetail/include/asm/xenomai/syscall.h | 5 -----
.../include/asm-generic/xenomai/syscall.h | 7 -------
kernel/cobalt/posix/syscall.c | 14 ++++++++++----
7 files changed, 45 insertions(+), 26 deletions(-)
diff --git a/include/cobalt/kernel/dovetail/pipeline/pipeline.h b/include/cobalt/kernel/dovetail/pipeline/pipeline.h
index a985e367a..2ee7b3210 100644
--- a/include/cobalt/kernel/dovetail/pipeline/pipeline.h
+++ b/include/cobalt/kernel/dovetail/pipeline/pipeline.h
@@ -9,6 +9,8 @@
#include <linux/cpumask.h>
#include <cobalt/kernel/assert.h>
#include <asm/xenomai/features.h>
+#include <asm/xenomai/syscall.h>
+#include <asm/syscall.h>
#include <pipeline/machine.h>
typedef unsigned long spl_t;
@@ -95,4 +97,13 @@ static inline void pipeline_collect_features(struct cobalt_featinfo *f)
f->clock_freq = 0; /* N/A */
}
+#ifndef pipeline_get_syscall_args
+static inline void pipeline_get_syscall_args(struct task_struct *task,
+ struct pt_regs *regs,
+ unsigned long *args)
+{
+ syscall_get_arguments(task, regs, args);
+}
+#endif /* !pipeline_get_syscall_args */
+
#endif /* !_COBALT_KERNEL_DOVETAIL_PIPELINE_H */
diff --git a/include/cobalt/kernel/ipipe/pipeline/pipeline.h b/include/cobalt/kernel/ipipe/pipeline/pipeline.h
index fda962568..ac9c92b1b 100644
--- a/include/cobalt/kernel/ipipe/pipeline/pipeline.h
+++ b/include/cobalt/kernel/ipipe/pipeline/pipeline.h
@@ -11,6 +11,7 @@
#include <pipeline/machine.h>
#include <asm/xenomai/features.h>
+#include <asm/xenomai/syscall.h>
#define xnsched_primary_domain cobalt_pipeline.domain
@@ -81,4 +82,15 @@ static inline void pipeline_collect_features(struct cobalt_featinfo *f)
f->clock_freq = cobalt_pipeline.clock_freq;
}
+static inline void pipeline_get_syscall_args(struct task_struct *task,
+ struct pt_regs *regs,
+ unsigned long *args)
+{
+ *args++ = __xn_reg_arg1(regs);
+ *args++ = __xn_reg_arg2(regs);
+ *args++ = __xn_reg_arg3(regs);
+ *args++ = __xn_reg_arg4(regs);
+ *args = __xn_reg_arg5(regs);
+}
+
#endif /* !_COBALT_KERNEL_IPIPE_PIPELINE_H */
diff --git a/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall.h b/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall.h
index eb4ec1bbe..d41b25768 100644
--- a/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall.h
+++ b/kernel/cobalt/arch/arm/dovetail/include/asm/xenomai/syscall.h
@@ -59,11 +59,6 @@
})
#define __xn_reg_rval(__regs) ((__regs)->ARM_r0)
-#define __xn_reg_arg1(__regs) ((__regs)->ARM_r1)
-#define __xn_reg_arg2(__regs) ((__regs)->ARM_r2)
-#define __xn_reg_arg3(__regs) ((__regs)->ARM_r3)
-#define __xn_reg_arg4(__regs) ((__regs)->ARM_r4)
-#define __xn_reg_arg5(__regs) ((__regs)->ARM_r5)
#define __xn_reg_pc(__regs) ((__regs)->ARM_ip)
#define __xn_reg_sp(__regs) ((__regs)->ARM_sp)
@@ -91,4 +86,16 @@ int xnarch_local_syscall(unsigned long a1, unsigned long a2,
return -ENOSYS;
}
+#define pipeline_get_syscall_args pipeline_get_syscall_args
+static inline void pipeline_get_syscall_args(struct task_struct *task,
+ struct pt_regs *regs,
+ unsigned long *args)
+{
+ args[0] = regs->ARM_r1;
+ args[1] = regs->ARM_r2;
+ args[2] = regs->ARM_r3;
+ args[3] = regs->ARM_r4;
+ args[4] = regs->ARM_r5;
+}
+
#endif /* !_COBALT_ARM_DOVETAIL_SYSCALL_H */
diff --git a/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall.h b/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall.h
index 33b4a710f..96871e247 100644
--- a/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall.h
+++ b/kernel/cobalt/arch/arm64/dovetail/include/asm/xenomai/syscall.h
@@ -23,11 +23,6 @@
#define __xn_syscall(__regs) ((unsigned long)(__xn_reg_sys(__regs) & ~__COBALT_SYSCALL_BIT))
#define __xn_reg_rval(__regs) ((__regs)->regs[0])
-#define __xn_reg_arg1(__regs) ((__regs)->regs[0])
-#define __xn_reg_arg2(__regs) ((__regs)->regs[1])
-#define __xn_reg_arg3(__regs) ((__regs)->regs[2])
-#define __xn_reg_arg4(__regs) ((__regs)->regs[3])
-#define __xn_reg_arg5(__regs) ((__regs)->regs[4])
#define __xn_reg_pc(__regs) ((__regs)->pc)
#define __xn_reg_sp(__regs) ((__regs)->sp)
diff --git a/kernel/cobalt/arch/x86/dovetail/include/asm/xenomai/syscall.h b/kernel/cobalt/arch/x86/dovetail/include/asm/xenomai/syscall.h
index c64196b6a..b2e158247 100644
--- a/kernel/cobalt/arch/x86/dovetail/include/asm/xenomai/syscall.h
+++ b/kernel/cobalt/arch/x86/dovetail/include/asm/xenomai/syscall.h
@@ -29,11 +29,6 @@
*/
#define __xn_reg_sys(regs) ((regs)->orig_ax)
#define __xn_reg_rval(regs) ((regs)->ax)
-#define __xn_reg_arg1(regs) ((regs)->di)
-#define __xn_reg_arg2(regs) ((regs)->si)
-#define __xn_reg_arg3(regs) ((regs)->dx)
-#define __xn_reg_arg4(regs) ((regs)->r10)
-#define __xn_reg_arg5(regs) ((regs)->r8)
#define __xn_reg_pc(regs) ((regs)->ip)
#define __xn_reg_sp(regs) ((regs)->sp)
diff --git a/kernel/cobalt/include/asm-generic/xenomai/syscall.h b/kernel/cobalt/include/asm-generic/xenomai/syscall.h
index 48c44c49d..117bb3ff8 100644
--- a/kernel/cobalt/include/asm-generic/xenomai/syscall.h
+++ b/kernel/cobalt/include/asm-generic/xenomai/syscall.h
@@ -36,13 +36,6 @@
#define access_wok(addr, size) access_ok(VERIFY_WRITE, (addr), (size))
#endif
-#define __xn_reg_arglist(regs) \
- __xn_reg_arg1(regs), \
- __xn_reg_arg2(regs), \
- __xn_reg_arg3(regs), \
- __xn_reg_arg4(regs), \
- __xn_reg_arg5(regs)
-
#define __xn_copy_from_user(dstP, srcP, n) raw_copy_from_user(dstP, srcP, n)
#define __xn_copy_to_user(dstP, srcP, n) raw_copy_to_user(dstP, srcP, n)
#define __xn_put_user(src, dstP) __put_user(src, dstP)
diff --git a/kernel/cobalt/posix/syscall.c b/kernel/cobalt/posix/syscall.c
index 30c33dda6..b7177042e 100644
--- a/kernel/cobalt/posix/syscall.c
+++ b/kernel/cobalt/posix/syscall.c
@@ -486,6 +486,7 @@ int handle_head_syscall(bool caller_is_relaxed, struct pt_regs *regs)
struct xnthread *thread;
cobalt_syshand handler;
struct task_struct *p;
+ unsigned long args[6];
unsigned int nr, code;
long ret;
@@ -592,7 +593,10 @@ restart:
* handler (lostage ones), or rejected by allowed_syscall().
*/
- ret = handler(__xn_reg_arglist(regs));
+ p = current;
+ pipeline_get_syscall_args(p, regs, args);
+
+ ret = handler(args[0], args[1], args[2], args[3], args[4]);
if (ret == -ENOSYS && (sysflags & __xn_exec_adaptive)) {
if (switched) {
ret = xnthread_harden();
@@ -611,7 +615,6 @@ done:
__xn_status_return(regs, ret);
sigs = 0;
if (!xnsched_root_p()) {
- p = current;
if (signal_pending(p) ||
xnthread_test_info(thread, XNKICKED)) {
sigs = 1;
@@ -677,6 +680,7 @@ int handle_root_syscall(struct pt_regs *regs)
struct xnthread *thread;
cobalt_syshand handler;
struct task_struct *p;
+ unsigned long args[6];
unsigned int nr, code;
long ret;
@@ -735,7 +739,10 @@ restart:
xnthread_propagate_schedparam(thread);
}
- ret = handler(__xn_reg_arglist(regs));
+ p = current;
+ pipeline_get_syscall_args(p, regs, args);
+
+ ret = handler(args[0], args[1], args[2], args[3], args[4]);
if (ret == -ENOSYS && (sysflags & __xn_exec_adaptive)) {
sysflags ^= __xn_exec_histage;
if (switched) {
@@ -756,7 +763,6 @@ restart:
* just invoked, so make sure to fetch it.
*/
thread = xnthread_current();
- p = current;
if (signal_pending(p)) {
sigs = 1;
prepare_for_signal(p, thread, regs, sysflags);
--
2.26.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 17/17] ci: Add arm, arm64 and x86 dovetail targets for kernel 5.10
2021-06-11 18:05 [PATCH 00/17] Dovetail integration - the finals Jan Kiszka
` (15 preceding siblings ...)
2021-06-11 18:05 ` [PATCH 16/17] cobalt/syscall: Account for different syscall argument marshaling Jan Kiszka
@ 2021-06-11 18:05 ` Jan Kiszka
2021-06-13 17:07 ` [PATCH 00/17] Dovetail integration - the finals Philippe Gerum
17 siblings, 0 replies; 25+ messages in thread
From: Jan Kiszka @ 2021-06-11 18:05 UTC (permalink / raw)
To: xenomai
From: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
.gitlab-ci.yml | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index f6d2ee884..764486110 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -21,6 +21,7 @@ variables:
USE_CCACHE: "1"
CCACHE_MAXSIZE: "1G"
IPIPE_MIRROR_URL: "https://source.denx.de/Xenomai"
+ DOVETAIL_URL: "https://source.denx.de/Xenomai/linux-dovetail.git"
PACKAGES: "gcc-8 u-boot-tools git make bc bison libelf-dev autotools-dev \
autoconf autogen libtool pkg-config ccache flex libssl-dev"
PACKAGES_arm: "gcc-8-arm-linux-gnueabihf libc6-dev-armhf-cross"
@@ -72,6 +73,7 @@ variables:
- ./scripts/config -d DEBUG_INFO
# I-pipe and Xenomai settings
- ./scripts/config -e IPIPE
+ - ./scripts/config -e DOVETAIL
- ./scripts/config -e XENOMAI
- ./scripts/config -e XENO_OPT_SCHED_CLASSES
- ./scripts/config -e XENO_OPT_SCHED_WEAK
@@ -170,6 +172,14 @@ variables:
paths:
- .ccache
+arm-5.10-head:
+ extends: .build
+ variables:
+ ARCH: arm
+ PIPELINE_REV: v5.10.y-dovetail-rebase
+ PIPELINE_KERNEL: ${DOVETAIL_URL}
+ KERNEL_DEFCONFIG: multi_v7_defconfig
+
arm-5.4-release:
extends: .build
variables:
@@ -186,6 +196,14 @@ arm-4.19-release:
PIPELINE_KERNEL: ${IPIPE_MIRROR_URL}/ipipe-arm.git
KERNEL_DEFCONFIG: multi_v7_defconfig
+arm64-5.10-head:
+ extends: .build
+ variables:
+ ARCH: arm64
+ PIPELINE_REV: v5.10.y-dovetail-rebase
+ PIPELINE_KERNEL: ${DOVETAIL_URL}
+ KERNEL_DEFCONFIG: defconfig
+
arm64-5.4-release:
extends: .build
variables:
@@ -202,6 +220,14 @@ arm64-4.19-cip-release:
PIPELINE_KERNEL: ${IPIPE_MIRROR_URL}/ipipe-arm64.git
KERNEL_DEFCONFIG: defconfig
+x86-5.10-head:
+ extends: .build
+ variables:
+ ARCH: x86
+ PIPELINE_REV: v5.10.y-dovetail-rebase
+ PIPELINE_KERNEL: ${DOVETAIL_URL}
+ KERNEL_DEFCONFIG: x86_64_defconfig
+
x86-5.4-head:
extends: .build
variables:
--
2.26.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [PATCH 00/17] Dovetail integration - the finals
2021-06-11 18:05 [PATCH 00/17] Dovetail integration - the finals Jan Kiszka
` (16 preceding siblings ...)
2021-06-11 18:05 ` [PATCH 17/17] ci: Add arm, arm64 and x86 dovetail targets for kernel 5.10 Jan Kiszka
@ 2021-06-13 17:07 ` Philippe Gerum
2021-06-13 17:12 ` Jan Kiszka
17 siblings, 1 reply; 25+ messages in thread
From: Philippe Gerum @ 2021-06-13 17:07 UTC (permalink / raw)
To: Jan Kiszka; +Cc: xenomai, Hongzhan Chen
Jan Kiszka <jan.kiszka@siemens.com> writes:
> On top, I would also still like to understand - and ideally address
> prior to releasing a new major version - if we cannot align the ARM
> syscall ABI of Xenomai to the kernel /wrt argument passing. Now is a
> good chance to do that.
That would be fairly simple implementation-wise, problem though: this
would require to add this change to the next I-pipe/arm releases,
invalidating all previous I-pipe releases for 3.2 in the same move. In
theory, all existing I-pipe/arm patches starting from kernel 3.10 can
run the -next branch (although it may be better to stick with 4.14+ in
practice).
--
Philippe.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 00/17] Dovetail integration - the finals
2021-06-13 17:07 ` [PATCH 00/17] Dovetail integration - the finals Philippe Gerum
@ 2021-06-13 17:12 ` Jan Kiszka
2021-06-14 6:24 ` Philippe Gerum
0 siblings, 1 reply; 25+ messages in thread
From: Jan Kiszka @ 2021-06-13 17:12 UTC (permalink / raw)
To: Philippe Gerum; +Cc: xenomai, Hongzhan Chen
On 13.06.21 19:07, Philippe Gerum wrote:
>
> Jan Kiszka <jan.kiszka@siemens.com> writes:
>
>> On top, I would also still like to understand - and ideally address
>> prior to releasing a new major version - if we cannot align the ARM
>> syscall ABI of Xenomai to the kernel /wrt argument passing. Now is a
>> good chance to do that.
>
> That would be fairly simple implementation-wise, problem though: this
> would require to add this change to the next I-pipe/arm releases,
> invalidating all previous I-pipe releases for 3.2 in the same move. In
> theory, all existing I-pipe/arm patches starting from kernel 3.10 can
> run the -next branch (although it may be better to stick with 4.14+ in
> practice).
>
4.14 is long dead, 4.19 could be the lowest target. Yes, that might
involve some some complications. Maybe we could add a feature flag for
the new ABI, start with dovetail and then look into lifting 4.19 and 5.4
I-pipe.
Jan
--
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 00/17] Dovetail integration - the finals
2021-06-13 17:12 ` Jan Kiszka
@ 2021-06-14 6:24 ` Philippe Gerum
0 siblings, 0 replies; 25+ messages in thread
From: Philippe Gerum @ 2021-06-14 6:24 UTC (permalink / raw)
To: Jan Kiszka; +Cc: xenomai, Hongzhan Chen
Jan Kiszka <jan.kiszka@siemens.com> writes:
> On 13.06.21 19:07, Philippe Gerum wrote:
>>
>> Jan Kiszka <jan.kiszka@siemens.com> writes:
>>
>>> On top, I would also still like to understand - and ideally address
>>> prior to releasing a new major version - if we cannot align the ARM
>>> syscall ABI of Xenomai to the kernel /wrt argument passing. Now is a
>>> good chance to do that.
>>
>> That would be fairly simple implementation-wise, problem though: this
>> would require to add this change to the next I-pipe/arm releases,
>> invalidating all previous I-pipe releases for 3.2 in the same move. In
>> theory, all existing I-pipe/arm patches starting from kernel 3.10 can
>> run the -next branch (although it may be better to stick with 4.14+ in
>> practice).
>>
>
> 4.14 is long dead, 4.19 could be the lowest target. Yes, that might
> involve some some complications. Maybe we could add a feature flag for
> the new ABI, start with dovetail and then look into lifting 4.19 and 5.4
> I-pipe.
>
Dovetail already follows the ARM EABI, only using a marker to
distinguish in-band syscalls from foreign ones in r7, on all archs. This
said, I plan to work on another option for conveying oob syscalls based
on prctl() asap, this would be saner than invalidating a whole range of
syscall numbers for the regular kernel. In addition, this might help
enabling tools like valgrind at little/no cost. The earlier the better
to investigate this, I'd rather not have to maintain multiple ABIs in
Dovetail.
--
Philippe.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 06/17] cobalt/thread: dovetail: keep hard irqs off on transition to in-band
2021-06-11 18:05 ` [PATCH 06/17] cobalt/thread: dovetail: keep hard irqs off on transition to in-band Jan Kiszka
@ 2022-01-24 16:15 ` Jan Kiszka
2022-01-24 16:55 ` Philippe Gerum
0 siblings, 1 reply; 25+ messages in thread
From: Jan Kiszka @ 2022-01-24 16:15 UTC (permalink / raw)
To: Philippe Gerum; +Cc: xenomai
On 11.06.21 20:05, Jan Kiszka via Xenomai wrote:
> From: Philippe Gerum <rpm@xenomai.org>
>
> Dovetail provides a fast service to escalate the caller to out-of-band
> mode for executing a routine (run_oob_call()), which we use to enforce
> primary mode in ___xnsched_run() to schedule out the relaxing thread.
>
> Due to the way run_oob_call() works, enabling hardirqs during this
> transition can trigger a subtle bug caused by the relaxing thread to
> be switched out, as a result of taking an interrupt during the irqs on
> section, before __xnsched_run() actually runs on behalf of
> xnthread_relax() -> xnthread_suspend().
>
> This may lead to a breakage of the inband interrupt state, revealed by
> lockdep complaining about a HARDIRQ-IN-W -> HARDIRQ-ON-W situation,
> when finalize_task_switch() runs for reconciling both the in-band and
> Xenomai scheduler states.
>
> Re-enabling hard irqs before switching out the relaxing thread was
> throught as a mean to reduce the scope of the interrupt-free section
> while relaxing a thread with the I-pipe, which unlike Dovetail
> requires us to open code an escalation service based on triggering a
> synthetic interrupt.
>
> We differentiate the behavior between the I-pipe and Dovetail by
> abstracting the related bits as pipeline_leave_oob_unlock(), keeping
> the current implementation unchanged for the I-pipe.
>
> Signed-off-by: Philippe Gerum <rpm@xenomai.org>
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
> .../cobalt/kernel/dovetail/pipeline/sched.h | 12 +++++++
> include/cobalt/kernel/ipipe/pipeline/sched.h | 20 ++++++++++++
> kernel/cobalt/thread.c | 32 +++++++------------
> 3 files changed, 43 insertions(+), 21 deletions(-)
>
> diff --git a/include/cobalt/kernel/dovetail/pipeline/sched.h b/include/cobalt/kernel/dovetail/pipeline/sched.h
> index b5d6c1773..45512b983 100644
> --- a/include/cobalt/kernel/dovetail/pipeline/sched.h
> +++ b/include/cobalt/kernel/dovetail/pipeline/sched.h
...
> @@ -2081,7 +2071,6 @@ void xnthread_relax(int notify, int reason)
> * We disable interrupts during the migration sequence, but
> * xnthread_suspend() has an interrupts-on section built in.
> */
> - splmax();
> trace_cobalt_lostage_request("wakeup", p);
> pipeline_post_inband_work(&wakework);
> /*
> @@ -2089,6 +2078,7 @@ void xnthread_relax(int notify, int reason)
> * manipulation with handle_sigwake_event. This lock will be
> * dropped by xnthread_suspend().
> */
> + splmax();
This moves pipeline_post_inband_work() outside of the irqs-off section,
no? I think this destabilizes our migration to in-band, allowing the
injected wake-event to be consumed by Linux prior to the oob thread
suspension. I have a test here that seems to trigger this reliably.
Currently validating of moving splmax up again helps. Also
Can you recall why you changed it this way?
Jan
> xnlock_get(&nklock);
> xnthread_run_handler_stack(thread, relax_thread);
> suspension = pipeline_leave_oob_prepare();
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 06/17] cobalt/thread: dovetail: keep hard irqs off on transition to in-band
2022-01-24 16:15 ` Jan Kiszka
@ 2022-01-24 16:55 ` Philippe Gerum
2022-01-24 17:00 ` Philippe Gerum
0 siblings, 1 reply; 25+ messages in thread
From: Philippe Gerum @ 2022-01-24 16:55 UTC (permalink / raw)
To: Jan Kiszka; +Cc: xenomai
Jan Kiszka <jan.kiszka@siemens.com> writes:
> On 11.06.21 20:05, Jan Kiszka via Xenomai wrote:
>> From: Philippe Gerum <rpm@xenomai.org>
>> Dovetail provides a fast service to escalate the caller to
>> out-of-band
>> mode for executing a routine (run_oob_call()), which we use to enforce
>> primary mode in ___xnsched_run() to schedule out the relaxing thread.
>> Due to the way run_oob_call() works, enabling hardirqs during this
>> transition can trigger a subtle bug caused by the relaxing thread to
>> be switched out, as a result of taking an interrupt during the irqs on
>> section, before __xnsched_run() actually runs on behalf of
>> xnthread_relax() -> xnthread_suspend().
>> This may lead to a breakage of the inband interrupt state, revealed
>> by
>> lockdep complaining about a HARDIRQ-IN-W -> HARDIRQ-ON-W situation,
>> when finalize_task_switch() runs for reconciling both the in-band and
>> Xenomai scheduler states.
>> Re-enabling hard irqs before switching out the relaxing thread was
>> throught as a mean to reduce the scope of the interrupt-free section
>> while relaxing a thread with the I-pipe, which unlike Dovetail
>> requires us to open code an escalation service based on triggering a
>> synthetic interrupt.
>> We differentiate the behavior between the I-pipe and Dovetail by
>> abstracting the related bits as pipeline_leave_oob_unlock(), keeping
>> the current implementation unchanged for the I-pipe.
>> Signed-off-by: Philippe Gerum <rpm@xenomai.org>
>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
>> ---
>> .../cobalt/kernel/dovetail/pipeline/sched.h | 12 +++++++
>> include/cobalt/kernel/ipipe/pipeline/sched.h | 20 ++++++++++++
>> kernel/cobalt/thread.c | 32 +++++++------------
>> 3 files changed, 43 insertions(+), 21 deletions(-)
>> diff --git a/include/cobalt/kernel/dovetail/pipeline/sched.h
>> b/include/cobalt/kernel/dovetail/pipeline/sched.h
>> index b5d6c1773..45512b983 100644
>> --- a/include/cobalt/kernel/dovetail/pipeline/sched.h
>> +++ b/include/cobalt/kernel/dovetail/pipeline/sched.h
>
> ...
>
>> @@ -2081,7 +2071,6 @@ void xnthread_relax(int notify, int reason)
>> * We disable interrupts during the migration sequence, but
>> * xnthread_suspend() has an interrupts-on section built in.
>> */
>> - splmax();
>> trace_cobalt_lostage_request("wakeup", p);
>> pipeline_post_inband_work(&wakework);
>> /*
>> @@ -2089,6 +2078,7 @@ void xnthread_relax(int notify, int reason)
>> * manipulation with handle_sigwake_event. This lock will be
>> * dropped by xnthread_suspend().
>> */
>> + splmax();
>
> This moves pipeline_post_inband_work() outside of the irqs-off
> section, no? I think this destabilizes our migration to in-band,
> allowing the injected wake-event to be consumed by Linux prior to the
> oob thread suspension. I have a test here that seems to trigger this
> reliably. Currently validating of moving splmax up again helps. Also
>
> Can you recall why you changed it this way?
>
The rationale was that current had to be running oob at this point in
xnthread_relax(), so there would be no way for an in-band irq to be
handled, causing the per-cpu work to run on the same CPU.
Now, the tricky part is that this assumption might be wrong if an oob
irq wakes up a thread which preempts on the same CPU, then switches back
to secondary mode when it resumes. In this case, the irq work would run
earlier than expected. Worth testing indeed.
--
Philippe.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 06/17] cobalt/thread: dovetail: keep hard irqs off on transition to in-band
2022-01-24 16:55 ` Philippe Gerum
@ 2022-01-24 17:00 ` Philippe Gerum
2022-01-24 17:05 ` Jan Kiszka
0 siblings, 1 reply; 25+ messages in thread
From: Philippe Gerum @ 2022-01-24 17:00 UTC (permalink / raw)
To: Jan Kiszka; +Cc: xenomai
Philippe Gerum <rpm@xenomai.org> writes:
> Jan Kiszka <jan.kiszka@siemens.com> writes:
>
>> On 11.06.21 20:05, Jan Kiszka via Xenomai wrote:
>>> From: Philippe Gerum <rpm@xenomai.org>
>>> Dovetail provides a fast service to escalate the caller to
>>> out-of-band
>>> mode for executing a routine (run_oob_call()), which we use to enforce
>>> primary mode in ___xnsched_run() to schedule out the relaxing thread.
>>> Due to the way run_oob_call() works, enabling hardirqs during this
>>> transition can trigger a subtle bug caused by the relaxing thread to
>>> be switched out, as a result of taking an interrupt during the irqs on
>>> section, before __xnsched_run() actually runs on behalf of
>>> xnthread_relax() -> xnthread_suspend().
>>> This may lead to a breakage of the inband interrupt state, revealed
>>> by
>>> lockdep complaining about a HARDIRQ-IN-W -> HARDIRQ-ON-W situation,
>>> when finalize_task_switch() runs for reconciling both the in-band and
>>> Xenomai scheduler states.
>>> Re-enabling hard irqs before switching out the relaxing thread was
>>> throught as a mean to reduce the scope of the interrupt-free section
>>> while relaxing a thread with the I-pipe, which unlike Dovetail
>>> requires us to open code an escalation service based on triggering a
>>> synthetic interrupt.
>>> We differentiate the behavior between the I-pipe and Dovetail by
>>> abstracting the related bits as pipeline_leave_oob_unlock(), keeping
>>> the current implementation unchanged for the I-pipe.
>>> Signed-off-by: Philippe Gerum <rpm@xenomai.org>
>>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
>>> ---
>>> .../cobalt/kernel/dovetail/pipeline/sched.h | 12 +++++++
>>> include/cobalt/kernel/ipipe/pipeline/sched.h | 20 ++++++++++++
>>> kernel/cobalt/thread.c | 32 +++++++------------
>>> 3 files changed, 43 insertions(+), 21 deletions(-)
>>> diff --git a/include/cobalt/kernel/dovetail/pipeline/sched.h
>>> b/include/cobalt/kernel/dovetail/pipeline/sched.h
>>> index b5d6c1773..45512b983 100644
>>> --- a/include/cobalt/kernel/dovetail/pipeline/sched.h
>>> +++ b/include/cobalt/kernel/dovetail/pipeline/sched.h
>>
>> ...
>>
>>> @@ -2081,7 +2071,6 @@ void xnthread_relax(int notify, int reason)
>>> * We disable interrupts during the migration sequence, but
>>> * xnthread_suspend() has an interrupts-on section built in.
>>> */
>>> - splmax();
>>> trace_cobalt_lostage_request("wakeup", p);
>>> pipeline_post_inband_work(&wakework);
>>> /*
>>> @@ -2089,6 +2078,7 @@ void xnthread_relax(int notify, int reason)
>>> * manipulation with handle_sigwake_event. This lock will be
>>> * dropped by xnthread_suspend().
>>> */
>>> + splmax();
>>
>> This moves pipeline_post_inband_work() outside of the irqs-off
>> section, no? I think this destabilizes our migration to in-band,
>> allowing the injected wake-event to be consumed by Linux prior to the
>> oob thread suspension. I have a test here that seems to trigger this
>> reliably. Currently validating of moving splmax up again helps. Also
>>
>> Can you recall why you changed it this way?
>>
>
> The rationale was that current had to be running oob at this point in
> xnthread_relax(), so there would be no way for an in-band irq to be
> handled, causing the per-cpu work to run on the same CPU.
>
> Now, the tricky part is that this assumption might be wrong if an oob
> irq wakes up a thread which preempts on the same CPU, then switches back
> to secondary mode when it resumes. In this case, the irq work would run
> earlier than expected. Worth testing indeed.
As a matter of fact, EVL keeps the section atomic, Cobalt should align
on this:
void evl_switch_inband(int cause)
...
hard_local_irq_disable();
irq_work_queue(&curr->inband_work);
--
Philippe.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 06/17] cobalt/thread: dovetail: keep hard irqs off on transition to in-band
2022-01-24 17:00 ` Philippe Gerum
@ 2022-01-24 17:05 ` Jan Kiszka
0 siblings, 0 replies; 25+ messages in thread
From: Jan Kiszka @ 2022-01-24 17:05 UTC (permalink / raw)
To: Philippe Gerum; +Cc: xenomai
On 24.01.22 18:00, Philippe Gerum wrote:
>
> Philippe Gerum <rpm@xenomai.org> writes:
>
>> Jan Kiszka <jan.kiszka@siemens.com> writes:
>>
>>> On 11.06.21 20:05, Jan Kiszka via Xenomai wrote:
>>>> From: Philippe Gerum <rpm@xenomai.org>
>>>> Dovetail provides a fast service to escalate the caller to
>>>> out-of-band
>>>> mode for executing a routine (run_oob_call()), which we use to enforce
>>>> primary mode in ___xnsched_run() to schedule out the relaxing thread.
>>>> Due to the way run_oob_call() works, enabling hardirqs during this
>>>> transition can trigger a subtle bug caused by the relaxing thread to
>>>> be switched out, as a result of taking an interrupt during the irqs on
>>>> section, before __xnsched_run() actually runs on behalf of
>>>> xnthread_relax() -> xnthread_suspend().
>>>> This may lead to a breakage of the inband interrupt state, revealed
>>>> by
>>>> lockdep complaining about a HARDIRQ-IN-W -> HARDIRQ-ON-W situation,
>>>> when finalize_task_switch() runs for reconciling both the in-band and
>>>> Xenomai scheduler states.
>>>> Re-enabling hard irqs before switching out the relaxing thread was
>>>> throught as a mean to reduce the scope of the interrupt-free section
>>>> while relaxing a thread with the I-pipe, which unlike Dovetail
>>>> requires us to open code an escalation service based on triggering a
>>>> synthetic interrupt.
>>>> We differentiate the behavior between the I-pipe and Dovetail by
>>>> abstracting the related bits as pipeline_leave_oob_unlock(), keeping
>>>> the current implementation unchanged for the I-pipe.
>>>> Signed-off-by: Philippe Gerum <rpm@xenomai.org>
>>>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
>>>> ---
>>>> .../cobalt/kernel/dovetail/pipeline/sched.h | 12 +++++++
>>>> include/cobalt/kernel/ipipe/pipeline/sched.h | 20 ++++++++++++
>>>> kernel/cobalt/thread.c | 32 +++++++------------
>>>> 3 files changed, 43 insertions(+), 21 deletions(-)
>>>> diff --git a/include/cobalt/kernel/dovetail/pipeline/sched.h
>>>> b/include/cobalt/kernel/dovetail/pipeline/sched.h
>>>> index b5d6c1773..45512b983 100644
>>>> --- a/include/cobalt/kernel/dovetail/pipeline/sched.h
>>>> +++ b/include/cobalt/kernel/dovetail/pipeline/sched.h
>>>
>>> ...
>>>
>>>> @@ -2081,7 +2071,6 @@ void xnthread_relax(int notify, int reason)
>>>> * We disable interrupts during the migration sequence, but
>>>> * xnthread_suspend() has an interrupts-on section built in.
>>>> */
>>>> - splmax();
>>>> trace_cobalt_lostage_request("wakeup", p);
>>>> pipeline_post_inband_work(&wakework);
>>>> /*
>>>> @@ -2089,6 +2078,7 @@ void xnthread_relax(int notify, int reason)
>>>> * manipulation with handle_sigwake_event. This lock will be
>>>> * dropped by xnthread_suspend().
>>>> */
>>>> + splmax();
>>>
>>> This moves pipeline_post_inband_work() outside of the irqs-off
>>> section, no? I think this destabilizes our migration to in-band,
>>> allowing the injected wake-event to be consumed by Linux prior to the
>>> oob thread suspension. I have a test here that seems to trigger this
>>> reliably. Currently validating of moving splmax up again helps. Also
>>>
>>> Can you recall why you changed it this way?
>>>
>>
>> The rationale was that current had to be running oob at this point in
>> xnthread_relax(), so there would be no way for an in-band irq to be
>> handled, causing the per-cpu work to run on the same CPU.
>>
>> Now, the tricky part is that this assumption might be wrong if an oob
>> irq wakes up a thread which preempts on the same CPU, then switches back
>> to secondary mode when it resumes. In this case, the irq work would run
>> earlier than expected. Worth testing indeed.
Yeah, the test actually involves remote suspending. That may explain why
only we saw it so far with a rather special setup, luckily.
>
> As a matter of fact, EVL keeps the section atomic, Cobalt should align
> on this:
>
> void evl_switch_inband(int cause)
> ...
> hard_local_irq_disable();
> irq_work_queue(&curr->inband_work);
>
>
>
OK, will send a patch the reverts that change above.
Thanks,
Jan
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2022-01-24 17:05 UTC | newest]
Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-11 18:05 [PATCH 00/17] Dovetail integration - the finals Jan Kiszka
2021-06-11 18:05 ` [PATCH 01/17] cobalt/thread: Rework kthread check for xnthread_signal Jan Kiszka
2021-06-11 18:05 ` [PATCH 02/17] cobalt/thread: Move xnthread_signal work off the stack Jan Kiszka
2021-06-11 18:05 ` [PATCH 03/17] cobalt/thread: Privatize xnthread_map to map_kthread Jan Kiszka
2021-06-11 18:05 ` [PATCH 04/17] cobalt/thread: Pull kthread inband work onto creator stack Jan Kiszka
2021-06-11 18:05 ` [PATCH 05/17] cobalt/thread: Make sure relax inband work is on a stack outliving the wakeup Jan Kiszka
2021-06-11 18:05 ` [PATCH 06/17] cobalt/thread: dovetail: keep hard irqs off on transition to in-band Jan Kiszka
2022-01-24 16:15 ` Jan Kiszka
2022-01-24 16:55 ` Philippe Gerum
2022-01-24 17:00 ` Philippe Gerum
2022-01-24 17:05 ` Jan Kiszka
2021-06-11 18:05 ` [PATCH 07/17] cobalt/sched: dovetail: prevent early scheduling on uninit runqueues Jan Kiszka
2021-06-11 18:05 ` [PATCH 08/17] cobalt/arm: dovetail: add architecture bits Jan Kiszka
2021-06-11 18:05 ` [PATCH 09/17] lib/cobalt: Reorder low_init for having cobalt_use_legacy_tsc earlier available Jan Kiszka
2021-06-11 18:05 ` [PATCH 10/17] lib/cobalt/arm: dovetail: skip detection of KUSER_TSC support Jan Kiszka
2021-06-11 18:05 ` [PATCH 11/17] lib/cobalt/arm64: " Jan Kiszka
2021-06-11 18:05 ` [PATCH 12/17] cobalt/tick: dovetail: improve accuracy of the host tick delay Jan Kiszka
2021-06-11 18:05 ` [PATCH 13/17] cobalt/tick: dovetail: Drop pointless inline specifier from pipeline_must_force_program_tick Jan Kiszka
2021-06-11 18:05 ` [PATCH 14/17] cobalt/clock: dovetail: abstract interface to hardware TSC Jan Kiszka
2021-06-11 18:05 ` [PATCH 15/17] cobalt/arm64: dovetail: add architecture bits Jan Kiszka
2021-06-11 18:05 ` [PATCH 16/17] cobalt/syscall: Account for different syscall argument marshaling Jan Kiszka
2021-06-11 18:05 ` [PATCH 17/17] ci: Add arm, arm64 and x86 dovetail targets for kernel 5.10 Jan Kiszka
2021-06-13 17:07 ` [PATCH 00/17] Dovetail integration - the finals Philippe Gerum
2021-06-13 17:12 ` Jan Kiszka
2021-06-14 6:24 ` Philippe Gerum
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.