From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753385AbbJTUhI (ORCPT ); Tue, 20 Oct 2015 16:37:08 -0400 Received: from mail-am1on0075.outbound.protection.outlook.com ([157.56.112.75]:4960 "EHLO emea01-am1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753228AbbJTUg5 (ORCPT ); Tue, 20 Oct 2015 16:36:57 -0400 Authentication-Results: spf=fail (sender IP is 12.216.194.146) smtp.mailfrom=ezchip.com; ezchip.com; dkim=none (message not signed) header.d=none;ezchip.com; dmarc=none action=none header.from=ezchip.com; From: Chris Metcalf To: Gilad Ben Yossef , Steven Rostedt , Ingo Molnar , Peter Zijlstra , Andrew Morton , "Rik van Riel" , Tejun Heo , Frederic Weisbecker , Thomas Gleixner , "Paul E. McKenney" , Christoph Lameter , Viresh Kumar , Catalin Marinas , Will Deacon , Andy Lutomirski , , CC: Chris Metcalf Subject: [PATCH v8 07/14] task_isolation: add debug boot flag Date: Tue, 20 Oct 2015 16:36:05 -0400 Message-ID: <1445373372-6567-8-git-send-email-cmetcalf@ezchip.com> X-Mailer: git-send-email 2.1.2 In-Reply-To: <1445373372-6567-1-git-send-email-cmetcalf@ezchip.com> References: <1445373372-6567-1-git-send-email-cmetcalf@ezchip.com> X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1;DB3FFO11FD045;1:b9rjU2aHXxrpVKOFfBppj6TtC/WyykRno5yOSsszq8sqHpcNLsI7EMIrSxEVLVt3N7Wuam6DqlE3T4Mk+vhJzRPqVOcX21eWcXT7aRgZVGZ2IuGVeg668gK/vctNz6P+a8FkDeURHr7eQ8LjmUcp2EEm1v0iLMgXQQLqnpkFTC1GMDayx3NqQqL/uEsnv7sul5A8kYOjZCqnCAOI7Nbw9grG5tW5pyCiXPQ8DuQwKPNh5WBaNQhImCzxrDdlWf0ATtIz/lGT7GWUw4qGz+/5pSvSPwbiP1GsuywBWnFw5yu/cfxUpHKvRr6z0EyN+L2BKu5GSy6PwvUV9Fq1vTTkpp1tfhy1SIqqrFto4Nn7VPY= X-Forefront-Antispam-Report: CIP:12.216.194.146;CTRY:US;IPV:NLI;EFV:NLI;SFV:NSPM;SFS:(10009020)(6009001)(2980300002)(1109001)(1110001)(339900001)(189002)(199003)(42186005)(92566002)(64706001)(5001960100002)(229853001)(2201001)(106466001)(19580395003)(50986999)(106476002)(76176999)(87936001)(19580405001)(106356001)(105606002)(97736004)(81156007)(86362001)(6806005)(5001770100001)(575784001)(5007970100001)(85426001)(104016004)(47776003)(50226001)(36756003)(50466002)(5008740100001)(48376002)(107886002)(5003940100001)(46102003)(5001920100001)(2950100001)(33646002)(189998001)(9376005)(921003)(4001430100001)(83996005)(2101003)(1121003);DIR:OUT;SFP:1101;SCL:1;SRVR:AM2PR02MB0772;H:ld-1.internal.tilera.com;FPR:;SPF:Fail;PTR:InfoNoRecords;MX:1;A:1;LANG:en; MIME-Version: 1.0 Content-Type: text/plain X-Microsoft-Exchange-Diagnostics: 1;AM2PR02MB0772;2:/ryGfHF3wqNOpsGDCrGCb9TlKcrnxzvuePJ4dZTE0JM9nXA6LraIkwLBXRIfA2lIjK8c1gNHC/VZnrJfi9YI0hJAn3UCaNN/gJusu57iSmz1KsixBe1w8+W7JA+QpEZwv9w40qSqbkJVUlCjmHw6FkRfPuknMrzS66tb3b5krhc=;3:JZc6Pe09zu9yjAVs33JuXCsPlMe0FwkoByOHsEE8/7BfX7gLA+XzAw7SMu68AVoLG66o8BKnqHP1GxUAIOJZ2PgkZ6OqGr/L7+/ldOnlF80pf0tQYthvFaBqPuWVhJqPyzVXGwBCuP5thmtYKEDrVeb77hPhod+vNVy7/f2t5VrT4iJHGC0eqPELAfk+RUMzOo4zJ1TtJBr22CgBiYXPz2MQ9PJiqIIWbK4Ia/sQgyRtDE7Dh+Y1fAyOMRQ5c086;25:IpT1EukyVC7XlAyVdD9qaMhvWmQY6hFS25C9Eeho8kHG8SlgMvIUkXKORvnbdqT4AuoPmrwEN6vS/HpKblZnN+aYkP6/1KmYGj7w0CwXAviRc1OtuBvOQB7n+kxEbfmKPy9yD1VxfDJ1I4Vywvf5oSaZm26xXq4KIW25lmq6pzUicrPZ2m7ZmGz93QjSVJh6RdNE6zbUyzb1YcozjzlK/5Zyg0cqMseZygBY3J0vzGZ8yCvD4HX2QQm8Ucq5gpxc4mbw0Zcmy0+VozOb23SG9A==;20:ClQC8bF7zhrM75DDbLFpaAF7noKuCRHZROgrwx80YSSId1+OnsgZpeO/gMD8aqyyHiuvf9C0FEdtUThb+b2b3O6ukmc4zhlCnxuhBeoDI6SSqGnMZ4mGpPeT6I8TT84iCi58yo9ZSCzMn/AmKPPcRCMh0gFuc0mXxHQZx/JsmdU= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:AM2PR02MB0772; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(121898900299872); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(5005006)(520078)(8121501046)(3002001);SRVR:AM2PR02MB0772;BCL:0;PCL:0;RULEID:;SRVR:AM2PR02MB0772; X-Microsoft-Exchange-Diagnostics: 1;AM2PR02MB0772;4:xsc/dVnmzvpem4hK+T9Jh4vz4A5V04HCJj3UILi1bsfP+bzq9L8NnjelveKk2xkbyxUvdTZaEge1AObW3fA/boqtCZlO7ZOIpMB2t2uMw0w9dPTAwPaJYSXSoc1dwRKxfhDAuEWMddC2Opis5JBV/momGcnbdK6HF4qg7Eu5G/tHqbwiXtHGe7Ie38bPL8u+i2zXaUFvg4OfJi+/66r+HeEIj3RTtaT8yfvyXWqYRQKqG4//RhiQchCiFKdxtdrO+QxIl8edXHT4kbH9WbUNFHcDhfMGDZCKUIj0++GwH+xzOXsPvhaqaaP1LIwfNZvhLNIsyaQlc1F68U8Dhp5IFDHVKJiGIEQECLeX/PJiZdk= X-Forefront-PRVS: 073515755F X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;AM2PR02MB0772;23:dOsHacxWxQ1Du7y9tyaBpvGH4XS/qTleZ07766FpY?= =?us-ascii?Q?ZvqA2kjfrL0EoFovCIfeYoCR9iGp+BYVKdYIwYvklSwOvftnaTY06JItjtdt?= =?us-ascii?Q?ym057BCnJyUAluRDf5b9DsEFhF9KmN+i06FRdDbc4TdymUIZRpzp53J9ygIu?= =?us-ascii?Q?56Ro+pONNe1jgsGG8xYsK9AgXh5cMxHT9j6zRWV11+2TUK/SPugdpO2L4Ixj?= =?us-ascii?Q?9xNFEty9bFm6JH7lO4f/ENrMMFd7kKBBNnbLymw53x5Q+xMu5P2al7aVk2b+?= =?us-ascii?Q?7aQ0jPYrmLnGSumZoZZiGVgNj6Qd4JnAF4Bi74aZdB0UhhNOefr6RrQeDuYE?= =?us-ascii?Q?eU2tfycZ1z9dSlNE+GV9V4sOrXeaeZGoBk+TauGhEzPEMedrhwC+OzZilnIC?= =?us-ascii?Q?/+HoUGWpU5bsQ1WCQauVaApXIZJoNvnGcaVJFLcYdvSvEHoX1tR56cdu4F0t?= =?us-ascii?Q?oXrbjwZxZeolnT2YEwLvIulIMbusAwTOWiT1xQ8zpe6CnNIaQ2CdovaW7EGQ?= =?us-ascii?Q?OXsDmqGj77/Tt2p0QRRK7Q9r0M+ZS8JV8g+nGxMeoxyLG/F2kXUE2Q3lOW2n?= =?us-ascii?Q?0JycoENApNbDBkXpCpXL/JXwXOJBjax+Z9p8WCNTaKq1IUCBMGEWLaMdEE0U?= =?us-ascii?Q?mUYVUzWB/0A/JnLKF2WAm32jzMJfjfOz4jmQgoSP329yfG6CnrkLY/ad87PS?= =?us-ascii?Q?3x9q2LFMETLCMdRTv1Aipj0et1OA6QZV7UuxfBdAEsh6L1OVqGrXVvTcYaYi?= =?us-ascii?Q?8NXV+16+kZamLfOB/e4B60yCmmHlCcBxfkQDa/WNWOorF81tn/MRVOczov61?= =?us-ascii?Q?wI8UV80OPmcS4shPNejOZwP9KDrPWo6Y7+GpyXchCLZkk4lr88LDBelul8XT?= =?us-ascii?Q?LRXzIcc3Hf8kt5aGqA8kCEfZPol7MilS+Aq5322x6mR+XATKehJYKkrnlZZC?= =?us-ascii?Q?4QLk+Llv0nnxDUqQ+QT4JsoHDKoSzO9Shwp5r/nMGr/EoEP6IeFaWDYKGzQq?= =?us-ascii?Q?VHBy60IcJ9WgR77c6iHgWCG3tMPoxGKC3la5r0CVxaQrI2i/2qDbq30HkPAp?= =?us-ascii?Q?4Y81YVFeIVUZBWbJ0foS634Gvl5dax3KYsZEV5qvD3sIEDycgI4ZP5Jm6kxl?= =?us-ascii?Q?AxcHD64bAbymH2jwtLfQQUEJ02cimBLQ92sghEGOpClLkKniZmi3xRRWCkJ+?= =?us-ascii?Q?G5kbQVKueH5g0BWTCJ6IDsHEjomlZxC9CnAkv9kuHSzWSw01kb2Suju+zmuR?= =?us-ascii?Q?1KL9l5klfNBoDqFNhHp9f0ICTnsF4OO4PJNFHX0RyGgOjFmF5s3dhrNkt85l?= =?us-ascii?Q?8j57DrYRRG+mtTN1552hlY=3D?= X-Microsoft-Exchange-Diagnostics: 1;AM2PR02MB0772;5:M2XsvC6IQNjC3hCbF3f4RysDmcXWaaJCBwJqR4Jt1onTxNL0VH/HG6w+Kfs267GRK/yzEeHZayWjaSkwk+7TTRPXcCQtljEuW1nK0QYBTjRR6ZYbgavpUD93PigM1mjctt5OSwihqie+xqwtspU88g==;24:X/Ye97o5BoUEGjkFoI3fL4SjINc+MZJ/OeKDn82xZBPg8xS9WTmh7DOTTirKb1/txJ5VSh9LUVLnhL/Mg2KrrlNL1hmfhf6LXeI+VGMsCHo=;20:DVVFgM41AivnKH5u3RkQo0dIhHvkS2HZ1PVjhBVrTaP01ZQWxXJNyi308agnqQJhtuhp93rfNsrP/o+Xm5GhQA== SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: ezchip.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Oct 2015 20:36:52.6423 (UTC) X-MS-Exchange-CrossTenant-Id: 0fc16e0a-3cd3-4092-8b2f-0a42cff122c3 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=0fc16e0a-3cd3-4092-8b2f-0a42cff122c3;Ip=[12.216.194.146];Helo=[ld-1.internal.tilera.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM2PR02MB0772 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The new "task_isolation_debug" flag simplifies debugging of TASK_ISOLATION kernels when processes are running in PR_TASK_ISOLATION_ENABLE mode. Such processes should get no interrupts from the kernel, and if they do, when this boot flag is specified a kernel stack dump on the console is generated. It's possible to use ftrace to simply detect whether a task_isolation core has unexpectedly entered the kernel. But what this boot flag does is allow the kernel to provide better diagnostics, e.g. by reporting in the IPI-generating code what remote core and context is preparing to deliver an interrupt to a task_isolation core. It may be worth considering other ways to generate useful debugging output rather than console spew, but for now that is simple and direct. Signed-off-by: Chris Metcalf --- Documentation/kernel-parameters.txt | 7 +++++++ include/linux/isolation.h | 2 ++ kernel/irq_work.c | 5 ++++- kernel/sched/core.c | 21 +++++++++++++++++++++ kernel/signal.c | 5 +++++ kernel/smp.c | 4 ++++ kernel/softirq.c | 7 +++++++ 7 files changed, 50 insertions(+), 1 deletion(-) diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index 22a4b687ea5b..48ff15f3166f 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3623,6 +3623,13 @@ bytes respectively. Such letter suffixes can also be entirely omitted. neutralize any effect of /proc/sys/kernel/sysrq. Useful for debugging. + task_isolation_debug [KNL] + In kernels built with CONFIG_TASK_ISOLATION and booted + in nohz_full= mode, this setting will generate console + backtraces when the kernel is about to interrupt a + task that has requested PR_TASK_ISOLATION_ENABLE + and is running on a nohz_full core. + tcpmhash_entries= [KNL,NET] Set the number of tcp_metrics_hash slots. Default value is 8192 or 16384 depending on total diff --git a/include/linux/isolation.h b/include/linux/isolation.h index dc14057a359c..ad94d1168c31 100644 --- a/include/linux/isolation.h +++ b/include/linux/isolation.h @@ -31,6 +31,7 @@ static inline void task_isolation_enter(void) extern bool task_isolation_syscall(int nr); extern bool task_isolation_exception(const char *fmt, ...); +extern void task_isolation_debug(int cpu); static inline bool task_isolation_strict(void) { @@ -54,6 +55,7 @@ static inline bool task_isolation_ready(void) { return true; } static inline void task_isolation_enter(void) { } static inline bool task_isolation_check_syscall(int nr) { return false; } static inline bool task_isolation_check_exception(const char *fmt, ...) { return false; } +static inline void task_isolation_debug(int cpu) { } #endif #endif diff --git a/kernel/irq_work.c b/kernel/irq_work.c index cbf9fb899d92..745c2ea6a4e4 100644 --- a/kernel/irq_work.c +++ b/kernel/irq_work.c @@ -17,6 +17,7 @@ #include #include #include +#include #include @@ -75,8 +76,10 @@ bool irq_work_queue_on(struct irq_work *work, int cpu) if (!irq_work_claim(work)) return false; - if (llist_add(&work->llnode, &per_cpu(raised_list, cpu))) + if (llist_add(&work->llnode, &per_cpu(raised_list, cpu))) { + task_isolation_debug(cpu); arch_send_call_function_single_ipi(cpu); + } return true; } diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 10a8faa1b0d4..b79f8e0aeffb 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -74,6 +74,7 @@ #include #include #include +#include #include #include @@ -746,6 +747,26 @@ bool sched_can_stop_tick(void) } #endif /* CONFIG_NO_HZ_FULL */ +#ifdef CONFIG_TASK_ISOLATION +/* Enable debugging of any interrupts of task_isolation cores. */ +static int task_isolation_debug_flag; +static int __init task_isolation_debug_func(char *str) +{ + task_isolation_debug_flag = true; + return 1; +} +__setup("task_isolation_debug", task_isolation_debug_func); + +void task_isolation_debug(int cpu) +{ + if (task_isolation_debug_flag && tick_nohz_full_cpu(cpu) && + (cpu_curr(cpu)->task_isolation_flags & PR_TASK_ISOLATION_ENABLE)) { + pr_err("Interrupt detected for task_isolation cpu %d\n", cpu); + dump_stack(); + } +} +#endif + void sched_avg_update(struct rq *rq) { s64 period = sched_avg_period(); diff --git a/kernel/signal.c b/kernel/signal.c index 0f6bbbe77b46..c6e09f0f7e24 100644 --- a/kernel/signal.c +++ b/kernel/signal.c @@ -684,6 +684,11 @@ int dequeue_signal(struct task_struct *tsk, sigset_t *mask, siginfo_t *info) */ void signal_wake_up_state(struct task_struct *t, unsigned int state) { +#ifdef CONFIG_TASK_ISOLATION + /* If the task is being killed, don't complain about task_isolation. */ + if (state & TASK_WAKEKILL) + t->task_isolation_flags = 0; +#endif set_tsk_thread_flag(t, TIF_SIGPENDING); /* * TASK_WAKEKILL also means wake it up in the stopped/traced/killable diff --git a/kernel/smp.c b/kernel/smp.c index 07854477c164..b0bddff2693d 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -14,6 +14,7 @@ #include #include #include +#include #include "smpboot.h" @@ -178,6 +179,7 @@ static int generic_exec_single(int cpu, struct call_single_data *csd, * locking and barrier primitives. Generic code isn't really * equipped to do the right thing... */ + task_isolation_debug(cpu); if (llist_add(&csd->llist, &per_cpu(call_single_queue, cpu))) arch_send_call_function_single_ipi(cpu); @@ -457,6 +459,8 @@ void smp_call_function_many(const struct cpumask *mask, } /* Send a message to all CPUs in the map */ + for_each_cpu(cpu, cfd->cpumask) + task_isolation_debug(cpu); arch_send_call_function_ipi_mask(cfd->cpumask); if (wait) { diff --git a/kernel/softirq.c b/kernel/softirq.c index 479e4436f787..ed762fec7265 100644 --- a/kernel/softirq.c +++ b/kernel/softirq.c @@ -24,8 +24,10 @@ #include #include #include +#include #include #include +#include #define CREATE_TRACE_POINTS #include @@ -335,6 +337,11 @@ void irq_enter(void) _local_bh_enable(); } + if (context_tracking_cpu_is_enabled() && + context_tracking_in_user() && + !in_interrupt()) + task_isolation_debug(smp_processor_id()); + __irq_enter(); } -- 2.1.2