From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934262AbbI1PSM (ORCPT ); Mon, 28 Sep 2015 11:18:12 -0400 Received: from mail-am1on0068.outbound.protection.outlook.com ([157.56.112.68]:63802 "EHLO emea01-am1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S934219AbbI1PSB (ORCPT ); Mon, 28 Sep 2015 11:18:01 -0400 Authentication-Results: spf=fail (sender IP is 12.216.194.146) smtp.mailfrom=ezchip.com; ezchip.com; dkim=none (message not signed) header.d=none;ezchip.com; dmarc=none action=none header.from=ezchip.com; From: Chris Metcalf To: Gilad Ben Yossef , Steven Rostedt , Ingo Molnar , Peter Zijlstra , Andrew Morton , "Rik van Riel" , Tejun Heo , Frederic Weisbecker , Thomas Gleixner , "Paul E. McKenney" , Christoph Lameter , Viresh Kumar , Catalin Marinas , Will Deacon , Andy Lutomirski , , CC: Chris Metcalf Subject: [PATCH v7 05/11] task_isolation: add debug boot flag Date: Mon, 28 Sep 2015 11:17:20 -0400 Message-ID: <1443453446-7827-6-git-send-email-cmetcalf@ezchip.com> X-Mailer: git-send-email 2.1.2 In-Reply-To: <1443453446-7827-1-git-send-email-cmetcalf@ezchip.com> References: <1443453446-7827-1-git-send-email-cmetcalf@ezchip.com> X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1;DB3FFO11OLC002;1:UcWn6mzCiVaq6ysULMuDdsjZ7T1i2dncQEVKW12sd1wS0AC10Y+PYDgcZ9qH2YaPuq+oESJvCIVKmliKhsQnLyftAJl483/ge9SL+eztkw9eoosAVNHdZ/kZGEhlkcJ8WHD4st5rPSM4cpsHmRVUjT1rwZX+luIpmYUfLM20yenrpSNXY73u/xY9yfw3nvBLuzJIzSiZjaOfJ4eoLLXq7aIuX/tFrIlBWSi0kJXwb1IX4Iy2ogV8Wy3hJUo2QG4PSj7FnGmUqqR5P6WTGko3eNbntS5iYWZknqJZAD7hEY87V+tew7AEyj7ZTFyif67fs+yJhfWHanPWk5beHLbUVkY2InvsR+Dwl8WwVq+d/Lo= X-Forefront-Antispam-Report: CIP:12.216.194.146;CTRY:US;IPV:NLI;EFV:NLI;SFV:NSPM;SFS:(10009020)(6009001)(2980300002)(1110001)(1109001)(339900001)(199003)(189002)(50466002)(50226001)(106356001)(33646002)(105606002)(92566002)(48376002)(106466001)(47776003)(19580395003)(86362001)(64706001)(106476002)(19580405001)(229853001)(4001540100001)(81156007)(85426001)(62966003)(104016004)(97736004)(5007970100001)(5001770100001)(77156002)(87936001)(6806005)(5001860100001)(50986999)(76176999)(5001830100001)(2201001)(2950100001)(42186005)(46102003)(575784001)(189998001)(5003940100001)(36756003)(11100500001)(107886002)(5001960100002)(68736005)(9376005)(921003)(83996005)(2101003)(4001430100001)(1121003);DIR:OUT;SFP:1101;SCL:1;SRVR:AM3PR02MB0421;H:ld-1.internal.tilera.com;FPR:;SPF:Fail;PTR:InfoNoRecords;A:1;MX:1;LANG:en; MIME-Version: 1.0 Content-Type: text/plain X-Microsoft-Exchange-Diagnostics: 1;AM3PR02MB0421;2:BkWl0tFMSunE8goJ28gdqXOf8L62K/A8hTvLNeafSqErjTWFT+WEKsxHwiqTKGn9mS8txNBEN2j5iQpttqjmVDC/fsHikkC4ZYj+eveiJ+f3pIUl2FDphBHLMcnUrvC0LHBv9TVunfbi+ZhPXpZ+LStaNcFPlQHNBxGrA/tPE7M=;3:vgt7iAEbssZ1rvoR3eYiVeqhCn0HtxNVa6jqfQZU6Sljl3NSnnc7lREnkUo+cWPZdI34xYV5xf/irfevRXMnaHDu92cEyOSKNiNnCVRx+n77vUcjZURw+2bUPnfjsRMoxvgKzL/nI7Q5PWNRuVrV6FtjhDQitIXjlQI55CCt9cVtxtsY4mKYEosAgah6i2OG+4+UQoDs2j+oGiN1/pdl7+39xP2bbOC/riofTIVzFOXyLvzkDaMB9qPFhlHR0/At;25:aBRFuwS+HUZ6LmaceMbL+wYVoIssyJjnuR3UgeHOYmv3rZr6YDAItmy3nQoYdTQuvyld8M4urn9audjOH4bXZg8ESrNIPVHrkS1eEo0NhfnSOjqh0V/ue3aXfGE3kbEtwcaw9U35Ql7YJEu98nfwjp35fW5C3yV9jmMSR4W1+bHQ548vhsjy8NV0HY/5tbo029sh35G9Qcyh37BoJc3nqgBb7plR/alxLjJXyjQikLxVikjDV0m4k9N86EAdKFvYi0Vavsa2IU6PPw/kaPtVJA==;20:JG4CSuLeZNuuexiepZ5xrtkfhKMY/otzrPRBt4/ZuftrXXTMNCCxMaUOAqcx6b8c6oKjJpuVTeAN1wT+AFyXKOjk228rdJIgOZri/+3aSQ/4x4VpgJbKpKr+x/IRtff+D8yY5+hg2igaoraor0umjbjeeOrdrN2Kxzo9oIttND0= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:AM3PR02MB0421; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(5005006)(520078)(8121501046)(3002001);SRVR:AM3PR02MB0421;BCL:0;PCL:0;RULEID:;SRVR:AM3PR02MB0421; X-Microsoft-Exchange-Diagnostics: 1;AM3PR02MB0421;4:L1cf1A6rny9AwrHwR7Hm7Gc4lqsU2t4UMkywS5xexVUSxYjqnRzmIC9A1pyDQeizJRFJCsmlOyH4PpYCMJjAbKauyIMGp1IZht9Et/gJr/9DDl5ubrVs+Cfp5XLA4zWYgp9qaBy8DcgtmLcs5RgqQteqg7DwBX2isZEGkkUhKtnRYcQUDfhF4wbe8sIcWMLSQkXD7SnAbZLyge9nM+W6T22oWKz/ZbtS30i2hhUS1XjPsPelixJ2wsFTj2F9JOB4NRX51MVr4qSBgEZbdEhthpJsnsenjfYLloRZ0RMDHB6R0jIYDhTKilxHhh3MPW3BERinonmFLxDyiUmk8LkFa3Qxop+Yzuj97/IdOK6j2Gc= X-Forefront-PRVS: 0713BC207F X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;AM3PR02MB0421;23:4VmjYw8JVrvsVS3MdB5FYryG8uPfWN163Oq49KIqK?= =?us-ascii?Q?uRv/sRXKvjal3jCEICsQWAq57c+LVMGkR87Eew083xEg1S8sUZYkt92gKM/w?= =?us-ascii?Q?avgx19Ogoqhpzd3B/VS7wHo3jqnZARf5ikVvb/hsaV+6/wj25qaQEgOuEnut?= =?us-ascii?Q?P0GhsRAyyHRhCL4JqE46YBYAPi+7xo+xPINL0UVDHXSAkAZ46DfZbgDpAolZ?= =?us-ascii?Q?2lZbFQkfKrS2AyBhyI3U+L8l+thkYxsXrz78eZ0MXW5PUp/qyE5svOSEKHnX?= =?us-ascii?Q?Rb2bCSRYu1NgYYDQJ66edIHVH6EuIF0TmPu1+t0zpaoM4zitPSzlBTQyoRF0?= =?us-ascii?Q?T+4G+DRUoSwPETvMs7YeUUrdxEJHEVUFtr6rpU9AZCaML3nZrXls/+a1Dujw?= =?us-ascii?Q?dfWizyMgUQ9jqiDi6quYZ9ULRNDDE1qm+FCAvvktopS/T6nhdRg9EBSi3XFt?= =?us-ascii?Q?x97gUOwppW9g6ZhfszXqMSb7+tkA6zQyCwpwrM57tUAEm95LTyeByP3jX6h1?= =?us-ascii?Q?vgbqJfL6h+ji/zEL5Pl7bQRkVVCUVrzqdYQ5IuB1CSO1vWe97Vc66awVcHW4?= =?us-ascii?Q?+UnrJFUMsg3jpF/zLbaxF+EadZs8B8AhWhbQEVRShH+fsyjvaWrQc++oXZy1?= =?us-ascii?Q?JBigNbzmcoipvh+qmhTg72X1ByAfABTwlo1zjrSCZQxtZyz4FGAT4H87bjAH?= =?us-ascii?Q?PgegfCJy84mqqEEuk/y28CTo1MMwK0iTADCPDOhOGqVFKHFu85iWzAuvoUi0?= =?us-ascii?Q?tjeJLZrjoGvActRJ/yRkdx6CTPdwI4giZFr+sm1IX4hRej0ii66BiAgkZM8p?= =?us-ascii?Q?DK9OAn8wTMsXCetX7EhxUnVVJ78vYNXlU5E8lzNRzEKklpgnewzSsyHkOVDQ?= =?us-ascii?Q?+BD2IKd9oRAWO8BmJP3dbdrdK0Lgw5jySGkVKp86V0jKa+rWMdN1JMqcokUd?= =?us-ascii?Q?EARpQxTcpj8yyLdrcITiit87QtUQt4D4hg18frbsHcjANw0alH3S/R81WjKB?= =?us-ascii?Q?6ENea1BCcDPUk9TSalRZkfuy7wJwpDRDhQHlPwvcUW9zNGI2/4QsDsrk5wSa?= =?us-ascii?Q?Ef3bwDoffA5W+MDsN+QMa+FnVgSFXy6qIxalnEChaYNbnGZ/wwTGMQpH/aWM?= =?us-ascii?Q?tOA18MinazX+ZrHpKkAN5ouIQZUMbVY3c3AXTdbuQWRshMAxocrCqg03jf6o?= =?us-ascii?Q?VlkFOHPSYr6YV1zdJ+HENeh2AXlGSH5KBakdDEqiMMv5FAnZJTMpNwPX0bcE?= =?us-ascii?Q?DIViQGjuadf13QkYMuJessmKkB8cWXekb0tZXXsHVCNvN5Zi5KYLupNCMeJc?= =?us-ascii?Q?OHniM3RWjnDxlER0vcmywHUVWLkQQR4JtoCPq6TL/pAMHM+YKXOGZ0Y9LnEp?= =?us-ascii?Q?C9ksw8mlb4P2DwoEto7ZiEp0dFKRBAwpcarNwu4HtAOWE4bXuXagl1kyJXyZ?= =?us-ascii?Q?prR1skz0ZLN7Xil6gMLr0GYm5Rm8+U=3D?= X-Microsoft-Exchange-Diagnostics: 1;AM3PR02MB0421;5:yaKAbxF5Vnwe0bh2s6oJOsVcnLgi+BL60dZ/KOK23row1uxV0kjN5h4qGrOXSuaMIuuOeuMNkdc41WOZUoWjhBpqgVkLqxaMX3Dh5HmU8VkLNGa7Vg8GdxqW5AZnutA4Wr5ZrZbF8aycEr+oiz0MwA==;24:QDP1TShZ0jrCVV2lW+2sxmTZFEFg7v/ukPqKaFDeqoAHzDE5X9HIka11bs2ls5O2tTDmpVdpJnVX4DPX3zlsPZdAHsLfqMccJ1Tn44JXLj8=;20:/6TExkeC6syyxI0N9xNjEwfC65op0UfWnMfAazvJKgorF7iNODW1P82h1Cfa/+KG++MbHOFCDrMkmzJNDlf+KQ== SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: ezchip.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Sep 2015 15:17:57.7554 (UTC) X-MS-Exchange-CrossTenant-Id: 0fc16e0a-3cd3-4092-8b2f-0a42cff122c3 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=0fc16e0a-3cd3-4092-8b2f-0a42cff122c3;Ip=[12.216.194.146];Helo=[ld-1.internal.tilera.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM3PR02MB0421 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The new "task_isolation_debug" flag simplifies debugging of TASK_ISOLATION kernels when processes are running in PR_TASK_ISOLATION_ENABLE mode. Such processes should get no interrupts from the kernel, and if they do, when this boot flag is specified a kernel stack dump on the console is generated. It's possible to use ftrace to simply detect whether a task_isolation core has unexpectedly entered the kernel. But what this boot flag does is allow the kernel to provide better diagnostics, e.g. by reporting in the IPI-generating code what remote core and context is preparing to deliver an interrupt to a task_isolation core. It may be worth considering other ways to generate useful debugging output rather than console spew, but for now that is simple and direct. Signed-off-by: Chris Metcalf --- Documentation/kernel-parameters.txt | 7 +++++++ include/linux/isolation.h | 2 ++ kernel/irq_work.c | 5 ++++- kernel/sched/core.c | 21 +++++++++++++++++++++ kernel/signal.c | 5 +++++ kernel/smp.c | 4 ++++ kernel/softirq.c | 7 +++++++ 7 files changed, 50 insertions(+), 1 deletion(-) diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index 22a4b687ea5b..48ff15f3166f 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3623,6 +3623,13 @@ bytes respectively. Such letter suffixes can also be entirely omitted. neutralize any effect of /proc/sys/kernel/sysrq. Useful for debugging. + task_isolation_debug [KNL] + In kernels built with CONFIG_TASK_ISOLATION and booted + in nohz_full= mode, this setting will generate console + backtraces when the kernel is about to interrupt a + task that has requested PR_TASK_ISOLATION_ENABLE + and is running on a nohz_full core. + tcpmhash_entries= [KNL,NET] Set the number of tcp_metrics_hash slots. Default value is 8192 or 16384 depending on total diff --git a/include/linux/isolation.h b/include/linux/isolation.h index 27a4469831c1..9f1747331a36 100644 --- a/include/linux/isolation.h +++ b/include/linux/isolation.h @@ -18,11 +18,13 @@ extern void task_isolation_enter(void); extern void task_isolation_syscall(int nr); extern void task_isolation_exception(void); extern void task_isolation_wait(void); +extern void task_isolation_debug(int cpu); #else static inline bool task_isolation_enabled(void) { return false; } static inline void task_isolation_enter(void) { } static inline void task_isolation_syscall(int nr) { } static inline void task_isolation_exception(void) { } +static inline void task_isolation_debug(int cpu) { } #endif static inline bool task_isolation_strict(void) diff --git a/kernel/irq_work.c b/kernel/irq_work.c index cbf9fb899d92..745c2ea6a4e4 100644 --- a/kernel/irq_work.c +++ b/kernel/irq_work.c @@ -17,6 +17,7 @@ #include #include #include +#include #include @@ -75,8 +76,10 @@ bool irq_work_queue_on(struct irq_work *work, int cpu) if (!irq_work_claim(work)) return false; - if (llist_add(&work->llnode, &per_cpu(raised_list, cpu))) + if (llist_add(&work->llnode, &per_cpu(raised_list, cpu))) { + task_isolation_debug(cpu); arch_send_call_function_single_ipi(cpu); + } return true; } diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 3595403921bd..8ddabb0d7510 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -74,6 +74,7 @@ #include #include #include +#include #include #include @@ -743,6 +744,26 @@ bool sched_can_stop_tick(void) } #endif /* CONFIG_NO_HZ_FULL */ +#ifdef CONFIG_TASK_ISOLATION +/* Enable debugging of any interrupts of task_isolation cores. */ +static int task_isolation_debug_flag; +static int __init task_isolation_debug_func(char *str) +{ + task_isolation_debug_flag = true; + return 1; +} +__setup("task_isolation_debug", task_isolation_debug_func); + +void task_isolation_debug(int cpu) +{ + if (task_isolation_debug_flag && tick_nohz_full_cpu(cpu) && + (cpu_curr(cpu)->task_isolation_flags & PR_TASK_ISOLATION_ENABLE)) { + pr_err("Interrupt detected for task_isolation cpu %d\n", cpu); + dump_stack(); + } +} +#endif + void sched_avg_update(struct rq *rq) { s64 period = sched_avg_period(); diff --git a/kernel/signal.c b/kernel/signal.c index 0f6bbbe77b46..c6e09f0f7e24 100644 --- a/kernel/signal.c +++ b/kernel/signal.c @@ -684,6 +684,11 @@ int dequeue_signal(struct task_struct *tsk, sigset_t *mask, siginfo_t *info) */ void signal_wake_up_state(struct task_struct *t, unsigned int state) { +#ifdef CONFIG_TASK_ISOLATION + /* If the task is being killed, don't complain about task_isolation. */ + if (state & TASK_WAKEKILL) + t->task_isolation_flags = 0; +#endif set_tsk_thread_flag(t, TIF_SIGPENDING); /* * TASK_WAKEKILL also means wake it up in the stopped/traced/killable diff --git a/kernel/smp.c b/kernel/smp.c index 07854477c164..b0bddff2693d 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -14,6 +14,7 @@ #include #include #include +#include #include "smpboot.h" @@ -178,6 +179,7 @@ static int generic_exec_single(int cpu, struct call_single_data *csd, * locking and barrier primitives. Generic code isn't really * equipped to do the right thing... */ + task_isolation_debug(cpu); if (llist_add(&csd->llist, &per_cpu(call_single_queue, cpu))) arch_send_call_function_single_ipi(cpu); @@ -457,6 +459,8 @@ void smp_call_function_many(const struct cpumask *mask, } /* Send a message to all CPUs in the map */ + for_each_cpu(cpu, cfd->cpumask) + task_isolation_debug(cpu); arch_send_call_function_ipi_mask(cfd->cpumask); if (wait) { diff --git a/kernel/softirq.c b/kernel/softirq.c index 479e4436f787..ed762fec7265 100644 --- a/kernel/softirq.c +++ b/kernel/softirq.c @@ -24,8 +24,10 @@ #include #include #include +#include #include #include +#include #define CREATE_TRACE_POINTS #include @@ -335,6 +337,11 @@ void irq_enter(void) _local_bh_enable(); } + if (context_tracking_cpu_is_enabled() && + context_tracking_in_user() && + !in_interrupt()) + task_isolation_debug(smp_processor_id()); + __irq_enter(); } -- 2.1.2