From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752922AbbGMT6Q (ORCPT ); Mon, 13 Jul 2015 15:58:16 -0400 Received: from mail-db3on0074.outbound.protection.outlook.com ([157.55.234.74]:33616 "EHLO emea01-db3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752069AbbGMT6O (ORCPT ); Mon, 13 Jul 2015 15:58:14 -0400 Authentication-Results: spf=fail (sender IP is 12.216.194.146) smtp.mailfrom=ezchip.com; ezchip.com; dkim=none (message not signed) header.d=none; From: Chris Metcalf To: Gilad Ben Yossef , Steven Rostedt , Ingo Molnar , Peter Zijlstra , Andrew Morton , Rik van Riel , Tejun Heo , Frederic Weisbecker , Thomas Gleixner , "Paul E. McKenney" , Christoph Lameter , Viresh Kumar , Catalin Marinas , Will Deacon , , , CC: Chris Metcalf Subject: [PATCH v4 0/5] support "cpu_isolated" mode for nohz_full Date: Mon, 13 Jul 2015 15:57:56 -0400 Message-ID: <1436817481-8732-1-git-send-email-cmetcalf@ezchip.com> X-Mailer: git-send-email 2.1.2 In-Reply-To: <1433345365-29506-1-git-send-email-cmetcalf@ezchip.com> X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1;DB3FFO11FD055;1:jFOerTlJTCdTUNWWw4YHfSiVawFoRx8YI30JthKrMPOYm9hwKcYb7ElnPT/XVE+IxntGLm6YCMLBuUvUb/krfVSObckcyxk9EVn8S28+mFDDCfZWe+Xvm1X9DzZ8jqYA1aoucfRvaVdNcft6eglI364RTbWb0eXP27KNuY2BROf3Bm10yjbD9znpjUZJ5yrXKyEukrU3FRTsdlm0akHTQ6rxub1laQ3Ypp1Gdh0DDiEAJ2PIuLfChZJUZd2rwhbEgf7Zf1jM18f+4fzoXmMt7ehKle2z9ySNAHeo6vnFC8x51sOI3ZcBm+L4Ypb/1fRtIFI+Pe5OzInQ1eMQZiuTaKFR8vYgyHve1+9/djZdNhBkYC52saR/RjXNmXHwO5KCAVxwlLJAUEhy8uocJlJnjQ== X-Forefront-Antispam-Report: CIP:12.216.194.146;CTRY:US;IPV:NLI;EFV:NLI;SFV:NSPM;SFS:(10009020)(6009001)(2980300002)(339900001)(199003)(189002)(5003940100001)(5001770100001)(50986999)(46102003)(50226001)(36756003)(47776003)(189998001)(5001920100001)(85426001)(42186005)(87936001)(6806004)(229853001)(2950100001)(19580395003)(15975445007)(2201001)(48376002)(86362001)(50466002)(5001960100002)(107886002)(106466001)(105606002)(104016003)(62966003)(77156002)(33646002)(92566002)(921003)(1121003)(4001430100001);DIR:OUT;SFP:1101;SCL:1;SRVR:AM2PR02MB0420;H:ld-1.internal.tilera.com;FPR:;SPF:Fail;MLV:sfv;A:1;MX:1;LANG:en; MIME-Version: 1.0 Content-Type: text/plain X-Microsoft-Exchange-Diagnostics: 1;AM2PR02MB0420;2:6xRSZKn/gKVzJvSbFlsksKCzUwZZ4scD/p4zx3Q1dAHbiwojF3E2hhLzxul3+9db;3:+jxJQ+Hbr7XKlg7ECAkLoWKbrDHQbE1Eg+NCiFbLU8jJZPgmtXgOD80JQW+4SIH/Kuc7GrpIDeOph+wKGaQhatrXB/Y+3wGbKDXZf5k/AGB2bbinrpWjKlPXzUCFAgg1SoAERmnAMUK/qrwKF6CKhlY3Ar37rlP04OQRsBcAgvjC/ye27cvLbCt3TnE/k8uFlHH6pgZaAzS4N1VICPQYVrMF5ZcNzniOg3WOxIqGvtveTA4M0kHIAeH3yEsLZMah;25:9KDpSS8WW4h73FF1OnB5PMLrlXrS0wJeTgdgcJnEfqDdnNZxezbT7OwZsDR83OSwzM2rvR3l5N/1BfeIsYTfpsVDF4fUKlIRhl31XNDLp3ANJPeVz16CNUrVvMWWOlfA2K5B3D6RQH+mY0K/GTiodZdP4GAx6mN7PgLV5th8fEInR96XAeuhzuYp4F5E3bRiGTxwEs33fKbG3qpJNyBkBZBeyFEhgM5/CWrC/3EiSRjxTyEfEs3ojNqL0g1dr9zjYlUm4s1ufi2AEMyRj1LvwQ==;20:NB4ZiDqK1yw1kgCaug0gew/XJOpzgFom7rmkuqI1i52LrEoWZfSO9NtZdQuj2VfFaVUz8szTjk3PTZOe1r1it6EQuDmkaORnDA7f3lVZq513mANvJ7G3c0CFUyY5NadlKzChFZ7Xq+PEpgDo2PScSIv8+tc7NgmT5y+AeF+vCYc= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:AM2PR02MB0420; AM2PR02MB0420: X-MS-Exchange-Organization-RulesExecuted X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(5005006)(3002001);SRVR:AM2PR02MB0420;BCL:0;PCL:0;RULEID:;SRVR:AM2PR02MB0420; X-Microsoft-Exchange-Diagnostics: 1;AM2PR02MB0420;4:ILymZA5+k4Q960DIs4Dz7QciNKrWrDsr4iHJ7Ncx47BGM0XdeAIbK8wVQCoccWVC2onnRDf2bDz6gKmrnyHifK8KDase8EwVXfwbXBUb4oDLg69m/fw7IFmJPTM9yjjjFUtef7qfVZcJwOAgUc89D3vbIQB1IKSjHMRQYgaOBGA1/FMvzvW0HAEIClaoDSFpIX/Bn2KaIjxAX17e3/ugIovvHYbZi51+0OiahADBSVujSokypHc5NckicSv16Lw53MM8u3zI8bQ2+zlxe1Ue8fGZnDvVf2Tw0uE1gR/nAfM= X-Forefront-PRVS: 0636271852 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;AM2PR02MB0420;23:uUiB5pqFKo03Ex2tjjlhPIv1Iu4aEsKHRo8G3Jbm/?= =?us-ascii?Q?CEsZoQwvHlcgENYo97G7V/odc2lF0+noh8Us5co7bWBhy8YluNcR+12GppZC?= =?us-ascii?Q?yy4NMKVkCnMP71ObhpBabqRxZ7t92MW6vRwErvpi/DXlTgaRniTT6NAS6ou9?= =?us-ascii?Q?OBK4W2Z6rwcNNsTAyE7kBxK2MmnYvJ9oPFLSXbwnRc5lDtAfm1mwlnL1wE/T?= =?us-ascii?Q?i5O2hYL5qc6YNu4NvtGLlUb+Bj37WnGf+I2pJVLVgq0BRwqq0ftjmGw2rr6i?= =?us-ascii?Q?aq96LPt0/0UYm2Qh1mbBU6Ovx7+A6tQF4DOHS/ASlf5YMiKTgD5YniEu2nO8?= =?us-ascii?Q?kWjVIFoWXksn7Ry/0SuXI9NCSaXnS7+r3Hgb+ghrG494R6HTztXLVCdiZ8jr?= =?us-ascii?Q?xJ6DQq3avxpqpE4YqgBIncytuE0LA+tDL6mBLa5orfkEPSeKATH832/190RS?= =?us-ascii?Q?i4a+qNe9iAep26wQMwVDQ2zjYQgXhBByCWPeHZAHqziX7WOGlU7i3HrNQiop?= =?us-ascii?Q?6uHqvuGrGuqQUxLzop07EBnhq13pfl8XcsCpyXFsR5NN4183F/JATOfSCD4x?= =?us-ascii?Q?LwiVGZYhn5/7obM8gJP4kPD3e9mivU5kMevIm3iyzppueS5Wv1Sk63zuaCji?= =?us-ascii?Q?JICbLTKgr7xRTfq6QfUme61EX4ddB+RWnk/7qcQuzkgPKlTIIg7P69uCl1wj?= =?us-ascii?Q?b/tDl1kqTvm2cQYq8wTO3iC3TTYcOm40QVD0QnnWn4h/mEA86760Q+XuHodi?= =?us-ascii?Q?bP8phkwf0ndaJGZPZV1kjJc1NvWPazC493GNOHI8X8lJg+J/Mr+WK3RtrDyf?= =?us-ascii?Q?tn75G+OvVe2F2gZmcGz03967guySlDM8uJBJFvCDqXsO7uwCCJDqgVAP3coD?= =?us-ascii?Q?c2z1wT1Caf/oajN8JKZuCmNk12o5Ci7TDuuhhpIYKN2xNf9OT/8NrwGfxxbq?= =?us-ascii?Q?A91CxVKG9T3wQXlnLnhLw8S0idOEct0HErV9V9pj1vFMrncApLxbJSqSZTY8?= =?us-ascii?Q?sTQKLQKPxyHTqlJLpT4AFEZ?= X-Microsoft-Exchange-Diagnostics: 1;AM2PR02MB0420;5:DiOenKrYmm4lgDl+Tye07Tcf3b03MT3ihbEgnI5g+/CSPL/2E39dyOfFYE4dUDrj+4Eif2tXGBBIWRjK9HM2w8H5OJo4f4zAG3AYJXn02qFcOoyZRgOfrkIcfR8oHvzhXIDjqu2G8o9iNlARca48uA==;24:lhAlNCPbbV/7cIIFm+NZ5u503UG1X2m5FBqSxCSf0Cmn2wijMTDR+STGvQ8bzid+Um85FkSg0z0bFNv4YAiTO32+3OSStNyI7vKRWg9RCzg=;20:o90x5RWrlPosOaeKwft8v11/9XUxqU1QgPMOLuHGmIguzvXGIc1KuihKQfQREb82t7RMO+1gFOhUv0Fbs0fmZQ== SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: ezchip.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jul 2015 19:58:10.0097 (UTC) X-MS-Exchange-CrossTenant-Id: 0fc16e0a-3cd3-4092-8b2f-0a42cff122c3 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=0fc16e0a-3cd3-4092-8b2f-0a42cff122c3;Ip=[12.216.194.146];Helo=[ld-1.internal.tilera.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM2PR02MB0420 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This posting of the series is basically a "ping" since there were no comments to the v3 version. I have rebased it to 4.2-rc1, added support for arm64 syscall tracking for "strict" mode, and retested it; are there any remaining concerns? Thomas, I haven't heard from you whether my removal of the cpu_idle calls sufficiently addresses your concerns about that aspect. Are there other concerns with this patch series at this point? Original patch series cover letter follows: The existing nohz_full mode does a nice job of suppressing extraneous kernel interrupts for cores that desire it. However, there is a need for a more deterministic mode that rigorously disallows kernel interrupts, even at a higher cost in user/kernel transition time: for example, high-speed networking applications running userspace drivers that will drop packets if they are ever interrupted. These changes attempt to provide an initial draft of such a framework; the changes do not add any overhead to the usual non-nohz_full mode, and only very small overhead to the typical nohz_full mode. A prctl() option (PR_SET_CPU_ISOLATED) is added to control whether processes have requested this stricter semantics, and within that prctl() option we provide a number of different bits for more precise control. Additionally, we add a new command-line boot argument to facilitate debugging where unexpected interrupts are being delivered from. Code that is conceptually similar has been in use in Tilera's Multicore Development Environment since 2008, known as Zero-Overhead Linux, and has seen wide adoption by a range of customers. This patch series represents the first serious attempt to upstream that functionality. Although the current state of the kernel isn't quite ready to run with absolutely no kernel interrupts (for example, workqueues on cpu_isolated cores still remain to be dealt with), this patch series provides a way to make dynamic tradeoffs between avoiding kernel interrupts on the one hand, and making voluntary calls in and out of the kernel more expensive, for tasks that want it. The series (based currently on v4.2-rc1) is available at: git://git.kernel.org/pub/scm/linux/kernel/git/cmetcalf/linux-tile.git dataplane v4: rebased on kernel v4.2-rc1 added support for detecting CPU_ISOLATED_STRICT syscalls on arm64 v3: remove dependency on cpu_idle subsystem (Thomas Gleixner) use READ_ONCE instead of ACCESS_ONCE in tick_nohz_cpu_isolated_enter use seconds for console messages instead of jiffies (Thomas Gleixner) updated commit description for patch 5/5 v2: rename "dataplane" to "cpu_isolated" drop ksoftirqd suppression changes (believed no longer needed) merge previous "QUIESCE" functionality into baseline functionality explicitly track syscalls and exceptions for "STRICT" functionality allow configuring a signal to be delivered for STRICT mode failures move debug tracking to irq_enter(), not irq_exit() Note: I have not removed the commit to disable the 1Hz timer tick fallback that was nack'ed by PeterZ, pending a decision on that thread as to what to do (https://lkml.org/lkml/2015/5/8/555); also since if we remove the 1Hz tick, cpu_isolated threads will never re-enter userspace since a tick will always be pending. Chris Metcalf (5): nohz_full: add support for "cpu_isolated" mode nohz: support PR_CPU_ISOLATED_STRICT mode nohz: cpu_isolated strict mode configurable signal nohz: add cpu_isolated_debug boot flag nohz: cpu_isolated: allow tick to be fully disabled Documentation/kernel-parameters.txt | 6 +++ arch/tile/kernel/process.c | 9 ++++ arch/tile/kernel/ptrace.c | 6 ++- arch/tile/mm/homecache.c | 5 +- arch/x86/kernel/ptrace.c | 2 + include/linux/context_tracking.h | 11 ++-- include/linux/sched.h | 3 ++ include/linux/tick.h | 28 ++++++++++ include/uapi/linux/prctl.h | 8 +++ kernel/context_tracking.c | 12 +++-- kernel/irq_work.c | 4 +- kernel/sched/core.c | 18 +++++++ kernel/signal.c | 5 ++ kernel/smp.c | 4 ++ kernel/softirq.c | 6 +++ kernel/sys.c | 8 +++ kernel/time/tick-sched.c | 104 +++++++++++++++++++++++++++++++++++- 17 files changed, 229 insertions(+), 10 deletions(-) -- 2.1.2