From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933722AbdKCRyJ (ORCPT ); Fri, 3 Nov 2017 13:54:09 -0400 Received: from mail-eopbgr50052.outbound.protection.outlook.com ([40.107.5.52]:35904 "EHLO EUR03-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S932672AbdKCRyF (ORCPT ); Fri, 3 Nov 2017 13:54:05 -0400 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=cmetcalf@mellanox.com; Subject: Re: [PATCH v16 09/13] arch/arm64: enable task isolation functionality To: Mark Rutland Cc: Steven Rostedt , Ingo Molnar , Peter Zijlstra , Andrew Morton , Rik van Riel , Tejun Heo , Frederic Weisbecker , Thomas Gleixner , "Paul E. McKenney" , Christoph Lameter , Viresh Kumar , Catalin Marinas , Will Deacon , Andy Lutomirski , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org References: <1509728692-10460-1-git-send-email-cmetcalf@mellanox.com> <1509728692-10460-10-git-send-email-cmetcalf@mellanox.com> <20171103173203.h7y7c65r4ml6r2jm@lakrids.cambridge.arm.com> From: Chris Metcalf Message-ID: Date: Fri, 3 Nov 2017 13:53:51 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: <20171103173203.h7y7c65r4ml6r2jm@lakrids.cambridge.arm.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Originating-IP: [216.156.69.42] X-ClientProxiedBy: BN6PR13CA0036.namprd13.prod.outlook.com (2603:10b6:404:13e::22) To DB6PR0501MB2757.eurprd05.prod.outlook.com (2603:10a6:4:82::9) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ce878d82-8d59-4d32-ff41-08d522e3dd73 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(48565401081)(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(2017052603238);SRVR:DB6PR0501MB2757; X-Microsoft-Exchange-Diagnostics: 1;DB6PR0501MB2757;3:wcBJmjR3cQhs8dsDlrk7vQ/5NoqRCu3/+33UHxFLh+XyAHj+nbSK3LuLzw6ZsmPwAqEzg305I9yhfxF17a/R2qWrkDV3qY+YJMLxObMfermmoGXBmHcD+9vBzI0J64rkS55JLbUgPL9Ywf5HKVskp5tjYWItwgrH3PRaV/lw9vFFpz2GSa0S+d81RkTFvyJmcOQt08LWPDIffT3RQadQd8ETfiPzgEDuY9cs8hmRTmyQf6wrUI9aVNiHD4f6+ydS;25:UJldAzg58hN+sSCk76G7v0eYw7BZQ3F0oxwnu9DpOcIezdflX5hu0es8+lD3cPL5uirPDbSkpfgJEDF5ecdqt2sGA2FSpQu5dnK6NrD2SNp8pPDhIIMV7xCNOcXQIqXw70d8+MO5vAQVlvFUZZjI7MC1/QwZQQ8m8ccw59Xx/JpZFy3IwxZjUTIPuo4wyf0QKLy4gJCd6YbJNCmNVt38uEAvU1e+zAzX0+OsrpoTo6rvnp5Em+xvMwMiQUrYsRD0TpFNAFJzYj5PJ7QE8QkvMvWpq05FvRpQUfXTpGpAgbQrlfZOlJzAklCsdcyAeYc1TKpU127+aD5DT7btVUYI5w==;31:GjJrlCjEP4xrKpN7igr2C41FroKSGD/ps9tocSh9ZuFBUO7iRG1d/oeADHTiD4Ja6keA4kWRJl9ktY+e+YkS2YVdHMmTW2tpSteaDIabmp27pm1BTcdAFLLLZB7LEIkWYBZpQbvVvEwOEy9vuUXlfljwTFeA4O5YI0XLYdTTwtwabTTAnacoh+FbXjZl3PlAKXuWzMLFloz5q25rfKUwb94DvlfFNFGJMzMEkxR4ko0= X-MS-TrafficTypeDiagnostic: DB6PR0501MB2757: X-Microsoft-Exchange-Diagnostics: 1;DB6PR0501MB2757;20:J819NTjWGuqUKYP1YmP1FZ66fDcy9HWNWRcdEx27mPTRm6r6KSD4FCrwlWL1mxYR+GOhbGnkExez3KrYdbHmI1J7DV3RXHSB4eYhexUnU2U18nnRZeenMikq8AXQ5Upmv36vw6Rmn/n2e+rW5911FJnUsf6XXN8x2+VQaM+7LftZQcjxOxjURizRZttN44zJY7HeI7ApS9YvstRmD9dis1MHQUJ5erRqVSJELcfJml1PpeUQl2MQjg0VYUR9fjQg1O87L877zlG2LK8I8g5OMecdBTMzGJfdU6TKdkG6hmsRdNJxxIn4BgUVBHMG9+HGKppyo/l3iJK449cbQ+iZSI/wbfCPRGlPbWifnf/5VsecvuzTkykJaT2j2U1MwnjQEzv3OWoydnxw/IvN/HIL2ul0Ltg7HVEgLUyNOZCrOwkT2DZK3CzE/atU5jcpPc7qgHUDvNcjDmv4Rri6Nh3kDxYRCSY66tvZhBTG/S/AG5b4pYRLi62Nqrg2Z97C4cNM;4:p7zarHOjKdbuwPldVbkSqpEgoO/NIhdRUJS9uXDvf4Sy+5ourMqtM8q7Zxp1Ho6Ljv4AjGGamwvT1tPDKt+ueWUZdmiBDYZp0C6JMPApcaseTMihH05zEIry/DAV1ychxkdYGtaku2yRauLBo0pTqVQtoc+h2Mbnfna+yk9dvlUX4Ej2qbBsQFT3WUGskQ+AexwkeN2VbgVxkwt6z8UepSMuRVxuLuqBEpkAgsGnq/hM/JyuvzTXls1NbbaZgnzXkTqPP74CIISRW3uimWnOFV28m0wgT8OceJmAUJWvldo2Yk7pjDzYCG3HB2ChNYDZ X-Exchange-Antispam-Report-Test: UriScan:(171992500451332); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(10201501046)(3002001)(3231021)(93006095)(93001095)(100000703101)(100105400095)(6055026)(6041248)(20161123562025)(20161123558100)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123564025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:DB6PR0501MB2757;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:DB6PR0501MB2757; X-Forefront-PRVS: 0480A51D4A X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(6009001)(6049001)(376002)(346002)(189002)(57704003)(24454002)(199003)(305945005)(50466002)(7736002)(316002)(105586002)(47776003)(229853002)(76176999)(68736007)(2906002)(64126003)(53546010)(101416001)(39060400002)(50986999)(3846002)(6116002)(54356999)(8936002)(106356001)(31686004)(81156014)(81166006)(6306002)(36756003)(58126008)(8676002)(65826007)(53936002)(16576012)(3260700006)(25786009)(31696002)(478600001)(16526018)(97736004)(6486002)(83506002)(2870700001)(2950100002)(6916009)(77096006)(966005)(65806001)(6246003)(66066001)(33646002)(65956001)(67846002)(23676003)(189998001)(7416002)(5660300001)(54906003)(4326008)(86362001)(6666003)(18886065003);DIR:OUT;SFP:1101;SCL:1;SRVR:DB6PR0501MB2757;H:[10.15.7.185];FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtEQjZQUjA1MDFNQjI3NTc7MjM6dHVkV2YvV2diRm5YcHdIREpGd0F6bjdP?= =?utf-8?B?Z3JOanFuZW9oem52YWRjVm9XcWZlNStna0ZVaFgwdjB2WlhXL2V1SkE1MVRG?= =?utf-8?B?OFBFRFZ3SUwvSkRkZHVjYWFsR3B0eUE1LzQvSmcrQUFtSTNIOFR5aWtqTVcw?= =?utf-8?B?dzNGNWlnYVlaYk1PZG5TM3diSGpvWDB2Wk1Kc3M5czdSVXJxZ0lIOUlSR2Jv?= =?utf-8?B?RFhQallGS0hKVHh0bXZtRVE1WC81VWF5YlNiSTl2MzBqVUlTVzhDODR5UVVM?= =?utf-8?B?bXo4S280NW45Z2ExTFh1dXZNN1l3NUljYzN0M2JFdXc4WVpLUUtGRWRIc3NR?= =?utf-8?B?NDJmYk1yRndQUCtERGtCY2s4S3Z5UXZDTlpsdko0LzlvWHBudGRSQmpib2ZT?= =?utf-8?B?YUk4NUlYOUdZL1FScngzZHVBS0VZM0lmSGJWMkNKNGhUZVlkUGtyWkpteG10?= =?utf-8?B?ZDZoYjFnU1RTTjlqOHdGaUZuK05DRU5LNmNyVnNTN0NRcHFOYmZkMVc1YnU5?= =?utf-8?B?Y3l0RWxNZ1NMeGVsZXQ4VXY1d1BtWmxITmRyamlmUm1waEoveEFiakFBWDBx?= =?utf-8?B?eHgycG5uRW03VWM1WTJCbWNmanFoY1JoQ0V6KzZNMkdsWUVqc2ZVR29UbC9V?= =?utf-8?B?SmZPMFNYNnk0dzRNZzJ2MGNQQ3ZVc3BodndBQ2V5emJzRnhQS3dIR3NGYUti?= =?utf-8?B?enkrczFkNGt2emd6cnNlZW10QTVmSlZoL2VDNHppa0Zxb0dKU2xNYUUyd1Fx?= =?utf-8?B?andwNmhaUHA1NFVpclA0Z3Q5UlRPaFVSbDdRMzY2amYxYUkwQVRjc2Ixc3JJ?= =?utf-8?B?bXBFZVFZd2hQa3JjVVkrdDlJMlZHOHJhTjVoRVFSdUtYUUJFWmFDWExqeWVN?= =?utf-8?B?ZVFRdkJwYWVDbW5WYmJ1VEFOSEdDTUJlOXJDTGNzRnhpMDZ2ZHROVDU1am15?= =?utf-8?B?T0N2S2pEcXJLSTVRVFdodGh1bTBPZWlHYXJja21vNFI0N2RIcTRTUUcvV1hw?= =?utf-8?B?NFgrMGJoUTlGT2dmSkUwRXRjVWc2T2VNNnFoL0Z0RHd6R0dwbElrQTEwZncx?= =?utf-8?B?dHl5QkNnUzN6WTMwaTVqb3pTdkdqc29waTNBSUVnVFNQU2FUM0ljRWtTQUJU?= =?utf-8?B?M3dYajAzYXZVeStUakF4ckw5bDF0QWtNaCtzYjRRVTRPM0grMUFLNFIzWHlD?= =?utf-8?B?Q2M3UXFVQjN2dFFleW53dnAzUmx1bnYrU0xxc293UXcvTW9ENzNNR3ArNkpx?= =?utf-8?B?dE9LL2J2TXJiRXFOdnIrTTR0YktVSUV6Z240QmlPZC9MS1A3dGRhaWM5Zmkz?= =?utf-8?B?RlVLVk5kVFlKaWExVHdSM0VLR3FxMk5Hb2p3MU93NUhKeEJDTFkzRTVYVVlS?= =?utf-8?B?NnZES1B4bHZVMXQ0UGRSNC96cE5LYkV5TmxQMXVvV3UrRFdyY1czdURqSU1G?= =?utf-8?B?Wk5KaXF6ek0xUm1EZ0oxNUVFTzlnSWpDbXRYZTlocW1kZTdHcDA5MkVSNUgz?= =?utf-8?B?R3NtOW4weEE0VUlYQU9XdTlQT2E0QUQzK25reHVMTERUM2QzNzdxdm1ld0F4?= =?utf-8?B?ekNRdmNMK0FuTEtZTCtwWEpJTTJNa1F3Rk95M0JleGhZSWpXSndVeTMvb3pH?= =?utf-8?B?MjgwRGUrNGZ0NVNvOTNNeHAxdGk4ZVNHZGpCR2RoOHhuMytuSVZXYngvbTBa?= =?utf-8?B?d3RMSWFndUdjWHdIaytjdWpSdEMwWEpzYTJiSXU2eWhCd0hibWdhQkUvZ3ZX?= =?utf-8?B?U214elJzMHZvSkZRSUNEbHNBNktnUzRjRkIrOVdOZVhNTklFcW5SSUlsYVV4?= =?utf-8?B?RFA4L1BEQW93bHB1Rk14dmF0RDNmaUVsYnJuaFl2VGxMMVVXRW9uMUtZQUJp?= =?utf-8?B?S0xqUDJRK3hWZlJVdi83RmNMWmtBVG9IU1VucHFaaXhoUTJ1bkxNTGdadDFm?= =?utf-8?B?RTRibzBpak8ycEdJWkxxR05ZOFArSDNyZ082NXF1dkxBeTZxQzRmTEpCQzI5?= =?utf-8?B?czFUVlN2Wm5pQ29ZejlkM3R4SmcwMDRnNjFkdGkwUmx2b2tVamk3NHlHNlRB?= =?utf-8?Q?tbBZKg=3D?= X-Microsoft-Exchange-Diagnostics: 1;DB6PR0501MB2757;6:kAO5T3dSv4yanMKib5CrTRP6WKYEHa4fViH67a8VVFgakPh4HqNdg1w15pujSpeq8SDkfegj6MG7s5aWszM5AZnY5LvXLg3+9nthyrLvnPX1NflNmBueClv7kO7QMq1kjnRq4vkjkpRZWeEDWgzgA58/KrZ8OOdl+t+FSAdLGTBmzOZneL0TO656tDsiqdTv1N6ChIzPN/mAAgzie6J4uMAYv8kSm0CZdQrGs3cAYJya5mGT4tLgnvZWHn0/ubE1zp3a+yTeTpaS4JP72DbAhdtgptv/AxZ0ypJZQkWgrnQN/qhkEBRO9gIfjMmL/rBMyLGLG3yzWWmOGsG0ZpxkWJyzFr+X1ZAOnWwLhy4Em4A=;5:rMbyevb43d0TOz/SNUFpF4OW1EPExQGoeVJXnsh8PxW+0ipe0G2wK2TocXHZrVZuhHAH/TS8HRIpeeYt5YuAs0k1ZknXupzNP19y0EGZ8DZ7IQcJYz8Z2d29bM3d/UXtd6cxrs5kJWpJVTQ4naU/UI0pTLlnBW1qaUtDZtXE/nk=;24:Mgk+Jz+mlBLMk9B2pz9BmPFEJTmXFMTTK48n7HFavAyjhZmOlqKS5JJ0Q5efp5xXvfvNc0sj2B0MQV2XIZAijbo9l9dojp4OxBSxdsm10e4=;7:e/tZpzs9IkwK/YhpLcCHozm2qNTZB0Mkq3FlUiRT73eKGB0I94eBJSKOV7V2nVUX8+Y11hSfO9VfXSY10lEL9nPWjoKfM8Nw/U92jgQZ8aXuUuml3Tam6C3EtFgpTLodk4odVaPsOxFOBJ69Cv3YaAPsGqlupTfi6+SAqMIzNp8lJ45do3um95jkMwZZAxankdydnHIIM8tqUx6HIBc3/KfMTWmS6bk25n0LOSGvZFl//BcznHoSiNHokNYnGRSl SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2017 17:53:58.8111 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ce878d82-8d59-4d32-ff41-08d522e3dd73 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0501MB2757 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 11/3/2017 1:32 PM, Mark Rutland wrote: > Hi Chris, > > On Fri, Nov 03, 2017 at 01:04:48PM -0400, Chris Metcalf wrote: >> In do_notify_resume(), call task_isolation_start() for >> TIF_TASK_ISOLATION tasks. Add _TIF_TASK_ISOLATION to _TIF_WORK_MASK, >> and define a local NOTIFY_RESUME_LOOP_FLAGS to check in the loop, >> since we don't clear _TIF_TASK_ISOLATION in the loop. >> >> We tweak syscall_trace_enter() slightly to carry the "flags" >> value from current_thread_info()->flags for each of the tests, >> rather than doing a volatile read from memory for each one. This >> avoids a small overhead for each test, and in particular avoids >> that overhead for TIF_NOHZ when TASK_ISOLATION is not enabled. >> >> We instrument the smp_send_reschedule() routine so that it checks for >> isolated tasks and generates a suitable warning if needed. >> >> Finally, report on page faults in task-isolation processes in >> do_page_faults(). > I don't have much context for this (I only received patches 9, 10, and > 12), and this commit message doesn't help me to understand why these > changes are necessary. Sorry, I missed having you on the cover letter.  I'll fix that for the next spin. The cover letter (and rest of the series) is here: https://lkml.org/lkml/2017/11/3/589 The core piece of the patch is here: https://lkml.org/lkml/2017/11/3/598 > Here we add to _TIF_WORK_MASK... > [...] > ... and here we open-code the *old* _TIF_WORK_MASK. > > Can we drop both in , building one in terms of the > other: > > #define _TIF_WORK_NOISOLATION_MASK \ > (_TIF_NEED_RESCHED | _TIF_SIGPENDING | _TIF_NOTIFY_RESUME | \ > _TIF_FOREIGN_FPSTATE | _TIF_UPROBE | _TIF_FSCHECK) > > #define _TIF_WORK_MASK \ > (_TIF_WORK_NOISOLATION_MASK | _TIF_TASK_ISOLATION) > > ... that avoids duplication, ensuring the two are kept in sync, and > makes it a little easier to understand. We certainly could do that.  I based my approach on the x86 model, which defines _TIF_ALLWORK_MASK in thread_info.h, and then a local EXIT_TO_USERMODE_WORK_FLAGS above exit_to_usermode_loop(). If you'd prefer to avoid the duplication, perhaps names more like this? _TIF_WORK_LOOP_MASK (without TIF_TASK_ISOLATION) _TIF_WORK_MASK as _TIF_WORK_LOOP_MASK | _TIF_TASK_ISOLATION That keeps the names reflective of the function (entry only vs loop). >> @@ -818,6 +819,7 @@ void arch_send_call_function_single_ipi(int cpu) >> #ifdef CONFIG_ARM64_ACPI_PARKING_PROTOCOL >> void arch_send_wakeup_ipi_mask(const struct cpumask *mask) >> { >> + task_isolation_remote_cpumask(mask, "wakeup IPI"); > What exactly does this do? Is it some kind of a tracepoint? It is intended to generate a diagnostic for a remote task that is trying to run isolated from the kernel (NOHZ_FULL on steroids, more or less), if the kernel is about to interrupt it. Similarly, the task_isolation_interrupt() hooks are diagnostics for the current task.  The intent is that by hooking a little deeper in the call path, you get actionable diagnostics for processes that are about to be signalled because they have lost task isolation for some reason. >> @@ -495,6 +496,10 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, >> */ >> if (likely(!(fault & (VM_FAULT_ERROR | VM_FAULT_BADMAP | >> VM_FAULT_BADACCESS)))) { >> + /* No signal was generated, but notify task-isolation tasks. */ >> + if (user_mode(regs)) >> + task_isolation_interrupt("page fault at %#lx", addr); > What exactly does the task receive here? Are these strings ABI? > > Do we need to do this for *every* exception? The strings are diagnostic messages; the process itself just gets a SIGKILL (or user-defined signal if requested).  To provide better diagnosis we emit a log message that can be examined to see what exactly caused the signal to be generated. Thanks! -- Chris Metcalf, Mellanox Technologies http://www.mellanox.com