From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3E17C4363C for ; Sun, 4 Oct 2020 14:47:28 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F11732067D for ; Sun, 4 Oct 2020 14:47:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="w5cc5fE0"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=marvell.com header.i=@marvell.com header.b="Cdu/99j/"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.b="MikUmlJf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F11732067D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=marvell.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Content-ID:In-Reply-To:References: Message-ID:Date:Subject:To:From:Reply-To:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=35u4VOYtQBgbP36aeK3i9TcfqTTi6b0kG8O97HGYvRs=; b=w5cc5fE0zNUxRDMOEGaURi4r2 WvYhGhe2aDmIO2Ct6188eTOOtBDypK3CxlJq2yFhYcp6ECmdJ/gfAUU8nV0d0nd+bc/WEY+YDecc8 vYdJh5HY3As58OpEqvR7GMynTsXWQAMHNMB3GvKRyy5JpQTOG3C9r57WhjQtPJ153bPYbTfswGMCI j2ozzPF1IyN/sqPGCQv/vTn60BcWBNXKFRSGlBjKhZZZ8VJwl0bwY9Z+SMwuNmNDXUCfhFqVWOdzK oZzJdh65cTG47KOmCjDEdzugzkD3snsgSPbMNeXYH0EANrLe0llOFq0IHO1VLN3mzhXwkZykCOQBS ukeHCmnIA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kP5Gf-0004cg-At; Sun, 04 Oct 2020 14:45:41 +0000 Received: from mx0a-0016f401.pphosted.com ([67.231.148.174] helo=mx0b-0016f401.pphosted.com) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kP5Gc-0004c9-CQ for linux-arm-kernel@lists.infradead.org; Sun, 04 Oct 2020 14:45:39 +0000 Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 094EaQx8005357; Sun, 4 Oct 2020 07:44:44 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0220; bh=BTvvMP4towAAuGQAjxTkThyAC9pveBzp1UXXXgEwLfk=; b=Cdu/99j/wketvDCnfTEVmXfN+/k/SefreTU9375zmwUdd2FXMFqlQbQymVdTZxsMUCgT xPVH3xaVOEkro4XMPbdNvs717usgihNzIAZpwpUHmYmnrMRtqLDheBRCwVK9vbSrwfoJ sQ2GmHNB3OeoFice+EOaSamfc1vkDp7RgzqDlxd0zivk/9Gc4jPwdSAQ2NQpDLIkR1FM OPhFKu9dgdY/RB9neJXvf4KxxN8R8p/D3yv+JLynmHZw4djlLUxFPAj45EfrM6BhX2RI X421GjVOSOIwedv4iNHnRmFvNuIMvEABNuKdF2r2aPabXTKV1Z/+WDujyy0B6UBSjPWa ig== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0a-0016f401.pphosted.com with ESMTP id 33xpnpjqv8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 04 Oct 2020 07:44:43 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sun, 4 Oct 2020 07:44:42 -0700 Received: from SC-EXCH03.marvell.com (10.93.176.83) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sun, 4 Oct 2020 07:44:41 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.168) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Sun, 4 Oct 2020 07:44:41 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Q/w+8VvX0SmEz0Tc014WXEH4mF8FOvvkZCyCCAZAIrdhazXIa9iBF2ovgxGE9etbdxSONjLr9Y5+NBIFzSHP/B7fjt2w0ZFVxTCdk2tJa97/DAWTiRM82ROb4+KYlZQNiJQZTp1vE/rP8AvN5q1/DF71WWJvKVzKaLWGhX+QXcxiLAM7lXRVRf+5DpsM/6lXBbv742HxNBl7fu7j1wcovPw2NqRlVPkJUgvS910oDCs+khFz4WOCS+fDA7ZPlBhb4f6StHsOwo3YD2IUWkEBjchgaORpLViwguDzYxOmfbG0fLwmr8b61MUobD2eDRh6BKyz1Xe0AOBKJq1BZg2NZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BTvvMP4towAAuGQAjxTkThyAC9pveBzp1UXXXgEwLfk=; b=WkRtm/vadj2Pdh1r2EEZKMMJ3XDBbieTUKfOYSFXnC3wYxn9rrRAk7bnSdDek+qakKvJl90Ba9MBoDiZMWzTXq6qwXChUVHfEP/7fErHZ/ZmgGnGYAHkVPWr0s6J9z2j/DXNMY2P3k4DjuIxuHaQtj28U1uiFfEW5zoYrpDpkQL9m/yVVd/KHz2ASCWz5VeD9DME/9ty0AZC3EqC8BFcqwUXrqDkbSoSMA2BnTskypOBUVwAGvosYrIW0Q3WUZ0M1bMBrXFELMJrqU9hBAMHylbao2qvs456ktFDhA2AtJsn/1NtikkBBE72BN9VxogDaJOAFoHny+48JsSMoiBJTA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BTvvMP4towAAuGQAjxTkThyAC9pveBzp1UXXXgEwLfk=; b=MikUmlJfHmNy9u9801jOg1VRf9saSNC4XfzJjPopUYlbPUVxe2hktNhf40wTXQ0xiCUuEesN6zZ9A0RPR8nViWeXZgPDjfcF0OT0LE1X6QzgA2Eq7qNLp5qrTNibl+j4SPF4EGXrNvpDcmvun3PqMHBrE5fZ/O9SM2D58WFaQq8= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MW2PR18MB2089.namprd18.prod.outlook.com (2603:10b6:907:10::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3433.39; Sun, 4 Oct 2020 14:44:39 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::54a1:710f:41a5:c76f]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::54a1:710f:41a5:c76f%5]) with mapi id 15.20.3433.043; Sun, 4 Oct 2020 14:44:39 +0000 From: Alex Belits To: "frederic@kernel.org" Subject: Re: [EXT] Re: [PATCH v4 03/13] task_isolation: userspace hard isolation from kernel Thread-Topic: [EXT] Re: [PATCH v4 03/13] task_isolation: userspace hard isolation from kernel Thread-Index: AQHWYDdZDqoeqvfF50CzG3l43t/UoamDNPAAgATEYwA= Date: Sun, 4 Oct 2020 14:44:39 +0000 Message-ID: <7e54b3c5e0d4c91eb64f2dd1583dd687bc34757e.camel@marvell.com> References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> <20201001135640.GA1748@lothringen> In-Reply-To: <20201001135640.GA1748@lothringen> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 71ebe096-584a-4721-7200-08d8687405ce x-ms-traffictypediagnostic: MW2PR18MB2089: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: kTtvDKKcdYaiaN8OhZSckNVTngMLjHY+nZE7KY6Y8Gn6Q/V23QjN0ZAUIXG1YO+5mKacJDEzDlnNfRgGFVNBAi0M16HTHoVLn1T4OZHn7ecxwk5cZ9Dr0rjsl/aPyMIUea+dMoCCTpRPF8G5MlIYPvl6GkupitCeMCjoWR63EZy0RT2e7RoxUMwclForVRvwSXEn77i1OvcoesnTCWiSFZ2qtjJSchqMOJIUU85lu6vBz/TvD0w+b8YukMgeGWT9EKDw644cuoCfMfXwPXj3rJNi53A82Iiww3qv90qiN4nUCo3c1VboSclu0/wcCF4kLgYFedP+vNAi14jXN9m6mQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MW2PR18MB2267.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(346002)(396003)(136003)(366004)(39850400004)(86362001)(71200400001)(83380400001)(2906002)(6506007)(8676002)(7416002)(36756003)(66476007)(54906003)(26005)(66446008)(64756008)(66556008)(8936002)(2616005)(6916009)(186003)(91956017)(76116006)(66946007)(6486002)(5660300002)(4326008)(478600001)(316002)(6512007); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: 8lDm45uqhgzqoCh43Hp1TVza0tHL09EbWv2SBlXrG+e+fSuf+fk4Y3AFUN8j3Mne2+3+pRq41iDyNH2elDQ/Qbb16tSQWad6U22+WGZ4vOgeSunbOQbo8WTfgYMjCtJVPdxWTn9Z60I3nnnWPystRxEnd71t6EGak+Q/U55qjnabh7Ifvn7EugDiutWpqxAbtcXFXixMFy9N6CNRdiSx25TpHXIATrKxnyHs+tX0lWLJFBQmGgwCCR0k5ZE4q3ex85pa3NxAZk6ZCfmdkkPBv7E2fxyDTrc8CEXaB6GhkHYsPCt0FWYFU53JDROITqJi6L25gCwJlwA95FQiZrLnAKaKz5lwVAdeMgs8QGI7cFtFjcZomcRDiSgKhJe5aXrU/HVd2J1p5owpBvZ0iVCYIbRiJXeIW5Vy3mkf8+bgyYewDHMxkLBy51Zaa8cch2gktfpwFe2xmxVTfY9ovVSC3zHunbBS++3VsEtOJE2VaBNee68LYhuj8lB0cnvCxIPnuoFXVkgh3ZxZl75SSoInPSuiCzsNzplajz2Mw3Ygg4F1swwQYPYyQ706dBVz841dKVdPLi3Mrt/DQ43mCERq8h6jD3fIAWkMX2TtxRzPj6tAW3Ot/dQkJsPv4J6hM2AzpoaURSxfBqm3n2mKBKUMzA== Content-ID: MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 71ebe096-584a-4721-7200-08d8687405ce X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Oct 2020 14:44:39.1938 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: dH3PGOl6j/ZcVgOrIZNX+NWpW3ETXha6S8FQDYsNUfxtH5UXLD938s8CBjhylcufAzdqg8tiRgjIl+Nqm8bzsw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR18MB2089 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-10-04_13:2020-10-02, 2020-10-04 signatures=0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201004_104538_576154_37687C8D X-CRM114-Status: GOOD ( 35.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "linux-arch@vger.kernel.org" , "peterz@infradead.org" , "linux-api@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "rostedt@goodmis.org" , "davem@davemloft.net" , "netdev@vger.kernel.org" , "catalin.marinas@arm.com" , Prasun Kapoor , "tglx@linutronix.de" , "will@kernel.org" , "mingo@kernel.org" , "linux-arm-kernel@lists.infradead.org" Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, 2020-10-01 at 15:56 +0200, Frederic Weisbecker wrote: > External Email > > ------------------------------------------------------------------- > --- > On Wed, Jul 22, 2020 at 02:49:49PM +0000, Alex Belits wrote: > > +/* > > + * Description of the last two tasks that ran isolated on a given > > CPU. > > + * This is intended only for messages about isolation breaking. We > > + * don't want any references to actual task while accessing this > > from > > + * CPU that caused isolation breaking -- we know nothing about > > timing > > + * and don't want to use locking or RCU. > > + */ > > +struct isol_task_desc { > > + atomic_t curr_index; > > + atomic_t curr_index_wr; > > + bool warned[2]; > > + pid_t pid[2]; > > + pid_t tgid[2]; > > + char comm[2][TASK_COMM_LEN]; > > +}; > > +static DEFINE_PER_CPU(struct isol_task_desc, isol_task_descs); > > So that's quite a huge patch that would have needed to be split up. > Especially this tracing engine. > > Speaking of which, I agree with Thomas that it's unnecessary. It's > too much > code and complexity. We can use the existing trace events and perform > the > analysis from userspace to find the source of the disturbance. The idea behind this is that isolation breaking events are supposed to be known to the applications while applications run normally, and they should not require any analysis or human intervention to be handled. A process may exit isolation because some leftover delayed work, for example, a timer or a workqueue, is still present on a CPU, or because a page fault or some other exception, normally handled silently, is caused by the task. It is also possible to direct an interrupt to a CPU that is running an isolated task -- currently it's perfectly valid to set interrupt smp affinity to a CPU running isolated task, and then interrupt will cause breaking isolation. While it's probably not the best way of handling interrupts, I would rather not prohibit this explicitly. There is also a matter of avoiding race conditions on entering isolation. Once CPU entered isolation, other CPUs should avoid disturbing it when they know that CPU is running a task in isolated mode. However for a short time after entering isolation other CPUs may be unaware of this, and will still send IPIs to it. Preventing this scenario completely would be very costly in terms of what other CPUs will have to do before notifying others, so similar to how EINTR works, we can simply specify that this is allowed, and task is supposed to re- enter isolation after this. It's still a bad idea to specify that isolation breaking can continue happening while application is running in isolated mode, however allowing some "grace period" after entering is acceptable as long as application is aware of this happening. In libtmc I have moved this handling of isolation breaking into a separate thread, intended to become a separate daemon if necessary. In part it was done because initial implementation of isolation made it very difficult to avoid repeating delayed work on isolated CPUs, so something had to watch for it from non-isolated CPU. It's possible that now, when delayed work does not appear on isolated CPUs out of nowhere, the need in isolation manager thread will disappear, and task itself will be able to handle all isolation breaking, like original implementation by Chris was supposed to. However in either case it's still useful for the task, or isolation manager, to get a description of the isolation-breaking event. This is what those things are intended for. Now they only produce log messages because this is where initially all description of isolation-breaking events went, however I would prefer to make logging optional but always let applications read those events descriptions, regardless of any tracing mechanism being used. I was more focused on making the reporting mechanism properly detect the cause of isolation breaking because that functionality was not quite working in earlier work by Chris and Yuri, so I have kept logging as the only output, but made it suitable for producing events that applications will be able to receive. Application, or isolation manager, will receive clear and unambiguous reporting, so there will be no need for any additional analysis or guesswork. After adding a proper "low-level" isolation flags, I got the idea that we might have a better yet reporting mechanism. Early isolation breaking detection on kernel entry may set a flag that says that isolation breaking happened, however its cause is unknown. Or, more likely, only some general information about isolation breaking is available, like a type of exception. Then, once a known isolation- breaking reporting mechanism is called from interrupt, syscall, IPI or exception processing, the flag is cleared, and reporting is supposed to be done. However if then kernel returns to userspace on isolated task but isolation breaking is not reported yet, an isolation breaking reporting with "unknown cause" will happen. We may even add some more optional lightweight tracing for debugging purposes, however the fact that reporting will be done, will allow us to make sure that no matter how complicated exception processing is, or how we managed to miss some subtle details of an architecture where we are implementing task isolation, there will be a reliable way to tell the user that something is wrong, and tell the task that there is something it has to react to. > > Thanks. > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel