From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752861AbcBJSIZ (ORCPT ); Wed, 10 Feb 2016 13:08:25 -0500 Received: from mail-by2on0080.outbound.protection.outlook.com ([207.46.100.80]:35354 "EHLO na01-by2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751470AbcBJSIX (ORCPT ); Wed, 10 Feb 2016 13:08:23 -0500 Authentication-Results: arm.com; dkim=none (message not signed) header.d=none;arm.com; dmarc=none action=none header.from=caviumnetworks.com; Message-ID: <56BB7C91.5010205@caviumnetworks.com> Date: Wed, 10 Feb 2016 10:08:17 -0800 From: David Daney User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130625 Thunderbird/17.0.7 MIME-Version: 1.0 To: Will Deacon CC: David Daney , , Mark Rutland , Catalin Marinas , Marc Zyngier , , Andrew Pinski , David Daney Subject: Re: [PATCH] arm64: Add workaround for Cavium erratum 27456 References: <1455046156-10582-1-git-send-email-ddaney.cavm@gmail.com> <20160210092822.GA1052@arm.com> In-Reply-To: <20160210092822.GA1052@arm.com> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [64.2.3.194] X-ClientProxiedBy: SN1PR0701CA0045.namprd07.prod.outlook.com (25.163.126.13) To DM3PR07MB2138.namprd07.prod.outlook.com (25.164.4.144) X-Microsoft-Exchange-Diagnostics: 1;DM3PR07MB2138;2:ZDutR+hVuPF68YexVh38Musl1DxhXTtT8IpQjMdjmw9DkNTxKq0EJAdBeNPtJZMnCrlgCNrKdWCz4AovF3FAZYUcgqUKibVjLNDDrLySfq+Hrpw/egpuoEeaH2pkDDdUqWgP+w4aRyviLveTwwReQw==;3:un1aqTyhOSao7bsakosxK71tWaMSsNyM6Y/Rcf6itGH+aPnIhr5m+jaaizzf1RgEWCLofnZkX8z1V9UuSlJVWMqqv3WH3IWmujJvvDqCZ6enECe1eq0gkuqOPglgTjh+;25:dBpCKkXXlddkKuslTk1pN+miRxkSx79ICTkLs5vbF+SvhxRaCh14A0iLTaI905zOXBVDPJ9n7sMBbto0MBvmVnMDjBLrj/CpIzUEFYO3jOEBThcrh/aX/ElOuL9Qj/CtmcifQ3gNGk3LZzfUvySfb46cidyLHYnq5h5Cj/6LGYTsbs5gEmQwOFS3+e3/pkOnFdjd+cKd2TPDZK6HajYxI6/uB2fJmEw7ws46l4bdInj36C+Gb+vaYcwDXErKxuJWQR5mDEkXM1PWJdZSLd5aXI/s4cAPXgHrkNju6g8NhMP6kkSiF5sRDvCczmcmRIrl X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DM3PR07MB2138; X-MS-Office365-Filtering-Correlation-Id: 2de0c77f-9347-4053-b98a-08d332452960 X-Microsoft-Exchange-Diagnostics: 1;DM3PR07MB2138;20:eH6cV5976UjPQEhv7+RYL9sUIWLPmCq9KDYTeuJ20lyTJX7M15Uq0QoR1cD5/q8NGmvQ82H8O+thLo8FH+iqFWBCqKbLCboI6mEj0FnQQYuxMefzOE6nrF1wez9iDm46x+vLnoGiYMDVoU+M3dMzkPN1t/+kerqEWgLlp7maAhzQterQWJT6qKX9IoxbUquCgjc485liKiLGXyzD9B99gbnNM31wU8cKuFguLQ+WkgIWL34WZBbVnYXjSYKElnucOcd10h8bSJd7USGM9SYwg7llDRv5CtHt2ELH8pVw9LXkUvPnZApspzzNrg0EyXsPzFFAoUMjMTMa+E6hqvKMnKgmy7IJZ9KYZHdbWIFj0og+MD6gzpK4YAqjGETwyX1VP484z4WsgDOGWnmFUT9At2GwpmZKANMkdoMr99egqxSC2rukcmlUVq0H288pXAWvEdG8u8q7+BZV0HS66ErFvkJVbwPKGkzgKMeARgUb84sXrjqUrnR/IvFVGkPW+SjW4A2QeK4L6huefIcraDzHRhbjz7bVnPmMbr9iH+nq+3wLFFQB8m13lVNjpIq6eWvg8AAcG0GVnKarzdXt2qyjI/eQvQiDFgjU1UZivA3HE4U= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001);SRVR:DM3PR07MB2138;BCL:0;PCL:0;RULEID:;SRVR:DM3PR07MB2138; X-Microsoft-Exchange-Diagnostics: 1;DM3PR07MB2138;4:bNYUBT7o22gwzjUP3yZ6/CTxG0mGwEsFvDSCoW6AnaOBMfVGUvGY3sgzHMje5crUPalOXZs3tTAzLLEb0xlK4CCmuszv04ecufsiJxy67mr9+5g0tgZ16GDL6gjbG2Uu0YoeBYz1OounXso7afRzo0Ewv+DmSoNtST0g1FdYP6Q/8Flx0K1ZBGay1XOMkTj9gN+M/dnK1VY45BlLyD+FVDgtkFxZxWj/LIJq1djXnh1ti2B2h/JOIUtH5+V+iTVN6OGEvIFqKwEPpW5p7F97tjbxW8uOxVVc9g2+Hq+LZi4RutL9h6iEGkGPcc121qbASjFu8zlwecxTSD3wiZfkzdb4XI3tv97QVySCrQidwpgW9/s695cWr1m2wPFdpmuy X-Forefront-PRVS: 0848C1A6AA X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(6009001)(479174004)(377454003)(24454002)(50986999)(2950100001)(65816999)(76176999)(54356999)(83506001)(122386002)(50466002)(92566002)(77096005)(87976001)(40100003)(4326007)(19580395003)(19580405001)(189998001)(586003)(3846002)(1096002)(53416004)(47776003)(36756003)(23756003)(4001350100001)(110136002)(5001960100002)(5008740100001)(33656002)(5004730100002)(65956001)(230700001)(66066001)(42186005)(2906002);DIR:OUT;SFP:1101;SCL:1;SRVR:DM3PR07MB2138;H:dl.caveonetworks.com;FPR:;SPF:None;MLV:sfv;LANG:en; X-Microsoft-Exchange-Diagnostics: =?iso-8859-1?Q?1;DM3PR07MB2138;23:UvW4KVmDNoWFtZrr1M/yk51x03EdVTIZUQIvP8W?= =?iso-8859-1?Q?0S9dgYcv1PJ4PIaaFSzXQm69yLK7cU3rFZt/oix/fm3CbSL5r6bpBcSPv3?= =?iso-8859-1?Q?iKgPIEQIePPeMdT4X77k17/wHErOKcgdnPZ4chnMYKWeaBEoh9er/y5byE?= =?iso-8859-1?Q?z9Zqw7qh3f2hjhOVOlOMfmDM/VXzHdWUElsTv7E8Hie4j0LLgAFzh48q9o?= =?iso-8859-1?Q?QY0MI6Lz+8Oa8788QCcLWp+3cpF4LfsEq1wDxZSCb1U+ebx1NIBGBMn1vF?= =?iso-8859-1?Q?mxqWAWdvr6vm4vmDvqFJ6jVDnXsJNAdfBIMis54qrKyKpbrEK6AlXKRP1K?= =?iso-8859-1?Q?CAtVcfUFUE50OE6kbiyKDxjdoyzEYT62FN2m9o6Y4E+8Jcd7VUdPHs+eOP?= =?iso-8859-1?Q?3MBKE14PT3j7xw7asTL1jObmtyaOFbXVJTuZ8gAdLZrccOjdTeSVaWZTZQ?= =?iso-8859-1?Q?oOupQvty34GN6GlUWOphWG64nUzqawLIp5ArzZZiCdw9U4LWkbx/2hSOYS?= =?iso-8859-1?Q?zxw+mAOh+NNsDyzt9ZYX7DBzkNJ3P3XgE220ybTJD2za38QDBm5Yxc6ozO?= =?iso-8859-1?Q?n/psuMHfjnHVSXif+56h4/cZo5JVekpNJkaGQRIjQdOPq/MnJvxklUNjoe?= =?iso-8859-1?Q?gx3QA9ZffH1Mx2J0js6Xkvy4JWVOnVPdgr+R/HVa96mEBC3pM7t4HrCJqY?= =?iso-8859-1?Q?ygM0zue8PeBw+dcGdkoLMf0WJsSavwQpjSREPBvtwuWU7RGSakLkU67Dt3?= =?iso-8859-1?Q?sFofOyyh6H2rXBUu+3q+mdhbxRvnrH6wh/70ah72Ll9yEU8x7Hv66VA5Xk?= =?iso-8859-1?Q?FNKHUZ/M60Jq7qFecgazmro/WG6c4mzVfAxPgjjbsEII5kTxhNHHubATRC?= =?iso-8859-1?Q?n+E69AqhBOtodMyyxHWoCbchdNk7cCJefkRzai8IIqk9MCMOrZjBddVuNy?= =?iso-8859-1?Q?2Ij2nmDVT53DobKPEpiGZjgUlZw4T7s7SSc6i0N+C4Bv4P0M7Yzxp4ccuL?= =?iso-8859-1?Q?CHUIOSo3/81Rgn8lbZ7qqmCbLHKOn3AULYAI4xZAbQ70MEy5sLi/4RX0Xs?= =?iso-8859-1?Q?AhMlNAFcvQXyzS8EtFoHw=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;DM3PR07MB2138;5:xDNjzCKTbkMTb8Cox/UnbD+Rr/ih4lyqnv+INheiDqBnY+k7Z5UaGqQw8fMPEjA6Mq2XuMMfxkJKjrCtknP/R9X1iqV+IP2VuX/XlaOEYf2ugkvwKCLOdcb1gQW4XLeV0bLjRJdQ5R/31V07J8NW/Q==;24:R5bPiujiw+WqybacZgMchx7x2wrMu0QRnPXWNrpbC1eLM3G/4NZ7G0Fzq9Wk/Qs7ikMSVYnlnPdoBZHVqf9YhbiIPhY377Bu42pGyx4AN90= SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Feb 2016 18:08:20.9583 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM3PR07MB2138 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 02/10/2016 01:28 AM, Will Deacon wrote: > On Tue, Feb 09, 2016 at 11:29:16AM -0800, David Daney wrote: >> From: Andrew Pinski >> >> On ThunderX T88 pass 1.x through 2.1 parts, broadcast TLBI >> instructions may cause the icache to become invalid if it contains >> data for a non-current ASID. >> >> This patch implements the workaround (which flushes the local icache >> when switching the mm) by using code patching. > > So, to be clear, is this "just" a performance problem as opposed to a > correctness issue? No. It is a correctness issue. Without this workaround in place, userspace programs end up executing the wrong instructions, which leads to unpredictable behavior and program crashes. > If so, do you have any numbers with and without this > change? We tried to measure it, but the impact is not measurable in the tests we have done. Switching the mm is not often done so the extra ICache invalidation is rare. Also note that for the non-workaround case, the code path is unchanged. Since the following function (__cpu_setup()) is not on the hot path of anything, any extra ICache pressure from the three NOPs is unlikely to matter. > > Will > From mboxrd@z Thu Jan 1 00:00:00 1970 From: ddaney@caviumnetworks.com (David Daney) Date: Wed, 10 Feb 2016 10:08:17 -0800 Subject: [PATCH] arm64: Add workaround for Cavium erratum 27456 In-Reply-To: <20160210092822.GA1052@arm.com> References: <1455046156-10582-1-git-send-email-ddaney.cavm@gmail.com> <20160210092822.GA1052@arm.com> Message-ID: <56BB7C91.5010205@caviumnetworks.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On 02/10/2016 01:28 AM, Will Deacon wrote: > On Tue, Feb 09, 2016 at 11:29:16AM -0800, David Daney wrote: >> From: Andrew Pinski >> >> On ThunderX T88 pass 1.x through 2.1 parts, broadcast TLBI >> instructions may cause the icache to become invalid if it contains >> data for a non-current ASID. >> >> This patch implements the workaround (which flushes the local icache >> when switching the mm) by using code patching. > > So, to be clear, is this "just" a performance problem as opposed to a > correctness issue? No. It is a correctness issue. Without this workaround in place, userspace programs end up executing the wrong instructions, which leads to unpredictable behavior and program crashes. > If so, do you have any numbers with and without this > change? We tried to measure it, but the impact is not measurable in the tests we have done. Switching the mm is not often done so the extra ICache invalidation is rare. Also note that for the non-workaround case, the code path is unchanged. Since the following function (__cpu_setup()) is not on the hot path of anything, any extra ICache pressure from the three NOPs is unlikely to matter. > > Will >