From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00883C43462 for ; Wed, 12 May 2021 19:53:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C16D461407 for ; Wed, 12 May 2021 19:52:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1383009AbhELTtH (ORCPT ); Wed, 12 May 2021 15:49:07 -0400 Received: from mail.kernel.org ([198.145.29.99]:52302 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352814AbhELSER (ORCPT ); Wed, 12 May 2021 14:04:17 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id C02BF61412; Wed, 12 May 2021 18:03:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1620842588; bh=opB93W4AGOPK9xdlwOLmjCuWmfcA9Q7hj5tZrzDtRxE=; h=From:To:Cc:Subject:Date:From; b=nAbqnlTkae2+xI6GaNbC0GqbY4cqKoDg1TbM+DjiTG+eB8ie1/kzlfdLYBWsAH0JE 9CZi5BYk3eDRgFzr/ikh0CArcQ8em21oTRp6nxPNGfejvZL4uYJBLe6i9tilbdHqyQ XnL4I5petiyvWsnGFyHHFEB01QKnvdwslS80qEnXduUFv1MGBJEly+tDUyJokeIZ4k xm9uS8njUl6yHYqmhlqjpBZmvupoqeplQNQgAs9rovC0UdD/m2eIPiViP0ZRAR712l HYSKDUfJn0+nOsbwaBfHueqSlaj4HGpbNG092xnfpYCbeFgz2MP8brJ5xQd7+FmH1T I6lwhsF8Ko1Hw== From: Sasha Levin To: linux-kernel@vger.kernel.org, stable@vger.kernel.org Cc: Ard Biesheuvel , Nicolas Pitre , Russell King , Sasha Levin , linux-arm-kernel@lists.infradead.org Subject: [PATCH AUTOSEL 5.10 01/34] ARM: 9058/1: cache-v7: refactor v7_invalidate_l1 to avoid clobbering r5/r6 Date: Wed, 12 May 2021 14:02:32 -0400 Message-Id: <20210512180306.664925-1-sashal@kernel.org> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-stable: review X-Patchwork-Hint: Ignore Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Ard Biesheuvel [ Upstream commit f9e7a99fb6b86aa6a00e53b34ee6973840e005aa ] The cache invalidation code in v7_invalidate_l1 can be tweaked to re-read the associativity from CCSIDR, and keep the way identifier component in a single register that is assigned in the outer loop. This way, we need 2 registers less. Given that the number of sets is typically much larger than the associativity, rearrange the code so that the outer loop has the fewer number of iterations, ensuring that the re-read of CCSIDR only occurs a handful of times in practice. Fix the whitespace while at it, and update the comment to indicate that this code is no longer a clone of anything else. Acked-by: Nicolas Pitre Signed-off-by: Ard Biesheuvel Signed-off-by: Russell King Signed-off-by: Sasha Levin --- arch/arm/mm/cache-v7.S | 51 +++++++++++++++++++++--------------------- 1 file changed, 25 insertions(+), 26 deletions(-) diff --git a/arch/arm/mm/cache-v7.S b/arch/arm/mm/cache-v7.S index dc8f152f3556..e3bc1d6e13d0 100644 --- a/arch/arm/mm/cache-v7.S +++ b/arch/arm/mm/cache-v7.S @@ -33,41 +33,40 @@ icache_size: * processor. We fix this by performing an invalidate, rather than a * clean + invalidate, before jumping into the kernel. * - * This function is cloned from arch/arm/mach-tegra/headsmp.S, and needs - * to be called for both secondary cores startup and primary core resume - * procedures. + * This function needs to be called for both secondary cores startup and + * primary core resume procedures. */ ENTRY(v7_invalidate_l1) mov r0, #0 mcr p15, 2, r0, c0, c0, 0 mrc p15, 1, r0, c0, c0, 0 - movw r1, #0x7fff - and r2, r1, r0, lsr #13 + movw r3, #0x3ff + and r3, r3, r0, lsr #3 @ 'Associativity' in CCSIDR[12:3] + clz r1, r3 @ WayShift + mov r2, #1 + mov r3, r3, lsl r1 @ NumWays-1 shifted into bits [31:...] + movs r1, r2, lsl r1 @ #1 shifted left by same amount + moveq r1, #1 @ r1 needs value > 0 even if only 1 way - movw r1, #0x3ff + and r2, r0, #0x7 + add r2, r2, #4 @ SetShift - and r3, r1, r0, lsr #3 @ NumWays - 1 - add r2, r2, #1 @ NumSets +1: movw r4, #0x7fff + and r0, r4, r0, lsr #13 @ 'NumSets' in CCSIDR[27:13] - and r0, r0, #0x7 - add r0, r0, #4 @ SetShift - - clz r1, r3 @ WayShift - add r4, r3, #1 @ NumWays -1: sub r2, r2, #1 @ NumSets-- - mov r3, r4 @ Temp = NumWays -2: subs r3, r3, #1 @ Temp-- - mov r5, r3, lsl r1 - mov r6, r2, lsl r0 - orr r5, r5, r6 @ Reg = (Temp< X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6411CC33CBB for ; Wed, 12 May 2021 18:07:00 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 406F661289 for ; Wed, 12 May 2021 18:06:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 406F661289 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=5KGuYoRj4Cy9R037oo7RJbZ+Ywt7tsMecBkmrIADDQU=; b=GK8W4Lm3h1tRlQjhVk7rotfm5o 3RoLY7DHKgxI3aVsY6hHiJCupHwTnF70lw2MlaMt0NpJTCPri3/ZCUx20A28Xfml6pH8meS60v47b 8H4Cw/wBLQNMxnABoGYBUP2MwqVxlnUu3JDdhwROhkEVu18QiIAMjLsxf1l8I0W5lTH8f0q03QcY+ hGmt0mXBckfodkJndCiqiEsNnJER49P+loEWCVux44agbt2SlHkwXYS5vO8BxXaBitp89kJv9Y1kR sFPuq5eYhieIHnmi+3TpBYE4tnupcyOF5gnUmC8piqKplqgUgXm0jFyTrfIM1dMtMWaNuQ9Olc2U8 17ZT0DKQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lgtEW-003Wwp-Sx; Wed, 12 May 2021 18:05:21 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgtCS-003Vy4-1V for linux-arm-kernel@desiato.infradead.org; Wed, 12 May 2021 18:03:12 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type: Content-ID:Content-Description:In-Reply-To:References; bh=h4fQnTql0IubEHEd6gRU16QnFpa87XYzgkIiNt+GmyA=; b=TAL3ZJmJl8zfuRGEIk2bpgUsao 1Jxu9eYBs3UKu5jJIurb6BXXlGXZU1FZqZuE/DaELGPsUuCng4xtIDaL9/jQUWoCvqQX0lKj6Cy6y dGZk3Q/DQEheyBhqN4qRKPLs/nY0Vqr63Or4rULUgHWGuG7CItUGYJTzrXo5MrGQiN/yaJWqUxR1p Iuty2vXYHVaX1Eynmue0+thNTqz6/7voGtIqVFk/0ivpjEnPFnjsiMKb9X9QIG3UHNpu/hjZWrD+h d/OArDYCwOi/pJOcuSGkh/nLrKh76do/o0/XJgHUOio+5w2Tllmg8wxF30Nkz8iBKsD0dcJBp86Es AeeUgAqg==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgtCP-00Afup-47 for linux-arm-kernel@lists.infradead.org; Wed, 12 May 2021 18:03:10 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id C02BF61412; Wed, 12 May 2021 18:03:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1620842588; bh=opB93W4AGOPK9xdlwOLmjCuWmfcA9Q7hj5tZrzDtRxE=; h=From:To:Cc:Subject:Date:From; b=nAbqnlTkae2+xI6GaNbC0GqbY4cqKoDg1TbM+DjiTG+eB8ie1/kzlfdLYBWsAH0JE 9CZi5BYk3eDRgFzr/ikh0CArcQ8em21oTRp6nxPNGfejvZL4uYJBLe6i9tilbdHqyQ XnL4I5petiyvWsnGFyHHFEB01QKnvdwslS80qEnXduUFv1MGBJEly+tDUyJokeIZ4k xm9uS8njUl6yHYqmhlqjpBZmvupoqeplQNQgAs9rovC0UdD/m2eIPiViP0ZRAR712l HYSKDUfJn0+nOsbwaBfHueqSlaj4HGpbNG092xnfpYCbeFgz2MP8brJ5xQd7+FmH1T I6lwhsF8Ko1Hw== From: Sasha Levin To: linux-kernel@vger.kernel.org, stable@vger.kernel.org Cc: Ard Biesheuvel , Nicolas Pitre , Russell King , Sasha Levin , linux-arm-kernel@lists.infradead.org Subject: [PATCH AUTOSEL 5.10 01/34] ARM: 9058/1: cache-v7: refactor v7_invalidate_l1 to avoid clobbering r5/r6 Date: Wed, 12 May 2021 14:02:32 -0400 Message-Id: <20210512180306.664925-1-sashal@kernel.org> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-stable: review X-Patchwork-Hint: Ignore X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210512_110309_479593_F5AAE605 X-CRM114-Status: GOOD ( 14.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel [ Upstream commit f9e7a99fb6b86aa6a00e53b34ee6973840e005aa ] The cache invalidation code in v7_invalidate_l1 can be tweaked to re-read the associativity from CCSIDR, and keep the way identifier component in a single register that is assigned in the outer loop. This way, we need 2 registers less. Given that the number of sets is typically much larger than the associativity, rearrange the code so that the outer loop has the fewer number of iterations, ensuring that the re-read of CCSIDR only occurs a handful of times in practice. Fix the whitespace while at it, and update the comment to indicate that this code is no longer a clone of anything else. Acked-by: Nicolas Pitre Signed-off-by: Ard Biesheuvel Signed-off-by: Russell King Signed-off-by: Sasha Levin --- arch/arm/mm/cache-v7.S | 51 +++++++++++++++++++++--------------------- 1 file changed, 25 insertions(+), 26 deletions(-) diff --git a/arch/arm/mm/cache-v7.S b/arch/arm/mm/cache-v7.S index dc8f152f3556..e3bc1d6e13d0 100644 --- a/arch/arm/mm/cache-v7.S +++ b/arch/arm/mm/cache-v7.S @@ -33,41 +33,40 @@ icache_size: * processor. We fix this by performing an invalidate, rather than a * clean + invalidate, before jumping into the kernel. * - * This function is cloned from arch/arm/mach-tegra/headsmp.S, and needs - * to be called for both secondary cores startup and primary core resume - * procedures. + * This function needs to be called for both secondary cores startup and + * primary core resume procedures. */ ENTRY(v7_invalidate_l1) mov r0, #0 mcr p15, 2, r0, c0, c0, 0 mrc p15, 1, r0, c0, c0, 0 - movw r1, #0x7fff - and r2, r1, r0, lsr #13 + movw r3, #0x3ff + and r3, r3, r0, lsr #3 @ 'Associativity' in CCSIDR[12:3] + clz r1, r3 @ WayShift + mov r2, #1 + mov r3, r3, lsl r1 @ NumWays-1 shifted into bits [31:...] + movs r1, r2, lsl r1 @ #1 shifted left by same amount + moveq r1, #1 @ r1 needs value > 0 even if only 1 way - movw r1, #0x3ff + and r2, r0, #0x7 + add r2, r2, #4 @ SetShift - and r3, r1, r0, lsr #3 @ NumWays - 1 - add r2, r2, #1 @ NumSets +1: movw r4, #0x7fff + and r0, r4, r0, lsr #13 @ 'NumSets' in CCSIDR[27:13] - and r0, r0, #0x7 - add r0, r0, #4 @ SetShift - - clz r1, r3 @ WayShift - add r4, r3, #1 @ NumWays -1: sub r2, r2, #1 @ NumSets-- - mov r3, r4 @ Temp = NumWays -2: subs r3, r3, #1 @ Temp-- - mov r5, r3, lsl r1 - mov r6, r2, lsl r0 - orr r5, r5, r6 @ Reg = (Temp<