From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F68DC433E0 for ; Mon, 8 Mar 2021 16:55:03 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 57AA9650DF for ; Mon, 8 Mar 2021 16:55:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 57AA9650DF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:Cc:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=PqXMGHN4YfidROFB+ZlCPlU57zhHNz9VuVW/PzU5LjY=; b=ag63s7obyGa1RKM4EGoCD/ddd T2ZMMdufeVh25Vo3MJrByh81IC4LZze8wDBomvBqAN2/hZzZkOfG+nXjSM9BJbgRdvoL7VaOCIdwl fnJQjCmdyb9QNZ4GPd5YBQCSDF2SV33gQah3YX02XwDF5ORO/nwug9Xbfk+2LHZ+hK8cmqqA7XdiU T3fU2Qb/wl4TZUZc02Twm+CfTfxuhwpHuWPMWSsFJnR8lAqYuVMbuIIk5b98Fv0ob3RsEuBD8/eJL 8eBTHQM63Mm96P2kUeRUDF15jBvLxiLZk3UsGBgQSXYWv0JAYR49U6oZStzbyjji5Wjfv8U4QTXEw c851VSjpg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lJJ8M-001BuA-PA; Mon, 08 Mar 2021 16:53:31 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lJJ8K-001BtJ-AK for linux-arm-kernel@desiato.infradead.org; Mon, 08 Mar 2021 16:53:28 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:Content-Type: In-Reply-To:MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender :Reply-To:Content-ID:Content-Description; bh=HMrKdmWybd8MCU7qOnocmzupx3KLmdTTzJbNXOBZU2M=; b=OGwiL6si1MjChfxyT/ZtRPXB+p qo8t/3CmC3DPzXlYI5s0wPKuwFMY/Mbcad5qQpgzugyYk+gXRKlodi7mhf2hVBzEYrpR6SUDnL5EY YFb40bqHLLmSBZS1SfWdif8nu65zYOtUMhmWEdJtbhIKPFkbMOiTrx7Cfg6SsN7LA2bW1Nc/jBuCs a752TiY1KyTLt5+j8Vjsh3JIDP6FxUyKDeeHgBVsY+1q8pvz43s+MskGwYsai1CEvGx4bBZ+Su0Gn SFNwlK6uulfK+gihG9NxxzR0W9mKnDh6HJg/HR7IKeHnZOkfElwqboa+Sz7xorc8AiSkNukbe9rKP hN6JrmfQ==; Received: from foss.arm.com ([217.140.110.172]) by casper.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lJJ7p-00Fihh-Ci for linux-arm-kernel@lists.infradead.org; Mon, 08 Mar 2021 16:53:16 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 25D24D6E; Mon, 8 Mar 2021 08:52:50 -0800 (PST) Received: from [192.168.0.110] (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A83C93F71B; Mon, 8 Mar 2021 08:52:48 -0800 (PST) Subject: Re: [PATCH] KVM: arm64: Ensure I-cache isolation between vcpus of a same VM To: Catalin Marinas , Marc Zyngier Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, kernel-team@android.com, James Morse , Julien Thierry , Suzuki K Poulose , Will Deacon , Mark Rutland References: <20210303164505.68492-1-maz@kernel.org> <20210305190708.GL23855@arm.com> <877dmksgaw.wl-maz@kernel.org> <20210306141546.GB2932@arm.com> From: Alexandru Elisei Message-ID: Date: Mon, 8 Mar 2021 16:53:09 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: <20210306141546.GB2932@arm.com> Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210308_165307_028784_988C1530 X-CRM114-Status: GOOD ( 24.88 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hello, It's not clear to me why this patch is needed. If one VCPU in the VM is generating code, is it not the software running in the VM responsible for keeping track of the MMU state of the other VCPUs and making sure the new code is executed correctly? Why should KVM get involved? I don't see how this is different than running on bare metal (no hypervisor), and one CPU with the MMU on generates code that another CPU with the MMU off must execute. Some comments below. On 3/6/21 2:15 PM, Catalin Marinas wrote: > On Sat, Mar 06, 2021 at 10:54:47AM +0000, Marc Zyngier wrote: >> On Fri, 05 Mar 2021 19:07:09 +0000, >> Catalin Marinas wrote: >>> On Wed, Mar 03, 2021 at 04:45:05PM +0000, Marc Zyngier wrote: >>>> It recently became apparent that the ARMv8 architecture has interesting >>>> rules regarding attributes being used when fetching instructions >>>> if the MMU is off at Stage-1. >>>> >>>> In this situation, the CPU is allowed to fetch from the PoC and >>>> allocate into the I-cache (unless the memory is mapped with >>>> the XN attribute at Stage-2). >>> Digging through the ARM ARM is hard. Do we have this behaviour with FWB >>> as well? >> The ARM ARM doesn't seem to mention FWB at all when it comes to >> instruction fetch, which is sort of expected as it only covers the >> D-side. I *think* we could sidestep this when CTR_EL0.DIC is set >> though, as the I-side would then snoop the D-side. > Not sure this helps. CTR_EL0.DIC refers to the need for maintenance to > PoU while the SCTLR_EL1.M == 0 causes the I-cache to fetch from PoC. I > don't think I-cache snooping the D-cache would happen to the PoU when > the S1 MMU is off. FEAT_FWB requires that CLIDR_EL1.{LoUIS, LoUU} = {0, 0} which means that no dcache clean is required for instruction to data coherence (page D13-3086). I interpret that as saying that with FEAT_FWB, CTR_EL0.IDC is effectively 1, which means that dcache clean is not required for instruction generation, and icache invalidation is required only if CTR_EL0.DIC = 0 (according to B2-158). > My reading of D4.4.4 is that when SCTLR_EL1.M == 0 both I and D accesses > are Normal Non-cacheable with a note in D4.4.6 that Non-cacheable > accesses may be held in the I-cache. Nitpicking, but SCTLR_EL1.M == 0 and SCTLR_EL1.I == 1 means that instruction fetches are to Normal Cacheable, Inner and Outer Read-Allocate memory (ARM DDI 0487G.a, pages D5-2709 and indirectly at D13-3586). Like you've pointed out, as mentioned in D4.4.6, it is always possible that instruction fetches are held in the instruction cache, regardless of the state of the SCTLR_EL1.M bit. > The FWB rules on combining S1 and S2 says that Normal Non-cacheable at > S1 is "upgraded" to cacheable. This should happen irrespective of > whether the S1 MMU is on or off and should apply to both I and D > accesses (since it does not explicitly says). So I think we could skip > this IC IALLU when FWB is present. > > The same logic should apply when the VMM copies the VM text. With FWB, > we probably only need D-cache maintenance to PoU and only if > CTR_EL0.IDC==0. I haven't checked what the code currently does. When FEAT_FWB, CTR_EL0.IDC is effectively 1 (see above), so we don't need a dcache clean in this case. Thanks, Alex _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel