From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89741C4338F for ; Tue, 3 Aug 2021 15:58:25 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 512A660F45 for ; Tue, 3 Aug 2021 15:58:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 512A660F45 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To:References: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=zAeRKUICUp1+CY+bdz3NtClIV2JcSkFCO179pX567Fc=; b=yDFKZB8imaU2TX RA2pJ014yW2qH68Jz7Ro5Tu7DL704C4f5WoUtJ2kgNOE6CnAzE7ZBwvtwZwal0Z6NBQTOaao9JyKg GB3XRRgTOqkR3O+8jAAkPKWPO9jvlcyOXkY9uDdlRlqFGseCJ4L+Zv+kCCxBw/yBBt3LhbmnWuQl5 vkFmYXEkPGBhMO0Q0ts2pzYMWsU4gqtCtf6FogCEflTArTyYKWssk5jVWsdbxwESYgGp1GTnFn6KN FqIqpPOyQ9yD/7t+nAK3NMyhaHORYRFd4kgagwhAEENT8uzc5k6xuewnIhFBlbCZsZo5sIUCL3IBc 01rWcbRVhQtRIyK8rISg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAwmF-003U1N-2U; Tue, 03 Aug 2021 15:56:23 +0000 Received: from szxga06-in.huawei.com ([45.249.212.32]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAwm7-003TxX-Ri for linux-arm-kernel@lists.infradead.org; Tue, 03 Aug 2021 15:56:18 +0000 Received: from dggems703-chm.china.huawei.com (unknown [172.30.72.60]) by szxga06-in.huawei.com (SkyGuard) with ESMTP id 4GfKCL1Y7lzmmZs; Tue, 3 Aug 2021 23:53:02 +0800 (CST) Received: from lhreml710-chm.china.huawei.com (10.201.108.61) by dggems703-chm.china.huawei.com (10.3.19.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Tue, 3 Aug 2021 23:56:10 +0800 Received: from lhreml710-chm.china.huawei.com ([169.254.81.184]) by lhreml710-chm.china.huawei.com ([169.254.81.184]) with mapi id 15.01.2176.012; Tue, 3 Aug 2021 16:56:09 +0100 From: Shameerali Kolothum Thodi To: Will Deacon CC: "linux-arm-kernel@lists.infradead.org" , "kvmarm@lists.cs.columbia.edu" , "linux-kernel@vger.kernel.org" , "maz@kernel.org" , "catalin.marinas@arm.com" , "james.morse@arm.com" , "julien.thierry.kdev@gmail.com" , "suzuki.poulose@arm.com" , "jean-philippe@linaro.org" , "Alexandru.Elisei@arm.com" , "qperret@google.com" , Linuxarm Subject: RE: [PATCH v3 4/4] KVM: arm64: Clear active_vmids on vCPU schedule out Thread-Topic: [PATCH v3 4/4] KVM: arm64: Clear active_vmids on vCPU schedule out Thread-Index: AQHXhGar1JrryboKrU+7h+yU/0K2/6thny2AgAAaZCCAACXigIAAEl3Q Date: Tue, 3 Aug 2021 15:56:08 +0000 Message-ID: <58bf7a27a4c440c685c6b1e7c9325441@huawei.com> References: <20210729104009.382-1-shameerali.kolothum.thodi@huawei.com> <20210729104009.382-5-shameerali.kolothum.thodi@huawei.com> <20210803114034.GB30853@willie-the-truck> <20210803153036.GA31125@willie-the-truck> In-Reply-To: <20210803153036.GA31125@willie-the-truck> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.47.86.18] MIME-Version: 1.0 X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210803_085616_490423_E6D3BB2B X-CRM114-Status: GOOD ( 30.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org > -----Original Message----- > From: Will Deacon [mailto:will@kernel.org] > Sent: 03 August 2021 16:31 > To: Shameerali Kolothum Thodi > Cc: linux-arm-kernel@lists.infradead.org; kvmarm@lists.cs.columbia.edu; > linux-kernel@vger.kernel.org; maz@kernel.org; catalin.marinas@arm.com; > james.morse@arm.com; julien.thierry.kdev@gmail.com; > suzuki.poulose@arm.com; jean-philippe@linaro.org; > Alexandru.Elisei@arm.com; qperret@google.com; Linuxarm > > Subject: Re: [PATCH v3 4/4] KVM: arm64: Clear active_vmids on vCPU > schedule out > > On Tue, Aug 03, 2021 at 12:55:25PM +0000, Shameerali Kolothum Thodi > wrote: > > > > diff --git a/arch/arm64/kvm/vmid.c b/arch/arm64/kvm/vmid.c > > > > index 5584e84aed95..5fd51f5445c1 100644 > > > > --- a/arch/arm64/kvm/vmid.c > > > > +++ b/arch/arm64/kvm/vmid.c > > > > @@ -116,6 +116,12 @@ static u64 new_vmid(struct kvm_vmid > > > *kvm_vmid) > > > > return idx2vmid(vmid) | generation; > > > > } > > > > > > > > +/* Call with preemption disabled */ > > > > +void kvm_arm_vmid_clear_active(void) > > > > +{ > > > > + atomic64_set(this_cpu_ptr(&active_vmids), 0); > > > > +} > > > > > > I think this is very broken, as it will force everybody to take the > > > slow-path when they see an active_vmid of 0. > > > > Yes. I have seen that happening in my test setup. > > Why didn't you say so?! Sorry. I thought of getting some performance numbers with and without this patch and measure the impact. But didn't quite get time to finish it yet. > > > > It also doesn't solve the issue I mentioned before, as an active_vmid of 0 > > > means that the reserved vmid is preserved. > > > > > > Needs more thought... > > > > How about we clear all the active_vmids in kvm_arch_free_vm() if it > > matches the kvm_vmid->id ? But we may have to hold the lock > > there > > I think we have to be really careful not to run into the "suspended > animation" problem described in ae120d9edfe9 ("ARM: 7767/1: let the ASID > allocator handle suspended animation") if we go down this road. Ok. I will go through that. > Maybe something along the lines of: > > ROLLOVER > > * Take lock > * Inc generation > => This will force everybody down the slow path > * Record active VMIDs > * Broadcast TLBI > => Only active VMIDs can be dirty > => Reserve active VMIDs and mark as allocated > > VCPU SCHED IN > > * Set active VMID > * Check generation > * If mismatch then: > * Take lock > * Try to match a reserved VMID > * If no reserved VMID, allocate new > > VCPU SCHED OUT > > * Clear active VMID > > but I'm not daft enough to think I got it right first time. I think it > needs both implementing *and* modelling in TLA+ before we merge it! > Ok. I need some time to digest the above first :). On another note, how serious do you think is the problem of extra reservation of the VMID space? Just wondering if we can skip this patch for now or not.. Thanks, Shameer _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel