From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06CD8C432BE for ; Mon, 9 Aug 2021 13:11:36 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A5B7B60F93 for ; Mon, 9 Aug 2021 13:11:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A5B7B60F93 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=B09z3V8IrgslZW4SGspLA+9MkuXDoCC4hCsrfTSTbnI=; b=XbX+DxyWcBjkao KNGHQBU4MkjbM4yYyyhRUwt8zIBpHIEGFBRYLBW6WR1NNuqJ5/tFX1ON6ZO3jQA6qYq2p/AIsQiMA GtQ89AZMxaaLY/gPAFN2F7RUi0ngLDGl48y1diPTlnBOXS72JwMI8Duw1op/R2oB8G3ognb7cygji RXyhZ0lKclHO4aqS5b1pP6HAXB8yYGFHoSx3poR4ahEAD73GmicVxHOwnTe1NXYqesLjScPCfvclH olnGbRE5FMXM81u4zrZhjqdxMU2efFntnbEDH26S7iCJOv3OrxOhqjn3xLSlM4d7nW4juOp88R+7B VJ1oSTkz1h+N9RkQB69Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD520-000h4S-49; Mon, 09 Aug 2021 13:09:28 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD51v-000h3G-EP for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 13:09:25 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 0BE6E60EC0; Mon, 9 Aug 2021 13:09:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1628514563; bh=c6YJj4mhhNAXx6eVWt/9O44DDuMRQSozD29Cp4nkeC4=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=dFUthAbq5qF7TRptNmUkrRcgE3Pm/TgwlG5vaCI/Ih4nOb914rc2Bq7Gr9nus0F8/ Vsa6CRs0lYZSkOP9/cr4LkgbhSuVGenhJwpPSL7P2PaET/jZr7ZHKDhDihvmLtxQDc s9TJwQeB4dhkYTlAmkI9sJ3tMFKGKQabR2B3WVd25EP1h1RJq2dnn29Q+xNHxqc9i8 nXswAkf6pI0o8d0y/fWall3OsWwIywpekINKrCHLjLruXzjGu1X0TBO9T9nbENsCZJ R+N2mE4pZfJGPXm1KU6tzZPY/5bXitA5fG8vCUna4bQwd/FOr8dvIAeZ/EGqSoHRzv D2cYdhCIQBTXQ== Date: Mon, 9 Aug 2021 14:09:17 +0100 From: Will Deacon To: Shameerali Kolothum Thodi Cc: "linux-arm-kernel@lists.infradead.org" , "kvmarm@lists.cs.columbia.edu" , "linux-kernel@vger.kernel.org" , "maz@kernel.org" , "catalin.marinas@arm.com" , "james.morse@arm.com" , "julien.thierry.kdev@gmail.com" , "suzuki.poulose@arm.com" , "jean-philippe@linaro.org" , "Alexandru.Elisei@arm.com" , "qperret@google.com" , Linuxarm Subject: Re: [PATCH v3 4/4] KVM: arm64: Clear active_vmids on vCPU schedule out Message-ID: <20210809130917.GA1207@willie-the-truck> References: <20210729104009.382-1-shameerali.kolothum.thodi@huawei.com> <20210729104009.382-5-shameerali.kolothum.thodi@huawei.com> <20210803114034.GB30853@willie-the-truck> <20210803153036.GA31125@willie-the-truck> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_060923_568509_C35567F0 X-CRM114-Status: GOOD ( 14.69 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Aug 06, 2021 at 12:24:36PM +0000, Shameerali Kolothum Thodi wrote: > These are some test numbers with and without this patch, run on two > different test setups. > > > a)Test Setup -1 > ----------------------- > > Platform: HiSilicon D06 with 128 CPUs, VMID bits = 16 > Run 128 VMs concurrently each with 2 vCPUs. Each Guest will execute hackbench > 5 times before exiting. > > Measurements taken avg. of 10 Runs. > > Image : 5.14-rc3 > --------------------------- > Time(s) 44.43813888 > No. of exits 145,348,264 > > Image: 5.14-rc3 + vmid-v3 > ---------------------------------------- > Time(s) 46.59789034 > No. of exits 133,587,307 > > %diff against 5.14-rc3 > Time: 4.8% more > Exits: 8% less > > Image: 5.14-rc3 + vmid-v3 + Without active_asid clear > --------------------------------------------------------------------------- > Time(s) 44.5031782 > No. of exits 144,443,188 > > %diff against 5.14-rc3 > Time: 0.15% more > Exits: 2.42% less > > b)Test Setup -2 > ----------------------- > > Platform: HiSilicon D06 + Kernel with maxcpus set to 8 and VMID bits set to 4. > Run 40 VMs concurrently each with 2 vCPUs. Each Guest will execute hackbench > 5 times before exiting. > > Measurements taken avg. of 10 Runs. > > Image : 5.14-rc3-vmid4bit > ------------------------------------ > Time(s) 46.19963266 > No. of exits 23,699,546 > > Image: 5.14-rc3-vmid4bit + vmid-v3 > --------------------------------------------------- > Time(s) 45.83307736 > No. of exits 23,260,203 > > %diff against 5.14-rc3-vmid4bit > Time: 0.8% less > Exits: 1.85% less > > Image: 5.14-rc3-vmid4bit + vmid-v3 + Without active_asid clear > ----------------------------------------------------------------------------------------- > Time(s) 44.5031782 > No. of exits 144,443,188 Really? The *exact* same numbers as the "Image: 5.14-rc3 + vmid-v3 + Without active_asid clear" configuration? Guessing a copy-paste error here. > %diff against 5.14-rc3-vmid4bit > Time: 1.05% less > Exits: 2.06% less > > As expected, the active_asid clear on schedule out is not helping. > But without this patch, the numbers seems to be better than the > vanilla kernel when we force the setup(cpus=8, vmd=4bits) > to perform rollover. I'm struggling a bit to understand these numbers. Are you saying that clearing the active_asid helps in the 16-bit VMID case but not in the 4-bit case? Why would the active_asid clear have any impact on the number of exits? The problem I see with not having the active_asid clear is that we will roll over more frequently as the number of reserved VMIDs increases. Will _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel