From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.5 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60994C432BE for ; Fri, 23 Jul 2021 07:16:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3F60A60E90 for ; Fri, 23 Jul 2021 07:16:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231191AbhGWGgP (ORCPT ); Fri, 23 Jul 2021 02:36:15 -0400 Received: from mga12.intel.com ([192.55.52.136]:54299 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229616AbhGWGgN (ORCPT ); Fri, 23 Jul 2021 02:36:13 -0400 X-IronPort-AV: E=McAfee;i="6200,9189,10053"; a="191418868" X-IronPort-AV: E=Sophos;i="5.84,263,1620716400"; d="scan'208";a="191418868" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jul 2021 00:16:47 -0700 X-IronPort-AV: E=Sophos;i="5.84,263,1620716400"; d="scan'208";a="471023868" Received: from zengguan-mobl.ccr.corp.intel.com (HELO [10.238.0.133]) ([10.238.0.133]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jul 2021 00:16:41 -0700 Subject: Re: [PATCH v2 0/6] IPI virtualization support for VM To: Wanpeng Li Cc: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm , Dave Hansen , "Luck, Tony" , Kan Liang , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Kim Phillips , Jarkko Sakkinen , Jethro Beekman , "Huang, Kai" , the arch/x86 maintainers , LKML , "Hu, Robert" , "Gao, Chao" References: <20210723051626.18364-1-guang.zeng@intel.com> From: Zeng Guang Message-ID: <4e8f8b7f-0b20-5c2a-f23d-3f5d5321dd3a@intel.com> Date: Fri, 23 Jul 2021 15:16:31 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0 Thunderbird/78.12.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 7/23/2021 2:11 PM, Wanpeng Li wrote: > On Fri, 23 Jul 2021 at 13:41, Zeng Guang wrote: > > -------------------------------------- > IPI microbenchmark: > (https://lore.kernel.org/kvm/20171219085010.4081-1-ynorov@caviumnetworks.com) > > 2 vCPUs, 1:1 pin vCPU to pCPU, guest VM runs with idle=poll, x2APIC mode > Improve the performance for unicast ipi is as expected, however, I > wonder whether the broadcast performance is worse than PV > IPIs/Thomas's IPI shorthands(IPI shorthands are supported by upstream > linux apic/x2apic driver). The hardware acceleration is not always > outstanding on AMD(https://lore.kernel.org/kvm/CANRm+Cx597FNRUCyVz1D=B6Vs2GX3Sw57X7Muk+yMpi_hb+v1w@mail.gmail.com/), > how about your Intel guys? Please try a big VM at least 96 vCPUs as > below or more bigger. Intel IPIv target to accelerate unicast ipi process, not benefit to broadcast performance. As to IPI benchmark, it's not big different to test with large or small scale of vCPUs. In essential, Normal IPI test try to send ipi to any other online CPU in sequence. The cost on IPI process itself should be similar. >> Result with IPIv enabled: >> >> Dry-run: 0, 272798 ns >> Self-IPI: 5094123, 11114037 ns >> Normal IPI: 131697087, 173321200 ns >> Broadcast IPI: 0, 155649075 ns >> Broadcast lock: 0, 161518031 ns >> >> Result with IPIv disabled: >> >> Dry-run: 0, 272766 ns >> Self-IPI: 5091788, 11123699 ns >> Normal IPI: 145215772, 174558920 ns >> Broadcast IPI: 0, 175785384 ns >> Broadcast lock: 0, 149076195 ns >> >> >> As IPIv can benefit unicast IPI to other CPU, Noraml IPI test case gain >> about 9.73% time saving on average out of 15 test runs when IPIv is >> enabled. >> >> w/o IPIv w/ IPIv >> Normal IPI: 145944306.6 ns 131742993.1 ns >> %Reduction -9.73% >> >> -------------------------------------- >> hackbench: >> >> 8 vCPUs, guest VM free run, x2APIC mode >> ./hackbench -p -l 100000 >> >> w/o IPIv w/ IPIv >> Time: 91.887 74.605 >> %Reduction: -18.808% >> >> 96 vCPUs, guest VM free run, x2APIC mode >> ./hackbench -p -l 1000000 >> >> w/o IPIv w/ IPIv >> Time: 287.504 235.185 >> %Reduction: -18.198% > Good to know this. > > Wanpeng