* [PATCH] x86/kvm: Introduce boot parameter no-kvm-pvipi @ 2021-10-20 12:07 zhenwei pi 2021-10-20 12:22 ` Wanpeng Li 0 siblings, 1 reply; 9+ messages in thread From: zhenwei pi @ 2021-10-20 12:07 UTC (permalink / raw) To: pbonzini, corbet; +Cc: wanpengli, seanjc, linux-kernel, linux-doc, zhenwei pi Although host side exposes KVM PV SEND IPI feature to guest side, guest should still have a chance to disable it. A typicall case of this parameter: If the host AMD server enables AVIC feature, the flat mode of APIC get better performance in the guest. Signed-off-by: zhenwei pi <pizhenwei@bytedance.com> --- Documentation/admin-guide/kernel-parameters.txt | 2 ++ arch/x86/kernel/kvm.c | 13 ++++++++++++- 2 files changed, 14 insertions(+), 1 deletion(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 43dc35fe5bc0..73b8712b94b0 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -3495,6 +3495,8 @@ no-kvmapf [X86,KVM] Disable paravirtualized asynchronous page fault handling. + no-kvm-pvipi [X86,KVM] Disable paravirtualized KVM send IPI. + no-vmw-sched-clock [X86,PV_OPS] Disable paravirtualized VMware scheduler clock and use the default one. diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index b656456c3a94..911f1cd2bec5 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -62,6 +62,17 @@ static int __init parse_no_stealacc(char *arg) early_param("no-steal-acc", parse_no_stealacc); +static int kvm_pvipi = 1; + +static int __init parse_no_kvm_pvipi(char *arg) +{ + kvm_pvipi = 0; + + return 0; +} + +early_param("no-kvm-pvipi", parse_no_kvm_pvipi); + static DEFINE_PER_CPU_DECRYPTED(struct kvm_vcpu_pv_apf_data, apf_reason) __aligned(64); DEFINE_PER_CPU_DECRYPTED(struct kvm_steal_time, steal_time) __aligned(64) __visible; static int has_steal_clock = 0; @@ -795,7 +806,7 @@ static uint32_t __init kvm_detect(void) static void __init kvm_apic_init(void) { #ifdef CONFIG_SMP - if (pv_ipi_supported()) + if (pv_ipi_supported() && kvm_pvipi) kvm_setup_pv_ipi(); #endif } -- 2.25.1 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH] x86/kvm: Introduce boot parameter no-kvm-pvipi 2021-10-20 12:07 [PATCH] x86/kvm: Introduce boot parameter no-kvm-pvipi zhenwei pi @ 2021-10-20 12:22 ` Wanpeng Li 2021-10-20 20:12 ` Sean Christopherson 0 siblings, 1 reply; 9+ messages in thread From: Wanpeng Li @ 2021-10-20 12:22 UTC (permalink / raw) To: zhenwei pi Cc: Paolo Bonzini, Jonathan Corbet, Wanpeng Li, Sean Christopherson, LKML, linux-doc On Wed, 20 Oct 2021 at 20:08, zhenwei pi <pizhenwei@bytedance.com> wrote: > > Although host side exposes KVM PV SEND IPI feature to guest side, > guest should still have a chance to disable it. > > A typicall case of this parameter: > If the host AMD server enables AVIC feature, the flat mode of APIC > get better performance in the guest. Hmm, I didn't find enough valuable information in your posting. We observe AMD a lot before. https://lore.kernel.org/all/CANRm+Cx597FNRUCyVz1D=B6Vs2GX3Sw57X7Muk+yMpi_hb+v1w@mail.gmail.com/T/#u Wanpeng ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] x86/kvm: Introduce boot parameter no-kvm-pvipi 2021-10-20 12:22 ` Wanpeng Li @ 2021-10-20 20:12 ` Sean Christopherson 2021-10-21 3:02 ` zhenwei pi 0 siblings, 1 reply; 9+ messages in thread From: Sean Christopherson @ 2021-10-20 20:12 UTC (permalink / raw) To: Wanpeng Li Cc: zhenwei pi, Paolo Bonzini, Jonathan Corbet, Wanpeng Li, LKML, linux-doc On Wed, Oct 20, 2021, Wanpeng Li wrote: > On Wed, 20 Oct 2021 at 20:08, zhenwei pi <pizhenwei@bytedance.com> wrote: > > > > Although host side exposes KVM PV SEND IPI feature to guest side, > > guest should still have a chance to disable it. > > > > A typicall case of this parameter: > > If the host AMD server enables AVIC feature, the flat mode of APIC > > get better performance in the guest. > > Hmm, I didn't find enough valuable information in your posting. We > observe AMD a lot before. > https://lore.kernel.org/all/CANRm+Cx597FNRUCyVz1D=B6Vs2GX3Sw57X7Muk+yMpi_hb+v1w@mail.gmail.com/T/#u I too would like to see numbers. I suspect the answer is going to be that AVIC performs poorly in CPU overcommit scenarios because of the cost of managing the tables and handling "failed delivery" exits, but that AVIC does quite well when vCPUs are pinned 1:1 and IPIs rarely require an exit to the host. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Re: [PATCH] x86/kvm: Introduce boot parameter no-kvm-pvipi 2021-10-20 20:12 ` Sean Christopherson @ 2021-10-21 3:02 ` zhenwei pi 2021-10-21 5:03 ` Wanpeng Li 0 siblings, 1 reply; 9+ messages in thread From: zhenwei pi @ 2021-10-21 3:02 UTC (permalink / raw) To: Sean Christopherson, Wanpeng Li Cc: Paolo Bonzini, Jonathan Corbet, Wanpeng Li, LKML, linux-doc On 10/21/21 4:12 AM, Sean Christopherson wrote: > On Wed, Oct 20, 2021, Wanpeng Li wrote: >> On Wed, 20 Oct 2021 at 20:08, zhenwei pi <pizhenwei@bytedance.com> wrote: >>> >>> Although host side exposes KVM PV SEND IPI feature to guest side, >>> guest should still have a chance to disable it. >>> >>> A typicall case of this parameter: >>> If the host AMD server enables AVIC feature, the flat mode of APIC >>> get better performance in the guest. >> >> Hmm, I didn't find enough valuable information in your posting. We >> observe AMD a lot before. >> https://lore.kernel.org/all/CANRm+Cx597FNRUCyVz1D=B6Vs2GX3Sw57X7Muk+yMpi_hb+v1w@mail.gmail.com/T/#u > > I too would like to see numbers. I suspect the answer is going to be that > AVIC performs poorly in CPU overcommit scenarios because of the cost of managing > the tables and handling "failed delivery" exits, but that AVIC does quite well > when vCPUs are pinned 1:1 and IPIs rarely require an exit to the host. > Test env: CPU: AMD EPYC 7642 48-Core Processor Kmod args(enable avic and disable nested): modprobe kvm-amd nested=0 avic=1 npt=1 QEMU args(disable x2apic): ... -cpu host,x2apic=off ... Benchmark tool: https://github.com/bytedance/kvm-utils/tree/master/microbenchmark/apic-ipi ~# insmod apic_ipi.ko options=5 && dmesg -c apic_ipi: 1 NUMA node(s) apic_ipi: apic [flat] apic_ipi: apic->send_IPI[default_send_IPI_single+0x0/0x40] apic_ipi: apic->send_IPI_mask[kvm_send_ipi_mask+0x0/0x10] apic_ipi: IPI[kvm_send_ipi_mask] from CPU[0] to CPU[1] apic_ipi: total cycles 375671259, avg 3756 apic_ipi: IPI[flat_send_IPI_mask] from CPU[0] to CPU[1] apic_ipi: total cycles 221961822, avg 2219 apic->send_IPI_mask[kvm_send_ipi_mask+0x0/0x10] -> This line show current send_IPI_mask is kvm_send_ipi_mask(because of PV SEND IPI FEATURE) apic_ipi: IPI[kvm_send_ipi_mask] from CPU[0] to CPU[1] apic_ipi: total cycles 375671259, avg 3756 -->These lines show the average cycles of each kvm_send_ipi_mask: 3756 apic_ipi: IPI[flat_send_IPI_mask] from CPU[0] to CPU[1] apic_ipi: total cycles 221961822, avg 2219 -->These lines show the average cycles of each flat_send_IPI_mask: 2219 -- zhenwei pi ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Re: [PATCH] x86/kvm: Introduce boot parameter no-kvm-pvipi 2021-10-21 3:02 ` zhenwei pi @ 2021-10-21 5:03 ` Wanpeng Li 2021-10-21 7:17 ` zhenwei pi 0 siblings, 1 reply; 9+ messages in thread From: Wanpeng Li @ 2021-10-21 5:03 UTC (permalink / raw) To: zhenwei pi Cc: Sean Christopherson, Paolo Bonzini, Jonathan Corbet, Wanpeng Li, LKML, linux-doc On Thu, 21 Oct 2021 at 11:05, zhenwei pi <pizhenwei@bytedance.com> wrote: > > > On 10/21/21 4:12 AM, Sean Christopherson wrote: > > On Wed, Oct 20, 2021, Wanpeng Li wrote: > >> On Wed, 20 Oct 2021 at 20:08, zhenwei pi <pizhenwei@bytedance.com> wrote: > >>> > >>> Although host side exposes KVM PV SEND IPI feature to guest side, > >>> guest should still have a chance to disable it. > >>> > >>> A typicall case of this parameter: > >>> If the host AMD server enables AVIC feature, the flat mode of APIC > >>> get better performance in the guest. > >> > >> Hmm, I didn't find enough valuable information in your posting. We > >> observe AMD a lot before. > >> https://lore.kernel.org/all/CANRm+Cx597FNRUCyVz1D=B6Vs2GX3Sw57X7Muk+yMpi_hb+v1w@mail.gmail.com/T/#u > > > > I too would like to see numbers. I suspect the answer is going to be that > > AVIC performs poorly in CPU overcommit scenarios because of the cost of managing > > the tables and handling "failed delivery" exits, but that AVIC does quite well > > when vCPUs are pinned 1:1 and IPIs rarely require an exit to the host. > > > > Test env: > CPU: AMD EPYC 7642 48-Core Processor > > Kmod args(enable avic and disable nested): > modprobe kvm-amd nested=0 avic=1 npt=1 > > QEMU args(disable x2apic): > ... -cpu host,x2apic=off ... > > Benchmark tool: > https://github.com/bytedance/kvm-utils/tree/master/microbenchmark/apic-ipi > > ~# insmod apic_ipi.ko options=5 && dmesg -c > > apic_ipi: 1 NUMA node(s) > apic_ipi: apic [flat] > apic_ipi: apic->send_IPI[default_send_IPI_single+0x0/0x40] > apic_ipi: apic->send_IPI_mask[kvm_send_ipi_mask+0x0/0x10] > apic_ipi: IPI[kvm_send_ipi_mask] from CPU[0] to CPU[1] > apic_ipi: total cycles 375671259, avg 3756 > apic_ipi: IPI[flat_send_IPI_mask] from CPU[0] to CPU[1] > apic_ipi: total cycles 221961822, avg 2219 > > > apic->send_IPI_mask[kvm_send_ipi_mask+0x0/0x10] > -> This line show current send_IPI_mask is kvm_send_ipi_mask(because > of PV SEND IPI FEATURE) > > apic_ipi: IPI[kvm_send_ipi_mask] from CPU[0] to CPU[1] > apic_ipi: total cycles 375671259, avg 3756 > -->These lines show the average cycles of each kvm_send_ipi_mask: 3756 > > apic_ipi: IPI[flat_send_IPI_mask] from CPU[0] to CPU[1] > apic_ipi: total cycles 221961822, avg 2219 > -->These lines show the average cycles of each flat_send_IPI_mask: 2219 Just single target IPI is not eough. Wanpeng ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Re: Re: [PATCH] x86/kvm: Introduce boot parameter no-kvm-pvipi 2021-10-21 5:03 ` Wanpeng Li @ 2021-10-21 7:17 ` zhenwei pi 2021-10-25 3:14 ` zhenwei pi 0 siblings, 1 reply; 9+ messages in thread From: zhenwei pi @ 2021-10-21 7:17 UTC (permalink / raw) To: Wanpeng Li Cc: Sean Christopherson, Paolo Bonzini, Jonathan Corbet, Wanpeng Li, LKML, linux-doc On 10/21/21 1:03 PM, Wanpeng Li wrote: > On Thu, 21 Oct 2021 at 11:05, zhenwei pi <pizhenwei@bytedance.com> wrote: >> >> >> On 10/21/21 4:12 AM, Sean Christopherson wrote: >>> On Wed, Oct 20, 2021, Wanpeng Li wrote: >>>> On Wed, 20 Oct 2021 at 20:08, zhenwei pi <pizhenwei@bytedance.com> wrote: >>>>> >>>>> Although host side exposes KVM PV SEND IPI feature to guest side, >>>>> guest should still have a chance to disable it. >>>>> >>>>> A typicall case of this parameter: >>>>> If the host AMD server enables AVIC feature, the flat mode of APIC >>>>> get better performance in the guest. >>>> >>>> Hmm, I didn't find enough valuable information in your posting. We >>>> observe AMD a lot before. >>>> https://lore.kernel.org/all/CANRm+Cx597FNRUCyVz1D=B6Vs2GX3Sw57X7Muk+yMpi_hb+v1w@mail.gmail.com/T/#u >>> >>> I too would like to see numbers. I suspect the answer is going to be that >>> AVIC performs poorly in CPU overcommit scenarios because of the cost of managing >>> the tables and handling "failed delivery" exits, but that AVIC does quite well >>> when vCPUs are pinned 1:1 and IPIs rarely require an exit to the host. >>> >> >> Test env: >> CPU: AMD EPYC 7642 48-Core Processor >> >> Kmod args(enable avic and disable nested): >> modprobe kvm-amd nested=0 avic=1 npt=1 >> >> QEMU args(disable x2apic): >> ... -cpu host,x2apic=off ... >> >> Benchmark tool: >> https://github.com/bytedance/kvm-utils/tree/master/microbenchmark/apic-ipi >> >> ~# insmod apic_ipi.ko options=5 && dmesg -c >> >> apic_ipi: 1 NUMA node(s) >> apic_ipi: apic [flat] >> apic_ipi: apic->send_IPI[default_send_IPI_single+0x0/0x40] >> apic_ipi: apic->send_IPI_mask[kvm_send_ipi_mask+0x0/0x10] >> apic_ipi: IPI[kvm_send_ipi_mask] from CPU[0] to CPU[1] >> apic_ipi: total cycles 375671259, avg 3756 >> apic_ipi: IPI[flat_send_IPI_mask] from CPU[0] to CPU[1] >> apic_ipi: total cycles 221961822, avg 2219 >> >> >> apic->send_IPI_mask[kvm_send_ipi_mask+0x0/0x10] >> -> This line show current send_IPI_mask is kvm_send_ipi_mask(because >> of PV SEND IPI FEATURE) >> >> apic_ipi: IPI[kvm_send_ipi_mask] from CPU[0] to CPU[1] >> apic_ipi: total cycles 375671259, avg 3756 >> -->These lines show the average cycles of each kvm_send_ipi_mask: 3756 >> >> apic_ipi: IPI[flat_send_IPI_mask] from CPU[0] to CPU[1] >> apic_ipi: total cycles 221961822, avg 2219 >> -->These lines show the average cycles of each flat_send_IPI_mask: 2219 > > Just single target IPI is not eough. > > Wanpeng > Benchmark smp_call_function_single (https://github.com/bytedance/kvm-utils/blob/master/microbenchmark/ipi-bench/ipi_bench.c): Test env: CPU: AMD EPYC 7642 48-Core Processor Kmod args(enable avic and disable nested): modprobe kvm-amd nested=0 avic=1 npt=1 QEMU args(disable x2apic): ... -cpu host,x2apic=off ... 1> without no-kvm-pvipi: ipi_bench_single wait[1], CPU0[NODE0] -> CPU1[NODE0], loop = 100000 elapsed = 424945631 cycles, average = 4249 cycles ipitime = 385246136 cycles, average = 3852 cycles ipi_bench_single wait[0], CPU0[NODE0] -> CPU1[NODE0], loop = 100000 elapsed = 419057953 cycles, average = 4190 cycles 2> with no-kvm-pvipi: ipi_bench_single wait[1], CPU0[NODE0] -> CPU1[NODE0], loop = 100000 elapsed = 321756407 cycles, average = 3217 cycles ipitime = 299433550 cycles, average = 2994 cycles ipi_bench_single wait[0], CPU0[NODE0] -> CPU1[NODE0], loop = 100000 elapsed = 295382146 cycles, average = 2953 cycles -- zhenwei pi ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] x86/kvm: Introduce boot parameter no-kvm-pvipi 2021-10-21 7:17 ` zhenwei pi @ 2021-10-25 3:14 ` zhenwei pi 2021-10-26 16:04 ` Sean Christopherson 0 siblings, 1 reply; 9+ messages in thread From: zhenwei pi @ 2021-10-25 3:14 UTC (permalink / raw) To: Wanpeng Li, Sean Christopherson Cc: Paolo Bonzini, Jonathan Corbet, Wanpeng Li, LKML, linux-doc On 10/21/21 3:17 PM, zhenwei pi wrote: > On 10/21/21 1:03 PM, Wanpeng Li wrote: >> On Thu, 21 Oct 2021 at 11:05, zhenwei pi <pizhenwei@bytedance.com> wrote: >>> >>> >>> On 10/21/21 4:12 AM, Sean Christopherson wrote: >>>> On Wed, Oct 20, 2021, Wanpeng Li wrote: >>>>> On Wed, 20 Oct 2021 at 20:08, zhenwei pi <pizhenwei@bytedance.com> >>>>> wrote: >>>>>> >>>>>> Although host side exposes KVM PV SEND IPI feature to guest side, >>>>>> guest should still have a chance to disable it. >>>>>> >>>>>> A typicall case of this parameter: >>>>>> If the host AMD server enables AVIC feature, the flat mode of APIC >>>>>> get better performance in the guest. >>>>> >>>>> Hmm, I didn't find enough valuable information in your posting. We >>>>> observe AMD a lot before. >>>>> https://lore.kernel.org/all/CANRm+Cx597FNRUCyVz1D=B6Vs2GX3Sw57X7Muk+yMpi_hb+v1w@mail.gmail.com/T/#u >>>>> >>>> >>>> I too would like to see numbers. I suspect the answer is going to >>>> be that >>>> AVIC performs poorly in CPU overcommit scenarios because of the cost >>>> of managing >>>> the tables and handling "failed delivery" exits, but that AVIC does >>>> quite well >>>> when vCPUs are pinned 1:1 and IPIs rarely require an exit to the host. >>>> >>> >>> Test env: >>> CPU: AMD EPYC 7642 48-Core Processor >>> >>> Kmod args(enable avic and disable nested): >>> modprobe kvm-amd nested=0 avic=1 npt=1 >>> >>> QEMU args(disable x2apic): >>> ... -cpu host,x2apic=off ... >>> >>> Benchmark tool: >>> https://github.com/bytedance/kvm-utils/tree/master/microbenchmark/apic-ipi >>> >>> >>> ~# insmod apic_ipi.ko options=5 && dmesg -c >>> >>> apic_ipi: 1 NUMA node(s) >>> apic_ipi: apic [flat] >>> apic_ipi: apic->send_IPI[default_send_IPI_single+0x0/0x40] >>> apic_ipi: apic->send_IPI_mask[kvm_send_ipi_mask+0x0/0x10] >>> apic_ipi: IPI[kvm_send_ipi_mask] from CPU[0] to CPU[1] >>> apic_ipi: total cycles 375671259, avg 3756 >>> apic_ipi: IPI[flat_send_IPI_mask] from CPU[0] to CPU[1] >>> apic_ipi: total cycles 221961822, avg 2219 >>> >>> >>> apic->send_IPI_mask[kvm_send_ipi_mask+0x0/0x10] >>> -> This line show current send_IPI_mask is kvm_send_ipi_mask(because >>> of PV SEND IPI FEATURE) >>> >>> apic_ipi: IPI[kvm_send_ipi_mask] from CPU[0] to CPU[1] >>> apic_ipi: total cycles 375671259, avg 3756 >>> -->These lines show the average cycles of each kvm_send_ipi_mask: >>> 3756 >>> >>> apic_ipi: IPI[flat_send_IPI_mask] from CPU[0] to CPU[1] >>> apic_ipi: total cycles 221961822, avg 2219 >>> -->These lines show the average cycles of each >>> flat_send_IPI_mask: 2219 >> >> Just single target IPI is not eough. >> >> Wanpeng >> > > Benchmark smp_call_function_single > (https://github.com/bytedance/kvm-utils/blob/master/microbenchmark/ipi-bench/ipi_bench.c): > > > Test env: > CPU: AMD EPYC 7642 48-Core Processor > > Kmod args(enable avic and disable nested): > modprobe kvm-amd nested=0 avic=1 npt=1 > > QEMU args(disable x2apic): > ... -cpu host,x2apic=off ... > > 1> without no-kvm-pvipi: > ipi_bench_single wait[1], CPU0[NODE0] -> CPU1[NODE0], loop = 100000 > elapsed = 424945631 cycles, average = 4249 cycles > ipitime = 385246136 cycles, average = 3852 cycles > ipi_bench_single wait[0], CPU0[NODE0] -> CPU1[NODE0], loop = 100000 > elapsed = 419057953 cycles, average = 4190 cycles > > 2> with no-kvm-pvipi: > ipi_bench_single wait[1], CPU0[NODE0] -> CPU1[NODE0], loop = 100000 > elapsed = 321756407 cycles, average = 3217 cycles > ipitime = 299433550 cycles, average = 2994 cycles > ipi_bench_single wait[0], CPU0[NODE0] -> CPU1[NODE0], loop = 100000 > elapsed = 295382146 cycles, average = 2953 cycles > > Hi, Wanpeng & Sean Also benchmark redis(by 127.0.0.1) in a guest(2vCPU), 'no-kvm-pvipi' gets better performance. Test env: Host side: pin 2vCPU on 2core in a die. Guest side: run command: taskset -c 1 ./redis-server --appendonly no taskset -c 0 ./redis-benchmark -h 127.0.0.1 -d 1024 -n 10000000 -t get 1> without no-kvm-pvipi: redis QPS: 193203.12 requests per second kvm_pv_send_ipi exit: ~18K/s 2> with no-kvm-pvipi: redis QPS: 196028.47 requests per second avic_incomplete_ipi_interception exit: ~5K/s -- zhenwei pi ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] x86/kvm: Introduce boot parameter no-kvm-pvipi 2021-10-25 3:14 ` zhenwei pi @ 2021-10-26 16:04 ` Sean Christopherson 2021-10-27 0:46 ` Wanpeng Li 0 siblings, 1 reply; 9+ messages in thread From: Sean Christopherson @ 2021-10-26 16:04 UTC (permalink / raw) To: zhenwei pi Cc: Wanpeng Li, Paolo Bonzini, Jonathan Corbet, Wanpeng Li, LKML, linux-doc On Mon, Oct 25, 2021, zhenwei pi wrote: > Hi, Wanpeng & Sean > > Also benchmark redis(by 127.0.0.1) in a guest(2vCPU), 'no-kvm-pvipi' gets > better performance. > > Test env: > Host side: pin 2vCPU on 2core in a die. > Guest side: run command: > taskset -c 1 ./redis-server --appendonly no > taskset -c 0 ./redis-benchmark -h 127.0.0.1 -d 1024 -n 10000000 -t get > > 1> without no-kvm-pvipi: > redis QPS: 193203.12 requests per second > kvm_pv_send_ipi exit: ~18K/s > > 2> with no-kvm-pvipi: > redis QPS: 196028.47 requests per second > avic_incomplete_ipi_interception exit: ~5K/s Numbers look sane, but I don't think that adding a guest-side kernel param is the correct "fix". As evidenced by Wanpeng's tests, PV IPI can outperform AVIC in overcommit scenarios, and there's also no guarantee that AVIC/APICv is even supported/enabled. In other words, blindly disabling PV IPIs from within the guest makes sense if and only if the guest knows that AVIC is enabled and that its vCPUs are pinned. If the guest has that info, then the host also has that info, in which case the correct way to handle this is to simply not advertise KVM_FEATURE_PV_SEND_IPI to the guest in CPUID. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] x86/kvm: Introduce boot parameter no-kvm-pvipi 2021-10-26 16:04 ` Sean Christopherson @ 2021-10-27 0:46 ` Wanpeng Li 0 siblings, 0 replies; 9+ messages in thread From: Wanpeng Li @ 2021-10-27 0:46 UTC (permalink / raw) To: Sean Christopherson Cc: zhenwei pi, Paolo Bonzini, Jonathan Corbet, Wanpeng Li, LKML, linux-doc On Wed, 27 Oct 2021 at 00:04, Sean Christopherson <seanjc@google.com> wrote: > > On Mon, Oct 25, 2021, zhenwei pi wrote: > > Hi, Wanpeng & Sean > > > > Also benchmark redis(by 127.0.0.1) in a guest(2vCPU), 'no-kvm-pvipi' gets > > better performance. > > > > Test env: > > Host side: pin 2vCPU on 2core in a die. > > Guest side: run command: > > taskset -c 1 ./redis-server --appendonly no > > taskset -c 0 ./redis-benchmark -h 127.0.0.1 -d 1024 -n 10000000 -t get > > > > 1> without no-kvm-pvipi: > > redis QPS: 193203.12 requests per second > > kvm_pv_send_ipi exit: ~18K/s > > > > 2> with no-kvm-pvipi: > > redis QPS: 196028.47 requests per second > > avic_incomplete_ipi_interception exit: ~5K/s > > Numbers look sane, but I don't think that adding a guest-side kernel param is > the correct "fix". As evidenced by Wanpeng's tests, PV IPI can outperform AVIC > in overcommit scenarios, and there's also no guarantee that AVIC/APICv is even Our evaluation is a dedicated scenario w/ big VM. The testing from above is a one-sided view. Wanpeng ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2021-10-27 0:47 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-10-20 12:07 [PATCH] x86/kvm: Introduce boot parameter no-kvm-pvipi zhenwei pi 2021-10-20 12:22 ` Wanpeng Li 2021-10-20 20:12 ` Sean Christopherson 2021-10-21 3:02 ` zhenwei pi 2021-10-21 5:03 ` Wanpeng Li 2021-10-21 7:17 ` zhenwei pi 2021-10-25 3:14 ` zhenwei pi 2021-10-26 16:04 ` Sean Christopherson 2021-10-27 0:46 ` Wanpeng Li
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.