From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA6ECC433F5 for ; Wed, 4 May 2022 11:46:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349050AbiEDLuB (ORCPT ); Wed, 4 May 2022 07:50:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57862 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237744AbiEDLt5 (ORCPT ); Wed, 4 May 2022 07:49:57 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 72318BE7 for ; Wed, 4 May 2022 04:46:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1651664781; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9F/b5dq34n9ypa7ZJsv/PbEE1g/MJwHsEsaylCTqSvM=; b=N67xjir9Lmlgs1j+vOmyJDncrnwN5hmtA2oVG/Yc1qiVasCtWd6a4Q7VWOSJWy/L4vQirx GhVB8UH5/2V91FOnDDVEAxeNW/0dEwhCyHC6R5ps5/9RM0nDQ7Uo/bfHSBD0Z2/h+CH44c sC1Vj47okSJfBjEcRVkf2VWgQ3WJHAE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-629-uuC6M4G2PXCfgvAm4tvRzg-1; Wed, 04 May 2022 07:46:18 -0400 X-MC-Unique: uuC6M4G2PXCfgvAm4tvRzg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2EB4A86B8AD; Wed, 4 May 2022 11:46:18 +0000 (UTC) Received: from starship (unknown [10.40.192.26]) by smtp.corp.redhat.com (Postfix) with ESMTP id 15D1D112131E; Wed, 4 May 2022 11:46:15 +0000 (UTC) Message-ID: <8a0b27436239a97cc486d8460662febb6b155069.camel@redhat.com> Subject: Re: [PATCH v2 08/12] KVM: SVM: Update AVIC settings when changing APIC mode From: Maxim Levitsky To: Suravee Suthikulpanit , linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: pbonzini@redhat.com, seanjc@google.com, joro@8bytes.org, jon.grimm@amd.com, wei.huang2@amd.com, terry.bowman@amd.com Date: Wed, 04 May 2022 14:46:14 +0300 In-Reply-To: <3196873f-0047-3411-d434-56d96ca31298@amd.com> References: <20220412115822.14351-1-suravee.suthikulpanit@amd.com> <20220412115822.14351-9-suravee.suthikulpanit@amd.com> <9307c734-3473-0bdc-57be-c39e96bca4d8@amd.com> <24b74f5bd8810c7f79777ed6898baeaf47bfe3e3.camel@redhat.com> <3196873f-0047-3411-d434-56d96ca31298@amd.com> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.36.5 (3.36.5-2.fc32) MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 2022-05-03 at 20:04 +0700, Suravee Suthikulpanit wrote: > Maxim, > > On 5/3/22 12:13 AM, Maxim Levitsky wrote: > > > In the kvm/queue branch, I found a regression on nested SVM guest, where L2 guest cannot > > > launch. The bad commit is: > > > > > > commit a4cfff3f0f8c07f1f7873a82bdeb3995807dac8c (bisect) > > > Merge: 42dcbe7d8bac 8d5678a76689 > > > Author: Paolo Bonzini > > > Date: Fri Apr 8 12:43:40 2022 -0400 > > > > > > Merge branch 'kvm-older-features' into HEAD > > > > > > Merge branch for features that did not make it into 5.18: > > > > > > * New ioctls to get/set TSC frequency for a whole VM > > > > > > * Allow userspace to opt out of hypercall patching > > > > > > Nested virtualization improvements for AMD: > > > > > > * Support for "nested nested" optimizations (nested vVMLOAD/VMSAVE, > > > nested vGIF) > > > > > > * Allow AVIC to co-exist with a nested guest running > > > > > > * Fixes for LBR virtualizations when a nested guest is running, > > > and nested LBR virtualization support > > > > > > * PAUSE filtering for nested hypervisors > > > > > > Guest support: > > > > > > * Decoupling of vcpu_is_preempted from PV spinlocks > > > > > > Signed-off-by: Paolo Bonzini > > > > > > I am still working on the bisect into the merge commits. > > > > > > Regards, > > > Suravee > > > > > What happens when the guest can't launch? It sure works for me for kvm/queue > > from yesterday. > > > > I'll test again tomorrow. > > I have bisected it to this commit: > > commit 74fd41ed16fd71725e69e2cb90b755505326c2e6 > Author: Maxim Levitsky > Date: Tue Mar 22 19:40:47 2022 +0200 > > KVM: x86: nSVM: support PAUSE filtering when L0 doesn't intercept PAUSE > > Expose the pause filtering and threshold in the guest CPUID > and support PAUSE filtering when possible: > > - If the L0 doesn't intercept PAUSE (cpu_pm=on), then allow L1 to > have full control over PAUSE filtering. > > - if the L1 doesn't intercept PAUSE, use host values and update > the adaptive count/threshold even when running nested. > > - Otherwise always exit to L1; it is not really possible to merge > the fields correctly. It is expected that in this case, userspace > will not enable this feature in the guest CPUID, to avoid having the > guest update both fields pointlessly. > > Signed-off-by: Maxim Levitsky > Message-Id: <20220322174050.241850-4-mlevitsk@redhat.com> > Signed-off-by: Paolo Bonzini > > I can revert this one or specify pause_filter_count=0 pause_filter_thresh=0, > and then I can boot the L2 guest. > > Regards, > Suravee > This is really wierd. Could you share the qemu command line for L1 and L2 guest, and as much as possible info on what happens when you boot L2? I tested latest kvm/queue and I don't see any issues with booting nested guest. Which hardware you test on? I test on Zen2 (3970X) mostly. How many vCPUs L2 has? Could you do a kvm trace of the L2, from L1, to see what it does prior to hang? Best regards, Maxim Levitsky