Linux-EDAC Archive on lore.kernel.org
 help / color / Atom feed
From: Sean Christopherson <sean.j.christopherson@intel.com>
To: Jacob Keller <jacob.e.keller@intel.com>
Cc: TonyWWang-oc@zhaoxin.com, acme@kernel.org,
	alexander.shishkin@linux.intel.com, bp@alien8.de, bp@suse.de,
	hpa@zytor.com, jacob.jun.pan@linux.intel.com,
	jarkko.sakkinen@linux.intel.com, jmattson@google.com,
	jolsa@redhat.com, joro@8bytes.org, kvm@vger.kernel.org,
	lenb@kernel.org, linux-edac@vger.kernel.org,
	linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org,
	linux-pm@vger.kernel.org, mark.rutland@arm.com, mingo@redhat.com,
	namhyung@kernel.org, pbonzini@redhat.com, peterz@infradead.org,
	rkrcmar@redhat.com, shuah@kernel.org, tglx@linutronix.de,
	tony.luck@intel.com, vkuznets@redhat.com, wanpengli@tencent.com,
	x86@kernel.org
Subject: Re: [PATCH v5 13/19] x86/cpufeatures: Add flag to track whether MSR IA32_FEAT_CTL is configured
Date: Tue, 25 Feb 2020 14:12:34 -0800
Message-ID: <20200225221234.GL9245@linux.intel.com> (raw)
In-Reply-To: <e741196d-52aa-0f5e-8f1e-a37ddf2e5025@intel.com>

On Tue, Feb 25, 2020 at 01:49:13PM -0800, Jacob Keller wrote:
> Hi Sean,
> 
> > Add a new feature flag, X86_FEATURE_MSR_IA32_FEAT_CTL, to track whether
> > IA32_FEAT_CTL has been initialized.  This will allow KVM, and any future
> > subsystems that depend on IA32_FEAT_CTL, to rely purely on cpufeatures
> > to query platform support, e.g. allows a future patch to remove KVM's
> > manual IA32_FEAT_CTL MSR checks.
> > 
> > Various features (on platforms that support IA32_FEAT_CTL) are dependent
> > on IA32_FEAT_CTL being configured and locked, e.g. VMX and LMCE.  The
> > MSR is always configured during boot, but only if the CPU vendor is
> > recognized by the kernel.  Because CPUID doesn't incorporate the current
> > IA32_FEAT_CTL value in its reporting of relevant features, it's possible
> > for a feature to be reported as supported in cpufeatures but not truly
> > enabled, e.g. if the CPU supports VMX but the kernel doesn't recognize
> > the CPU.
> > 
> > As a result, without the flag, KVM would see VMX as supported even if
> > IA32_FEAT_CTL hasn't been initialized, and so would need to manually
> > read the MSR and check the various enabling bits to avoid taking an
> > unexpected #GP on VMXON.
> 
> 
> I recently ran into a general protection fault that I believe is the
> fault of this patch:
> 
> > [   32.189584] general protection fault, maybe for address 0xffffb567801bcf58: 0000 [#1] SMP PTI
> > [   32.198103] CPU: 1 PID: 2600 Comm: rngd Not tainted 5.6.0-rc2-jk+ #2
> > [   32.204454] Hardware name: Intel Corporation S2600STQ/S2600STQ, BIOS SE5C620.86B.02.01.0008.031920191559 03/19/2019
> > [   32.214887] RIP: 0010:hardware_enable+0x100/0x1a0 [kvm_intel]
> > [   32.220628] Code: 00 00 48 39 f8 74 0f 65 48 89 3d 43 a2 cb 3c e8 66 d3 cc c5 66 90 48 89 df 57 9d 0f 1f 44 00 00 bf 01 00 00 00 e8 90 3d ca c5 <f3> 0f c7 34 24 31 c0 80 3d 59 8d 03 00 00 75 36 48 8b 5c 24 10 65
> > [   32.239373] RSP: 0000:ffffb567801bcf58 EFLAGS: 00010002
> > [   32.244598] RAX: 0000000000300000 RBX: 0000000000000086 RCX: ffff8f2650440000
> > [   32.251730] RDX: 0000000000300000 RSI: 0000000000000000 RDI: ffff8f2650457020
> > [   32.258862] RBP: 0000000000000007 R08: 000000077ea5d531 R09: 0000000000000000
> > [   32.265986] R10: 000001432bf20982 R11: 0000000000000000 R12: ffffd55b80467110
> > [   32.273118] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> > [   32.280243] FS:  00007facfe66f700(0000) GS:ffff8f2650440000(0000) knlGS:0000000000000000
> > [   32.288329] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [   32.294077] CR2: 00007facf0003000 CR3: 0000000b7d402006 CR4: 00000000007626e0
> > [   32.301210] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > [   32.308342] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> > [   32.315474] PKRU: 55555554
> > [   32.318186] Call Trace:
> > [   32.320642]  <IRQ>
> > [   32.322689]  kvm_arch_hardware_enable+0x84/0x240 [kvm]
> > [   32.327836]  hardware_enable_nolock+0x31/0x60 [kvm]
> > [   32.332717]  flush_smp_call_function_queue+0x4d/0xe0
> > [   32.337683]  smp_call_function_interrupt+0x3a/0xd0
> > [   32.342471]  call_function_interrupt+0xf/0x20
> > [   32.346830]  </IRQ>
> > [   32.348935] RIP: 0033:0x7facffd4c753
> > [   32.352514] Code: e8 48 c7 45 e0 00 00 00 00 eb 5f 48 8b 45 c8 48 8b 50 38 48 8b 45 c8 8b 40 40 89 c0 48 01 d0 48 89 45 f0 48 8b 45 f0 0f b6 00 <83> c0 01 89 c2 48 8b 45 f0 88 10 48 8b 45 c8 8b 50 40 48 8b 45 c8
> > [   32.371263] RSP: 002b:00007facfe66ebf0 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff03
> > [   32.378826] RAX: 00000000000000ee RBX: 0000000000004097 RCX: 0000000000000000
> > [   32.385961] RDX: 0000562781dbadf0 RSI: 0000000000000000 RDI: 00007ffd7edf9080
> > [   32.393092] RBP: 00007facfe66ec30 R08: 00007ffd7edf9080 R09: 000000000000cd4a
> > [   32.400226] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
> > [   32.407358] R13: 00007facf0000b20 R14: 0000562781dba2e8 R15: 00007facfe66ed10
> > [   32.414493] Modules linked in: ip6table_mangle ip6table_nat iptable_mangle iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rfkill ib_isert iscsi
> > _target_mod ib_srpt target_core_mod ib_srp scsi_transport_srp ib_ipoib vfat fat ib_umad rpcrdma sunrpc intel_rapl_msr intel_rapl_common rdma_ucm ib_iser rdma_cm isst_if_common iw_cm ib_cm libiscsi skx_edac scsi_transport_iscsi nfit libnv
> > dimm x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel intel_cstate i40iw qat_c62x iTCO_wdt ipmi_ssif iTCO_vendor_support ib_uverbs mei_me intel_qat intel_uncore ib_c
> > ore joydev intel_rapl_perf pcspkr ipmi_si authenc ioatdma mei i2c_i801 lpc_ich dca ipmi_devintf ipmi_msghandler acpi_power_meter acpi_pad ip_tables ast i2c_algo_bit drm_vram_helper drm_ttm_helper ttm drm_kms_helper cec drm ice i40e crc32
> > c_intel wmi fuse
> > [   32.498314] ---[ end trace bfeeeba337a01208 ]---
> 
> I noticed that a slightly older commit from before this does not fail.
> Additionally, the system reports the following during boot:
> 
> kvm: disabled by bios
> 
> I looked into the vmx_disabled_by_bios and noticed that it checks for
> both X86_FEATURE_MSR_IA32_FEAT_CTL and X86_FEATURE_VMX.
> 
> Compared to the older code before commit a4d0b2fdbcf7 ("KVM: VMX: Use
> VMX feature flag to query BIOS enabling") it's not clear to me how
> exactly this could fail to match up.
> 
> I suspect something is wrong and the features are enabled even though
> the BIOS has it disabled, leading to later failure because of this.

Hrm.  On the failing kernel, what are the values of MSR 0x3a for all CPUs,
i.e. what's the output of 'sudo rdmsr -a 0x3a'?

  reply index

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-21  4:44 [PATCH v5 00/19] x86/cpu: Clean up handling of VMX features Sean Christopherson
2019-12-21  4:44 ` [PATCH v5 01/19] x86/msr-index: Clean up bit defines for IA32_FEATURE_CONTROL MSR Sean Christopherson
2019-12-21  4:44 ` [PATCH v5 02/19] selftests: kvm: Replace manual MSR defs with common msr-index.h Sean Christopherson
2019-12-21  4:44 ` [PATCH v5 03/19] tools arch x86: Sync msr-index.h from kernel sources Sean Christopherson
2019-12-21  4:44 ` [PATCH v5 04/19] x86/intel: Initialize IA32_FEAT_CTL MSR at boot Sean Christopherson
2019-12-21  4:44 ` [PATCH v5 05/19] x86/mce: WARN once if IA32_FEAT_CTL MSR is left unlocked Sean Christopherson
2019-12-21  4:45 ` [PATCH v5 06/19] x86/centaur: Use common IA32_FEAT_CTL MSR initialization Sean Christopherson
2019-12-21  4:45 ` [PATCH v5 07/19] x86/zhaoxin: " Sean Christopherson
2019-12-21  4:45 ` [PATCH v5 08/19] x86/cpu: Clear VMX feature flag if VMX is not fully enabled Sean Christopherson
2019-12-21  4:45 ` [PATCH v5 09/19] x86/vmx: Introduce VMX_FEATURES_* Sean Christopherson
2019-12-21  4:45 ` [PATCH v5 10/19] x86/cpu: Detect VMX features on Intel, Centaur and Zhaoxin CPUs Sean Christopherson
2019-12-21  4:45 ` [PATCH v5 11/19] x86/cpu: Print VMX flags in /proc/cpuinfo using VMX_FEATURES_* Sean Christopherson
2019-12-21  4:45 ` [PATCH v5 12/19] x86/cpu: Set synthetic VMX cpufeatures during init_ia32_feat_ctl() Sean Christopherson
2019-12-21  4:45 ` [PATCH v5 13/19] x86/cpufeatures: Add flag to track whether MSR IA32_FEAT_CTL is configured Sean Christopherson
2020-02-25 21:49   ` Jacob Keller
2020-02-25 22:12     ` Sean Christopherson [this message]
2020-02-25 22:52       ` Jacob Keller
2020-02-25 23:29         ` Sean Christopherson
2020-02-25 23:35           ` Jacob Keller
2020-02-25 23:54           ` Jacob Keller
2020-02-26  0:41             ` Jacob Keller
2020-02-26  0:42             ` Sean Christopherson
2020-02-26  0:58               ` Jacob Keller
2020-02-26 20:41                 ` Jacob Keller
2020-02-26 20:57                   ` Sean Christopherson
2020-02-26 21:03                     ` Jacob Keller
2020-02-26 21:25                       ` Sean Christopherson
2020-02-26 21:53                         ` Jacob Keller
2020-02-27  2:12     ` Sean Christopherson
2020-02-27  4:20       ` Huang, Kai
2020-02-27 18:09       ` Jacob Keller
2019-12-21  4:45 ` [PATCH v5 14/19] KVM: VMX: Drop initialization of IA32_FEAT_CTL MSR Sean Christopherson
2019-12-21  4:45 ` [PATCH v5 15/19] KVM: VMX: Use VMX feature flag to query BIOS enabling Sean Christopherson
2019-12-21  4:45 ` [PATCH v5 16/19] KVM: VMX: Check for full VMX support when verifying CPU compatibility Sean Christopherson
2019-12-21  4:45 ` [PATCH v5 17/19] KVM: VMX: Use VMX_FEATURE_* flags to define VMCS control bits Sean Christopherson
2020-01-13 18:32   ` Borislav Petkov
2020-01-13 18:37     ` Sean Christopherson
2020-01-13 18:38       ` Borislav Petkov
2020-01-13 18:42         ` Sean Christopherson
2020-01-13 18:52           ` [PATCH] KVM: VMX: Rename define to CPU_BASED_USE_TSC_OFFSETTING Borislav Petkov
2020-01-13 20:16             ` Sean Christopherson
2020-01-14  9:31               ` Borislav Petkov
2020-01-14 17:27                 ` Sean Christopherson
2019-12-21  4:45 ` [PATCH v5 18/19] perf/x86: Provide stubs of KVM helpers for non-Intel CPUs Sean Christopherson
2019-12-21  4:45 ` [PATCH v5 19/19] KVM: VMX: Allow KVM_INTEL when building for Centaur and/or Zhaoxin CPUs Sean Christopherson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200225221234.GL9245@linux.intel.com \
    --to=sean.j.christopherson@intel.com \
    --cc=TonyWWang-oc@zhaoxin.com \
    --cc=acme@kernel.org \
    --cc=alexander.shishkin@linux.intel.com \
    --cc=bp@alien8.de \
    --cc=bp@suse.de \
    --cc=hpa@zytor.com \
    --cc=jacob.e.keller@intel.com \
    --cc=jacob.jun.pan@linux.intel.com \
    --cc=jarkko.sakkinen@linux.intel.com \
    --cc=jmattson@google.com \
    --cc=jolsa@redhat.com \
    --cc=joro@8bytes.org \
    --cc=kvm@vger.kernel.org \
    --cc=lenb@kernel.org \
    --cc=linux-edac@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=linux-pm@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=mingo@redhat.com \
    --cc=namhyung@kernel.org \
    --cc=pbonzini@redhat.com \
    --cc=peterz@infradead.org \
    --cc=rkrcmar@redhat.com \
    --cc=shuah@kernel.org \
    --cc=tglx@linutronix.de \
    --cc=tony.luck@intel.com \
    --cc=vkuznets@redhat.com \
    --cc=wanpengli@tencent.com \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-EDAC Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-edac/0 linux-edac/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-edac linux-edac/ https://lore.kernel.org/linux-edac \
		linux-edac@vger.kernel.org
	public-inbox-index linux-edac

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-edac


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git