From: Vitaly Kuznetsov <vkuznets@redhat.com>
To: Roman Kagan <rkagan@virtuozzo.com>
Cc: "kvm\@vger.kernel.org" <kvm@vger.kernel.org>,
Michael Kelley <mikelley@microsoft.com>,
Lan Tianyu <Tianyu.Lan@microsoft.com>,
Joerg Roedel <jroedel@suse.de>,
"K. Y. Srinivasan" <kys@microsoft.com>,
Haiyang Zhang <haiyangz@microsoft.com>,
Stephen Hemminger <sthemmin@microsoft.com>,
Sasha Levin <sashal@kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
"H. Peter Anvin" <hpa@zytor.com>,
"x86\@kernel.org" <x86@kernel.org>,
"linux-hyperv\@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
"linux-kernel\@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v2] x86/hyperv: make vapic support x2apic mode
Date: Thu, 03 Oct 2019 12:54:03 +0200 [thread overview]
Message-ID: <87muei14ms.fsf@vitty.brq.redhat.com> (raw)
In-Reply-To: <20191002101923.4981-1-rkagan@virtuozzo.com>
Roman Kagan <rkagan@virtuozzo.com> writes:
> Now that there's Hyper-V IOMMU driver, Linux can switch to x2apic mode
> when supported by the vcpus.
>
> However, the apic access functions for Hyper-V enlightened apic assume
> xapic mode only.
>
> As a result, Linux fails to bring up secondary cpus when run as a guest
> in QEMU/KVM with both hv_apic and x2apic enabled.
>
> I didn't manage to make my instance of Hyper-V expose x2apic to the
> guest; nor does Hyper-V spec document the expected behavior. However,
> a Windows guest running in QEMU/KVM with hv_apic and x2apic and a big
> number of vcpus (so that it turns on x2apic mode) does use enlightened
> apic MSRs passing unshifted 32bit destination id and falls back to the
> regular x2apic MSRs for less frequently used apic fields.
>
> So implement the same behavior, by replacing enlightened apic access
> functions (only those where it makes a difference) with their
> x2apic-aware versions when x2apic is in use.
>
> Fixes: 29217a474683 ("iommu/hyper-v: Add Hyper-V stub IOMMU driver")
> Fixes: 6b48cb5f8347 ("X86/Hyper-V: Enlighten APIC access")
> Cc: stable@vger.kernel.org
> Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
> ---
> v1 -> v2:
> - add ifdefs to handle !CONFIG_X86_X2APIC
>
> arch/x86/hyperv/hv_apic.c | 54 ++++++++++++++++++++++++++++++++++++---
> 1 file changed, 51 insertions(+), 3 deletions(-)
>
> diff --git a/arch/x86/hyperv/hv_apic.c b/arch/x86/hyperv/hv_apic.c
> index 5c056b8aebef..eb1434ae9e46 100644
> --- a/arch/x86/hyperv/hv_apic.c
> +++ b/arch/x86/hyperv/hv_apic.c
> @@ -84,6 +84,44 @@ static void hv_apic_write(u32 reg, u32 val)
> }
> }
>
> +#ifdef CONFIG_X86_X2APIC
> +static void hv_x2apic_icr_write(u32 low, u32 id)
> +{
> + wrmsr(HV_X64_MSR_ICR, low, id);
> +}
AFAIU you're trying to mirror native_x2apic_icr_write() here but this is
different from what hv_apic_icr_write() does
(SET_APIC_DEST_FIELD(id)). Is it actually correct? (I think you've
tested this and it is but) Michael, could you please shed some light
here?
> +
> +static u32 hv_x2apic_read(u32 reg)
> +{
> + u32 reg_val, hi;
> +
> + switch (reg) {
> + case APIC_EOI:
> + rdmsr(HV_X64_MSR_EOI, reg_val, hi);
> + return reg_val;
> + case APIC_TASKPRI:
> + rdmsr(HV_X64_MSR_TPR, reg_val, hi);
> + return reg_val;
> +
> + default:
> + return native_apic_msr_read(reg);
> + }
> +}
> +
> +static void hv_x2apic_write(u32 reg, u32 val)
> +{
> + switch (reg) {
> + case APIC_EOI:
> + wrmsr(HV_X64_MSR_EOI, val, 0);
> + break;
> + case APIC_TASKPRI:
> + wrmsr(HV_X64_MSR_TPR, val, 0);
> + break;
> + default:
> + native_apic_msr_write(reg, val);
> + }
> +}
> +#endif /* CONFIG_X86_X2APIC */
> +
> static void hv_apic_eoi_write(u32 reg, u32 val)
> {
> struct hv_vp_assist_page *hvp = hv_vp_assist_page[smp_processor_id()];
> @@ -262,9 +300,19 @@ void __init hv_apic_init(void)
> if (ms_hyperv.hints & HV_X64_APIC_ACCESS_RECOMMENDED) {
> pr_info("Hyper-V: Using MSR based APIC access\n");
> apic_set_eoi_write(hv_apic_eoi_write);
> - apic->read = hv_apic_read;
> - apic->write = hv_apic_write;
> - apic->icr_write = hv_apic_icr_write;
> +#ifdef CONFIG_X86_X2APIC
> + if (x2apic_enabled()) {
> + apic->read = hv_x2apic_read;
> + apic->write = hv_x2apic_write;
> + apic->icr_write = hv_x2apic_icr_write;
> + } else {
> +#endif
> + apic->read = hv_apic_read;
> + apic->write = hv_apic_write;
> + apic->icr_write = hv_apic_icr_write;
(just wondering): Is it always safe to assume that we cannot switch
between apic_flat/x2apic in runtime? Moreover, the only difference
between hv_apic_read/hv_apic_write and hv_x2apic_read/hv_x2apic_write is
native_apic_mem_{read,write} -> native_apic_msr_{read,write}. Would it
make sense to move if (x2apic_enabled()) and merge these functions?
> +#ifdef CONFIG_X86_X2APIC
> + }
> +#endif
> apic->icr_read = hv_apic_icr_read;
> }
> }
--
Vitaly
next prev parent reply other threads:[~2019-10-03 10:54 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-02 10:19 [PATCH v2] x86/hyperv: make vapic support x2apic mode Roman Kagan
2019-10-03 10:54 ` Vitaly Kuznetsov [this message]
2019-10-03 12:52 ` Roman Kagan
2019-10-04 3:01 ` Michael Kelley
2019-10-04 9:18 ` Roman Kagan
2019-10-04 22:33 ` Michael Kelley
2019-10-08 20:54 ` Michael Kelley
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87muei14ms.fsf@vitty.brq.redhat.com \
--to=vkuznets@redhat.com \
--cc=Tianyu.Lan@microsoft.com \
--cc=bp@alien8.de \
--cc=haiyangz@microsoft.com \
--cc=hpa@zytor.com \
--cc=jroedel@suse.de \
--cc=kvm@vger.kernel.org \
--cc=kys@microsoft.com \
--cc=linux-hyperv@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mikelley@microsoft.com \
--cc=mingo@redhat.com \
--cc=rkagan@virtuozzo.com \
--cc=sashal@kernel.org \
--cc=sthemmin@microsoft.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).