RE: [PATCH v7 07/10] arm64: hyperv: Initialize hypervisor on boot

From: Michael Kelley <mikelley@microsoft.com>
To: Arnd Bergmann <arnd@arndb.de>
Cc: Will Deacon <will@kernel.org>, Ard Biesheuvel <ardb@kernel.org>,
	Catalin Marinas <catalin.marinas@arm.com>,
	"Mark.Rutland@arm.com" <Mark.Rutland@arm.com>,
	Marc Zyngier <maz@kernel.org>,
	Linux ARM <linux-arm-kernel@lists.infradead.org>,
	gregkh <gregkh@linuxfoundation.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	linux-efi <linux-efi@vger.kernel.org>,
	linux-arch <linux-arch@vger.kernel.org>,
	"wei.liu@kernel.org" <wei.liu@kernel.org>,
	vkuznets <vkuznets@redhat.com>, KY Srinivasan <kys@microsoft.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>,
	Boqun Feng <boqun.feng@gmail.com>
Subject: RE: [PATCH v7 07/10] arm64: hyperv: Initialize hypervisor on boot
Date: Tue, 25 Aug 2020 21:20:04 +0000	[thread overview]
Message-ID: <MW2PR2101MB1052AD42CC4F9A71F87EFE95D7570@MW2PR2101MB1052.namprd21.prod.outlook.com> (raw)
In-Reply-To: <CAK8P3a1NXVJON+apBZeVDdx_bqQmenab8srqJDWS_VFVpAncRA@mail.gmail.com>

From: Arnd Bergmann <arnd@arndb.de> Sent: Monday, August 24, 2020 11:34 AM
> 
> On Mon, Aug 24, 2020 at 6:48 PM Michael Kelley <mikelley@microsoft.com> wrote:
> >
> >  /*
> > + * This function is invoked via the ACPI clocksource probe mechanism. We
> > + * don't actually use any values from the ACPI GTDT table, but we set up
> > + * the Hyper-V synthetic clocksource and do other initialization for
> > + * interacting with Hyper-V the first time.  Using early_initcall to invoke
> > + * this function is too late because interrupts are already enabled at that
> > + * point, and hv_init_clocksource() must run before interrupts are enabled.
> > + *
> > + * 1. Setup the guest ID.
> > + * 2. Get features and hints info from Hyper-V
> > + * 3. Setup per-cpu VP indices.
> > + * 4. Initialize the Hyper-V clocksource.
> > + */
> > +
> > +static int __init hyperv_init(struct acpi_table_header *table)
> > +{
> > +       struct hv_get_vp_registers_output result;
> > +       u32     a, b, c, d;
> > +       u64     guest_id;
> > +       int     i, cpuhp;
> > +
> > +       /*
> > +        * If we're in a VM on Hyper-V, the ACPI hypervisor_id field will
> > +        * have the string "MsHyperV".
> > +        */
> > +       if (strncmp((char *)&acpi_gbl_FADT.hypervisor_id, "MsHyperV", 8))
> > +               return -EINVAL;
> > +
> > +       /* Setup the guest ID */
> > +       guest_id = generate_guest_id(0, LINUX_VERSION_CODE, 0);
> > +       hv_set_vpreg(HV_REGISTER_GUEST_OSID, guest_id);
> > +
> > +       /* Get the features and hints from Hyper-V */
> > +       hv_get_vpreg_128(HV_REGISTER_FEATURES, &result);
> > +       ms_hyperv.features = result.as32.a;
> > +       ms_hyperv.misc_features = result.as32.c;
> > +
> > +       hv_get_vpreg_128(HV_REGISTER_ENLIGHTENMENTS, &result);
> > +       ms_hyperv.hints = result.as32.a;
> > +
> > +       pr_info("Hyper-V: Features 0x%x, hints 0x%x, misc 0x%x\n",
> > +               ms_hyperv.features, ms_hyperv.hints, ms_hyperv.misc_features);
> > +
> > +       /*
> > +        * If Hyper-V has crash notifications, set crash_kexec_post_notifiers
> > +        * so that we will report the panic to Hyper-V before running kdump.
> > +        */
> > +       if (ms_hyperv.misc_features & HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE)
> > +               crash_kexec_post_notifiers = true;
> > +
> > +       /* Get information about the Hyper-V host version */
> > +       hv_get_vpreg_128(HV_REGISTER_HYPERVISOR_VERSION, &result);
> > +       a = result.as32.a;
> > +       b = result.as32.b;
> > +       c = result.as32.c;
> > +       d = result.as32.d;
> > +       pr_info("Hyper-V: Host Build %d.%d.%d.%d-%d-%d\n",
> > +               b >> 16, b & 0xFFFF, a, d & 0xFFFFFF, c, d >> 24);
> > +
> > +       /* Allocate and initialize percpu VP index array */
> > +       hv_vp_index = kmalloc_array(num_possible_cpus(), sizeof(*hv_vp_index),
> > +                                   GFP_KERNEL);
> > +       if (!hv_vp_index)
> > +               return -ENOMEM;
> > +
> > +       for (i = 0; i < num_possible_cpus(); i++)
> > +               hv_vp_index[i] = VP_INVAL;
> > +
> > +       cpuhp = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
> > +                       "arm64/hyperv_init:online", hv_cpu_init, NULL);
> > +       if (cpuhp < 0)
> > +               goto free_vp_index;
> > +
> > +       hv_init_clocksource();
> > +       if (hv_stimer_alloc())
> > +               goto remove_cpuhp_state;
> > +
> > +       hyperv_initialized = true;
> > +       return 0;
> > +
> > +remove_cpuhp_state:
> > +       cpuhp_remove_state(cpuhp);
> > +free_vp_index:
> > +       kfree(hv_vp_index);
> > +       hv_vp_index = NULL;
> > +       return -EINVAL;
> > +}
> > +TIMER_ACPI_DECLARE(hyperv, ACPI_SIG_GTDT, hyperv_init);
> 
> I think this has come up before, and I still don't consider it an acceptable
> hack to hook platform initialization code into the timer code.
> 
> Please split out the timer into a standalone driver in drivers/clocksource
> that can get reviewed by the clocksource maintainers.

I see two related topics here.  First, the Hyper-V clocksource driver is
drivers/clocksource/hyperv_timer.c.  The code is architecture independent
and is used today on the x86 side and for ARM64 in this patch series.  A few
architecture specific calls are satisfied by code under arch/x86, and in this
patch series, under arch/arm64.  Is there some aspect of this driver that
needs reconsideration?  I just want to make sure to understand what you
are getting at.

Second is the question of where/how to do Hyper-V specific initialization.
I agree that hanging it off the timer initialization isn't a great approach.
Should I add a Hyper-V specific initialization call at the appropriate point
in the ARM64 init sequence?  The x86 side has some structure for handling
multiple hypervisors, and the Hyper-V initialization code naturally plugs into
that structure.  I'm certainly open to suggestions on the best way to handle
it for ARM64.

Michael

> 
>       Arnd