From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?UTF-8?B?QW5kcmVhcyBGw6RyYmVy?= Subject: Re: [Qemu-devel] -cpu host (was Re: KVM call minutes for 2013-08-06) Date: Thu, 08 Aug 2013 17:55:09 +0200 Message-ID: <5203BF5D.4030007@suse.de> References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: quintela@redhat.com, qemu-devel , KVM devel mailing list , kvmarm@lists.cs.columbia.edu To: Peter Maydell Return-path: Received: from cantor2.suse.de ([195.135.220.15]:49037 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S965398Ab3HHPzO (ORCPT ); Thu, 8 Aug 2013 11:55:14 -0400 In-Reply-To: Sender: kvm-owner@vger.kernel.org List-ID: Hi Peter, Am 08.08.2013 14:51, schrieb Peter Maydell: > [I missed this KVM call but the stuff about -cpu host ties into > an issue we've been grappling with for ARM KVM, so it seems > a reasonable jumping-off-point.] >=20 > On 6 August 2013 16:15, Juan Quintela wrote: >> 2013-08-06 >> ---------- >> >> What libvirt needs/miss Today? >> - how to handle machine types? creating them inside qemu? >> - qemu --cpu help >> only shows cpus, not what features qemu will use >> - qemu -cpu host >> what does this exactly means? kvm removes same flags. >> - Important to know if migration would work. >> - Machine types sometimes disable some feature, so cpu alone is not >> enough. >=20 >> - kernel removes some features because it knows it can't be virtuali= sed >> - qemu adds some others because it knows it don't need host support >> - and then lots of features in the middle >=20 > So, coming at this from an ARM perspective: > Should any target arch that supports KVM also support "-cpu host"? > If so, what should it do? I think that depends on the target and whether/what is useful. > Is there a description somewhere of > what the x86 and PPC semantics of -cpu host are? I'm afraid our usual documentation will be reading the source code. ;) x86 was first to implement -cpu host and passed through pretty much all host features even if they would not work without additional support code. I've seen a bunch of bugs where that leads to GMP and others breaking badly. Lately in the case of PMU we've started to limit that. Alex proposed -cpu best, which was never merged to date. It was similar to how ppc's -cpu host works: ppc matches the Processor Version Register (PVR) in kvm.c against its known models from cpu-models.c (strictly today, mask being discussed). The PVR can be read from userspace via mfpvr alias to mfspr (Move From Special Purpose Register; possibly emulated for userspace by kernel?). CPU features are all QEMU-driven AFAIU, through the "CPU families" in translate_init.c. Beware, everything is highly macro'fied in ppc code. > For ARM you can't get at feature info of the host from userspace > (unless you want to get into parsing /proc/cpuinfo), so my current > idea is to have KVM_ARM_VCPU_INIT support a target-cpu-type > which means "whatever host CPU is". Then when we've created the > vcpu we can populate QEMU's idea of what the CPU features are > by using the existing ioctls for reading the cp15 registers of > the vcpu. Sounds sane to me iff those cp15 registers all work with KVM and don't need any additional KVM/QEMU/device code. > The other unresolved thing is what "-cpu host" ought to mean > for the CPU's on-chip peripherals (of which the major one is > the interrupt controller) -- if the host is an A57 should > this imply that you always get the A57's GICv3, or is it OK > to provide an A57 with a GICv2? At the moment QEMU models the > per-cpu peripherals in a somewhat more semi-detached fashion > than is the case in silicon, treating them as more a part > of the board model than of the cpu itself. =46eel free to submit patches changing that. Prerequisite should then b= e to have those devices be pure TYPE_DEVICE rather than TYPE_SYS_BUS_DEVICE, or otherwise you'll run into the same hot-plug tra= p as we did with the x86 APIC (we had to invent a hotpluggable ICC bus as interim solution). > Having '-cpu host' > not affect them might be the pragmatic choice, since it fits > with what QEMU currently does and with kernel-side situations > where the host CPU may only be able to show the guest VM a > GICv2 view of the world (or only a GICv3, as the case may be). > For this to work it does require that guests figure out what > their per-cpu peripherals are by looking at the device tree > rather than saying "oh, this is an A57, I know all A57s > have this", of course... Without directly answering the question and continuing from above, my personal view has been that we need to get away from the current CPU model to a) how hardware is structured and b) how we want to have thing= s behave in virtualized environments. Take x86 as an example: CPUState corresponds to a hyperthread today, bu= t we want hotplug to work like it does on a physical machine: hot-adding on socket-level only. Beyond just building the topology with Container objects, that means having a Xeon-X5-4242 object that has-a CPU core has-a CPU thread and any devices the particular layers bring along. =46or SoCs I have been proposing - for sh7750 and lately tegra2 - to mo= del "the black chip on the board" as a TYPE_DEVICE for encapsulation across boards. Meaning the GIC would no longer be instantiated on the board bu= t as part of an object, and -smp and -cpu would as a consequence loose in influence. We could interpret -cpu host as instantiate the host's SoC object. But the mainstream SoC for KVM virtualization is exynos5, and no one sat down to model exynos5 in QEMU so far, so that would be moot. Versatile Express is rather unlikely to match the host environment KVM is used in= , and when using Soft Macros (or what ARM calls their FPGA-based emulation) then things get fuzzy anyway. Similar problem for CPU hotplug: there is no real match in physical ARM hardware that we can copy for KVM/QEMU. It's all mixed in one chip wher= e we can only enable/disable things via MMIO in physical reality. You recently proposed to have the CPUs in the a*mpcore_priv object, which also happens to own the GIC. Having the CPU model be a property o= f a*mpcore would complicate a lot of things QOM-wise but for the question at hand would allow to exchange the GIC based on CPU model, so I'm undecided. a9mpcore_priv with a cortex-a15 doesn't make much sense though, given there's a15mpcore_priv with different amount of IRQs and less/different child devices. Given that ARM SoCs are much less standardized then x86 PCs, I would conclude that passing random CPUs into a board/SoC does not make sense and should at least be limited to known-good combinations such as Cortex-A7 vs. Cortex-A15. If someone wants to experiment with modifications of SoCs/boards then we should rather provide more easy ways to let someone derive a custom board config then pretending that the user can just set a magical command line argument to cherry-pick parts of the system. Regards, Andreas --=20 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 N=C3=BCrnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imend=C3=B6rffer; HRB 16746 AG N=C3= =BCrnberg From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56258) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V7SYb-00083c-Dr for qemu-devel@nongnu.org; Thu, 08 Aug 2013 11:55:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1V7SYU-0001l6-3V for qemu-devel@nongnu.org; Thu, 08 Aug 2013 11:55:21 -0400 Received: from cantor2.suse.de ([195.135.220.15]:49034 helo=mx2.suse.de) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V7SYT-0001ku-Nb for qemu-devel@nongnu.org; Thu, 08 Aug 2013 11:55:14 -0400 Message-ID: <5203BF5D.4030007@suse.de> Date: Thu, 08 Aug 2013 17:55:09 +0200 From: =?UTF-8?B?QW5kcmVhcyBGw6RyYmVy?= MIME-Version: 1.0 References: In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] -cpu host (was Re: KVM call minutes for 2013-08-06) List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Peter Maydell Cc: kvmarm@lists.cs.columbia.edu, qemu-devel , KVM devel mailing list , quintela@redhat.com Hi Peter, Am 08.08.2013 14:51, schrieb Peter Maydell: > [I missed this KVM call but the stuff about -cpu host ties into > an issue we've been grappling with for ARM KVM, so it seems > a reasonable jumping-off-point.] >=20 > On 6 August 2013 16:15, Juan Quintela wrote: >> 2013-08-06 >> ---------- >> >> What libvirt needs/miss Today? >> - how to handle machine types? creating them inside qemu? >> - qemu --cpu help >> only shows cpus, not what features qemu will use >> - qemu -cpu host >> what does this exactly means? kvm removes same flags. >> - Important to know if migration would work. >> - Machine types sometimes disable some feature, so cpu alone is not >> enough. >=20 >> - kernel removes some features because it knows it can't be virtualise= d >> - qemu adds some others because it knows it don't need host support >> - and then lots of features in the middle >=20 > So, coming at this from an ARM perspective: > Should any target arch that supports KVM also support "-cpu host"? > If so, what should it do? I think that depends on the target and whether/what is useful. > Is there a description somewhere of > what the x86 and PPC semantics of -cpu host are? I'm afraid our usual documentation will be reading the source code. ;) x86 was first to implement -cpu host and passed through pretty much all host features even if they would not work without additional support code. I've seen a bunch of bugs where that leads to GMP and others breaking badly. Lately in the case of PMU we've started to limit that. Alex proposed -cpu best, which was never merged to date. It was similar to how ppc's -cpu host works: ppc matches the Processor Version Register (PVR) in kvm.c against its known models from cpu-models.c (strictly today, mask being discussed). The PVR can be read from userspace via mfpvr alias to mfspr (Move From Special Purpose Register; possibly emulated for userspace by kernel?). CPU features are all QEMU-driven AFAIU, through the "CPU families" in translate_init.c. Beware, everything is highly macro'fied in ppc code. > For ARM you can't get at feature info of the host from userspace > (unless you want to get into parsing /proc/cpuinfo), so my current > idea is to have KVM_ARM_VCPU_INIT support a target-cpu-type > which means "whatever host CPU is". Then when we've created the > vcpu we can populate QEMU's idea of what the CPU features are > by using the existing ioctls for reading the cp15 registers of > the vcpu. Sounds sane to me iff those cp15 registers all work with KVM and don't need any additional KVM/QEMU/device code. > The other unresolved thing is what "-cpu host" ought to mean > for the CPU's on-chip peripherals (of which the major one is > the interrupt controller) -- if the host is an A57 should > this imply that you always get the A57's GICv3, or is it OK > to provide an A57 with a GICv2? At the moment QEMU models the > per-cpu peripherals in a somewhat more semi-detached fashion > than is the case in silicon, treating them as more a part > of the board model than of the cpu itself. Feel free to submit patches changing that. Prerequisite should then be to have those devices be pure TYPE_DEVICE rather than TYPE_SYS_BUS_DEVICE, or otherwise you'll run into the same hot-plug trap as we did with the x86 APIC (we had to invent a hotpluggable ICC bus as interim solution). > Having '-cpu host' > not affect them might be the pragmatic choice, since it fits > with what QEMU currently does and with kernel-side situations > where the host CPU may only be able to show the guest VM a > GICv2 view of the world (or only a GICv3, as the case may be). > For this to work it does require that guests figure out what > their per-cpu peripherals are by looking at the device tree > rather than saying "oh, this is an A57, I know all A57s > have this", of course... Without directly answering the question and continuing from above, my personal view has been that we need to get away from the current CPU model to a) how hardware is structured and b) how we want to have things behave in virtualized environments. Take x86 as an example: CPUState corresponds to a hyperthread today, but we want hotplug to work like it does on a physical machine: hot-adding on socket-level only. Beyond just building the topology with Container objects, that means having a Xeon-X5-4242 object that has-a CPU core has-a CPU thread and any devices the particular layers bring along. For SoCs I have been proposing - for sh7750 and lately tegra2 - to model "the black chip on the board" as a TYPE_DEVICE for encapsulation across boards. Meaning the GIC would no longer be instantiated on the board but as part of an object, and -smp and -cpu would as a consequence loose in influence. We could interpret -cpu host as instantiate the host's SoC object. But the mainstream SoC for KVM virtualization is exynos5, and no one sat down to model exynos5 in QEMU so far, so that would be moot. Versatile Express is rather unlikely to match the host environment KVM is used in, and when using Soft Macros (or what ARM calls their FPGA-based emulation) then things get fuzzy anyway. Similar problem for CPU hotplug: there is no real match in physical ARM hardware that we can copy for KVM/QEMU. It's all mixed in one chip where we can only enable/disable things via MMIO in physical reality. You recently proposed to have the CPUs in the a*mpcore_priv object, which also happens to own the GIC. Having the CPU model be a property of a*mpcore would complicate a lot of things QOM-wise but for the question at hand would allow to exchange the GIC based on CPU model, so I'm undecided. a9mpcore_priv with a cortex-a15 doesn't make much sense though, given there's a15mpcore_priv with different amount of IRQs and less/different child devices. Given that ARM SoCs are much less standardized then x86 PCs, I would conclude that passing random CPUs into a board/SoC does not make sense and should at least be limited to known-good combinations such as Cortex-A7 vs. Cortex-A15. If someone wants to experiment with modifications of SoCs/boards then we should rather provide more easy ways to let someone derive a custom board config then pretending that the user can just set a magical command line argument to cherry-pick parts of the system. Regards, Andreas --=20 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 N=C3=BCrnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imend=C3=B6rffer; HRB 16746 AG N=C3=BC= rnberg