From: David Woodhouse <dwmw2@infradead.org>
To: qemu-devel@nongnu.org
Cc: "Paolo Bonzini" <pbonzini@redhat.com>,
"Paul Durrant" <paul@xen.org>,
"Joao Martins" <joao.m.martins@oracle.com>,
"Ankur Arora" <ankur.a.arora@oracle.com>,
"Philippe Mathieu-Daudé" <philmd@linaro.org>,
"Thomas Huth" <thuth@redhat.com>,
"Alex Bennée" <alex.bennee@linaro.org>,
"Juan Quintela" <quintela@redhat.com>,
"Dr . David Alan Gilbert" <dgilbert@redhat.com>,
"Claudio Fontana" <cfontana@suse.de>,
"Julien Grall" <julien@xen.org>
Subject: [RFC PATCH v5 00/52] Xen support under KVM
Date: Fri, 30 Dec 2022 12:11:43 +0000 [thread overview]
Message-ID: <20221230121235.1282915-1-dwmw2@infradead.org> (raw)
Round 5, in which it gains a XenStore implementation. This just returns
ENOSYS to every request for now, but that's enough to let older Linux
guests boot, and let the XTF tests run.
As noted, I'd like to hook that up to a real xenstored via the UNIX
socket and an XS_SU command to let that connection be seen as the client
domain. If I can just get xenstored to be a little less crashy when run
without actual Xen, that is...
Having even this much XenStore allows the GSI and PCI_INTX event
delivery to be tested properly, and it's working OK with userspace
IOAPIC and the hack to poll for deassertion on vmexit. I do still want
to clean that (and VFIO) up with a hook on EOI in the PIC/IOAPIC instead
of the excessive polling, but that's a pre-existing problem and I can
live with what we have for now.
The in-kernel PIC/IOAPIC *do* have the right notifiers for acked GSIs
but they're deprecated, so I don't think I want to add code to the Xen
support in the kernel to support them; I think I'll just declare that
Xen support requires split-irqchip mode.
XenStore also means I can do some more kexec testing (since I'm lazy and
the kernels *inside* the disk images I'm using are old enough to need
it). So there are some fixes for SHUTDOWN_soft_reset in there too.
I think this is mostly ready to merge as it is, as an implementation of
the basic platform support.
Next steps.. with an actual xenstore hooked up, I can work on a set of
gnttab backend operations like the evtchn ones, and then selecting the
right set of ops according to whether we're running on true Xen or under
KVM, and the existing back end drivers ought to work without a large
amount of work (except perhaps for migration).
v5: https://git.infradead.org/users/dwmw2/qemu.git/shortlog/refs/heads/xenfv-kvm-5
• Add backend implementation of event channel support, to parallel the
libxenevtchn API used by existing backend drivers.
• Add basic XenStore ring implementation, test migration and kexec.
• Some kexec/soft reset fixes (clear port pending bits, kernel timer virq).
• Fix race with setting the xen_callback_asserted flag before actually
doing so, which could lead to it being *cleared* again before we even
assert it... and leave it asserted for ever.
v4: https://lore.kernel.org/qemu-devel/20221221010623.1000191-1-dwmw2@infradead.org/
• Add soft reset support near the beginning and thread it through the
rest of the feature enablement.
• Add PV timer support and advertise XENFEAT_safe_hvm_pvclock.
• Add basic grant table mapping and [gs]et_version / query_size support.
• Make xen_platform device build (and work) without CONFIG_XEN.
• Fix Xen HVM mode not to require --xen-attach.
v3: https://lore.kernel.org/qemu-devel/20221216004117.862106-1-dwmw2@infradead.org/
• Switch back to xen-version as KVM accelerator property, other review
feedback and bug fixes.
• Fix Hyper-V coexistence (ick, calling kvm_xen_init() again because
hyperv_enabled() doesn't return the right answer the first time).
• Implement event channel support, including GSI/PCI_INTX callback.
• Implement 32-bit guest support.
v2: https://lore.kernel.org/qemu-devel/20221209095612.689243-1-dwmw2@infradead.org/
• Attempt to implement migration support; every Xen enlightenment is
now recorded either from vmstate_x86_cpu or from a new sysdev device
created for that purpose. And — I believe — correctly restored, in
the right order, on vmload.
• The shared_info page is created as a proper overlay instead of abusing
the underlying guest page. This is important because Windows doesn't
even select a GPA which had RAM behind it beforehand. This will be
extended to handle the grant frames too, in the fullness of time.
• Set vCPU attributes from the correct vCPU thread to avoid deadlocks.
• Carefully copy the entire hypercall argument structure from userspace
instead of assuming that it's contiguous in HVA space.
• Distinguish between "handled but intentionally returns -ENOSYS" and
"no idea what that was" in hypercalls, allowing us to emit a
GUEST_ERROR (actually, shouldn't that change to UNIMP?) on the
latter. Experience shows that to we'll end up having to intentionally
return -ENOSYS to a bunch of weird crap that ancient guests still
attempt to use, including XenServer local hacks that nobody even
remembers what they were (hvmop 0x101, anyone? Some old Windows
PV driver appears to be trying to use it...).
* Drop the '+xen' CPU property and present Xen CPUID instead of KVM
unconditionally when running in Xen mode. Make the Xen CPUID coexist
with Hyper-V CPUID as it should, though.
• Add XEN_EMU and XENFV_MACHINE (the latter to be XEN_EMU||XEN) config
options. Some more work on this, and the incestuous relationships
between the KVM target code and the 'platform' code, is going to be
required but it's probably better to get on with implementing the
real code so we can see those interactions in all their glory,
before losing too much sleep over the details here.
• Drop the GSI-2 hack, and also the patch which made the PCI platform
device have real RAM (which isn't needed now we have overlays, qv).
• Drop the XenState and XenVcpuState from KVMState and CPUArchState
respectively. The Xen-specific fields are natively included in
CPUArchState now though, for migration purposes. And we don't
keep a host pointer to the shared_info or vcpu_info at all any
more. With the kernel doing everything for us, we don't actually
need them.
v1: https://lore.kernel.org/qemu-devel/20221205173137.607044-1-dwmw2@infradead.org/
v0: https://github.com/jpemartins/qemu/commits/xen-shim-rfc (Joao et al.)
Ankur Arora (2):
i386/xen: implement HVMOP_set_evtchn_upcall_vector
i386/xen: implement HVMOP_set_param
David Woodhouse (32):
xen: add CONFIG_XENFV_MACHINE and CONFIG_XEN_EMU options for Xen emulation
xen: Add XEN_DISABLED mode and make it default
i386/kvm: Add xen-version KVM accelerator property and init KVM Xen support
i386/hvm: Set Xen vCPU ID in KVM
i386/xen: Implement SCHEDOP_poll and SCHEDOP_yield
hw/xen: Add xen_overlay device for emulating shared xenheap pages
i386/xen: add pc_machine_kvm_type to initialize XEN_EMULATE mode
i386/xen: manage and save/restore Xen guest long_mode setting
i386/xen: implement XENMEM_add_to_physmap_batch
hw/xen: Add xen_evtchn device for event channel emulation
i386/xen: Add support for Xen event channel delivery to vCPU
hw/xen: Implement EVTCHNOP_status
hw/xen: Implement EVTCHNOP_close
hw/xen: Implement EVTCHNOP_unmask
hw/xen: Implement EVTCHNOP_bind_virq
hw/xen: Implement EVTCHNOP_bind_ipi
hw/xen: Implement EVTCHNOP_send
hw/xen: Implement EVTCHNOP_alloc_unbound
hw/xen: Implement EVTCHNOP_bind_interdomain
hw/xen: Implement EVTCHNOP_bind_vcpu
hw/xen: Implement EVTCHNOP_reset
hw/xen: Support HVM_PARAM_CALLBACK_TYPE_GSI callback
hw/xen: Support HVM_PARAM_CALLBACK_TYPE_PCI_INTX callback
kvm/i386: Add xen-gnttab-max-frames property
hw/xen: Add xen_gnttab device for grant table emulation
hw/xen: Support mapping grant frames
i386/xen: Implement HYPERVISOR_grant_table_op and GNTTABOP_[gs]et_verson
hw/xen: Implement GNTTABOP_query_size
i386/xen: Reserve Xen special pages for console, xenstore rings
hw/xen: Add backend implementation of interdomain event channel support
hw/xen: Add xen_xenstore device for xenstore emulation
hw/xen: Add basic ring handling to xenstore
Joao Martins (18):
include: import Xen public headers to include/standard-headers/
i386/kvm: handle Xen HVM cpuid leaves
xen-platform: exclude vfio-pci from the PCI platform unplug
xen-platform: allow its creation with XEN_EMULATE mode
hw/xen_backend: refactor xen_be_init()
i386/xen: handle guest hypercalls
i386/xen: implement HYPERVISOR_xen_version
i386/xen: implement HYPERVISOR_sched_op, SCHEDOP_shutdown
i386/xen: implement HYPERVISOR_memory_op
i386/xen: implement HYPERVISOR_hvm_op
i386/xen: implement HYPERVISOR_vcpu_op
i386/xen: handle VCPUOP_register_vcpu_info
i386/xen: handle VCPUOP_register_vcpu_time_info
i386/xen: handle VCPUOP_register_runstate_memory_area
i386/xen: implement HYPERVISOR_event_channel_op
i386/xen: add monitor commands to test event injection
i386/xen: handle PV timer hypercalls
i386/xen: handle HVMOP_get_param
accel/kvm/kvm-all.c | 2 +
accel/xen/xen-all.c | 2 +
hmp-commands.hx | 29 +
hw/Kconfig | 1 +
hw/i386/Kconfig | 5 +
hw/i386/kvm/meson.build | 6 +
hw/i386/kvm/xen_evtchn.c | 1598 +++++++++++++++++++
hw/i386/kvm/xen_evtchn.h | 69 +
hw/i386/kvm/xen_gnttab.c | 241 +++
hw/i386/kvm/xen_gnttab.h | 25 +
hw/i386/kvm/xen_overlay.c | 265 ++++
hw/i386/kvm/xen_overlay.h | 26 +
hw/i386/kvm/xen_xenstore.c | 469 ++++++
hw/i386/kvm/xen_xenstore.h | 20 +
hw/i386/pc.c | 23 +
hw/i386/xen/meson.build | 5 +-
hw/i386/xen/xen_platform.c | 57 +-
hw/xen/Kconfig | 3 +
hw/xen/xen-legacy-backend.c | 40 +-
include/hw/i386/pc.h | 3 +
include/hw/xen/xen-legacy-backend.h | 3 +
include/hw/xen/xen.h | 5 +-
include/standard-headers/xen/arch-x86/cpuid.h | 118 ++
include/standard-headers/xen/arch-x86/xen-x86_32.h | 194 +++
include/standard-headers/xen/arch-x86/xen-x86_64.h | 241 +++
include/standard-headers/xen/arch-x86/xen.h | 398 +++++
include/standard-headers/xen/event_channel.h | 388 +++++
include/standard-headers/xen/features.h | 143 ++
include/standard-headers/xen/grant_table.h | 686 ++++++++
include/standard-headers/xen/hvm/hvm_op.h | 395 +++++
include/standard-headers/xen/hvm/params.h | 318 ++++
include/standard-headers/xen/io/console.h | 56 +
include/standard-headers/xen/io/ring.h | 495 ++++++
include/standard-headers/xen/io/xs_wire.h | 153 ++
include/standard-headers/xen/memory.h | 754 +++++++++
include/standard-headers/xen/physdev.h | 383 +++++
include/standard-headers/xen/sched.h | 202 +++
include/standard-headers/xen/trace.h | 341 ++++
include/standard-headers/xen/vcpu.h | 248 +++
include/standard-headers/xen/version.h | 113 ++
include/standard-headers/xen/xen-compat.h | 46 +
include/standard-headers/xen/xen.h | 1049 +++++++++++++
include/sysemu/kvm_int.h | 3 +
include/sysemu/kvm_xen.h | 36 +
meson.build | 1 +
monitor/misc.c | 4 +
softmmu/globals.c | 2 +-
target/i386/cpu.c | 1 +
target/i386/cpu.h | 17 +
target/i386/kvm/kvm.c | 208 ++-
target/i386/kvm/meson.build | 2 +
target/i386/kvm/trace-events | 6 +
target/i386/kvm/xen-compat.h | 51 +
target/i386/kvm/xen-emu.c | 1657 ++++++++++++++++++++
target/i386/kvm/xen-emu.h | 33 +
target/i386/machine.c | 25 +
56 files changed, 11632 insertions(+), 32 deletions(-)
next reply other threads:[~2022-12-30 12:18 UTC|newest]
Thread overview: 67+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-30 12:11 David Woodhouse [this message]
2022-12-30 12:11 ` [RFC PATCH v5 01/52] include: import Xen public headers to include/standard-headers/ David Woodhouse
2022-12-30 12:11 ` [RFC PATCH v5 02/52] xen: add CONFIG_XENFV_MACHINE and CONFIG_XEN_EMU options for Xen emulation David Woodhouse
2022-12-30 12:11 ` [RFC PATCH v5 03/52] xen: Add XEN_DISABLED mode and make it default David Woodhouse
2022-12-30 12:11 ` [RFC PATCH v5 04/52] i386/kvm: Add xen-version KVM accelerator property and init KVM Xen support David Woodhouse
2022-12-30 12:11 ` [RFC PATCH v5 05/52] i386/kvm: handle Xen HVM cpuid leaves David Woodhouse
2022-12-30 12:11 ` [RFC PATCH v5 06/52] i386/hvm: Set Xen vCPU ID in KVM David Woodhouse
2022-12-30 12:11 ` [RFC PATCH v5 07/52] xen-platform: exclude vfio-pci from the PCI platform unplug David Woodhouse
2022-12-30 12:11 ` [RFC PATCH v5 08/52] xen-platform: allow its creation with XEN_EMULATE mode David Woodhouse
2022-12-30 12:11 ` [RFC PATCH v5 09/52] hw/xen_backend: refactor xen_be_init() David Woodhouse
2022-12-30 12:11 ` [RFC PATCH v5 10/52] i386/xen: handle guest hypercalls David Woodhouse
2022-12-30 12:11 ` [RFC PATCH v5 11/52] i386/xen: implement HYPERVISOR_xen_version David Woodhouse
2022-12-30 12:11 ` [RFC PATCH v5 12/52] i386/xen: implement HYPERVISOR_sched_op, SCHEDOP_shutdown David Woodhouse
2022-12-30 12:11 ` [RFC PATCH v5 13/52] i386/xen: Implement SCHEDOP_poll and SCHEDOP_yield David Woodhouse
2022-12-30 12:11 ` [RFC PATCH v5 14/52] hw/xen: Add xen_overlay device for emulating shared xenheap pages David Woodhouse
2023-01-03 17:54 ` Dr. David Alan Gilbert
2022-12-30 12:11 ` [RFC PATCH v5 15/52] i386/xen: add pc_machine_kvm_type to initialize XEN_EMULATE mode David Woodhouse
2022-12-30 12:11 ` [RFC PATCH v5 16/52] i386/xen: manage and save/restore Xen guest long_mode setting David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 17/52] i386/xen: implement HYPERVISOR_memory_op David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 18/52] i386/xen: implement XENMEM_add_to_physmap_batch David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 19/52] i386/xen: implement HYPERVISOR_hvm_op David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 20/52] i386/xen: implement HYPERVISOR_vcpu_op David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 21/52] i386/xen: handle VCPUOP_register_vcpu_info David Woodhouse
2023-01-03 18:13 ` Dr. David Alan Gilbert
2022-12-30 12:12 ` [RFC PATCH v5 22/52] i386/xen: handle VCPUOP_register_vcpu_time_info David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 23/52] i386/xen: handle VCPUOP_register_runstate_memory_area David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 24/52] i386/xen: implement HYPERVISOR_event_channel_op David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 25/52] i386/xen: implement HVMOP_set_evtchn_upcall_vector David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 26/52] i386/xen: implement HVMOP_set_param David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 27/52] hw/xen: Add xen_evtchn device for event channel emulation David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 28/52] i386/xen: Add support for Xen event channel delivery to vCPU David Woodhouse
2023-01-09 21:18 ` David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 29/52] hw/xen: Implement EVTCHNOP_status David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 30/52] hw/xen: Implement EVTCHNOP_close David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 31/52] hw/xen: Implement EVTCHNOP_unmask David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 32/52] hw/xen: Implement EVTCHNOP_bind_virq David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 33/52] hw/xen: Implement EVTCHNOP_bind_ipi David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 34/52] hw/xen: Implement EVTCHNOP_send David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 35/52] hw/xen: Implement EVTCHNOP_alloc_unbound David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 36/52] hw/xen: Implement EVTCHNOP_bind_interdomain David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 37/52] hw/xen: Implement EVTCHNOP_bind_vcpu David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 38/52] hw/xen: Implement EVTCHNOP_reset David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 39/52] i386/xen: add monitor commands to test event injection David Woodhouse
2023-01-04 12:48 ` Dr. David Alan Gilbert
2023-01-05 19:42 ` David Woodhouse
2023-01-05 20:09 ` Dr. David Alan Gilbert
2023-01-09 17:24 ` David Woodhouse
2023-01-09 18:51 ` Dr. David Alan Gilbert
2023-01-09 19:49 ` David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 40/52] hw/xen: Support HVM_PARAM_CALLBACK_TYPE_GSI callback David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 41/52] hw/xen: Support HVM_PARAM_CALLBACK_TYPE_PCI_INTX callback David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 42/52] kvm/i386: Add xen-gnttab-max-frames property David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 43/52] hw/xen: Add xen_gnttab device for grant table emulation David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 44/52] hw/xen: Support mapping grant frames David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 45/52] i386/xen: Implement HYPERVISOR_grant_table_op and GNTTABOP_[gs]et_verson David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 46/52] hw/xen: Implement GNTTABOP_query_size David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 47/52] i386/xen: handle PV timer hypercalls David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 48/52] i386/xen: Reserve Xen special pages for console, xenstore rings David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 49/52] i386/xen: handle HVMOP_get_param David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 50/52] hw/xen: Add backend implementation of interdomain event channel support David Woodhouse
2023-01-04 11:22 ` Dr. David Alan Gilbert
2023-01-04 14:33 ` David Woodhouse
2023-01-04 11:52 ` Dr. David Alan Gilbert
2022-12-30 12:12 ` [RFC PATCH v5 51/52] hw/xen: Add xen_xenstore device for xenstore emulation David Woodhouse
2023-01-04 12:01 ` Dr. David Alan Gilbert
2023-01-04 14:35 ` David Woodhouse
2022-12-30 12:12 ` [RFC PATCH v5 52/52] hw/xen: Add basic ring handling to xenstore David Woodhouse
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221230121235.1282915-1-dwmw2@infradead.org \
--to=dwmw2@infradead.org \
--cc=alex.bennee@linaro.org \
--cc=ankur.a.arora@oracle.com \
--cc=cfontana@suse.de \
--cc=dgilbert@redhat.com \
--cc=joao.m.martins@oracle.com \
--cc=julien@xen.org \
--cc=paul@xen.org \
--cc=pbonzini@redhat.com \
--cc=philmd@linaro.org \
--cc=qemu-devel@nongnu.org \
--cc=quintela@redhat.com \
--cc=thuth@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).