* [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
@ 2021-01-25 19:08 Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 01/22] x86/ioreq: Prepare IOREQ feature for making it common Oleksandr Tyshchenko
` (23 more replies)
0 siblings, 24 replies; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Tim Deegan,
Daniel De Graaf, Volodymyr Babchuk, Bertrand Marquis, Wei Chen,
Kaly Xin, Artem Mygaiev, Alex Bennée
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Hello all.
The purpose of this patch series is to add IOREQ/DM support to Xen on Arm.
You can find an initial discussion at [1] and RFC-V4 series at [2]-[6].
Xen on Arm requires some implementation to forward guest MMIO access to a device
model in order to implement virtio-mmio backend or even mediator outside of hypervisor.
As Xen on x86 already contains required support this series tries to make it common
and introduce Arm specific bits plus some new functionality. Patch series is based on
Julien's PoC "xen/arm: Add support for Guest IO forwarding to a device emulator".
***
IMPORTANT NOTE:
Current patch series doesn't contain VirtIO related changes for the toolstack
(but they are still available at the GitHub repo [8]):
- libxl: Introduce basic virtio-mmio support on Arm
- [RFC] libxl: Add support for virtio-disk configuration
I decided to skip these patches for now since they require some rework (not Xen 4.15 materials),
I will resume pushing them once we get *common* IOREQ in.
***
According to the initial/subsequent discussions there are a few open
questions/concerns regarding security, performance in VirtIO solution:
1. virtio-mmio vs virtio-pci, SPI vs MSI, or even a composition of virtio-mmio + MSI,
different use-cases require different transport...
2. virtio backend is able to access all guest memory, some kind of protection
is needed: 'virtio-iommu in Xen' vs 'pre-shared-memory & memcpys in guest', etc
(for these Alex have provided some input at [7])
3. interface between toolstack and 'out-of-qemu' virtio backend, avoid using
Xenstore in virtio backend if possible. Also, there is a desire to make VirtIO
backend hypervisor-agnostic.
4. a lot of 'foreing mapping' could lead to the memory exhaustion at the host side,
as we are stealing the page from host memory in order to map the guest page.
Julien has some idea regarding that.
5. Julien also has some ideas how to optimize the IOREQ code:
5.1 vcpu_ioreq_handle_completion (former handle_hvm_io_completion) which is called in
an hotpath on Arm (everytime we are re-entering to the guest):
Ideally, vcpu_ioreq_handle_completion should be a NOP (at max a few instructions)
if there is nothing to do (if we don't have I/O forwarded to an IOREQ server).
Maybe we want to introduce a per-vCPU flag indicating if an I/O has been
forwarded to an IOREQ server. This would allow us to bypass most of the function
if there is nothing to do.
5.2 The current way to handle MMIO is the following:
- Pause the vCPU
- Forward the access to the backend domain
- Schedule the backend domain
- Wait for the access to be handled
- Unpause the vCPU
The sequence is going to be fairly expensive on Xen.
It might be possible to optimize the ACK and avoid to wait for the backend
to handle the access.
Looks like all of them are valid and worth considering, but the first thing
which we need on Arm is a mechanism to forward guest IO to a device emulator,
so let's focus on it in the first place.
There are a lot of changes since RFC series, almost all TODOs were resolved on Arm,
Arm code was improved and hardened, common IOREQ/DM code became really arch-agnostic
(without HVM-ism), the "legacy" mechanism of mapping magic pages for the IOREQ servers
was left x86 specific, etc. But one TODO still remains which is "PIO handling" on Arm.
The "PIO handling" TODO is expected to left unaddressed for the current series.
It is not an big issue for now while Xen doesn't have support for vPCI on Arm.
On Arm64 they are only used for PCI IO Bar and we would probably want to expose
them to emulator as PIO access to make a DM completely arch-agnostic. So "PIO handling"
should be implemented when we add support for vPCI.
There are patches on review this series depends on:
https://patchwork.kernel.org/patch/11816689
https://patchwork.kernel.org/patch/11803383
Please note, that IOREQ feature is disabled by default on Arm within current series.
***
Patch series [8] was rebased on recent "staging branch"
(5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is unmapped) and tested on
Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio disk backend [9]
running in driver domain and unmodified Linux Guest running on existing
virtio-blk driver (frontend). No issues were observed. Guest domain 'reboot/destroy'
use-cases work properly. Patch series was only build-tested on x86.
Please note, build-test passed for the following modes:
1. x86: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y (default)
2. x86: #CONFIG_HVM is not set / #CONFIG_IOREQ_SERVER is not set
3. Arm64: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
4. Arm64: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
5. Arm32: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
6. Arm32: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
***
Any feedback/help would be highly appreciated.
[1] https://lists.xenproject.org/archives/html/xen-devel/2020-07/msg00825.html
[2] https://lists.xenproject.org/archives/html/xen-devel/2020-08/msg00071.html
[3] https://lists.xenproject.org/archives/html/xen-devel/2020-09/msg00732.html
[4] https://lists.xenproject.org/archives/html/xen-devel/2020-10/msg01077.html
[5] https://lists.xenproject.org/archives/html/xen-devel/2020-11/msg02188.html
[6] https://lists.xenproject.org/archives/html/xen-devel/2021-01/msg00749.html
[7] https://lists.xenproject.org/archives/html/xen-devel/2020-11/msg02212.html
[8] https://github.com/otyshchenko1/xen/commits/ioreq_4.14_ml6
[9] https://github.com/xen-troops/virtio-disk/commits/ioreq_ml1
Julien Grall (3):
xen/ioreq: Make x86's IOREQ related dm-op handling common
xen/mm: Make x86's XENMEM_resource_ioreq_server handling common
arm/ioreq: Introduce arch specific bits for IOREQ/DM features
Oleksandr Tyshchenko (19):
x86/ioreq: Prepare IOREQ feature for making it common
x86/ioreq: Add IOREQ_STATUS_* #define-s and update code for moving
x86/ioreq: Provide out-of-line wrapper for the handle_mmio()
xen/ioreq: Make x86's IOREQ feature common
xen/ioreq: Make x86's hvm_ioreq_needs_completion() common
xen/ioreq: Make x86's hvm_mmio_first(last)_byte() common
xen/ioreq: Make x86's hvm_ioreq_(page/vcpu/server) structs common
xen/ioreq: Move x86's ioreq_server to struct domain
xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
xen/ioreq: Remove "hvm" prefixes from involved function names
xen/ioreq: Use guest_cmpxchg64() instead of cmpxchg()
xen/arm: Call vcpu_ioreq_handle_completion() in check_for_vcpu_work()
xen/mm: Handle properly reference in set_foreign_p2m_entry() on Arm
xen/ioreq: Introduce domain_has_ioreq_server()
xen/dm: Introduce xendevicemodel_set_irq_level DM op
xen/arm: io: Abstract sign-extension
xen/arm: io: Harden sign extension check
xen/ioreq: Make x86's send_invalidate_req() common
xen/arm: Add mapcache invalidation handling
MAINTAINERS | 9 +-
tools/include/xendevicemodel.h | 4 +
tools/libs/devicemodel/core.c | 18 +
tools/libs/devicemodel/libxendevicemodel.map | 1 +
xen/arch/arm/Makefile | 2 +
xen/arch/arm/dm.c | 149 +++
xen/arch/arm/domain.c | 9 +
xen/arch/arm/io.c | 30 +-
xen/arch/arm/ioreq.c | 200 ++++
xen/arch/arm/p2m.c | 51 +-
xen/arch/arm/traps.c | 55 +-
xen/arch/x86/Kconfig | 2 +-
xen/arch/x86/hvm/dm.c | 134 +--
xen/arch/x86/hvm/emulate.c | 220 ++--
xen/arch/x86/hvm/hvm.c | 14 +-
xen/arch/x86/hvm/hypercall.c | 9 +-
xen/arch/x86/hvm/intercept.c | 5 +-
xen/arch/x86/hvm/io.c | 52 +-
xen/arch/x86/hvm/ioreq.c | 1368 ++----------------------
xen/arch/x86/hvm/stdvga.c | 12 +-
xen/arch/x86/hvm/svm/nestedsvm.c | 2 +-
xen/arch/x86/hvm/vmx/realmode.c | 8 +-
xen/arch/x86/hvm/vmx/vvmx.c | 5 +-
xen/arch/x86/mm.c | 46 +-
xen/arch/x86/mm/p2m.c | 17 +-
xen/arch/x86/mm/shadow/common.c | 2 +-
xen/common/Kconfig | 6 +-
xen/common/Makefile | 2 +
xen/common/dm.c | 55 +
xen/common/ioreq.c | 1426 ++++++++++++++++++++++++++
xen/common/memory.c | 72 +-
xen/include/asm-arm/domain.h | 2 +
xen/include/asm-arm/ioreq.h | 70 ++
xen/include/asm-arm/mmio.h | 1 +
xen/include/asm-arm/p2m.h | 19 +-
xen/include/asm-arm/traps.h | 25 +
xen/include/asm-x86/hvm/domain.h | 43 -
xen/include/asm-x86/hvm/emulate.h | 2 +-
xen/include/asm-x86/hvm/io.h | 17 -
xen/include/asm-x86/hvm/ioreq.h | 39 +-
xen/include/asm-x86/hvm/vcpu.h | 18 -
xen/include/asm-x86/ioreq.h | 37 +
xen/include/asm-x86/mm.h | 4 -
xen/include/asm-x86/p2m.h | 22 +-
xen/include/public/hvm/dm_op.h | 16 +
xen/include/xen/dm.h | 43 +
xen/include/xen/ioreq.h | 140 +++
xen/include/xen/mm.h | 9 -
xen/include/xen/p2m-common.h | 4 +
xen/include/xen/sched.h | 34 +
xen/include/xsm/dummy.h | 4 +-
xen/include/xsm/xsm.h | 6 +-
xen/xsm/dummy.c | 2 +-
xen/xsm/flask/hooks.c | 5 +-
54 files changed, 2712 insertions(+), 1835 deletions(-)
create mode 100644 xen/arch/arm/dm.c
create mode 100644 xen/arch/arm/ioreq.c
create mode 100644 xen/common/dm.c
create mode 100644 xen/common/ioreq.c
create mode 100644 xen/include/asm-arm/ioreq.h
create mode 100644 xen/include/asm-x86/ioreq.h
create mode 100644 xen/include/xen/dm.h
create mode 100644 xen/include/xen/ioreq.h
--
2.7.4
^ permalink raw reply [flat|nested] 93+ messages in thread
* [PATCH V5 01/22] x86/ioreq: Prepare IOREQ feature for making it common
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-27 16:48 ` Jan Beulich
2021-01-25 19:08 ` [PATCH V5 02/22] x86/ioreq: Add IOREQ_STATUS_* #define-s and update code for moving Oleksandr Tyshchenko
` (22 subsequent siblings)
23 siblings, 1 reply; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, Stefano Stabellini, Julien Grall, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
As a lot of x86 code can be re-used on Arm later on, this
patch makes some preparation to x86/hvm/ioreq.c before moving
to the common code. This way we will get a verbatim copy
for a code movement in subsequent patch.
This patch mostly introduces specific hooks to abstract arch
specific materials taking into the account the requirment to leave
the "legacy" mechanism of mapping magic pages for the IOREQ servers
x86 specific and not expose it to the common code.
These hooks are named according to the more consistent new naming
scheme right away (including dropping the "hvm" prefixes and infixes):
- IOREQ server functions should start with "ioreq_server_"
- IOREQ functions should start with "ioreq_"
other functions will be renamed in subsequent patches.
Introduce common ioreq.h right away and put arch hook declarations
there.
Also re-order #include-s alphabetically.
This support is going to be used on Arm to be able run device
emulator outside of Xen hypervisor.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Paul Durrant <paul@xen.org>
CC: Julien Grall <julien.grall@arm.com>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- new patch, was split from:
"[RFC PATCH V1 01/12] hvm/ioreq: Make x86's IOREQ feature common"
- fold the check of p->type into hvm_get_ioreq_server_range_type()
and make it return success/failure
- remove relocate_portio_handler() call from arch_hvm_ioreq_destroy()
in arch/x86/hvm/ioreq.c
- introduce arch_hvm_destroy_ioreq_server()/arch_handle_hvm_io_completion()
Changes V1 -> V2:
- update patch description
- make arch functions inline and put them into arch header
to achieve a truly rename by the subsequent patch
- return void in arch_hvm_destroy_ioreq_server()
- return bool in arch_hvm_ioreq_destroy()
- bring relocate_portio_handler() back to arch_hvm_ioreq_destroy()
- rename IOREQ_IO* to IOREQ_STATUS*
- remove *handle* from arch_handle_hvm_io_completion()
- re-order #include-s alphabetically
- rename hvm_get_ioreq_server_range_type() to hvm_ioreq_server_get_type_addr()
and add "const" to several arguments
Changes V2 -> V3:
- update patch description
- name new arch hooks according to the new naming scheme
- don't make arch hooks inline, move them ioreq.c
- make get_ioreq_server() local again
- rework the whole patch taking into the account that "legacy" interface
should remain x86 specific (additional arch hooks, etc)
- update the code to be able to use hvm_map_mem_type_to_ioreq_server()
in the common code (an extra arch hook, etc)
- don’t include <asm/hvm/emulate.h> from arch header
- add "arch" prefix to hvm_ioreq_server_get_type_addr()
- move IOREQ_STATUS_* #define-s introduction to the separate patch
- move HANDLE_BUFIOREQ to the arch header
- just return relocate_portio_handler() from arch_ioreq_server_destroy_all()
- misc adjustments proposed by Jan (adding const, unsigned int instead of uint32_t)
Changes V3 -> V4:
- add Alex's R-b
- update patch description
- make arch_ioreq_server_get_type_addr return bool
- drop #include <xen/ctype.h>
- use two arch hooks in hvm_map_mem_type_to_ioreq_server()
to avoid calling p2m_change_entry_type_global() with lock held
Changes V4 -> V5:
- add Julien's and Paul's R-b
- update patch description
- remove single use variable in arch_ioreq_server_map_mem_type_completed()
- put multiple function parameters on a single line in the header
where possible
- introduce common ioreq.h right away and put arch hooks declarations
there instead of doing that in patch #4
---
---
xen/arch/x86/hvm/ioreq.c | 175 +++++++++++++++++++++++++++++++----------------
xen/include/xen/ioreq.h | 54 +++++++++++++++
2 files changed, 169 insertions(+), 60 deletions(-)
create mode 100644 xen/include/xen/ioreq.h
diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 1cc27df..3c3c173 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -16,16 +16,16 @@
* this program; If not, see <http://www.gnu.org/licenses/>.
*/
-#include <xen/ctype.h>
+#include <xen/domain.h>
+#include <xen/event.h>
#include <xen/init.h>
+#include <xen/ioreq.h>
+#include <xen/irq.h>
#include <xen/lib.h>
-#include <xen/trace.h>
+#include <xen/paging.h>
#include <xen/sched.h>
-#include <xen/irq.h>
#include <xen/softirq.h>
-#include <xen/domain.h>
-#include <xen/event.h>
-#include <xen/paging.h>
+#include <xen/trace.h>
#include <xen/vpci.h>
#include <asm/hvm/emulate.h>
@@ -170,6 +170,29 @@ static bool hvm_wait_for_io(struct hvm_ioreq_vcpu *sv, ioreq_t *p)
return true;
}
+bool arch_vcpu_ioreq_completion(enum hvm_io_completion io_completion)
+{
+ switch ( io_completion )
+ {
+ case HVMIO_realmode_completion:
+ {
+ struct hvm_emulate_ctxt ctxt;
+
+ hvm_emulate_init_once(&ctxt, NULL, guest_cpu_user_regs());
+ vmx_realmode_emulate_one(&ctxt);
+ hvm_emulate_writeback(&ctxt);
+
+ break;
+ }
+
+ default:
+ ASSERT_UNREACHABLE();
+ break;
+ }
+
+ return true;
+}
+
bool handle_hvm_io_completion(struct vcpu *v)
{
struct domain *d = v->domain;
@@ -209,19 +232,8 @@ bool handle_hvm_io_completion(struct vcpu *v)
return handle_pio(vio->io_req.addr, vio->io_req.size,
vio->io_req.dir);
- case HVMIO_realmode_completion:
- {
- struct hvm_emulate_ctxt ctxt;
-
- hvm_emulate_init_once(&ctxt, NULL, guest_cpu_user_regs());
- vmx_realmode_emulate_one(&ctxt);
- hvm_emulate_writeback(&ctxt);
-
- break;
- }
default:
- ASSERT_UNREACHABLE();
- break;
+ return arch_vcpu_ioreq_completion(io_completion);
}
return true;
@@ -477,9 +489,6 @@ static void hvm_update_ioreq_evtchn(struct hvm_ioreq_server *s,
}
}
-#define HANDLE_BUFIOREQ(s) \
- ((s)->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF)
-
static int hvm_ioreq_server_add_vcpu(struct hvm_ioreq_server *s,
struct vcpu *v)
{
@@ -586,7 +595,7 @@ static void hvm_ioreq_server_remove_all_vcpus(struct hvm_ioreq_server *s)
spin_unlock(&s->lock);
}
-static int hvm_ioreq_server_map_pages(struct hvm_ioreq_server *s)
+int arch_ioreq_server_map_pages(struct hvm_ioreq_server *s)
{
int rc;
@@ -601,7 +610,7 @@ static int hvm_ioreq_server_map_pages(struct hvm_ioreq_server *s)
return rc;
}
-static void hvm_ioreq_server_unmap_pages(struct hvm_ioreq_server *s)
+void arch_ioreq_server_unmap_pages(struct hvm_ioreq_server *s)
{
hvm_unmap_ioreq_gfn(s, true);
hvm_unmap_ioreq_gfn(s, false);
@@ -674,6 +683,12 @@ static int hvm_ioreq_server_alloc_rangesets(struct hvm_ioreq_server *s,
return rc;
}
+void arch_ioreq_server_enable(struct hvm_ioreq_server *s)
+{
+ hvm_remove_ioreq_gfn(s, false);
+ hvm_remove_ioreq_gfn(s, true);
+}
+
static void hvm_ioreq_server_enable(struct hvm_ioreq_server *s)
{
struct hvm_ioreq_vcpu *sv;
@@ -683,8 +698,7 @@ static void hvm_ioreq_server_enable(struct hvm_ioreq_server *s)
if ( s->enabled )
goto done;
- hvm_remove_ioreq_gfn(s, false);
- hvm_remove_ioreq_gfn(s, true);
+ arch_ioreq_server_enable(s);
s->enabled = true;
@@ -697,6 +711,12 @@ static void hvm_ioreq_server_enable(struct hvm_ioreq_server *s)
spin_unlock(&s->lock);
}
+void arch_ioreq_server_disable(struct hvm_ioreq_server *s)
+{
+ hvm_add_ioreq_gfn(s, true);
+ hvm_add_ioreq_gfn(s, false);
+}
+
static void hvm_ioreq_server_disable(struct hvm_ioreq_server *s)
{
spin_lock(&s->lock);
@@ -704,8 +724,7 @@ static void hvm_ioreq_server_disable(struct hvm_ioreq_server *s)
if ( !s->enabled )
goto done;
- hvm_add_ioreq_gfn(s, true);
- hvm_add_ioreq_gfn(s, false);
+ arch_ioreq_server_disable(s);
s->enabled = false;
@@ -750,7 +769,7 @@ static int hvm_ioreq_server_init(struct hvm_ioreq_server *s,
fail_add:
hvm_ioreq_server_remove_all_vcpus(s);
- hvm_ioreq_server_unmap_pages(s);
+ arch_ioreq_server_unmap_pages(s);
hvm_ioreq_server_free_rangesets(s);
@@ -764,7 +783,7 @@ static void hvm_ioreq_server_deinit(struct hvm_ioreq_server *s)
hvm_ioreq_server_remove_all_vcpus(s);
/*
- * NOTE: It is safe to call both hvm_ioreq_server_unmap_pages() and
+ * NOTE: It is safe to call both arch_ioreq_server_unmap_pages() and
* hvm_ioreq_server_free_pages() in that order.
* This is because the former will do nothing if the pages
* are not mapped, leaving the page to be freed by the latter.
@@ -772,7 +791,7 @@ static void hvm_ioreq_server_deinit(struct hvm_ioreq_server *s)
* the page_info pointer to NULL, meaning the latter will do
* nothing.
*/
- hvm_ioreq_server_unmap_pages(s);
+ arch_ioreq_server_unmap_pages(s);
hvm_ioreq_server_free_pages(s);
hvm_ioreq_server_free_rangesets(s);
@@ -836,6 +855,12 @@ int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
return rc;
}
+/* Called when target domain is paused */
+void arch_ioreq_server_destroy(struct hvm_ioreq_server *s)
+{
+ p2m_set_ioreq_server(s->target, 0, s);
+}
+
int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
{
struct hvm_ioreq_server *s;
@@ -855,7 +880,7 @@ int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
domain_pause(d);
- p2m_set_ioreq_server(d, 0, s);
+ arch_ioreq_server_destroy(s);
hvm_ioreq_server_disable(s);
@@ -900,7 +925,7 @@ int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
if ( ioreq_gfn || bufioreq_gfn )
{
- rc = hvm_ioreq_server_map_pages(s);
+ rc = arch_ioreq_server_map_pages(s);
if ( rc )
goto out;
}
@@ -1080,6 +1105,22 @@ int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
return rc;
}
+/* Called with ioreq_server lock held */
+int arch_ioreq_server_map_mem_type(struct domain *d,
+ struct hvm_ioreq_server *s,
+ uint32_t flags)
+{
+ return p2m_set_ioreq_server(d, flags, s);
+}
+
+void arch_ioreq_server_map_mem_type_completed(struct domain *d,
+ struct hvm_ioreq_server *s,
+ uint32_t flags)
+{
+ if ( flags == 0 && read_atomic(&p2m_get_hostp2m(d)->ioreq.entry_count) )
+ p2m_change_entry_type_global(d, p2m_ioreq_server, p2m_ram_rw);
+}
+
/*
* Map or unmap an ioreq server to specific memory type. For now, only
* HVMMEM_ioreq_server is supported, and in the future new types can be
@@ -1112,18 +1153,13 @@ int hvm_map_mem_type_to_ioreq_server(struct domain *d, ioservid_t id,
if ( s->emulator != current->domain )
goto out;
- rc = p2m_set_ioreq_server(d, flags, s);
+ rc = arch_ioreq_server_map_mem_type(d, s, flags);
out:
spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
- if ( rc == 0 && flags == 0 )
- {
- struct p2m_domain *p2m = p2m_get_hostp2m(d);
-
- if ( read_atomic(&p2m->ioreq.entry_count) )
- p2m_change_entry_type_global(d, p2m_ioreq_server, p2m_ram_rw);
- }
+ if ( rc == 0 )
+ arch_ioreq_server_map_mem_type_completed(d, s, flags);
return rc;
}
@@ -1210,12 +1246,17 @@ void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v)
spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
}
+bool arch_ioreq_server_destroy_all(struct domain *d)
+{
+ return relocate_portio_handler(d, 0xcf8, 0xcf8, 4);
+}
+
void hvm_destroy_all_ioreq_servers(struct domain *d)
{
struct hvm_ioreq_server *s;
unsigned int id;
- if ( !relocate_portio_handler(d, 0xcf8, 0xcf8, 4) )
+ if ( !arch_ioreq_server_destroy_all(d) )
return;
spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
@@ -1239,33 +1280,28 @@ void hvm_destroy_all_ioreq_servers(struct domain *d)
spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
}
-struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
- ioreq_t *p)
+bool arch_ioreq_server_get_type_addr(const struct domain *d,
+ const ioreq_t *p,
+ uint8_t *type,
+ uint64_t *addr)
{
- struct hvm_ioreq_server *s;
- uint32_t cf8;
- uint8_t type;
- uint64_t addr;
- unsigned int id;
+ unsigned int cf8 = d->arch.hvm.pci_cf8;
if ( p->type != IOREQ_TYPE_COPY && p->type != IOREQ_TYPE_PIO )
- return NULL;
-
- cf8 = d->arch.hvm.pci_cf8;
+ return false;
if ( p->type == IOREQ_TYPE_PIO &&
(p->addr & ~3) == 0xcfc &&
CF8_ENABLED(cf8) )
{
- uint32_t x86_fam;
+ unsigned int x86_fam, reg;
pci_sbdf_t sbdf;
- unsigned int reg;
reg = hvm_pci_decode_addr(cf8, p->addr, &sbdf);
/* PCI config data cycle */
- type = XEN_DMOP_IO_RANGE_PCI;
- addr = ((uint64_t)sbdf.sbdf << 32) | reg;
+ *type = XEN_DMOP_IO_RANGE_PCI;
+ *addr = ((uint64_t)sbdf.sbdf << 32) | reg;
/* AMD extended configuration space access? */
if ( CF8_ADDR_HI(cf8) &&
d->arch.cpuid->x86_vendor == X86_VENDOR_AMD &&
@@ -1277,16 +1313,30 @@ struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
if ( !rdmsr_safe(MSR_AMD64_NB_CFG, msr_val) &&
(msr_val & (1ULL << AMD64_NB_CFG_CF8_EXT_ENABLE_BIT)) )
- addr |= CF8_ADDR_HI(cf8);
+ *addr |= CF8_ADDR_HI(cf8);
}
}
else
{
- type = (p->type == IOREQ_TYPE_PIO) ?
- XEN_DMOP_IO_RANGE_PORT : XEN_DMOP_IO_RANGE_MEMORY;
- addr = p->addr;
+ *type = (p->type == IOREQ_TYPE_PIO) ?
+ XEN_DMOP_IO_RANGE_PORT : XEN_DMOP_IO_RANGE_MEMORY;
+ *addr = p->addr;
}
+ return true;
+}
+
+struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
+ ioreq_t *p)
+{
+ struct hvm_ioreq_server *s;
+ uint8_t type;
+ uint64_t addr;
+ unsigned int id;
+
+ if ( !arch_ioreq_server_get_type_addr(d, p, &type, &addr) )
+ return NULL;
+
FOR_EACH_IOREQ_SERVER(d, id, s)
{
struct rangeset *r;
@@ -1515,11 +1565,16 @@ static int hvm_access_cf8(
return X86EMUL_UNHANDLEABLE;
}
+void arch_ioreq_domain_init(struct domain *d)
+{
+ register_portio_handler(d, 0xcf8, 4, hvm_access_cf8);
+}
+
void hvm_ioreq_init(struct domain *d)
{
spin_lock_init(&d->arch.hvm.ioreq_server.lock);
- register_portio_handler(d, 0xcf8, 4, hvm_access_cf8);
+ arch_ioreq_domain_init(d);
}
/*
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
new file mode 100644
index 0000000..d0980c5
--- /dev/null
+++ b/xen/include/xen/ioreq.h
@@ -0,0 +1,54 @@
+/*
+ * ioreq.h: Hardware virtual machine assist interface definitions.
+ *
+ * Copyright (c) 2016 Citrix Systems Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __XEN_IOREQ_H__
+#define __XEN_IOREQ_H__
+
+#include <xen/sched.h>
+
+#define HANDLE_BUFIOREQ(s) \
+ ((s)->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF)
+
+bool arch_vcpu_ioreq_completion(enum hvm_io_completion io_completion);
+int arch_ioreq_server_map_pages(struct hvm_ioreq_server *s);
+void arch_ioreq_server_unmap_pages(struct hvm_ioreq_server *s);
+void arch_ioreq_server_enable(struct hvm_ioreq_server *s);
+void arch_ioreq_server_disable(struct hvm_ioreq_server *s);
+void arch_ioreq_server_destroy(struct hvm_ioreq_server *s);
+int arch_ioreq_server_map_mem_type(struct domain *d,
+ struct hvm_ioreq_server *s,
+ uint32_t flags);
+void arch_ioreq_server_map_mem_type_completed(struct domain *d,
+ struct hvm_ioreq_server *s,
+ uint32_t flags);
+bool arch_ioreq_server_destroy_all(struct domain *d);
+bool arch_ioreq_server_get_type_addr(const struct domain *d, const ioreq_t *p,
+ uint8_t *type, uint64_t *addr);
+void arch_ioreq_domain_init(struct domain *d);
+
+#endif /* __XEN_IOREQ_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 02/22] x86/ioreq: Add IOREQ_STATUS_* #define-s and update code for moving
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 01/22] x86/ioreq: Prepare IOREQ feature for making it common Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 03/22] x86/ioreq: Provide out-of-line wrapper for the handle_mmio() Oleksandr Tyshchenko
` (21 subsequent siblings)
23 siblings, 0 replies; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, Stefano Stabellini, Julien Grall, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
This patch continues to make some preparation to x86/hvm/ioreq.c
before moving to the common code.
Add IOREQ_STATUS_* #define-s and update candidates for moving
since X86EMUL_* shouldn't be exposed to the common code in
that form.
This support is going to be used on Arm to be able run device
emulator outside of Xen hypervisor.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Paul Durrant <paul@xen.org>
CC: Julien Grall <julien.grall@arm.com>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes V2 -> V3:
- new patch, was split from
[PATCH V2 01/23] x86/ioreq: Prepare IOREQ feature for making it common
Changes V3 -> V4:
- add Alex's R-b and Jan's A-b
- add a comment above IOREQ_STATUS_* #define-s
Changes V4 -> V5:
- rebase
- add Julien's and Paul's R-b
---
---
xen/arch/x86/hvm/ioreq.c | 16 ++++++++--------
xen/include/asm-x86/hvm/ioreq.h | 5 +++++
2 files changed, 13 insertions(+), 8 deletions(-)
diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 3c3c173..27a4a6f 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -1401,7 +1401,7 @@ static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
pg = iorp->va;
if ( !pg )
- return X86EMUL_UNHANDLEABLE;
+ return IOREQ_STATUS_UNHANDLED;
/*
* Return 0 for the cases we can't deal with:
@@ -1431,7 +1431,7 @@ static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
break;
default:
gdprintk(XENLOG_WARNING, "unexpected ioreq size: %u\n", p->size);
- return X86EMUL_UNHANDLEABLE;
+ return IOREQ_STATUS_UNHANDLED;
}
spin_lock(&s->bufioreq_lock);
@@ -1441,7 +1441,7 @@ static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
{
/* The queue is full: send the iopacket through the normal path. */
spin_unlock(&s->bufioreq_lock);
- return X86EMUL_UNHANDLEABLE;
+ return IOREQ_STATUS_UNHANDLED;
}
pg->buf_ioreq[pg->ptrs.write_pointer % IOREQ_BUFFER_SLOT_NUM] = bp;
@@ -1472,7 +1472,7 @@ static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
notify_via_xen_event_channel(d, s->bufioreq_evtchn);
spin_unlock(&s->bufioreq_lock);
- return X86EMUL_OKAY;
+ return IOREQ_STATUS_HANDLED;
}
int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
@@ -1488,7 +1488,7 @@ int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
return hvm_send_buffered_ioreq(s, proto_p);
if ( unlikely(!vcpu_start_shutdown_deferral(curr)) )
- return X86EMUL_RETRY;
+ return IOREQ_STATUS_RETRY;
list_for_each_entry ( sv,
&s->ioreq_vcpu_list,
@@ -1528,11 +1528,11 @@ int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
notify_via_xen_event_channel(d, port);
sv->pending = true;
- return X86EMUL_RETRY;
+ return IOREQ_STATUS_RETRY;
}
}
- return X86EMUL_UNHANDLEABLE;
+ return IOREQ_STATUS_UNHANDLED;
}
unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered)
@@ -1546,7 +1546,7 @@ unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered)
if ( !s->enabled )
continue;
- if ( hvm_send_ioreq(s, p, buffered) == X86EMUL_UNHANDLEABLE )
+ if ( hvm_send_ioreq(s, p, buffered) == IOREQ_STATUS_UNHANDLED )
failed++;
}
diff --git a/xen/include/asm-x86/hvm/ioreq.h b/xen/include/asm-x86/hvm/ioreq.h
index e2588e9..df0c292 100644
--- a/xen/include/asm-x86/hvm/ioreq.h
+++ b/xen/include/asm-x86/hvm/ioreq.h
@@ -55,6 +55,11 @@ unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered);
void hvm_ioreq_init(struct domain *d);
+/* This correlation must not be altered */
+#define IOREQ_STATUS_HANDLED X86EMUL_OKAY
+#define IOREQ_STATUS_UNHANDLED X86EMUL_UNHANDLEABLE
+#define IOREQ_STATUS_RETRY X86EMUL_RETRY
+
#endif /* __ASM_X86_HVM_IOREQ_H__ */
/*
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 03/22] x86/ioreq: Provide out-of-line wrapper for the handle_mmio()
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 01/22] x86/ioreq: Prepare IOREQ feature for making it common Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 02/22] x86/ioreq: Add IOREQ_STATUS_* #define-s and update code for moving Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common Oleksandr Tyshchenko
` (20 subsequent siblings)
23 siblings, 0 replies; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, Stefano Stabellini, Julien Grall, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
The IOREQ is about to be common feature and Arm will have its own
implementation.
But the name of the function is pretty generic and can be confusing
on Arm (we already have a try_handle_mmio()).
In order not to rename the function (which is used for a varying
set of purposes on x86) globally and get non-confusing variant on Arm
provide a wrapper arch_ioreq_complete_mmio() to be used on common
and Arm code.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Paul Durrant <paul@xen.org>
CC: Julien Grall <julien.grall@arm.com>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- new patch
Changes V1 -> V2:
- remove "handle"
- add Jan's A-b
Changes V2 -> V3:
- remove Jan's A-b
- update patch subject/description
- use out-of-line function instead of #define
- put earlier in the series to avoid breakage
Changes V3 -> V4:
- add Jan's R-b
- rename ioreq_complete_mmio() to arch_ioreq_complete_mmio()
Changes V4 -> V5:
- rebase
- add Alex's, Julien's and Paul's R-b
---
---
xen/arch/x86/hvm/ioreq.c | 7 ++++++-
xen/include/xen/ioreq.h | 1 +
2 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 27a4a6f..30e8724 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -36,6 +36,11 @@
#include <public/hvm/ioreq.h>
#include <public/hvm/params.h>
+bool arch_ioreq_complete_mmio(void)
+{
+ return handle_mmio();
+}
+
static void set_ioreq_server(struct domain *d, unsigned int id,
struct hvm_ioreq_server *s)
{
@@ -226,7 +231,7 @@ bool handle_hvm_io_completion(struct vcpu *v)
break;
case HVMIO_mmio_completion:
- return handle_mmio();
+ return arch_ioreq_complete_mmio();
case HVMIO_pio_completion:
return handle_pio(vio->io_req.addr, vio->io_req.size,
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
index d0980c5..b95d3ef 100644
--- a/xen/include/xen/ioreq.h
+++ b/xen/include/xen/ioreq.h
@@ -24,6 +24,7 @@
#define HANDLE_BUFIOREQ(s) \
((s)->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF)
+bool arch_ioreq_complete_mmio(void);
bool arch_vcpu_ioreq_completion(enum hvm_io_completion io_completion);
int arch_ioreq_server_map_pages(struct hvm_ioreq_server *s);
void arch_ioreq_server_unmap_pages(struct hvm_ioreq_server *s);
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (2 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 03/22] x86/ioreq: Provide out-of-line wrapper for the handle_mmio() Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-25 23:13 ` Julien Grall
2021-01-27 16:58 ` Jan Beulich
2021-01-25 19:08 ` [PATCH V5 05/22] xen/ioreq: Make x86's hvm_ioreq_needs_completion() common Oleksandr Tyshchenko
` (19 subsequent siblings)
23 siblings, 2 replies; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Andrew Cooper, George Dunlap, Ian Jackson,
Jan Beulich, Julien Grall, Stefano Stabellini, Wei Liu,
Roger Pau Monné,
Paul Durrant, Jun Nakajima, Kevin Tian, Tim Deegan, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
As a lot of x86 code can be re-used on Arm later on, this patch
moves previously prepared IOREQ support to the common code
(the code movement is verbatim copy).
The "legacy" mechanism of mapping magic pages for the IOREQ servers
remains x86 specific and not exposed to the common code.
The common IOREQ feature is supposed to be built with IOREQ_SERVER
option enabled, which is selected for x86's config HVM for now.
In order to avoid having a gigantic patch here, the subsequent
patches will update remaining bits in the common code step by step:
- Make IOREQ related structs/materials common
- Drop the "hvm" prefixes and infixes
- Remove layering violation by moving corresponding fields
out of *arch.hvm* or abstracting away accesses to them
Introduce asm/ioreq.h wrapper to be included by common ioreq.h
instead of asm/hvm/ioreq.h to avoid HVM-ism in the code common.
Also include <xen/domain_page.h> which will be needed on Arm
to avoid touch the common code again when introducing Arm specific bits.
This support is going to be used on Arm to be able run device
emulator outside of Xen hypervisor.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Reviewed-by: Paul Durrant <paul@xen.org>
CC: Julien Grall <julien.grall@arm.com>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
***
Please note, this patch depends on the following which is
on review:
https://patchwork.kernel.org/patch/11816689/
The effort (to get it upstreamed) was paused because of
the security issue around that code (XSA-348).
***
Changes RFC -> V1:
- was split into three patches:
- x86/ioreq: Prepare IOREQ feature for making it common
- xen/ioreq: Make x86's IOREQ feature common
- xen/ioreq: Make x86's hvm_ioreq_needs_completion() common
- update MAINTAINERS file
- do not use a separate subdir for the IOREQ stuff, move it to:
- xen/common/ioreq.c
- xen/include/xen/ioreq.h
- update x86's files to include xen/ioreq.h
- remove unneeded headers in arch/x86/hvm/ioreq.c
- re-order the headers alphabetically in common/ioreq.c
- update common/ioreq.c according to the newly introduced arch functions:
arch_hvm_destroy_ioreq_server()/arch_handle_hvm_io_completion()
Changes V1 -> V2:
- update patch description
- make everything needed in the previous patch to achieve
a truly rename here
- don't include unnecessary headers from asm-x86/hvm/ioreq.h
and xen/ioreq.h
- use __XEN_IOREQ_H__ instead of __IOREQ_H__
- move get_ioreq_server() to common/ioreq.c
Changes V2 -> V3:
- update patch description
- make everything needed in the previous patch to not
expose "legacy" interface to the common code here
- update patch according the "legacy interface" is x86 specific
- include <xen/domain_page.h> in common ioreq.c
Changes V3 -> V4:
- rebase
- don't include <xen/ioreq.h> from arch header
- мove all arch hook declarations to the common header
Change V4 -> V5:
- rebase
- introduce asm-x86/ioreq.h wrapper:
- update MAINTAINERS file
- include asm/ioreq.h instead of asm/hvm/ioreq.h from common ioreq.c
- include public/hvm/dm_op.h from common ioreq.h
- add Paul's R-b
---
---
MAINTAINERS | 9 +-
xen/arch/x86/Kconfig | 1 +
xen/arch/x86/hvm/dm.c | 2 +-
xen/arch/x86/hvm/emulate.c | 2 +-
xen/arch/x86/hvm/hvm.c | 2 +-
xen/arch/x86/hvm/io.c | 2 +-
xen/arch/x86/hvm/ioreq.c | 1316 ++-------------------------------------
xen/arch/x86/hvm/stdvga.c | 2 +-
xen/arch/x86/hvm/vmx/vvmx.c | 3 +-
xen/arch/x86/mm.c | 2 +-
xen/arch/x86/mm/shadow/common.c | 2 +-
xen/common/Kconfig | 3 +
xen/common/Makefile | 1 +
xen/common/ioreq.c | 1290 ++++++++++++++++++++++++++++++++++++++
xen/include/asm-x86/hvm/ioreq.h | 36 --
xen/include/asm-x86/ioreq.h | 37 ++
xen/include/xen/ioreq.h | 38 ++
17 files changed, 1422 insertions(+), 1326 deletions(-)
create mode 100644 xen/common/ioreq.c
create mode 100644 xen/include/asm-x86/ioreq.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 5079b83..c4f9aff 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -333,6 +333,13 @@ X: xen/drivers/passthrough/vtd/
X: xen/drivers/passthrough/device_tree.c
F: xen/include/xen/iommu.h
+I/O EMULATION (IOREQ)
+M: Paul Durrant <paul@xen.org>
+S: Supported
+F: xen/common/ioreq.c
+F: xen/include/xen/ioreq.h
+F: xen/include/public/hvm/ioreq.h
+
KCONFIG
M: Doug Goldstein <cardoe@cardoe.com>
S: Supported
@@ -549,7 +556,7 @@ F: xen/arch/x86/hvm/ioreq.c
F: xen/include/asm-x86/hvm/emulate.h
F: xen/include/asm-x86/hvm/io.h
F: xen/include/asm-x86/hvm/ioreq.h
-F: xen/include/public/hvm/ioreq.h
+F: xen/include/asm-x86/ioreq.h
X86 MEMORY MANAGEMENT
M: Jan Beulich <jbeulich@suse.com>
diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index 78f351f..ea9a9ea 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -92,6 +92,7 @@ config PV_LINEAR_PT
config HVM
def_bool !PV_SHIM_EXCLUSIVE
+ select IOREQ_SERVER
prompt "HVM support"
---help---
Interfaces to support HVM domains. HVM domains require hardware
diff --git a/xen/arch/x86/hvm/dm.c b/xen/arch/x86/hvm/dm.c
index 71f5ca4..d3e2a9e 100644
--- a/xen/arch/x86/hvm/dm.c
+++ b/xen/arch/x86/hvm/dm.c
@@ -17,12 +17,12 @@
#include <xen/event.h>
#include <xen/guest_access.h>
#include <xen/hypercall.h>
+#include <xen/ioreq.h>
#include <xen/nospec.h>
#include <xen/sched.h>
#include <asm/hap.h>
#include <asm/hvm/cacheattr.h>
-#include <asm/hvm/ioreq.h>
#include <asm/shadow.h>
#include <xsm/xsm.h>
diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index 24cf85f..60ca465 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -10,6 +10,7 @@
*/
#include <xen/init.h>
+#include <xen/ioreq.h>
#include <xen/lib.h>
#include <xen/sched.h>
#include <xen/paging.h>
@@ -20,7 +21,6 @@
#include <asm/xstate.h>
#include <asm/hvm/emulate.h>
#include <asm/hvm/hvm.h>
-#include <asm/hvm/ioreq.h>
#include <asm/hvm/monitor.h>
#include <asm/hvm/trace.h>
#include <asm/hvm/support.h>
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 54e32e4..bc96947 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -20,6 +20,7 @@
#include <xen/ctype.h>
#include <xen/init.h>
+#include <xen/ioreq.h>
#include <xen/lib.h>
#include <xen/trace.h>
#include <xen/sched.h>
@@ -64,7 +65,6 @@
#include <asm/hvm/trace.h>
#include <asm/hvm/nestedhvm.h>
#include <asm/hvm/monitor.h>
-#include <asm/hvm/ioreq.h>
#include <asm/hvm/viridian.h>
#include <asm/hvm/vm_event.h>
#include <asm/altp2m.h>
diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
index 3e09d9b..11e007d 100644
--- a/xen/arch/x86/hvm/io.c
+++ b/xen/arch/x86/hvm/io.c
@@ -19,6 +19,7 @@
*/
#include <xen/init.h>
+#include <xen/ioreq.h>
#include <xen/mm.h>
#include <xen/lib.h>
#include <xen/errno.h>
@@ -35,7 +36,6 @@
#include <asm/shadow.h>
#include <asm/p2m.h>
#include <asm/hvm/hvm.h>
-#include <asm/hvm/ioreq.h>
#include <asm/hvm/support.h>
#include <asm/hvm/vpt.h>
#include <asm/hvm/vpic.h>
diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 30e8724..666d695 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -30,7 +30,6 @@
#include <asm/hvm/emulate.h>
#include <asm/hvm/hvm.h>
-#include <asm/hvm/ioreq.h>
#include <asm/hvm/vmx/vmx.h>
#include <public/hvm/ioreq.h>
@@ -41,140 +40,6 @@ bool arch_ioreq_complete_mmio(void)
return handle_mmio();
}
-static void set_ioreq_server(struct domain *d, unsigned int id,
- struct hvm_ioreq_server *s)
-{
- ASSERT(id < MAX_NR_IOREQ_SERVERS);
- ASSERT(!s || !d->arch.hvm.ioreq_server.server[id]);
-
- d->arch.hvm.ioreq_server.server[id] = s;
-}
-
-#define GET_IOREQ_SERVER(d, id) \
- (d)->arch.hvm.ioreq_server.server[id]
-
-static struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
- unsigned int id)
-{
- if ( id >= MAX_NR_IOREQ_SERVERS )
- return NULL;
-
- return GET_IOREQ_SERVER(d, id);
-}
-
-/*
- * Iterate over all possible ioreq servers.
- *
- * NOTE: The iteration is backwards such that more recently created
- * ioreq servers are favoured in hvm_select_ioreq_server().
- * This is a semantic that previously existed when ioreq servers
- * were held in a linked list.
- */
-#define FOR_EACH_IOREQ_SERVER(d, id, s) \
- for ( (id) = MAX_NR_IOREQ_SERVERS; (id) != 0; ) \
- if ( !(s = GET_IOREQ_SERVER(d, --(id))) ) \
- continue; \
- else
-
-static ioreq_t *get_ioreq(struct hvm_ioreq_server *s, struct vcpu *v)
-{
- shared_iopage_t *p = s->ioreq.va;
-
- ASSERT((v == current) || !vcpu_runnable(v));
- ASSERT(p != NULL);
-
- return &p->vcpu_ioreq[v->vcpu_id];
-}
-
-static struct hvm_ioreq_vcpu *get_pending_vcpu(const struct vcpu *v,
- struct hvm_ioreq_server **srvp)
-{
- struct domain *d = v->domain;
- struct hvm_ioreq_server *s;
- unsigned int id;
-
- FOR_EACH_IOREQ_SERVER(d, id, s)
- {
- struct hvm_ioreq_vcpu *sv;
-
- list_for_each_entry ( sv,
- &s->ioreq_vcpu_list,
- list_entry )
- {
- if ( sv->vcpu == v && sv->pending )
- {
- if ( srvp )
- *srvp = s;
- return sv;
- }
- }
- }
-
- return NULL;
-}
-
-bool hvm_io_pending(struct vcpu *v)
-{
- return get_pending_vcpu(v, NULL);
-}
-
-static bool hvm_wait_for_io(struct hvm_ioreq_vcpu *sv, ioreq_t *p)
-{
- unsigned int prev_state = STATE_IOREQ_NONE;
- unsigned int state = p->state;
- uint64_t data = ~0;
-
- smp_rmb();
-
- /*
- * The only reason we should see this condition be false is when an
- * emulator dying races with I/O being requested.
- */
- while ( likely(state != STATE_IOREQ_NONE) )
- {
- if ( unlikely(state < prev_state) )
- {
- gdprintk(XENLOG_ERR, "Weird HVM ioreq state transition %u -> %u\n",
- prev_state, state);
- sv->pending = false;
- domain_crash(sv->vcpu->domain);
- return false; /* bail */
- }
-
- switch ( prev_state = state )
- {
- case STATE_IORESP_READY: /* IORESP_READY -> NONE */
- p->state = STATE_IOREQ_NONE;
- data = p->data;
- break;
-
- case STATE_IOREQ_READY: /* IOREQ_{READY,INPROCESS} -> IORESP_READY */
- case STATE_IOREQ_INPROCESS:
- wait_on_xen_event_channel(sv->ioreq_evtchn,
- ({ state = p->state;
- smp_rmb();
- state != prev_state; }));
- continue;
-
- default:
- gdprintk(XENLOG_ERR, "Weird HVM iorequest state %u\n", state);
- sv->pending = false;
- domain_crash(sv->vcpu->domain);
- return false; /* bail */
- }
-
- break;
- }
-
- p = &sv->vcpu->arch.hvm.hvm_io.io_req;
- if ( hvm_ioreq_needs_completion(p) )
- p->data = data;
-
- sv->pending = false;
-
- return true;
-}
-
bool arch_vcpu_ioreq_completion(enum hvm_io_completion io_completion)
{
switch ( io_completion )
@@ -198,52 +63,6 @@ bool arch_vcpu_ioreq_completion(enum hvm_io_completion io_completion)
return true;
}
-bool handle_hvm_io_completion(struct vcpu *v)
-{
- struct domain *d = v->domain;
- struct hvm_vcpu_io *vio = &v->arch.hvm.hvm_io;
- struct hvm_ioreq_server *s;
- struct hvm_ioreq_vcpu *sv;
- enum hvm_io_completion io_completion;
-
- if ( has_vpci(d) && vpci_process_pending(v) )
- {
- raise_softirq(SCHEDULE_SOFTIRQ);
- return false;
- }
-
- sv = get_pending_vcpu(v, &s);
- if ( sv && !hvm_wait_for_io(sv, get_ioreq(s, v)) )
- return false;
-
- vio->io_req.state = hvm_ioreq_needs_completion(&vio->io_req) ?
- STATE_IORESP_READY : STATE_IOREQ_NONE;
-
- msix_write_completion(v);
- vcpu_end_shutdown_deferral(v);
-
- io_completion = vio->io_completion;
- vio->io_completion = HVMIO_no_completion;
-
- switch ( io_completion )
- {
- case HVMIO_no_completion:
- break;
-
- case HVMIO_mmio_completion:
- return arch_ioreq_complete_mmio();
-
- case HVMIO_pio_completion:
- return handle_pio(vio->io_req.addr, vio->io_req.size,
- vio->io_req.dir);
-
- default:
- return arch_vcpu_ioreq_completion(io_completion);
- }
-
- return true;
-}
-
static gfn_t hvm_alloc_legacy_ioreq_gfn(struct hvm_ioreq_server *s)
{
struct domain *d = s->target;
@@ -360,93 +179,6 @@ static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
return rc;
}
-static int hvm_alloc_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
-{
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
- struct page_info *page;
-
- if ( iorp->page )
- {
- /*
- * If a guest frame has already been mapped (which may happen
- * on demand if hvm_get_ioreq_server_info() is called), then
- * allocating a page is not permitted.
- */
- if ( !gfn_eq(iorp->gfn, INVALID_GFN) )
- return -EPERM;
-
- return 0;
- }
-
- page = alloc_domheap_page(s->target, MEMF_no_refcount);
-
- if ( !page )
- return -ENOMEM;
-
- if ( !get_page_and_type(page, s->target, PGT_writable_page) )
- {
- /*
- * The domain can't possibly know about this page yet, so failure
- * here is a clear indication of something fishy going on.
- */
- domain_crash(s->emulator);
- return -ENODATA;
- }
-
- iorp->va = __map_domain_page_global(page);
- if ( !iorp->va )
- goto fail;
-
- iorp->page = page;
- clear_page(iorp->va);
- return 0;
-
- fail:
- put_page_alloc_ref(page);
- put_page_and_type(page);
-
- return -ENOMEM;
-}
-
-static void hvm_free_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
-{
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
- struct page_info *page = iorp->page;
-
- if ( !page )
- return;
-
- iorp->page = NULL;
-
- unmap_domain_page_global(iorp->va);
- iorp->va = NULL;
-
- put_page_alloc_ref(page);
- put_page_and_type(page);
-}
-
-bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
-{
- const struct hvm_ioreq_server *s;
- unsigned int id;
- bool found = false;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- FOR_EACH_IOREQ_SERVER(d, id, s)
- {
- if ( (s->ioreq.page == page) || (s->bufioreq.page == page) )
- {
- found = true;
- break;
- }
- }
-
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- return found;
-}
-
static void hvm_remove_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
{
@@ -481,125 +213,6 @@ static int hvm_add_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
return rc;
}
-static void hvm_update_ioreq_evtchn(struct hvm_ioreq_server *s,
- struct hvm_ioreq_vcpu *sv)
-{
- ASSERT(spin_is_locked(&s->lock));
-
- if ( s->ioreq.va != NULL )
- {
- ioreq_t *p = get_ioreq(s, sv->vcpu);
-
- p->vp_eport = sv->ioreq_evtchn;
- }
-}
-
-static int hvm_ioreq_server_add_vcpu(struct hvm_ioreq_server *s,
- struct vcpu *v)
-{
- struct hvm_ioreq_vcpu *sv;
- int rc;
-
- sv = xzalloc(struct hvm_ioreq_vcpu);
-
- rc = -ENOMEM;
- if ( !sv )
- goto fail1;
-
- spin_lock(&s->lock);
-
- rc = alloc_unbound_xen_event_channel(v->domain, v->vcpu_id,
- s->emulator->domain_id, NULL);
- if ( rc < 0 )
- goto fail2;
-
- sv->ioreq_evtchn = rc;
-
- if ( v->vcpu_id == 0 && HANDLE_BUFIOREQ(s) )
- {
- rc = alloc_unbound_xen_event_channel(v->domain, 0,
- s->emulator->domain_id, NULL);
- if ( rc < 0 )
- goto fail3;
-
- s->bufioreq_evtchn = rc;
- }
-
- sv->vcpu = v;
-
- list_add(&sv->list_entry, &s->ioreq_vcpu_list);
-
- if ( s->enabled )
- hvm_update_ioreq_evtchn(s, sv);
-
- spin_unlock(&s->lock);
- return 0;
-
- fail3:
- free_xen_event_channel(v->domain, sv->ioreq_evtchn);
-
- fail2:
- spin_unlock(&s->lock);
- xfree(sv);
-
- fail1:
- return rc;
-}
-
-static void hvm_ioreq_server_remove_vcpu(struct hvm_ioreq_server *s,
- struct vcpu *v)
-{
- struct hvm_ioreq_vcpu *sv;
-
- spin_lock(&s->lock);
-
- list_for_each_entry ( sv,
- &s->ioreq_vcpu_list,
- list_entry )
- {
- if ( sv->vcpu != v )
- continue;
-
- list_del(&sv->list_entry);
-
- if ( v->vcpu_id == 0 && HANDLE_BUFIOREQ(s) )
- free_xen_event_channel(v->domain, s->bufioreq_evtchn);
-
- free_xen_event_channel(v->domain, sv->ioreq_evtchn);
-
- xfree(sv);
- break;
- }
-
- spin_unlock(&s->lock);
-}
-
-static void hvm_ioreq_server_remove_all_vcpus(struct hvm_ioreq_server *s)
-{
- struct hvm_ioreq_vcpu *sv, *next;
-
- spin_lock(&s->lock);
-
- list_for_each_entry_safe ( sv,
- next,
- &s->ioreq_vcpu_list,
- list_entry )
- {
- struct vcpu *v = sv->vcpu;
-
- list_del(&sv->list_entry);
-
- if ( v->vcpu_id == 0 && HANDLE_BUFIOREQ(s) )
- free_xen_event_channel(v->domain, s->bufioreq_evtchn);
-
- free_xen_event_channel(v->domain, sv->ioreq_evtchn);
-
- xfree(sv);
- }
-
- spin_unlock(&s->lock);
-}
-
int arch_ioreq_server_map_pages(struct hvm_ioreq_server *s)
{
int rc;
@@ -621,688 +234,63 @@ void arch_ioreq_server_unmap_pages(struct hvm_ioreq_server *s)
hvm_unmap_ioreq_gfn(s, false);
}
-static int hvm_ioreq_server_alloc_pages(struct hvm_ioreq_server *s)
-{
- int rc;
-
- rc = hvm_alloc_ioreq_mfn(s, false);
-
- if ( !rc && (s->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF) )
- rc = hvm_alloc_ioreq_mfn(s, true);
-
- if ( rc )
- hvm_free_ioreq_mfn(s, false);
-
- return rc;
-}
-
-static void hvm_ioreq_server_free_pages(struct hvm_ioreq_server *s)
-{
- hvm_free_ioreq_mfn(s, true);
- hvm_free_ioreq_mfn(s, false);
-}
-
-static void hvm_ioreq_server_free_rangesets(struct hvm_ioreq_server *s)
-{
- unsigned int i;
-
- for ( i = 0; i < NR_IO_RANGE_TYPES; i++ )
- rangeset_destroy(s->range[i]);
-}
-
-static int hvm_ioreq_server_alloc_rangesets(struct hvm_ioreq_server *s,
- ioservid_t id)
-{
- unsigned int i;
- int rc;
-
- for ( i = 0; i < NR_IO_RANGE_TYPES; i++ )
- {
- char *name;
-
- rc = asprintf(&name, "ioreq_server %d %s", id,
- (i == XEN_DMOP_IO_RANGE_PORT) ? "port" :
- (i == XEN_DMOP_IO_RANGE_MEMORY) ? "memory" :
- (i == XEN_DMOP_IO_RANGE_PCI) ? "pci" :
- "");
- if ( rc )
- goto fail;
-
- s->range[i] = rangeset_new(s->target, name,
- RANGESETF_prettyprint_hex);
-
- xfree(name);
-
- rc = -ENOMEM;
- if ( !s->range[i] )
- goto fail;
-
- rangeset_limit(s->range[i], MAX_NR_IO_RANGES);
- }
-
- return 0;
-
- fail:
- hvm_ioreq_server_free_rangesets(s);
-
- return rc;
-}
-
void arch_ioreq_server_enable(struct hvm_ioreq_server *s)
{
hvm_remove_ioreq_gfn(s, false);
hvm_remove_ioreq_gfn(s, true);
}
-static void hvm_ioreq_server_enable(struct hvm_ioreq_server *s)
-{
- struct hvm_ioreq_vcpu *sv;
-
- spin_lock(&s->lock);
-
- if ( s->enabled )
- goto done;
-
- arch_ioreq_server_enable(s);
-
- s->enabled = true;
-
- list_for_each_entry ( sv,
- &s->ioreq_vcpu_list,
- list_entry )
- hvm_update_ioreq_evtchn(s, sv);
-
- done:
- spin_unlock(&s->lock);
-}
-
void arch_ioreq_server_disable(struct hvm_ioreq_server *s)
{
hvm_add_ioreq_gfn(s, true);
hvm_add_ioreq_gfn(s, false);
}
-static void hvm_ioreq_server_disable(struct hvm_ioreq_server *s)
+/* Called when target domain is paused */
+void arch_ioreq_server_destroy(struct hvm_ioreq_server *s)
{
- spin_lock(&s->lock);
-
- if ( !s->enabled )
- goto done;
-
- arch_ioreq_server_disable(s);
+ p2m_set_ioreq_server(s->target, 0, s);
+}
- s->enabled = false;
+/* Called with ioreq_server lock held */
+int arch_ioreq_server_map_mem_type(struct domain *d,
+ struct hvm_ioreq_server *s,
+ uint32_t flags)
+{
+ return p2m_set_ioreq_server(d, flags, s);
+}
- done:
- spin_unlock(&s->lock);
+void arch_ioreq_server_map_mem_type_completed(struct domain *d,
+ struct hvm_ioreq_server *s,
+ uint32_t flags)
+{
+ if ( flags == 0 && read_atomic(&p2m_get_hostp2m(d)->ioreq.entry_count) )
+ p2m_change_entry_type_global(d, p2m_ioreq_server, p2m_ram_rw);
}
-static int hvm_ioreq_server_init(struct hvm_ioreq_server *s,
- struct domain *d, int bufioreq_handling,
- ioservid_t id)
+bool arch_ioreq_server_destroy_all(struct domain *d)
{
- struct domain *currd = current->domain;
- struct vcpu *v;
- int rc;
+ return relocate_portio_handler(d, 0xcf8, 0xcf8, 4);
+}
- s->target = d;
+bool arch_ioreq_server_get_type_addr(const struct domain *d,
+ const ioreq_t *p,
+ uint8_t *type,
+ uint64_t *addr)
+{
+ unsigned int cf8 = d->arch.hvm.pci_cf8;
- get_knownalive_domain(currd);
- s->emulator = currd;
+ if ( p->type != IOREQ_TYPE_COPY && p->type != IOREQ_TYPE_PIO )
+ return false;
- spin_lock_init(&s->lock);
- INIT_LIST_HEAD(&s->ioreq_vcpu_list);
- spin_lock_init(&s->bufioreq_lock);
+ if ( p->type == IOREQ_TYPE_PIO &&
+ (p->addr & ~3) == 0xcfc &&
+ CF8_ENABLED(cf8) )
+ {
+ unsigned int x86_fam, reg;
+ pci_sbdf_t sbdf;
- s->ioreq.gfn = INVALID_GFN;
- s->bufioreq.gfn = INVALID_GFN;
-
- rc = hvm_ioreq_server_alloc_rangesets(s, id);
- if ( rc )
- return rc;
-
- s->bufioreq_handling = bufioreq_handling;
-
- for_each_vcpu ( d, v )
- {
- rc = hvm_ioreq_server_add_vcpu(s, v);
- if ( rc )
- goto fail_add;
- }
-
- return 0;
-
- fail_add:
- hvm_ioreq_server_remove_all_vcpus(s);
- arch_ioreq_server_unmap_pages(s);
-
- hvm_ioreq_server_free_rangesets(s);
-
- put_domain(s->emulator);
- return rc;
-}
-
-static void hvm_ioreq_server_deinit(struct hvm_ioreq_server *s)
-{
- ASSERT(!s->enabled);
- hvm_ioreq_server_remove_all_vcpus(s);
-
- /*
- * NOTE: It is safe to call both arch_ioreq_server_unmap_pages() and
- * hvm_ioreq_server_free_pages() in that order.
- * This is because the former will do nothing if the pages
- * are not mapped, leaving the page to be freed by the latter.
- * However if the pages are mapped then the former will set
- * the page_info pointer to NULL, meaning the latter will do
- * nothing.
- */
- arch_ioreq_server_unmap_pages(s);
- hvm_ioreq_server_free_pages(s);
-
- hvm_ioreq_server_free_rangesets(s);
-
- put_domain(s->emulator);
-}
-
-int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
- ioservid_t *id)
-{
- struct hvm_ioreq_server *s;
- unsigned int i;
- int rc;
-
- if ( bufioreq_handling > HVM_IOREQSRV_BUFIOREQ_ATOMIC )
- return -EINVAL;
-
- s = xzalloc(struct hvm_ioreq_server);
- if ( !s )
- return -ENOMEM;
-
- domain_pause(d);
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- for ( i = 0; i < MAX_NR_IOREQ_SERVERS; i++ )
- {
- if ( !GET_IOREQ_SERVER(d, i) )
- break;
- }
-
- rc = -ENOSPC;
- if ( i >= MAX_NR_IOREQ_SERVERS )
- goto fail;
-
- /*
- * It is safe to call set_ioreq_server() prior to
- * hvm_ioreq_server_init() since the target domain is paused.
- */
- set_ioreq_server(d, i, s);
-
- rc = hvm_ioreq_server_init(s, d, bufioreq_handling, i);
- if ( rc )
- {
- set_ioreq_server(d, i, NULL);
- goto fail;
- }
-
- if ( id )
- *id = i;
-
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
- domain_unpause(d);
-
- return 0;
-
- fail:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
- domain_unpause(d);
-
- xfree(s);
- return rc;
-}
-
-/* Called when target domain is paused */
-void arch_ioreq_server_destroy(struct hvm_ioreq_server *s)
-{
- p2m_set_ioreq_server(s->target, 0, s);
-}
-
-int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
-{
- struct hvm_ioreq_server *s;
- int rc;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- s = get_ioreq_server(d, id);
-
- rc = -ENOENT;
- if ( !s )
- goto out;
-
- rc = -EPERM;
- if ( s->emulator != current->domain )
- goto out;
-
- domain_pause(d);
-
- arch_ioreq_server_destroy(s);
-
- hvm_ioreq_server_disable(s);
-
- /*
- * It is safe to call hvm_ioreq_server_deinit() prior to
- * set_ioreq_server() since the target domain is paused.
- */
- hvm_ioreq_server_deinit(s);
- set_ioreq_server(d, id, NULL);
-
- domain_unpause(d);
-
- xfree(s);
-
- rc = 0;
-
- out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- return rc;
-}
-
-int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
- unsigned long *ioreq_gfn,
- unsigned long *bufioreq_gfn,
- evtchn_port_t *bufioreq_port)
-{
- struct hvm_ioreq_server *s;
- int rc;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- s = get_ioreq_server(d, id);
-
- rc = -ENOENT;
- if ( !s )
- goto out;
-
- rc = -EPERM;
- if ( s->emulator != current->domain )
- goto out;
-
- if ( ioreq_gfn || bufioreq_gfn )
- {
- rc = arch_ioreq_server_map_pages(s);
- if ( rc )
- goto out;
- }
-
- if ( ioreq_gfn )
- *ioreq_gfn = gfn_x(s->ioreq.gfn);
-
- if ( HANDLE_BUFIOREQ(s) )
- {
- if ( bufioreq_gfn )
- *bufioreq_gfn = gfn_x(s->bufioreq.gfn);
-
- if ( bufioreq_port )
- *bufioreq_port = s->bufioreq_evtchn;
- }
-
- rc = 0;
-
- out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- return rc;
-}
-
-int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
- unsigned long idx, mfn_t *mfn)
-{
- struct hvm_ioreq_server *s;
- int rc;
-
- ASSERT(is_hvm_domain(d));
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- s = get_ioreq_server(d, id);
-
- rc = -ENOENT;
- if ( !s )
- goto out;
-
- rc = -EPERM;
- if ( s->emulator != current->domain )
- goto out;
-
- rc = hvm_ioreq_server_alloc_pages(s);
- if ( rc )
- goto out;
-
- switch ( idx )
- {
- case XENMEM_resource_ioreq_server_frame_bufioreq:
- rc = -ENOENT;
- if ( !HANDLE_BUFIOREQ(s) )
- goto out;
-
- *mfn = page_to_mfn(s->bufioreq.page);
- rc = 0;
- break;
-
- case XENMEM_resource_ioreq_server_frame_ioreq(0):
- *mfn = page_to_mfn(s->ioreq.page);
- rc = 0;
- break;
-
- default:
- rc = -EINVAL;
- break;
- }
-
- out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- return rc;
-}
-
-int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint64_t start,
- uint64_t end)
-{
- struct hvm_ioreq_server *s;
- struct rangeset *r;
- int rc;
-
- if ( start > end )
- return -EINVAL;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- s = get_ioreq_server(d, id);
-
- rc = -ENOENT;
- if ( !s )
- goto out;
-
- rc = -EPERM;
- if ( s->emulator != current->domain )
- goto out;
-
- switch ( type )
- {
- case XEN_DMOP_IO_RANGE_PORT:
- case XEN_DMOP_IO_RANGE_MEMORY:
- case XEN_DMOP_IO_RANGE_PCI:
- r = s->range[type];
- break;
-
- default:
- r = NULL;
- break;
- }
-
- rc = -EINVAL;
- if ( !r )
- goto out;
-
- rc = -EEXIST;
- if ( rangeset_overlaps_range(r, start, end) )
- goto out;
-
- rc = rangeset_add_range(r, start, end);
-
- out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- return rc;
-}
-
-int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint64_t start,
- uint64_t end)
-{
- struct hvm_ioreq_server *s;
- struct rangeset *r;
- int rc;
-
- if ( start > end )
- return -EINVAL;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- s = get_ioreq_server(d, id);
-
- rc = -ENOENT;
- if ( !s )
- goto out;
-
- rc = -EPERM;
- if ( s->emulator != current->domain )
- goto out;
-
- switch ( type )
- {
- case XEN_DMOP_IO_RANGE_PORT:
- case XEN_DMOP_IO_RANGE_MEMORY:
- case XEN_DMOP_IO_RANGE_PCI:
- r = s->range[type];
- break;
-
- default:
- r = NULL;
- break;
- }
-
- rc = -EINVAL;
- if ( !r )
- goto out;
-
- rc = -ENOENT;
- if ( !rangeset_contains_range(r, start, end) )
- goto out;
-
- rc = rangeset_remove_range(r, start, end);
-
- out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- return rc;
-}
-
-/* Called with ioreq_server lock held */
-int arch_ioreq_server_map_mem_type(struct domain *d,
- struct hvm_ioreq_server *s,
- uint32_t flags)
-{
- return p2m_set_ioreq_server(d, flags, s);
-}
-
-void arch_ioreq_server_map_mem_type_completed(struct domain *d,
- struct hvm_ioreq_server *s,
- uint32_t flags)
-{
- if ( flags == 0 && read_atomic(&p2m_get_hostp2m(d)->ioreq.entry_count) )
- p2m_change_entry_type_global(d, p2m_ioreq_server, p2m_ram_rw);
-}
-
-/*
- * Map or unmap an ioreq server to specific memory type. For now, only
- * HVMMEM_ioreq_server is supported, and in the future new types can be
- * introduced, e.g. HVMMEM_ioreq_serverX mapped to ioreq server X. And
- * currently, only write operations are to be forwarded to an ioreq server.
- * Support for the emulation of read operations can be added when an ioreq
- * server has such requirement in the future.
- */
-int hvm_map_mem_type_to_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint32_t flags)
-{
- struct hvm_ioreq_server *s;
- int rc;
-
- if ( type != HVMMEM_ioreq_server )
- return -EINVAL;
-
- if ( flags & ~XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
- return -EINVAL;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- s = get_ioreq_server(d, id);
-
- rc = -ENOENT;
- if ( !s )
- goto out;
-
- rc = -EPERM;
- if ( s->emulator != current->domain )
- goto out;
-
- rc = arch_ioreq_server_map_mem_type(d, s, flags);
-
- out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- if ( rc == 0 )
- arch_ioreq_server_map_mem_type_completed(d, s, flags);
-
- return rc;
-}
-
-int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
- bool enabled)
-{
- struct hvm_ioreq_server *s;
- int rc;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- s = get_ioreq_server(d, id);
-
- rc = -ENOENT;
- if ( !s )
- goto out;
-
- rc = -EPERM;
- if ( s->emulator != current->domain )
- goto out;
-
- domain_pause(d);
-
- if ( enabled )
- hvm_ioreq_server_enable(s);
- else
- hvm_ioreq_server_disable(s);
-
- domain_unpause(d);
-
- rc = 0;
-
- out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
- return rc;
-}
-
-int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
-{
- struct hvm_ioreq_server *s;
- unsigned int id;
- int rc;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- FOR_EACH_IOREQ_SERVER(d, id, s)
- {
- rc = hvm_ioreq_server_add_vcpu(s, v);
- if ( rc )
- goto fail;
- }
-
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- return 0;
-
- fail:
- while ( ++id != MAX_NR_IOREQ_SERVERS )
- {
- s = GET_IOREQ_SERVER(d, id);
-
- if ( !s )
- continue;
-
- hvm_ioreq_server_remove_vcpu(s, v);
- }
-
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- return rc;
-}
-
-void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v)
-{
- struct hvm_ioreq_server *s;
- unsigned int id;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- FOR_EACH_IOREQ_SERVER(d, id, s)
- hvm_ioreq_server_remove_vcpu(s, v);
-
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-}
-
-bool arch_ioreq_server_destroy_all(struct domain *d)
-{
- return relocate_portio_handler(d, 0xcf8, 0xcf8, 4);
-}
-
-void hvm_destroy_all_ioreq_servers(struct domain *d)
-{
- struct hvm_ioreq_server *s;
- unsigned int id;
-
- if ( !arch_ioreq_server_destroy_all(d) )
- return;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- /* No need to domain_pause() as the domain is being torn down */
-
- FOR_EACH_IOREQ_SERVER(d, id, s)
- {
- hvm_ioreq_server_disable(s);
-
- /*
- * It is safe to call hvm_ioreq_server_deinit() prior to
- * set_ioreq_server() since the target domain is being destroyed.
- */
- hvm_ioreq_server_deinit(s);
- set_ioreq_server(d, id, NULL);
-
- xfree(s);
- }
-
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-}
-
-bool arch_ioreq_server_get_type_addr(const struct domain *d,
- const ioreq_t *p,
- uint8_t *type,
- uint64_t *addr)
-{
- unsigned int cf8 = d->arch.hvm.pci_cf8;
-
- if ( p->type != IOREQ_TYPE_COPY && p->type != IOREQ_TYPE_PIO )
- return false;
-
- if ( p->type == IOREQ_TYPE_PIO &&
- (p->addr & ~3) == 0xcfc &&
- CF8_ENABLED(cf8) )
- {
- unsigned int x86_fam, reg;
- pci_sbdf_t sbdf;
-
- reg = hvm_pci_decode_addr(cf8, p->addr, &sbdf);
+ reg = hvm_pci_decode_addr(cf8, p->addr, &sbdf);
/* PCI config data cycle */
*type = XEN_DMOP_IO_RANGE_PCI;
@@ -1331,233 +319,6 @@ bool arch_ioreq_server_get_type_addr(const struct domain *d,
return true;
}
-struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
- ioreq_t *p)
-{
- struct hvm_ioreq_server *s;
- uint8_t type;
- uint64_t addr;
- unsigned int id;
-
- if ( !arch_ioreq_server_get_type_addr(d, p, &type, &addr) )
- return NULL;
-
- FOR_EACH_IOREQ_SERVER(d, id, s)
- {
- struct rangeset *r;
-
- if ( !s->enabled )
- continue;
-
- r = s->range[type];
-
- switch ( type )
- {
- unsigned long start, end;
-
- case XEN_DMOP_IO_RANGE_PORT:
- start = addr;
- end = start + p->size - 1;
- if ( rangeset_contains_range(r, start, end) )
- return s;
-
- break;
-
- case XEN_DMOP_IO_RANGE_MEMORY:
- start = hvm_mmio_first_byte(p);
- end = hvm_mmio_last_byte(p);
-
- if ( rangeset_contains_range(r, start, end) )
- return s;
-
- break;
-
- case XEN_DMOP_IO_RANGE_PCI:
- if ( rangeset_contains_singleton(r, addr >> 32) )
- {
- p->type = IOREQ_TYPE_PCI_CONFIG;
- p->addr = addr;
- return s;
- }
-
- break;
- }
- }
-
- return NULL;
-}
-
-static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
-{
- struct domain *d = current->domain;
- struct hvm_ioreq_page *iorp;
- buffered_iopage_t *pg;
- buf_ioreq_t bp = { .data = p->data,
- .addr = p->addr,
- .type = p->type,
- .dir = p->dir };
- /* Timeoffset sends 64b data, but no address. Use two consecutive slots. */
- int qw = 0;
-
- /* Ensure buffered_iopage fits in a page */
- BUILD_BUG_ON(sizeof(buffered_iopage_t) > PAGE_SIZE);
-
- iorp = &s->bufioreq;
- pg = iorp->va;
-
- if ( !pg )
- return IOREQ_STATUS_UNHANDLED;
-
- /*
- * Return 0 for the cases we can't deal with:
- * - 'addr' is only a 20-bit field, so we cannot address beyond 1MB
- * - we cannot buffer accesses to guest memory buffers, as the guest
- * may expect the memory buffer to be synchronously accessed
- * - the count field is usually used with data_is_ptr and since we don't
- * support data_is_ptr we do not waste space for the count field either
- */
- if ( (p->addr > 0xffffful) || p->data_is_ptr || (p->count != 1) )
- return 0;
-
- switch ( p->size )
- {
- case 1:
- bp.size = 0;
- break;
- case 2:
- bp.size = 1;
- break;
- case 4:
- bp.size = 2;
- break;
- case 8:
- bp.size = 3;
- qw = 1;
- break;
- default:
- gdprintk(XENLOG_WARNING, "unexpected ioreq size: %u\n", p->size);
- return IOREQ_STATUS_UNHANDLED;
- }
-
- spin_lock(&s->bufioreq_lock);
-
- if ( (pg->ptrs.write_pointer - pg->ptrs.read_pointer) >=
- (IOREQ_BUFFER_SLOT_NUM - qw) )
- {
- /* The queue is full: send the iopacket through the normal path. */
- spin_unlock(&s->bufioreq_lock);
- return IOREQ_STATUS_UNHANDLED;
- }
-
- pg->buf_ioreq[pg->ptrs.write_pointer % IOREQ_BUFFER_SLOT_NUM] = bp;
-
- if ( qw )
- {
- bp.data = p->data >> 32;
- pg->buf_ioreq[(pg->ptrs.write_pointer+1) % IOREQ_BUFFER_SLOT_NUM] = bp;
- }
-
- /* Make the ioreq_t visible /before/ write_pointer. */
- smp_wmb();
- pg->ptrs.write_pointer += qw ? 2 : 1;
-
- /* Canonicalize read/write pointers to prevent their overflow. */
- while ( (s->bufioreq_handling == HVM_IOREQSRV_BUFIOREQ_ATOMIC) &&
- qw++ < IOREQ_BUFFER_SLOT_NUM &&
- pg->ptrs.read_pointer >= IOREQ_BUFFER_SLOT_NUM )
- {
- union bufioreq_pointers old = pg->ptrs, new;
- unsigned int n = old.read_pointer / IOREQ_BUFFER_SLOT_NUM;
-
- new.read_pointer = old.read_pointer - n * IOREQ_BUFFER_SLOT_NUM;
- new.write_pointer = old.write_pointer - n * IOREQ_BUFFER_SLOT_NUM;
- cmpxchg(&pg->ptrs.full, old.full, new.full);
- }
-
- notify_via_xen_event_channel(d, s->bufioreq_evtchn);
- spin_unlock(&s->bufioreq_lock);
-
- return IOREQ_STATUS_HANDLED;
-}
-
-int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
- bool buffered)
-{
- struct vcpu *curr = current;
- struct domain *d = curr->domain;
- struct hvm_ioreq_vcpu *sv;
-
- ASSERT(s);
-
- if ( buffered )
- return hvm_send_buffered_ioreq(s, proto_p);
-
- if ( unlikely(!vcpu_start_shutdown_deferral(curr)) )
- return IOREQ_STATUS_RETRY;
-
- list_for_each_entry ( sv,
- &s->ioreq_vcpu_list,
- list_entry )
- {
- if ( sv->vcpu == curr )
- {
- evtchn_port_t port = sv->ioreq_evtchn;
- ioreq_t *p = get_ioreq(s, curr);
-
- if ( unlikely(p->state != STATE_IOREQ_NONE) )
- {
- gprintk(XENLOG_ERR, "device model set bad IO state %d\n",
- p->state);
- break;
- }
-
- if ( unlikely(p->vp_eport != port) )
- {
- gprintk(XENLOG_ERR, "device model set bad event channel %d\n",
- p->vp_eport);
- break;
- }
-
- proto_p->state = STATE_IOREQ_NONE;
- proto_p->vp_eport = port;
- *p = *proto_p;
-
- prepare_wait_on_xen_event_channel(port);
-
- /*
- * Following happens /after/ blocking and setting up ioreq
- * contents. prepare_wait_on_xen_event_channel() is an implicit
- * barrier.
- */
- p->state = STATE_IOREQ_READY;
- notify_via_xen_event_channel(d, port);
-
- sv->pending = true;
- return IOREQ_STATUS_RETRY;
- }
- }
-
- return IOREQ_STATUS_UNHANDLED;
-}
-
-unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered)
-{
- struct domain *d = current->domain;
- struct hvm_ioreq_server *s;
- unsigned int id, failed = 0;
-
- FOR_EACH_IOREQ_SERVER(d, id, s)
- {
- if ( !s->enabled )
- continue;
-
- if ( hvm_send_ioreq(s, p, buffered) == IOREQ_STATUS_UNHANDLED )
- failed++;
- }
-
- return failed;
-}
-
static int hvm_access_cf8(
int dir, unsigned int port, unsigned int bytes, uint32_t *val)
{
@@ -1575,13 +336,6 @@ void arch_ioreq_domain_init(struct domain *d)
register_portio_handler(d, 0xcf8, 4, hvm_access_cf8);
}
-void hvm_ioreq_init(struct domain *d)
-{
- spin_lock_init(&d->arch.hvm.ioreq_server.lock);
-
- arch_ioreq_domain_init(d);
-}
-
/*
* Local variables:
* mode: C
diff --git a/xen/arch/x86/hvm/stdvga.c b/xen/arch/x86/hvm/stdvga.c
index e267513..fd7cadb 100644
--- a/xen/arch/x86/hvm/stdvga.c
+++ b/xen/arch/x86/hvm/stdvga.c
@@ -27,10 +27,10 @@
* can have side effects.
*/
+#include <xen/ioreq.h>
#include <xen/types.h>
#include <xen/sched.h>
#include <xen/domain_page.h>
-#include <asm/hvm/ioreq.h>
#include <asm/hvm/support.h>
#include <xen/numa.h>
#include <xen/paging.h>
diff --git a/xen/arch/x86/hvm/vmx/vvmx.c b/xen/arch/x86/hvm/vmx/vvmx.c
index 3a37e9e..0ddb6a4 100644
--- a/xen/arch/x86/hvm/vmx/vvmx.c
+++ b/xen/arch/x86/hvm/vmx/vvmx.c
@@ -19,10 +19,11 @@
*
*/
+#include <xen/ioreq.h>
+
#include <asm/types.h>
#include <asm/mtrr.h>
#include <asm/p2m.h>
-#include <asm/hvm/ioreq.h>
#include <asm/hvm/vmx/vmx.h>
#include <asm/hvm/vmx/vvmx.h>
#include <asm/hvm/nestedhvm.h>
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 63e9fae..59eb5c8 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -100,6 +100,7 @@
*/
#include <xen/init.h>
+#include <xen/ioreq.h>
#include <xen/kernel.h>
#include <xen/lib.h>
#include <xen/mm.h>
@@ -140,7 +141,6 @@
#include <asm/io_apic.h>
#include <asm/pci.h>
#include <asm/guest.h>
-#include <asm/hvm/ioreq.h>
#include <asm/pv/domain.h>
#include <asm/pv/mm.h>
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index df95c4d..4a88824 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -20,6 +20,7 @@
* along with this program; If not, see <http://www.gnu.org/licenses/>.
*/
+#include <xen/ioreq.h>
#include <xen/types.h>
#include <xen/mm.h>
#include <xen/trace.h>
@@ -34,7 +35,6 @@
#include <asm/current.h>
#include <asm/flushtlb.h>
#include <asm/shadow.h>
-#include <asm/hvm/ioreq.h>
#include <xen/numa.h>
#include "private.h"
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index b5c91a1..cf32a07 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -139,6 +139,9 @@ config HYPFS_CONFIG
Disable this option in case you want to spare some memory or you
want to hide the .config contents from dom0.
+config IOREQ_SERVER
+ bool
+
config KEXEC
bool "kexec support"
default y
diff --git a/xen/common/Makefile b/xen/common/Makefile
index d751315..2e3c234 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -14,6 +14,7 @@ obj-$(CONFIG_GRANT_TABLE) += grant_table.o
obj-y += guestcopy.o
obj-bin-y += gunzip.init.o
obj-$(CONFIG_HYPFS) += hypfs.o
+obj-$(CONFIG_IOREQ_SERVER) += ioreq.o
obj-y += irq.o
obj-y += kernel.o
obj-y += keyhandler.o
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
new file mode 100644
index 0000000..4e7d91b
--- /dev/null
+++ b/xen/common/ioreq.c
@@ -0,0 +1,1290 @@
+/*
+ * ioreq.c: hardware virtual machine I/O emulation
+ *
+ * Copyright (c) 2016 Citrix Systems Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/domain.h>
+#include <xen/domain_page.h>
+#include <xen/event.h>
+#include <xen/init.h>
+#include <xen/ioreq.h>
+#include <xen/irq.h>
+#include <xen/lib.h>
+#include <xen/paging.h>
+#include <xen/sched.h>
+#include <xen/softirq.h>
+#include <xen/trace.h>
+#include <xen/vpci.h>
+
+#include <asm/ioreq.h>
+
+#include <public/hvm/ioreq.h>
+#include <public/hvm/params.h>
+
+static void set_ioreq_server(struct domain *d, unsigned int id,
+ struct hvm_ioreq_server *s)
+{
+ ASSERT(id < MAX_NR_IOREQ_SERVERS);
+ ASSERT(!s || !d->arch.hvm.ioreq_server.server[id]);
+
+ d->arch.hvm.ioreq_server.server[id] = s;
+}
+
+#define GET_IOREQ_SERVER(d, id) \
+ (d)->arch.hvm.ioreq_server.server[id]
+
+static struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
+ unsigned int id)
+{
+ if ( id >= MAX_NR_IOREQ_SERVERS )
+ return NULL;
+
+ return GET_IOREQ_SERVER(d, id);
+}
+
+/*
+ * Iterate over all possible ioreq servers.
+ *
+ * NOTE: The iteration is backwards such that more recently created
+ * ioreq servers are favoured in hvm_select_ioreq_server().
+ * This is a semantic that previously existed when ioreq servers
+ * were held in a linked list.
+ */
+#define FOR_EACH_IOREQ_SERVER(d, id, s) \
+ for ( (id) = MAX_NR_IOREQ_SERVERS; (id) != 0; ) \
+ if ( !(s = GET_IOREQ_SERVER(d, --(id))) ) \
+ continue; \
+ else
+
+static ioreq_t *get_ioreq(struct hvm_ioreq_server *s, struct vcpu *v)
+{
+ shared_iopage_t *p = s->ioreq.va;
+
+ ASSERT((v == current) || !vcpu_runnable(v));
+ ASSERT(p != NULL);
+
+ return &p->vcpu_ioreq[v->vcpu_id];
+}
+
+static struct hvm_ioreq_vcpu *get_pending_vcpu(const struct vcpu *v,
+ struct hvm_ioreq_server **srvp)
+{
+ struct domain *d = v->domain;
+ struct hvm_ioreq_server *s;
+ unsigned int id;
+
+ FOR_EACH_IOREQ_SERVER(d, id, s)
+ {
+ struct hvm_ioreq_vcpu *sv;
+
+ list_for_each_entry ( sv,
+ &s->ioreq_vcpu_list,
+ list_entry )
+ {
+ if ( sv->vcpu == v && sv->pending )
+ {
+ if ( srvp )
+ *srvp = s;
+ return sv;
+ }
+ }
+ }
+
+ return NULL;
+}
+
+bool hvm_io_pending(struct vcpu *v)
+{
+ return get_pending_vcpu(v, NULL);
+}
+
+static bool hvm_wait_for_io(struct hvm_ioreq_vcpu *sv, ioreq_t *p)
+{
+ unsigned int prev_state = STATE_IOREQ_NONE;
+ unsigned int state = p->state;
+ uint64_t data = ~0;
+
+ smp_rmb();
+
+ /*
+ * The only reason we should see this condition be false is when an
+ * emulator dying races with I/O being requested.
+ */
+ while ( likely(state != STATE_IOREQ_NONE) )
+ {
+ if ( unlikely(state < prev_state) )
+ {
+ gdprintk(XENLOG_ERR, "Weird HVM ioreq state transition %u -> %u\n",
+ prev_state, state);
+ sv->pending = false;
+ domain_crash(sv->vcpu->domain);
+ return false; /* bail */
+ }
+
+ switch ( prev_state = state )
+ {
+ case STATE_IORESP_READY: /* IORESP_READY -> NONE */
+ p->state = STATE_IOREQ_NONE;
+ data = p->data;
+ break;
+
+ case STATE_IOREQ_READY: /* IOREQ_{READY,INPROCESS} -> IORESP_READY */
+ case STATE_IOREQ_INPROCESS:
+ wait_on_xen_event_channel(sv->ioreq_evtchn,
+ ({ state = p->state;
+ smp_rmb();
+ state != prev_state; }));
+ continue;
+
+ default:
+ gdprintk(XENLOG_ERR, "Weird HVM iorequest state %u\n", state);
+ sv->pending = false;
+ domain_crash(sv->vcpu->domain);
+ return false; /* bail */
+ }
+
+ break;
+ }
+
+ p = &sv->vcpu->arch.hvm.hvm_io.io_req;
+ if ( hvm_ioreq_needs_completion(p) )
+ p->data = data;
+
+ sv->pending = false;
+
+ return true;
+}
+
+bool handle_hvm_io_completion(struct vcpu *v)
+{
+ struct domain *d = v->domain;
+ struct hvm_vcpu_io *vio = &v->arch.hvm.hvm_io;
+ struct hvm_ioreq_server *s;
+ struct hvm_ioreq_vcpu *sv;
+ enum hvm_io_completion io_completion;
+
+ if ( has_vpci(d) && vpci_process_pending(v) )
+ {
+ raise_softirq(SCHEDULE_SOFTIRQ);
+ return false;
+ }
+
+ sv = get_pending_vcpu(v, &s);
+ if ( sv && !hvm_wait_for_io(sv, get_ioreq(s, v)) )
+ return false;
+
+ vio->io_req.state = hvm_ioreq_needs_completion(&vio->io_req) ?
+ STATE_IORESP_READY : STATE_IOREQ_NONE;
+
+ msix_write_completion(v);
+ vcpu_end_shutdown_deferral(v);
+
+ io_completion = vio->io_completion;
+ vio->io_completion = HVMIO_no_completion;
+
+ switch ( io_completion )
+ {
+ case HVMIO_no_completion:
+ break;
+
+ case HVMIO_mmio_completion:
+ return arch_ioreq_complete_mmio();
+
+ case HVMIO_pio_completion:
+ return handle_pio(vio->io_req.addr, vio->io_req.size,
+ vio->io_req.dir);
+
+ default:
+ return arch_vcpu_ioreq_completion(io_completion);
+ }
+
+ return true;
+}
+
+static int hvm_alloc_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
+{
+ struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+ struct page_info *page;
+
+ if ( iorp->page )
+ {
+ /*
+ * If a guest frame has already been mapped (which may happen
+ * on demand if hvm_get_ioreq_server_info() is called), then
+ * allocating a page is not permitted.
+ */
+ if ( !gfn_eq(iorp->gfn, INVALID_GFN) )
+ return -EPERM;
+
+ return 0;
+ }
+
+ page = alloc_domheap_page(s->target, MEMF_no_refcount);
+
+ if ( !page )
+ return -ENOMEM;
+
+ if ( !get_page_and_type(page, s->target, PGT_writable_page) )
+ {
+ /*
+ * The domain can't possibly know about this page yet, so failure
+ * here is a clear indication of something fishy going on.
+ */
+ domain_crash(s->emulator);
+ return -ENODATA;
+ }
+
+ iorp->va = __map_domain_page_global(page);
+ if ( !iorp->va )
+ goto fail;
+
+ iorp->page = page;
+ clear_page(iorp->va);
+ return 0;
+
+ fail:
+ put_page_alloc_ref(page);
+ put_page_and_type(page);
+
+ return -ENOMEM;
+}
+
+static void hvm_free_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
+{
+ struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+ struct page_info *page = iorp->page;
+
+ if ( !page )
+ return;
+
+ iorp->page = NULL;
+
+ unmap_domain_page_global(iorp->va);
+ iorp->va = NULL;
+
+ put_page_alloc_ref(page);
+ put_page_and_type(page);
+}
+
+bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
+{
+ const struct hvm_ioreq_server *s;
+ unsigned int id;
+ bool found = false;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ FOR_EACH_IOREQ_SERVER(d, id, s)
+ {
+ if ( (s->ioreq.page == page) || (s->bufioreq.page == page) )
+ {
+ found = true;
+ break;
+ }
+ }
+
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ return found;
+}
+
+static void hvm_update_ioreq_evtchn(struct hvm_ioreq_server *s,
+ struct hvm_ioreq_vcpu *sv)
+{
+ ASSERT(spin_is_locked(&s->lock));
+
+ if ( s->ioreq.va != NULL )
+ {
+ ioreq_t *p = get_ioreq(s, sv->vcpu);
+
+ p->vp_eport = sv->ioreq_evtchn;
+ }
+}
+
+static int hvm_ioreq_server_add_vcpu(struct hvm_ioreq_server *s,
+ struct vcpu *v)
+{
+ struct hvm_ioreq_vcpu *sv;
+ int rc;
+
+ sv = xzalloc(struct hvm_ioreq_vcpu);
+
+ rc = -ENOMEM;
+ if ( !sv )
+ goto fail1;
+
+ spin_lock(&s->lock);
+
+ rc = alloc_unbound_xen_event_channel(v->domain, v->vcpu_id,
+ s->emulator->domain_id, NULL);
+ if ( rc < 0 )
+ goto fail2;
+
+ sv->ioreq_evtchn = rc;
+
+ if ( v->vcpu_id == 0 && HANDLE_BUFIOREQ(s) )
+ {
+ rc = alloc_unbound_xen_event_channel(v->domain, 0,
+ s->emulator->domain_id, NULL);
+ if ( rc < 0 )
+ goto fail3;
+
+ s->bufioreq_evtchn = rc;
+ }
+
+ sv->vcpu = v;
+
+ list_add(&sv->list_entry, &s->ioreq_vcpu_list);
+
+ if ( s->enabled )
+ hvm_update_ioreq_evtchn(s, sv);
+
+ spin_unlock(&s->lock);
+ return 0;
+
+ fail3:
+ free_xen_event_channel(v->domain, sv->ioreq_evtchn);
+
+ fail2:
+ spin_unlock(&s->lock);
+ xfree(sv);
+
+ fail1:
+ return rc;
+}
+
+static void hvm_ioreq_server_remove_vcpu(struct hvm_ioreq_server *s,
+ struct vcpu *v)
+{
+ struct hvm_ioreq_vcpu *sv;
+
+ spin_lock(&s->lock);
+
+ list_for_each_entry ( sv,
+ &s->ioreq_vcpu_list,
+ list_entry )
+ {
+ if ( sv->vcpu != v )
+ continue;
+
+ list_del(&sv->list_entry);
+
+ if ( v->vcpu_id == 0 && HANDLE_BUFIOREQ(s) )
+ free_xen_event_channel(v->domain, s->bufioreq_evtchn);
+
+ free_xen_event_channel(v->domain, sv->ioreq_evtchn);
+
+ xfree(sv);
+ break;
+ }
+
+ spin_unlock(&s->lock);
+}
+
+static void hvm_ioreq_server_remove_all_vcpus(struct hvm_ioreq_server *s)
+{
+ struct hvm_ioreq_vcpu *sv, *next;
+
+ spin_lock(&s->lock);
+
+ list_for_each_entry_safe ( sv,
+ next,
+ &s->ioreq_vcpu_list,
+ list_entry )
+ {
+ struct vcpu *v = sv->vcpu;
+
+ list_del(&sv->list_entry);
+
+ if ( v->vcpu_id == 0 && HANDLE_BUFIOREQ(s) )
+ free_xen_event_channel(v->domain, s->bufioreq_evtchn);
+
+ free_xen_event_channel(v->domain, sv->ioreq_evtchn);
+
+ xfree(sv);
+ }
+
+ spin_unlock(&s->lock);
+}
+
+static int hvm_ioreq_server_alloc_pages(struct hvm_ioreq_server *s)
+{
+ int rc;
+
+ rc = hvm_alloc_ioreq_mfn(s, false);
+
+ if ( !rc && (s->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF) )
+ rc = hvm_alloc_ioreq_mfn(s, true);
+
+ if ( rc )
+ hvm_free_ioreq_mfn(s, false);
+
+ return rc;
+}
+
+static void hvm_ioreq_server_free_pages(struct hvm_ioreq_server *s)
+{
+ hvm_free_ioreq_mfn(s, true);
+ hvm_free_ioreq_mfn(s, false);
+}
+
+static void hvm_ioreq_server_free_rangesets(struct hvm_ioreq_server *s)
+{
+ unsigned int i;
+
+ for ( i = 0; i < NR_IO_RANGE_TYPES; i++ )
+ rangeset_destroy(s->range[i]);
+}
+
+static int hvm_ioreq_server_alloc_rangesets(struct hvm_ioreq_server *s,
+ ioservid_t id)
+{
+ unsigned int i;
+ int rc;
+
+ for ( i = 0; i < NR_IO_RANGE_TYPES; i++ )
+ {
+ char *name;
+
+ rc = asprintf(&name, "ioreq_server %d %s", id,
+ (i == XEN_DMOP_IO_RANGE_PORT) ? "port" :
+ (i == XEN_DMOP_IO_RANGE_MEMORY) ? "memory" :
+ (i == XEN_DMOP_IO_RANGE_PCI) ? "pci" :
+ "");
+ if ( rc )
+ goto fail;
+
+ s->range[i] = rangeset_new(s->target, name,
+ RANGESETF_prettyprint_hex);
+
+ xfree(name);
+
+ rc = -ENOMEM;
+ if ( !s->range[i] )
+ goto fail;
+
+ rangeset_limit(s->range[i], MAX_NR_IO_RANGES);
+ }
+
+ return 0;
+
+ fail:
+ hvm_ioreq_server_free_rangesets(s);
+
+ return rc;
+}
+
+static void hvm_ioreq_server_enable(struct hvm_ioreq_server *s)
+{
+ struct hvm_ioreq_vcpu *sv;
+
+ spin_lock(&s->lock);
+
+ if ( s->enabled )
+ goto done;
+
+ arch_ioreq_server_enable(s);
+
+ s->enabled = true;
+
+ list_for_each_entry ( sv,
+ &s->ioreq_vcpu_list,
+ list_entry )
+ hvm_update_ioreq_evtchn(s, sv);
+
+ done:
+ spin_unlock(&s->lock);
+}
+
+static void hvm_ioreq_server_disable(struct hvm_ioreq_server *s)
+{
+ spin_lock(&s->lock);
+
+ if ( !s->enabled )
+ goto done;
+
+ arch_ioreq_server_disable(s);
+
+ s->enabled = false;
+
+ done:
+ spin_unlock(&s->lock);
+}
+
+static int hvm_ioreq_server_init(struct hvm_ioreq_server *s,
+ struct domain *d, int bufioreq_handling,
+ ioservid_t id)
+{
+ struct domain *currd = current->domain;
+ struct vcpu *v;
+ int rc;
+
+ s->target = d;
+
+ get_knownalive_domain(currd);
+ s->emulator = currd;
+
+ spin_lock_init(&s->lock);
+ INIT_LIST_HEAD(&s->ioreq_vcpu_list);
+ spin_lock_init(&s->bufioreq_lock);
+
+ s->ioreq.gfn = INVALID_GFN;
+ s->bufioreq.gfn = INVALID_GFN;
+
+ rc = hvm_ioreq_server_alloc_rangesets(s, id);
+ if ( rc )
+ return rc;
+
+ s->bufioreq_handling = bufioreq_handling;
+
+ for_each_vcpu ( d, v )
+ {
+ rc = hvm_ioreq_server_add_vcpu(s, v);
+ if ( rc )
+ goto fail_add;
+ }
+
+ return 0;
+
+ fail_add:
+ hvm_ioreq_server_remove_all_vcpus(s);
+ arch_ioreq_server_unmap_pages(s);
+
+ hvm_ioreq_server_free_rangesets(s);
+
+ put_domain(s->emulator);
+ return rc;
+}
+
+static void hvm_ioreq_server_deinit(struct hvm_ioreq_server *s)
+{
+ ASSERT(!s->enabled);
+ hvm_ioreq_server_remove_all_vcpus(s);
+
+ /*
+ * NOTE: It is safe to call both arch_ioreq_server_unmap_pages() and
+ * hvm_ioreq_server_free_pages() in that order.
+ * This is because the former will do nothing if the pages
+ * are not mapped, leaving the page to be freed by the latter.
+ * However if the pages are mapped then the former will set
+ * the page_info pointer to NULL, meaning the latter will do
+ * nothing.
+ */
+ arch_ioreq_server_unmap_pages(s);
+ hvm_ioreq_server_free_pages(s);
+
+ hvm_ioreq_server_free_rangesets(s);
+
+ put_domain(s->emulator);
+}
+
+int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
+ ioservid_t *id)
+{
+ struct hvm_ioreq_server *s;
+ unsigned int i;
+ int rc;
+
+ if ( bufioreq_handling > HVM_IOREQSRV_BUFIOREQ_ATOMIC )
+ return -EINVAL;
+
+ s = xzalloc(struct hvm_ioreq_server);
+ if ( !s )
+ return -ENOMEM;
+
+ domain_pause(d);
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ for ( i = 0; i < MAX_NR_IOREQ_SERVERS; i++ )
+ {
+ if ( !GET_IOREQ_SERVER(d, i) )
+ break;
+ }
+
+ rc = -ENOSPC;
+ if ( i >= MAX_NR_IOREQ_SERVERS )
+ goto fail;
+
+ /*
+ * It is safe to call set_ioreq_server() prior to
+ * hvm_ioreq_server_init() since the target domain is paused.
+ */
+ set_ioreq_server(d, i, s);
+
+ rc = hvm_ioreq_server_init(s, d, bufioreq_handling, i);
+ if ( rc )
+ {
+ set_ioreq_server(d, i, NULL);
+ goto fail;
+ }
+
+ if ( id )
+ *id = i;
+
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ domain_unpause(d);
+
+ return 0;
+
+ fail:
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ domain_unpause(d);
+
+ xfree(s);
+ return rc;
+}
+
+int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
+{
+ struct hvm_ioreq_server *s;
+ int rc;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ s = get_ioreq_server(d, id);
+
+ rc = -ENOENT;
+ if ( !s )
+ goto out;
+
+ rc = -EPERM;
+ if ( s->emulator != current->domain )
+ goto out;
+
+ domain_pause(d);
+
+ arch_ioreq_server_destroy(s);
+
+ hvm_ioreq_server_disable(s);
+
+ /*
+ * It is safe to call hvm_ioreq_server_deinit() prior to
+ * set_ioreq_server() since the target domain is paused.
+ */
+ hvm_ioreq_server_deinit(s);
+ set_ioreq_server(d, id, NULL);
+
+ domain_unpause(d);
+
+ xfree(s);
+
+ rc = 0;
+
+ out:
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ return rc;
+}
+
+int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
+ unsigned long *ioreq_gfn,
+ unsigned long *bufioreq_gfn,
+ evtchn_port_t *bufioreq_port)
+{
+ struct hvm_ioreq_server *s;
+ int rc;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ s = get_ioreq_server(d, id);
+
+ rc = -ENOENT;
+ if ( !s )
+ goto out;
+
+ rc = -EPERM;
+ if ( s->emulator != current->domain )
+ goto out;
+
+ if ( ioreq_gfn || bufioreq_gfn )
+ {
+ rc = arch_ioreq_server_map_pages(s);
+ if ( rc )
+ goto out;
+ }
+
+ if ( ioreq_gfn )
+ *ioreq_gfn = gfn_x(s->ioreq.gfn);
+
+ if ( HANDLE_BUFIOREQ(s) )
+ {
+ if ( bufioreq_gfn )
+ *bufioreq_gfn = gfn_x(s->bufioreq.gfn);
+
+ if ( bufioreq_port )
+ *bufioreq_port = s->bufioreq_evtchn;
+ }
+
+ rc = 0;
+
+ out:
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ return rc;
+}
+
+int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
+ unsigned long idx, mfn_t *mfn)
+{
+ struct hvm_ioreq_server *s;
+ int rc;
+
+ ASSERT(is_hvm_domain(d));
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ s = get_ioreq_server(d, id);
+
+ rc = -ENOENT;
+ if ( !s )
+ goto out;
+
+ rc = -EPERM;
+ if ( s->emulator != current->domain )
+ goto out;
+
+ rc = hvm_ioreq_server_alloc_pages(s);
+ if ( rc )
+ goto out;
+
+ switch ( idx )
+ {
+ case XENMEM_resource_ioreq_server_frame_bufioreq:
+ rc = -ENOENT;
+ if ( !HANDLE_BUFIOREQ(s) )
+ goto out;
+
+ *mfn = page_to_mfn(s->bufioreq.page);
+ rc = 0;
+ break;
+
+ case XENMEM_resource_ioreq_server_frame_ioreq(0):
+ *mfn = page_to_mfn(s->ioreq.page);
+ rc = 0;
+ break;
+
+ default:
+ rc = -EINVAL;
+ break;
+ }
+
+ out:
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ return rc;
+}
+
+int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
+ uint32_t type, uint64_t start,
+ uint64_t end)
+{
+ struct hvm_ioreq_server *s;
+ struct rangeset *r;
+ int rc;
+
+ if ( start > end )
+ return -EINVAL;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ s = get_ioreq_server(d, id);
+
+ rc = -ENOENT;
+ if ( !s )
+ goto out;
+
+ rc = -EPERM;
+ if ( s->emulator != current->domain )
+ goto out;
+
+ switch ( type )
+ {
+ case XEN_DMOP_IO_RANGE_PORT:
+ case XEN_DMOP_IO_RANGE_MEMORY:
+ case XEN_DMOP_IO_RANGE_PCI:
+ r = s->range[type];
+ break;
+
+ default:
+ r = NULL;
+ break;
+ }
+
+ rc = -EINVAL;
+ if ( !r )
+ goto out;
+
+ rc = -EEXIST;
+ if ( rangeset_overlaps_range(r, start, end) )
+ goto out;
+
+ rc = rangeset_add_range(r, start, end);
+
+ out:
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ return rc;
+}
+
+int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
+ uint32_t type, uint64_t start,
+ uint64_t end)
+{
+ struct hvm_ioreq_server *s;
+ struct rangeset *r;
+ int rc;
+
+ if ( start > end )
+ return -EINVAL;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ s = get_ioreq_server(d, id);
+
+ rc = -ENOENT;
+ if ( !s )
+ goto out;
+
+ rc = -EPERM;
+ if ( s->emulator != current->domain )
+ goto out;
+
+ switch ( type )
+ {
+ case XEN_DMOP_IO_RANGE_PORT:
+ case XEN_DMOP_IO_RANGE_MEMORY:
+ case XEN_DMOP_IO_RANGE_PCI:
+ r = s->range[type];
+ break;
+
+ default:
+ r = NULL;
+ break;
+ }
+
+ rc = -EINVAL;
+ if ( !r )
+ goto out;
+
+ rc = -ENOENT;
+ if ( !rangeset_contains_range(r, start, end) )
+ goto out;
+
+ rc = rangeset_remove_range(r, start, end);
+
+ out:
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ return rc;
+}
+
+/*
+ * Map or unmap an ioreq server to specific memory type. For now, only
+ * HVMMEM_ioreq_server is supported, and in the future new types can be
+ * introduced, e.g. HVMMEM_ioreq_serverX mapped to ioreq server X. And
+ * currently, only write operations are to be forwarded to an ioreq server.
+ * Support for the emulation of read operations can be added when an ioreq
+ * server has such requirement in the future.
+ */
+int hvm_map_mem_type_to_ioreq_server(struct domain *d, ioservid_t id,
+ uint32_t type, uint32_t flags)
+{
+ struct hvm_ioreq_server *s;
+ int rc;
+
+ if ( type != HVMMEM_ioreq_server )
+ return -EINVAL;
+
+ if ( flags & ~XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
+ return -EINVAL;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ s = get_ioreq_server(d, id);
+
+ rc = -ENOENT;
+ if ( !s )
+ goto out;
+
+ rc = -EPERM;
+ if ( s->emulator != current->domain )
+ goto out;
+
+ rc = arch_ioreq_server_map_mem_type(d, s, flags);
+
+ out:
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ if ( rc == 0 )
+ arch_ioreq_server_map_mem_type_completed(d, s, flags);
+
+ return rc;
+}
+
+int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
+ bool enabled)
+{
+ struct hvm_ioreq_server *s;
+ int rc;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ s = get_ioreq_server(d, id);
+
+ rc = -ENOENT;
+ if ( !s )
+ goto out;
+
+ rc = -EPERM;
+ if ( s->emulator != current->domain )
+ goto out;
+
+ domain_pause(d);
+
+ if ( enabled )
+ hvm_ioreq_server_enable(s);
+ else
+ hvm_ioreq_server_disable(s);
+
+ domain_unpause(d);
+
+ rc = 0;
+
+ out:
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ return rc;
+}
+
+int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
+{
+ struct hvm_ioreq_server *s;
+ unsigned int id;
+ int rc;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ FOR_EACH_IOREQ_SERVER(d, id, s)
+ {
+ rc = hvm_ioreq_server_add_vcpu(s, v);
+ if ( rc )
+ goto fail;
+ }
+
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ return 0;
+
+ fail:
+ while ( ++id != MAX_NR_IOREQ_SERVERS )
+ {
+ s = GET_IOREQ_SERVER(d, id);
+
+ if ( !s )
+ continue;
+
+ hvm_ioreq_server_remove_vcpu(s, v);
+ }
+
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ return rc;
+}
+
+void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v)
+{
+ struct hvm_ioreq_server *s;
+ unsigned int id;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ FOR_EACH_IOREQ_SERVER(d, id, s)
+ hvm_ioreq_server_remove_vcpu(s, v);
+
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+}
+
+void hvm_destroy_all_ioreq_servers(struct domain *d)
+{
+ struct hvm_ioreq_server *s;
+ unsigned int id;
+
+ if ( !arch_ioreq_server_destroy_all(d) )
+ return;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ /* No need to domain_pause() as the domain is being torn down */
+
+ FOR_EACH_IOREQ_SERVER(d, id, s)
+ {
+ hvm_ioreq_server_disable(s);
+
+ /*
+ * It is safe to call hvm_ioreq_server_deinit() prior to
+ * set_ioreq_server() since the target domain is being destroyed.
+ */
+ hvm_ioreq_server_deinit(s);
+ set_ioreq_server(d, id, NULL);
+
+ xfree(s);
+ }
+
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+}
+
+struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
+ ioreq_t *p)
+{
+ struct hvm_ioreq_server *s;
+ uint8_t type;
+ uint64_t addr;
+ unsigned int id;
+
+ if ( !arch_ioreq_server_get_type_addr(d, p, &type, &addr) )
+ return NULL;
+
+ FOR_EACH_IOREQ_SERVER(d, id, s)
+ {
+ struct rangeset *r;
+
+ if ( !s->enabled )
+ continue;
+
+ r = s->range[type];
+
+ switch ( type )
+ {
+ unsigned long start, end;
+
+ case XEN_DMOP_IO_RANGE_PORT:
+ start = addr;
+ end = start + p->size - 1;
+ if ( rangeset_contains_range(r, start, end) )
+ return s;
+
+ break;
+
+ case XEN_DMOP_IO_RANGE_MEMORY:
+ start = hvm_mmio_first_byte(p);
+ end = hvm_mmio_last_byte(p);
+
+ if ( rangeset_contains_range(r, start, end) )
+ return s;
+
+ break;
+
+ case XEN_DMOP_IO_RANGE_PCI:
+ if ( rangeset_contains_singleton(r, addr >> 32) )
+ {
+ p->type = IOREQ_TYPE_PCI_CONFIG;
+ p->addr = addr;
+ return s;
+ }
+
+ break;
+ }
+ }
+
+ return NULL;
+}
+
+static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
+{
+ struct domain *d = current->domain;
+ struct hvm_ioreq_page *iorp;
+ buffered_iopage_t *pg;
+ buf_ioreq_t bp = { .data = p->data,
+ .addr = p->addr,
+ .type = p->type,
+ .dir = p->dir };
+ /* Timeoffset sends 64b data, but no address. Use two consecutive slots. */
+ int qw = 0;
+
+ /* Ensure buffered_iopage fits in a page */
+ BUILD_BUG_ON(sizeof(buffered_iopage_t) > PAGE_SIZE);
+
+ iorp = &s->bufioreq;
+ pg = iorp->va;
+
+ if ( !pg )
+ return IOREQ_STATUS_UNHANDLED;
+
+ /*
+ * Return 0 for the cases we can't deal with:
+ * - 'addr' is only a 20-bit field, so we cannot address beyond 1MB
+ * - we cannot buffer accesses to guest memory buffers, as the guest
+ * may expect the memory buffer to be synchronously accessed
+ * - the count field is usually used with data_is_ptr and since we don't
+ * support data_is_ptr we do not waste space for the count field either
+ */
+ if ( (p->addr > 0xffffful) || p->data_is_ptr || (p->count != 1) )
+ return 0;
+
+ switch ( p->size )
+ {
+ case 1:
+ bp.size = 0;
+ break;
+ case 2:
+ bp.size = 1;
+ break;
+ case 4:
+ bp.size = 2;
+ break;
+ case 8:
+ bp.size = 3;
+ qw = 1;
+ break;
+ default:
+ gdprintk(XENLOG_WARNING, "unexpected ioreq size: %u\n", p->size);
+ return IOREQ_STATUS_UNHANDLED;
+ }
+
+ spin_lock(&s->bufioreq_lock);
+
+ if ( (pg->ptrs.write_pointer - pg->ptrs.read_pointer) >=
+ (IOREQ_BUFFER_SLOT_NUM - qw) )
+ {
+ /* The queue is full: send the iopacket through the normal path. */
+ spin_unlock(&s->bufioreq_lock);
+ return IOREQ_STATUS_UNHANDLED;
+ }
+
+ pg->buf_ioreq[pg->ptrs.write_pointer % IOREQ_BUFFER_SLOT_NUM] = bp;
+
+ if ( qw )
+ {
+ bp.data = p->data >> 32;
+ pg->buf_ioreq[(pg->ptrs.write_pointer+1) % IOREQ_BUFFER_SLOT_NUM] = bp;
+ }
+
+ /* Make the ioreq_t visible /before/ write_pointer. */
+ smp_wmb();
+ pg->ptrs.write_pointer += qw ? 2 : 1;
+
+ /* Canonicalize read/write pointers to prevent their overflow. */
+ while ( (s->bufioreq_handling == HVM_IOREQSRV_BUFIOREQ_ATOMIC) &&
+ qw++ < IOREQ_BUFFER_SLOT_NUM &&
+ pg->ptrs.read_pointer >= IOREQ_BUFFER_SLOT_NUM )
+ {
+ union bufioreq_pointers old = pg->ptrs, new;
+ unsigned int n = old.read_pointer / IOREQ_BUFFER_SLOT_NUM;
+
+ new.read_pointer = old.read_pointer - n * IOREQ_BUFFER_SLOT_NUM;
+ new.write_pointer = old.write_pointer - n * IOREQ_BUFFER_SLOT_NUM;
+ cmpxchg(&pg->ptrs.full, old.full, new.full);
+ }
+
+ notify_via_xen_event_channel(d, s->bufioreq_evtchn);
+ spin_unlock(&s->bufioreq_lock);
+
+ return IOREQ_STATUS_HANDLED;
+}
+
+int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
+ bool buffered)
+{
+ struct vcpu *curr = current;
+ struct domain *d = curr->domain;
+ struct hvm_ioreq_vcpu *sv;
+
+ ASSERT(s);
+
+ if ( buffered )
+ return hvm_send_buffered_ioreq(s, proto_p);
+
+ if ( unlikely(!vcpu_start_shutdown_deferral(curr)) )
+ return IOREQ_STATUS_RETRY;
+
+ list_for_each_entry ( sv,
+ &s->ioreq_vcpu_list,
+ list_entry )
+ {
+ if ( sv->vcpu == curr )
+ {
+ evtchn_port_t port = sv->ioreq_evtchn;
+ ioreq_t *p = get_ioreq(s, curr);
+
+ if ( unlikely(p->state != STATE_IOREQ_NONE) )
+ {
+ gprintk(XENLOG_ERR, "device model set bad IO state %d\n",
+ p->state);
+ break;
+ }
+
+ if ( unlikely(p->vp_eport != port) )
+ {
+ gprintk(XENLOG_ERR, "device model set bad event channel %d\n",
+ p->vp_eport);
+ break;
+ }
+
+ proto_p->state = STATE_IOREQ_NONE;
+ proto_p->vp_eport = port;
+ *p = *proto_p;
+
+ prepare_wait_on_xen_event_channel(port);
+
+ /*
+ * Following happens /after/ blocking and setting up ioreq
+ * contents. prepare_wait_on_xen_event_channel() is an implicit
+ * barrier.
+ */
+ p->state = STATE_IOREQ_READY;
+ notify_via_xen_event_channel(d, port);
+
+ sv->pending = true;
+ return IOREQ_STATUS_RETRY;
+ }
+ }
+
+ return IOREQ_STATUS_UNHANDLED;
+}
+
+unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered)
+{
+ struct domain *d = current->domain;
+ struct hvm_ioreq_server *s;
+ unsigned int id, failed = 0;
+
+ FOR_EACH_IOREQ_SERVER(d, id, s)
+ {
+ if ( !s->enabled )
+ continue;
+
+ if ( hvm_send_ioreq(s, p, buffered) == IOREQ_STATUS_UNHANDLED )
+ failed++;
+ }
+
+ return failed;
+}
+
+void hvm_ioreq_init(struct domain *d)
+{
+ spin_lock_init(&d->arch.hvm.ioreq_server.lock);
+
+ arch_ioreq_domain_init(d);
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-x86/hvm/ioreq.h b/xen/include/asm-x86/hvm/ioreq.h
index df0c292..9b2eb6f 100644
--- a/xen/include/asm-x86/hvm/ioreq.h
+++ b/xen/include/asm-x86/hvm/ioreq.h
@@ -19,42 +19,6 @@
#ifndef __ASM_X86_HVM_IOREQ_H__
#define __ASM_X86_HVM_IOREQ_H__
-bool hvm_io_pending(struct vcpu *v);
-bool handle_hvm_io_completion(struct vcpu *v);
-bool is_ioreq_server_page(struct domain *d, const struct page_info *page);
-
-int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
- ioservid_t *id);
-int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id);
-int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
- unsigned long *ioreq_gfn,
- unsigned long *bufioreq_gfn,
- evtchn_port_t *bufioreq_port);
-int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
- unsigned long idx, mfn_t *mfn);
-int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint64_t start,
- uint64_t end);
-int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint64_t start,
- uint64_t end);
-int hvm_map_mem_type_to_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint32_t flags);
-int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
- bool enabled);
-
-int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v);
-void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v);
-void hvm_destroy_all_ioreq_servers(struct domain *d);
-
-struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
- ioreq_t *p);
-int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
- bool buffered);
-unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered);
-
-void hvm_ioreq_init(struct domain *d);
-
/* This correlation must not be altered */
#define IOREQ_STATUS_HANDLED X86EMUL_OKAY
#define IOREQ_STATUS_UNHANDLED X86EMUL_UNHANDLEABLE
diff --git a/xen/include/asm-x86/ioreq.h b/xen/include/asm-x86/ioreq.h
new file mode 100644
index 0000000..2aed60c
--- /dev/null
+++ b/xen/include/asm-x86/ioreq.h
@@ -0,0 +1,37 @@
+/*
+ * ioreq.h: Hardware virtual machine assist interface definitions.
+ *
+ * This is a wrapper which purpose is to not include arch HVM specific header
+ * from the common code.
+ *
+ * Copyright (c) 2016 Citrix Systems Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ASM_X86_IOREQ_H__
+#define __ASM_X86_IOREQ_H__
+
+#include <asm/hvm/ioreq.h>
+
+#endif /* __ASM_X86_IOREQ_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
index b95d3ef..430fc22 100644
--- a/xen/include/xen/ioreq.h
+++ b/xen/include/xen/ioreq.h
@@ -21,9 +21,47 @@
#include <xen/sched.h>
+#include <public/hvm/dm_op.h>
+
#define HANDLE_BUFIOREQ(s) \
((s)->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF)
+bool hvm_io_pending(struct vcpu *v);
+bool handle_hvm_io_completion(struct vcpu *v);
+bool is_ioreq_server_page(struct domain *d, const struct page_info *page);
+
+int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
+ ioservid_t *id);
+int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id);
+int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
+ unsigned long *ioreq_gfn,
+ unsigned long *bufioreq_gfn,
+ evtchn_port_t *bufioreq_port);
+int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
+ unsigned long idx, mfn_t *mfn);
+int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
+ uint32_t type, uint64_t start,
+ uint64_t end);
+int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
+ uint32_t type, uint64_t start,
+ uint64_t end);
+int hvm_map_mem_type_to_ioreq_server(struct domain *d, ioservid_t id,
+ uint32_t type, uint32_t flags);
+int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
+ bool enabled);
+
+int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v);
+void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v);
+void hvm_destroy_all_ioreq_servers(struct domain *d);
+
+struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
+ ioreq_t *p);
+int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
+ bool buffered);
+unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered);
+
+void hvm_ioreq_init(struct domain *d);
+
bool arch_ioreq_complete_mmio(void);
bool arch_vcpu_ioreq_completion(enum hvm_io_completion io_completion);
int arch_ioreq_server_map_pages(struct hvm_ioreq_server *s);
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 05/22] xen/ioreq: Make x86's hvm_ioreq_needs_completion() common
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (3 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 06/22] xen/ioreq: Make x86's hvm_mmio_first(last)_byte() common Oleksandr Tyshchenko
` (18 subsequent siblings)
23 siblings, 0 replies; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, Stefano Stabellini, Julien Grall, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
The IOREQ is a common feature now and this helper will be used
on Arm as is. Move it to xen/ioreq.h and remove "hvm" prefix.
Although PIO handling on Arm is not introduced with the current series
(it will be implemented when we add support for vPCI), technically
the PIOs exist on Arm (however they are accessed the same way as MMIO)
and it would be better not to diverge now.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
CC: Julien Grall <julien.grall@arm.com>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- new patch, was split from:
"[RFC PATCH V1 01/12] hvm/ioreq: Make x86's IOREQ feature common"
Changes V1 -> V2:
- remove "hvm" prefix
Changes V2 -> V3:
- add Paul's R-b
Changes V3 -> V4:
- add Jan's A-b
Changes V4 -> V5:
- rebase
- add Julien's and Alex's R-b
---
---
xen/arch/x86/hvm/emulate.c | 4 ++--
xen/arch/x86/hvm/io.c | 2 +-
xen/common/ioreq.c | 4 ++--
xen/include/asm-x86/hvm/vcpu.h | 7 -------
xen/include/xen/ioreq.h | 7 +++++++
5 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index 60ca465..c3487b5 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -336,7 +336,7 @@ static int hvmemul_do_io(
rc = hvm_send_ioreq(s, &p, 0);
if ( rc != X86EMUL_RETRY || currd->is_shutting_down )
vio->io_req.state = STATE_IOREQ_NONE;
- else if ( !hvm_ioreq_needs_completion(&vio->io_req) )
+ else if ( !ioreq_needs_completion(&vio->io_req) )
rc = X86EMUL_OKAY;
}
break;
@@ -2649,7 +2649,7 @@ static int _hvm_emulate_one(struct hvm_emulate_ctxt *hvmemul_ctxt,
if ( rc == X86EMUL_OKAY && vio->mmio_retry )
rc = X86EMUL_RETRY;
- if ( !hvm_ioreq_needs_completion(&vio->io_req) )
+ if ( !ioreq_needs_completion(&vio->io_req) )
completion = HVMIO_no_completion;
else if ( completion == HVMIO_no_completion )
completion = (vio->io_req.type != IOREQ_TYPE_PIO ||
diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
index 11e007d..ef8286b 100644
--- a/xen/arch/x86/hvm/io.c
+++ b/xen/arch/x86/hvm/io.c
@@ -135,7 +135,7 @@ bool handle_pio(uint16_t port, unsigned int size, int dir)
rc = hvmemul_do_pio_buffer(port, size, dir, &data);
- if ( hvm_ioreq_needs_completion(&vio->io_req) )
+ if ( ioreq_needs_completion(&vio->io_req) )
vio->io_completion = HVMIO_pio_completion;
switch ( rc )
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index 4e7d91b..61ddd54 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -160,7 +160,7 @@ static bool hvm_wait_for_io(struct hvm_ioreq_vcpu *sv, ioreq_t *p)
}
p = &sv->vcpu->arch.hvm.hvm_io.io_req;
- if ( hvm_ioreq_needs_completion(p) )
+ if ( ioreq_needs_completion(p) )
p->data = data;
sv->pending = false;
@@ -186,7 +186,7 @@ bool handle_hvm_io_completion(struct vcpu *v)
if ( sv && !hvm_wait_for_io(sv, get_ioreq(s, v)) )
return false;
- vio->io_req.state = hvm_ioreq_needs_completion(&vio->io_req) ?
+ vio->io_req.state = ioreq_needs_completion(&vio->io_req) ?
STATE_IORESP_READY : STATE_IOREQ_NONE;
msix_write_completion(v);
diff --git a/xen/include/asm-x86/hvm/vcpu.h b/xen/include/asm-x86/hvm/vcpu.h
index 5ccd075..6c1feda 100644
--- a/xen/include/asm-x86/hvm/vcpu.h
+++ b/xen/include/asm-x86/hvm/vcpu.h
@@ -91,13 +91,6 @@ struct hvm_vcpu_io {
const struct g2m_ioport *g2m_ioport;
};
-static inline bool hvm_ioreq_needs_completion(const ioreq_t *ioreq)
-{
- return ioreq->state == STATE_IOREQ_READY &&
- !ioreq->data_is_ptr &&
- (ioreq->type != IOREQ_TYPE_PIO || ioreq->dir != IOREQ_WRITE);
-}
-
struct nestedvcpu {
bool_t nv_guestmode; /* vcpu in guestmode? */
void *nv_vvmcx; /* l1 guest virtual VMCB/VMCS */
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
index 430fc22..e957b52 100644
--- a/xen/include/xen/ioreq.h
+++ b/xen/include/xen/ioreq.h
@@ -23,6 +23,13 @@
#include <public/hvm/dm_op.h>
+static inline bool ioreq_needs_completion(const ioreq_t *ioreq)
+{
+ return ioreq->state == STATE_IOREQ_READY &&
+ !ioreq->data_is_ptr &&
+ (ioreq->type != IOREQ_TYPE_PIO || ioreq->dir != IOREQ_WRITE);
+}
+
#define HANDLE_BUFIOREQ(s) \
((s)->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF)
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 06/22] xen/ioreq: Make x86's hvm_mmio_first(last)_byte() common
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (4 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 05/22] xen/ioreq: Make x86's hvm_ioreq_needs_completion() common Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 07/22] xen/ioreq: Make x86's hvm_ioreq_(page/vcpu/server) structs common Oleksandr Tyshchenko
` (17 subsequent siblings)
23 siblings, 0 replies; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, Stefano Stabellini, Julien Grall, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
The IOREQ is a common feature now and these helpers will be used
on Arm as is. Move them to xen/ioreq.h and replace "hvm" prefixes
with "ioreq".
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
CC: Julien Grall <julien.grall@arm.com>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- new patch
Changes V1 -> V2:
- replace "hvm" prefix by "ioreq"
Changes V2 -> V3:
- add Paul's R-b
Changes V3 -> V4:
- add Jan's A-b
Changes V4 -> V5:
- rebase
- add Julien's and Alex's R-b
---
---
xen/arch/x86/hvm/intercept.c | 5 +++--
xen/arch/x86/hvm/stdvga.c | 4 ++--
xen/common/ioreq.c | 4 ++--
xen/include/asm-x86/hvm/io.h | 16 ----------------
xen/include/xen/ioreq.h | 16 ++++++++++++++++
5 files changed, 23 insertions(+), 22 deletions(-)
diff --git a/xen/arch/x86/hvm/intercept.c b/xen/arch/x86/hvm/intercept.c
index cd4c4c1..02ca3b0 100644
--- a/xen/arch/x86/hvm/intercept.c
+++ b/xen/arch/x86/hvm/intercept.c
@@ -17,6 +17,7 @@
* this program; If not, see <http://www.gnu.org/licenses/>.
*/
+#include <xen/ioreq.h>
#include <xen/types.h>
#include <xen/sched.h>
#include <asm/regs.h>
@@ -34,7 +35,7 @@
static bool_t hvm_mmio_accept(const struct hvm_io_handler *handler,
const ioreq_t *p)
{
- paddr_t first = hvm_mmio_first_byte(p), last;
+ paddr_t first = ioreq_mmio_first_byte(p), last;
BUG_ON(handler->type != IOREQ_TYPE_COPY);
@@ -42,7 +43,7 @@ static bool_t hvm_mmio_accept(const struct hvm_io_handler *handler,
return 0;
/* Make sure the handler will accept the whole access. */
- last = hvm_mmio_last_byte(p);
+ last = ioreq_mmio_last_byte(p);
if ( last != first &&
!handler->mmio.ops->check(current, last) )
domain_crash(current->domain);
diff --git a/xen/arch/x86/hvm/stdvga.c b/xen/arch/x86/hvm/stdvga.c
index fd7cadb..17dee74 100644
--- a/xen/arch/x86/hvm/stdvga.c
+++ b/xen/arch/x86/hvm/stdvga.c
@@ -524,8 +524,8 @@ static bool_t stdvga_mem_accept(const struct hvm_io_handler *handler,
* deadlock when hvm_mmio_internal() is called from
* hvm_copy_to/from_guest_phys() in hvm_process_io_intercept().
*/
- if ( (hvm_mmio_first_byte(p) < VGA_MEM_BASE) ||
- (hvm_mmio_last_byte(p) >= (VGA_MEM_BASE + VGA_MEM_SIZE)) )
+ if ( (ioreq_mmio_first_byte(p) < VGA_MEM_BASE) ||
+ (ioreq_mmio_last_byte(p) >= (VGA_MEM_BASE + VGA_MEM_SIZE)) )
return 0;
spin_lock(&s->lock);
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index 61ddd54..89e75ff 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -1078,8 +1078,8 @@ struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
break;
case XEN_DMOP_IO_RANGE_MEMORY:
- start = hvm_mmio_first_byte(p);
- end = hvm_mmio_last_byte(p);
+ start = ioreq_mmio_first_byte(p);
+ end = ioreq_mmio_last_byte(p);
if ( rangeset_contains_range(r, start, end) )
return s;
diff --git a/xen/include/asm-x86/hvm/io.h b/xen/include/asm-x86/hvm/io.h
index 9453b9b..6bc80db 100644
--- a/xen/include/asm-x86/hvm/io.h
+++ b/xen/include/asm-x86/hvm/io.h
@@ -40,22 +40,6 @@ struct hvm_mmio_ops {
hvm_mmio_write_t write;
};
-static inline paddr_t hvm_mmio_first_byte(const ioreq_t *p)
-{
- return unlikely(p->df) ?
- p->addr - (p->count - 1ul) * p->size :
- p->addr;
-}
-
-static inline paddr_t hvm_mmio_last_byte(const ioreq_t *p)
-{
- unsigned long size = p->size;
-
- return unlikely(p->df) ?
- p->addr + size - 1:
- p->addr + (p->count * size) - 1;
-}
-
typedef int (*portio_action_t)(
int dir, unsigned int port, unsigned int bytes, uint32_t *val);
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
index e957b52..6853aa3 100644
--- a/xen/include/xen/ioreq.h
+++ b/xen/include/xen/ioreq.h
@@ -23,6 +23,22 @@
#include <public/hvm/dm_op.h>
+static inline paddr_t ioreq_mmio_first_byte(const ioreq_t *p)
+{
+ return unlikely(p->df) ?
+ p->addr - (p->count - 1ul) * p->size :
+ p->addr;
+}
+
+static inline paddr_t ioreq_mmio_last_byte(const ioreq_t *p)
+{
+ unsigned long size = p->size;
+
+ return unlikely(p->df) ?
+ p->addr + size - 1:
+ p->addr + (p->count * size) - 1;
+}
+
static inline bool ioreq_needs_completion(const ioreq_t *ioreq)
{
return ioreq->state == STATE_IOREQ_READY &&
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 07/22] xen/ioreq: Make x86's hvm_ioreq_(page/vcpu/server) structs common
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (5 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 06/22] xen/ioreq: Make x86's hvm_mmio_first(last)_byte() common Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 08/22] xen/ioreq: Move x86's ioreq_server to struct domain Oleksandr Tyshchenko
` (16 subsequent siblings)
23 siblings, 0 replies; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Stefano Stabellini, Julien Grall,
Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
The IOREQ is a common feature now and these structs will be used
on Arm as is. Move them to xen/ioreq.h and remove "hvm" prefixes.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
CC: Julien Grall <julien.grall@arm.com>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- new patch
Changes V1 -> V2:
- remove "hvm" prefix
Changes V2 -> V3:
- update patch according the "legacy interface" is x86 specific
Changes V3 -> V4:
- add Jan's A-b
Changes V4 -> V5:
- rebase
- add Julien's, Alex's and Paul's R-b
- fix alignment issue in domain.h
---
---
xen/arch/x86/hvm/emulate.c | 2 +-
xen/arch/x86/hvm/ioreq.c | 38 +++++++-------
xen/arch/x86/hvm/stdvga.c | 2 +-
xen/arch/x86/mm/p2m.c | 8 +--
xen/common/ioreq.c | 108 +++++++++++++++++++--------------------
xen/include/asm-x86/hvm/domain.h | 36 +------------
xen/include/asm-x86/p2m.h | 8 +--
xen/include/xen/ioreq.h | 54 ++++++++++++++++----
8 files changed, 128 insertions(+), 128 deletions(-)
diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index c3487b5..4d62199 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -287,7 +287,7 @@ static int hvmemul_do_io(
* However, there's no cheap approach to avoid above situations in xen,
* so the device model side needs to check the incoming ioreq event.
*/
- struct hvm_ioreq_server *s = NULL;
+ struct ioreq_server *s = NULL;
p2m_type_t p2mt = p2m_invalid;
if ( is_mmio )
diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 666d695..0cadf34 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -63,7 +63,7 @@ bool arch_vcpu_ioreq_completion(enum hvm_io_completion io_completion)
return true;
}
-static gfn_t hvm_alloc_legacy_ioreq_gfn(struct hvm_ioreq_server *s)
+static gfn_t hvm_alloc_legacy_ioreq_gfn(struct ioreq_server *s)
{
struct domain *d = s->target;
unsigned int i;
@@ -79,7 +79,7 @@ static gfn_t hvm_alloc_legacy_ioreq_gfn(struct hvm_ioreq_server *s)
return INVALID_GFN;
}
-static gfn_t hvm_alloc_ioreq_gfn(struct hvm_ioreq_server *s)
+static gfn_t hvm_alloc_ioreq_gfn(struct ioreq_server *s)
{
struct domain *d = s->target;
unsigned int i;
@@ -97,7 +97,7 @@ static gfn_t hvm_alloc_ioreq_gfn(struct hvm_ioreq_server *s)
return hvm_alloc_legacy_ioreq_gfn(s);
}
-static bool hvm_free_legacy_ioreq_gfn(struct hvm_ioreq_server *s,
+static bool hvm_free_legacy_ioreq_gfn(struct ioreq_server *s,
gfn_t gfn)
{
struct domain *d = s->target;
@@ -115,7 +115,7 @@ static bool hvm_free_legacy_ioreq_gfn(struct hvm_ioreq_server *s,
return true;
}
-static void hvm_free_ioreq_gfn(struct hvm_ioreq_server *s, gfn_t gfn)
+static void hvm_free_ioreq_gfn(struct ioreq_server *s, gfn_t gfn)
{
struct domain *d = s->target;
unsigned int i = gfn_x(gfn) - d->arch.hvm.ioreq_gfn.base;
@@ -129,9 +129,9 @@ static void hvm_free_ioreq_gfn(struct hvm_ioreq_server *s, gfn_t gfn)
}
}
-static void hvm_unmap_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
+static void hvm_unmap_ioreq_gfn(struct ioreq_server *s, bool buf)
{
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+ struct ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
if ( gfn_eq(iorp->gfn, INVALID_GFN) )
return;
@@ -143,10 +143,10 @@ static void hvm_unmap_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
iorp->gfn = INVALID_GFN;
}
-static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
+static int hvm_map_ioreq_gfn(struct ioreq_server *s, bool buf)
{
struct domain *d = s->target;
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+ struct ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
int rc;
if ( iorp->page )
@@ -179,11 +179,11 @@ static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
return rc;
}
-static void hvm_remove_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
+static void hvm_remove_ioreq_gfn(struct ioreq_server *s, bool buf)
{
struct domain *d = s->target;
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+ struct ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
if ( gfn_eq(iorp->gfn, INVALID_GFN) )
return;
@@ -194,10 +194,10 @@ static void hvm_remove_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
clear_page(iorp->va);
}
-static int hvm_add_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
+static int hvm_add_ioreq_gfn(struct ioreq_server *s, bool buf)
{
struct domain *d = s->target;
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+ struct ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
int rc;
if ( gfn_eq(iorp->gfn, INVALID_GFN) )
@@ -213,7 +213,7 @@ static int hvm_add_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
return rc;
}
-int arch_ioreq_server_map_pages(struct hvm_ioreq_server *s)
+int arch_ioreq_server_map_pages(struct ioreq_server *s)
{
int rc;
@@ -228,40 +228,40 @@ int arch_ioreq_server_map_pages(struct hvm_ioreq_server *s)
return rc;
}
-void arch_ioreq_server_unmap_pages(struct hvm_ioreq_server *s)
+void arch_ioreq_server_unmap_pages(struct ioreq_server *s)
{
hvm_unmap_ioreq_gfn(s, true);
hvm_unmap_ioreq_gfn(s, false);
}
-void arch_ioreq_server_enable(struct hvm_ioreq_server *s)
+void arch_ioreq_server_enable(struct ioreq_server *s)
{
hvm_remove_ioreq_gfn(s, false);
hvm_remove_ioreq_gfn(s, true);
}
-void arch_ioreq_server_disable(struct hvm_ioreq_server *s)
+void arch_ioreq_server_disable(struct ioreq_server *s)
{
hvm_add_ioreq_gfn(s, true);
hvm_add_ioreq_gfn(s, false);
}
/* Called when target domain is paused */
-void arch_ioreq_server_destroy(struct hvm_ioreq_server *s)
+void arch_ioreq_server_destroy(struct ioreq_server *s)
{
p2m_set_ioreq_server(s->target, 0, s);
}
/* Called with ioreq_server lock held */
int arch_ioreq_server_map_mem_type(struct domain *d,
- struct hvm_ioreq_server *s,
+ struct ioreq_server *s,
uint32_t flags)
{
return p2m_set_ioreq_server(d, flags, s);
}
void arch_ioreq_server_map_mem_type_completed(struct domain *d,
- struct hvm_ioreq_server *s,
+ struct ioreq_server *s,
uint32_t flags)
{
if ( flags == 0 && read_atomic(&p2m_get_hostp2m(d)->ioreq.entry_count) )
diff --git a/xen/arch/x86/hvm/stdvga.c b/xen/arch/x86/hvm/stdvga.c
index 17dee74..ee13449 100644
--- a/xen/arch/x86/hvm/stdvga.c
+++ b/xen/arch/x86/hvm/stdvga.c
@@ -466,7 +466,7 @@ static int stdvga_mem_write(const struct hvm_io_handler *handler,
.dir = IOREQ_WRITE,
.data = data,
};
- struct hvm_ioreq_server *srv;
+ struct ioreq_server *srv;
if ( !stdvga_cache_is_enabled(s) || !s->stdvga )
goto done;
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index a32301c..c1dd45b 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -372,7 +372,7 @@ void p2m_memory_type_changed(struct domain *d)
int p2m_set_ioreq_server(struct domain *d,
unsigned int flags,
- struct hvm_ioreq_server *s)
+ struct ioreq_server *s)
{
struct p2m_domain *p2m = p2m_get_hostp2m(d);
int rc;
@@ -420,11 +420,11 @@ int p2m_set_ioreq_server(struct domain *d,
return rc;
}
-struct hvm_ioreq_server *p2m_get_ioreq_server(struct domain *d,
- unsigned int *flags)
+struct ioreq_server *p2m_get_ioreq_server(struct domain *d,
+ unsigned int *flags)
{
struct p2m_domain *p2m = p2m_get_hostp2m(d);
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
spin_lock(&p2m->ioreq.lock);
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index 89e75ff..7320f23 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -35,7 +35,7 @@
#include <public/hvm/params.h>
static void set_ioreq_server(struct domain *d, unsigned int id,
- struct hvm_ioreq_server *s)
+ struct ioreq_server *s)
{
ASSERT(id < MAX_NR_IOREQ_SERVERS);
ASSERT(!s || !d->arch.hvm.ioreq_server.server[id]);
@@ -46,8 +46,8 @@ static void set_ioreq_server(struct domain *d, unsigned int id,
#define GET_IOREQ_SERVER(d, id) \
(d)->arch.hvm.ioreq_server.server[id]
-static struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
- unsigned int id)
+static struct ioreq_server *get_ioreq_server(const struct domain *d,
+ unsigned int id)
{
if ( id >= MAX_NR_IOREQ_SERVERS )
return NULL;
@@ -69,7 +69,7 @@ static struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
continue; \
else
-static ioreq_t *get_ioreq(struct hvm_ioreq_server *s, struct vcpu *v)
+static ioreq_t *get_ioreq(struct ioreq_server *s, struct vcpu *v)
{
shared_iopage_t *p = s->ioreq.va;
@@ -79,16 +79,16 @@ static ioreq_t *get_ioreq(struct hvm_ioreq_server *s, struct vcpu *v)
return &p->vcpu_ioreq[v->vcpu_id];
}
-static struct hvm_ioreq_vcpu *get_pending_vcpu(const struct vcpu *v,
- struct hvm_ioreq_server **srvp)
+static struct ioreq_vcpu *get_pending_vcpu(const struct vcpu *v,
+ struct ioreq_server **srvp)
{
struct domain *d = v->domain;
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
unsigned int id;
FOR_EACH_IOREQ_SERVER(d, id, s)
{
- struct hvm_ioreq_vcpu *sv;
+ struct ioreq_vcpu *sv;
list_for_each_entry ( sv,
&s->ioreq_vcpu_list,
@@ -111,7 +111,7 @@ bool hvm_io_pending(struct vcpu *v)
return get_pending_vcpu(v, NULL);
}
-static bool hvm_wait_for_io(struct hvm_ioreq_vcpu *sv, ioreq_t *p)
+static bool hvm_wait_for_io(struct ioreq_vcpu *sv, ioreq_t *p)
{
unsigned int prev_state = STATE_IOREQ_NONE;
unsigned int state = p->state;
@@ -172,8 +172,8 @@ bool handle_hvm_io_completion(struct vcpu *v)
{
struct domain *d = v->domain;
struct hvm_vcpu_io *vio = &v->arch.hvm.hvm_io;
- struct hvm_ioreq_server *s;
- struct hvm_ioreq_vcpu *sv;
+ struct ioreq_server *s;
+ struct ioreq_vcpu *sv;
enum hvm_io_completion io_completion;
if ( has_vpci(d) && vpci_process_pending(v) )
@@ -214,9 +214,9 @@ bool handle_hvm_io_completion(struct vcpu *v)
return true;
}
-static int hvm_alloc_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
+static int hvm_alloc_ioreq_mfn(struct ioreq_server *s, bool buf)
{
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+ struct ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
struct page_info *page;
if ( iorp->page )
@@ -262,9 +262,9 @@ static int hvm_alloc_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
return -ENOMEM;
}
-static void hvm_free_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
+static void hvm_free_ioreq_mfn(struct ioreq_server *s, bool buf)
{
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+ struct ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
struct page_info *page = iorp->page;
if ( !page )
@@ -281,7 +281,7 @@ static void hvm_free_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
{
- const struct hvm_ioreq_server *s;
+ const struct ioreq_server *s;
unsigned int id;
bool found = false;
@@ -301,8 +301,8 @@ bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
return found;
}
-static void hvm_update_ioreq_evtchn(struct hvm_ioreq_server *s,
- struct hvm_ioreq_vcpu *sv)
+static void hvm_update_ioreq_evtchn(struct ioreq_server *s,
+ struct ioreq_vcpu *sv)
{
ASSERT(spin_is_locked(&s->lock));
@@ -314,13 +314,13 @@ static void hvm_update_ioreq_evtchn(struct hvm_ioreq_server *s,
}
}
-static int hvm_ioreq_server_add_vcpu(struct hvm_ioreq_server *s,
+static int hvm_ioreq_server_add_vcpu(struct ioreq_server *s,
struct vcpu *v)
{
- struct hvm_ioreq_vcpu *sv;
+ struct ioreq_vcpu *sv;
int rc;
- sv = xzalloc(struct hvm_ioreq_vcpu);
+ sv = xzalloc(struct ioreq_vcpu);
rc = -ENOMEM;
if ( !sv )
@@ -366,10 +366,10 @@ static int hvm_ioreq_server_add_vcpu(struct hvm_ioreq_server *s,
return rc;
}
-static void hvm_ioreq_server_remove_vcpu(struct hvm_ioreq_server *s,
+static void hvm_ioreq_server_remove_vcpu(struct ioreq_server *s,
struct vcpu *v)
{
- struct hvm_ioreq_vcpu *sv;
+ struct ioreq_vcpu *sv;
spin_lock(&s->lock);
@@ -394,9 +394,9 @@ static void hvm_ioreq_server_remove_vcpu(struct hvm_ioreq_server *s,
spin_unlock(&s->lock);
}
-static void hvm_ioreq_server_remove_all_vcpus(struct hvm_ioreq_server *s)
+static void hvm_ioreq_server_remove_all_vcpus(struct ioreq_server *s)
{
- struct hvm_ioreq_vcpu *sv, *next;
+ struct ioreq_vcpu *sv, *next;
spin_lock(&s->lock);
@@ -420,7 +420,7 @@ static void hvm_ioreq_server_remove_all_vcpus(struct hvm_ioreq_server *s)
spin_unlock(&s->lock);
}
-static int hvm_ioreq_server_alloc_pages(struct hvm_ioreq_server *s)
+static int hvm_ioreq_server_alloc_pages(struct ioreq_server *s)
{
int rc;
@@ -435,13 +435,13 @@ static int hvm_ioreq_server_alloc_pages(struct hvm_ioreq_server *s)
return rc;
}
-static void hvm_ioreq_server_free_pages(struct hvm_ioreq_server *s)
+static void hvm_ioreq_server_free_pages(struct ioreq_server *s)
{
hvm_free_ioreq_mfn(s, true);
hvm_free_ioreq_mfn(s, false);
}
-static void hvm_ioreq_server_free_rangesets(struct hvm_ioreq_server *s)
+static void hvm_ioreq_server_free_rangesets(struct ioreq_server *s)
{
unsigned int i;
@@ -449,7 +449,7 @@ static void hvm_ioreq_server_free_rangesets(struct hvm_ioreq_server *s)
rangeset_destroy(s->range[i]);
}
-static int hvm_ioreq_server_alloc_rangesets(struct hvm_ioreq_server *s,
+static int hvm_ioreq_server_alloc_rangesets(struct ioreq_server *s,
ioservid_t id)
{
unsigned int i;
@@ -487,9 +487,9 @@ static int hvm_ioreq_server_alloc_rangesets(struct hvm_ioreq_server *s,
return rc;
}
-static void hvm_ioreq_server_enable(struct hvm_ioreq_server *s)
+static void hvm_ioreq_server_enable(struct ioreq_server *s)
{
- struct hvm_ioreq_vcpu *sv;
+ struct ioreq_vcpu *sv;
spin_lock(&s->lock);
@@ -509,7 +509,7 @@ static void hvm_ioreq_server_enable(struct hvm_ioreq_server *s)
spin_unlock(&s->lock);
}
-static void hvm_ioreq_server_disable(struct hvm_ioreq_server *s)
+static void hvm_ioreq_server_disable(struct ioreq_server *s)
{
spin_lock(&s->lock);
@@ -524,7 +524,7 @@ static void hvm_ioreq_server_disable(struct hvm_ioreq_server *s)
spin_unlock(&s->lock);
}
-static int hvm_ioreq_server_init(struct hvm_ioreq_server *s,
+static int hvm_ioreq_server_init(struct ioreq_server *s,
struct domain *d, int bufioreq_handling,
ioservid_t id)
{
@@ -569,7 +569,7 @@ static int hvm_ioreq_server_init(struct hvm_ioreq_server *s,
return rc;
}
-static void hvm_ioreq_server_deinit(struct hvm_ioreq_server *s)
+static void hvm_ioreq_server_deinit(struct ioreq_server *s)
{
ASSERT(!s->enabled);
hvm_ioreq_server_remove_all_vcpus(s);
@@ -594,14 +594,14 @@ static void hvm_ioreq_server_deinit(struct hvm_ioreq_server *s)
int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
ioservid_t *id)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
unsigned int i;
int rc;
if ( bufioreq_handling > HVM_IOREQSRV_BUFIOREQ_ATOMIC )
return -EINVAL;
- s = xzalloc(struct hvm_ioreq_server);
+ s = xzalloc(struct ioreq_server);
if ( !s )
return -ENOMEM;
@@ -649,7 +649,7 @@ int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
int rc;
spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
@@ -694,7 +694,7 @@ int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
unsigned long *bufioreq_gfn,
evtchn_port_t *bufioreq_port)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
int rc;
spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
@@ -739,7 +739,7 @@ int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
unsigned long idx, mfn_t *mfn)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
int rc;
ASSERT(is_hvm_domain(d));
@@ -791,7 +791,7 @@ int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
uint32_t type, uint64_t start,
uint64_t end)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
struct rangeset *r;
int rc;
@@ -843,7 +843,7 @@ int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
uint32_t type, uint64_t start,
uint64_t end)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
struct rangeset *r;
int rc;
@@ -902,7 +902,7 @@ int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
int hvm_map_mem_type_to_ioreq_server(struct domain *d, ioservid_t id,
uint32_t type, uint32_t flags)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
int rc;
if ( type != HVMMEM_ioreq_server )
@@ -937,7 +937,7 @@ int hvm_map_mem_type_to_ioreq_server(struct domain *d, ioservid_t id,
int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
bool enabled)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
int rc;
spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
@@ -970,7 +970,7 @@ int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
unsigned int id;
int rc;
@@ -1005,7 +1005,7 @@ int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
unsigned int id;
spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
@@ -1018,7 +1018,7 @@ void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v)
void hvm_destroy_all_ioreq_servers(struct domain *d)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
unsigned int id;
if ( !arch_ioreq_server_destroy_all(d) )
@@ -1045,10 +1045,10 @@ void hvm_destroy_all_ioreq_servers(struct domain *d)
spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
}
-struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
- ioreq_t *p)
+struct ioreq_server *hvm_select_ioreq_server(struct domain *d,
+ ioreq_t *p)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
uint8_t type;
uint64_t addr;
unsigned int id;
@@ -1101,10 +1101,10 @@ struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
return NULL;
}
-static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
+static int hvm_send_buffered_ioreq(struct ioreq_server *s, ioreq_t *p)
{
struct domain *d = current->domain;
- struct hvm_ioreq_page *iorp;
+ struct ioreq_page *iorp;
buffered_iopage_t *pg;
buf_ioreq_t bp = { .data = p->data,
.addr = p->addr,
@@ -1194,12 +1194,12 @@ static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
return IOREQ_STATUS_HANDLED;
}
-int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
+int hvm_send_ioreq(struct ioreq_server *s, ioreq_t *proto_p,
bool buffered)
{
struct vcpu *curr = current;
struct domain *d = curr->domain;
- struct hvm_ioreq_vcpu *sv;
+ struct ioreq_vcpu *sv;
ASSERT(s);
@@ -1257,7 +1257,7 @@ int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered)
{
struct domain *d = current->domain;
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
unsigned int id, failed = 0;
FOR_EACH_IOREQ_SERVER(d, id, s)
diff --git a/xen/include/asm-x86/hvm/domain.h b/xen/include/asm-x86/hvm/domain.h
index 9d247ba..3b36c2f 100644
--- a/xen/include/asm-x86/hvm/domain.h
+++ b/xen/include/asm-x86/hvm/domain.h
@@ -30,40 +30,6 @@
#include <public/hvm/dm_op.h>
-struct hvm_ioreq_page {
- gfn_t gfn;
- struct page_info *page;
- void *va;
-};
-
-struct hvm_ioreq_vcpu {
- struct list_head list_entry;
- struct vcpu *vcpu;
- evtchn_port_t ioreq_evtchn;
- bool pending;
-};
-
-#define NR_IO_RANGE_TYPES (XEN_DMOP_IO_RANGE_PCI + 1)
-#define MAX_NR_IO_RANGES 256
-
-struct hvm_ioreq_server {
- struct domain *target, *emulator;
-
- /* Lock to serialize toolstack modifications */
- spinlock_t lock;
-
- struct hvm_ioreq_page ioreq;
- struct list_head ioreq_vcpu_list;
- struct hvm_ioreq_page bufioreq;
-
- /* Lock to serialize access to buffered ioreq ring */
- spinlock_t bufioreq_lock;
- evtchn_port_t bufioreq_evtchn;
- struct rangeset *range[NR_IO_RANGE_TYPES];
- bool enabled;
- uint8_t bufioreq_handling;
-};
-
#ifdef CONFIG_MEM_SHARING
struct mem_sharing_domain
{
@@ -110,7 +76,7 @@ struct hvm_domain {
/* Lock protects all other values in the sub-struct and the default */
struct {
spinlock_t lock;
- struct hvm_ioreq_server *server[MAX_NR_IOREQ_SERVERS];
+ struct ioreq_server *server[MAX_NR_IOREQ_SERVERS];
} ioreq_server;
/* Cached CF8 for guest PCI config cycles */
diff --git a/xen/include/asm-x86/p2m.h b/xen/include/asm-x86/p2m.h
index 6acbd1e..5d7836d 100644
--- a/xen/include/asm-x86/p2m.h
+++ b/xen/include/asm-x86/p2m.h
@@ -363,7 +363,7 @@ struct p2m_domain {
* ioreq server who's responsible for the emulation of
* gfns with specific p2m type(for now, p2m_ioreq_server).
*/
- struct hvm_ioreq_server *server;
+ struct ioreq_server *server;
/*
* flags specifies whether read, write or both operations
* are to be emulated by an ioreq server.
@@ -937,9 +937,9 @@ static inline unsigned int p2m_get_iommu_flags(p2m_type_t p2mt, mfn_t mfn)
}
int p2m_set_ioreq_server(struct domain *d, unsigned int flags,
- struct hvm_ioreq_server *s);
-struct hvm_ioreq_server *p2m_get_ioreq_server(struct domain *d,
- unsigned int *flags);
+ struct ioreq_server *s);
+struct ioreq_server *p2m_get_ioreq_server(struct domain *d,
+ unsigned int *flags);
static inline int p2m_entry_modify(struct p2m_domain *p2m, p2m_type_t nt,
p2m_type_t ot, mfn_t nfn, mfn_t ofn,
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
index 6853aa3..5a6c11d 100644
--- a/xen/include/xen/ioreq.h
+++ b/xen/include/xen/ioreq.h
@@ -23,6 +23,40 @@
#include <public/hvm/dm_op.h>
+struct ioreq_page {
+ gfn_t gfn;
+ struct page_info *page;
+ void *va;
+};
+
+struct ioreq_vcpu {
+ struct list_head list_entry;
+ struct vcpu *vcpu;
+ evtchn_port_t ioreq_evtchn;
+ bool pending;
+};
+
+#define NR_IO_RANGE_TYPES (XEN_DMOP_IO_RANGE_PCI + 1)
+#define MAX_NR_IO_RANGES 256
+
+struct ioreq_server {
+ struct domain *target, *emulator;
+
+ /* Lock to serialize toolstack modifications */
+ spinlock_t lock;
+
+ struct ioreq_page ioreq;
+ struct list_head ioreq_vcpu_list;
+ struct ioreq_page bufioreq;
+
+ /* Lock to serialize access to buffered ioreq ring */
+ spinlock_t bufioreq_lock;
+ evtchn_port_t bufioreq_evtchn;
+ struct rangeset *range[NR_IO_RANGE_TYPES];
+ bool enabled;
+ uint8_t bufioreq_handling;
+};
+
static inline paddr_t ioreq_mmio_first_byte(const ioreq_t *p)
{
return unlikely(p->df) ?
@@ -77,9 +111,9 @@ int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v);
void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v);
void hvm_destroy_all_ioreq_servers(struct domain *d);
-struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
- ioreq_t *p);
-int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
+struct ioreq_server *hvm_select_ioreq_server(struct domain *d,
+ ioreq_t *p);
+int hvm_send_ioreq(struct ioreq_server *s, ioreq_t *proto_p,
bool buffered);
unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered);
@@ -87,16 +121,16 @@ void hvm_ioreq_init(struct domain *d);
bool arch_ioreq_complete_mmio(void);
bool arch_vcpu_ioreq_completion(enum hvm_io_completion io_completion);
-int arch_ioreq_server_map_pages(struct hvm_ioreq_server *s);
-void arch_ioreq_server_unmap_pages(struct hvm_ioreq_server *s);
-void arch_ioreq_server_enable(struct hvm_ioreq_server *s);
-void arch_ioreq_server_disable(struct hvm_ioreq_server *s);
-void arch_ioreq_server_destroy(struct hvm_ioreq_server *s);
+int arch_ioreq_server_map_pages(struct ioreq_server *s);
+void arch_ioreq_server_unmap_pages(struct ioreq_server *s);
+void arch_ioreq_server_enable(struct ioreq_server *s);
+void arch_ioreq_server_disable(struct ioreq_server *s);
+void arch_ioreq_server_destroy(struct ioreq_server *s);
int arch_ioreq_server_map_mem_type(struct domain *d,
- struct hvm_ioreq_server *s,
+ struct ioreq_server *s,
uint32_t flags);
void arch_ioreq_server_map_mem_type_completed(struct domain *d,
- struct hvm_ioreq_server *s,
+ struct ioreq_server *s,
uint32_t flags);
bool arch_ioreq_server_destroy_all(struct domain *d);
bool arch_ioreq_server_get_type_addr(const struct domain *d, const ioreq_t *p,
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 08/22] xen/ioreq: Move x86's ioreq_server to struct domain
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (6 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 07/22] xen/ioreq: Make x86's hvm_ioreq_(page/vcpu/server) structs common Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 09/22] xen/ioreq: Make x86's IOREQ related dm-op handling common Oleksandr Tyshchenko
` (15 subsequent siblings)
23 siblings, 0 replies; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Andrew Cooper, George Dunlap,
Ian Jackson, Jan Beulich, Julien Grall, Stefano Stabellini,
Wei Liu, Roger Pau Monné,
Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
The IOREQ is a common feature now and this struct will be used
on Arm as is. Move it to common struct domain. This also
significantly reduces the layering violation in the common code
(*arch.hvm* usage).
We don't move ioreq_gfn since it is not used in the common code
(the "legacy" mechanism is x86 specific).
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
CC: Julien Grall <julien.grall@arm.com>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes V1 -> V2:
- new patch
Changes V2 -> V3:
- remove the mention of "ioreq_gfn" from patch subject/description
- update patch according the "legacy interface" is x86 specific
- drop hvm_params related changes in arch/x86/hvm/hvm.c
- leave ioreq_gfn in hvm_domain
Changes V3 -> V4:
- rebase
- drop the stale part of the comment above struct ioreq_server
- add Jan's A-b
Changes V4 -> V5:
- add Julien's, Alex's and Paul's R-b
---
---
xen/common/ioreq.c | 60 ++++++++++++++++++++--------------------
xen/include/asm-x86/hvm/domain.h | 8 ------
xen/include/xen/sched.h | 10 +++++++
3 files changed, 40 insertions(+), 38 deletions(-)
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index 7320f23..4cb26e6 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -38,13 +38,13 @@ static void set_ioreq_server(struct domain *d, unsigned int id,
struct ioreq_server *s)
{
ASSERT(id < MAX_NR_IOREQ_SERVERS);
- ASSERT(!s || !d->arch.hvm.ioreq_server.server[id]);
+ ASSERT(!s || !d->ioreq_server.server[id]);
- d->arch.hvm.ioreq_server.server[id] = s;
+ d->ioreq_server.server[id] = s;
}
#define GET_IOREQ_SERVER(d, id) \
- (d)->arch.hvm.ioreq_server.server[id]
+ (d)->ioreq_server.server[id]
static struct ioreq_server *get_ioreq_server(const struct domain *d,
unsigned int id)
@@ -285,7 +285,7 @@ bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
unsigned int id;
bool found = false;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
FOR_EACH_IOREQ_SERVER(d, id, s)
{
@@ -296,7 +296,7 @@ bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
}
}
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
return found;
}
@@ -606,7 +606,7 @@ int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
return -ENOMEM;
domain_pause(d);
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
for ( i = 0; i < MAX_NR_IOREQ_SERVERS; i++ )
{
@@ -634,13 +634,13 @@ int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
if ( id )
*id = i;
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
domain_unpause(d);
return 0;
fail:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
domain_unpause(d);
xfree(s);
@@ -652,7 +652,7 @@ int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
struct ioreq_server *s;
int rc;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
s = get_ioreq_server(d, id);
@@ -684,7 +684,7 @@ int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
rc = 0;
out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
return rc;
}
@@ -697,7 +697,7 @@ int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
struct ioreq_server *s;
int rc;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
s = get_ioreq_server(d, id);
@@ -731,7 +731,7 @@ int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
rc = 0;
out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
return rc;
}
@@ -744,7 +744,7 @@ int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
ASSERT(is_hvm_domain(d));
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
s = get_ioreq_server(d, id);
@@ -782,7 +782,7 @@ int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
}
out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
return rc;
}
@@ -798,7 +798,7 @@ int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
if ( start > end )
return -EINVAL;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
s = get_ioreq_server(d, id);
@@ -834,7 +834,7 @@ int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
rc = rangeset_add_range(r, start, end);
out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
return rc;
}
@@ -850,7 +850,7 @@ int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
if ( start > end )
return -EINVAL;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
s = get_ioreq_server(d, id);
@@ -886,7 +886,7 @@ int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
rc = rangeset_remove_range(r, start, end);
out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
return rc;
}
@@ -911,7 +911,7 @@ int hvm_map_mem_type_to_ioreq_server(struct domain *d, ioservid_t id,
if ( flags & ~XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
return -EINVAL;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
s = get_ioreq_server(d, id);
@@ -926,7 +926,7 @@ int hvm_map_mem_type_to_ioreq_server(struct domain *d, ioservid_t id,
rc = arch_ioreq_server_map_mem_type(d, s, flags);
out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
if ( rc == 0 )
arch_ioreq_server_map_mem_type_completed(d, s, flags);
@@ -940,7 +940,7 @@ int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
struct ioreq_server *s;
int rc;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
s = get_ioreq_server(d, id);
@@ -964,7 +964,7 @@ int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
rc = 0;
out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
return rc;
}
@@ -974,7 +974,7 @@ int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
unsigned int id;
int rc;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
FOR_EACH_IOREQ_SERVER(d, id, s)
{
@@ -983,7 +983,7 @@ int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
goto fail;
}
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
return 0;
@@ -998,7 +998,7 @@ int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
hvm_ioreq_server_remove_vcpu(s, v);
}
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
return rc;
}
@@ -1008,12 +1008,12 @@ void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v)
struct ioreq_server *s;
unsigned int id;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
FOR_EACH_IOREQ_SERVER(d, id, s)
hvm_ioreq_server_remove_vcpu(s, v);
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
}
void hvm_destroy_all_ioreq_servers(struct domain *d)
@@ -1024,7 +1024,7 @@ void hvm_destroy_all_ioreq_servers(struct domain *d)
if ( !arch_ioreq_server_destroy_all(d) )
return;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
/* No need to domain_pause() as the domain is being torn down */
@@ -1042,7 +1042,7 @@ void hvm_destroy_all_ioreq_servers(struct domain *d)
xfree(s);
}
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
}
struct ioreq_server *hvm_select_ioreq_server(struct domain *d,
@@ -1274,7 +1274,7 @@ unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered)
void hvm_ioreq_init(struct domain *d)
{
- spin_lock_init(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_init(&d->ioreq_server.lock);
arch_ioreq_domain_init(d);
}
diff --git a/xen/include/asm-x86/hvm/domain.h b/xen/include/asm-x86/hvm/domain.h
index 3b36c2f..b8be1ad 100644
--- a/xen/include/asm-x86/hvm/domain.h
+++ b/xen/include/asm-x86/hvm/domain.h
@@ -63,8 +63,6 @@ struct hvm_pi_ops {
void (*vcpu_block)(struct vcpu *);
};
-#define MAX_NR_IOREQ_SERVERS 8
-
struct hvm_domain {
/* Guest page range used for non-default ioreq servers */
struct {
@@ -73,12 +71,6 @@ struct hvm_domain {
unsigned long legacy_mask; /* indexed by HVM param number */
} ioreq_gfn;
- /* Lock protects all other values in the sub-struct and the default */
- struct {
- spinlock_t lock;
- struct ioreq_server *server[MAX_NR_IOREQ_SERVERS];
- } ioreq_server;
-
/* Cached CF8 for guest PCI config cycles */
uint32_t pci_cf8;
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index da19f4e..f437ee3 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -318,6 +318,8 @@ struct sched_unit {
struct evtchn_port_ops;
+#define MAX_NR_IOREQ_SERVERS 8
+
struct domain
{
domid_t domain_id;
@@ -534,6 +536,14 @@ struct domain
unsigned int val;
struct vcpu *vcpu;
} teardown;
+
+#ifdef CONFIG_IOREQ_SERVER
+ /* Lock protects all other values in the sub-struct */
+ struct {
+ spinlock_t lock;
+ struct ioreq_server *server[MAX_NR_IOREQ_SERVERS];
+ } ioreq_server;
+#endif
};
static inline struct page_list_head *page_to_list(
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 09/22] xen/ioreq: Make x86's IOREQ related dm-op handling common
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (7 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 08/22] xen/ioreq: Move x86's ioreq_server to struct domain Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-26 13:31 ` Paul Durrant
2021-01-28 10:52 ` Jan Beulich
2021-01-25 19:08 ` [PATCH V5 10/22] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu Oleksandr Tyshchenko
` (14 subsequent siblings)
23 siblings, 2 replies; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Julien Grall, Jan Beulich, Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Paul Durrant, Daniel De Graaf,
Oleksandr Tyshchenko
From: Julien Grall <julien.grall@arm.com>
As a lot of x86 code can be re-used on Arm later on, this patch
moves the IOREQ related dm-op handling to the common code.
The idea is to have the top level dm-op handling arch-specific
and call into ioreq_server_dm_op() for otherwise unhandled ops.
Pros:
- More natural than doing it other way around (top level dm-op
handling common).
- Leave compat_dm_op() in x86 code.
Cons:
- Code duplication. Both arches have to duplicate dm_op(), etc.
Make the corresponding functions static and rename them according
to the new naming scheme (including dropping the "hvm" prefixes).
Introduce common dm.c file as a resting place for the do_dm_op()
(which is identical for both Arm and x86) to minimize code duplication.
The common DM feature is supposed to be built with IOREQ_SERVER
option enabled (as well as the IOREQ feature), which is selected
for x86's config HVM for now.
Also update XSM code a bit to let dm-op be used on Arm.
This support is going to be used on Arm to be able run device
emulator outside of Xen hypervisor.
Signed-off-by: Julien Grall <julien.grall@arm.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- update XSM, related changes were pulled from:
[RFC PATCH V1 04/12] xen/arm: Introduce arch specific bits for IOREQ/DM features
Changes V1 -> V2:
- update the author of a patch
- update patch description
- introduce xen/dm.h and move definitions here
Changes V2 -> V3:
- no changes
Changes V3 -> V4:
- rework to have the top level dm-op handling arch-specific
- update patch subject/description, was "xen/dm: Make x86's DM feature common"
- make a few functions static in common ioreq.c
Changes V4 -> V5:
- update patch description
- add Jan's A-b
- drop the 'hvm_' prefixes of touched functions and rename them
instead of doing that in patch #12
- add common dm.c to keep do_dm_op(), make dm_op() global
---
---
xen/arch/x86/hvm/dm.c | 132 ++--------------------------------------------
xen/common/Makefile | 1 +
xen/common/dm.c | 55 +++++++++++++++++++
xen/common/ioreq.c | 137 ++++++++++++++++++++++++++++++++++++++++++------
xen/include/xen/dm.h | 41 +++++++++++++++
xen/include/xen/ioreq.h | 21 ++------
xen/include/xsm/dummy.h | 4 +-
xen/include/xsm/xsm.h | 6 +--
xen/xsm/dummy.c | 2 +-
xen/xsm/flask/hooks.c | 5 +-
10 files changed, 235 insertions(+), 169 deletions(-)
create mode 100644 xen/common/dm.c
create mode 100644 xen/include/xen/dm.h
diff --git a/xen/arch/x86/hvm/dm.c b/xen/arch/x86/hvm/dm.c
index d3e2a9e..5bc172a 100644
--- a/xen/arch/x86/hvm/dm.c
+++ b/xen/arch/x86/hvm/dm.c
@@ -16,6 +16,7 @@
#include <xen/event.h>
#include <xen/guest_access.h>
+#include <xen/dm.h>
#include <xen/hypercall.h>
#include <xen/ioreq.h>
#include <xen/nospec.h>
@@ -29,13 +30,6 @@
#include <public/hvm/hvm_op.h>
-struct dmop_args {
- domid_t domid;
- unsigned int nr_bufs;
- /* Reserve enough buf elements for all current hypercalls. */
- struct xen_dm_op_buf buf[2];
-};
-
static bool _raw_copy_from_guest_buf_offset(void *dst,
const struct dmop_args *args,
unsigned int buf_idx,
@@ -338,7 +332,7 @@ static int inject_event(struct domain *d,
return 0;
}
-static int dm_op(const struct dmop_args *op_args)
+int dm_op(const struct dmop_args *op_args)
{
struct domain *d;
struct xen_dm_op op;
@@ -408,71 +402,6 @@ static int dm_op(const struct dmop_args *op_args)
switch ( op.op )
{
- case XEN_DMOP_create_ioreq_server:
- {
- struct xen_dm_op_create_ioreq_server *data =
- &op.u.create_ioreq_server;
-
- const_op = false;
-
- rc = -EINVAL;
- if ( data->pad[0] || data->pad[1] || data->pad[2] )
- break;
-
- rc = hvm_create_ioreq_server(d, data->handle_bufioreq,
- &data->id);
- break;
- }
-
- case XEN_DMOP_get_ioreq_server_info:
- {
- struct xen_dm_op_get_ioreq_server_info *data =
- &op.u.get_ioreq_server_info;
- const uint16_t valid_flags = XEN_DMOP_no_gfns;
-
- const_op = false;
-
- rc = -EINVAL;
- if ( data->flags & ~valid_flags )
- break;
-
- rc = hvm_get_ioreq_server_info(d, data->id,
- (data->flags & XEN_DMOP_no_gfns) ?
- NULL : &data->ioreq_gfn,
- (data->flags & XEN_DMOP_no_gfns) ?
- NULL : &data->bufioreq_gfn,
- &data->bufioreq_port);
- break;
- }
-
- case XEN_DMOP_map_io_range_to_ioreq_server:
- {
- const struct xen_dm_op_ioreq_server_range *data =
- &op.u.map_io_range_to_ioreq_server;
-
- rc = -EINVAL;
- if ( data->pad )
- break;
-
- rc = hvm_map_io_range_to_ioreq_server(d, data->id, data->type,
- data->start, data->end);
- break;
- }
-
- case XEN_DMOP_unmap_io_range_from_ioreq_server:
- {
- const struct xen_dm_op_ioreq_server_range *data =
- &op.u.unmap_io_range_from_ioreq_server;
-
- rc = -EINVAL;
- if ( data->pad )
- break;
-
- rc = hvm_unmap_io_range_from_ioreq_server(d, data->id, data->type,
- data->start, data->end);
- break;
- }
-
case XEN_DMOP_map_mem_type_to_ioreq_server:
{
struct xen_dm_op_map_mem_type_to_ioreq_server *data =
@@ -486,8 +415,8 @@ static int dm_op(const struct dmop_args *op_args)
break;
if ( first_gfn == 0 )
- rc = hvm_map_mem_type_to_ioreq_server(d, data->id,
- data->type, data->flags);
+ rc = ioreq_server_map_mem_type(d, data->id,
+ data->type, data->flags);
else
rc = 0;
@@ -523,32 +452,6 @@ static int dm_op(const struct dmop_args *op_args)
break;
}
- case XEN_DMOP_set_ioreq_server_state:
- {
- const struct xen_dm_op_set_ioreq_server_state *data =
- &op.u.set_ioreq_server_state;
-
- rc = -EINVAL;
- if ( data->pad )
- break;
-
- rc = hvm_set_ioreq_server_state(d, data->id, !!data->enabled);
- break;
- }
-
- case XEN_DMOP_destroy_ioreq_server:
- {
- const struct xen_dm_op_destroy_ioreq_server *data =
- &op.u.destroy_ioreq_server;
-
- rc = -EINVAL;
- if ( data->pad )
- break;
-
- rc = hvm_destroy_ioreq_server(d, data->id);
- break;
- }
-
case XEN_DMOP_track_dirty_vram:
{
const struct xen_dm_op_track_dirty_vram *data =
@@ -703,7 +606,7 @@ static int dm_op(const struct dmop_args *op_args)
}
default:
- rc = -EOPNOTSUPP;
+ rc = ioreq_server_dm_op(&op, d, &const_op);
break;
}
@@ -776,31 +679,6 @@ int compat_dm_op(domid_t domid,
return rc;
}
-long do_dm_op(domid_t domid,
- unsigned int nr_bufs,
- XEN_GUEST_HANDLE_PARAM(xen_dm_op_buf_t) bufs)
-{
- struct dmop_args args;
- int rc;
-
- if ( nr_bufs > ARRAY_SIZE(args.buf) )
- return -E2BIG;
-
- args.domid = domid;
- args.nr_bufs = array_index_nospec(nr_bufs, ARRAY_SIZE(args.buf) + 1);
-
- if ( copy_from_guest_offset(&args.buf[0], bufs, 0, args.nr_bufs) )
- return -EFAULT;
-
- rc = dm_op(&args);
-
- if ( rc == -ERESTART )
- rc = hypercall_create_continuation(__HYPERVISOR_dm_op, "iih",
- domid, nr_bufs, bufs);
-
- return rc;
-}
-
/*
* Local variables:
* mode: C
diff --git a/xen/common/Makefile b/xen/common/Makefile
index 2e3c234..71c1d46 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -5,6 +5,7 @@ obj-$(CONFIG_CORE_PARKING) += core_parking.o
obj-y += cpu.o
obj-$(CONFIG_DEBUG_TRACE) += debugtrace.o
obj-$(CONFIG_HAS_DEVICE_TREE) += device_tree.o
+obj-$(CONFIG_IOREQ_SERVER) += dm.o
obj-y += domain.o
obj-y += event_2l.o
obj-y += event_channel.o
diff --git a/xen/common/dm.c b/xen/common/dm.c
new file mode 100644
index 0000000..2d1d98c
--- /dev/null
+++ b/xen/common/dm.c
@@ -0,0 +1,55 @@
+/*
+ * Copyright (c) 2016 Citrix Systems Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/dm.h>
+#include <xen/guest_access.h>
+#include <xen/hypercall.h>
+#include <xen/nospec.h>
+
+long do_dm_op(domid_t domid,
+ unsigned int nr_bufs,
+ XEN_GUEST_HANDLE_PARAM(xen_dm_op_buf_t) bufs)
+{
+ struct dmop_args args;
+ int rc;
+
+ if ( nr_bufs > ARRAY_SIZE(args.buf) )
+ return -E2BIG;
+
+ args.domid = domid;
+ args.nr_bufs = array_index_nospec(nr_bufs, ARRAY_SIZE(args.buf) + 1);
+
+ if ( copy_from_guest_offset(&args.buf[0], bufs, 0, args.nr_bufs) )
+ return -EFAULT;
+
+ rc = dm_op(&args);
+
+ if ( rc == -ERESTART )
+ rc = hypercall_create_continuation(__HYPERVISOR_dm_op, "iih",
+ domid, nr_bufs, bufs);
+
+ return rc;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index 4cb26e6..84bce36 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -223,7 +223,7 @@ static int hvm_alloc_ioreq_mfn(struct ioreq_server *s, bool buf)
{
/*
* If a guest frame has already been mapped (which may happen
- * on demand if hvm_get_ioreq_server_info() is called), then
+ * on demand if ioreq_server_get_info() is called), then
* allocating a page is not permitted.
*/
if ( !gfn_eq(iorp->gfn, INVALID_GFN) )
@@ -591,8 +591,8 @@ static void hvm_ioreq_server_deinit(struct ioreq_server *s)
put_domain(s->emulator);
}
-int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
- ioservid_t *id)
+static int ioreq_server_create(struct domain *d, int bufioreq_handling,
+ ioservid_t *id)
{
struct ioreq_server *s;
unsigned int i;
@@ -647,7 +647,7 @@ int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
return rc;
}
-int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
+static int ioreq_server_destroy(struct domain *d, ioservid_t id)
{
struct ioreq_server *s;
int rc;
@@ -689,10 +689,10 @@ int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
return rc;
}
-int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
- unsigned long *ioreq_gfn,
- unsigned long *bufioreq_gfn,
- evtchn_port_t *bufioreq_port)
+static int ioreq_server_get_info(struct domain *d, ioservid_t id,
+ unsigned long *ioreq_gfn,
+ unsigned long *bufioreq_gfn,
+ evtchn_port_t *bufioreq_port)
{
struct ioreq_server *s;
int rc;
@@ -787,7 +787,7 @@ int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
return rc;
}
-int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
+static int ioreq_server_map_io_range(struct domain *d, ioservid_t id,
uint32_t type, uint64_t start,
uint64_t end)
{
@@ -839,9 +839,9 @@ int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
return rc;
}
-int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint64_t start,
- uint64_t end)
+static int ioreq_server_unmap_io_range(struct domain *d, ioservid_t id,
+ uint32_t type, uint64_t start,
+ uint64_t end)
{
struct ioreq_server *s;
struct rangeset *r;
@@ -899,8 +899,8 @@ int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
* Support for the emulation of read operations can be added when an ioreq
* server has such requirement in the future.
*/
-int hvm_map_mem_type_to_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint32_t flags)
+int ioreq_server_map_mem_type(struct domain *d, ioservid_t id,
+ uint32_t type, uint32_t flags)
{
struct ioreq_server *s;
int rc;
@@ -934,8 +934,8 @@ int hvm_map_mem_type_to_ioreq_server(struct domain *d, ioservid_t id,
return rc;
}
-int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
- bool enabled)
+static int ioreq_server_set_state(struct domain *d, ioservid_t id,
+ bool enabled)
{
struct ioreq_server *s;
int rc;
@@ -1279,6 +1279,111 @@ void hvm_ioreq_init(struct domain *d)
arch_ioreq_domain_init(d);
}
+int ioreq_server_dm_op(struct xen_dm_op *op, struct domain *d, bool *const_op)
+{
+ long rc;
+
+ switch ( op->op )
+ {
+ case XEN_DMOP_create_ioreq_server:
+ {
+ struct xen_dm_op_create_ioreq_server *data =
+ &op->u.create_ioreq_server;
+
+ *const_op = false;
+
+ rc = -EINVAL;
+ if ( data->pad[0] || data->pad[1] || data->pad[2] )
+ break;
+
+ rc = ioreq_server_create(d, data->handle_bufioreq,
+ &data->id);
+ break;
+ }
+
+ case XEN_DMOP_get_ioreq_server_info:
+ {
+ struct xen_dm_op_get_ioreq_server_info *data =
+ &op->u.get_ioreq_server_info;
+ const uint16_t valid_flags = XEN_DMOP_no_gfns;
+
+ *const_op = false;
+
+ rc = -EINVAL;
+ if ( data->flags & ~valid_flags )
+ break;
+
+ rc = ioreq_server_get_info(d, data->id,
+ (data->flags & XEN_DMOP_no_gfns) ?
+ NULL : (unsigned long *)&data->ioreq_gfn,
+ (data->flags & XEN_DMOP_no_gfns) ?
+ NULL : (unsigned long *)&data->bufioreq_gfn,
+ &data->bufioreq_port);
+ break;
+ }
+
+ case XEN_DMOP_map_io_range_to_ioreq_server:
+ {
+ const struct xen_dm_op_ioreq_server_range *data =
+ &op->u.map_io_range_to_ioreq_server;
+
+ rc = -EINVAL;
+ if ( data->pad )
+ break;
+
+ rc = ioreq_server_map_io_range(d, data->id, data->type,
+ data->start, data->end);
+ break;
+ }
+
+ case XEN_DMOP_unmap_io_range_from_ioreq_server:
+ {
+ const struct xen_dm_op_ioreq_server_range *data =
+ &op->u.unmap_io_range_from_ioreq_server;
+
+ rc = -EINVAL;
+ if ( data->pad )
+ break;
+
+ rc = ioreq_server_unmap_io_range(d, data->id, data->type,
+ data->start, data->end);
+ break;
+ }
+
+ case XEN_DMOP_set_ioreq_server_state:
+ {
+ const struct xen_dm_op_set_ioreq_server_state *data =
+ &op->u.set_ioreq_server_state;
+
+ rc = -EINVAL;
+ if ( data->pad )
+ break;
+
+ rc = ioreq_server_set_state(d, data->id, !!data->enabled);
+ break;
+ }
+
+ case XEN_DMOP_destroy_ioreq_server:
+ {
+ const struct xen_dm_op_destroy_ioreq_server *data =
+ &op->u.destroy_ioreq_server;
+
+ rc = -EINVAL;
+ if ( data->pad )
+ break;
+
+ rc = ioreq_server_destroy(d, data->id);
+ break;
+ }
+
+ default:
+ rc = -EOPNOTSUPP;
+ break;
+ }
+
+ return rc;
+}
+
/*
* Local variables:
* mode: C
diff --git a/xen/include/xen/dm.h b/xen/include/xen/dm.h
new file mode 100644
index 0000000..4bd7e57
--- /dev/null
+++ b/xen/include/xen/dm.h
@@ -0,0 +1,41 @@
+/*
+ * Copyright (c) 2016 Citrix Systems Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __XEN_DM_H__
+#define __XEN_DM_H__
+
+#include <xen/sched.h>
+
+struct dmop_args {
+ domid_t domid;
+ unsigned int nr_bufs;
+ /* Reserve enough buf elements for all current hypercalls. */
+ struct xen_dm_op_buf buf[2];
+};
+
+int dm_op(const struct dmop_args *op_args);
+
+#endif /* __XEN_DM_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
index 5a6c11d..60e864d 100644
--- a/xen/include/xen/ioreq.h
+++ b/xen/include/xen/ioreq.h
@@ -87,25 +87,10 @@ bool hvm_io_pending(struct vcpu *v);
bool handle_hvm_io_completion(struct vcpu *v);
bool is_ioreq_server_page(struct domain *d, const struct page_info *page);
-int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
- ioservid_t *id);
-int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id);
-int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
- unsigned long *ioreq_gfn,
- unsigned long *bufioreq_gfn,
- evtchn_port_t *bufioreq_port);
int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
unsigned long idx, mfn_t *mfn);
-int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint64_t start,
- uint64_t end);
-int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint64_t start,
- uint64_t end);
-int hvm_map_mem_type_to_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint32_t flags);
-int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
- bool enabled);
+int ioreq_server_map_mem_type(struct domain *d, ioservid_t id,
+ uint32_t type, uint32_t flags);
int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v);
void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v);
@@ -119,6 +104,8 @@ unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered);
void hvm_ioreq_init(struct domain *d);
+int ioreq_server_dm_op(struct xen_dm_op *op, struct domain *d, bool *const_op);
+
bool arch_ioreq_complete_mmio(void);
bool arch_vcpu_ioreq_completion(enum hvm_io_completion io_completion);
int arch_ioreq_server_map_pages(struct ioreq_server *s);
diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index fa40e88..10739e7 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -707,14 +707,14 @@ static XSM_INLINE int xsm_pmu_op (XSM_DEFAULT_ARG struct domain *d, unsigned int
}
}
+#endif /* CONFIG_X86 */
+
static XSM_INLINE int xsm_dm_op(XSM_DEFAULT_ARG struct domain *d)
{
XSM_ASSERT_ACTION(XSM_DM_PRIV);
return xsm_default_action(action, current->domain, d);
}
-#endif /* CONFIG_X86 */
-
#ifdef CONFIG_ARGO
static XSM_INLINE int xsm_argo_enable(const struct domain *d)
{
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 7bd03d8..91ecff4 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -176,8 +176,8 @@ struct xsm_operations {
int (*ioport_permission) (struct domain *d, uint32_t s, uint32_t e, uint8_t allow);
int (*ioport_mapping) (struct domain *d, uint32_t s, uint32_t e, uint8_t allow);
int (*pmu_op) (struct domain *d, unsigned int op);
- int (*dm_op) (struct domain *d);
#endif
+ int (*dm_op) (struct domain *d);
int (*xen_version) (uint32_t cmd);
int (*domain_resource_map) (struct domain *d);
#ifdef CONFIG_ARGO
@@ -682,13 +682,13 @@ static inline int xsm_pmu_op (xsm_default_t def, struct domain *d, unsigned int
return xsm_ops->pmu_op(d, op);
}
+#endif /* CONFIG_X86 */
+
static inline int xsm_dm_op(xsm_default_t def, struct domain *d)
{
return xsm_ops->dm_op(d);
}
-#endif /* CONFIG_X86 */
-
static inline int xsm_xen_version (xsm_default_t def, uint32_t op)
{
return xsm_ops->xen_version(op);
diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c
index 9e09512..8bdffe7 100644
--- a/xen/xsm/dummy.c
+++ b/xen/xsm/dummy.c
@@ -147,8 +147,8 @@ void __init xsm_fixup_ops (struct xsm_operations *ops)
set_to_dummy_if_null(ops, ioport_permission);
set_to_dummy_if_null(ops, ioport_mapping);
set_to_dummy_if_null(ops, pmu_op);
- set_to_dummy_if_null(ops, dm_op);
#endif
+ set_to_dummy_if_null(ops, dm_op);
set_to_dummy_if_null(ops, xen_version);
set_to_dummy_if_null(ops, domain_resource_map);
#ifdef CONFIG_ARGO
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index 19b0d9e..11784d7 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -1656,14 +1656,13 @@ static int flask_pmu_op (struct domain *d, unsigned int op)
return -EPERM;
}
}
+#endif /* CONFIG_X86 */
static int flask_dm_op(struct domain *d)
{
return current_has_perm(d, SECCLASS_HVM, HVM__DM);
}
-#endif /* CONFIG_X86 */
-
static int flask_xen_version (uint32_t op)
{
u32 dsid = domain_sid(current->domain);
@@ -1865,8 +1864,8 @@ static struct xsm_operations flask_ops = {
.ioport_permission = flask_ioport_permission,
.ioport_mapping = flask_ioport_mapping,
.pmu_op = flask_pmu_op,
- .dm_op = flask_dm_op,
#endif
+ .dm_op = flask_dm_op,
.xen_version = flask_xen_version,
.domain_resource_map = flask_domain_resource_map,
#ifdef CONFIG_ARGO
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 10/22] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (8 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 09/22] xen/ioreq: Make x86's IOREQ related dm-op handling common Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-28 13:41 ` Julien Grall
2021-01-25 19:08 ` [PATCH V5 11/22] xen/mm: Make x86's XENMEM_resource_ioreq_server handling common Oleksandr Tyshchenko
` (13 subsequent siblings)
23 siblings, 1 reply; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
The IOREQ is a common feature now and these fields will be used
on Arm as is. Move them to common struct vcpu as a part of new
struct vcpu_io and drop duplicating "io" prefixes. Also move
enum hvm_io_completion to xen/sched.h and remove "hvm" prefixes.
This patch completely removes layering violation in the common code.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
CC: Julien Grall <julien.grall@arm.com>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes V1 -> V2:
- new patch
Changes V2 -> V3:
- update patch according the "legacy interface" is x86 specific
- update patch description
- drop the "io" prefixes from the field names
- wrap IO_realmode_completion
Changes V3 -> V4:
- rename all hvm_vcpu_io locals to "hvio"
- rename according to the new renaming scheme IO_ -> VIO_ (io_ -> vio_)
- drop "io" prefix from io_completion locals
Changes V4 -> V5:
- add Julien's and Paul's R-b, Jan's A-b
---
---
xen/arch/x86/hvm/emulate.c | 210 +++++++++++++++++++-------------------
xen/arch/x86/hvm/hvm.c | 2 +-
xen/arch/x86/hvm/io.c | 32 +++---
xen/arch/x86/hvm/ioreq.c | 6 +-
xen/arch/x86/hvm/svm/nestedsvm.c | 2 +-
xen/arch/x86/hvm/vmx/realmode.c | 8 +-
xen/common/ioreq.c | 26 ++---
xen/include/asm-x86/hvm/emulate.h | 2 +-
xen/include/asm-x86/hvm/vcpu.h | 11 --
xen/include/xen/ioreq.h | 2 +-
xen/include/xen/sched.h | 19 ++++
11 files changed, 164 insertions(+), 156 deletions(-)
diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index 4d62199..21051ce 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -140,15 +140,15 @@ static const struct hvm_io_handler ioreq_server_handler = {
*/
void hvmemul_cancel(struct vcpu *v)
{
- struct hvm_vcpu_io *vio = &v->arch.hvm.hvm_io;
+ struct hvm_vcpu_io *hvio = &v->arch.hvm.hvm_io;
- vio->io_req.state = STATE_IOREQ_NONE;
- vio->io_completion = HVMIO_no_completion;
- vio->mmio_cache_count = 0;
- vio->mmio_insn_bytes = 0;
- vio->mmio_access = (struct npfec){};
- vio->mmio_retry = false;
- vio->g2m_ioport = NULL;
+ v->io.req.state = STATE_IOREQ_NONE;
+ v->io.completion = VIO_no_completion;
+ hvio->mmio_cache_count = 0;
+ hvio->mmio_insn_bytes = 0;
+ hvio->mmio_access = (struct npfec){};
+ hvio->mmio_retry = false;
+ hvio->g2m_ioport = NULL;
hvmemul_cache_disable(v);
}
@@ -159,7 +159,7 @@ static int hvmemul_do_io(
{
struct vcpu *curr = current;
struct domain *currd = curr->domain;
- struct hvm_vcpu_io *vio = &curr->arch.hvm.hvm_io;
+ struct vcpu_io *vio = &curr->io;
ioreq_t p = {
.type = is_mmio ? IOREQ_TYPE_COPY : IOREQ_TYPE_PIO,
.addr = addr,
@@ -184,13 +184,13 @@ static int hvmemul_do_io(
return X86EMUL_UNHANDLEABLE;
}
- switch ( vio->io_req.state )
+ switch ( vio->req.state )
{
case STATE_IOREQ_NONE:
break;
case STATE_IORESP_READY:
- vio->io_req.state = STATE_IOREQ_NONE;
- p = vio->io_req;
+ vio->req.state = STATE_IOREQ_NONE;
+ p = vio->req;
/* Verify the emulation request has been correctly re-issued */
if ( (p.type != (is_mmio ? IOREQ_TYPE_COPY : IOREQ_TYPE_PIO)) ||
@@ -238,7 +238,7 @@ static int hvmemul_do_io(
}
ASSERT(p.count);
- vio->io_req = p;
+ vio->req = p;
rc = hvm_io_intercept(&p);
@@ -247,12 +247,12 @@ static int hvmemul_do_io(
* our callers and mirror this into latched state.
*/
ASSERT(p.count <= *reps);
- *reps = vio->io_req.count = p.count;
+ *reps = vio->req.count = p.count;
switch ( rc )
{
case X86EMUL_OKAY:
- vio->io_req.state = STATE_IOREQ_NONE;
+ vio->req.state = STATE_IOREQ_NONE;
break;
case X86EMUL_UNHANDLEABLE:
{
@@ -305,7 +305,7 @@ static int hvmemul_do_io(
if ( s == NULL )
{
rc = X86EMUL_RETRY;
- vio->io_req.state = STATE_IOREQ_NONE;
+ vio->req.state = STATE_IOREQ_NONE;
break;
}
@@ -316,7 +316,7 @@ static int hvmemul_do_io(
if ( dir == IOREQ_READ )
{
rc = hvm_process_io_intercept(&ioreq_server_handler, &p);
- vio->io_req.state = STATE_IOREQ_NONE;
+ vio->req.state = STATE_IOREQ_NONE;
break;
}
}
@@ -329,14 +329,14 @@ static int hvmemul_do_io(
if ( !s )
{
rc = hvm_process_io_intercept(&null_handler, &p);
- vio->io_req.state = STATE_IOREQ_NONE;
+ vio->req.state = STATE_IOREQ_NONE;
}
else
{
rc = hvm_send_ioreq(s, &p, 0);
if ( rc != X86EMUL_RETRY || currd->is_shutting_down )
- vio->io_req.state = STATE_IOREQ_NONE;
- else if ( !ioreq_needs_completion(&vio->io_req) )
+ vio->req.state = STATE_IOREQ_NONE;
+ else if ( !ioreq_needs_completion(&vio->req) )
rc = X86EMUL_OKAY;
}
break;
@@ -1005,14 +1005,14 @@ static int hvmemul_phys_mmio_access(
* cache indexed by linear MMIO address.
*/
static struct hvm_mmio_cache *hvmemul_find_mmio_cache(
- struct hvm_vcpu_io *vio, unsigned long gla, uint8_t dir, bool create)
+ struct hvm_vcpu_io *hvio, unsigned long gla, uint8_t dir, bool create)
{
unsigned int i;
struct hvm_mmio_cache *cache;
- for ( i = 0; i < vio->mmio_cache_count; i ++ )
+ for ( i = 0; i < hvio->mmio_cache_count; i ++ )
{
- cache = &vio->mmio_cache[i];
+ cache = &hvio->mmio_cache[i];
if ( gla == cache->gla &&
dir == cache->dir )
@@ -1022,13 +1022,13 @@ static struct hvm_mmio_cache *hvmemul_find_mmio_cache(
if ( !create )
return NULL;
- i = vio->mmio_cache_count;
- if( i == ARRAY_SIZE(vio->mmio_cache) )
+ i = hvio->mmio_cache_count;
+ if( i == ARRAY_SIZE(hvio->mmio_cache) )
return NULL;
- ++vio->mmio_cache_count;
+ ++hvio->mmio_cache_count;
- cache = &vio->mmio_cache[i];
+ cache = &hvio->mmio_cache[i];
memset(cache, 0, sizeof (*cache));
cache->gla = gla;
@@ -1037,26 +1037,26 @@ static struct hvm_mmio_cache *hvmemul_find_mmio_cache(
return cache;
}
-static void latch_linear_to_phys(struct hvm_vcpu_io *vio, unsigned long gla,
+static void latch_linear_to_phys(struct hvm_vcpu_io *hvio, unsigned long gla,
unsigned long gpa, bool_t write)
{
- if ( vio->mmio_access.gla_valid )
+ if ( hvio->mmio_access.gla_valid )
return;
- vio->mmio_gla = gla & PAGE_MASK;
- vio->mmio_gpfn = PFN_DOWN(gpa);
- vio->mmio_access = (struct npfec){ .gla_valid = 1,
- .read_access = 1,
- .write_access = write };
+ hvio->mmio_gla = gla & PAGE_MASK;
+ hvio->mmio_gpfn = PFN_DOWN(gpa);
+ hvio->mmio_access = (struct npfec){ .gla_valid = 1,
+ .read_access = 1,
+ .write_access = write };
}
static int hvmemul_linear_mmio_access(
unsigned long gla, unsigned int size, uint8_t dir, void *buffer,
uint32_t pfec, struct hvm_emulate_ctxt *hvmemul_ctxt, bool_t known_gpfn)
{
- struct hvm_vcpu_io *vio = ¤t->arch.hvm.hvm_io;
+ struct hvm_vcpu_io *hvio = ¤t->arch.hvm.hvm_io;
unsigned long offset = gla & ~PAGE_MASK;
- struct hvm_mmio_cache *cache = hvmemul_find_mmio_cache(vio, gla, dir, true);
+ struct hvm_mmio_cache *cache = hvmemul_find_mmio_cache(hvio, gla, dir, true);
unsigned int chunk, buffer_offset = 0;
paddr_t gpa;
unsigned long one_rep = 1;
@@ -1068,7 +1068,7 @@ static int hvmemul_linear_mmio_access(
chunk = min_t(unsigned int, size, PAGE_SIZE - offset);
if ( known_gpfn )
- gpa = pfn_to_paddr(vio->mmio_gpfn) | offset;
+ gpa = pfn_to_paddr(hvio->mmio_gpfn) | offset;
else
{
rc = hvmemul_linear_to_phys(gla, &gpa, chunk, &one_rep, pfec,
@@ -1076,7 +1076,7 @@ static int hvmemul_linear_mmio_access(
if ( rc != X86EMUL_OKAY )
return rc;
- latch_linear_to_phys(vio, gla, gpa, dir == IOREQ_WRITE);
+ latch_linear_to_phys(hvio, gla, gpa, dir == IOREQ_WRITE);
}
for ( ;; )
@@ -1122,22 +1122,22 @@ static inline int hvmemul_linear_mmio_write(
static bool known_gla(unsigned long addr, unsigned int bytes, uint32_t pfec)
{
- const struct hvm_vcpu_io *vio = ¤t->arch.hvm.hvm_io;
+ const struct hvm_vcpu_io *hvio = ¤t->arch.hvm.hvm_io;
if ( pfec & PFEC_write_access )
{
- if ( !vio->mmio_access.write_access )
+ if ( !hvio->mmio_access.write_access )
return false;
}
else if ( pfec & PFEC_insn_fetch )
{
- if ( !vio->mmio_access.insn_fetch )
+ if ( !hvio->mmio_access.insn_fetch )
return false;
}
- else if ( !vio->mmio_access.read_access )
+ else if ( !hvio->mmio_access.read_access )
return false;
- return (vio->mmio_gla == (addr & PAGE_MASK) &&
+ return (hvio->mmio_gla == (addr & PAGE_MASK) &&
(addr & ~PAGE_MASK) + bytes <= PAGE_SIZE);
}
@@ -1145,7 +1145,7 @@ static int linear_read(unsigned long addr, unsigned int bytes, void *p_data,
uint32_t pfec, struct hvm_emulate_ctxt *hvmemul_ctxt)
{
pagefault_info_t pfinfo;
- struct hvm_vcpu_io *vio = ¤t->arch.hvm.hvm_io;
+ struct hvm_vcpu_io *hvio = ¤t->arch.hvm.hvm_io;
unsigned int offset = addr & ~PAGE_MASK;
int rc = HVMTRANS_bad_gfn_to_mfn;
@@ -1167,7 +1167,7 @@ static int linear_read(unsigned long addr, unsigned int bytes, void *p_data,
* we handle this access in the same way to guarantee completion and hence
* clean up any interim state.
*/
- if ( !hvmemul_find_mmio_cache(vio, addr, IOREQ_READ, false) )
+ if ( !hvmemul_find_mmio_cache(hvio, addr, IOREQ_READ, false) )
rc = hvm_copy_from_guest_linear(p_data, addr, bytes, pfec, &pfinfo);
switch ( rc )
@@ -1200,7 +1200,7 @@ static int linear_write(unsigned long addr, unsigned int bytes, void *p_data,
uint32_t pfec, struct hvm_emulate_ctxt *hvmemul_ctxt)
{
pagefault_info_t pfinfo;
- struct hvm_vcpu_io *vio = ¤t->arch.hvm.hvm_io;
+ struct hvm_vcpu_io *hvio = ¤t->arch.hvm.hvm_io;
unsigned int offset = addr & ~PAGE_MASK;
int rc = HVMTRANS_bad_gfn_to_mfn;
@@ -1222,7 +1222,7 @@ static int linear_write(unsigned long addr, unsigned int bytes, void *p_data,
* we handle this access in the same way to guarantee completion and hence
* clean up any interim state.
*/
- if ( !hvmemul_find_mmio_cache(vio, addr, IOREQ_WRITE, false) )
+ if ( !hvmemul_find_mmio_cache(hvio, addr, IOREQ_WRITE, false) )
rc = hvm_copy_to_guest_linear(addr, p_data, bytes, pfec, &pfinfo);
switch ( rc )
@@ -1599,7 +1599,7 @@ static int hvmemul_cmpxchg(
struct vcpu *curr = current;
unsigned long addr;
uint32_t pfec = PFEC_page_present | PFEC_write_access;
- struct hvm_vcpu_io *vio = &curr->arch.hvm.hvm_io;
+ struct hvm_vcpu_io *hvio = &curr->arch.hvm.hvm_io;
int rc;
void *mapping = NULL;
@@ -1625,8 +1625,8 @@ static int hvmemul_cmpxchg(
/* Fix this in case the guest is really relying on r-m-w atomicity. */
return hvmemul_linear_mmio_write(addr, bytes, p_new, pfec,
hvmemul_ctxt,
- vio->mmio_access.write_access &&
- vio->mmio_gla == (addr & PAGE_MASK));
+ hvio->mmio_access.write_access &&
+ hvio->mmio_gla == (addr & PAGE_MASK));
}
switch ( bytes )
@@ -1823,7 +1823,7 @@ static int hvmemul_rep_movs(
struct hvm_emulate_ctxt *hvmemul_ctxt =
container_of(ctxt, struct hvm_emulate_ctxt, ctxt);
struct vcpu *curr = current;
- struct hvm_vcpu_io *vio = &curr->arch.hvm.hvm_io;
+ struct hvm_vcpu_io *hvio = &curr->arch.hvm.hvm_io;
unsigned long saddr, daddr, bytes;
paddr_t sgpa, dgpa;
uint32_t pfec = PFEC_page_present;
@@ -1846,18 +1846,18 @@ static int hvmemul_rep_movs(
if ( hvmemul_ctxt->seg_reg[x86_seg_ss].dpl == 3 )
pfec |= PFEC_user_mode;
- if ( vio->mmio_access.read_access &&
- (vio->mmio_gla == (saddr & PAGE_MASK)) &&
+ if ( hvio->mmio_access.read_access &&
+ (hvio->mmio_gla == (saddr & PAGE_MASK)) &&
/*
* Upon initial invocation don't truncate large batches just because
* of a hit for the translation: Doing the guest page table walk is
* cheaper than multiple round trips through the device model. Yet
* when processing a response we can always re-use the translation.
*/
- (vio->io_req.state == STATE_IORESP_READY ||
+ (curr->io.req.state == STATE_IORESP_READY ||
((!df || *reps == 1) &&
PAGE_SIZE - (saddr & ~PAGE_MASK) >= *reps * bytes_per_rep)) )
- sgpa = pfn_to_paddr(vio->mmio_gpfn) | (saddr & ~PAGE_MASK);
+ sgpa = pfn_to_paddr(hvio->mmio_gpfn) | (saddr & ~PAGE_MASK);
else
{
rc = hvmemul_linear_to_phys(saddr, &sgpa, bytes_per_rep, reps, pfec,
@@ -1867,13 +1867,13 @@ static int hvmemul_rep_movs(
}
bytes = PAGE_SIZE - (daddr & ~PAGE_MASK);
- if ( vio->mmio_access.write_access &&
- (vio->mmio_gla == (daddr & PAGE_MASK)) &&
+ if ( hvio->mmio_access.write_access &&
+ (hvio->mmio_gla == (daddr & PAGE_MASK)) &&
/* See comment above. */
- (vio->io_req.state == STATE_IORESP_READY ||
+ (curr->io.req.state == STATE_IORESP_READY ||
((!df || *reps == 1) &&
PAGE_SIZE - (daddr & ~PAGE_MASK) >= *reps * bytes_per_rep)) )
- dgpa = pfn_to_paddr(vio->mmio_gpfn) | (daddr & ~PAGE_MASK);
+ dgpa = pfn_to_paddr(hvio->mmio_gpfn) | (daddr & ~PAGE_MASK);
else
{
rc = hvmemul_linear_to_phys(daddr, &dgpa, bytes_per_rep, reps,
@@ -1892,14 +1892,14 @@ static int hvmemul_rep_movs(
if ( sp2mt == p2m_mmio_dm )
{
- latch_linear_to_phys(vio, saddr, sgpa, 0);
+ latch_linear_to_phys(hvio, saddr, sgpa, 0);
return hvmemul_do_mmio_addr(
sgpa, reps, bytes_per_rep, IOREQ_READ, df, dgpa);
}
if ( dp2mt == p2m_mmio_dm )
{
- latch_linear_to_phys(vio, daddr, dgpa, 1);
+ latch_linear_to_phys(hvio, daddr, dgpa, 1);
return hvmemul_do_mmio_addr(
dgpa, reps, bytes_per_rep, IOREQ_WRITE, df, sgpa);
}
@@ -1992,7 +1992,7 @@ static int hvmemul_rep_stos(
struct hvm_emulate_ctxt *hvmemul_ctxt =
container_of(ctxt, struct hvm_emulate_ctxt, ctxt);
struct vcpu *curr = current;
- struct hvm_vcpu_io *vio = &curr->arch.hvm.hvm_io;
+ struct hvm_vcpu_io *hvio = &curr->arch.hvm.hvm_io;
unsigned long addr, bytes;
paddr_t gpa;
p2m_type_t p2mt;
@@ -2004,13 +2004,13 @@ static int hvmemul_rep_stos(
return rc;
bytes = PAGE_SIZE - (addr & ~PAGE_MASK);
- if ( vio->mmio_access.write_access &&
- (vio->mmio_gla == (addr & PAGE_MASK)) &&
+ if ( hvio->mmio_access.write_access &&
+ (hvio->mmio_gla == (addr & PAGE_MASK)) &&
/* See respective comment in MOVS processing. */
- (vio->io_req.state == STATE_IORESP_READY ||
+ (curr->io.req.state == STATE_IORESP_READY ||
((!df || *reps == 1) &&
PAGE_SIZE - (addr & ~PAGE_MASK) >= *reps * bytes_per_rep)) )
- gpa = pfn_to_paddr(vio->mmio_gpfn) | (addr & ~PAGE_MASK);
+ gpa = pfn_to_paddr(hvio->mmio_gpfn) | (addr & ~PAGE_MASK);
else
{
uint32_t pfec = PFEC_page_present | PFEC_write_access;
@@ -2103,7 +2103,7 @@ static int hvmemul_rep_stos(
return X86EMUL_UNHANDLEABLE;
case p2m_mmio_dm:
- latch_linear_to_phys(vio, addr, gpa, 1);
+ latch_linear_to_phys(hvio, addr, gpa, 1);
return hvmemul_do_mmio_buffer(gpa, reps, bytes_per_rep, IOREQ_WRITE, df,
p_data);
}
@@ -2613,18 +2613,18 @@ static const struct x86_emulate_ops hvm_emulate_ops_no_write = {
};
/*
- * Note that passing HVMIO_no_completion into this function serves as kind
+ * Note that passing VIO_no_completion into this function serves as kind
* of (but not fully) an "auto select completion" indicator. When there's
* no completion needed, the passed in value will be ignored in any case.
*/
static int _hvm_emulate_one(struct hvm_emulate_ctxt *hvmemul_ctxt,
const struct x86_emulate_ops *ops,
- enum hvm_io_completion completion)
+ enum vio_completion completion)
{
const struct cpu_user_regs *regs = hvmemul_ctxt->ctxt.regs;
struct vcpu *curr = current;
uint32_t new_intr_shadow;
- struct hvm_vcpu_io *vio = &curr->arch.hvm.hvm_io;
+ struct hvm_vcpu_io *hvio = &curr->arch.hvm.hvm_io;
int rc;
/*
@@ -2632,45 +2632,45 @@ static int _hvm_emulate_one(struct hvm_emulate_ctxt *hvmemul_ctxt,
* untouched if it's already enabled, for re-execution to consume
* entries populated by an earlier pass.
*/
- if ( vio->cache->num_ents > vio->cache->max_ents )
+ if ( hvio->cache->num_ents > hvio->cache->max_ents )
{
- ASSERT(vio->io_req.state == STATE_IOREQ_NONE);
- vio->cache->num_ents = 0;
+ ASSERT(curr->io.req.state == STATE_IOREQ_NONE);
+ hvio->cache->num_ents = 0;
}
else
- ASSERT(vio->io_req.state == STATE_IORESP_READY);
+ ASSERT(curr->io.req.state == STATE_IORESP_READY);
- hvm_emulate_init_per_insn(hvmemul_ctxt, vio->mmio_insn,
- vio->mmio_insn_bytes);
+ hvm_emulate_init_per_insn(hvmemul_ctxt, hvio->mmio_insn,
+ hvio->mmio_insn_bytes);
- vio->mmio_retry = 0;
+ hvio->mmio_retry = 0;
rc = x86_emulate(&hvmemul_ctxt->ctxt, ops);
- if ( rc == X86EMUL_OKAY && vio->mmio_retry )
+ if ( rc == X86EMUL_OKAY && hvio->mmio_retry )
rc = X86EMUL_RETRY;
- if ( !ioreq_needs_completion(&vio->io_req) )
- completion = HVMIO_no_completion;
- else if ( completion == HVMIO_no_completion )
- completion = (vio->io_req.type != IOREQ_TYPE_PIO ||
- hvmemul_ctxt->is_mem_access) ? HVMIO_mmio_completion
- : HVMIO_pio_completion;
+ if ( !ioreq_needs_completion(&curr->io.req) )
+ completion = VIO_no_completion;
+ else if ( completion == VIO_no_completion )
+ completion = (curr->io.req.type != IOREQ_TYPE_PIO ||
+ hvmemul_ctxt->is_mem_access) ? VIO_mmio_completion
+ : VIO_pio_completion;
- switch ( vio->io_completion = completion )
+ switch ( curr->io.completion = completion )
{
- case HVMIO_no_completion:
- case HVMIO_pio_completion:
- vio->mmio_cache_count = 0;
- vio->mmio_insn_bytes = 0;
- vio->mmio_access = (struct npfec){};
+ case VIO_no_completion:
+ case VIO_pio_completion:
+ hvio->mmio_cache_count = 0;
+ hvio->mmio_insn_bytes = 0;
+ hvio->mmio_access = (struct npfec){};
hvmemul_cache_disable(curr);
break;
- case HVMIO_mmio_completion:
- case HVMIO_realmode_completion:
- BUILD_BUG_ON(sizeof(vio->mmio_insn) < sizeof(hvmemul_ctxt->insn_buf));
- vio->mmio_insn_bytes = hvmemul_ctxt->insn_buf_bytes;
- memcpy(vio->mmio_insn, hvmemul_ctxt->insn_buf, vio->mmio_insn_bytes);
+ case VIO_mmio_completion:
+ case VIO_realmode_completion:
+ BUILD_BUG_ON(sizeof(hvio->mmio_insn) < sizeof(hvmemul_ctxt->insn_buf));
+ hvio->mmio_insn_bytes = hvmemul_ctxt->insn_buf_bytes;
+ memcpy(hvio->mmio_insn, hvmemul_ctxt->insn_buf, hvio->mmio_insn_bytes);
break;
default:
@@ -2716,7 +2716,7 @@ static int _hvm_emulate_one(struct hvm_emulate_ctxt *hvmemul_ctxt,
int hvm_emulate_one(
struct hvm_emulate_ctxt *hvmemul_ctxt,
- enum hvm_io_completion completion)
+ enum vio_completion completion)
{
return _hvm_emulate_one(hvmemul_ctxt, &hvm_emulate_ops, completion);
}
@@ -2754,7 +2754,7 @@ int hvm_emulate_one_mmio(unsigned long mfn, unsigned long gla)
guest_cpu_user_regs());
ctxt.ctxt.data = &mmio_ro_ctxt;
- switch ( rc = _hvm_emulate_one(&ctxt, ops, HVMIO_no_completion) )
+ switch ( rc = _hvm_emulate_one(&ctxt, ops, VIO_no_completion) )
{
case X86EMUL_UNHANDLEABLE:
case X86EMUL_UNIMPLEMENTED:
@@ -2782,28 +2782,28 @@ void hvm_emulate_one_vm_event(enum emul_kind kind, unsigned int trapnr,
{
case EMUL_KIND_NOWRITE:
rc = _hvm_emulate_one(&ctx, &hvm_emulate_ops_no_write,
- HVMIO_no_completion);
+ VIO_no_completion);
break;
case EMUL_KIND_SET_CONTEXT_INSN: {
struct vcpu *curr = current;
- struct hvm_vcpu_io *vio = &curr->arch.hvm.hvm_io;
+ struct hvm_vcpu_io *hvio = &curr->arch.hvm.hvm_io;
- BUILD_BUG_ON(sizeof(vio->mmio_insn) !=
+ BUILD_BUG_ON(sizeof(hvio->mmio_insn) !=
sizeof(curr->arch.vm_event->emul.insn.data));
- ASSERT(!vio->mmio_insn_bytes);
+ ASSERT(!hvio->mmio_insn_bytes);
/*
* Stash insn buffer into mmio buffer here instead of ctx
* to avoid having to add more logic to hvm_emulate_one.
*/
- vio->mmio_insn_bytes = sizeof(vio->mmio_insn);
- memcpy(vio->mmio_insn, curr->arch.vm_event->emul.insn.data,
- vio->mmio_insn_bytes);
+ hvio->mmio_insn_bytes = sizeof(hvio->mmio_insn);
+ memcpy(hvio->mmio_insn, curr->arch.vm_event->emul.insn.data,
+ hvio->mmio_insn_bytes);
}
/* Fall-through */
default:
ctx.set_context = (kind == EMUL_KIND_SET_CONTEXT_DATA);
- rc = hvm_emulate_one(&ctx, HVMIO_no_completion);
+ rc = hvm_emulate_one(&ctx, VIO_no_completion);
}
switch ( rc )
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index bc96947..4ed929c 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -3800,7 +3800,7 @@ void hvm_ud_intercept(struct cpu_user_regs *regs)
return;
}
- switch ( hvm_emulate_one(&ctxt, HVMIO_no_completion) )
+ switch ( hvm_emulate_one(&ctxt, VIO_no_completion) )
{
case X86EMUL_UNHANDLEABLE:
case X86EMUL_UNIMPLEMENTED:
diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
index ef8286b..dd733e1 100644
--- a/xen/arch/x86/hvm/io.c
+++ b/xen/arch/x86/hvm/io.c
@@ -85,7 +85,7 @@ bool hvm_emulate_one_insn(hvm_emulate_validate_t *validate, const char *descr)
hvm_emulate_init_once(&ctxt, validate, guest_cpu_user_regs());
- switch ( rc = hvm_emulate_one(&ctxt, HVMIO_no_completion) )
+ switch ( rc = hvm_emulate_one(&ctxt, VIO_no_completion) )
{
case X86EMUL_UNHANDLEABLE:
hvm_dump_emulation_state(XENLOG_G_WARNING, descr, &ctxt, rc);
@@ -109,20 +109,20 @@ bool hvm_emulate_one_insn(hvm_emulate_validate_t *validate, const char *descr)
bool handle_mmio_with_translation(unsigned long gla, unsigned long gpfn,
struct npfec access)
{
- struct hvm_vcpu_io *vio = ¤t->arch.hvm.hvm_io;
+ struct hvm_vcpu_io *hvio = ¤t->arch.hvm.hvm_io;
- vio->mmio_access = access.gla_valid &&
- access.kind == npfec_kind_with_gla
- ? access : (struct npfec){};
- vio->mmio_gla = gla & PAGE_MASK;
- vio->mmio_gpfn = gpfn;
+ hvio->mmio_access = access.gla_valid &&
+ access.kind == npfec_kind_with_gla
+ ? access : (struct npfec){};
+ hvio->mmio_gla = gla & PAGE_MASK;
+ hvio->mmio_gpfn = gpfn;
return handle_mmio();
}
bool handle_pio(uint16_t port, unsigned int size, int dir)
{
struct vcpu *curr = current;
- struct hvm_vcpu_io *vio = &curr->arch.hvm.hvm_io;
+ struct vcpu_io *vio = &curr->io;
unsigned int data;
int rc;
@@ -135,8 +135,8 @@ bool handle_pio(uint16_t port, unsigned int size, int dir)
rc = hvmemul_do_pio_buffer(port, size, dir, &data);
- if ( ioreq_needs_completion(&vio->io_req) )
- vio->io_completion = HVMIO_pio_completion;
+ if ( ioreq_needs_completion(&vio->req) )
+ vio->completion = VIO_pio_completion;
switch ( rc )
{
@@ -175,7 +175,7 @@ static bool_t g2m_portio_accept(const struct hvm_io_handler *handler,
{
struct vcpu *curr = current;
const struct hvm_domain *hvm = &curr->domain->arch.hvm;
- struct hvm_vcpu_io *vio = &curr->arch.hvm.hvm_io;
+ struct hvm_vcpu_io *hvio = &curr->arch.hvm.hvm_io;
struct g2m_ioport *g2m_ioport;
unsigned int start, end;
@@ -185,7 +185,7 @@ static bool_t g2m_portio_accept(const struct hvm_io_handler *handler,
end = start + g2m_ioport->np;
if ( (p->addr >= start) && (p->addr + p->size <= end) )
{
- vio->g2m_ioport = g2m_ioport;
+ hvio->g2m_ioport = g2m_ioport;
return 1;
}
}
@@ -196,8 +196,8 @@ static bool_t g2m_portio_accept(const struct hvm_io_handler *handler,
static int g2m_portio_read(const struct hvm_io_handler *handler,
uint64_t addr, uint32_t size, uint64_t *data)
{
- struct hvm_vcpu_io *vio = ¤t->arch.hvm.hvm_io;
- const struct g2m_ioport *g2m_ioport = vio->g2m_ioport;
+ struct hvm_vcpu_io *hvio = ¤t->arch.hvm.hvm_io;
+ const struct g2m_ioport *g2m_ioport = hvio->g2m_ioport;
unsigned int mport = (addr - g2m_ioport->gport) + g2m_ioport->mport;
switch ( size )
@@ -221,8 +221,8 @@ static int g2m_portio_read(const struct hvm_io_handler *handler,
static int g2m_portio_write(const struct hvm_io_handler *handler,
uint64_t addr, uint32_t size, uint64_t data)
{
- struct hvm_vcpu_io *vio = ¤t->arch.hvm.hvm_io;
- const struct g2m_ioport *g2m_ioport = vio->g2m_ioport;
+ struct hvm_vcpu_io *hvio = ¤t->arch.hvm.hvm_io;
+ const struct g2m_ioport *g2m_ioport = hvio->g2m_ioport;
unsigned int mport = (addr - g2m_ioport->gport) + g2m_ioport->mport;
switch ( size )
diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 0cadf34..62a4b33 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -40,11 +40,11 @@ bool arch_ioreq_complete_mmio(void)
return handle_mmio();
}
-bool arch_vcpu_ioreq_completion(enum hvm_io_completion io_completion)
+bool arch_vcpu_ioreq_completion(enum vio_completion completion)
{
- switch ( io_completion )
+ switch ( completion )
{
- case HVMIO_realmode_completion:
+ case VIO_realmode_completion:
{
struct hvm_emulate_ctxt ctxt;
diff --git a/xen/arch/x86/hvm/svm/nestedsvm.c b/xen/arch/x86/hvm/svm/nestedsvm.c
index fcfccf7..6d90630 100644
--- a/xen/arch/x86/hvm/svm/nestedsvm.c
+++ b/xen/arch/x86/hvm/svm/nestedsvm.c
@@ -1266,7 +1266,7 @@ enum hvm_intblk nsvm_intr_blocked(struct vcpu *v)
* Delay the injection because this would result in delivering
* an interrupt *within* the execution of an instruction.
*/
- if ( v->arch.hvm.hvm_io.io_req.state != STATE_IOREQ_NONE )
+ if ( v->io.req.state != STATE_IOREQ_NONE )
return hvm_intblk_shadow;
if ( !nv->nv_vmexit_pending && n2vmcb->exit_int_info.v )
diff --git a/xen/arch/x86/hvm/vmx/realmode.c b/xen/arch/x86/hvm/vmx/realmode.c
index 768f01e..cc23afa 100644
--- a/xen/arch/x86/hvm/vmx/realmode.c
+++ b/xen/arch/x86/hvm/vmx/realmode.c
@@ -101,7 +101,7 @@ void vmx_realmode_emulate_one(struct hvm_emulate_ctxt *hvmemul_ctxt)
perfc_incr(realmode_emulations);
- rc = hvm_emulate_one(hvmemul_ctxt, HVMIO_realmode_completion);
+ rc = hvm_emulate_one(hvmemul_ctxt, VIO_realmode_completion);
if ( rc == X86EMUL_UNHANDLEABLE )
{
@@ -153,7 +153,7 @@ void vmx_realmode(struct cpu_user_regs *regs)
struct vcpu *curr = current;
struct hvm_emulate_ctxt hvmemul_ctxt;
struct segment_register *sreg;
- struct hvm_vcpu_io *vio = &curr->arch.hvm.hvm_io;
+ struct hvm_vcpu_io *hvio = &curr->arch.hvm.hvm_io;
unsigned long intr_info;
unsigned int emulations = 0;
@@ -188,7 +188,7 @@ void vmx_realmode(struct cpu_user_regs *regs)
vmx_realmode_emulate_one(&hvmemul_ctxt);
- if ( vio->io_req.state != STATE_IOREQ_NONE || vio->mmio_retry )
+ if ( curr->io.req.state != STATE_IOREQ_NONE || hvio->mmio_retry )
break;
/* Stop emulating unless our segment state is not safe */
@@ -202,7 +202,7 @@ void vmx_realmode(struct cpu_user_regs *regs)
}
/* Need to emulate next time if we've started an IO operation */
- if ( vio->io_req.state != STATE_IOREQ_NONE )
+ if ( curr->io.req.state != STATE_IOREQ_NONE )
curr->arch.hvm.vmx.vmx_emulate = 1;
if ( !curr->arch.hvm.vmx.vmx_emulate && !curr->arch.hvm.vmx.vmx_realmode )
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index 84bce36..ce3ef59 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -159,7 +159,7 @@ static bool hvm_wait_for_io(struct ioreq_vcpu *sv, ioreq_t *p)
break;
}
- p = &sv->vcpu->arch.hvm.hvm_io.io_req;
+ p = &sv->vcpu->io.req;
if ( ioreq_needs_completion(p) )
p->data = data;
@@ -171,10 +171,10 @@ static bool hvm_wait_for_io(struct ioreq_vcpu *sv, ioreq_t *p)
bool handle_hvm_io_completion(struct vcpu *v)
{
struct domain *d = v->domain;
- struct hvm_vcpu_io *vio = &v->arch.hvm.hvm_io;
+ struct vcpu_io *vio = &v->io;
struct ioreq_server *s;
struct ioreq_vcpu *sv;
- enum hvm_io_completion io_completion;
+ enum vio_completion completion;
if ( has_vpci(d) && vpci_process_pending(v) )
{
@@ -186,29 +186,29 @@ bool handle_hvm_io_completion(struct vcpu *v)
if ( sv && !hvm_wait_for_io(sv, get_ioreq(s, v)) )
return false;
- vio->io_req.state = ioreq_needs_completion(&vio->io_req) ?
+ vio->req.state = ioreq_needs_completion(&vio->req) ?
STATE_IORESP_READY : STATE_IOREQ_NONE;
msix_write_completion(v);
vcpu_end_shutdown_deferral(v);
- io_completion = vio->io_completion;
- vio->io_completion = HVMIO_no_completion;
+ completion = vio->completion;
+ vio->completion = VIO_no_completion;
- switch ( io_completion )
+ switch ( completion )
{
- case HVMIO_no_completion:
+ case VIO_no_completion:
break;
- case HVMIO_mmio_completion:
+ case VIO_mmio_completion:
return arch_ioreq_complete_mmio();
- case HVMIO_pio_completion:
- return handle_pio(vio->io_req.addr, vio->io_req.size,
- vio->io_req.dir);
+ case VIO_pio_completion:
+ return handle_pio(vio->req.addr, vio->req.size,
+ vio->req.dir);
default:
- return arch_vcpu_ioreq_completion(io_completion);
+ return arch_vcpu_ioreq_completion(completion);
}
return true;
diff --git a/xen/include/asm-x86/hvm/emulate.h b/xen/include/asm-x86/hvm/emulate.h
index 1620cc7..610078b 100644
--- a/xen/include/asm-x86/hvm/emulate.h
+++ b/xen/include/asm-x86/hvm/emulate.h
@@ -65,7 +65,7 @@ bool __nonnull(1, 2) hvm_emulate_one_insn(
const char *descr);
int hvm_emulate_one(
struct hvm_emulate_ctxt *hvmemul_ctxt,
- enum hvm_io_completion completion);
+ enum vio_completion completion);
void hvm_emulate_one_vm_event(enum emul_kind kind,
unsigned int trapnr,
unsigned int errcode);
diff --git a/xen/include/asm-x86/hvm/vcpu.h b/xen/include/asm-x86/hvm/vcpu.h
index 6c1feda..8adf455 100644
--- a/xen/include/asm-x86/hvm/vcpu.h
+++ b/xen/include/asm-x86/hvm/vcpu.h
@@ -28,13 +28,6 @@
#include <asm/mtrr.h>
#include <public/hvm/ioreq.h>
-enum hvm_io_completion {
- HVMIO_no_completion,
- HVMIO_mmio_completion,
- HVMIO_pio_completion,
- HVMIO_realmode_completion
-};
-
struct hvm_vcpu_asid {
uint64_t generation;
uint32_t asid;
@@ -52,10 +45,6 @@ struct hvm_mmio_cache {
};
struct hvm_vcpu_io {
- /* I/O request in flight to device model. */
- enum hvm_io_completion io_completion;
- ioreq_t io_req;
-
/*
* HVM emulation:
* Linear address @mmio_gla maps to MMIO physical frame @mmio_gpfn.
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
index 60e864d..eace1d3 100644
--- a/xen/include/xen/ioreq.h
+++ b/xen/include/xen/ioreq.h
@@ -107,7 +107,7 @@ void hvm_ioreq_init(struct domain *d);
int ioreq_server_dm_op(struct xen_dm_op *op, struct domain *d, bool *const_op);
bool arch_ioreq_complete_mmio(void);
-bool arch_vcpu_ioreq_completion(enum hvm_io_completion io_completion);
+bool arch_vcpu_ioreq_completion(enum vio_completion completion);
int arch_ioreq_server_map_pages(struct ioreq_server *s);
void arch_ioreq_server_unmap_pages(struct ioreq_server *s);
void arch_ioreq_server_enable(struct ioreq_server *s);
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index f437ee3..59e5b6a 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -147,6 +147,21 @@ void evtchn_destroy_final(struct domain *d); /* from complete_domain_destroy */
struct waitqueue_vcpu;
+enum vio_completion {
+ VIO_no_completion,
+ VIO_mmio_completion,
+ VIO_pio_completion,
+#ifdef CONFIG_X86
+ VIO_realmode_completion,
+#endif
+};
+
+struct vcpu_io {
+ /* I/O request in flight to device model. */
+ enum vio_completion completion;
+ ioreq_t req;
+};
+
struct vcpu
{
int vcpu_id;
@@ -258,6 +273,10 @@ struct vcpu
struct vpci_vcpu vpci;
struct arch_vcpu arch;
+
+#ifdef CONFIG_IOREQ_SERVER
+ struct vcpu_io io;
+#endif
};
struct sched_unit {
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 11/22] xen/mm: Make x86's XENMEM_resource_ioreq_server handling common
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (9 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 10/22] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 12/22] xen/ioreq: Remove "hvm" prefixes from involved function names Oleksandr Tyshchenko
` (12 subsequent siblings)
23 siblings, 0 replies; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Julien Grall, Jan Beulich, Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Oleksandr Tyshchenko
From: Julien Grall <julien.grall@arm.com>
As x86 implementation of XENMEM_resource_ioreq_server can be
re-used on Arm later on, this patch makes it common and removes
arch_acquire_resource (and the corresponding option) as unneeded.
Also re-order #include-s alphabetically.
This support is going to be used on Arm to be able run device
emulator outside of Xen hypervisor.
Signed-off-by: Julien Grall <julien.grall@arm.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul@xen.org>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- no changes
Changes V1 -> V2:
- update the author of a patch
Changes V2 -> V3:
- don't wrap #include <xen/ioreq.h>
- limit the number of #ifdef-s
- re-order #include-s alphabetically
Changes V3 -> V4:
- rebase
- Add Jan's R-b
Changes V4 -> V5:
- add Paul's R-b
- update patch description
- remove ARCH_ACQUIRE_RESOURCE option, etc
---
---
xen/arch/x86/Kconfig | 1 -
xen/arch/x86/mm.c | 44 ---------------------------------
xen/common/Kconfig | 3 ---
xen/common/memory.c | 63 +++++++++++++++++++++++++++++++++++++++---------
xen/include/asm-x86/mm.h | 4 ---
xen/include/xen/mm.h | 9 -------
6 files changed, 51 insertions(+), 73 deletions(-)
diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index ea9a9ea..abe0fce 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -6,7 +6,6 @@ config X86
select ACPI
select ACPI_LEGACY_TABLES_LOOKUP
select ARCH_SUPPORTS_INT128
- select ARCH_ACQUIRE_RESOURCE
select COMPAT
select CORE_PARKING
select HAS_ALTERNATIVE
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 59eb5c8..4366ea3 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -4587,50 +4587,6 @@ static int handle_iomem_range(unsigned long s, unsigned long e, void *p)
return err || s > e ? err : _handle_iomem_range(s, e, p);
}
-int arch_acquire_resource(struct domain *d, unsigned int type,
- unsigned int id, unsigned long frame,
- unsigned int nr_frames, xen_pfn_t mfn_list[])
-{
- int rc;
-
- switch ( type )
- {
-#ifdef CONFIG_HVM
- case XENMEM_resource_ioreq_server:
- {
- ioservid_t ioservid = id;
- unsigned int i;
-
- rc = -EINVAL;
- if ( !is_hvm_domain(d) )
- break;
-
- if ( id != (unsigned int)ioservid )
- break;
-
- rc = 0;
- for ( i = 0; i < nr_frames; i++ )
- {
- mfn_t mfn;
-
- rc = hvm_get_ioreq_server_frame(d, id, frame + i, &mfn);
- if ( rc )
- break;
-
- mfn_list[i] = mfn_x(mfn);
- }
- break;
- }
-#endif
-
- default:
- rc = -EOPNOTSUPP;
- break;
- }
-
- return rc;
-}
-
long arch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
{
int rc;
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index cf32a07..fa049a6 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -22,9 +22,6 @@ config GRANT_TABLE
If unsure, say Y.
-config ARCH_ACQUIRE_RESOURCE
- bool
-
config HAS_ALTERNATIVE
bool
diff --git a/xen/common/memory.c b/xen/common/memory.c
index ccb4d49..2f274a6 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -8,22 +8,23 @@
*/
#include <xen/domain_page.h>
-#include <xen/types.h>
+#include <xen/errno.h>
+#include <xen/event.h>
+#include <xen/grant_table.h>
+#include <xen/guest_access.h>
+#include <xen/hypercall.h>
+#include <xen/iocap.h>
+#include <xen/ioreq.h>
#include <xen/lib.h>
+#include <xen/mem_access.h>
#include <xen/mm.h>
+#include <xen/numa.h>
+#include <xen/paging.h>
#include <xen/param.h>
#include <xen/perfc.h>
#include <xen/sched.h>
-#include <xen/event.h>
-#include <xen/paging.h>
-#include <xen/iocap.h>
-#include <xen/guest_access.h>
-#include <xen/hypercall.h>
-#include <xen/errno.h>
-#include <xen/numa.h>
-#include <xen/mem_access.h>
#include <xen/trace.h>
-#include <xen/grant_table.h>
+#include <xen/types.h>
#include <asm/current.h>
#include <asm/hardirq.h>
#include <asm/p2m.h>
@@ -1091,6 +1092,40 @@ static int acquire_grant_table(struct domain *d, unsigned int id,
return 0;
}
+static int acquire_ioreq_server(struct domain *d,
+ unsigned int id,
+ unsigned long frame,
+ unsigned int nr_frames,
+ xen_pfn_t mfn_list[])
+{
+#ifdef CONFIG_IOREQ_SERVER
+ ioservid_t ioservid = id;
+ unsigned int i;
+ int rc;
+
+ if ( !is_hvm_domain(d) )
+ return -EINVAL;
+
+ if ( id != (unsigned int)ioservid )
+ return -EINVAL;
+
+ for ( i = 0; i < nr_frames; i++ )
+ {
+ mfn_t mfn;
+
+ rc = hvm_get_ioreq_server_frame(d, id, frame + i, &mfn);
+ if ( rc )
+ return rc;
+
+ mfn_list[i] = mfn_x(mfn);
+ }
+
+ return 0;
+#else
+ return -EOPNOTSUPP;
+#endif
+}
+
static int acquire_resource(
XEN_GUEST_HANDLE_PARAM(xen_mem_acquire_resource_t) arg)
{
@@ -1149,9 +1184,13 @@ static int acquire_resource(
mfn_list);
break;
+ case XENMEM_resource_ioreq_server:
+ rc = acquire_ioreq_server(d, xmar.id, xmar.frame, xmar.nr_frames,
+ mfn_list);
+ break;
+
default:
- rc = arch_acquire_resource(d, xmar.type, xmar.id, xmar.frame,
- xmar.nr_frames, mfn_list);
+ rc = -EOPNOTSUPP;
break;
}
diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h
index 1fdb4eb..041c158 100644
--- a/xen/include/asm-x86/mm.h
+++ b/xen/include/asm-x86/mm.h
@@ -640,8 +640,4 @@ static inline bool arch_mfn_in_directmap(unsigned long mfn)
return mfn <= (virt_to_mfn(eva - 1) + 1);
}
-int arch_acquire_resource(struct domain *d, unsigned int type,
- unsigned int id, unsigned long frame,
- unsigned int nr_frames, xen_pfn_t mfn_list[]);
-
#endif /* __ASM_X86_MM_H__ */
diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
index 636a125..667f9da 100644
--- a/xen/include/xen/mm.h
+++ b/xen/include/xen/mm.h
@@ -612,13 +612,4 @@ static inline void put_page_alloc_ref(struct page_info *page)
}
}
-#ifndef CONFIG_ARCH_ACQUIRE_RESOURCE
-static inline int arch_acquire_resource(
- struct domain *d, unsigned int type, unsigned int id, unsigned long frame,
- unsigned int nr_frames, xen_pfn_t mfn_list[])
-{
- return -EOPNOTSUPP;
-}
-#endif /* !CONFIG_ARCH_ACQUIRE_RESOURCE */
-
#endif /* __XEN_MM_H__ */
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 12/22] xen/ioreq: Remove "hvm" prefixes from involved function names
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (10 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 11/22] xen/mm: Make x86's XENMEM_resource_ioreq_server handling common Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 13/22] xen/ioreq: Use guest_cmpxchg64() instead of cmpxchg() Oleksandr Tyshchenko
` (11 subsequent siblings)
23 siblings, 0 replies; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
This patch removes "hvm" prefixes and infixes from IOREQ related
function names in the common code and performs a renaming where
appropriate according to the more consistent new naming scheme:
- IOREQ server functions should start with "ioreq_server_"
- IOREQ functions should start with "ioreq_"
A few function names are clarified to better fit into their purposes:
handle_hvm_io_completion -> vcpu_ioreq_handle_completion
hvm_io_pending -> vcpu_ioreq_pending
hvm_ioreq_init -> ioreq_domain_init
hvm_alloc_ioreq_mfn -> ioreq_server_alloc_mfn
hvm_free_ioreq_mfn -> ioreq_server_free_mfn
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul@xen.org>
CC: Julien Grall <julien.grall@arm.com>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes V1 -> V2:
- new patch
Changes V2 -> V3:
- update patch according the "legacy interface" is x86 specific
- update patch description
- rename everything touched according to new naming scheme
Changes V3 -> V4:
- rebase
- rename ioreq_update_evtchn() to ioreq_server_update_evtchn()
- add Jan's R-b
Changes V4 -> V5:
- rebase
- add Paul's R-b
---
---
xen/arch/x86/hvm/emulate.c | 6 +-
xen/arch/x86/hvm/hvm.c | 10 ++--
xen/arch/x86/hvm/io.c | 6 +-
xen/arch/x86/hvm/ioreq.c | 2 +-
xen/arch/x86/hvm/stdvga.c | 4 +-
xen/arch/x86/hvm/vmx/vvmx.c | 2 +-
xen/common/ioreq.c | 138 ++++++++++++++++++++++----------------------
xen/common/memory.c | 2 +-
xen/include/xen/ioreq.h | 26 ++++-----
9 files changed, 98 insertions(+), 98 deletions(-)
diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index 21051ce..425c8dd 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -261,7 +261,7 @@ static int hvmemul_do_io(
* an ioreq server that can handle it.
*
* Rules:
- * A> PIO or MMIO accesses run through hvm_select_ioreq_server() to
+ * A> PIO or MMIO accesses run through ioreq_server_select() to
* choose the ioreq server by range. If no server is found, the access
* is ignored.
*
@@ -323,7 +323,7 @@ static int hvmemul_do_io(
}
if ( !s )
- s = hvm_select_ioreq_server(currd, &p);
+ s = ioreq_server_select(currd, &p);
/* If there is no suitable backing DM, just ignore accesses */
if ( !s )
@@ -333,7 +333,7 @@ static int hvmemul_do_io(
}
else
{
- rc = hvm_send_ioreq(s, &p, 0);
+ rc = ioreq_send(s, &p, 0);
if ( rc != X86EMUL_RETRY || currd->is_shutting_down )
vio->req.state = STATE_IOREQ_NONE;
else if ( !ioreq_needs_completion(&vio->req) )
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 4ed929c..0d7bb42 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -546,7 +546,7 @@ void hvm_do_resume(struct vcpu *v)
pt_restore_timer(v);
- if ( !handle_hvm_io_completion(v) )
+ if ( !vcpu_ioreq_handle_completion(v) )
return;
if ( unlikely(v->arch.vm_event) )
@@ -677,7 +677,7 @@ int hvm_domain_initialise(struct domain *d)
register_g2m_portio_handler(d);
register_vpci_portio_handler(d);
- hvm_ioreq_init(d);
+ ioreq_domain_init(d);
hvm_init_guest_time(d);
@@ -739,7 +739,7 @@ void hvm_domain_relinquish_resources(struct domain *d)
viridian_domain_deinit(d);
- hvm_destroy_all_ioreq_servers(d);
+ ioreq_server_destroy_all(d);
msixtbl_pt_cleanup(d);
@@ -1582,7 +1582,7 @@ int hvm_vcpu_initialise(struct vcpu *v)
if ( rc )
goto fail5;
- rc = hvm_all_ioreq_servers_add_vcpu(d, v);
+ rc = ioreq_server_add_vcpu_all(d, v);
if ( rc != 0 )
goto fail6;
@@ -1618,7 +1618,7 @@ void hvm_vcpu_destroy(struct vcpu *v)
{
viridian_vcpu_deinit(v);
- hvm_all_ioreq_servers_remove_vcpu(v->domain, v);
+ ioreq_server_remove_vcpu_all(v->domain, v);
if ( hvm_altp2m_supported() )
altp2m_vcpu_destroy(v);
diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
index dd733e1..66a37ee 100644
--- a/xen/arch/x86/hvm/io.c
+++ b/xen/arch/x86/hvm/io.c
@@ -60,7 +60,7 @@ void send_timeoffset_req(unsigned long timeoff)
if ( timeoff == 0 )
return;
- if ( hvm_broadcast_ioreq(&p, true) != 0 )
+ if ( ioreq_broadcast(&p, true) != 0 )
gprintk(XENLOG_ERR, "Unsuccessful timeoffset update\n");
}
@@ -74,7 +74,7 @@ void send_invalidate_req(void)
.data = ~0UL, /* flush all */
};
- if ( hvm_broadcast_ioreq(&p, false) != 0 )
+ if ( ioreq_broadcast(&p, false) != 0 )
gprintk(XENLOG_ERR, "Unsuccessful map-cache invalidate\n");
}
@@ -155,7 +155,7 @@ bool handle_pio(uint16_t port, unsigned int size, int dir)
* We should not advance RIP/EIP if the domain is shutting down or
* if X86EMUL_RETRY has been returned by an internal handler.
*/
- if ( curr->domain->is_shutting_down || !hvm_io_pending(curr) )
+ if ( curr->domain->is_shutting_down || !vcpu_ioreq_pending(curr) )
return false;
break;
diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 62a4b33..02ad9db 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -153,7 +153,7 @@ static int hvm_map_ioreq_gfn(struct ioreq_server *s, bool buf)
{
/*
* If a page has already been allocated (which will happen on
- * demand if hvm_get_ioreq_server_frame() is called), then
+ * demand if ioreq_server_get_frame() is called), then
* mapping a guest frame is not permitted.
*/
if ( gfn_eq(iorp->gfn, INVALID_GFN) )
diff --git a/xen/arch/x86/hvm/stdvga.c b/xen/arch/x86/hvm/stdvga.c
index ee13449..ab9781d 100644
--- a/xen/arch/x86/hvm/stdvga.c
+++ b/xen/arch/x86/hvm/stdvga.c
@@ -507,11 +507,11 @@ static int stdvga_mem_write(const struct hvm_io_handler *handler,
}
done:
- srv = hvm_select_ioreq_server(current->domain, &p);
+ srv = ioreq_server_select(current->domain, &p);
if ( !srv )
return X86EMUL_UNHANDLEABLE;
- return hvm_send_ioreq(srv, &p, 1);
+ return ioreq_send(srv, &p, 1);
}
static bool_t stdvga_mem_accept(const struct hvm_io_handler *handler,
diff --git a/xen/arch/x86/hvm/vmx/vvmx.c b/xen/arch/x86/hvm/vmx/vvmx.c
index 0ddb6a4..e9f94da 100644
--- a/xen/arch/x86/hvm/vmx/vvmx.c
+++ b/xen/arch/x86/hvm/vmx/vvmx.c
@@ -1517,7 +1517,7 @@ void nvmx_switch_guest(void)
* don't want to continue as this setup is not implemented nor supported
* as of right now.
*/
- if ( hvm_io_pending(v) )
+ if ( vcpu_ioreq_pending(v) )
return;
/*
* a softirq may interrupt us between a virtual vmentry is
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index ce3ef59..de3066a 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -59,7 +59,7 @@ static struct ioreq_server *get_ioreq_server(const struct domain *d,
* Iterate over all possible ioreq servers.
*
* NOTE: The iteration is backwards such that more recently created
- * ioreq servers are favoured in hvm_select_ioreq_server().
+ * ioreq servers are favoured in ioreq_server_select().
* This is a semantic that previously existed when ioreq servers
* were held in a linked list.
*/
@@ -106,12 +106,12 @@ static struct ioreq_vcpu *get_pending_vcpu(const struct vcpu *v,
return NULL;
}
-bool hvm_io_pending(struct vcpu *v)
+bool vcpu_ioreq_pending(struct vcpu *v)
{
return get_pending_vcpu(v, NULL);
}
-static bool hvm_wait_for_io(struct ioreq_vcpu *sv, ioreq_t *p)
+static bool wait_for_io(struct ioreq_vcpu *sv, ioreq_t *p)
{
unsigned int prev_state = STATE_IOREQ_NONE;
unsigned int state = p->state;
@@ -168,7 +168,7 @@ static bool hvm_wait_for_io(struct ioreq_vcpu *sv, ioreq_t *p)
return true;
}
-bool handle_hvm_io_completion(struct vcpu *v)
+bool vcpu_ioreq_handle_completion(struct vcpu *v)
{
struct domain *d = v->domain;
struct vcpu_io *vio = &v->io;
@@ -183,7 +183,7 @@ bool handle_hvm_io_completion(struct vcpu *v)
}
sv = get_pending_vcpu(v, &s);
- if ( sv && !hvm_wait_for_io(sv, get_ioreq(s, v)) )
+ if ( sv && !wait_for_io(sv, get_ioreq(s, v)) )
return false;
vio->req.state = ioreq_needs_completion(&vio->req) ?
@@ -214,7 +214,7 @@ bool handle_hvm_io_completion(struct vcpu *v)
return true;
}
-static int hvm_alloc_ioreq_mfn(struct ioreq_server *s, bool buf)
+static int ioreq_server_alloc_mfn(struct ioreq_server *s, bool buf)
{
struct ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
struct page_info *page;
@@ -262,7 +262,7 @@ static int hvm_alloc_ioreq_mfn(struct ioreq_server *s, bool buf)
return -ENOMEM;
}
-static void hvm_free_ioreq_mfn(struct ioreq_server *s, bool buf)
+static void ioreq_server_free_mfn(struct ioreq_server *s, bool buf)
{
struct ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
struct page_info *page = iorp->page;
@@ -301,8 +301,8 @@ bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
return found;
}
-static void hvm_update_ioreq_evtchn(struct ioreq_server *s,
- struct ioreq_vcpu *sv)
+static void ioreq_server_update_evtchn(struct ioreq_server *s,
+ struct ioreq_vcpu *sv)
{
ASSERT(spin_is_locked(&s->lock));
@@ -314,8 +314,8 @@ static void hvm_update_ioreq_evtchn(struct ioreq_server *s,
}
}
-static int hvm_ioreq_server_add_vcpu(struct ioreq_server *s,
- struct vcpu *v)
+static int ioreq_server_add_vcpu(struct ioreq_server *s,
+ struct vcpu *v)
{
struct ioreq_vcpu *sv;
int rc;
@@ -350,7 +350,7 @@ static int hvm_ioreq_server_add_vcpu(struct ioreq_server *s,
list_add(&sv->list_entry, &s->ioreq_vcpu_list);
if ( s->enabled )
- hvm_update_ioreq_evtchn(s, sv);
+ ioreq_server_update_evtchn(s, sv);
spin_unlock(&s->lock);
return 0;
@@ -366,8 +366,8 @@ static int hvm_ioreq_server_add_vcpu(struct ioreq_server *s,
return rc;
}
-static void hvm_ioreq_server_remove_vcpu(struct ioreq_server *s,
- struct vcpu *v)
+static void ioreq_server_remove_vcpu(struct ioreq_server *s,
+ struct vcpu *v)
{
struct ioreq_vcpu *sv;
@@ -394,7 +394,7 @@ static void hvm_ioreq_server_remove_vcpu(struct ioreq_server *s,
spin_unlock(&s->lock);
}
-static void hvm_ioreq_server_remove_all_vcpus(struct ioreq_server *s)
+static void ioreq_server_remove_all_vcpus(struct ioreq_server *s)
{
struct ioreq_vcpu *sv, *next;
@@ -420,28 +420,28 @@ static void hvm_ioreq_server_remove_all_vcpus(struct ioreq_server *s)
spin_unlock(&s->lock);
}
-static int hvm_ioreq_server_alloc_pages(struct ioreq_server *s)
+static int ioreq_server_alloc_pages(struct ioreq_server *s)
{
int rc;
- rc = hvm_alloc_ioreq_mfn(s, false);
+ rc = ioreq_server_alloc_mfn(s, false);
if ( !rc && (s->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF) )
- rc = hvm_alloc_ioreq_mfn(s, true);
+ rc = ioreq_server_alloc_mfn(s, true);
if ( rc )
- hvm_free_ioreq_mfn(s, false);
+ ioreq_server_free_mfn(s, false);
return rc;
}
-static void hvm_ioreq_server_free_pages(struct ioreq_server *s)
+static void ioreq_server_free_pages(struct ioreq_server *s)
{
- hvm_free_ioreq_mfn(s, true);
- hvm_free_ioreq_mfn(s, false);
+ ioreq_server_free_mfn(s, true);
+ ioreq_server_free_mfn(s, false);
}
-static void hvm_ioreq_server_free_rangesets(struct ioreq_server *s)
+static void ioreq_server_free_rangesets(struct ioreq_server *s)
{
unsigned int i;
@@ -449,8 +449,8 @@ static void hvm_ioreq_server_free_rangesets(struct ioreq_server *s)
rangeset_destroy(s->range[i]);
}
-static int hvm_ioreq_server_alloc_rangesets(struct ioreq_server *s,
- ioservid_t id)
+static int ioreq_server_alloc_rangesets(struct ioreq_server *s,
+ ioservid_t id)
{
unsigned int i;
int rc;
@@ -482,12 +482,12 @@ static int hvm_ioreq_server_alloc_rangesets(struct ioreq_server *s,
return 0;
fail:
- hvm_ioreq_server_free_rangesets(s);
+ ioreq_server_free_rangesets(s);
return rc;
}
-static void hvm_ioreq_server_enable(struct ioreq_server *s)
+static void ioreq_server_enable(struct ioreq_server *s)
{
struct ioreq_vcpu *sv;
@@ -503,13 +503,13 @@ static void hvm_ioreq_server_enable(struct ioreq_server *s)
list_for_each_entry ( sv,
&s->ioreq_vcpu_list,
list_entry )
- hvm_update_ioreq_evtchn(s, sv);
+ ioreq_server_update_evtchn(s, sv);
done:
spin_unlock(&s->lock);
}
-static void hvm_ioreq_server_disable(struct ioreq_server *s)
+static void ioreq_server_disable(struct ioreq_server *s)
{
spin_lock(&s->lock);
@@ -524,9 +524,9 @@ static void hvm_ioreq_server_disable(struct ioreq_server *s)
spin_unlock(&s->lock);
}
-static int hvm_ioreq_server_init(struct ioreq_server *s,
- struct domain *d, int bufioreq_handling,
- ioservid_t id)
+static int ioreq_server_init(struct ioreq_server *s,
+ struct domain *d, int bufioreq_handling,
+ ioservid_t id)
{
struct domain *currd = current->domain;
struct vcpu *v;
@@ -544,7 +544,7 @@ static int hvm_ioreq_server_init(struct ioreq_server *s,
s->ioreq.gfn = INVALID_GFN;
s->bufioreq.gfn = INVALID_GFN;
- rc = hvm_ioreq_server_alloc_rangesets(s, id);
+ rc = ioreq_server_alloc_rangesets(s, id);
if ( rc )
return rc;
@@ -552,7 +552,7 @@ static int hvm_ioreq_server_init(struct ioreq_server *s,
for_each_vcpu ( d, v )
{
- rc = hvm_ioreq_server_add_vcpu(s, v);
+ rc = ioreq_server_add_vcpu(s, v);
if ( rc )
goto fail_add;
}
@@ -560,23 +560,23 @@ static int hvm_ioreq_server_init(struct ioreq_server *s,
return 0;
fail_add:
- hvm_ioreq_server_remove_all_vcpus(s);
+ ioreq_server_remove_all_vcpus(s);
arch_ioreq_server_unmap_pages(s);
- hvm_ioreq_server_free_rangesets(s);
+ ioreq_server_free_rangesets(s);
put_domain(s->emulator);
return rc;
}
-static void hvm_ioreq_server_deinit(struct ioreq_server *s)
+static void ioreq_server_deinit(struct ioreq_server *s)
{
ASSERT(!s->enabled);
- hvm_ioreq_server_remove_all_vcpus(s);
+ ioreq_server_remove_all_vcpus(s);
/*
* NOTE: It is safe to call both arch_ioreq_server_unmap_pages() and
- * hvm_ioreq_server_free_pages() in that order.
+ * ioreq_server_free_pages() in that order.
* This is because the former will do nothing if the pages
* are not mapped, leaving the page to be freed by the latter.
* However if the pages are mapped then the former will set
@@ -584,9 +584,9 @@ static void hvm_ioreq_server_deinit(struct ioreq_server *s)
* nothing.
*/
arch_ioreq_server_unmap_pages(s);
- hvm_ioreq_server_free_pages(s);
+ ioreq_server_free_pages(s);
- hvm_ioreq_server_free_rangesets(s);
+ ioreq_server_free_rangesets(s);
put_domain(s->emulator);
}
@@ -620,11 +620,11 @@ static int ioreq_server_create(struct domain *d, int bufioreq_handling,
/*
* It is safe to call set_ioreq_server() prior to
- * hvm_ioreq_server_init() since the target domain is paused.
+ * ioreq_server_init() since the target domain is paused.
*/
set_ioreq_server(d, i, s);
- rc = hvm_ioreq_server_init(s, d, bufioreq_handling, i);
+ rc = ioreq_server_init(s, d, bufioreq_handling, i);
if ( rc )
{
set_ioreq_server(d, i, NULL);
@@ -668,13 +668,13 @@ static int ioreq_server_destroy(struct domain *d, ioservid_t id)
arch_ioreq_server_destroy(s);
- hvm_ioreq_server_disable(s);
+ ioreq_server_disable(s);
/*
- * It is safe to call hvm_ioreq_server_deinit() prior to
+ * It is safe to call ioreq_server_deinit() prior to
* set_ioreq_server() since the target domain is paused.
*/
- hvm_ioreq_server_deinit(s);
+ ioreq_server_deinit(s);
set_ioreq_server(d, id, NULL);
domain_unpause(d);
@@ -736,8 +736,8 @@ static int ioreq_server_get_info(struct domain *d, ioservid_t id,
return rc;
}
-int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
- unsigned long idx, mfn_t *mfn)
+int ioreq_server_get_frame(struct domain *d, ioservid_t id,
+ unsigned long idx, mfn_t *mfn)
{
struct ioreq_server *s;
int rc;
@@ -756,7 +756,7 @@ int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
if ( s->emulator != current->domain )
goto out;
- rc = hvm_ioreq_server_alloc_pages(s);
+ rc = ioreq_server_alloc_pages(s);
if ( rc )
goto out;
@@ -955,9 +955,9 @@ static int ioreq_server_set_state(struct domain *d, ioservid_t id,
domain_pause(d);
if ( enabled )
- hvm_ioreq_server_enable(s);
+ ioreq_server_enable(s);
else
- hvm_ioreq_server_disable(s);
+ ioreq_server_disable(s);
domain_unpause(d);
@@ -968,7 +968,7 @@ static int ioreq_server_set_state(struct domain *d, ioservid_t id,
return rc;
}
-int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
+int ioreq_server_add_vcpu_all(struct domain *d, struct vcpu *v)
{
struct ioreq_server *s;
unsigned int id;
@@ -978,7 +978,7 @@ int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
FOR_EACH_IOREQ_SERVER(d, id, s)
{
- rc = hvm_ioreq_server_add_vcpu(s, v);
+ rc = ioreq_server_add_vcpu(s, v);
if ( rc )
goto fail;
}
@@ -995,7 +995,7 @@ int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
if ( !s )
continue;
- hvm_ioreq_server_remove_vcpu(s, v);
+ ioreq_server_remove_vcpu(s, v);
}
spin_unlock_recursive(&d->ioreq_server.lock);
@@ -1003,7 +1003,7 @@ int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
return rc;
}
-void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v)
+void ioreq_server_remove_vcpu_all(struct domain *d, struct vcpu *v)
{
struct ioreq_server *s;
unsigned int id;
@@ -1011,12 +1011,12 @@ void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v)
spin_lock_recursive(&d->ioreq_server.lock);
FOR_EACH_IOREQ_SERVER(d, id, s)
- hvm_ioreq_server_remove_vcpu(s, v);
+ ioreq_server_remove_vcpu(s, v);
spin_unlock_recursive(&d->ioreq_server.lock);
}
-void hvm_destroy_all_ioreq_servers(struct domain *d)
+void ioreq_server_destroy_all(struct domain *d)
{
struct ioreq_server *s;
unsigned int id;
@@ -1030,13 +1030,13 @@ void hvm_destroy_all_ioreq_servers(struct domain *d)
FOR_EACH_IOREQ_SERVER(d, id, s)
{
- hvm_ioreq_server_disable(s);
+ ioreq_server_disable(s);
/*
- * It is safe to call hvm_ioreq_server_deinit() prior to
+ * It is safe to call ioreq_server_deinit() prior to
* set_ioreq_server() since the target domain is being destroyed.
*/
- hvm_ioreq_server_deinit(s);
+ ioreq_server_deinit(s);
set_ioreq_server(d, id, NULL);
xfree(s);
@@ -1045,8 +1045,8 @@ void hvm_destroy_all_ioreq_servers(struct domain *d)
spin_unlock_recursive(&d->ioreq_server.lock);
}
-struct ioreq_server *hvm_select_ioreq_server(struct domain *d,
- ioreq_t *p)
+struct ioreq_server *ioreq_server_select(struct domain *d,
+ ioreq_t *p)
{
struct ioreq_server *s;
uint8_t type;
@@ -1101,7 +1101,7 @@ struct ioreq_server *hvm_select_ioreq_server(struct domain *d,
return NULL;
}
-static int hvm_send_buffered_ioreq(struct ioreq_server *s, ioreq_t *p)
+static int ioreq_send_buffered(struct ioreq_server *s, ioreq_t *p)
{
struct domain *d = current->domain;
struct ioreq_page *iorp;
@@ -1194,8 +1194,8 @@ static int hvm_send_buffered_ioreq(struct ioreq_server *s, ioreq_t *p)
return IOREQ_STATUS_HANDLED;
}
-int hvm_send_ioreq(struct ioreq_server *s, ioreq_t *proto_p,
- bool buffered)
+int ioreq_send(struct ioreq_server *s, ioreq_t *proto_p,
+ bool buffered)
{
struct vcpu *curr = current;
struct domain *d = curr->domain;
@@ -1204,7 +1204,7 @@ int hvm_send_ioreq(struct ioreq_server *s, ioreq_t *proto_p,
ASSERT(s);
if ( buffered )
- return hvm_send_buffered_ioreq(s, proto_p);
+ return ioreq_send_buffered(s, proto_p);
if ( unlikely(!vcpu_start_shutdown_deferral(curr)) )
return IOREQ_STATUS_RETRY;
@@ -1254,7 +1254,7 @@ int hvm_send_ioreq(struct ioreq_server *s, ioreq_t *proto_p,
return IOREQ_STATUS_UNHANDLED;
}
-unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered)
+unsigned int ioreq_broadcast(ioreq_t *p, bool buffered)
{
struct domain *d = current->domain;
struct ioreq_server *s;
@@ -1265,14 +1265,14 @@ unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered)
if ( !s->enabled )
continue;
- if ( hvm_send_ioreq(s, p, buffered) == IOREQ_STATUS_UNHANDLED )
+ if ( ioreq_send(s, p, buffered) == IOREQ_STATUS_UNHANDLED )
failed++;
}
return failed;
}
-void hvm_ioreq_init(struct domain *d)
+void ioreq_domain_init(struct domain *d)
{
spin_lock_init(&d->ioreq_server.lock);
diff --git a/xen/common/memory.c b/xen/common/memory.c
index 2f274a6..8eb05b1 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -1113,7 +1113,7 @@ static int acquire_ioreq_server(struct domain *d,
{
mfn_t mfn;
- rc = hvm_get_ioreq_server_frame(d, id, frame + i, &mfn);
+ rc = ioreq_server_get_frame(d, id, frame + i, &mfn);
if ( rc )
return rc;
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
index eace1d3..0b433e2 100644
--- a/xen/include/xen/ioreq.h
+++ b/xen/include/xen/ioreq.h
@@ -83,26 +83,26 @@ static inline bool ioreq_needs_completion(const ioreq_t *ioreq)
#define HANDLE_BUFIOREQ(s) \
((s)->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF)
-bool hvm_io_pending(struct vcpu *v);
-bool handle_hvm_io_completion(struct vcpu *v);
+bool vcpu_ioreq_pending(struct vcpu *v);
+bool vcpu_ioreq_handle_completion(struct vcpu *v);
bool is_ioreq_server_page(struct domain *d, const struct page_info *page);
-int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
- unsigned long idx, mfn_t *mfn);
+int ioreq_server_get_frame(struct domain *d, ioservid_t id,
+ unsigned long idx, mfn_t *mfn);
int ioreq_server_map_mem_type(struct domain *d, ioservid_t id,
uint32_t type, uint32_t flags);
-int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v);
-void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v);
-void hvm_destroy_all_ioreq_servers(struct domain *d);
+int ioreq_server_add_vcpu_all(struct domain *d, struct vcpu *v);
+void ioreq_server_remove_vcpu_all(struct domain *d, struct vcpu *v);
+void ioreq_server_destroy_all(struct domain *d);
-struct ioreq_server *hvm_select_ioreq_server(struct domain *d,
- ioreq_t *p);
-int hvm_send_ioreq(struct ioreq_server *s, ioreq_t *proto_p,
- bool buffered);
-unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered);
+struct ioreq_server *ioreq_server_select(struct domain *d,
+ ioreq_t *p);
+int ioreq_send(struct ioreq_server *s, ioreq_t *proto_p,
+ bool buffered);
+unsigned int ioreq_broadcast(ioreq_t *p, bool buffered);
-void hvm_ioreq_init(struct domain *d);
+void ioreq_domain_init(struct domain *d);
int ioreq_server_dm_op(struct xen_dm_op *op, struct domain *d, bool *const_op);
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 13/22] xen/ioreq: Use guest_cmpxchg64() instead of cmpxchg()
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (11 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 12/22] xen/ioreq: Remove "hvm" prefixes from involved function names Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features Oleksandr Tyshchenko
` (10 subsequent siblings)
23 siblings, 0 replies; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Stefano Stabellini,
Julien Grall, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
The cmpxchg() in ioreq_send_buffered() operates on memory shared
with the emulator domain (and the target domain if the legacy
interface is used).
In order to be on the safe side we need to switch
to guest_cmpxchg64() to prevent a domain to DoS Xen on Arm.
The point to use 64-bit version of helper is to support Arm32
since the IOREQ code uses cmpxchg() with 64-bit value.
As there is no plan to support the legacy interface on Arm,
we will have a page to be mapped in a single domain at the time,
so we can use s->emulator in guest_cmpxchg64() safely.
Thankfully the only user of the legacy interface is x86 so far
and there is not concern regarding the atomics operations.
Please note, that the legacy interface *must* not be used on Arm
without revisiting the code.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Paul Durrant <paul@xen.org>
CC: Julien Grall <julien.grall@arm.com>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- new patch
Changes V1 -> V2:
- move earlier to avoid breaking arm32 compilation
- add an explanation to commit description and hvm_allow_set_param()
- pass s->emulator
Changes V2 -> V3:
- update patch description
Changes V3 -> V4:
- add Stefano's A-b
- drop comment from arm/hvm.c
Changes V4 -> V5:
- update patch description
- rebase
- add Paul's R-b
---
---
xen/common/ioreq.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index de3066a..07572a5 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -29,6 +29,7 @@
#include <xen/trace.h>
#include <xen/vpci.h>
+#include <asm/guest_atomics.h>
#include <asm/ioreq.h>
#include <public/hvm/ioreq.h>
@@ -1185,7 +1186,7 @@ static int ioreq_send_buffered(struct ioreq_server *s, ioreq_t *p)
new.read_pointer = old.read_pointer - n * IOREQ_BUFFER_SLOT_NUM;
new.write_pointer = old.write_pointer - n * IOREQ_BUFFER_SLOT_NUM;
- cmpxchg(&pg->ptrs.full, old.full, new.full);
+ guest_cmpxchg64(s->emulator, &pg->ptrs.full, old.full, new.full);
}
notify_via_xen_event_channel(d, s->bufioreq_evtchn);
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (12 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 13/22] xen/ioreq: Use guest_cmpxchg64() instead of cmpxchg() Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-25 20:19 ` Stefano Stabellini
` (2 more replies)
2021-01-25 19:08 ` [PATCH V5 15/22] xen/arm: Call vcpu_ioreq_handle_completion() in check_for_vcpu_work() Oleksandr Tyshchenko
` (9 subsequent siblings)
23 siblings, 3 replies; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Julien Grall, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Andrew Cooper, George Dunlap, Ian Jackson,
Jan Beulich, Wei Liu, Oleksandr Tyshchenko
From: Julien Grall <julien.grall@arm.com>
This patch adds basic IOREQ/DM support on Arm. The subsequent
patches will improve functionality and add remaining bits.
The IOREQ/DM features are supposed to be built with IOREQ_SERVER
option enabled, which is disabled by default on Arm for now.
Please note, the "PIO handling" TODO is expected to left unaddressed
for the current series. It is not an big issue for now while Xen
doesn't have support for vPCI on Arm. On Arm64 they are only used
for PCI IO Bar and we would probably want to expose them to emulator
as PIO access to make a DM completely arch-agnostic. So "PIO handling"
should be implemented when we add support for vPCI.
Signed-off-by: Julien Grall <julien.grall@arm.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
***
I admit, I didn't resolve header dependencies completely.
For now, public/hvm/dm_op.h is included by xen/dm.h, but ought to be included
by arch/arm/dm.c. Details here:
https://lore.kernel.org/xen-devel/e0bc7f80-974e-945d-4605-173bd05302af@gmail.com/
***
Changes RFC -> V1:
- was split into:
- arm/ioreq: Introduce arch specific bits for IOREQ/DM features
- xen/mm: Handle properly reference in set_foreign_p2m_entry() on Arm
- update patch description
- update asm-arm/hvm/ioreq.h according to the newly introduced arch functions:
- arch_hvm_destroy_ioreq_server()
- arch_handle_hvm_io_completion()
- update arch files to include xen/ioreq.h
- remove HVMOP plumbing
- rewrite a logic to handle properly case when hvm_send_ioreq() returns IO_RETRY
- add a logic to handle properly handle_hvm_io_completion() return value
- rename handle_mmio() to ioreq_handle_complete_mmio()
- move paging_mark_pfn_dirty() to asm-arm/paging.h
- remove forward declaration for hvm_ioreq_server in asm-arm/paging.h
- move try_fwd_ioserv() to ioreq.c, provide stubs if !CONFIG_IOREQ_SERVER
- do not remove #ifdef CONFIG_IOREQ_SERVER in memory.c for guarding xen/ioreq.h
- use gdprintk in try_fwd_ioserv(), remove unneeded prints
- update list of #include-s
- move has_vpci() to asm-arm/domain.h
- add a comment (TODO) to unimplemented yet handle_pio()
- remove hvm_mmio_first(last)_byte() and hvm_ioreq_(page/vcpu/server) structs
from the arch files, they were already moved to the common code
- remove set_foreign_p2m_entry() changes, they will be properly implemented
in the follow-up patch
- select IOREQ_SERVER for Arm instead of Arm64 in Kconfig
- remove x86's realmode and other unneeded stubs from xen/ioreq.h
- clafify ioreq_t p.df usage in try_fwd_ioserv()
- set ioreq_t p.count to 1 in try_fwd_ioserv()
Changes V1 -> V2:
- was split into:
- arm/ioreq: Introduce arch specific bits for IOREQ/DM features
- xen/arm: Stick around in leave_hypervisor_to_guest until I/O has completed
- update the author of a patch
- update patch description
- move a loop in leave_hypervisor_to_guest() to a separate patch
- set IOREQ_SERVER disabled by default
- remove already clarified /* XXX */
- replace BUG() by ASSERT_UNREACHABLE() in handle_pio()
- remove default case for handling the return value of try_handle_mmio()
- remove struct hvm_domain, enum hvm_io_completion, struct hvm_vcpu_io,
struct hvm_vcpu from asm-arm/domain.h, these are common materials now
- update everything according to the recent changes (IOREQ related function
names don't contain "hvm" prefixes/infixes anymore, IOREQ related fields
are part of common struct vcpu/domain now, etc)
Changes V2 -> V3:
- update patch according the "legacy interface" is x86 specific
- add dummy arch hooks
- remove dummy paging_mark_pfn_dirty()
- don’t include <xen/domain_page.h> in common ioreq.c
- don’t include <public/hvm/ioreq.h> in arch ioreq.h
- remove #define ioreq_params(d, i)
Changes V3 -> V4:
- rebase
- update patch according to the renaming IO_ -> VIO_ (io_ -> vio_)
and misc changes to arch hooks
- update patch according to the IOREQ related dm-op handling changes
- don't include <xen/ioreq.h> from arch header
- make all arch hooks out-of-line
- add a comment above IOREQ_STATUS_* #define-s
Changes V4 -> V5:
- change the placement of ioreq_server_destroy_all() in arm/domain.c
- don't include public/hvm/dm_op.h by asm-arm/domain.h
- include public/hvm/dm_op.h by xen/dm.h
- put arch ioreq.h directly into asm-arm subdir
- remove do_dm_op() in arm/dm.c, this is a common material now
- remove obsolete ioreq_complete_mmio() from asm-arm/ioreq.h
- optimize arch_ioreq_complete_mmio() to not call try_handle_mmio(),
but try_handle_mmio(), use ASSERT_UNREACHABLE() if state is incorrect
- split changes to check_for_vcpu_work() to be squashed with patch #15
---
---
xen/arch/arm/Makefile | 2 +
xen/arch/arm/dm.c | 97 +++++++++++++++++++
xen/arch/arm/domain.c | 9 ++
xen/arch/arm/io.c | 12 ++-
xen/arch/arm/ioreq.c | 215 +++++++++++++++++++++++++++++++++++++++++++
xen/arch/arm/traps.c | 6 ++
xen/include/asm-arm/domain.h | 2 +
xen/include/asm-arm/ioreq.h | 70 ++++++++++++++
xen/include/asm-arm/mmio.h | 1 +
xen/include/xen/dm.h | 2 +
10 files changed, 415 insertions(+), 1 deletion(-)
create mode 100644 xen/arch/arm/dm.c
create mode 100644 xen/arch/arm/ioreq.c
create mode 100644 xen/include/asm-arm/ioreq.h
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 512ffdd..16e6523 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -13,6 +13,7 @@ obj-y += cpuerrata.o
obj-y += cpufeature.o
obj-y += decode.o
obj-y += device.o
+obj-$(CONFIG_IOREQ_SERVER) += dm.o
obj-y += domain.o
obj-y += domain_build.init.o
obj-y += domctl.o
@@ -27,6 +28,7 @@ obj-y += guest_atomics.o
obj-y += guest_walk.o
obj-y += hvm.o
obj-y += io.o
+obj-$(CONFIG_IOREQ_SERVER) += ioreq.o
obj-y += irq.o
obj-y += kernel.init.o
obj-$(CONFIG_LIVEPATCH) += livepatch.o
diff --git a/xen/arch/arm/dm.c b/xen/arch/arm/dm.c
new file mode 100644
index 0000000..f254ed7
--- /dev/null
+++ b/xen/arch/arm/dm.c
@@ -0,0 +1,97 @@
+/*
+ * Copyright (c) 2019 Arm ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/dm.h>
+#include <xen/guest_access.h>
+#include <xen/hypercall.h>
+#include <xen/ioreq.h>
+#include <xen/nospec.h>
+
+int dm_op(const struct dmop_args *op_args)
+{
+ struct domain *d;
+ struct xen_dm_op op;
+ bool const_op = true;
+ long rc;
+ size_t offset;
+
+ static const uint8_t op_size[] = {
+ [XEN_DMOP_create_ioreq_server] = sizeof(struct xen_dm_op_create_ioreq_server),
+ [XEN_DMOP_get_ioreq_server_info] = sizeof(struct xen_dm_op_get_ioreq_server_info),
+ [XEN_DMOP_map_io_range_to_ioreq_server] = sizeof(struct xen_dm_op_ioreq_server_range),
+ [XEN_DMOP_unmap_io_range_from_ioreq_server] = sizeof(struct xen_dm_op_ioreq_server_range),
+ [XEN_DMOP_set_ioreq_server_state] = sizeof(struct xen_dm_op_set_ioreq_server_state),
+ [XEN_DMOP_destroy_ioreq_server] = sizeof(struct xen_dm_op_destroy_ioreq_server),
+ };
+
+ rc = rcu_lock_remote_domain_by_id(op_args->domid, &d);
+ if ( rc )
+ return rc;
+
+ rc = xsm_dm_op(XSM_DM_PRIV, d);
+ if ( rc )
+ goto out;
+
+ offset = offsetof(struct xen_dm_op, u);
+
+ rc = -EFAULT;
+ if ( op_args->buf[0].size < offset )
+ goto out;
+
+ if ( copy_from_guest_offset((void *)&op, op_args->buf[0].h, 0, offset) )
+ goto out;
+
+ if ( op.op >= ARRAY_SIZE(op_size) )
+ {
+ rc = -EOPNOTSUPP;
+ goto out;
+ }
+
+ op.op = array_index_nospec(op.op, ARRAY_SIZE(op_size));
+
+ if ( op_args->buf[0].size < offset + op_size[op.op] )
+ goto out;
+
+ if ( copy_from_guest_offset((void *)&op.u, op_args->buf[0].h, offset,
+ op_size[op.op]) )
+ goto out;
+
+ rc = -EINVAL;
+ if ( op.pad )
+ goto out;
+
+ rc = ioreq_server_dm_op(&op, d, &const_op);
+
+ if ( (!rc || rc == -ERESTART) &&
+ !const_op && copy_to_guest_offset(op_args->buf[0].h, offset,
+ (void *)&op.u, op_size[op.op]) )
+ rc = -EFAULT;
+
+ out:
+ rcu_unlock_domain(d);
+
+ return rc;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 18cafcd..bdd3d3e 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -15,6 +15,7 @@
#include <xen/guest_access.h>
#include <xen/hypercall.h>
#include <xen/init.h>
+#include <xen/ioreq.h>
#include <xen/lib.h>
#include <xen/livepatch.h>
#include <xen/sched.h>
@@ -696,6 +697,10 @@ int arch_domain_create(struct domain *d,
ASSERT(config != NULL);
+#ifdef CONFIG_IOREQ_SERVER
+ ioreq_domain_init(d);
+#endif
+
/* p2m_init relies on some value initialized by the IOMMU subsystem */
if ( (rc = iommu_domain_init(d, config->iommu_opts)) != 0 )
goto fail;
@@ -1009,6 +1014,10 @@ int domain_relinquish_resources(struct domain *d)
*/
domain_vpl011_deinit(d);
+#ifdef CONFIG_IOREQ_SERVER
+ ioreq_server_destroy_all(d);
+#endif
+
PROGRESS(tee):
ret = tee_relinquish_resources(d);
if (ret )
diff --git a/xen/arch/arm/io.c b/xen/arch/arm/io.c
index ae7ef96..7ac0303 100644
--- a/xen/arch/arm/io.c
+++ b/xen/arch/arm/io.c
@@ -16,12 +16,14 @@
* GNU General Public License for more details.
*/
+#include <xen/ioreq.h>
#include <xen/lib.h>
#include <xen/spinlock.h>
#include <xen/sched.h>
#include <xen/sort.h>
#include <asm/cpuerrata.h>
#include <asm/current.h>
+#include <asm/ioreq.h>
#include <asm/mmio.h>
#include "decode.h"
@@ -123,7 +125,15 @@ enum io_state try_handle_mmio(struct cpu_user_regs *regs,
handler = find_mmio_handler(v->domain, info.gpa);
if ( !handler )
- return IO_UNHANDLED;
+ {
+ int rc;
+
+ rc = try_fwd_ioserv(regs, v, &info);
+ if ( rc == IO_HANDLED )
+ return handle_ioserv(regs, v);
+
+ return rc;
+ }
/* All the instructions used on emulated MMIO region should be valid */
if ( !dabt.valid )
diff --git a/xen/arch/arm/ioreq.c b/xen/arch/arm/ioreq.c
new file mode 100644
index 0000000..ffeeb0b
--- /dev/null
+++ b/xen/arch/arm/ioreq.c
@@ -0,0 +1,215 @@
+/*
+ * arm/ioreq.c: hardware virtual machine I/O emulation
+ *
+ * Copyright (c) 2019 Arm ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/domain.h>
+#include <xen/ioreq.h>
+
+#include <asm/traps.h>
+
+#include <public/hvm/ioreq.h>
+
+enum io_state handle_ioserv(struct cpu_user_regs *regs, struct vcpu *v)
+{
+ const union hsr hsr = { .bits = regs->hsr };
+ const struct hsr_dabt dabt = hsr.dabt;
+ /* Code is similar to handle_read */
+ uint8_t size = (1 << dabt.size) * 8;
+ register_t r = v->io.req.data;
+
+ /* We are done with the IO */
+ v->io.req.state = STATE_IOREQ_NONE;
+
+ if ( dabt.write )
+ return IO_HANDLED;
+
+ /*
+ * Sign extend if required.
+ * Note that we expect the read handler to have zeroed the bits
+ * outside the requested access size.
+ */
+ if ( dabt.sign && (r & (1UL << (size - 1))) )
+ {
+ /*
+ * We are relying on register_t using the same as
+ * an unsigned long in order to keep the 32-bit assembly
+ * code smaller.
+ */
+ BUILD_BUG_ON(sizeof(register_t) != sizeof(unsigned long));
+ r |= (~0UL) << size;
+ }
+
+ set_user_reg(regs, dabt.reg, r);
+
+ return IO_HANDLED;
+}
+
+enum io_state try_fwd_ioserv(struct cpu_user_regs *regs,
+ struct vcpu *v, mmio_info_t *info)
+{
+ struct vcpu_io *vio = &v->io;
+ ioreq_t p = {
+ .type = IOREQ_TYPE_COPY,
+ .addr = info->gpa,
+ .size = 1 << info->dabt.size,
+ .count = 1,
+ .dir = !info->dabt.write,
+ /*
+ * On x86, df is used by 'rep' instruction to tell the direction
+ * to iterate (forward or backward).
+ * On Arm, all the accesses to MMIO region will do a single
+ * memory access. So for now, we can safely always set to 0.
+ */
+ .df = 0,
+ .data = get_user_reg(regs, info->dabt.reg),
+ .state = STATE_IOREQ_READY,
+ };
+ struct ioreq_server *s = NULL;
+ enum io_state rc;
+
+ switch ( vio->req.state )
+ {
+ case STATE_IOREQ_NONE:
+ break;
+
+ default:
+ gdprintk(XENLOG_ERR, "wrong state %u\n", vio->req.state);
+ return IO_ABORT;
+ }
+
+ s = ioreq_server_select(v->domain, &p);
+ if ( !s )
+ return IO_UNHANDLED;
+
+ if ( !info->dabt.valid )
+ return IO_ABORT;
+
+ vio->req = p;
+
+ rc = ioreq_send(s, &p, 0);
+ if ( rc != IO_RETRY || v->domain->is_shutting_down )
+ vio->req.state = STATE_IOREQ_NONE;
+ else if ( !ioreq_needs_completion(&vio->req) )
+ rc = IO_HANDLED;
+ else
+ vio->completion = VIO_mmio_completion;
+
+ return rc;
+}
+
+bool arch_ioreq_complete_mmio(void)
+{
+ struct vcpu *v = current;
+ struct cpu_user_regs *regs = guest_cpu_user_regs();
+ const union hsr hsr = { .bits = regs->hsr };
+
+ if ( v->io.req.state != STATE_IORESP_READY )
+ {
+ ASSERT_UNREACHABLE();
+ return false;
+ }
+
+ if ( handle_ioserv(regs, v) == IO_HANDLED )
+ {
+ advance_pc(regs, hsr);
+ return true;
+ }
+
+ return false;
+}
+
+bool arch_vcpu_ioreq_completion(enum vio_completion completion)
+{
+ ASSERT_UNREACHABLE();
+ return true;
+}
+
+/*
+ * The "legacy" mechanism of mapping magic pages for the IOREQ servers
+ * is x86 specific, so the following hooks don't need to be implemented on Arm:
+ * - arch_ioreq_server_map_pages
+ * - arch_ioreq_server_unmap_pages
+ * - arch_ioreq_server_enable
+ * - arch_ioreq_server_disable
+ */
+int arch_ioreq_server_map_pages(struct ioreq_server *s)
+{
+ return -EOPNOTSUPP;
+}
+
+void arch_ioreq_server_unmap_pages(struct ioreq_server *s)
+{
+}
+
+void arch_ioreq_server_enable(struct ioreq_server *s)
+{
+}
+
+void arch_ioreq_server_disable(struct ioreq_server *s)
+{
+}
+
+void arch_ioreq_server_destroy(struct ioreq_server *s)
+{
+}
+
+int arch_ioreq_server_map_mem_type(struct domain *d,
+ struct ioreq_server *s,
+ uint32_t flags)
+{
+ return -EOPNOTSUPP;
+}
+
+void arch_ioreq_server_map_mem_type_completed(struct domain *d,
+ struct ioreq_server *s,
+ uint32_t flags)
+{
+}
+
+bool arch_ioreq_server_destroy_all(struct domain *d)
+{
+ return true;
+}
+
+bool arch_ioreq_server_get_type_addr(const struct domain *d,
+ const ioreq_t *p,
+ uint8_t *type,
+ uint64_t *addr)
+{
+ if ( p->type != IOREQ_TYPE_COPY && p->type != IOREQ_TYPE_PIO )
+ return false;
+
+ *type = (p->type == IOREQ_TYPE_PIO) ?
+ XEN_DMOP_IO_RANGE_PORT : XEN_DMOP_IO_RANGE_MEMORY;
+ *addr = p->addr;
+
+ return true;
+}
+
+void arch_ioreq_domain_init(struct domain *d)
+{
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 1af1bb9..b0cd8f9 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -1385,6 +1385,9 @@ static arm_hypercall_t arm_hypercall_table[] = {
#ifdef CONFIG_HYPFS
HYPERCALL(hypfs_op, 5),
#endif
+#ifdef CONFIG_IOREQ_SERVER
+ HYPERCALL(dm_op, 3),
+#endif
};
#ifndef NDEBUG
@@ -1956,6 +1959,9 @@ static void do_trap_stage2_abort_guest(struct cpu_user_regs *regs,
case IO_HANDLED:
advance_pc(regs, hsr);
return;
+ case IO_RETRY:
+ /* finish later */
+ return;
case IO_UNHANDLED:
/* IO unhandled, try another way to handle it. */
break;
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 6819a3b..1da90f2 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -262,6 +262,8 @@ static inline void arch_vcpu_block(struct vcpu *v) {}
#define arch_vm_assist_valid_mask(d) (1UL << VMASST_TYPE_runstate_update_flag)
+#define has_vpci(d) ({ (void)(d); false; })
+
#endif /* __ASM_DOMAIN_H__ */
/*
diff --git a/xen/include/asm-arm/ioreq.h b/xen/include/asm-arm/ioreq.h
new file mode 100644
index 0000000..5018597
--- /dev/null
+++ b/xen/include/asm-arm/ioreq.h
@@ -0,0 +1,70 @@
+/*
+ * ioreq.h: Hardware virtual machine assist interface definitions.
+ *
+ * Copyright (c) 2016 Citrix Systems Inc.
+ * Copyright (c) 2019 Arm ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ASM_ARM_IOREQ_H__
+#define __ASM_ARM_IOREQ_H__
+
+#ifdef CONFIG_IOREQ_SERVER
+enum io_state handle_ioserv(struct cpu_user_regs *regs, struct vcpu *v);
+enum io_state try_fwd_ioserv(struct cpu_user_regs *regs,
+ struct vcpu *v, mmio_info_t *info);
+#else
+static inline enum io_state handle_ioserv(struct cpu_user_regs *regs,
+ struct vcpu *v)
+{
+ return IO_UNHANDLED;
+}
+
+static inline enum io_state try_fwd_ioserv(struct cpu_user_regs *regs,
+ struct vcpu *v, mmio_info_t *info)
+{
+ return IO_UNHANDLED;
+}
+#endif
+
+static inline bool handle_pio(uint16_t port, unsigned int size, int dir)
+{
+ /*
+ * TODO: For Arm64, the main user will be PCI. So this should be
+ * implemented when we add support for vPCI.
+ */
+ ASSERT_UNREACHABLE();
+ return true;
+}
+
+static inline void msix_write_completion(struct vcpu *v)
+{
+}
+
+/* This correlation must not be altered */
+#define IOREQ_STATUS_HANDLED IO_HANDLED
+#define IOREQ_STATUS_UNHANDLED IO_UNHANDLED
+#define IOREQ_STATUS_RETRY IO_RETRY
+
+#endif /* __ASM_ARM_IOREQ_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/mmio.h b/xen/include/asm-arm/mmio.h
index 8dbfb27..7ab873c 100644
--- a/xen/include/asm-arm/mmio.h
+++ b/xen/include/asm-arm/mmio.h
@@ -37,6 +37,7 @@ enum io_state
IO_ABORT, /* The IO was handled by the helper and led to an abort. */
IO_HANDLED, /* The IO was successfully handled by the helper. */
IO_UNHANDLED, /* The IO was not handled by the helper. */
+ IO_RETRY, /* Retry the emulation for some reason */
};
typedef int (*mmio_read_t)(struct vcpu *v, mmio_info_t *info,
diff --git a/xen/include/xen/dm.h b/xen/include/xen/dm.h
index 4bd7e57..7f9f0bd 100644
--- a/xen/include/xen/dm.h
+++ b/xen/include/xen/dm.h
@@ -19,6 +19,8 @@
#include <xen/sched.h>
+#include <public/hvm/dm_op.h>
+
struct dmop_args {
domid_t domid;
unsigned int nr_bufs;
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 15/22] xen/arm: Call vcpu_ioreq_handle_completion() in check_for_vcpu_work()
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (13 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-27 14:49 ` Julien Grall
2021-01-25 19:08 ` [PATCH V5 16/22] xen/mm: Handle properly reference in set_foreign_p2m_entry() on Arm Oleksandr Tyshchenko
` (8 subsequent siblings)
23 siblings, 1 reply; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
This patch adds remaining bits needed for the IOREQ support on Arm.
Besides just calling vcpu_ioreq_handle_completion() we need to handle
it's return value to make sure that all the vCPU works are done before
we return to the guest (the vcpu_ioreq_handle_completion() may return
false if there is vCPU work to do or IOREQ state is invalid).
For that reason we use an unbounded loop in leave_hypervisor_to_guest().
The worse that can happen here if the vCPU will never run again
(the I/O will never complete). But, in Xen case, if the I/O never
completes then it most likely means that something went horribly
wrong with the Device Emulator. And it is most likely not safe
to continue. So letting the vCPU to spin forever if the I/O never
completes is a safer action than letting it continue and leaving
the guest in unclear state and is the best what we can do for now.
Please note, using this loop we will not spin forever on a pCPU,
preventing any other vCPUs from being scheduled. At every loop
we will call check_for_pcpu_work() that will process pending
softirqs. In case of failure, the guest will crash and the vCPU
will be unscheduled. In normal case, if the rescheduling is necessary
the vCPU will be rescheduled to give place to someone else.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes V1 -> V2:
- new patch, changes were derived from (+ new explanation):
arm/ioreq: Introduce arch specific bits for IOREQ/DM features
Changes V2 -> V3:
- update patch description
Changes V3 -> V4:
- update patch description and comment in code
Changes V4 -> V5:
- add Stefano's R-b
- update patch subject/description and comment in code,
was "xen/arm: Stick around in leave_hypervisor_to_guest until I/O has completed"
- change loop logic a bit
- squash with changes to check_for_vcpu_work() from patch #14
---
---
xen/arch/arm/traps.c | 26 +++++++++++++++++++++++---
1 file changed, 23 insertions(+), 3 deletions(-)
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index b0cd8f9..2039ff5 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -21,6 +21,7 @@
#include <xen/hypercall.h>
#include <xen/init.h>
#include <xen/iocap.h>
+#include <xen/ioreq.h>
#include <xen/irq.h>
#include <xen/lib.h>
#include <xen/mem_access.h>
@@ -2261,12 +2262,23 @@ static void check_for_pcpu_work(void)
* Process pending work for the vCPU. Any call should be fast or
* implement preemption.
*/
-static void check_for_vcpu_work(void)
+static bool check_for_vcpu_work(void)
{
struct vcpu *v = current;
+#ifdef CONFIG_IOREQ_SERVER
+ bool handled;
+
+ local_irq_enable();
+ handled = vcpu_ioreq_handle_completion(v);
+ local_irq_disable();
+
+ if ( !handled )
+ return true;
+#endif
+
if ( likely(!v->arch.need_flush_to_ram) )
- return;
+ return false;
/*
* Give a chance for the pCPU to process work before handling the vCPU
@@ -2277,6 +2289,8 @@ static void check_for_vcpu_work(void)
local_irq_enable();
p2m_flush_vm(v);
local_irq_disable();
+
+ return false;
}
/*
@@ -2289,7 +2303,13 @@ void leave_hypervisor_to_guest(void)
{
local_irq_disable();
- check_for_vcpu_work();
+ /*
+ * check_for_vcpu_work() may return true if there are more work to before
+ * the vCPU can safely resume. This gives us an opportunity to deschedule
+ * the vCPU if needed.
+ */
+ while ( check_for_vcpu_work() )
+ check_for_pcpu_work();
check_for_pcpu_work();
vgic_sync_to_lrs();
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 16/22] xen/mm: Handle properly reference in set_foreign_p2m_entry() on Arm
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (14 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 15/22] xen/arm: Call vcpu_ioreq_handle_completion() in check_for_vcpu_work() Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 17/22] xen/ioreq: Introduce domain_has_ioreq_server() Oleksandr Tyshchenko
` (7 subsequent siblings)
23 siblings, 0 replies; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Andrew Cooper, George Dunlap, Ian Jackson,
Jan Beulich, Wei Liu, Roger Pau Monné,
Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
This patch implements reference counting of foreign entries in
in set_foreign_p2m_entry() on Arm. This is a mandatory action if
we want to run emulator (IOREQ server) in other than dom0 domain,
as we can't trust it to do the right thing if it is not running
in dom0. So we need to grab a reference on the page to avoid it
disappearing.
It is valid to always pass "p2m_map_foreign_rw" type to
guest_physmap_add_entry() since the current and foreign domains
would be always different. A case when they are equal would be
rejected by rcu_lock_remote_domain_by_id(). Besides the similar
comment in the code put a respective ASSERT() to catch incorrect
usage in future.
It was tested with IOREQ feature to confirm that all the pages given
to this function belong to a domain, so we can use the same approach
as for XENMAPSPACE_gmfn_foreign handling in xenmem_add_to_physmap_one().
This involves adding an extra parameter for the foreign domain to
set_foreign_p2m_entry() and a helper to indicate whether the arch
supports the reference counting of foreign entries and the restriction
for the hardware domain in the common code can be skipped for it.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- new patch, was split from:
"[RFC PATCH V1 04/12] xen/arm: Introduce arch specific bits for IOREQ/DM features"
- rewrite a logic to handle properly reference in set_foreign_p2m_entry()
instead of treating foreign entries as p2m_ram_rw
Changes V1 -> V2:
- rebase according to the recent changes to acquire_resource()
- update patch description
- introduce arch_refcounts_p2m()
- add an explanation why p2m_map_foreign_rw is valid
- move set_foreign_p2m_entry() to p2m-common.h
- add const to new parameter
Changes V2 -> V3:
- update patch description
- rename arch_refcounts_p2m() to arch_acquire_resource_check()
- move comment to x86’s arch_acquire_resource_check()
- return rc in Arm's set_foreign_p2m_entry()
- put a respective ASSERT() into Arm's set_foreign_p2m_entry()
Changes V3 -> V4:
- update arch_acquire_resource_check() implementation on x86
and common code which uses it, pass struct domain to the function
- put ASSERT() to x86/Arm set_foreign_p2m_entry()
- use arch_acquire_resource_check() in p2m_add_foreign()
instead of open-coding it
Changes V4 -> V5:
- update x86's arch_acquire_resource_check()
- use single return statement
- update comment
- add Jan's and Julien's R-b, Stefano's A-b
---
---
xen/arch/arm/p2m.c | 26 ++++++++++++++++++++++++++
xen/arch/x86/mm/p2m.c | 9 ++++++---
xen/common/memory.c | 9 ++-------
xen/include/asm-arm/p2m.h | 19 +++++++++----------
xen/include/asm-x86/p2m.h | 14 +++++++++++---
xen/include/xen/p2m-common.h | 4 ++++
6 files changed, 58 insertions(+), 23 deletions(-)
diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index 4eeb867..d41c4fa 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -1380,6 +1380,32 @@ int guest_physmap_remove_page(struct domain *d, gfn_t gfn, mfn_t mfn,
return p2m_remove_mapping(d, gfn, (1 << page_order), mfn);
}
+int set_foreign_p2m_entry(struct domain *d, const struct domain *fd,
+ unsigned long gfn, mfn_t mfn)
+{
+ struct page_info *page = mfn_to_page(mfn);
+ int rc;
+
+ ASSERT(arch_acquire_resource_check(d));
+
+ if ( !get_page(page, fd) )
+ return -EINVAL;
+
+ /*
+ * It is valid to always use p2m_map_foreign_rw here as if this gets
+ * called then d != fd. A case when d == fd would be rejected by
+ * rcu_lock_remote_domain_by_id() earlier. Put a respective ASSERT()
+ * to catch incorrect usage in future.
+ */
+ ASSERT(d != fd);
+
+ rc = guest_physmap_add_entry(d, _gfn(gfn), mfn, 0, p2m_map_foreign_rw);
+ if ( rc )
+ put_page(page);
+
+ return rc;
+}
+
static struct page_info *p2m_allocate_root(void)
{
struct page_info *page;
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index c1dd45b..2091aed 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -1323,8 +1323,11 @@ static int set_typed_p2m_entry(struct domain *d, unsigned long gfn_l,
}
/* Set foreign mfn in the given guest's p2m table. */
-int set_foreign_p2m_entry(struct domain *d, unsigned long gfn, mfn_t mfn)
+int set_foreign_p2m_entry(struct domain *d, const struct domain *fd,
+ unsigned long gfn, mfn_t mfn)
{
+ ASSERT(arch_acquire_resource_check(d));
+
return set_typed_p2m_entry(d, gfn, mfn, PAGE_ORDER_4K, p2m_map_foreign,
p2m_get_hostp2m(d)->default_access);
}
@@ -2587,7 +2590,7 @@ static int p2m_add_foreign(struct domain *tdom, unsigned long fgfn,
* hvm fixme: until support is added to p2m teardown code to cleanup any
* foreign entries, limit this to hardware domain only.
*/
- if ( !is_hardware_domain(tdom) )
+ if ( !arch_acquire_resource_check(tdom) )
return -EPERM;
if ( foreigndom == DOMID_XEN )
@@ -2643,7 +2646,7 @@ static int p2m_add_foreign(struct domain *tdom, unsigned long fgfn,
* will update the m2p table which will result in mfn -> gpfn of dom0
* and not fgfn of domU.
*/
- rc = set_foreign_p2m_entry(tdom, gpfn, mfn);
+ rc = set_foreign_p2m_entry(tdom, fdom, gpfn, mfn);
if ( rc )
gdprintk(XENLOG_WARNING, "set_foreign_p2m_entry failed. "
"gpfn:%lx mfn:%lx fgfn:%lx td:%d fd:%d\n",
diff --git a/xen/common/memory.c b/xen/common/memory.c
index 8eb05b1..33296e6 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -1139,12 +1139,7 @@ static int acquire_resource(
xen_pfn_t mfn_list[32];
int rc;
- /*
- * FIXME: Until foreign pages inserted into the P2M are properly
- * reference counted, it is unsafe to allow mapping of
- * resource pages unless the caller is the hardware domain.
- */
- if ( paging_mode_translate(currd) && !is_hardware_domain(currd) )
+ if ( !arch_acquire_resource_check(currd) )
return -EACCES;
if ( copy_from_guest(&xmar, arg, 1) )
@@ -1212,7 +1207,7 @@ static int acquire_resource(
for ( i = 0; !rc && i < xmar.nr_frames; i++ )
{
- rc = set_foreign_p2m_entry(currd, gfn_list[i],
+ rc = set_foreign_p2m_entry(currd, d, gfn_list[i],
_mfn(mfn_list[i]));
/* rc should be -EIO for any iteration other than the first */
if ( rc && i )
diff --git a/xen/include/asm-arm/p2m.h b/xen/include/asm-arm/p2m.h
index 28ca9a8..4f8b3b0 100644
--- a/xen/include/asm-arm/p2m.h
+++ b/xen/include/asm-arm/p2m.h
@@ -161,6 +161,15 @@ typedef enum {
#endif
#include <xen/p2m-common.h>
+static inline bool arch_acquire_resource_check(struct domain *d)
+{
+ /*
+ * The reference counting of foreign entries in set_foreign_p2m_entry()
+ * is supported on Arm.
+ */
+ return true;
+}
+
static inline
void p2m_altp2m_check(struct vcpu *v, uint16_t idx)
{
@@ -392,16 +401,6 @@ static inline gfn_t gfn_next_boundary(gfn_t gfn, unsigned int order)
return gfn_add(gfn, 1UL << order);
}
-static inline int set_foreign_p2m_entry(struct domain *d, unsigned long gfn,
- mfn_t mfn)
-{
- /*
- * NOTE: If this is implemented then proper reference counting of
- * foreign entries will need to be implemented.
- */
- return -EOPNOTSUPP;
-}
-
/*
* A vCPU has cache enabled only when the MMU is enabled and data cache
* is enabled.
diff --git a/xen/include/asm-x86/p2m.h b/xen/include/asm-x86/p2m.h
index 5d7836d..7d63f57 100644
--- a/xen/include/asm-x86/p2m.h
+++ b/xen/include/asm-x86/p2m.h
@@ -382,6 +382,17 @@ struct p2m_domain {
#endif
#include <xen/p2m-common.h>
+static inline bool arch_acquire_resource_check(struct domain *d)
+{
+ /*
+ * FIXME: Until foreign pages inserted into the P2M are properly
+ * reference counted, it is unsafe to allow mapping of
+ * resource pages unless the caller is the hardware domain
+ * (see set_foreign_p2m_entry()).
+ */
+ return !paging_mode_translate(d) || is_hardware_domain(d);
+}
+
/*
* Updates vCPU's n2pm to match its np2m_base in VMCx12 and returns that np2m.
*/
@@ -647,9 +658,6 @@ int p2m_finish_type_change(struct domain *d,
int p2m_is_logdirty_range(struct p2m_domain *, unsigned long start,
unsigned long end);
-/* Set foreign entry in the p2m table (for priv-mapping) */
-int set_foreign_p2m_entry(struct domain *d, unsigned long gfn, mfn_t mfn);
-
/* Set mmio addresses in the p2m table (for pass-through) */
int set_mmio_p2m_entry(struct domain *d, gfn_t gfn, mfn_t mfn,
unsigned int order);
diff --git a/xen/include/xen/p2m-common.h b/xen/include/xen/p2m-common.h
index 3753bc0..a322e73 100644
--- a/xen/include/xen/p2m-common.h
+++ b/xen/include/xen/p2m-common.h
@@ -3,6 +3,10 @@
#include <xen/mm-frame.h>
+/* Set foreign entry in the p2m table */
+int set_foreign_p2m_entry(struct domain *d, const struct domain *fd,
+ unsigned long gfn, mfn_t mfn);
+
/* Remove a page from a domain's p2m table */
int __must_check
guest_physmap_remove_page(struct domain *d, gfn_t gfn, mfn_t mfn,
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 17/22] xen/ioreq: Introduce domain_has_ioreq_server()
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (15 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 16/22] xen/mm: Handle properly reference in set_foreign_p2m_entry() on Arm Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 18/22] xen/dm: Introduce xendevicemodel_set_irq_level DM op Oleksandr Tyshchenko
` (6 subsequent siblings)
23 siblings, 0 replies; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Paul Durrant, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
This patch introduces a helper the main purpose of which is to check
if a domain is using IOREQ server(s).
On Arm the current benefit is to avoid calling vcpu_ioreq_handle_completion()
(which implies iterating over all possible IOREQ servers anyway)
on every return in leave_hypervisor_to_guest() if there is no active
servers for the particular domain.
Also this helper will be used by one of the subsequent patches on Arm.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Paul Durrant <paul@xen.org>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- new patch
Changes V1 -> V2:
- update patch description
- guard helper with CONFIG_IOREQ_SERVER
- remove "hvm" prefix
- modify helper to just return d->arch.hvm.ioreq_server.nr_servers
- put suitable ASSERT()s
- use ASSERT(d->ioreq_server.server[id] ? !s : !!s) in set_ioreq_server()
- remove d->ioreq_server.nr_servers = 0 from hvm_ioreq_init()
Changes V2 -> V3:
- update patch description
- remove ASSERT()s from the helper, add a comment
- use #ifdef CONFIG_IOREQ_SERVER inside function body
- use new ASSERT() construction in set_ioreq_server()
Changes V3 -> V4:
- update patch description
- drop per-domain variable "nr_servers"
- reimplement a helper to count the non-NULL entries
- make the helper out-of-line
Changes V4 -> V5:
- add Stefano's and Paul's R-b
---
---
xen/arch/arm/traps.c | 15 +++++++++------
xen/common/ioreq.c | 16 ++++++++++++++++
xen/include/xen/ioreq.h | 2 ++
3 files changed, 27 insertions(+), 6 deletions(-)
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 2039ff5..4cdd343 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -2267,14 +2267,17 @@ static bool check_for_vcpu_work(void)
struct vcpu *v = current;
#ifdef CONFIG_IOREQ_SERVER
- bool handled;
+ if ( domain_has_ioreq_server(v->domain) )
+ {
+ bool handled;
- local_irq_enable();
- handled = vcpu_ioreq_handle_completion(v);
- local_irq_disable();
+ local_irq_enable();
+ handled = vcpu_ioreq_handle_completion(v);
+ local_irq_disable();
- if ( !handled )
- return true;
+ if ( !handled )
+ return true;
+ }
#endif
if ( likely(!v->arch.need_flush_to_ram) )
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index 07572a5..5b0f03e 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -80,6 +80,22 @@ static ioreq_t *get_ioreq(struct ioreq_server *s, struct vcpu *v)
return &p->vcpu_ioreq[v->vcpu_id];
}
+/*
+ * This should only be used when d == current->domain or when they're
+ * distinct and d is paused. Otherwise the result is stale before
+ * the caller can inspect it.
+ */
+bool domain_has_ioreq_server(const struct domain *d)
+{
+ const struct ioreq_server *s;
+ unsigned int id;
+
+ FOR_EACH_IOREQ_SERVER(d, id, s)
+ return true;
+
+ return false;
+}
+
static struct ioreq_vcpu *get_pending_vcpu(const struct vcpu *v,
struct ioreq_server **srvp)
{
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
index 0b433e2..89ee171 100644
--- a/xen/include/xen/ioreq.h
+++ b/xen/include/xen/ioreq.h
@@ -83,6 +83,8 @@ static inline bool ioreq_needs_completion(const ioreq_t *ioreq)
#define HANDLE_BUFIOREQ(s) \
((s)->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF)
+bool domain_has_ioreq_server(const struct domain *d);
+
bool vcpu_ioreq_pending(struct vcpu *v);
bool vcpu_ioreq_handle_completion(struct vcpu *v);
bool is_ioreq_server_page(struct domain *d, const struct page_info *page);
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 18/22] xen/dm: Introduce xendevicemodel_set_irq_level DM op
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (16 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 17/22] xen/ioreq: Introduce domain_has_ioreq_server() Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 19/22] xen/arm: io: Abstract sign-extension Oleksandr Tyshchenko
` (5 subsequent siblings)
23 siblings, 0 replies; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Ian Jackson, Wei Liu, Andrew Cooper,
George Dunlap, Jan Beulich, Julien Grall, Stefano Stabellini,
Volodymyr Babchuk, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
This patch adds ability to the device emulator to notify otherend
(some entity running in the guest) using a SPI and implements Arm
specific bits for it. Proposed interface allows emulator to set
the logical level of a one of a domain's IRQ lines.
We can't reuse the existing DM op (xen_dm_op_set_isa_irq_level)
to inject an interrupt as the "isa_irq" field is only 8-bit and
able to cover IRQ 0 - 255, whereas we need a wider range (0 - 1020).
Please note, for egde-triggered interrupt (which is used for
the virtio-mmio emulation) we only trigger the interrupt on Arm
if the level is asserted (rising edge) and do nothing if the level
is deasserted (falling edge), so the call could be named "trigger_irq"
(without the level parameter). But, in order to model the line closely
(to be able to support level-triggered interrupt) we need to know whether
the line is low or high, so the proposed interface has been chosen.
However, it is worth mentioning that in case of the level-triggered
interrupt, we should keep injecting the interrupt to the guest until
the line is deasserted (this is not covered by current patch).
Signed-off-by: Julien Grall <julien.grall@arm.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- check incoming parameters in arch_dm_op()
- add explicit padding to struct xen_dm_op_set_irq_level
Changes V1 -> V2:
- update the author of a patch
- update patch description
- check that padding is always 0
- mention that interface is Arm only and only SPIs are
supported for now
- allow to set the logical level of a line for non-allocated
interrupts only
- add xen_dm_op_set_irq_level_t
Changes V2 -> V3:
- no changes
Changes V3 -> V4:
- update patch description
- update patch according to the IOREQ related dm-op handling changes
Changes V4 -> V5:
- rebase
- add Stefano-s A-b
---
---
tools/include/xendevicemodel.h | 4 +++
tools/libs/devicemodel/core.c | 18 ++++++++++
tools/libs/devicemodel/libxendevicemodel.map | 1 +
xen/arch/arm/dm.c | 54 +++++++++++++++++++++++++++-
xen/include/public/hvm/dm_op.h | 16 +++++++++
5 files changed, 92 insertions(+), 1 deletion(-)
diff --git a/tools/include/xendevicemodel.h b/tools/include/xendevicemodel.h
index e877f5c..c06b3c8 100644
--- a/tools/include/xendevicemodel.h
+++ b/tools/include/xendevicemodel.h
@@ -209,6 +209,10 @@ int xendevicemodel_set_isa_irq_level(
xendevicemodel_handle *dmod, domid_t domid, uint8_t irq,
unsigned int level);
+int xendevicemodel_set_irq_level(
+ xendevicemodel_handle *dmod, domid_t domid, unsigned int irq,
+ unsigned int level);
+
/**
* This function maps a PCI INTx line to a an IRQ line.
*
diff --git a/tools/libs/devicemodel/core.c b/tools/libs/devicemodel/core.c
index 4d40639..30bd79f 100644
--- a/tools/libs/devicemodel/core.c
+++ b/tools/libs/devicemodel/core.c
@@ -430,6 +430,24 @@ int xendevicemodel_set_isa_irq_level(
return xendevicemodel_op(dmod, domid, 1, &op, sizeof(op));
}
+int xendevicemodel_set_irq_level(
+ xendevicemodel_handle *dmod, domid_t domid, uint32_t irq,
+ unsigned int level)
+{
+ struct xen_dm_op op;
+ struct xen_dm_op_set_irq_level *data;
+
+ memset(&op, 0, sizeof(op));
+
+ op.op = XEN_DMOP_set_irq_level;
+ data = &op.u.set_irq_level;
+
+ data->irq = irq;
+ data->level = level;
+
+ return xendevicemodel_op(dmod, domid, 1, &op, sizeof(op));
+}
+
int xendevicemodel_set_pci_link_route(
xendevicemodel_handle *dmod, domid_t domid, uint8_t link, uint8_t irq)
{
diff --git a/tools/libs/devicemodel/libxendevicemodel.map b/tools/libs/devicemodel/libxendevicemodel.map
index 561c62d..a0c3012 100644
--- a/tools/libs/devicemodel/libxendevicemodel.map
+++ b/tools/libs/devicemodel/libxendevicemodel.map
@@ -32,6 +32,7 @@ VERS_1.2 {
global:
xendevicemodel_relocate_memory;
xendevicemodel_pin_memory_cacheattr;
+ xendevicemodel_set_irq_level;
} VERS_1.1;
VERS_1.3 {
diff --git a/xen/arch/arm/dm.c b/xen/arch/arm/dm.c
index f254ed7..7854133 100644
--- a/xen/arch/arm/dm.c
+++ b/xen/arch/arm/dm.c
@@ -20,6 +20,8 @@
#include <xen/ioreq.h>
#include <xen/nospec.h>
+#include <asm/vgic.h>
+
int dm_op(const struct dmop_args *op_args)
{
struct domain *d;
@@ -35,6 +37,7 @@ int dm_op(const struct dmop_args *op_args)
[XEN_DMOP_unmap_io_range_from_ioreq_server] = sizeof(struct xen_dm_op_ioreq_server_range),
[XEN_DMOP_set_ioreq_server_state] = sizeof(struct xen_dm_op_set_ioreq_server_state),
[XEN_DMOP_destroy_ioreq_server] = sizeof(struct xen_dm_op_destroy_ioreq_server),
+ [XEN_DMOP_set_irq_level] = sizeof(struct xen_dm_op_set_irq_level),
};
rc = rcu_lock_remote_domain_by_id(op_args->domid, &d);
@@ -73,7 +76,56 @@ int dm_op(const struct dmop_args *op_args)
if ( op.pad )
goto out;
- rc = ioreq_server_dm_op(&op, d, &const_op);
+ switch ( op.op )
+ {
+ case XEN_DMOP_set_irq_level:
+ {
+ const struct xen_dm_op_set_irq_level *data =
+ &op.u.set_irq_level;
+ unsigned int i;
+
+ /* Only SPIs are supported */
+ if ( (data->irq < NR_LOCAL_IRQS) || (data->irq >= vgic_num_irqs(d)) )
+ {
+ rc = -EINVAL;
+ break;
+ }
+
+ if ( data->level != 0 && data->level != 1 )
+ {
+ rc = -EINVAL;
+ break;
+ }
+
+ /* Check that padding is always 0 */
+ for ( i = 0; i < sizeof(data->pad); i++ )
+ {
+ if ( data->pad[i] )
+ {
+ rc = -EINVAL;
+ break;
+ }
+ }
+
+ /*
+ * Allow to set the logical level of a line for non-allocated
+ * interrupts only.
+ */
+ if ( test_bit(data->irq, d->arch.vgic.allocated_irqs) )
+ {
+ rc = -EINVAL;
+ break;
+ }
+
+ vgic_inject_irq(d, NULL, data->irq, data->level);
+ rc = 0;
+ break;
+ }
+
+ default:
+ rc = ioreq_server_dm_op(&op, d, &const_op);
+ break;
+ }
if ( (!rc || rc == -ERESTART) &&
!const_op && copy_to_guest_offset(op_args->buf[0].h, offset,
diff --git a/xen/include/public/hvm/dm_op.h b/xen/include/public/hvm/dm_op.h
index 66cae1a..1f70d58 100644
--- a/xen/include/public/hvm/dm_op.h
+++ b/xen/include/public/hvm/dm_op.h
@@ -434,6 +434,21 @@ struct xen_dm_op_pin_memory_cacheattr {
};
typedef struct xen_dm_op_pin_memory_cacheattr xen_dm_op_pin_memory_cacheattr_t;
+/*
+ * XEN_DMOP_set_irq_level: Set the logical level of a one of a domain's
+ * IRQ lines (currently Arm only).
+ * Only SPIs are supported.
+ */
+#define XEN_DMOP_set_irq_level 19
+
+struct xen_dm_op_set_irq_level {
+ uint32_t irq;
+ /* IN - Level: 0 -> deasserted, 1 -> asserted */
+ uint8_t level;
+ uint8_t pad[3];
+};
+typedef struct xen_dm_op_set_irq_level xen_dm_op_set_irq_level_t;
+
struct xen_dm_op {
uint32_t op;
uint32_t pad;
@@ -447,6 +462,7 @@ struct xen_dm_op {
xen_dm_op_track_dirty_vram_t track_dirty_vram;
xen_dm_op_set_pci_intx_level_t set_pci_intx_level;
xen_dm_op_set_isa_irq_level_t set_isa_irq_level;
+ xen_dm_op_set_irq_level_t set_irq_level;
xen_dm_op_set_pci_link_route_t set_pci_link_route;
xen_dm_op_modified_memory_t modified_memory;
xen_dm_op_set_mem_type_t set_mem_type;
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 19/22] xen/arm: io: Abstract sign-extension
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (17 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 18/22] xen/dm: Introduce xendevicemodel_set_irq_level DM op Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 20/22] xen/arm: io: Harden sign extension check Oleksandr Tyshchenko
` (4 subsequent siblings)
23 siblings, 0 replies; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
In order to avoid code duplication (both handle_read() and
handle_ioserv() contain the same code for the sign-extension)
put this code to a common helper to be used for both.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes V1 -> V2:
- new patch
Changes V2 -> V3:
- no changes
Changes V3 -> V4:
- no changes here, but in new patch:
"xen/arm: io: Harden sign extension check"
Changes V4 -> V5:
- add Stefano-s R-b
---
---
xen/arch/arm/io.c | 18 ++----------------
xen/arch/arm/ioreq.c | 17 +----------------
xen/include/asm-arm/traps.h | 24 ++++++++++++++++++++++++
3 files changed, 27 insertions(+), 32 deletions(-)
diff --git a/xen/arch/arm/io.c b/xen/arch/arm/io.c
index 7ac0303..729287e 100644
--- a/xen/arch/arm/io.c
+++ b/xen/arch/arm/io.c
@@ -25,6 +25,7 @@
#include <asm/current.h>
#include <asm/ioreq.h>
#include <asm/mmio.h>
+#include <asm/traps.h>
#include "decode.h"
@@ -40,26 +41,11 @@ static enum io_state handle_read(const struct mmio_handler *handler,
* setting r).
*/
register_t r = 0;
- uint8_t size = (1 << dabt.size) * 8;
if ( !handler->ops->read(v, info, &r, handler->priv) )
return IO_ABORT;
- /*
- * Sign extend if required.
- * Note that we expect the read handler to have zeroed the bits
- * outside the requested access size.
- */
- if ( dabt.sign && (r & (1UL << (size - 1))) )
- {
- /*
- * We are relying on register_t using the same as
- * an unsigned long in order to keep the 32-bit assembly
- * code smaller.
- */
- BUILD_BUG_ON(sizeof(register_t) != sizeof(unsigned long));
- r |= (~0UL) << size;
- }
+ r = sign_extend(dabt, r);
set_user_reg(regs, dabt.reg, r);
diff --git a/xen/arch/arm/ioreq.c b/xen/arch/arm/ioreq.c
index ffeeb0b..c7ee1a7 100644
--- a/xen/arch/arm/ioreq.c
+++ b/xen/arch/arm/ioreq.c
@@ -28,7 +28,6 @@ enum io_state handle_ioserv(struct cpu_user_regs *regs, struct vcpu *v)
const union hsr hsr = { .bits = regs->hsr };
const struct hsr_dabt dabt = hsr.dabt;
/* Code is similar to handle_read */
- uint8_t size = (1 << dabt.size) * 8;
register_t r = v->io.req.data;
/* We are done with the IO */
@@ -37,21 +36,7 @@ enum io_state handle_ioserv(struct cpu_user_regs *regs, struct vcpu *v)
if ( dabt.write )
return IO_HANDLED;
- /*
- * Sign extend if required.
- * Note that we expect the read handler to have zeroed the bits
- * outside the requested access size.
- */
- if ( dabt.sign && (r & (1UL << (size - 1))) )
- {
- /*
- * We are relying on register_t using the same as
- * an unsigned long in order to keep the 32-bit assembly
- * code smaller.
- */
- BUILD_BUG_ON(sizeof(register_t) != sizeof(unsigned long));
- r |= (~0UL) << size;
- }
+ r = sign_extend(dabt, r);
set_user_reg(regs, dabt.reg, r);
diff --git a/xen/include/asm-arm/traps.h b/xen/include/asm-arm/traps.h
index c4a3d0f..c6b3cc7 100644
--- a/xen/include/asm-arm/traps.h
+++ b/xen/include/asm-arm/traps.h
@@ -84,6 +84,30 @@ static inline bool VABORT_GEN_BY_GUEST(const struct cpu_user_regs *regs)
(unsigned long)abort_guest_exit_end == regs->pc;
}
+/* Check whether the sign extension is required and perform it */
+static inline register_t sign_extend(const struct hsr_dabt dabt, register_t r)
+{
+ uint8_t size = (1 << dabt.size) * 8;
+
+ /*
+ * Sign extend if required.
+ * Note that we expect the read handler to have zeroed the bits
+ * outside the requested access size.
+ */
+ if ( dabt.sign && (r & (1UL << (size - 1))) )
+ {
+ /*
+ * We are relying on register_t using the same as
+ * an unsigned long in order to keep the 32-bit assembly
+ * code smaller.
+ */
+ BUILD_BUG_ON(sizeof(register_t) != sizeof(unsigned long));
+ r |= (~0UL) << size;
+ }
+
+ return r;
+}
+
#endif /* __ASM_ARM_TRAPS__ */
/*
* Local variables:
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 20/22] xen/arm: io: Harden sign extension check
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (18 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 19/22] xen/arm: io: Abstract sign-extension Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 21/22] xen/ioreq: Make x86's send_invalidate_req() common Oleksandr Tyshchenko
` (3 subsequent siblings)
23 siblings, 0 replies; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
In the ideal world we would never get an undefined behavior when
propagating the sign bit since that bit can only be set for access
size smaller than the register size (i.e byte/half-word for aarch32,
byte/half-word/word for aarch64).
In the real world we need to care for *possible* hardware bug such as
advertising a sign extension for either 64-bit (or 32-bit) on Arm64
(resp. Arm32).
So harden a bit more the code to prevent undefined behavior when
propagating the sign bit in case of buggy hardware.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
CC: Julien Grall <julien.grall@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes V3 -> V4:
- new patch
Changes V4 -> V5:
- add Stefano's and Volodymyr's R-b
---
---
xen/include/asm-arm/traps.h | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/xen/include/asm-arm/traps.h b/xen/include/asm-arm/traps.h
index c6b3cc7..2ed2b85 100644
--- a/xen/include/asm-arm/traps.h
+++ b/xen/include/asm-arm/traps.h
@@ -94,7 +94,8 @@ static inline register_t sign_extend(const struct hsr_dabt dabt, register_t r)
* Note that we expect the read handler to have zeroed the bits
* outside the requested access size.
*/
- if ( dabt.sign && (r & (1UL << (size - 1))) )
+ if ( dabt.sign && (size < sizeof(register_t) * 8) &&
+ (r & (1UL << (size - 1))) )
{
/*
* We are relying on register_t using the same as
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 21/22] xen/ioreq: Make x86's send_invalidate_req() common
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (19 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 20/22] xen/arm: io: Harden sign extension check Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 22/22] xen/arm: Add mapcache invalidation handling Oleksandr Tyshchenko
` (2 subsequent siblings)
23 siblings, 0 replies; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Paul Durrant, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
As the IOREQ is a common feature now and we also need to
invalidate qemu/demu mapcache on Arm when the required condition
occurs this patch moves this function to the common code
(and remames it to ioreq_signal_mapcache_invalidate).
This patch also moves per-domain qemu_mapcache_invalidate
variable out of the arch sub-struct (and drops "qemu" prefix).
We don't put this variable inside the #ifdef CONFIG_IOREQ_SERVER
at the end of struct domain, but in the hole next to the group
of 5 bools further up which is more efficient.
The subsequent patch will add mapcache invalidation handling on Arm.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- move send_invalidate_req() to the common code
- update patch subject/description
- move qemu_mapcache_invalidate out of the arch sub-struct,
update checks
- remove #if defined(CONFIG_ARM64) from the common code
Changes V1 -> V2:
- was split into:
- xen/ioreq: Make x86's send_invalidate_req() common
- xen/arm: Add mapcache invalidation handling
- update patch description/subject
- move Arm bits to a separate patch
- don't alter the common code, the flag is set by arch code
- rename send_invalidate_req() to send_invalidate_ioreq()
- guard qemu_mapcache_invalidate with CONFIG_IOREQ_SERVER
- use bool instead of bool_t
- remove blank line blank line between head comment and #include-s
Changes V2 -> V3:
- update patch description
- drop "qemu" prefix from the variable name
- rename send_invalidate_req() to ioreq_signal_mapcache_invalidate()
Changes V3 -> V4:
- change variable location in struct domain
Changes V4 -> V5:
- add Jan's A-b and Paul's R-b
---
---
xen/arch/x86/hvm/hypercall.c | 9 +++++----
xen/arch/x86/hvm/io.c | 14 --------------
xen/common/ioreq.c | 14 ++++++++++++++
xen/include/asm-x86/hvm/domain.h | 1 -
xen/include/asm-x86/hvm/io.h | 1 -
xen/include/xen/ioreq.h | 1 +
xen/include/xen/sched.h | 5 +++++
7 files changed, 25 insertions(+), 20 deletions(-)
diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c
index ac573c8..6d41c56 100644
--- a/xen/arch/x86/hvm/hypercall.c
+++ b/xen/arch/x86/hvm/hypercall.c
@@ -20,6 +20,7 @@
*/
#include <xen/lib.h>
#include <xen/hypercall.h>
+#include <xen/ioreq.h>
#include <xen/nospec.h>
#include <asm/hvm/emulate.h>
@@ -47,7 +48,7 @@ static long hvm_memory_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
rc = compat_memory_op(cmd, arg);
if ( (cmd & MEMOP_CMD_MASK) == XENMEM_decrease_reservation )
- curr->domain->arch.hvm.qemu_mapcache_invalidate = true;
+ curr->domain->mapcache_invalidate = true;
return rc;
}
@@ -326,9 +327,9 @@ int hvm_hypercall(struct cpu_user_regs *regs)
HVM_DBG_LOG(DBG_LEVEL_HCALL, "hcall%lu -> %lx", eax, regs->rax);
- if ( unlikely(currd->arch.hvm.qemu_mapcache_invalidate) &&
- test_and_clear_bool(currd->arch.hvm.qemu_mapcache_invalidate) )
- send_invalidate_req();
+ if ( unlikely(currd->mapcache_invalidate) &&
+ test_and_clear_bool(currd->mapcache_invalidate) )
+ ioreq_signal_mapcache_invalidate();
return curr->hcall_preempted ? HVM_HCALL_preempted : HVM_HCALL_completed;
}
diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
index 66a37ee..046a8eb 100644
--- a/xen/arch/x86/hvm/io.c
+++ b/xen/arch/x86/hvm/io.c
@@ -64,20 +64,6 @@ void send_timeoffset_req(unsigned long timeoff)
gprintk(XENLOG_ERR, "Unsuccessful timeoffset update\n");
}
-/* Ask ioemu mapcache to invalidate mappings. */
-void send_invalidate_req(void)
-{
- ioreq_t p = {
- .type = IOREQ_TYPE_INVALIDATE,
- .size = 4,
- .dir = IOREQ_WRITE,
- .data = ~0UL, /* flush all */
- };
-
- if ( ioreq_broadcast(&p, false) != 0 )
- gprintk(XENLOG_ERR, "Unsuccessful map-cache invalidate\n");
-}
-
bool hvm_emulate_one_insn(hvm_emulate_validate_t *validate, const char *descr)
{
struct hvm_emulate_ctxt ctxt;
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index 5b0f03e..67ef1f7 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -35,6 +35,20 @@
#include <public/hvm/ioreq.h>
#include <public/hvm/params.h>
+/* Ask ioemu mapcache to invalidate mappings. */
+void ioreq_signal_mapcache_invalidate(void)
+{
+ ioreq_t p = {
+ .type = IOREQ_TYPE_INVALIDATE,
+ .size = 4,
+ .dir = IOREQ_WRITE,
+ .data = ~0UL, /* flush all */
+ };
+
+ if ( ioreq_broadcast(&p, false) != 0 )
+ gprintk(XENLOG_ERR, "Unsuccessful map-cache invalidate\n");
+}
+
static void set_ioreq_server(struct domain *d, unsigned int id,
struct ioreq_server *s)
{
diff --git a/xen/include/asm-x86/hvm/domain.h b/xen/include/asm-x86/hvm/domain.h
index b8be1ad..cf959f6 100644
--- a/xen/include/asm-x86/hvm/domain.h
+++ b/xen/include/asm-x86/hvm/domain.h
@@ -122,7 +122,6 @@ struct hvm_domain {
struct viridian_domain *viridian;
- bool_t qemu_mapcache_invalidate;
bool_t is_s3_suspended;
/*
diff --git a/xen/include/asm-x86/hvm/io.h b/xen/include/asm-x86/hvm/io.h
index 6bc80db..5eb4ddb 100644
--- a/xen/include/asm-x86/hvm/io.h
+++ b/xen/include/asm-x86/hvm/io.h
@@ -97,7 +97,6 @@ bool relocate_portio_handler(
unsigned int size);
void send_timeoffset_req(unsigned long timeoff);
-void send_invalidate_req(void);
bool handle_mmio_with_translation(unsigned long gla, unsigned long gpfn,
struct npfec);
bool handle_pio(uint16_t port, unsigned int size, int dir);
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
index 89ee171..2d635e9 100644
--- a/xen/include/xen/ioreq.h
+++ b/xen/include/xen/ioreq.h
@@ -103,6 +103,7 @@ struct ioreq_server *ioreq_server_select(struct domain *d,
int ioreq_send(struct ioreq_server *s, ioreq_t *proto_p,
bool buffered);
unsigned int ioreq_broadcast(ioreq_t *p, bool buffered);
+void ioreq_signal_mapcache_invalidate(void);
void ioreq_domain_init(struct domain *d);
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 59e5b6a..06dba1a 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -444,6 +444,11 @@ struct domain
* unpaused for the first time by the systemcontroller.
*/
bool creation_finished;
+ /*
+ * Indicates that mapcache invalidation request should be sent to
+ * the device emulator.
+ */
+ bool mapcache_invalidate;
/* Which guest this guest has privileges on */
struct domain *target;
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH V5 22/22] xen/arm: Add mapcache invalidation handling
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (20 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 21/22] xen/ioreq: Make x86's send_invalidate_req() common Oleksandr Tyshchenko
@ 2021-01-25 19:08 ` Oleksandr Tyshchenko
2021-01-28 9:55 ` Julien Grall
2021-01-27 16:43 ` [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Julien Grall
2021-01-28 16:11 ` Julien Grall
23 siblings, 1 reply; 93+ messages in thread
From: Oleksandr Tyshchenko @ 2021-01-25 19:08 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
We need to send mapcache invalidation request to qemu/demu everytime
the page gets removed from a guest.
At the moment, the Arm code doesn't explicitely remove the existing
mapping before inserting the new mapping. Instead, this is done
implicitely by __p2m_set_entry().
So we need to recognize a case when old entry is a RAM page *and*
the new MFN is different in order to set the corresponding flag.
The most suitable place to do this is p2m_free_entry(), there
we can find the correct leaf type. The invalidation request
will be sent in do_trap_hypercall() later on.
Taking into the account the following the do_trap_hypercall()
is the best place to send invalidation request:
- The only way a guest can modify its P2M on Arm is via an hypercall
- When sending the invalidation request, the vCPU will be blocked
until all the IOREQ servers have acknowledged the invalidation
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
[On Arm only]
Tested-by: Wei Chen <Wei.Chen@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
***
Please note, this patch depends on the following which is
on review:
https://patchwork.kernel.org/patch/11803383/
This patch is on par with x86 code (whether it is buggy or not).
If there is a need to improve/harden something, this can be done on
a follow-up.
***
Changes V1 -> V2:
- new patch, some changes were derived from (+ new explanation):
xen/ioreq: Make x86's invalidate qemu mapcache handling common
- put setting of the flag into __p2m_set_entry()
- clarify the conditions when the flag should be set
- use domain_has_ioreq_server()
- update do_trap_hypercall() by adding local variable
Changes V2 -> V3:
- update patch description
- move check to p2m_free_entry()
- add a comment
- use "curr" instead of "v" in do_trap_hypercall()
Changes V3 -> V4:
- update patch description
- re-order check in p2m_free_entry() to call domain_has_ioreq_server()
only if p2m->domain == current->domain
- add a comment in do_trap_hypercall()
Changes V4 -> V5:
- add Stefano's R-b
- update comment in do_trap_hypercall()
---
---
xen/arch/arm/p2m.c | 25 +++++++++++++++++--------
xen/arch/arm/traps.c | 20 +++++++++++++++++---
2 files changed, 34 insertions(+), 11 deletions(-)
diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index d41c4fa..26acb95d 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -1,6 +1,7 @@
#include <xen/cpu.h>
#include <xen/domain_page.h>
#include <xen/iocap.h>
+#include <xen/ioreq.h>
#include <xen/lib.h>
#include <xen/sched.h>
#include <xen/softirq.h>
@@ -749,17 +750,25 @@ static void p2m_free_entry(struct p2m_domain *p2m,
if ( !p2m_is_valid(entry) )
return;
- /* Nothing to do but updating the stats if the entry is a super-page. */
- if ( p2m_is_superpage(entry, level) )
+ if ( p2m_is_superpage(entry, level) || (level == 3) )
{
- p2m->stats.mappings[level]--;
- return;
- }
+#ifdef CONFIG_IOREQ_SERVER
+ /*
+ * If this gets called (non-recursively) then either the entry
+ * was replaced by an entry with a different base (valid case) or
+ * the shattering of a superpage was failed (error case).
+ * So, at worst, the spurious mapcache invalidation might be sent.
+ */
+ if ( (p2m->domain == current->domain) &&
+ domain_has_ioreq_server(p2m->domain) &&
+ p2m_is_ram(entry.p2m.type) )
+ p2m->domain->mapcache_invalidate = true;
+#endif
- if ( level == 3 )
- {
p2m->stats.mappings[level]--;
- p2m_put_l3_page(entry);
+ /* Nothing to do if the entry is a super-page. */
+ if ( level == 3 )
+ p2m_put_l3_page(entry);
return;
}
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 4cdd343..64b740b 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -1443,6 +1443,7 @@ static void do_trap_hypercall(struct cpu_user_regs *regs, register_t *nr,
const union hsr hsr)
{
arm_hypercall_fn_t call = NULL;
+ struct vcpu *curr = current;
BUILD_BUG_ON(NR_hypercalls < ARRAY_SIZE(arm_hypercall_table) );
@@ -1459,7 +1460,7 @@ static void do_trap_hypercall(struct cpu_user_regs *regs, register_t *nr,
return;
}
- current->hcall_preempted = false;
+ curr->hcall_preempted = false;
perfc_incra(hypercalls, *nr);
call = arm_hypercall_table[*nr].fn;
@@ -1472,7 +1473,7 @@ static void do_trap_hypercall(struct cpu_user_regs *regs, register_t *nr,
HYPERCALL_RESULT_REG(regs) = call(HYPERCALL_ARGS(regs));
#ifndef NDEBUG
- if ( !current->hcall_preempted )
+ if ( !curr->hcall_preempted )
{
/* Deliberately corrupt parameter regs used by this hypercall. */
switch ( arm_hypercall_table[*nr].nr_args ) {
@@ -1489,8 +1490,21 @@ static void do_trap_hypercall(struct cpu_user_regs *regs, register_t *nr,
#endif
/* Ensure the hypercall trap instruction is re-executed. */
- if ( current->hcall_preempted )
+ if ( curr->hcall_preempted )
regs->pc -= 4; /* re-execute 'hvc #XEN_HYPERCALL_TAG' */
+
+#ifdef CONFIG_IOREQ_SERVER
+ /*
+ * We call ioreq_signal_mapcache_invalidate from do_trap_hypercall()
+ * because the only way a guest can modify its P2M on Arm is via an
+ * hypercall.
+ * Note that sending the invalidation request causes the vCPU to block
+ * until all the IOREQ servers have acknowledged the invalidation.
+ */
+ if ( unlikely(curr->domain->mapcache_invalidate) &&
+ test_and_clear_bool(curr->domain->mapcache_invalidate) )
+ ioreq_signal_mapcache_invalidate();
+#endif
}
void arch_hypercall_tasklet_result(struct vcpu *v, long res)
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* Re: [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2021-01-25 19:08 ` [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features Oleksandr Tyshchenko
@ 2021-01-25 20:19 ` Stefano Stabellini
2021-01-26 9:15 ` Jan Beulich
2021-01-27 18:33 ` Julien Grall
2 siblings, 0 replies; 93+ messages in thread
From: Stefano Stabellini @ 2021-01-25 20:19 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: xen-devel, Julien Grall, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Andrew Cooper, George Dunlap, Ian Jackson,
Jan Beulich, Wei Liu, Oleksandr Tyshchenko
[-- Attachment #1: Type: text/plain, Size: 22717 bytes --]
On Mon, 25 Jan 2021, Oleksandr Tyshchenko wrote:
> From: Julien Grall <julien.grall@arm.com>
>
> This patch adds basic IOREQ/DM support on Arm. The subsequent
> patches will improve functionality and add remaining bits.
>
> The IOREQ/DM features are supposed to be built with IOREQ_SERVER
> option enabled, which is disabled by default on Arm for now.
>
> Please note, the "PIO handling" TODO is expected to left unaddressed
> for the current series. It is not an big issue for now while Xen
> doesn't have support for vPCI on Arm. On Arm64 they are only used
> for PCI IO Bar and we would probably want to expose them to emulator
> as PIO access to make a DM completely arch-agnostic. So "PIO handling"
> should be implemented when we add support for vPCI.
>
> Signed-off-by: Julien Grall <julien.grall@arm.com>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> [On Arm only]
> Tested-by: Wei Chen <Wei.Chen@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> ---
> Please note, this is a split/cleanup/hardening of Julien's PoC:
> "Add support for Guest IO forwarding to a device emulator"
>
> ***
> I admit, I didn't resolve header dependencies completely.
> For now, public/hvm/dm_op.h is included by xen/dm.h, but ought to be included
> by arch/arm/dm.c. Details here:
> https://lore.kernel.org/xen-devel/e0bc7f80-974e-945d-4605-173bd05302af@gmail.com/
> ***
>
> Changes RFC -> V1:
> - was split into:
> - arm/ioreq: Introduce arch specific bits for IOREQ/DM features
> - xen/mm: Handle properly reference in set_foreign_p2m_entry() on Arm
> - update patch description
> - update asm-arm/hvm/ioreq.h according to the newly introduced arch functions:
> - arch_hvm_destroy_ioreq_server()
> - arch_handle_hvm_io_completion()
> - update arch files to include xen/ioreq.h
> - remove HVMOP plumbing
> - rewrite a logic to handle properly case when hvm_send_ioreq() returns IO_RETRY
> - add a logic to handle properly handle_hvm_io_completion() return value
> - rename handle_mmio() to ioreq_handle_complete_mmio()
> - move paging_mark_pfn_dirty() to asm-arm/paging.h
> - remove forward declaration for hvm_ioreq_server in asm-arm/paging.h
> - move try_fwd_ioserv() to ioreq.c, provide stubs if !CONFIG_IOREQ_SERVER
> - do not remove #ifdef CONFIG_IOREQ_SERVER in memory.c for guarding xen/ioreq.h
> - use gdprintk in try_fwd_ioserv(), remove unneeded prints
> - update list of #include-s
> - move has_vpci() to asm-arm/domain.h
> - add a comment (TODO) to unimplemented yet handle_pio()
> - remove hvm_mmio_first(last)_byte() and hvm_ioreq_(page/vcpu/server) structs
> from the arch files, they were already moved to the common code
> - remove set_foreign_p2m_entry() changes, they will be properly implemented
> in the follow-up patch
> - select IOREQ_SERVER for Arm instead of Arm64 in Kconfig
> - remove x86's realmode and other unneeded stubs from xen/ioreq.h
> - clafify ioreq_t p.df usage in try_fwd_ioserv()
> - set ioreq_t p.count to 1 in try_fwd_ioserv()
>
> Changes V1 -> V2:
> - was split into:
> - arm/ioreq: Introduce arch specific bits for IOREQ/DM features
> - xen/arm: Stick around in leave_hypervisor_to_guest until I/O has completed
> - update the author of a patch
> - update patch description
> - move a loop in leave_hypervisor_to_guest() to a separate patch
> - set IOREQ_SERVER disabled by default
> - remove already clarified /* XXX */
> - replace BUG() by ASSERT_UNREACHABLE() in handle_pio()
> - remove default case for handling the return value of try_handle_mmio()
> - remove struct hvm_domain, enum hvm_io_completion, struct hvm_vcpu_io,
> struct hvm_vcpu from asm-arm/domain.h, these are common materials now
> - update everything according to the recent changes (IOREQ related function
> names don't contain "hvm" prefixes/infixes anymore, IOREQ related fields
> are part of common struct vcpu/domain now, etc)
>
> Changes V2 -> V3:
> - update patch according the "legacy interface" is x86 specific
> - add dummy arch hooks
> - remove dummy paging_mark_pfn_dirty()
> - don’t include <xen/domain_page.h> in common ioreq.c
> - don’t include <public/hvm/ioreq.h> in arch ioreq.h
> - remove #define ioreq_params(d, i)
>
> Changes V3 -> V4:
> - rebase
> - update patch according to the renaming IO_ -> VIO_ (io_ -> vio_)
> and misc changes to arch hooks
> - update patch according to the IOREQ related dm-op handling changes
> - don't include <xen/ioreq.h> from arch header
> - make all arch hooks out-of-line
> - add a comment above IOREQ_STATUS_* #define-s
>
> Changes V4 -> V5:
> - change the placement of ioreq_server_destroy_all() in arm/domain.c
> - don't include public/hvm/dm_op.h by asm-arm/domain.h
> - include public/hvm/dm_op.h by xen/dm.h
> - put arch ioreq.h directly into asm-arm subdir
> - remove do_dm_op() in arm/dm.c, this is a common material now
> - remove obsolete ioreq_complete_mmio() from asm-arm/ioreq.h
> - optimize arch_ioreq_complete_mmio() to not call try_handle_mmio(),
> but try_handle_mmio(), use ASSERT_UNREACHABLE() if state is incorrect
> - split changes to check_for_vcpu_work() to be squashed with patch #15
> ---
> ---
> xen/arch/arm/Makefile | 2 +
> xen/arch/arm/dm.c | 97 +++++++++++++++++++
> xen/arch/arm/domain.c | 9 ++
> xen/arch/arm/io.c | 12 ++-
> xen/arch/arm/ioreq.c | 215 +++++++++++++++++++++++++++++++++++++++++++
> xen/arch/arm/traps.c | 6 ++
> xen/include/asm-arm/domain.h | 2 +
> xen/include/asm-arm/ioreq.h | 70 ++++++++++++++
> xen/include/asm-arm/mmio.h | 1 +
> xen/include/xen/dm.h | 2 +
> 10 files changed, 415 insertions(+), 1 deletion(-)
> create mode 100644 xen/arch/arm/dm.c
> create mode 100644 xen/arch/arm/ioreq.c
> create mode 100644 xen/include/asm-arm/ioreq.h
>
> diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
> index 512ffdd..16e6523 100644
> --- a/xen/arch/arm/Makefile
> +++ b/xen/arch/arm/Makefile
> @@ -13,6 +13,7 @@ obj-y += cpuerrata.o
> obj-y += cpufeature.o
> obj-y += decode.o
> obj-y += device.o
> +obj-$(CONFIG_IOREQ_SERVER) += dm.o
> obj-y += domain.o
> obj-y += domain_build.init.o
> obj-y += domctl.o
> @@ -27,6 +28,7 @@ obj-y += guest_atomics.o
> obj-y += guest_walk.o
> obj-y += hvm.o
> obj-y += io.o
> +obj-$(CONFIG_IOREQ_SERVER) += ioreq.o
> obj-y += irq.o
> obj-y += kernel.init.o
> obj-$(CONFIG_LIVEPATCH) += livepatch.o
> diff --git a/xen/arch/arm/dm.c b/xen/arch/arm/dm.c
> new file mode 100644
> index 0000000..f254ed7
> --- /dev/null
> +++ b/xen/arch/arm/dm.c
> @@ -0,0 +1,97 @@
> +/*
> + * Copyright (c) 2019 Arm ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms and conditions of the GNU General Public License,
> + * version 2, as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
> + * more details.
> + *
> + * You should have received a copy of the GNU General Public License along with
> + * this program; If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/dm.h>
> +#include <xen/guest_access.h>
> +#include <xen/hypercall.h>
> +#include <xen/ioreq.h>
> +#include <xen/nospec.h>
> +
> +int dm_op(const struct dmop_args *op_args)
> +{
> + struct domain *d;
> + struct xen_dm_op op;
> + bool const_op = true;
> + long rc;
> + size_t offset;
> +
> + static const uint8_t op_size[] = {
> + [XEN_DMOP_create_ioreq_server] = sizeof(struct xen_dm_op_create_ioreq_server),
> + [XEN_DMOP_get_ioreq_server_info] = sizeof(struct xen_dm_op_get_ioreq_server_info),
> + [XEN_DMOP_map_io_range_to_ioreq_server] = sizeof(struct xen_dm_op_ioreq_server_range),
> + [XEN_DMOP_unmap_io_range_from_ioreq_server] = sizeof(struct xen_dm_op_ioreq_server_range),
> + [XEN_DMOP_set_ioreq_server_state] = sizeof(struct xen_dm_op_set_ioreq_server_state),
> + [XEN_DMOP_destroy_ioreq_server] = sizeof(struct xen_dm_op_destroy_ioreq_server),
> + };
> +
> + rc = rcu_lock_remote_domain_by_id(op_args->domid, &d);
> + if ( rc )
> + return rc;
> +
> + rc = xsm_dm_op(XSM_DM_PRIV, d);
> + if ( rc )
> + goto out;
> +
> + offset = offsetof(struct xen_dm_op, u);
> +
> + rc = -EFAULT;
> + if ( op_args->buf[0].size < offset )
> + goto out;
> +
> + if ( copy_from_guest_offset((void *)&op, op_args->buf[0].h, 0, offset) )
> + goto out;
> +
> + if ( op.op >= ARRAY_SIZE(op_size) )
> + {
> + rc = -EOPNOTSUPP;
> + goto out;
> + }
> +
> + op.op = array_index_nospec(op.op, ARRAY_SIZE(op_size));
> +
> + if ( op_args->buf[0].size < offset + op_size[op.op] )
> + goto out;
> +
> + if ( copy_from_guest_offset((void *)&op.u, op_args->buf[0].h, offset,
> + op_size[op.op]) )
> + goto out;
> +
> + rc = -EINVAL;
> + if ( op.pad )
> + goto out;
> +
> + rc = ioreq_server_dm_op(&op, d, &const_op);
> +
> + if ( (!rc || rc == -ERESTART) &&
> + !const_op && copy_to_guest_offset(op_args->buf[0].h, offset,
> + (void *)&op.u, op_size[op.op]) )
> + rc = -EFAULT;
> +
> + out:
> + rcu_unlock_domain(d);
> +
> + return rc;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> index 18cafcd..bdd3d3e 100644
> --- a/xen/arch/arm/domain.c
> +++ b/xen/arch/arm/domain.c
> @@ -15,6 +15,7 @@
> #include <xen/guest_access.h>
> #include <xen/hypercall.h>
> #include <xen/init.h>
> +#include <xen/ioreq.h>
> #include <xen/lib.h>
> #include <xen/livepatch.h>
> #include <xen/sched.h>
> @@ -696,6 +697,10 @@ int arch_domain_create(struct domain *d,
>
> ASSERT(config != NULL);
>
> +#ifdef CONFIG_IOREQ_SERVER
> + ioreq_domain_init(d);
> +#endif
> +
> /* p2m_init relies on some value initialized by the IOMMU subsystem */
> if ( (rc = iommu_domain_init(d, config->iommu_opts)) != 0 )
> goto fail;
> @@ -1009,6 +1014,10 @@ int domain_relinquish_resources(struct domain *d)
> */
> domain_vpl011_deinit(d);
>
> +#ifdef CONFIG_IOREQ_SERVER
> + ioreq_server_destroy_all(d);
> +#endif
> +
> PROGRESS(tee):
> ret = tee_relinquish_resources(d);
> if (ret )
> diff --git a/xen/arch/arm/io.c b/xen/arch/arm/io.c
> index ae7ef96..7ac0303 100644
> --- a/xen/arch/arm/io.c
> +++ b/xen/arch/arm/io.c
> @@ -16,12 +16,14 @@
> * GNU General Public License for more details.
> */
>
> +#include <xen/ioreq.h>
> #include <xen/lib.h>
> #include <xen/spinlock.h>
> #include <xen/sched.h>
> #include <xen/sort.h>
> #include <asm/cpuerrata.h>
> #include <asm/current.h>
> +#include <asm/ioreq.h>
> #include <asm/mmio.h>
>
> #include "decode.h"
> @@ -123,7 +125,15 @@ enum io_state try_handle_mmio(struct cpu_user_regs *regs,
>
> handler = find_mmio_handler(v->domain, info.gpa);
> if ( !handler )
> - return IO_UNHANDLED;
> + {
> + int rc;
> +
> + rc = try_fwd_ioserv(regs, v, &info);
> + if ( rc == IO_HANDLED )
> + return handle_ioserv(regs, v);
> +
> + return rc;
> + }
>
> /* All the instructions used on emulated MMIO region should be valid */
> if ( !dabt.valid )
> diff --git a/xen/arch/arm/ioreq.c b/xen/arch/arm/ioreq.c
> new file mode 100644
> index 0000000..ffeeb0b
> --- /dev/null
> +++ b/xen/arch/arm/ioreq.c
> @@ -0,0 +1,215 @@
> +/*
> + * arm/ioreq.c: hardware virtual machine I/O emulation
> + *
> + * Copyright (c) 2019 Arm ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms and conditions of the GNU General Public License,
> + * version 2, as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
> + * more details.
> + *
> + * You should have received a copy of the GNU General Public License along with
> + * this program; If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/domain.h>
> +#include <xen/ioreq.h>
> +
> +#include <asm/traps.h>
> +
> +#include <public/hvm/ioreq.h>
> +
> +enum io_state handle_ioserv(struct cpu_user_regs *regs, struct vcpu *v)
> +{
> + const union hsr hsr = { .bits = regs->hsr };
> + const struct hsr_dabt dabt = hsr.dabt;
> + /* Code is similar to handle_read */
> + uint8_t size = (1 << dabt.size) * 8;
> + register_t r = v->io.req.data;
> +
> + /* We are done with the IO */
> + v->io.req.state = STATE_IOREQ_NONE;
> +
> + if ( dabt.write )
> + return IO_HANDLED;
> +
> + /*
> + * Sign extend if required.
> + * Note that we expect the read handler to have zeroed the bits
> + * outside the requested access size.
> + */
> + if ( dabt.sign && (r & (1UL << (size - 1))) )
> + {
> + /*
> + * We are relying on register_t using the same as
> + * an unsigned long in order to keep the 32-bit assembly
> + * code smaller.
> + */
> + BUILD_BUG_ON(sizeof(register_t) != sizeof(unsigned long));
> + r |= (~0UL) << size;
> + }
> +
> + set_user_reg(regs, dabt.reg, r);
> +
> + return IO_HANDLED;
> +}
> +
> +enum io_state try_fwd_ioserv(struct cpu_user_regs *regs,
> + struct vcpu *v, mmio_info_t *info)
> +{
> + struct vcpu_io *vio = &v->io;
> + ioreq_t p = {
> + .type = IOREQ_TYPE_COPY,
> + .addr = info->gpa,
> + .size = 1 << info->dabt.size,
> + .count = 1,
> + .dir = !info->dabt.write,
> + /*
> + * On x86, df is used by 'rep' instruction to tell the direction
> + * to iterate (forward or backward).
> + * On Arm, all the accesses to MMIO region will do a single
> + * memory access. So for now, we can safely always set to 0.
> + */
> + .df = 0,
> + .data = get_user_reg(regs, info->dabt.reg),
> + .state = STATE_IOREQ_READY,
> + };
> + struct ioreq_server *s = NULL;
> + enum io_state rc;
> +
> + switch ( vio->req.state )
> + {
> + case STATE_IOREQ_NONE:
> + break;
> +
> + default:
> + gdprintk(XENLOG_ERR, "wrong state %u\n", vio->req.state);
> + return IO_ABORT;
> + }
> +
> + s = ioreq_server_select(v->domain, &p);
> + if ( !s )
> + return IO_UNHANDLED;
> +
> + if ( !info->dabt.valid )
> + return IO_ABORT;
> +
> + vio->req = p;
> +
> + rc = ioreq_send(s, &p, 0);
> + if ( rc != IO_RETRY || v->domain->is_shutting_down )
> + vio->req.state = STATE_IOREQ_NONE;
> + else if ( !ioreq_needs_completion(&vio->req) )
> + rc = IO_HANDLED;
> + else
> + vio->completion = VIO_mmio_completion;
> +
> + return rc;
> +}
> +
> +bool arch_ioreq_complete_mmio(void)
> +{
> + struct vcpu *v = current;
> + struct cpu_user_regs *regs = guest_cpu_user_regs();
> + const union hsr hsr = { .bits = regs->hsr };
> +
> + if ( v->io.req.state != STATE_IORESP_READY )
> + {
> + ASSERT_UNREACHABLE();
> + return false;
> + }
> +
> + if ( handle_ioserv(regs, v) == IO_HANDLED )
> + {
> + advance_pc(regs, hsr);
> + return true;
> + }
> +
> + return false;
> +}
> +
> +bool arch_vcpu_ioreq_completion(enum vio_completion completion)
> +{
> + ASSERT_UNREACHABLE();
> + return true;
> +}
> +
> +/*
> + * The "legacy" mechanism of mapping magic pages for the IOREQ servers
> + * is x86 specific, so the following hooks don't need to be implemented on Arm:
> + * - arch_ioreq_server_map_pages
> + * - arch_ioreq_server_unmap_pages
> + * - arch_ioreq_server_enable
> + * - arch_ioreq_server_disable
> + */
> +int arch_ioreq_server_map_pages(struct ioreq_server *s)
> +{
> + return -EOPNOTSUPP;
> +}
> +
> +void arch_ioreq_server_unmap_pages(struct ioreq_server *s)
> +{
> +}
> +
> +void arch_ioreq_server_enable(struct ioreq_server *s)
> +{
> +}
> +
> +void arch_ioreq_server_disable(struct ioreq_server *s)
> +{
> +}
> +
> +void arch_ioreq_server_destroy(struct ioreq_server *s)
> +{
> +}
> +
> +int arch_ioreq_server_map_mem_type(struct domain *d,
> + struct ioreq_server *s,
> + uint32_t flags)
> +{
> + return -EOPNOTSUPP;
> +}
> +
> +void arch_ioreq_server_map_mem_type_completed(struct domain *d,
> + struct ioreq_server *s,
> + uint32_t flags)
> +{
> +}
> +
> +bool arch_ioreq_server_destroy_all(struct domain *d)
> +{
> + return true;
> +}
> +
> +bool arch_ioreq_server_get_type_addr(const struct domain *d,
> + const ioreq_t *p,
> + uint8_t *type,
> + uint64_t *addr)
> +{
> + if ( p->type != IOREQ_TYPE_COPY && p->type != IOREQ_TYPE_PIO )
> + return false;
> +
> + *type = (p->type == IOREQ_TYPE_PIO) ?
> + XEN_DMOP_IO_RANGE_PORT : XEN_DMOP_IO_RANGE_MEMORY;
> + *addr = p->addr;
> +
> + return true;
> +}
> +
> +void arch_ioreq_domain_init(struct domain *d)
> +{
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index 1af1bb9..b0cd8f9 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -1385,6 +1385,9 @@ static arm_hypercall_t arm_hypercall_table[] = {
> #ifdef CONFIG_HYPFS
> HYPERCALL(hypfs_op, 5),
> #endif
> +#ifdef CONFIG_IOREQ_SERVER
> + HYPERCALL(dm_op, 3),
> +#endif
> };
>
> #ifndef NDEBUG
> @@ -1956,6 +1959,9 @@ static void do_trap_stage2_abort_guest(struct cpu_user_regs *regs,
> case IO_HANDLED:
> advance_pc(regs, hsr);
> return;
> + case IO_RETRY:
> + /* finish later */
> + return;
> case IO_UNHANDLED:
> /* IO unhandled, try another way to handle it. */
> break;
> diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
> index 6819a3b..1da90f2 100644
> --- a/xen/include/asm-arm/domain.h
> +++ b/xen/include/asm-arm/domain.h
> @@ -262,6 +262,8 @@ static inline void arch_vcpu_block(struct vcpu *v) {}
>
> #define arch_vm_assist_valid_mask(d) (1UL << VMASST_TYPE_runstate_update_flag)
>
> +#define has_vpci(d) ({ (void)(d); false; })
> +
> #endif /* __ASM_DOMAIN_H__ */
>
> /*
> diff --git a/xen/include/asm-arm/ioreq.h b/xen/include/asm-arm/ioreq.h
> new file mode 100644
> index 0000000..5018597
> --- /dev/null
> +++ b/xen/include/asm-arm/ioreq.h
> @@ -0,0 +1,70 @@
> +/*
> + * ioreq.h: Hardware virtual machine assist interface definitions.
> + *
> + * Copyright (c) 2016 Citrix Systems Inc.
> + * Copyright (c) 2019 Arm ltd.
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms and conditions of the GNU General Public License,
> + * version 2, as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
> + * more details.
> + *
> + * You should have received a copy of the GNU General Public License along with
> + * this program; If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#ifndef __ASM_ARM_IOREQ_H__
> +#define __ASM_ARM_IOREQ_H__
> +
> +#ifdef CONFIG_IOREQ_SERVER
> +enum io_state handle_ioserv(struct cpu_user_regs *regs, struct vcpu *v);
> +enum io_state try_fwd_ioserv(struct cpu_user_regs *regs,
> + struct vcpu *v, mmio_info_t *info);
> +#else
> +static inline enum io_state handle_ioserv(struct cpu_user_regs *regs,
> + struct vcpu *v)
> +{
> + return IO_UNHANDLED;
> +}
> +
> +static inline enum io_state try_fwd_ioserv(struct cpu_user_regs *regs,
> + struct vcpu *v, mmio_info_t *info)
> +{
> + return IO_UNHANDLED;
> +}
> +#endif
> +
> +static inline bool handle_pio(uint16_t port, unsigned int size, int dir)
> +{
> + /*
> + * TODO: For Arm64, the main user will be PCI. So this should be
> + * implemented when we add support for vPCI.
> + */
> + ASSERT_UNREACHABLE();
> + return true;
> +}
> +
> +static inline void msix_write_completion(struct vcpu *v)
> +{
> +}
> +
> +/* This correlation must not be altered */
> +#define IOREQ_STATUS_HANDLED IO_HANDLED
> +#define IOREQ_STATUS_UNHANDLED IO_UNHANDLED
> +#define IOREQ_STATUS_RETRY IO_RETRY
> +
> +#endif /* __ASM_ARM_IOREQ_H__ */
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/xen/include/asm-arm/mmio.h b/xen/include/asm-arm/mmio.h
> index 8dbfb27..7ab873c 100644
> --- a/xen/include/asm-arm/mmio.h
> +++ b/xen/include/asm-arm/mmio.h
> @@ -37,6 +37,7 @@ enum io_state
> IO_ABORT, /* The IO was handled by the helper and led to an abort. */
> IO_HANDLED, /* The IO was successfully handled by the helper. */
> IO_UNHANDLED, /* The IO was not handled by the helper. */
> + IO_RETRY, /* Retry the emulation for some reason */
> };
>
> typedef int (*mmio_read_t)(struct vcpu *v, mmio_info_t *info,
> diff --git a/xen/include/xen/dm.h b/xen/include/xen/dm.h
> index 4bd7e57..7f9f0bd 100644
> --- a/xen/include/xen/dm.h
> +++ b/xen/include/xen/dm.h
> @@ -19,6 +19,8 @@
>
> #include <xen/sched.h>
>
> +#include <public/hvm/dm_op.h>
> +
> struct dmop_args {
> domid_t domid;
> unsigned int nr_bufs;
> --
> 2.7.4
>
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common
2021-01-25 19:08 ` [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common Oleksandr Tyshchenko
@ 2021-01-25 23:13 ` Julien Grall
2021-01-25 23:40 ` Oleksandr
2021-01-27 16:58 ` Jan Beulich
1 sibling, 1 reply; 93+ messages in thread
From: Julien Grall @ 2021-01-25 23:13 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: xen-devel, Oleksandr Tyshchenko, Andrew Cooper, George Dunlap,
Ian Jackson, Jan Beulich, Stefano Stabellini, Wei Liu,
Roger Pau Monné,
Paul Durrant, Jun Nakajima, Kevin Tian, Tim Deegan, Julien Grall
Hi,
On Mon, 25 Jan 2021 at 19:09, Oleksandr Tyshchenko <olekstysh@gmail.com> wrote:
> ***
> Please note, this patch depends on the following which is
> on review:
> https://patchwork.kernel.org/patch/11816689/
> The effort (to get it upstreamed) was paused because of
> the security issue around that code (XSA-348).
> ***
I read this comment as "This series should be applied on top the patch
X". However, looking at your branch, I can't see the patch. What did I
miss?
Cheers,
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common
2021-01-25 23:13 ` Julien Grall
@ 2021-01-25 23:40 ` Oleksandr
2021-01-27 18:43 ` Julien Grall
0 siblings, 1 reply; 93+ messages in thread
From: Oleksandr @ 2021-01-25 23:40 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, Oleksandr Tyshchenko, Andrew Cooper, George Dunlap,
Ian Jackson, Jan Beulich, Stefano Stabellini, Wei Liu,
Roger Pau Monné,
Paul Durrant, Jun Nakajima, Kevin Tian, Tim Deegan, Julien Grall
On 26.01.21 01:13, Julien Grall wrote:
> Hi,
Hi Julien
>
> On Mon, 25 Jan 2021 at 19:09, Oleksandr Tyshchenko <olekstysh@gmail.com> wrote:
>> ***
>> Please note, this patch depends on the following which is
>> on review:
>> https://patchwork.kernel.org/patch/11816689/
>> The effort (to get it upstreamed) was paused because of
>> the security issue around that code (XSA-348).
>> ***
> I read this comment as "This series should be applied on top the patch
> X". However, looking at your branch, I can't see the patch. What did I
> miss?
You didn't miss anything. Patch series doesn't contain it. I mentioned
about this patch in order not to forget about it
and draw reviewer's attention. Looks like, the activity (to get it
upstreamed) hasn't been resumed yet and I don't know what we should do
with that dependency
in the context of this series...
>
> Cheers,
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2021-01-25 19:08 ` [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features Oleksandr Tyshchenko
2021-01-25 20:19 ` Stefano Stabellini
@ 2021-01-26 9:15 ` Jan Beulich
2021-01-27 9:54 ` Oleksandr
2021-01-27 17:01 ` Julien Grall
2021-01-27 18:33 ` Julien Grall
2 siblings, 2 replies; 93+ messages in thread
From: Jan Beulich @ 2021-01-26 9:15 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: Julien Grall, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Andrew Cooper, George Dunlap, Ian Jackson,
Wei Liu, Oleksandr Tyshchenko, xen-devel
On 25.01.2021 20:08, Oleksandr Tyshchenko wrote:
> --- a/xen/include/xen/dm.h
> +++ b/xen/include/xen/dm.h
> @@ -19,6 +19,8 @@
>
> #include <xen/sched.h>
>
> +#include <public/hvm/dm_op.h>
> +
> struct dmop_args {
> domid_t domid;
> unsigned int nr_bufs;
How come this becomes necessary at this point in the series, when
nothing else in this header changes, and nothing changes in the
public headers at all? Is it perhaps a .c file that needs the
#include instead? Headers shouldn't pull in other headers without
clear need - as indicated in reply to a prior version, we have
way too many bad examples (causing headaches in certain cases),
and we'd like to avoid gaining more. (Oh, I notice you actually
have a post-commit-message remark about this, but then this
patch should be marked RFC until the issue was resolved.)
Jan
^ permalink raw reply [flat|nested] 93+ messages in thread
* RE: [PATCH V5 09/22] xen/ioreq: Make x86's IOREQ related dm-op handling common
2021-01-25 19:08 ` [PATCH V5 09/22] xen/ioreq: Make x86's IOREQ related dm-op handling common Oleksandr Tyshchenko
@ 2021-01-26 13:31 ` Paul Durrant
2021-01-28 10:52 ` Jan Beulich
1 sibling, 0 replies; 93+ messages in thread
From: Paul Durrant @ 2021-01-26 13:31 UTC (permalink / raw)
To: 'Oleksandr Tyshchenko', xen-devel
Cc: 'Julien Grall', 'Jan Beulich',
'Andrew Cooper', 'Roger Pau Monné',
'Wei Liu', 'George Dunlap', 'Ian Jackson',
'Julien Grall', 'Stefano Stabellini',
'Daniel De Graaf', 'Oleksandr Tyshchenko'
> -----Original Message-----
> From: Oleksandr Tyshchenko <olekstysh@gmail.com>
> Sent: 25 January 2021 19:08
> To: xen-devel@lists.xenproject.org
> Cc: Julien Grall <julien.grall@arm.com>; Jan Beulich <jbeulich@suse.com>; Andrew Cooper
> <andrew.cooper3@citrix.com>; Roger Pau Monné <roger.pau@citrix.com>; Wei Liu <wl@xen.org>; George
> Dunlap <george.dunlap@citrix.com>; Ian Jackson <iwj@xenproject.org>; Julien Grall <julien@xen.org>;
> Stefano Stabellini <sstabellini@kernel.org>; Paul Durrant <paul@xen.org>; Daniel De Graaf
> <dgdegra@tycho.nsa.gov>; Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> Subject: [PATCH V5 09/22] xen/ioreq: Make x86's IOREQ related dm-op handling common
>
> From: Julien Grall <julien.grall@arm.com>
>
> As a lot of x86 code can be re-used on Arm later on, this patch
> moves the IOREQ related dm-op handling to the common code.
>
> The idea is to have the top level dm-op handling arch-specific
> and call into ioreq_server_dm_op() for otherwise unhandled ops.
> Pros:
> - More natural than doing it other way around (top level dm-op
> handling common).
> - Leave compat_dm_op() in x86 code.
> Cons:
> - Code duplication. Both arches have to duplicate dm_op(), etc.
>
> Make the corresponding functions static and rename them according
> to the new naming scheme (including dropping the "hvm" prefixes).
>
> Introduce common dm.c file as a resting place for the do_dm_op()
> (which is identical for both Arm and x86) to minimize code duplication.
> The common DM feature is supposed to be built with IOREQ_SERVER
> option enabled (as well as the IOREQ feature), which is selected
> for x86's config HVM for now.
>
> Also update XSM code a bit to let dm-op be used on Arm.
>
> This support is going to be used on Arm to be able run device
> emulator outside of Xen hypervisor.
>
> Signed-off-by: Julien Grall <julien.grall@arm.com>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>
> [On Arm only]
> Tested-by: Wei Chen <Wei.Chen@arm.com>
>
Reviewed-by: Paul Durrant <paul@xen.org>
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2021-01-26 9:15 ` Jan Beulich
@ 2021-01-27 9:54 ` Oleksandr
2021-01-27 10:15 ` Jan Beulich
2021-01-27 17:01 ` Julien Grall
1 sibling, 1 reply; 93+ messages in thread
From: Oleksandr @ 2021-01-27 9:54 UTC (permalink / raw)
To: Jan Beulich
Cc: Julien Grall, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Andrew Cooper, George Dunlap, Ian Jackson,
Wei Liu, Oleksandr Tyshchenko, xen-devel
On 26.01.21 11:15, Jan Beulich wrote:
Hi Jan
> On 25.01.2021 20:08, Oleksandr Tyshchenko wrote:
>> --- a/xen/include/xen/dm.h
>> +++ b/xen/include/xen/dm.h
>> @@ -19,6 +19,8 @@
>>
>> #include <xen/sched.h>
>>
>> +#include <public/hvm/dm_op.h>
>> +
>> struct dmop_args {
>> domid_t domid;
>> unsigned int nr_bufs;
> How come this becomes necessary at this point in the series, when
> nothing else in this header changes, and nothing changes in the
> public headers at all? Is it perhaps a .c file that needs the
> #include instead? Headers shouldn't pull in other headers without
> clear need - as indicated in reply to a prior version, we have
> way too many bad examples (causing headaches in certain cases),
> and we'd like to avoid gaining more.
Yes, I understand this and completely agree. I remember last discussion
on that, this is not forgotten. The only reason I made this (non
entirely correct) change is to make
series compilable on Arm with IOREQ support enabled. If I considered
this change as a correct one I would make it from the very beginning (in
patch #9 which adds this common header)...
I added remark to draw reviewer's attention on the fact that I got stuck
with resolving that, what I did wrong and how it should be done
properly. The problem is that I didn't manage to make it properly.
Of course, I tried to include it directly by dm.c, but this didn't help
much without violating "alphabetical order" rule. Details here:
https://lore.kernel.org/xen-devel/e0bc7f80-974e-945d-4605-173bd05302af@gmail.com/
I would appreciate any input on that.
> (Oh, I notice you actually
> have a post-commit-message remark about this, but then this
> patch should be marked RFC until the issue was resolved.)
Agree, I should have marked this patch as RFC to avoid misunderstanding.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2021-01-27 9:54 ` Oleksandr
@ 2021-01-27 10:15 ` Jan Beulich
0 siblings, 0 replies; 93+ messages in thread
From: Jan Beulich @ 2021-01-27 10:15 UTC (permalink / raw)
To: Oleksandr
Cc: Julien Grall, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Andrew Cooper, George Dunlap, Ian Jackson,
Wei Liu, Oleksandr Tyshchenko, xen-devel
On 27.01.2021 10:54, Oleksandr wrote:
>
> On 26.01.21 11:15, Jan Beulich wrote:
>
> Hi Jan
>
>> On 25.01.2021 20:08, Oleksandr Tyshchenko wrote:
>>> --- a/xen/include/xen/dm.h
>>> +++ b/xen/include/xen/dm.h
>>> @@ -19,6 +19,8 @@
>>>
>>> #include <xen/sched.h>
>>>
>>> +#include <public/hvm/dm_op.h>
>>> +
>>> struct dmop_args {
>>> domid_t domid;
>>> unsigned int nr_bufs;
>> How come this becomes necessary at this point in the series, when
>> nothing else in this header changes, and nothing changes in the
>> public headers at all? Is it perhaps a .c file that needs the
>> #include instead? Headers shouldn't pull in other headers without
>> clear need - as indicated in reply to a prior version, we have
>> way too many bad examples (causing headaches in certain cases),
>> and we'd like to avoid gaining more.
>
> Yes, I understand this and completely agree. I remember last discussion
> on that, this is not forgotten. The only reason I made this (non
> entirely correct) change is to make
> series compilable on Arm with IOREQ support enabled. If I considered
> this change as a correct one I would make it from the very beginning (in
> patch #9 which adds this common header)...
> I added remark to draw reviewer's attention on the fact that I got stuck
> with resolving that, what I did wrong and how it should be done
> properly. The problem is that I didn't manage to make it properly.
>
> Of course, I tried to include it directly by dm.c, but this didn't help
> much without violating "alphabetical order" rule. Details here:
> https://lore.kernel.org/xen-devel/e0bc7f80-974e-945d-4605-173bd05302af@gmail.com/
>
> I would appreciate any input on that.
I'll try to reply there.
Jan
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 15/22] xen/arm: Call vcpu_ioreq_handle_completion() in check_for_vcpu_work()
2021-01-25 19:08 ` [PATCH V5 15/22] xen/arm: Call vcpu_ioreq_handle_completion() in check_for_vcpu_work() Oleksandr Tyshchenko
@ 2021-01-27 14:49 ` Julien Grall
2021-01-27 15:56 ` Oleksandr
0 siblings, 1 reply; 93+ messages in thread
From: Julien Grall @ 2021-01-27 14:49 UTC (permalink / raw)
To: Oleksandr Tyshchenko, xen-devel
Cc: Oleksandr Tyshchenko, Stefano Stabellini, Volodymyr Babchuk,
Julien Grall
Hi Oleksandr,
On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> This patch adds remaining bits needed for the IOREQ support on Arm.
> Besides just calling vcpu_ioreq_handle_completion() we need to handle
> it's return value to make sure that all the vCPU works are done before
> we return to the guest (the vcpu_ioreq_handle_completion() may return
> false if there is vCPU work to do or IOREQ state is invalid).
> For that reason we use an unbounded loop in leave_hypervisor_to_guest().
>
> The worse that can happen here if the vCPU will never run again
> (the I/O will never complete). But, in Xen case, if the I/O never
> completes then it most likely means that something went horribly
> wrong with the Device Emulator. And it is most likely not safe
> to continue. So letting the vCPU to spin forever if the I/O never
> completes is a safer action than letting it continue and leaving
> the guest in unclear state and is the best what we can do for now.
>
> Please note, using this loop we will not spin forever on a pCPU,
> preventing any other vCPUs from being scheduled. At every loop
> we will call check_for_pcpu_work() that will process pending
> softirqs. In case of failure, the guest will crash and the vCPU
> will be unscheduled. In normal case, if the rescheduling is necessary
> the vCPU will be rescheduled to give place to someone else.
>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> CC: Julien Grall <julien.grall@arm.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> [On Arm only]
> Tested-by: Wei Chen <Wei.Chen@arm.com>
>
> ---
> Please note, this is a split/cleanup/hardening of Julien's PoC:
> "Add support for Guest IO forwarding to a device emulator"
>
> Changes V1 -> V2:
> - new patch, changes were derived from (+ new explanation):
> arm/ioreq: Introduce arch specific bits for IOREQ/DM features
>
> Changes V2 -> V3:
> - update patch description
>
> Changes V3 -> V4:
> - update patch description and comment in code
>
> Changes V4 -> V5:
> - add Stefano's R-b
Reviewed-by means the person reviewed the code and confirmed it is
correct. Given the changes you made below, I don't think this tag can hold.
Please confirm with Stefano he is happy with the tag to be carried.
Other than that, the change looks good to me:
Acked-by: Julien Grall <jgrall@amazon.com>
Cheers,
> - update patch subject/description and comment in code,
> was "xen/arm: Stick around in leave_hypervisor_to_guest until I/O has completed"
> - change loop logic a bit
> - squash with changes to check_for_vcpu_work() from patch #14
>
> ---
> ---
> xen/arch/arm/traps.c | 26 +++++++++++++++++++++++---
> 1 file changed, 23 insertions(+), 3 deletions(-)
>
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index b0cd8f9..2039ff5 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -21,6 +21,7 @@
> #include <xen/hypercall.h>
> #include <xen/init.h>
> #include <xen/iocap.h>
> +#include <xen/ioreq.h>
> #include <xen/irq.h>
> #include <xen/lib.h>
> #include <xen/mem_access.h>
> @@ -2261,12 +2262,23 @@ static void check_for_pcpu_work(void)
> * Process pending work for the vCPU. Any call should be fast or
> * implement preemption.
> */
> -static void check_for_vcpu_work(void)
> +static bool check_for_vcpu_work(void)
> {
> struct vcpu *v = current;
>
> +#ifdef CONFIG_IOREQ_SERVER
> + bool handled;
> +
> + local_irq_enable();
> + handled = vcpu_ioreq_handle_completion(v);
> + local_irq_disable();
> +
> + if ( !handled )
> + return true;
> +#endif
> +
> if ( likely(!v->arch.need_flush_to_ram) )
> - return;
> + return false;
>
> /*
> * Give a chance for the pCPU to process work before handling the vCPU
> @@ -2277,6 +2289,8 @@ static void check_for_vcpu_work(void)
> local_irq_enable();
> p2m_flush_vm(v);
> local_irq_disable();
> +
> + return false;
> }
>
> /*
> @@ -2289,7 +2303,13 @@ void leave_hypervisor_to_guest(void)
> {
> local_irq_disable();
>
> - check_for_vcpu_work();
> + /*
> + * check_for_vcpu_work() may return true if there are more work to before
> + * the vCPU can safely resume. This gives us an opportunity to deschedule
> + * the vCPU if needed.
> + */
> + while ( check_for_vcpu_work() )
> + check_for_pcpu_work();
> check_for_pcpu_work();
>
> vgic_sync_to_lrs();
>
--
Julien Grall
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 15/22] xen/arm: Call vcpu_ioreq_handle_completion() in check_for_vcpu_work()
2021-01-27 14:49 ` Julien Grall
@ 2021-01-27 15:56 ` Oleksandr
2021-01-27 20:34 ` Stefano Stabellini
0 siblings, 1 reply; 93+ messages in thread
From: Oleksandr @ 2021-01-27 15:56 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: xen-devel, Oleksandr Tyshchenko, Volodymyr Babchuk, Julien Grall
On 27.01.21 16:49, Julien Grall wrote:
> Hi Oleksandr,
Hi Julien, Stefano
>
> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> This patch adds remaining bits needed for the IOREQ support on Arm.
>> Besides just calling vcpu_ioreq_handle_completion() we need to handle
>> it's return value to make sure that all the vCPU works are done before
>> we return to the guest (the vcpu_ioreq_handle_completion() may return
>> false if there is vCPU work to do or IOREQ state is invalid).
>> For that reason we use an unbounded loop in leave_hypervisor_to_guest().
>>
>> The worse that can happen here if the vCPU will never run again
>> (the I/O will never complete). But, in Xen case, if the I/O never
>> completes then it most likely means that something went horribly
>> wrong with the Device Emulator. And it is most likely not safe
>> to continue. So letting the vCPU to spin forever if the I/O never
>> completes is a safer action than letting it continue and leaving
>> the guest in unclear state and is the best what we can do for now.
>>
>> Please note, using this loop we will not spin forever on a pCPU,
>> preventing any other vCPUs from being scheduled. At every loop
>> we will call check_for_pcpu_work() that will process pending
>> softirqs. In case of failure, the guest will crash and the vCPU
>> will be unscheduled. In normal case, if the rescheduling is necessary
>> the vCPU will be rescheduled to give place to someone else.
>>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> CC: Julien Grall <julien.grall@arm.com>
>> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
>> [On Arm only]
>> Tested-by: Wei Chen <Wei.Chen@arm.com>
>>
>> ---
>> Please note, this is a split/cleanup/hardening of Julien's PoC:
>> "Add support for Guest IO forwarding to a device emulator"
>>
>> Changes V1 -> V2:
>> - new patch, changes were derived from (+ new explanation):
>> arm/ioreq: Introduce arch specific bits for IOREQ/DM features
>>
>> Changes V2 -> V3:
>> - update patch description
>>
>> Changes V3 -> V4:
>> - update patch description and comment in code
>>
>> Changes V4 -> V5:
>> - add Stefano's R-b
>
> Reviewed-by means the person reviewed the code and confirmed it is
> correct. Given the changes you made below, I don't think this tag can
> hold.
>
> Please confirm with Stefano he is happy with the tag to be carried.
I think you are right, sorry for that. I should have either clarified
this question with Stefano in advance or dropped this tag.
@Stefano, are you happy with the changes for V4 -> V5 (would you mind if
your R-b still stands)?
>
> Other than that, the change looks good to me:
>
> Acked-by: Julien Grall <jgrall@amazon.com>
Thanks.
>
> Cheers,
>
>> - update patch subject/description and comment in code,
>> was "xen/arm: Stick around in leave_hypervisor_to_guest until
>> I/O has completed"
>> - change loop logic a bit
>> - squash with changes to check_for_vcpu_work() from patch #14
>>
>> ---
>> ---
>> xen/arch/arm/traps.c | 26 +++++++++++++++++++++++---
>> 1 file changed, 23 insertions(+), 3 deletions(-)
>>
>> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
>> index b0cd8f9..2039ff5 100644
>> --- a/xen/arch/arm/traps.c
>> +++ b/xen/arch/arm/traps.c
>> @@ -21,6 +21,7 @@
>> #include <xen/hypercall.h>
>> #include <xen/init.h>
>> #include <xen/iocap.h>
>> +#include <xen/ioreq.h>
>> #include <xen/irq.h>
>> #include <xen/lib.h>
>> #include <xen/mem_access.h>
>> @@ -2261,12 +2262,23 @@ static void check_for_pcpu_work(void)
>> * Process pending work for the vCPU. Any call should be fast or
>> * implement preemption.
>> */
>> -static void check_for_vcpu_work(void)
>> +static bool check_for_vcpu_work(void)
>> {
>> struct vcpu *v = current;
>> +#ifdef CONFIG_IOREQ_SERVER
>> + bool handled;
>> +
>> + local_irq_enable();
>> + handled = vcpu_ioreq_handle_completion(v);
>> + local_irq_disable();
>> +
>> + if ( !handled )
>> + return true;
>> +#endif
>> +
>> if ( likely(!v->arch.need_flush_to_ram) )
>> - return;
>> + return false;
>> /*
>> * Give a chance for the pCPU to process work before handling
>> the vCPU
>> @@ -2277,6 +2289,8 @@ static void check_for_vcpu_work(void)
>> local_irq_enable();
>> p2m_flush_vm(v);
>> local_irq_disable();
>> +
>> + return false;
>> }
>> /*
>> @@ -2289,7 +2303,13 @@ void leave_hypervisor_to_guest(void)
>> {
>> local_irq_disable();
>> - check_for_vcpu_work();
>> + /*
>> + * check_for_vcpu_work() may return true if there are more work
>> to before
>> + * the vCPU can safely resume. This gives us an opportunity to
>> deschedule
>> + * the vCPU if needed.
>> + */
>> + while ( check_for_vcpu_work() )
>> + check_for_pcpu_work();
>> check_for_pcpu_work();
>> vgic_sync_to_lrs();
>>
>
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (21 preceding siblings ...)
2021-01-25 19:08 ` [PATCH V5 22/22] xen/arm: Add mapcache invalidation handling Oleksandr Tyshchenko
@ 2021-01-27 16:43 ` Julien Grall
2021-01-27 16:50 ` Oleksandr
2021-01-28 16:11 ` Julien Grall
23 siblings, 1 reply; 93+ messages in thread
From: Julien Grall @ 2021-01-27 16:43 UTC (permalink / raw)
To: Oleksandr Tyshchenko, xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Tim Deegan,
Daniel De Graaf, Volodymyr Babchuk, Bertrand Marquis, Wei Chen,
Kaly Xin, Artem Mygaiev, Alex Bennée
Hi Oleksandr,
On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
> ***
>
> Patch series [8] was rebased on recent "staging branch"
> (5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is unmapped) and tested on
> Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio disk backend [9]
> running in driver domain and unmodified Linux Guest running on existing
> virtio-blk driver (frontend). No issues were observed. Guest domain 'reboot/destroy'
> use-cases work properly. Patch series was only build-tested on x86.
>
> Please note, build-test passed for the following modes:
> 1. x86: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y (default)
> 2. x86: #CONFIG_HVM is not set / #CONFIG_IOREQ_SERVER is not set
> 3. Arm64: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
> 4. Arm64: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
> 5. Arm32: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
> 6. Arm32: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
I thought I woudl give a try to test the code, but I can't find a way to
enable CONFIG_IOREQ_SERVER from the UI.
Looking at the Kconfig, it looks like CONFIG_IOREQ_SERVER doesn't have a
prompt and is not selected by Arm.
Can you provide details how this can be built on Arm?
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 01/22] x86/ioreq: Prepare IOREQ feature for making it common
2021-01-25 19:08 ` [PATCH V5 01/22] x86/ioreq: Prepare IOREQ feature for making it common Oleksandr Tyshchenko
@ 2021-01-27 16:48 ` Jan Beulich
0 siblings, 0 replies; 93+ messages in thread
From: Jan Beulich @ 2021-01-27 16:48 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: Oleksandr Tyshchenko, Paul Durrant, Andrew Cooper,
Roger Pau Monné,
Wei Liu, Stefano Stabellini, Julien Grall, Julien Grall,
xen-devel
On 25.01.2021 20:08, Oleksandr Tyshchenko wrote:
> Changes V4 -> V5:
> - add Julien's and Paul's R-b
> - update patch description
> - remove single use variable in arch_ioreq_server_map_mem_type_completed()
> - put multiple function parameters on a single line in the header
> where possible
> - introduce common ioreq.h right away and put arch hooks declarations
> there instead of doing that in patch #4
And with this, faod (the patch doesn't strictly need it),
Acked-by: Jan Beulich <jbeulich@suse.com>
Jan
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
2021-01-27 16:43 ` [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Julien Grall
@ 2021-01-27 16:50 ` Oleksandr
2021-01-27 17:33 ` Julien Grall
0 siblings, 1 reply; 93+ messages in thread
From: Oleksandr @ 2021-01-27 16:50 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Andrew Cooper, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Tim Deegan,
Daniel De Graaf, Volodymyr Babchuk, Bertrand Marquis, Wei Chen,
Kaly Xin, Artem Mygaiev, Alex Bennée
[-- Attachment #1: Type: text/plain, Size: 1425 bytes --]
On 27.01.21 18:43, Julien Grall wrote:
> Hi Oleksandr,
Hi Julien
>
> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>> ***
>>
>> Patch series [8] was rebased on recent "staging branch"
>> (5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is unmapped)
>> and tested on
>> Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio disk
>> backend [9]
>> running in driver domain and unmodified Linux Guest running on existing
>> virtio-blk driver (frontend). No issues were observed. Guest domain
>> 'reboot/destroy'
>> use-cases work properly. Patch series was only build-tested on x86.
>>
>> Please note, build-test passed for the following modes:
>> 1. x86: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y (default)
>> 2. x86: #CONFIG_HVM is not set / #CONFIG_IOREQ_SERVER is not set
>> 3. Arm64: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
>> 4. Arm64: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
>> 5. Arm32: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
>> 6. Arm32: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
>
> I thought I woudl give a try to test the code, but I can't find a way
> to enable CONFIG_IOREQ_SERVER from the UI.
>
> Looking at the Kconfig, it looks like CONFIG_IOREQ_SERVER doesn't have
> a prompt and is not selected by Arm.
>
> Can you provide details how this can be built on Arm?
Please apply the attached patch to select IOREQ on Arm.
--
Regards,
Oleksandr Tyshchenko
[-- Attachment #2: 0001-NOT-FOR-MERGE-enable-ioreq.patch --]
[-- Type: text/x-patch, Size: 656 bytes --]
From c0ef23f7f0788783f9fccbf1e4e4935711de67ea Mon Sep 17 00:00:00 2001
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Date: Thu, 8 Oct 2020 22:30:24 +0300
Subject: [PATCH] [NOT FOR MERGE] enable ioreq
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
xen/arch/arm/Kconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index c3eb13e..5cf1e84 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -21,6 +21,7 @@ config ARM
select HAS_PASSTHROUGH
select HAS_PDX
select IOMMU_FORCE_PT_SHARE
+ select IOREQ_SERVER
config ARCH_DEFCONFIG
string
--
2.7.4
^ permalink raw reply related [flat|nested] 93+ messages in thread
* Re: [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common
2021-01-25 19:08 ` [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common Oleksandr Tyshchenko
2021-01-25 23:13 ` Julien Grall
@ 2021-01-27 16:58 ` Jan Beulich
2021-01-27 20:14 ` Oleksandr
1 sibling, 1 reply; 93+ messages in thread
From: Jan Beulich @ 2021-01-27 16:58 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: Oleksandr Tyshchenko, Andrew Cooper, George Dunlap, Ian Jackson,
Julien Grall, Stefano Stabellini, Wei Liu, Roger Pau Monné,
Paul Durrant, Jun Nakajima, Kevin Tian, Tim Deegan, Julien Grall,
xen-devel
On 25.01.2021 20:08, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> As a lot of x86 code can be re-used on Arm later on, this patch
> moves previously prepared IOREQ support to the common code
> (the code movement is verbatim copy).
>
> The "legacy" mechanism of mapping magic pages for the IOREQ servers
> remains x86 specific and not exposed to the common code.
>
> The common IOREQ feature is supposed to be built with IOREQ_SERVER
> option enabled, which is selected for x86's config HVM for now.
>
> In order to avoid having a gigantic patch here, the subsequent
> patches will update remaining bits in the common code step by step:
> - Make IOREQ related structs/materials common
> - Drop the "hvm" prefixes and infixes
> - Remove layering violation by moving corresponding fields
> out of *arch.hvm* or abstracting away accesses to them
>
> Introduce asm/ioreq.h wrapper to be included by common ioreq.h
> instead of asm/hvm/ioreq.h to avoid HVM-ism in the code common.
>
> Also include <xen/domain_page.h> which will be needed on Arm
> to avoid touch the common code again when introducing Arm specific bits.
>
> This support is going to be used on Arm to be able run device
> emulator outside of Xen hypervisor.
>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> Reviewed-by: Paul Durrant <paul@xen.org>
> CC: Julien Grall <julien.grall@arm.com>
> [On Arm only]
> Tested-by: Wei Chen <Wei.Chen@arm.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
with ...
> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -92,6 +92,7 @@ config PV_LINEAR_PT
>
> config HVM
> def_bool !PV_SHIM_EXCLUSIVE
> + select IOREQ_SERVER
> prompt "HVM support"
> ---help---
... the addition moved below the prompt line (could probably
be taken care of while committing, if no other need for a v6
arises).
(Personally I think this should be
config HVM
bool "HVM support"
default !PV_SHIM_EXCLUSIVE
select IOREQ_SERVER
---help---
anyway, but that's nothing you need to care about.)
> --- /dev/null
> +++ b/xen/include/asm-x86/ioreq.h
> @@ -0,0 +1,37 @@
> +/*
> + * ioreq.h: Hardware virtual machine assist interface definitions.
> + *
> + * This is a wrapper which purpose is to not include arch HVM specific header
> + * from the common code.
> + *
> + * Copyright (c) 2016 Citrix Systems Inc.
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms and conditions of the GNU General Public License,
> + * version 2, as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
> + * more details.
> + *
> + * You should have received a copy of the GNU General Public License along with
> + * this program; If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#ifndef __ASM_X86_IOREQ_H__
> +#define __ASM_X86_IOREQ_H__
> +
> +#include <asm/hvm/ioreq.h>
> +
> +#endif /* __ASM_X86_IOREQ_H__ */
Not necessarily for taking care of right away, I think in the
longer run this wants wrapping by #ifdef CONFIG_HVM, such that
in !HVM builds the dependency on the "chained" header goes
away (reducing the amount of rebuilding in incremental builds).
Jan
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2021-01-26 9:15 ` Jan Beulich
2021-01-27 9:54 ` Oleksandr
@ 2021-01-27 17:01 ` Julien Grall
2021-01-27 17:04 ` Jan Beulich
1 sibling, 1 reply; 93+ messages in thread
From: Julien Grall @ 2021-01-27 17:01 UTC (permalink / raw)
To: Jan Beulich, Oleksandr Tyshchenko
Cc: Julien Grall, Stefano Stabellini, Volodymyr Babchuk,
Andrew Cooper, George Dunlap, Ian Jackson, Wei Liu,
Oleksandr Tyshchenko, xen-devel
Hi Jan,
On 26/01/2021 09:15, Jan Beulich wrote:
> On 25.01.2021 20:08, Oleksandr Tyshchenko wrote:
>> --- a/xen/include/xen/dm.h
>> +++ b/xen/include/xen/dm.h
>> @@ -19,6 +19,8 @@
>>
>> #include <xen/sched.h>
>>
>> +#include <public/hvm/dm_op.h>
>> +
>> struct dmop_args {
>> domid_t domid;
>> unsigned int nr_bufs;
>
> How come this becomes necessary at this point in the series, when
> nothing else in this header changes, and nothing changes in the
> public headers at all? Is it perhaps a .c file that needs the
> #include instead? Headers shouldn't pull in other headers without
> clear need - as indicated in reply to a prior version, we have
> way too many bad examples (causing headaches in certain cases),
> and we'd like to avoid gaining more. (Oh, I notice you actually
> have a post-commit-message remark about this, but then this
> patch should be marked RFC until the issue was resolved.)
dm.h contains the following:
struct dmop_args {
domid_t domid;
unsigned int nr_bufs;
/* Reserve enough buf elements for all current hypercalls. */
struct xen_dm_op_buf buf[2];
};
The struct xen_dm_op_buf is defined public/hvm/dm_op.h. On x86, this
will be indirectly included via:
-> xen/sched.h
-> xen/domain.h
-> asm/domain.h
-> asm/hvm/domain.h
-> public/hvm/dm_op.h
It looks like this was include from asm/hvm/domain.h because, before
this series, NR_IO_RANGE_TYPES made use of a DMOP definition.
With this series, the type is now moved in ioreq.h. So I think we may be
able to drop the include from asm/hvm/domain.h (this would avoid to
include it everywhere...).
I also think we want to include public/hvm/dm_op.h in xen/dm.h because
it is included directly by *.c.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2021-01-27 17:01 ` Julien Grall
@ 2021-01-27 17:04 ` Jan Beulich
0 siblings, 0 replies; 93+ messages in thread
From: Jan Beulich @ 2021-01-27 17:04 UTC (permalink / raw)
To: Julien Grall, Oleksandr Tyshchenko
Cc: Julien Grall, Stefano Stabellini, Volodymyr Babchuk,
Andrew Cooper, George Dunlap, Ian Jackson, Wei Liu,
Oleksandr Tyshchenko, xen-devel
On 27.01.2021 18:01, Julien Grall wrote:
> Hi Jan,
>
> On 26/01/2021 09:15, Jan Beulich wrote:
>> On 25.01.2021 20:08, Oleksandr Tyshchenko wrote:
>>> --- a/xen/include/xen/dm.h
>>> +++ b/xen/include/xen/dm.h
>>> @@ -19,6 +19,8 @@
>>>
>>> #include <xen/sched.h>
>>>
>>> +#include <public/hvm/dm_op.h>
>>> +
>>> struct dmop_args {
>>> domid_t domid;
>>> unsigned int nr_bufs;
>>
>> How come this becomes necessary at this point in the series, when
>> nothing else in this header changes, and nothing changes in the
>> public headers at all? Is it perhaps a .c file that needs the
>> #include instead? Headers shouldn't pull in other headers without
>> clear need - as indicated in reply to a prior version, we have
>> way too many bad examples (causing headaches in certain cases),
>> and we'd like to avoid gaining more. (Oh, I notice you actually
>> have a post-commit-message remark about this, but then this
>> patch should be marked RFC until the issue was resolved.)
>
> dm.h contains the following:
>
> struct dmop_args {
> domid_t domid;
> unsigned int nr_bufs;
> /* Reserve enough buf elements for all current hypercalls. */
> struct xen_dm_op_buf buf[2];
> };
Which means the public header should be included, yes, but
right away at the point this dependency appears, not at a
random point later in the series.
Jan
> The struct xen_dm_op_buf is defined public/hvm/dm_op.h. On x86, this
> will be indirectly included via:
> -> xen/sched.h
> -> xen/domain.h
> -> asm/domain.h
> -> asm/hvm/domain.h
> -> public/hvm/dm_op.h
>
> It looks like this was include from asm/hvm/domain.h because, before
> this series, NR_IO_RANGE_TYPES made use of a DMOP definition.
>
> With this series, the type is now moved in ioreq.h. So I think we may be
> able to drop the include from asm/hvm/domain.h (this would avoid to
> include it everywhere...).
>
> I also think we want to include public/hvm/dm_op.h in xen/dm.h because
> it is included directly by *.c.
>
> Cheers,
>
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
2021-01-27 16:50 ` Oleksandr
@ 2021-01-27 17:33 ` Julien Grall
2021-01-27 17:37 ` Oleksandr
0 siblings, 1 reply; 93+ messages in thread
From: Julien Grall @ 2021-01-27 17:33 UTC (permalink / raw)
To: Oleksandr
Cc: xen-devel, Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Andrew Cooper, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Tim Deegan,
Daniel De Graaf, Volodymyr Babchuk, Bertrand Marquis, Wei Chen,
Kaly Xin, Artem Mygaiev, Alex Bennée
On 27/01/2021 16:50, Oleksandr wrote:
>
> On 27.01.21 18:43, Julien Grall wrote:
>> Hi Oleksandr,
>
> Hi Julien
>
>
>>
>> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>>> ***
>>>
>>> Patch series [8] was rebased on recent "staging branch"
>>> (5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is unmapped)
>>> and tested on
>>> Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio disk
>>> backend [9]
>>> running in driver domain and unmodified Linux Guest running on existing
>>> virtio-blk driver (frontend). No issues were observed. Guest domain
>>> 'reboot/destroy'
>>> use-cases work properly. Patch series was only build-tested on x86.
>>>
>>> Please note, build-test passed for the following modes:
>>> 1. x86: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y (default)
>>> 2. x86: #CONFIG_HVM is not set / #CONFIG_IOREQ_SERVER is not set
>>> 3. Arm64: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
>>> 4. Arm64: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
>>> 5. Arm32: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
>>> 6. Arm32: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
>>
>> I thought I woudl give a try to test the code, but I can't find a way
>> to enable CONFIG_IOREQ_SERVER from the UI.
>>
>> Looking at the Kconfig, it looks like CONFIG_IOREQ_SERVER doesn't have
>> a prompt and is not selected by Arm.
>>
>> Can you provide details how this can be built on Arm?
>
> Please apply the attached patch to select IOREQ on Arm.
This is roughly what I wrote. I think a user should be able to select
IOREQ via the menuconfig without any additional patch on top of your series.
Can you include a patch that would enable that?
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
2021-01-27 17:33 ` Julien Grall
@ 2021-01-27 17:37 ` Oleksandr
2021-01-27 17:42 ` Julien Grall
0 siblings, 1 reply; 93+ messages in thread
From: Oleksandr @ 2021-01-27 17:37 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Andrew Cooper, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Tim Deegan,
Daniel De Graaf, Volodymyr Babchuk, Bertrand Marquis, Wei Chen,
Kaly Xin, Artem Mygaiev, Alex Bennée
On 27.01.21 19:33, Julien Grall wrote:
Hi Julien
>
>
> On 27/01/2021 16:50, Oleksandr wrote:
>>
>> On 27.01.21 18:43, Julien Grall wrote:
>>> Hi Oleksandr,
>>
>> Hi Julien
>>
>>
>>>
>>> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>>>> ***
>>>>
>>>> Patch series [8] was rebased on recent "staging branch"
>>>> (5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is
>>>> unmapped) and tested on
>>>> Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio
>>>> disk backend [9]
>>>> running in driver domain and unmodified Linux Guest running on
>>>> existing
>>>> virtio-blk driver (frontend). No issues were observed. Guest domain
>>>> 'reboot/destroy'
>>>> use-cases work properly. Patch series was only build-tested on x86.
>>>>
>>>> Please note, build-test passed for the following modes:
>>>> 1. x86: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y (default)
>>>> 2. x86: #CONFIG_HVM is not set / #CONFIG_IOREQ_SERVER is not set
>>>> 3. Arm64: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
>>>> 4. Arm64: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
>>>> 5. Arm32: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
>>>> 6. Arm32: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
>>>
>>> I thought I woudl give a try to test the code, but I can't find a
>>> way to enable CONFIG_IOREQ_SERVER from the UI.
>>>
>>> Looking at the Kconfig, it looks like CONFIG_IOREQ_SERVER doesn't
>>> have a prompt and is not selected by Arm.
>>>
>>> Can you provide details how this can be built on Arm?
>>
>> Please apply the attached patch to select IOREQ on Arm.
>
> This is roughly what I wrote. I think a user should be able to select
> IOREQ via the menuconfig without any additional patch on top of your
> series.
>
> Can you include a patch that would enable that?
Yes, do you prefer a separate patch or required changes could be folded
in patch #14?
>
>
> Cheers,
>
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
2021-01-27 17:37 ` Oleksandr
@ 2021-01-27 17:42 ` Julien Grall
2021-01-27 17:45 ` Oleksandr
0 siblings, 1 reply; 93+ messages in thread
From: Julien Grall @ 2021-01-27 17:42 UTC (permalink / raw)
To: Oleksandr
Cc: xen-devel, Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Andrew Cooper, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Tim Deegan,
Daniel De Graaf, Volodymyr Babchuk, Bertrand Marquis, Wei Chen,
Kaly Xin, Artem Mygaiev, Alex Bennée
On 27/01/2021 17:37, Oleksandr wrote:
>
> On 27.01.21 19:33, Julien Grall wrote:
>
> Hi Julien
>
>>
>>
>> On 27/01/2021 16:50, Oleksandr wrote:
>>>
>>> On 27.01.21 18:43, Julien Grall wrote:
>>>> Hi Oleksandr,
>>>
>>> Hi Julien
>>>
>>>
>>>>
>>>> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>>>>> ***
>>>>>
>>>>> Patch series [8] was rebased on recent "staging branch"
>>>>> (5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is
>>>>> unmapped) and tested on
>>>>> Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio
>>>>> disk backend [9]
>>>>> running in driver domain and unmodified Linux Guest running on
>>>>> existing
>>>>> virtio-blk driver (frontend). No issues were observed. Guest domain
>>>>> 'reboot/destroy'
>>>>> use-cases work properly. Patch series was only build-tested on x86.
>>>>>
>>>>> Please note, build-test passed for the following modes:
>>>>> 1. x86: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y (default)
>>>>> 2. x86: #CONFIG_HVM is not set / #CONFIG_IOREQ_SERVER is not set
>>>>> 3. Arm64: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
>>>>> 4. Arm64: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
>>>>> 5. Arm32: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
>>>>> 6. Arm32: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
>>>>
>>>> I thought I woudl give a try to test the code, but I can't find a
>>>> way to enable CONFIG_IOREQ_SERVER from the UI.
>>>>
>>>> Looking at the Kconfig, it looks like CONFIG_IOREQ_SERVER doesn't
>>>> have a prompt and is not selected by Arm.
>>>>
>>>> Can you provide details how this can be built on Arm?
>>>
>>> Please apply the attached patch to select IOREQ on Arm.
>>
>> This is roughly what I wrote. I think a user should be able to select
>> IOREQ via the menuconfig without any additional patch on top of your
>> series.
>>
>> Can you include a patch that would enable that?
>
> Yes, do you prefer a separate patch or required changes could be folded
> in patch #14?
I would do a separate patch as IOREQ only really work after the full
series applies.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
2021-01-27 17:42 ` Julien Grall
@ 2021-01-27 17:45 ` Oleksandr
2021-01-28 14:37 ` Oleksandr
0 siblings, 1 reply; 93+ messages in thread
From: Oleksandr @ 2021-01-27 17:45 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Andrew Cooper, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Tim Deegan,
Daniel De Graaf, Volodymyr Babchuk, Bertrand Marquis, Wei Chen,
Kaly Xin, Artem Mygaiev, Alex Bennée
On 27.01.21 19:42, Julien Grall wrote:
Hi
>
>
> On 27/01/2021 17:37, Oleksandr wrote:
>>
>> On 27.01.21 19:33, Julien Grall wrote:
>>
>> Hi Julien
>>
>>>
>>>
>>> On 27/01/2021 16:50, Oleksandr wrote:
>>>>
>>>> On 27.01.21 18:43, Julien Grall wrote:
>>>>> Hi Oleksandr,
>>>>
>>>> Hi Julien
>>>>
>>>>
>>>>>
>>>>> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>>>>>> ***
>>>>>>
>>>>>> Patch series [8] was rebased on recent "staging branch"
>>>>>> (5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is
>>>>>> unmapped) and tested on
>>>>>> Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio
>>>>>> disk backend [9]
>>>>>> running in driver domain and unmodified Linux Guest running on
>>>>>> existing
>>>>>> virtio-blk driver (frontend). No issues were observed. Guest
>>>>>> domain 'reboot/destroy'
>>>>>> use-cases work properly. Patch series was only build-tested on x86.
>>>>>>
>>>>>> Please note, build-test passed for the following modes:
>>>>>> 1. x86: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y (default)
>>>>>> 2. x86: #CONFIG_HVM is not set / #CONFIG_IOREQ_SERVER is not set
>>>>>> 3. Arm64: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
>>>>>> 4. Arm64: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
>>>>>> 5. Arm32: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
>>>>>> 6. Arm32: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
>>>>>
>>>>> I thought I woudl give a try to test the code, but I can't find a
>>>>> way to enable CONFIG_IOREQ_SERVER from the UI.
>>>>>
>>>>> Looking at the Kconfig, it looks like CONFIG_IOREQ_SERVER doesn't
>>>>> have a prompt and is not selected by Arm.
>>>>>
>>>>> Can you provide details how this can be built on Arm?
>>>>
>>>> Please apply the attached patch to select IOREQ on Arm.
>>>
>>> This is roughly what I wrote. I think a user should be able to
>>> select IOREQ via the menuconfig without any additional patch on top
>>> of your series.
>>>
>>> Can you include a patch that would enable that?
>>
>> Yes, do you prefer a separate patch or required changes could be
>> folded in patch #14?
>
> I would do a separate patch as IOREQ only really work after the full
> series applies.
Makes sense, I will do it for V6
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2021-01-25 19:08 ` [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features Oleksandr Tyshchenko
2021-01-25 20:19 ` Stefano Stabellini
2021-01-26 9:15 ` Jan Beulich
@ 2021-01-27 18:33 ` Julien Grall
2021-01-27 19:20 ` Oleksandr
2 siblings, 1 reply; 93+ messages in thread
From: Julien Grall @ 2021-01-27 18:33 UTC (permalink / raw)
To: Oleksandr Tyshchenko, xen-devel
Cc: Julien Grall, Stefano Stabellini, Volodymyr Babchuk,
Andrew Cooper, George Dunlap, Ian Jackson, Jan Beulich, Wei Liu,
Oleksandr Tyshchenko
Hi,
On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
> +enum io_state try_fwd_ioserv(struct cpu_user_regs *regs,
> + struct vcpu *v, mmio_info_t *info)
> +{
> + struct vcpu_io *vio = &v->io;
> + ioreq_t p = {
> + .type = IOREQ_TYPE_COPY,
> + .addr = info->gpa,
> + .size = 1 << info->dabt.size,
> + .count = 1,
> + .dir = !info->dabt.write,
> + /*
> + * On x86, df is used by 'rep' instruction to tell the direction
> + * to iterate (forward or backward).
> + * On Arm, all the accesses to MMIO region will do a single
> + * memory access. So for now, we can safely always set to 0.
> + */
> + .df = 0,
> + .data = get_user_reg(regs, info->dabt.reg),
> + .state = STATE_IOREQ_READY,
> + };
> + struct ioreq_server *s = NULL;
> + enum io_state rc;
> +
> + switch ( vio->req.state )
> + {
> + case STATE_IOREQ_NONE:
> + break;
> +
> + default:
> + gdprintk(XENLOG_ERR, "wrong state %u\n", vio->req.state);
> + return IO_ABORT;
> + }
NIT: Given there is only one case, I think this can become a:
if ( vio->req.state != STATE_IOREQ_NONE )
{
gdprintk(...);
return IO_ABORT;
}
It is up to you whether you want to fix it now, later, or never :).
Aside the discussion about dm.h, the rest of the code looks good to me.
It is nice to see the arch part of IOREQ is small :).
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common
2021-01-25 23:40 ` Oleksandr
@ 2021-01-27 18:43 ` Julien Grall
2021-01-27 20:22 ` Oleksandr
0 siblings, 1 reply; 93+ messages in thread
From: Julien Grall @ 2021-01-27 18:43 UTC (permalink / raw)
To: Oleksandr, Julien Grall
Cc: xen-devel, Oleksandr Tyshchenko, Andrew Cooper, George Dunlap,
Ian Jackson, Jan Beulich, Stefano Stabellini, Wei Liu,
Roger Pau Monné,
Paul Durrant, Jun Nakajima, Kevin Tian, Tim Deegan, Julien Grall
On 25/01/2021 23:40, Oleksandr wrote:
>
> On 26.01.21 01:13, Julien Grall wrote:
>> Hi,
>
> Hi Julien
Hi Oleksandr,
>
>>
>> On Mon, 25 Jan 2021 at 19:09, Oleksandr Tyshchenko
>> <olekstysh@gmail.com> wrote:
>>> ***
>>> Please note, this patch depends on the following which is
>>> on review:
>>> https://patchwork.kernel.org/patch/11816689/
>>> The effort (to get it upstreamed) was paused because of
>>> the security issue around that code (XSA-348).
>>> ***
>> I read this comment as "This series should be applied on top the patch
>> X". However, looking at your branch, I can't see the patch. What did I
>> miss?
> You didn't miss anything. Patch series doesn't contain it. I mentioned
> about this patch in order not to forget about it
> and draw reviewer's attention. Looks like, the activity (to get it
> upstreamed) hasn't been resumed yet and I don't know what we should do
> with that dependency
> in the context of this series...
I will reply what I wrote on IRC privately :). I think this fine to
defer it to after Xen 4.15 because IOREQ will be in tech preview.
Note that the bug can only be hit if the IOREQ server disappear behind
your back. AFAICT, this is mostly an issue with device model running in
non-trusted domain.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2021-01-27 18:33 ` Julien Grall
@ 2021-01-27 19:20 ` Oleksandr
2021-01-28 9:40 ` Julien Grall
0 siblings, 1 reply; 93+ messages in thread
From: Oleksandr @ 2021-01-27 19:20 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, Julien Grall, Stefano Stabellini, Volodymyr Babchuk,
Andrew Cooper, George Dunlap, Ian Jackson, Jan Beulich, Wei Liu,
Oleksandr Tyshchenko
On 27.01.21 20:33, Julien Grall wrote:
Hi Julien
> Hi,
>
> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>> +enum io_state try_fwd_ioserv(struct cpu_user_regs *regs,
>> + struct vcpu *v, mmio_info_t *info)
>> +{
>> + struct vcpu_io *vio = &v->io;
>> + ioreq_t p = {
>> + .type = IOREQ_TYPE_COPY,
>> + .addr = info->gpa,
>> + .size = 1 << info->dabt.size,
>> + .count = 1,
>> + .dir = !info->dabt.write,
>> + /*
>> + * On x86, df is used by 'rep' instruction to tell the
>> direction
>> + * to iterate (forward or backward).
>> + * On Arm, all the accesses to MMIO region will do a single
>> + * memory access. So for now, we can safely always set to 0.
>> + */
>> + .df = 0,
>> + .data = get_user_reg(regs, info->dabt.reg),
>> + .state = STATE_IOREQ_READY,
>> + };
>> + struct ioreq_server *s = NULL;
>> + enum io_state rc;
>> +
>> + switch ( vio->req.state )
>> + {
>> + case STATE_IOREQ_NONE:
>> + break;
>> +
>> + default:
>> + gdprintk(XENLOG_ERR, "wrong state %u\n", vio->req.state);
>> + return IO_ABORT;
>> + }
>
> NIT: Given there is only one case, I think this can become a:
>
> if ( vio->req.state != STATE_IOREQ_NONE )
> {
> gdprintk(...);
> return IO_ABORT;
> }
>
> It is up to you whether you want to fix it now, later, or never :).
V6 is planned, so will fix)
>
> Aside the discussion about dm.h, the rest of the code looks good to
> me. It is nice to see the arch part of IOREQ is small :).
I do agree, it is much smaller than it was for RFC series)
Yes, I will add required changes to dm.h in the patch which introduces
that header, but ...
>>> So I think we may be able to drop the include from asm/hvm/domain.h
(this would avoid to include it everywhere...).
I have tried that, but other CUs use definitions from
public/hvm/dm_op.h, for example:
p2m-pt.c: In function 'p2m_type_to_flags':
p2m-pt.c:87:33: error: 'XEN_DMOP_IOREQ_MEM_ACCESS_WRITE' undeclared
(first use in this function)
if ( p2m->ioreq.flags & XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
^
So, I would prefer to leave it as is, please let me know if you think
otherwise.
>
> Cheers,
>
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common
2021-01-27 16:58 ` Jan Beulich
@ 2021-01-27 20:14 ` Oleksandr
2021-01-28 8:06 ` Jan Beulich
0 siblings, 1 reply; 93+ messages in thread
From: Oleksandr @ 2021-01-27 20:14 UTC (permalink / raw)
To: Jan Beulich
Cc: Oleksandr Tyshchenko, Andrew Cooper, George Dunlap, Ian Jackson,
Julien Grall, Stefano Stabellini, Wei Liu, Roger Pau Monné,
Paul Durrant, Jun Nakajima, Kevin Tian, Tim Deegan, Julien Grall,
xen-devel
On 27.01.21 18:58, Jan Beulich wrote:
Hi Jan
> On 25.01.2021 20:08, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> As a lot of x86 code can be re-used on Arm later on, this patch
>> moves previously prepared IOREQ support to the common code
>> (the code movement is verbatim copy).
>>
>> The "legacy" mechanism of mapping magic pages for the IOREQ servers
>> remains x86 specific and not exposed to the common code.
>>
>> The common IOREQ feature is supposed to be built with IOREQ_SERVER
>> option enabled, which is selected for x86's config HVM for now.
>>
>> In order to avoid having a gigantic patch here, the subsequent
>> patches will update remaining bits in the common code step by step:
>> - Make IOREQ related structs/materials common
>> - Drop the "hvm" prefixes and infixes
>> - Remove layering violation by moving corresponding fields
>> out of *arch.hvm* or abstracting away accesses to them
>>
>> Introduce asm/ioreq.h wrapper to be included by common ioreq.h
>> instead of asm/hvm/ioreq.h to avoid HVM-ism in the code common.
>>
>> Also include <xen/domain_page.h> which will be needed on Arm
>> to avoid touch the common code again when introducing Arm specific bits.
>>
>> This support is going to be used on Arm to be able run device
>> emulator outside of Xen hypervisor.
>>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> Reviewed-by: Paul Durrant <paul@xen.org>
>> CC: Julien Grall <julien.grall@arm.com>
>> [On Arm only]
>> Tested-by: Wei Chen <Wei.Chen@arm.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>
Thanks
> with ...
>
>> --- a/xen/arch/x86/Kconfig
>> +++ b/xen/arch/x86/Kconfig
>> @@ -92,6 +92,7 @@ config PV_LINEAR_PT
>>
>> config HVM
>> def_bool !PV_SHIM_EXCLUSIVE
>> + select IOREQ_SERVER
>> prompt "HVM support"
>> ---help---
> ... the addition moved below the prompt line (could probably
> be taken care of while committing, if no other need for a v6
> arises).
V6 is planned anyway, so will do, but ...
>
> (Personally I think this should be
>
> config HVM
> bool "HVM support"
> default !PV_SHIM_EXCLUSIVE
... def_bool is changed to default by intention or this is a typo?
> select IOREQ_SERVER
> ---help---
>
> anyway, but that's nothing you need to care about.)
>
>> --- /dev/null
>> +++ b/xen/include/asm-x86/ioreq.h
>> @@ -0,0 +1,37 @@
>> +/*
>> + * ioreq.h: Hardware virtual machine assist interface definitions.
>> + *
>> + * This is a wrapper which purpose is to not include arch HVM specific header
>> + * from the common code.
>> + *
>> + * Copyright (c) 2016 Citrix Systems Inc.
>> + *
>> + * This program is free software; you can redistribute it and/or modify it
>> + * under the terms and conditions of the GNU General Public License,
>> + * version 2, as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope it will be useful, but WITHOUT
>> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
>> + * more details.
>> + *
>> + * You should have received a copy of the GNU General Public License along with
>> + * this program; If not, see <http://www.gnu.org/licenses/>.
>> + */
>> +
>> +#ifndef __ASM_X86_IOREQ_H__
>> +#define __ASM_X86_IOREQ_H__
>> +
>> +#include <asm/hvm/ioreq.h>
>> +
>> +#endif /* __ASM_X86_IOREQ_H__ */
> Not necessarily for taking care of right away, I think in the
> longer run this wants wrapping by #ifdef CONFIG_HVM, such that
> in !HVM builds the dependency on the "chained" header goes
> away (reducing the amount of rebuilding in incremental builds).
I don't mind wrapping it right away.
>
> Jan
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common
2021-01-27 18:43 ` Julien Grall
@ 2021-01-27 20:22 ` Oleksandr
2021-01-27 20:46 ` Stefano Stabellini
0 siblings, 1 reply; 93+ messages in thread
From: Oleksandr @ 2021-01-27 20:22 UTC (permalink / raw)
To: Julien Grall, Julien Grall
Cc: xen-devel, Oleksandr Tyshchenko, Andrew Cooper, George Dunlap,
Ian Jackson, Jan Beulich, Stefano Stabellini, Wei Liu,
Roger Pau Monné,
Paul Durrant, Jun Nakajima, Kevin Tian, Tim Deegan, Julien Grall
On 27.01.21 20:43, Julien Grall wrote:
>
>
> On 25/01/2021 23:40, Oleksandr wrote:
>>
>> On 26.01.21 01:13, Julien Grall wrote:
>>> Hi,
>>
>> Hi Julien
>
> Hi Oleksandr,
Hi Julien
>
>>
>>>
>>> On Mon, 25 Jan 2021 at 19:09, Oleksandr Tyshchenko
>>> <olekstysh@gmail.com> wrote:
>>>> ***
>>>> Please note, this patch depends on the following which is
>>>> on review:
>>>> https://patchwork.kernel.org/patch/11816689/
>>>> The effort (to get it upstreamed) was paused because of
>>>> the security issue around that code (XSA-348).
>>>> ***
>>> I read this comment as "This series should be applied on top the patch
>>> X". However, looking at your branch, I can't see the patch. What did I
>>> miss?
>> You didn't miss anything. Patch series doesn't contain it. I
>> mentioned about this patch in order not to forget about it
>> and draw reviewer's attention. Looks like, the activity (to get it
>> upstreamed) hasn't been resumed yet and I don't know what we should
>> do with that dependency
>> in the context of this series...
>
> I will reply what I wrote on IRC privately :). I think this fine to
> defer it to after Xen 4.15 because IOREQ will be in tech preview.
Thank you. I got a request to make a possibility for user to select
IOREQ via the menuconfig on Arm. Saying tech preview do you mean that I
also need to put it under CONFIG_EXPERT on Arm?
>
> Note that the bug can only be hit if the IOREQ server disappear behind
> your back. AFAICT, this is mostly an issue with device model running
> in non-trusted domain.
I got it.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 15/22] xen/arm: Call vcpu_ioreq_handle_completion() in check_for_vcpu_work()
2021-01-27 15:56 ` Oleksandr
@ 2021-01-27 20:34 ` Stefano Stabellini
0 siblings, 0 replies; 93+ messages in thread
From: Stefano Stabellini @ 2021-01-27 20:34 UTC (permalink / raw)
To: Oleksandr
Cc: Julien Grall, Stefano Stabellini, xen-devel,
Oleksandr Tyshchenko, Volodymyr Babchuk, Julien Grall
[-- Attachment #1: Type: text/plain, Size: 2938 bytes --]
On Wed, 27 Jan 2021, Oleksandr wrote:
> On 27.01.21 16:49, Julien Grall wrote:
> > Hi Oleksandr,
>
> Hi Julien, Stefano
>
>
> >
> > On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
> > > From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> > >
> > > This patch adds remaining bits needed for the IOREQ support on Arm.
> > > Besides just calling vcpu_ioreq_handle_completion() we need to handle
> > > it's return value to make sure that all the vCPU works are done before
> > > we return to the guest (the vcpu_ioreq_handle_completion() may return
> > > false if there is vCPU work to do or IOREQ state is invalid).
> > > For that reason we use an unbounded loop in leave_hypervisor_to_guest().
> > >
> > > The worse that can happen here if the vCPU will never run again
> > > (the I/O will never complete). But, in Xen case, if the I/O never
> > > completes then it most likely means that something went horribly
> > > wrong with the Device Emulator. And it is most likely not safe
> > > to continue. So letting the vCPU to spin forever if the I/O never
> > > completes is a safer action than letting it continue and leaving
> > > the guest in unclear state and is the best what we can do for now.
> > >
> > > Please note, using this loop we will not spin forever on a pCPU,
> > > preventing any other vCPUs from being scheduled. At every loop
> > > we will call check_for_pcpu_work() that will process pending
> > > softirqs. In case of failure, the guest will crash and the vCPU
> > > will be unscheduled. In normal case, if the rescheduling is necessary
> > > the vCPU will be rescheduled to give place to someone else.
> > >
> > > Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> > > CC: Julien Grall <julien.grall@arm.com>
> > > Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> > > [On Arm only]
> > > Tested-by: Wei Chen <Wei.Chen@arm.com>
> > >
> > > ---
> > > Please note, this is a split/cleanup/hardening of Julien's PoC:
> > > "Add support for Guest IO forwarding to a device emulator"
> > >
> > > Changes V1 -> V2:
> > > - new patch, changes were derived from (+ new explanation):
> > > arm/ioreq: Introduce arch specific bits for IOREQ/DM features
> > >
> > > Changes V2 -> V3:
> > > - update patch description
> > >
> > > Changes V3 -> V4:
> > > - update patch description and comment in code
> > >
> > > Changes V4 -> V5:
> > > - add Stefano's R-b
> >
> > Reviewed-by means the person reviewed the code and confirmed it is correct.
> > Given the changes you made below, I don't think this tag can hold.
> >
> > Please confirm with Stefano he is happy with the tag to be carried.
>
> I think you are right, sorry for that. I should have either clarified this
> question with Stefano in advance or dropped this tag.
>
> @Stefano, are you happy with the changes for V4 -> V5 (would you mind if your
> R-b still stands)?
Yes, I am.
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common
2021-01-27 20:22 ` Oleksandr
@ 2021-01-27 20:46 ` Stefano Stabellini
2021-01-28 11:01 ` Oleksandr
0 siblings, 1 reply; 93+ messages in thread
From: Stefano Stabellini @ 2021-01-27 20:46 UTC (permalink / raw)
To: Oleksandr
Cc: Julien Grall, Julien Grall, xen-devel, Oleksandr Tyshchenko,
Andrew Cooper, George Dunlap, Ian Jackson, Jan Beulich,
Stefano Stabellini, Wei Liu, Roger Pau Monné,
Paul Durrant, Jun Nakajima, Kevin Tian, Tim Deegan, Julien Grall
On Wed, 27 Jan 2021, Oleksandr wrote:
> > > > On Mon, 25 Jan 2021 at 19:09, Oleksandr Tyshchenko <olekstysh@gmail.com>
> > > > wrote:
> > > > > ***
> > > > > Please note, this patch depends on the following which is
> > > > > on review:
> > > > > https://patchwork.kernel.org/patch/11816689/
> > > > > The effort (to get it upstreamed) was paused because of
> > > > > the security issue around that code (XSA-348).
> > > > > ***
> > > > I read this comment as "This series should be applied on top the patch
> > > > X". However, looking at your branch, I can't see the patch. What did I
> > > > miss?
> > > You didn't miss anything. Patch series doesn't contain it. I mentioned
> > > about this patch in order not to forget about it
> > > and draw reviewer's attention. Looks like, the activity (to get it
> > > upstreamed) hasn't been resumed yet and I don't know what we should do
> > > with that dependency
> > > in the context of this series...
> >
> > I will reply what I wrote on IRC privately :). I think this fine to defer it
> > to after Xen 4.15 because IOREQ will be in tech preview.
>
> Thank you. I got a request to make a possibility for user to select IOREQ via
> the menuconfig on Arm. Saying tech preview do you mean that I also need to put
> it under CONFIG_EXPERT on Arm?
Normally I would say that unless it actually takes an expert to enable
the feature, it is better to make it depend on CONFIG_UNSUPPORTED [1].
However, in this case, it might actually take an expert :-)
Additional patches are still required to enable the feature at the
toolstack level, and also the user needs to build a userspace ioreq
server. So in this case I am fine either way. I'll leave it up to you
and Julien to pick the best one.
Either way, please add "(EXPERT)" or "(UNSUPPORTED)" in the one-line
kconfig description for ARM if possible.
[1] https://marc.info/?l=xen-devel&m=161168780401884
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common
2021-01-27 20:14 ` Oleksandr
@ 2021-01-28 8:06 ` Jan Beulich
2021-01-28 11:16 ` Oleksandr
0 siblings, 1 reply; 93+ messages in thread
From: Jan Beulich @ 2021-01-28 8:06 UTC (permalink / raw)
To: Oleksandr
Cc: Oleksandr Tyshchenko, Andrew Cooper, George Dunlap, Ian Jackson,
Julien Grall, Stefano Stabellini, Wei Liu, Roger Pau Monné,
Paul Durrant, Jun Nakajima, Kevin Tian, Tim Deegan, Julien Grall,
xen-devel
On 27.01.2021 21:14, Oleksandr wrote:
> On 27.01.21 18:58, Jan Beulich wrote:
>> On 25.01.2021 20:08, Oleksandr Tyshchenko wrote:
>>> --- a/xen/arch/x86/Kconfig
>>> +++ b/xen/arch/x86/Kconfig
>>> @@ -92,6 +92,7 @@ config PV_LINEAR_PT
>>>
>>> config HVM
>>> def_bool !PV_SHIM_EXCLUSIVE
>>> + select IOREQ_SERVER
>>> prompt "HVM support"
>>> ---help---
>> ... the addition moved below the prompt line (could probably
>> be taken care of while committing, if no other need for a v6
>> arises).
>
> V6 is planned anyway, so will do, but ...
>
>
>>
>> (Personally I think this should be
>>
>> config HVM
>> bool "HVM support"
>> default !PV_SHIM_EXCLUSIVE
>
> ... def_bool is changed to default by intention or this is a typo?
No, it's not a typo. "def_bool" is just a shorthand for "bool"
and "default", which is useful when there's no prompt, but
gets abused (in my view at least) in a number of other cases.
But as said ...
>> anyway, but that's nothing you need to care about.)
... here.
>>> --- /dev/null
>>> +++ b/xen/include/asm-x86/ioreq.h
>>> @@ -0,0 +1,37 @@
>>> +/*
>>> + * ioreq.h: Hardware virtual machine assist interface definitions.
>>> + *
>>> + * This is a wrapper which purpose is to not include arch HVM specific header
>>> + * from the common code.
>>> + *
>>> + * Copyright (c) 2016 Citrix Systems Inc.
>>> + *
>>> + * This program is free software; you can redistribute it and/or modify it
>>> + * under the terms and conditions of the GNU General Public License,
>>> + * version 2, as published by the Free Software Foundation.
>>> + *
>>> + * This program is distributed in the hope it will be useful, but WITHOUT
>>> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
>>> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
>>> + * more details.
>>> + *
>>> + * You should have received a copy of the GNU General Public License along with
>>> + * this program; If not, see <http://www.gnu.org/licenses/>.
>>> + */
>>> +
>>> +#ifndef __ASM_X86_IOREQ_H__
>>> +#define __ASM_X86_IOREQ_H__
>>> +
>>> +#include <asm/hvm/ioreq.h>
>>> +
>>> +#endif /* __ASM_X86_IOREQ_H__ */
>> Not necessarily for taking care of right away, I think in the
>> longer run this wants wrapping by #ifdef CONFIG_HVM, such that
>> in !HVM builds the dependency on the "chained" header goes
>> away (reducing the amount of rebuilding in incremental builds).
>
> I don't mind wrapping it right away.
Well, if that doesn't break the !HVM build, I'd of course
appreciate it. I'd expect fallout, though.
Jan
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2021-01-27 19:20 ` Oleksandr
@ 2021-01-28 9:40 ` Julien Grall
2021-01-28 11:33 ` Oleksandr
0 siblings, 1 reply; 93+ messages in thread
From: Julien Grall @ 2021-01-28 9:40 UTC (permalink / raw)
To: Oleksandr
Cc: xen-devel, Julien Grall, Stefano Stabellini, Volodymyr Babchuk,
Andrew Cooper, George Dunlap, Ian Jackson, Jan Beulich, Wei Liu,
Oleksandr Tyshchenko
Hi Oleksandr,
On 27/01/2021 19:20, Oleksandr wrote:
> >>> So I think we may be able to drop the include from
asm/hvm/domain.h
> (this would avoid to include it everywhere...).
>
> I have tried that, but other CUs use definitions from
> public/hvm/dm_op.h, for example:
>
> p2m-pt.c: In function 'p2m_type_to_flags':
> p2m-pt.c:87:33: error: 'XEN_DMOP_IOREQ_MEM_ACCESS_WRITE' undeclared
> (first use in this function)
> if ( p2m->ioreq.flags & XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
> ^
> So, I would prefer to leave it as is, please let me know if you think
> otherwise.
AFAICT, there is only 2 places (p2m-pt.c and p2m-ept.c) that requires
<public/hvm/dm_op.h> but doesn't directly include it. Folding the diff
below in patch #4 should do the job:
diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
index 975ab403f235..23d411f01d2f 100644
--- a/xen/arch/x86/mm/p2m-ept.c
+++ b/xen/arch/x86/mm/p2m-ept.c
@@ -17,6 +17,7 @@
#include <xen/domain_page.h>
#include <xen/sched.h>
+#include <public/hvm/dm_op.h>
#include <asm/altp2m.h>
#include <asm/current.h>
#include <asm/paging.h>
diff --git a/xen/arch/x86/mm/p2m-pt.c b/xen/arch/x86/mm/p2m-pt.c
index c43d5d0413a1..f2afcf49a368 100644
--- a/xen/arch/x86/mm/p2m-pt.c
+++ b/xen/arch/x86/mm/p2m-pt.c
@@ -27,6 +27,7 @@
#include <xen/vm_event.h>
#include <xen/event.h>
#include <xen/trace.h>
+#include <public/hvm/dm_op.h>
#include <public/vm_event.h>
#include <asm/altp2m.h>
#include <asm/domain.h>
diff --git a/xen/include/asm-x86/hvm/domain.h
b/xen/include/asm-x86/hvm/domain.h
index 3b36c2f41fa1..f26c1a2e2d5f 100644
--- a/xen/include/asm-x86/hvm/domain.h
+++ b/xen/include/asm-x86/hvm/domain.h
@@ -28,8 +28,6 @@
#include <asm/hvm/vmx/vmcs.h>
#include <asm/hvm/svm/vmcb.h>
-#include <public/hvm/dm_op.h>
-
#ifdef CONFIG_MEM_SHARING
struct mem_sharing_domain
{
You would then need to move the include of <public/hvm/dm_op.h> in
<xen/dm.h> from this patch to patch #9.
Cheers,
--
Julien Grall
^ permalink raw reply related [flat|nested] 93+ messages in thread
* Re: [PATCH V5 22/22] xen/arm: Add mapcache invalidation handling
2021-01-25 19:08 ` [PATCH V5 22/22] xen/arm: Add mapcache invalidation handling Oleksandr Tyshchenko
@ 2021-01-28 9:55 ` Julien Grall
2021-01-28 13:12 ` Oleksandr
0 siblings, 1 reply; 93+ messages in thread
From: Julien Grall @ 2021-01-28 9:55 UTC (permalink / raw)
To: Oleksandr Tyshchenko, xen-devel
Cc: Oleksandr Tyshchenko, Stefano Stabellini, Volodymyr Babchuk,
Julien Grall
Hi Oleksandr,
On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> We need to send mapcache invalidation request to qemu/demu everytime
> the page gets removed from a guest.
>
> At the moment, the Arm code doesn't explicitely remove the existing
> mapping before inserting the new mapping. Instead, this is done
> implicitely by __p2m_set_entry().
>
> So we need to recognize a case when old entry is a RAM page *and*
> the new MFN is different in order to set the corresponding flag.
AFAICT, this sentence doesn't match the code. Instead, you will request
the mapcache invalidation whenever the "freed" entry contains some RAM page.
I am fine with the approach used in the code. However, it would be good
that the commit message reflects the code.
> The most suitable place to do this is p2m_free_entry(), there
> we can find the correct leaf type. The invalidation request
> will be sent in do_trap_hypercall() later on.
>
> Taking into the account the following the do_trap_hypercall()
> is the best place to send invalidation request:
> - The only way a guest can modify its P2M on Arm is via an hypercall
> - When sending the invalidation request, the vCPU will be blocked
> until all the IOREQ servers have acknowledged the invalidation
>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> CC: Julien Grall <julien.grall@arm.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> [On Arm only]
> Tested-by: Wei Chen <Wei.Chen@arm.com>
>
> ---
> Please note, this is a split/cleanup/hardening of Julien's PoC:
> "Add support for Guest IO forwarding to a device emulator"
>
> ***
> Please note, this patch depends on the following which is
> on review:
> https://patchwork.kernel.org/patch/11803383/
I think we can defer this post this series.
>
> This patch is on par with x86 code (whether it is buggy or not).
> If there is a need to improve/harden something, this can be done on
> a follow-up.
> ***
>
> Changes V1 -> V2:
> - new patch, some changes were derived from (+ new explanation):
> xen/ioreq: Make x86's invalidate qemu mapcache handling common
> - put setting of the flag into __p2m_set_entry()
> - clarify the conditions when the flag should be set
> - use domain_has_ioreq_server()
> - update do_trap_hypercall() by adding local variable
>
> Changes V2 -> V3:
> - update patch description
> - move check to p2m_free_entry()
> - add a comment
> - use "curr" instead of "v" in do_trap_hypercall()
>
> Changes V3 -> V4:
> - update patch description
> - re-order check in p2m_free_entry() to call domain_has_ioreq_server()
> only if p2m->domain == current->domain
> - add a comment in do_trap_hypercall()
>
> Changes V4 -> V5:
> - add Stefano's R-b
> - update comment in do_trap_hypercall()
> ---
> ---
> xen/arch/arm/p2m.c | 25 +++++++++++++++++--------
> xen/arch/arm/traps.c | 20 +++++++++++++++++---
> 2 files changed, 34 insertions(+), 11 deletions(-)
>
> diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
> index d41c4fa..26acb95d 100644
> --- a/xen/arch/arm/p2m.c
> +++ b/xen/arch/arm/p2m.c
> @@ -1,6 +1,7 @@
> #include <xen/cpu.h>
> #include <xen/domain_page.h>
> #include <xen/iocap.h>
> +#include <xen/ioreq.h>
> #include <xen/lib.h>
> #include <xen/sched.h>
> #include <xen/softirq.h>
> @@ -749,17 +750,25 @@ static void p2m_free_entry(struct p2m_domain *p2m,
> if ( !p2m_is_valid(entry) )
> return;
>
> - /* Nothing to do but updating the stats if the entry is a super-page. */
> - if ( p2m_is_superpage(entry, level) )
> + if ( p2m_is_superpage(entry, level) || (level == 3) )
> {
> - p2m->stats.mappings[level]--;
> - return;
> - }
> +#ifdef CONFIG_IOREQ_SERVER
> + /*
> + * If this gets called (non-recursively) then either the entry
I am not sure why you specify "non-recursively". You still want to
invalidate the mapcache if we replaced a table with a superpage mapping.
> + * was replaced by an entry with a different base (valid case) or
> + * the shattering of a superpage was failed (error case).
s/was/has/ I think.
> + * So, at worst, the spurious mapcache invalidation might be sent.
> + */
> + if ( (p2m->domain == current->domain) &&
> + domain_has_ioreq_server(p2m->domain) &&
> + p2m_is_ram(entry.p2m.type) )
> + p2m->domain->mapcache_invalidate = true;
> +#endif
>
> - if ( level == 3 )
> - {
> p2m->stats.mappings[level]--;
> - p2m_put_l3_page(entry);
> + /* Nothing to do if the entry is a super-page. */
> + if ( level == 3 )
> + p2m_put_l3_page(entry);
> return;
> }
>
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index 4cdd343..64b740b 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -1443,6 +1443,7 @@ static void do_trap_hypercall(struct cpu_user_regs *regs, register_t *nr,
> const union hsr hsr)
> {
> arm_hypercall_fn_t call = NULL;
> + struct vcpu *curr = current;
>
> BUILD_BUG_ON(NR_hypercalls < ARRAY_SIZE(arm_hypercall_table) );
>
> @@ -1459,7 +1460,7 @@ static void do_trap_hypercall(struct cpu_user_regs *regs, register_t *nr,
> return;
> }
>
> - current->hcall_preempted = false;
> + curr->hcall_preempted = false;
>
> perfc_incra(hypercalls, *nr);
> call = arm_hypercall_table[*nr].fn;
> @@ -1472,7 +1473,7 @@ static void do_trap_hypercall(struct cpu_user_regs *regs, register_t *nr,
> HYPERCALL_RESULT_REG(regs) = call(HYPERCALL_ARGS(regs));
>
> #ifndef NDEBUG
> - if ( !current->hcall_preempted )
> + if ( !curr->hcall_preempted )
> {
> /* Deliberately corrupt parameter regs used by this hypercall. */
> switch ( arm_hypercall_table[*nr].nr_args ) {
> @@ -1489,8 +1490,21 @@ static void do_trap_hypercall(struct cpu_user_regs *regs, register_t *nr,
> #endif
>
> /* Ensure the hypercall trap instruction is re-executed. */
> - if ( current->hcall_preempted )
> + if ( curr->hcall_preempted )
> regs->pc -= 4; /* re-execute 'hvc #XEN_HYPERCALL_TAG' */
> +
> +#ifdef CONFIG_IOREQ_SERVER
> + /*
> + * We call ioreq_signal_mapcache_invalidate from do_trap_hypercall()
> + * because the only way a guest can modify its P2M on Arm is via an
> + * hypercall.
> + * Note that sending the invalidation request causes the vCPU to block
> + * until all the IOREQ servers have acknowledged the invalidation.
> + */
> + if ( unlikely(curr->domain->mapcache_invalidate) &&
> + test_and_clear_bool(curr->domain->mapcache_invalidate) )
> + ioreq_signal_mapcache_invalidate();
> +#endif
> }
>
> void arch_hypercall_tasklet_result(struct vcpu *v, long res)
>
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 09/22] xen/ioreq: Make x86's IOREQ related dm-op handling common
2021-01-25 19:08 ` [PATCH V5 09/22] xen/ioreq: Make x86's IOREQ related dm-op handling common Oleksandr Tyshchenko
2021-01-26 13:31 ` Paul Durrant
@ 2021-01-28 10:52 ` Jan Beulich
2021-01-28 12:06 ` Oleksandr
1 sibling, 1 reply; 93+ messages in thread
From: Jan Beulich @ 2021-01-28 10:52 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: Julien Grall, Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Paul Durrant, Daniel De Graaf,
Oleksandr Tyshchenko, xen-devel
On 25.01.2021 20:08, Oleksandr Tyshchenko wrote:
> --- /dev/null
> +++ b/xen/include/xen/dm.h
> @@ -0,0 +1,41 @@
> +/*
> + * Copyright (c) 2016 Citrix Systems Inc.
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms and conditions of the GNU General Public License,
> + * version 2, as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
> + * more details.
> + *
> + * You should have received a copy of the GNU General Public License along with
> + * this program; If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#ifndef __XEN_DM_H__
> +#define __XEN_DM_H__
> +
> +#include <xen/sched.h>
> +
> +struct dmop_args {
> + domid_t domid;
> + unsigned int nr_bufs;
> + /* Reserve enough buf elements for all current hypercalls. */
> + struct xen_dm_op_buf buf[2];
So this is then the patch where the public header needs including,
to satisfy this use of a struct declared there independent of what
xen/sched.h includes. In fact public/xen.h needs including here
as well, I think, for domid_t. Otoh I can't see why you include
xen/sched.h.
My previously given ack applies only with this properly sorted.
Jan
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common
2021-01-27 20:46 ` Stefano Stabellini
@ 2021-01-28 11:01 ` Oleksandr
2021-01-28 11:21 ` Jan Beulich
0 siblings, 1 reply; 93+ messages in thread
From: Oleksandr @ 2021-01-28 11:01 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall, Jan Beulich
Cc: Julien Grall, xen-devel, Oleksandr Tyshchenko, Andrew Cooper,
George Dunlap, Ian Jackson, Wei Liu, Roger Pau Monné,
Paul Durrant, Jun Nakajima, Kevin Tian, Tim Deegan, Julien Grall
On 27.01.21 22:46, Stefano Stabellini wrote:
Hi Stefano, all
> On Wed, 27 Jan 2021, Oleksandr wrote:
>>>>> On Mon, 25 Jan 2021 at 19:09, Oleksandr Tyshchenko <olekstysh@gmail.com>
>>>>> wrote:
>>>>>> ***
>>>>>> Please note, this patch depends on the following which is
>>>>>> on review:
>>>>>> https://patchwork.kernel.org/patch/11816689/
>>>>>> The effort (to get it upstreamed) was paused because of
>>>>>> the security issue around that code (XSA-348).
>>>>>> ***
>>>>> I read this comment as "This series should be applied on top the patch
>>>>> X". However, looking at your branch, I can't see the patch. What did I
>>>>> miss?
>>>> You didn't miss anything. Patch series doesn't contain it. I mentioned
>>>> about this patch in order not to forget about it
>>>> and draw reviewer's attention. Looks like, the activity (to get it
>>>> upstreamed) hasn't been resumed yet and I don't know what we should do
>>>> with that dependency
>>>> in the context of this series...
>>> I will reply what I wrote on IRC privately :). I think this fine to defer it
>>> to after Xen 4.15 because IOREQ will be in tech preview.
>> Thank you. I got a request to make a possibility for user to select IOREQ via
>> the menuconfig on Arm. Saying tech preview do you mean that I also need to put
>> it under CONFIG_EXPERT on Arm?
> Normally I would say that unless it actually takes an expert to enable
> the feature, it is better to make it depend on CONFIG_UNSUPPORTED [1].
>
> However, in this case, it might actually take an expert :-)
> Additional patches are still required to enable the feature at the
> toolstack level, and also the user needs to build a userspace ioreq
> server. So in this case I am fine either way. I'll leave it up to you
> and Julien to pick the best one.
>
> Either way, please add "(EXPERT)" or "(UNSUPPORTED)" in the one-line
> kconfig description for ARM if possible.
>
>
> [1] https://marc.info/?l=xen-devel&m=161168780401884
Thank you for the explanation, personally I would prefer EXPERT, but...
It would be really nice if we could agreed regarding that *common*
Kconfig option before I submit V6 (which would probably avoid me to send
V7 I hope).
Now I see it as following (please note, it is a subject for new patch in
this series, which should come last):
+++ b/xen/common/Kconfig
@@ -137,7 +137,13 @@ config HYPFS_CONFIG
want to hide the .config contents from dom0.
config IOREQ_SERVER
- bool
+ bool "IOREQ support" if EXPERT || X86
+ default X86
+ depends on HVM
+ ---help---
+ Enables generic mechanism for providing emulated devices to
the guests.
+
+ If unsure, say Y.
config KEXEC
bool "kexec support"
Looks like, it works this way for both archs, but for me it looks a bit ugly
I would appreciate your input on that.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common
2021-01-28 8:06 ` Jan Beulich
@ 2021-01-28 11:16 ` Oleksandr
2021-01-28 11:24 ` Jan Beulich
0 siblings, 1 reply; 93+ messages in thread
From: Oleksandr @ 2021-01-28 11:16 UTC (permalink / raw)
To: Jan Beulich
Cc: Oleksandr Tyshchenko, Andrew Cooper, George Dunlap, Ian Jackson,
Julien Grall, Stefano Stabellini, Wei Liu, Roger Pau Monné,
Paul Durrant, Jun Nakajima, Kevin Tian, Tim Deegan, Julien Grall,
xen-devel
On 28.01.21 10:06, Jan Beulich wrote:
Hi Jan
> On 27.01.2021 21:14, Oleksandr wrote:
>> On 27.01.21 18:58, Jan Beulich wrote:
>>> On 25.01.2021 20:08, Oleksandr Tyshchenko wrote:
>>>> --- a/xen/arch/x86/Kconfig
>>>> +++ b/xen/arch/x86/Kconfig
>>>> @@ -92,6 +92,7 @@ config PV_LINEAR_PT
>>>>
>>>> config HVM
>>>> def_bool !PV_SHIM_EXCLUSIVE
>>>> + select IOREQ_SERVER
>>>> prompt "HVM support"
>>>> ---help---
>>> ... the addition moved below the prompt line (could probably
>>> be taken care of while committing, if no other need for a v6
>>> arises).
>> V6 is planned anyway, so will do, but ...
>>
>>
>>> (Personally I think this should be
>>>
>>> config HVM
>>> bool "HVM support"
>>> default !PV_SHIM_EXCLUSIVE
>> ... def_bool is changed to default by intention or this is a typo?
> No, it's not a typo. "def_bool" is just a shorthand for "bool"
> and "default", which is useful when there's no prompt, but
> gets abused (in my view at least) in a number of other cases.
> But as said ...
>
>>> anyway, but that's nothing you need to care about.)
> ... here.
ok, well, thank you, but FYI playing with default instead of def_bool I
noticed the difference in behavior.
I don't understand why, but HVM option disappears from menuconfig
somehow... Or I do something completely wrong...
>
>>>> --- /dev/null
>>>> +++ b/xen/include/asm-x86/ioreq.h
>>>> @@ -0,0 +1,37 @@
>>>> +/*
>>>> + * ioreq.h: Hardware virtual machine assist interface definitions.
>>>> + *
>>>> + * This is a wrapper which purpose is to not include arch HVM specific header
>>>> + * from the common code.
>>>> + *
>>>> + * Copyright (c) 2016 Citrix Systems Inc.
>>>> + *
>>>> + * This program is free software; you can redistribute it and/or modify it
>>>> + * under the terms and conditions of the GNU General Public License,
>>>> + * version 2, as published by the Free Software Foundation.
>>>> + *
>>>> + * This program is distributed in the hope it will be useful, but WITHOUT
>>>> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
>>>> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
>>>> + * more details.
>>>> + *
>>>> + * You should have received a copy of the GNU General Public License along with
>>>> + * this program; If not, see <http://www.gnu.org/licenses/>.
>>>> + */
>>>> +
>>>> +#ifndef __ASM_X86_IOREQ_H__
>>>> +#define __ASM_X86_IOREQ_H__
>>>> +
>>>> +#include <asm/hvm/ioreq.h>
>>>> +
>>>> +#endif /* __ASM_X86_IOREQ_H__ */
>>> Not necessarily for taking care of right away, I think in the
>>> longer run this wants wrapping by #ifdef CONFIG_HVM, such that
>>> in !HVM builds the dependency on the "chained" header goes
>>> away (reducing the amount of rebuilding in incremental builds).
>> I don't mind wrapping it right away.
> Well, if that doesn't break the !HVM build, I'd of course
> appreciate it. I'd expect fallout, though.
Hmm, I didn't notice a fallout with that change when CONFIG_HVM is not set
+#ifdef CONFIG_HVM
#include <asm/hvm/ioreq.h>
+#endif
>
> Jan
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common
2021-01-28 11:01 ` Oleksandr
@ 2021-01-28 11:21 ` Jan Beulich
2021-01-28 12:49 ` Oleksandr
0 siblings, 1 reply; 93+ messages in thread
From: Jan Beulich @ 2021-01-28 11:21 UTC (permalink / raw)
To: Oleksandr
Cc: Julien Grall, xen-devel, Oleksandr Tyshchenko, Andrew Cooper,
George Dunlap, Ian Jackson, Wei Liu, Roger Pau Monné,
Paul Durrant, Jun Nakajima, Kevin Tian, Tim Deegan, Julien Grall,
Stefano Stabellini, Julien Grall
On 28.01.2021 12:01, Oleksandr wrote:
> On 27.01.21 22:46, Stefano Stabellini wrote:
>> On Wed, 27 Jan 2021, Oleksandr wrote:
>>> Thank you. I got a request to make a possibility for user to select IOREQ via
>>> the menuconfig on Arm. Saying tech preview do you mean that I also need to put
>>> it under CONFIG_EXPERT on Arm?
>> Normally I would say that unless it actually takes an expert to enable
>> the feature, it is better to make it depend on CONFIG_UNSUPPORTED [1].
>>
>> However, in this case, it might actually take an expert :-)
>> Additional patches are still required to enable the feature at the
>> toolstack level, and also the user needs to build a userspace ioreq
>> server. So in this case I am fine either way. I'll leave it up to you
>> and Julien to pick the best one.
>>
>> Either way, please add "(EXPERT)" or "(UNSUPPORTED)" in the one-line
>> kconfig description for ARM if possible.
>>
>>
>> [1] https://marc.info/?l=xen-devel&m=161168780401884
>
> Thank you for the explanation, personally I would prefer EXPERT, but...
>
> It would be really nice if we could agreed regarding that *common*
> Kconfig option before I submit V6 (which would probably avoid me to send
> V7 I hope).
>
> Now I see it as following (please note, it is a subject for new patch in
> this series, which should come last):
>
> +++ b/xen/common/Kconfig
> @@ -137,7 +137,13 @@ config HYPFS_CONFIG
> want to hide the .config contents from dom0.
>
> config IOREQ_SERVER
> - bool
> + bool "IOREQ support" if EXPERT || X86
This would make the prompt visible for x86, which we don't
want. But I guess Stefano was anyway after ...
> + default X86
> + depends on HVM
> + ---help---
> + Enables generic mechanism for providing emulated devices to
> the guests.
> +
> + If unsure, say Y.
config IOREQ_SERVER
bool "IOREQ support (EXPERT)" if EXPERT && !X86
default X86
depends on HVM
As the prompt should be invisible this way on x86, the (EXPERT)
doesn't need further qualifying, except maybe in the help text
(just to avoid this being taken as a possible mistake).
Jan
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common
2021-01-28 11:16 ` Oleksandr
@ 2021-01-28 11:24 ` Jan Beulich
0 siblings, 0 replies; 93+ messages in thread
From: Jan Beulich @ 2021-01-28 11:24 UTC (permalink / raw)
To: Oleksandr
Cc: Oleksandr Tyshchenko, Andrew Cooper, George Dunlap, Ian Jackson,
Julien Grall, Stefano Stabellini, Wei Liu, Roger Pau Monné,
Paul Durrant, Jun Nakajima, Kevin Tian, Tim Deegan, Julien Grall,
xen-devel
On 28.01.2021 12:16, Oleksandr wrote:
> On 28.01.21 10:06, Jan Beulich wrote:
>> On 27.01.2021 21:14, Oleksandr wrote:
>>> On 27.01.21 18:58, Jan Beulich wrote:
>>>> On 25.01.2021 20:08, Oleksandr Tyshchenko wrote:
>>>>> --- a/xen/arch/x86/Kconfig
>>>>> +++ b/xen/arch/x86/Kconfig
>>>>> @@ -92,6 +92,7 @@ config PV_LINEAR_PT
>>>>>
>>>>> config HVM
>>>>> def_bool !PV_SHIM_EXCLUSIVE
>>>>> + select IOREQ_SERVER
>>>>> prompt "HVM support"
>>>>> ---help---
>>>> ... the addition moved below the prompt line (could probably
>>>> be taken care of while committing, if no other need for a v6
>>>> arises).
>>> V6 is planned anyway, so will do, but ...
>>>
>>>
>>>> (Personally I think this should be
>>>>
>>>> config HVM
>>>> bool "HVM support"
>>>> default !PV_SHIM_EXCLUSIVE
>>> ... def_bool is changed to default by intention or this is a typo?
>> No, it's not a typo. "def_bool" is just a shorthand for "bool"
>> and "default", which is useful when there's no prompt, but
>> gets abused (in my view at least) in a number of other cases.
>> But as said ...
>>
>>>> anyway, but that's nothing you need to care about.)
>> ... here.
>
> ok, well, thank you, but FYI playing with default instead of def_bool I
> noticed the difference in behavior.
>
> I don't understand why, but HVM option disappears from menuconfig
> somehow... Or I do something completely wrong...
Hmm, this sounds odd indeed, but I confess I don't ever use
menuconfig, so I don't think I have an explanation.
>>>>> --- /dev/null
>>>>> +++ b/xen/include/asm-x86/ioreq.h
>>>>> @@ -0,0 +1,37 @@
>>>>> +/*
>>>>> + * ioreq.h: Hardware virtual machine assist interface definitions.
>>>>> + *
>>>>> + * This is a wrapper which purpose is to not include arch HVM specific header
>>>>> + * from the common code.
>>>>> + *
>>>>> + * Copyright (c) 2016 Citrix Systems Inc.
>>>>> + *
>>>>> + * This program is free software; you can redistribute it and/or modify it
>>>>> + * under the terms and conditions of the GNU General Public License,
>>>>> + * version 2, as published by the Free Software Foundation.
>>>>> + *
>>>>> + * This program is distributed in the hope it will be useful, but WITHOUT
>>>>> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
>>>>> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
>>>>> + * more details.
>>>>> + *
>>>>> + * You should have received a copy of the GNU General Public License along with
>>>>> + * this program; If not, see <http://www.gnu.org/licenses/>.
>>>>> + */
>>>>> +
>>>>> +#ifndef __ASM_X86_IOREQ_H__
>>>>> +#define __ASM_X86_IOREQ_H__
>>>>> +
>>>>> +#include <asm/hvm/ioreq.h>
>>>>> +
>>>>> +#endif /* __ASM_X86_IOREQ_H__ */
>>>> Not necessarily for taking care of right away, I think in the
>>>> longer run this wants wrapping by #ifdef CONFIG_HVM, such that
>>>> in !HVM builds the dependency on the "chained" header goes
>>>> away (reducing the amount of rebuilding in incremental builds).
>>> I don't mind wrapping it right away.
>> Well, if that doesn't break the !HVM build, I'd of course
>> appreciate it. I'd expect fallout, though.
>
> Hmm, I didn't notice a fallout with that change when CONFIG_HVM is not set
>
> +#ifdef CONFIG_HVM
> #include <asm/hvm/ioreq.h>
> +#endif
Good.
Jan
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2021-01-28 9:40 ` Julien Grall
@ 2021-01-28 11:33 ` Oleksandr
2021-01-28 13:39 ` Oleksandr
0 siblings, 1 reply; 93+ messages in thread
From: Oleksandr @ 2021-01-28 11:33 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, Julien Grall, Stefano Stabellini, Volodymyr Babchuk,
Andrew Cooper, George Dunlap, Ian Jackson, Jan Beulich, Wei Liu,
Oleksandr Tyshchenko
On 28.01.21 11:40, Julien Grall wrote:
Hi Julien
> Hi Oleksandr,
>
> On 27/01/2021 19:20, Oleksandr wrote:
> > >>> So I think we may be able to drop the include from
> asm/hvm/domain.h
>> (this would avoid to include it everywhere...).
>>
>> I have tried that, but other CUs use definitions from
>> public/hvm/dm_op.h, for example:
>>
>> p2m-pt.c: In function 'p2m_type_to_flags':
>> p2m-pt.c:87:33: error: 'XEN_DMOP_IOREQ_MEM_ACCESS_WRITE' undeclared
>> (first use in this function)
>> if ( p2m->ioreq.flags & XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
>> ^
>> So, I would prefer to leave it as is, please let me know if you think
>> otherwise.
>
> AFAICT, there is only 2 places (p2m-pt.c and p2m-ept.c) that requires
> <public/hvm/dm_op.h> but doesn't directly include it. Folding the diff
> below in patch #4 should do the job:
ok, will do
>
> diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
> index 975ab403f235..23d411f01d2f 100644
> --- a/xen/arch/x86/mm/p2m-ept.c
> +++ b/xen/arch/x86/mm/p2m-ept.c
> @@ -17,6 +17,7 @@
>
> #include <xen/domain_page.h>
> #include <xen/sched.h>
> +#include <public/hvm/dm_op.h>
> #include <asm/altp2m.h>
> #include <asm/current.h>
> #include <asm/paging.h>
> diff --git a/xen/arch/x86/mm/p2m-pt.c b/xen/arch/x86/mm/p2m-pt.c
> index c43d5d0413a1..f2afcf49a368 100644
> --- a/xen/arch/x86/mm/p2m-pt.c
> +++ b/xen/arch/x86/mm/p2m-pt.c
> @@ -27,6 +27,7 @@
> #include <xen/vm_event.h>
> #include <xen/event.h>
> #include <xen/trace.h>
> +#include <public/hvm/dm_op.h>
> #include <public/vm_event.h>
> #include <asm/altp2m.h>
> #include <asm/domain.h>
> diff --git a/xen/include/asm-x86/hvm/domain.h
> b/xen/include/asm-x86/hvm/domain.h
> index 3b36c2f41fa1..f26c1a2e2d5f 100644
> --- a/xen/include/asm-x86/hvm/domain.h
> +++ b/xen/include/asm-x86/hvm/domain.h
> @@ -28,8 +28,6 @@
> #include <asm/hvm/vmx/vmcs.h>
> #include <asm/hvm/svm/vmcb.h>
>
> -#include <public/hvm/dm_op.h>
> -
> #ifdef CONFIG_MEM_SHARING
> struct mem_sharing_domain
> {
>
> You would then need to move the include of <public/hvm/dm_op.h> in
> <xen/dm.h> from this patch to patch #9.
yes, sure
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 09/22] xen/ioreq: Make x86's IOREQ related dm-op handling common
2021-01-28 10:52 ` Jan Beulich
@ 2021-01-28 12:06 ` Oleksandr
2021-01-28 13:18 ` Jan Beulich
0 siblings, 1 reply; 93+ messages in thread
From: Oleksandr @ 2021-01-28 12:06 UTC (permalink / raw)
To: Jan Beulich
Cc: Julien Grall, Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Paul Durrant, Daniel De Graaf,
Oleksandr Tyshchenko, xen-devel
On 28.01.21 12:52, Jan Beulich wrote:
Hi Jan
> On 25.01.2021 20:08, Oleksandr Tyshchenko wrote:
>> --- /dev/null
>> +++ b/xen/include/xen/dm.h
>> @@ -0,0 +1,41 @@
>> +/*
>> + * Copyright (c) 2016 Citrix Systems Inc.
>> + *
>> + * This program is free software; you can redistribute it and/or modify it
>> + * under the terms and conditions of the GNU General Public License,
>> + * version 2, as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope it will be useful, but WITHOUT
>> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
>> + * more details.
>> + *
>> + * You should have received a copy of the GNU General Public License along with
>> + * this program; If not, see <http://www.gnu.org/licenses/>.
>> + */
>> +
>> +#ifndef __XEN_DM_H__
>> +#define __XEN_DM_H__
>> +
>> +#include <xen/sched.h>
>> +
>> +struct dmop_args {
>> + domid_t domid;
>> + unsigned int nr_bufs;
>> + /* Reserve enough buf elements for all current hypercalls. */
>> + struct xen_dm_op_buf buf[2];
> So this is then the patch where the public header needs including,
> to satisfy this use of a struct declared there independent of what
> xen/sched.h includes. In fact public/xen.h needs including here
> as well, I think, for domid_t. Otoh I can't see why you include
> xen/sched.h.
Yes, xen/sched.h indeed doesn't need to be included here, I mentioned
that we could just replace
it by xen/types.h (this is the minimum what we need here I think). As I
understand public/xen.h is already included by public/hvm/dm_op.h...
>
> My previously given ack applies only with this properly sorted.
Got it, sure.
>
> Jan
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common
2021-01-28 11:21 ` Jan Beulich
@ 2021-01-28 12:49 ` Oleksandr
2021-01-28 17:13 ` Stefano Stabellini
0 siblings, 1 reply; 93+ messages in thread
From: Oleksandr @ 2021-01-28 12:49 UTC (permalink / raw)
To: Jan Beulich
Cc: Julien Grall, xen-devel, Oleksandr Tyshchenko, Andrew Cooper,
George Dunlap, Ian Jackson, Wei Liu, Roger Pau Monné,
Paul Durrant, Jun Nakajima, Kevin Tian, Tim Deegan, Julien Grall,
Stefano Stabellini, Julien Grall
On 28.01.21 13:21, Jan Beulich wrote:
Hi Jan
> On 28.01.2021 12:01, Oleksandr wrote:
>> On 27.01.21 22:46, Stefano Stabellini wrote:
>>> On Wed, 27 Jan 2021, Oleksandr wrote:
>>>> Thank you. I got a request to make a possibility for user to select IOREQ via
>>>> the menuconfig on Arm. Saying tech preview do you mean that I also need to put
>>>> it under CONFIG_EXPERT on Arm?
>>> Normally I would say that unless it actually takes an expert to enable
>>> the feature, it is better to make it depend on CONFIG_UNSUPPORTED [1].
>>>
>>> However, in this case, it might actually take an expert :-)
>>> Additional patches are still required to enable the feature at the
>>> toolstack level, and also the user needs to build a userspace ioreq
>>> server. So in this case I am fine either way. I'll leave it up to you
>>> and Julien to pick the best one.
>>>
>>> Either way, please add "(EXPERT)" or "(UNSUPPORTED)" in the one-line
>>> kconfig description for ARM if possible.
>>>
>>>
>>> [1] https://marc.info/?l=xen-devel&m=161168780401884
>> Thank you for the explanation, personally I would prefer EXPERT, but...
>>
>> It would be really nice if we could agreed regarding that *common*
>> Kconfig option before I submit V6 (which would probably avoid me to send
>> V7 I hope).
>>
>> Now I see it as following (please note, it is a subject for new patch in
>> this series, which should come last):
>>
>> +++ b/xen/common/Kconfig
>> @@ -137,7 +137,13 @@ config HYPFS_CONFIG
>> want to hide the .config contents from dom0.
>>
>> config IOREQ_SERVER
>> - bool
>> + bool "IOREQ support" if EXPERT || X86
> This would make the prompt visible for x86, which we don't
> want.
Hmm, I missed that point.
> But I guess Stefano was anyway after ...
>
>> + default X86
>> + depends on HVM
>> + ---help---
>> + Enables generic mechanism for providing emulated devices to
>> the guests.
>> +
>> + If unsure, say Y.
> config IOREQ_SERVER
> bool "IOREQ support (EXPERT)" if EXPERT && !X86
> default X86
> depends on HVM
>
> As the prompt should be invisible this way on x86, the (EXPERT)
> doesn't need further qualifying, except maybe in the help text
> (just to avoid this being taken as a possible mistake).
Yes, this works as my initial version *but* is invisible on x86,
what is more that indeed the *EXPERT* tag for IOREQ on x86 will look
quite odd...
Perfect, thank you.
>
> Jan
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 22/22] xen/arm: Add mapcache invalidation handling
2021-01-28 9:55 ` Julien Grall
@ 2021-01-28 13:12 ` Oleksandr
0 siblings, 0 replies; 93+ messages in thread
From: Oleksandr @ 2021-01-28 13:12 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, Oleksandr Tyshchenko, Stefano Stabellini,
Volodymyr Babchuk, Julien Grall
On 28.01.21 11:55, Julien Grall wrote:
> Hi Oleksandr,
Hi Julien
>
> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> We need to send mapcache invalidation request to qemu/demu everytime
>> the page gets removed from a guest.
>>
>> At the moment, the Arm code doesn't explicitely remove the existing
>> mapping before inserting the new mapping. Instead, this is done
>> implicitely by __p2m_set_entry().
>>
>> So we need to recognize a case when old entry is a RAM page *and*
>> the new MFN is different in order to set the corresponding flag.
>
> AFAICT, this sentence doesn't match the code. Instead, you will
> request the mapcache invalidation whenever the "freed" entry contains
> some RAM page.
>
> I am fine with the approach used in the code. However, it would be
> good that the commit message reflects the code.
ok, will update it.
>
>
>> The most suitable place to do this is p2m_free_entry(), there
>> we can find the correct leaf type. The invalidation request
>> will be sent in do_trap_hypercall() later on.
>>
>> Taking into the account the following the do_trap_hypercall()
>> is the best place to send invalidation request:
>> - The only way a guest can modify its P2M on Arm is via an hypercall
>> - When sending the invalidation request, the vCPU will be blocked
>> until all the IOREQ servers have acknowledged the invalidation
>>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> CC: Julien Grall <julien.grall@arm.com>
>> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
>> [On Arm only]
>> Tested-by: Wei Chen <Wei.Chen@arm.com>
>>
>> ---
>> Please note, this is a split/cleanup/hardening of Julien's PoC:
>> "Add support for Guest IO forwarding to a device emulator"
>>
>> ***
>> Please note, this patch depends on the following which is
>> on review:
>> https://patchwork.kernel.org/patch/11803383/
>
> I think we can defer this post this series.
ok
>
>
>>
>> This patch is on par with x86 code (whether it is buggy or not).
>> If there is a need to improve/harden something, this can be done on
>> a follow-up.
>> ***
>>
>> Changes V1 -> V2:
>> - new patch, some changes were derived from (+ new explanation):
>> xen/ioreq: Make x86's invalidate qemu mapcache handling common
>> - put setting of the flag into __p2m_set_entry()
>> - clarify the conditions when the flag should be set
>> - use domain_has_ioreq_server()
>> - update do_trap_hypercall() by adding local variable
>>
>> Changes V2 -> V3:
>> - update patch description
>> - move check to p2m_free_entry()
>> - add a comment
>> - use "curr" instead of "v" in do_trap_hypercall()
>>
>> Changes V3 -> V4:
>> - update patch description
>> - re-order check in p2m_free_entry() to call
>> domain_has_ioreq_server()
>> only if p2m->domain == current->domain
>> - add a comment in do_trap_hypercall()
>>
>> Changes V4 -> V5:
>> - add Stefano's R-b
>> - update comment in do_trap_hypercall()
>> ---
>> ---
>> xen/arch/arm/p2m.c | 25 +++++++++++++++++--------
>> xen/arch/arm/traps.c | 20 +++++++++++++++++---
>> 2 files changed, 34 insertions(+), 11 deletions(-)
>>
>> diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
>> index d41c4fa..26acb95d 100644
>> --- a/xen/arch/arm/p2m.c
>> +++ b/xen/arch/arm/p2m.c
>> @@ -1,6 +1,7 @@
>> #include <xen/cpu.h>
>> #include <xen/domain_page.h>
>> #include <xen/iocap.h>
>> +#include <xen/ioreq.h>
>> #include <xen/lib.h>
>> #include <xen/sched.h>
>> #include <xen/softirq.h>
>> @@ -749,17 +750,25 @@ static void p2m_free_entry(struct p2m_domain *p2m,
>> if ( !p2m_is_valid(entry) )
>> return;
>> - /* Nothing to do but updating the stats if the entry is a
>> super-page. */
>> - if ( p2m_is_superpage(entry, level) )
>> + if ( p2m_is_superpage(entry, level) || (level == 3) )
>> {
>> - p2m->stats.mappings[level]--;
>> - return;
>> - }
>> +#ifdef CONFIG_IOREQ_SERVER
>> + /*
>> + * If this gets called (non-recursively) then either the entry
>
> I am not sure why you specify "non-recursively". You still want to
> invalidate the mapcache if we replaced a table with a superpage mapping.
yes agree, sorry I don't quite remember why I decided to put emphasis on
this word. I will just remove it.
>
>
>> + * was replaced by an entry with a different base (valid
>> case) or
>> + * the shattering of a superpage was failed (error case).
>
> s/was/has/ I think.
ok, I assume this applies for the second part of the sentence.
>
>> + * So, at worst, the spurious mapcache invalidation might be
>> sent.
>> + */
>> + if ( (p2m->domain == current->domain) &&
>> + domain_has_ioreq_server(p2m->domain) &&
>> + p2m_is_ram(entry.p2m.type) )
>> + p2m->domain->mapcache_invalidate = true;
>> +#endif
>> - if ( level == 3 )
>> - {
>> p2m->stats.mappings[level]--;
>> - p2m_put_l3_page(entry);
>> + /* Nothing to do if the entry is a super-page. */
>> + if ( level == 3 )
>> + p2m_put_l3_page(entry);
>> return;
>> }
>> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
>> index 4cdd343..64b740b 100644
>> --- a/xen/arch/arm/traps.c
>> +++ b/xen/arch/arm/traps.c
>> @@ -1443,6 +1443,7 @@ static void do_trap_hypercall(struct
>> cpu_user_regs *regs, register_t *nr,
>> const union hsr hsr)
>> {
>> arm_hypercall_fn_t call = NULL;
>> + struct vcpu *curr = current;
>> BUILD_BUG_ON(NR_hypercalls < ARRAY_SIZE(arm_hypercall_table) );
>> @@ -1459,7 +1460,7 @@ static void do_trap_hypercall(struct
>> cpu_user_regs *regs, register_t *nr,
>> return;
>> }
>> - current->hcall_preempted = false;
>> + curr->hcall_preempted = false;
>> perfc_incra(hypercalls, *nr);
>> call = arm_hypercall_table[*nr].fn;
>> @@ -1472,7 +1473,7 @@ static void do_trap_hypercall(struct
>> cpu_user_regs *regs, register_t *nr,
>> HYPERCALL_RESULT_REG(regs) = call(HYPERCALL_ARGS(regs));
>> #ifndef NDEBUG
>> - if ( !current->hcall_preempted )
>> + if ( !curr->hcall_preempted )
>> {
>> /* Deliberately corrupt parameter regs used by this
>> hypercall. */
>> switch ( arm_hypercall_table[*nr].nr_args ) {
>> @@ -1489,8 +1490,21 @@ static void do_trap_hypercall(struct
>> cpu_user_regs *regs, register_t *nr,
>> #endif
>> /* Ensure the hypercall trap instruction is re-executed. */
>> - if ( current->hcall_preempted )
>> + if ( curr->hcall_preempted )
>> regs->pc -= 4; /* re-execute 'hvc #XEN_HYPERCALL_TAG' */
>> +
>> +#ifdef CONFIG_IOREQ_SERVER
>> + /*
>> + * We call ioreq_signal_mapcache_invalidate from
>> do_trap_hypercall()
>> + * because the only way a guest can modify its P2M on Arm is via an
>> + * hypercall.
>> + * Note that sending the invalidation request causes the vCPU to
>> block
>> + * until all the IOREQ servers have acknowledged the invalidation.
>> + */
>> + if ( unlikely(curr->domain->mapcache_invalidate) &&
>> + test_and_clear_bool(curr->domain->mapcache_invalidate) )
>> + ioreq_signal_mapcache_invalidate();
>> +#endif
>> }
>> void arch_hypercall_tasklet_result(struct vcpu *v, long res)
>>
>
> Cheers,
>
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 09/22] xen/ioreq: Make x86's IOREQ related dm-op handling common
2021-01-28 12:06 ` Oleksandr
@ 2021-01-28 13:18 ` Jan Beulich
2021-01-28 20:01 ` Oleksandr
0 siblings, 1 reply; 93+ messages in thread
From: Jan Beulich @ 2021-01-28 13:18 UTC (permalink / raw)
To: Oleksandr
Cc: Julien Grall, Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Paul Durrant, Daniel De Graaf,
Oleksandr Tyshchenko, xen-devel
On 28.01.2021 13:06, Oleksandr wrote:
>
> On 28.01.21 12:52, Jan Beulich wrote:
>
> Hi Jan
>
>> On 25.01.2021 20:08, Oleksandr Tyshchenko wrote:
>>> --- /dev/null
>>> +++ b/xen/include/xen/dm.h
>>> @@ -0,0 +1,41 @@
>>> +/*
>>> + * Copyright (c) 2016 Citrix Systems Inc.
>>> + *
>>> + * This program is free software; you can redistribute it and/or modify it
>>> + * under the terms and conditions of the GNU General Public License,
>>> + * version 2, as published by the Free Software Foundation.
>>> + *
>>> + * This program is distributed in the hope it will be useful, but WITHOUT
>>> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
>>> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
>>> + * more details.
>>> + *
>>> + * You should have received a copy of the GNU General Public License along with
>>> + * this program; If not, see <http://www.gnu.org/licenses/>.
>>> + */
>>> +
>>> +#ifndef __XEN_DM_H__
>>> +#define __XEN_DM_H__
>>> +
>>> +#include <xen/sched.h>
>>> +
>>> +struct dmop_args {
>>> + domid_t domid;
>>> + unsigned int nr_bufs;
>>> + /* Reserve enough buf elements for all current hypercalls. */
>>> + struct xen_dm_op_buf buf[2];
>> So this is then the patch where the public header needs including,
>> to satisfy this use of a struct declared there independent of what
>> xen/sched.h includes. In fact public/xen.h needs including here
>> as well, I think, for domid_t. Otoh I can't see why you include
>> xen/sched.h.
>
> Yes, xen/sched.h indeed doesn't need to be included here, I mentioned
> that we could just replace
>
> it by xen/types.h (this is the minimum what we need here I think). As I
> understand public/xen.h is already included by public/hvm/dm_op.h...
But that's something you better wouldn't depend on anywhere.
What one public header does or does not include may change over
time.
Jan
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2021-01-28 11:33 ` Oleksandr
@ 2021-01-28 13:39 ` Oleksandr
2021-01-28 14:29 ` Oleksandr
0 siblings, 1 reply; 93+ messages in thread
From: Oleksandr @ 2021-01-28 13:39 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, Julien Grall, Stefano Stabellini, Volodymyr Babchuk,
Andrew Cooper, George Dunlap, Ian Jackson, Jan Beulich, Wei Liu,
Oleksandr Tyshchenko
On 28.01.21 13:33, Oleksandr wrote:
Hi Julien
>
> On 28.01.21 11:40, Julien Grall wrote:
>
> Hi Julien
>
>> Hi Oleksandr,
>>
>> On 27/01/2021 19:20, Oleksandr wrote:
>> > >>> So I think we may be able to drop the include from
>> asm/hvm/domain.h
>>> (this would avoid to include it everywhere...).
>>>
>>> I have tried that, but other CUs use definitions from
>>> public/hvm/dm_op.h, for example:
>>>
>>> p2m-pt.c: In function 'p2m_type_to_flags':
>>> p2m-pt.c:87:33: error: 'XEN_DMOP_IOREQ_MEM_ACCESS_WRITE' undeclared
>>> (first use in this function)
>>> if ( p2m->ioreq.flags & XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
>>> ^
>>> So, I would prefer to leave it as is, please let me know if you
>>> think otherwise.
>>
>> AFAICT, there is only 2 places (p2m-pt.c and p2m-ept.c) that requires
>> <public/hvm/dm_op.h> but doesn't directly include it. Folding the
>> diff below in patch #4 should do the job:
>
> ok, will do
Just to clarify, you mentioned about patch #4, but shouldn't we make
these changes in patch #9 which actually tries to sort dm related stuff?
>
>
>>
>> diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
>> index 975ab403f235..23d411f01d2f 100644
>> --- a/xen/arch/x86/mm/p2m-ept.c
>> +++ b/xen/arch/x86/mm/p2m-ept.c
>> @@ -17,6 +17,7 @@
>>
>> #include <xen/domain_page.h>
>> #include <xen/sched.h>
>> +#include <public/hvm/dm_op.h>
>> #include <asm/altp2m.h>
>> #include <asm/current.h>
>> #include <asm/paging.h>
>> diff --git a/xen/arch/x86/mm/p2m-pt.c b/xen/arch/x86/mm/p2m-pt.c
>> index c43d5d0413a1..f2afcf49a368 100644
>> --- a/xen/arch/x86/mm/p2m-pt.c
>> +++ b/xen/arch/x86/mm/p2m-pt.c
>> @@ -27,6 +27,7 @@
>> #include <xen/vm_event.h>
>> #include <xen/event.h>
>> #include <xen/trace.h>
>> +#include <public/hvm/dm_op.h>
>> #include <public/vm_event.h>
>> #include <asm/altp2m.h>
>> #include <asm/domain.h>
>> diff --git a/xen/include/asm-x86/hvm/domain.h
>> b/xen/include/asm-x86/hvm/domain.h
>> index 3b36c2f41fa1..f26c1a2e2d5f 100644
>> --- a/xen/include/asm-x86/hvm/domain.h
>> +++ b/xen/include/asm-x86/hvm/domain.h
>> @@ -28,8 +28,6 @@
>> #include <asm/hvm/vmx/vmcs.h>
>> #include <asm/hvm/svm/vmcb.h>
>>
>> -#include <public/hvm/dm_op.h>
>> -
>> #ifdef CONFIG_MEM_SHARING
>> struct mem_sharing_domain
>> {
>>
>> You would then need to move the include of <public/hvm/dm_op.h> in
>> <xen/dm.h> from this patch to patch #9.
>
> yes, sure
>
>
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 10/22] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
2021-01-25 19:08 ` [PATCH V5 10/22] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu Oleksandr Tyshchenko
@ 2021-01-28 13:41 ` Julien Grall
2021-01-28 13:53 ` Jan Beulich
2021-01-28 14:05 ` Oleksandr
0 siblings, 2 replies; 93+ messages in thread
From: Julien Grall @ 2021-01-28 13:41 UTC (permalink / raw)
To: Oleksandr Tyshchenko, xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Stefano Stabellini,
Jun Nakajima, Kevin Tian, Julien Grall
Hi Oleksandr,
On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> The IOREQ is a common feature now and these fields will be used
> on Arm as is. Move them to common struct vcpu as a part of new
> struct vcpu_io and drop duplicating "io" prefixes. Also move
> enum hvm_io_completion to xen/sched.h and remove "hvm" prefixes.
>
> This patch completely removes layering violation in the common code.
>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> Reviewed-by: Julien Grall <jgrall@amazon.com>
> Reviewed-by: Paul Durrant <paul@xen.org>
> Acked-by: Jan Beulich <jbeulich@suse.com>
> CC: Julien Grall <julien.grall@arm.com>
> [On Arm only]
> Tested-by: Wei Chen <Wei.Chen@arm.com>
I seem to have trouble running Xen on x86 platform after this patch is
applied (see trace below).
The bisector pointed out to this patch but I can't quite figure out why
this is breaking.
Does this ring any bell to someone?
(XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
(XEN) ----[ Xen-4.15-unstable x86_64 debug=n gcov=y Tainted: C ]----
(XEN) CPU: 1
(XEN) RIP: e008:[<ffff82d04041c1c7>]
x86_64/entry.S#restore_all_guest+0x7/0x145
(XEN) RFLAGS: 0000000000010002 CONTEXT: hypervisor (d0v0)
(XEN) rax: 00000000000000ff rbx: ffff83027c806000 rcx: ffff82d0406c9a80
(XEN) rdx: 0000000000000000 rsi: fffffffffffffed9 rdi: 0000000000000001
(XEN) rbp: ffff83027c887df0 rsp: ffff83027c887ef8 r8: 00000000aaa8946e
(XEN) r9: 0000000000000002 r10: ffff83027c806040 r11: ffff83027c8cc020
(XEN) r12: ffff83027c80f000 r13: ffff83027c895000 r14: 0000000000000000
(XEN) r15: 0000000000000000 cr0: 0000000080050033 cr4: 00000000003426e0
(XEN) cr3: 0000000273a2d000 cr2: 0000000000000000
(XEN) fsb: 0000000000000000 gsb: 0000000000000000 gss: 0000000000000000
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
(XEN) Xen code around <ffff82d04041c1c7>
(x86_64/entry.S#restore_all_guest+0x7/0x145):
(XEN) 00 48 8b 93 98 0d 00 00 <44> 8b 3a 4c 8b 8b 68 0b 00 00 ba ff 7f
00 00 48
(XEN) Xen stack trace from rsp=ffff83027c887ef8:
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 ffffffff83a2c000 0000000000000000 0000000000000000
(XEN) ffffffff82c9e160 000000000000e033 0000000000000200 ffffffff83a4f000
(XEN) 000000000000e02b 0000000000000000 0000000000000000 00000000ffffffff
(XEN) 0000000000000000 0000e01000000001 ffff83027c806000 000000323c1d9000
(XEN) 00000000003426e0 0000000000000000 0000000000000000 0000060100000000
(XEN) 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82d04041c1c7>] R x86_64/entry.S#restore_all_guest+0x7/0x145
(XEN)
(XEN) Pagetable walk from 0000000000000000:
(XEN) L4[0x000] = 0000000000000000 ffffffffffffffff
(XEN) Xen lock profile info SHOW (now = 4770175959 total = 4770175959)
(XEN) Global xenpf_lock: addr=ffff82d04052c4a0, lockval=00000000, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global rtc_lock: addr=ffff82d04052c480, lockval=00010001, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global pit_lock: addr=ffff82d04052c470, lockval=00000000, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global platform_timer_lock: addr=ffff82d04052c460,
lockval=000e000e, not locked
(XEN) lock:6(57390), block:0(0)
(XEN) Global sync_lock: addr=ffff82d04052c440, lockval=00000000, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global lock: addr=ffff82d04052c450, lockval=00000000, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global flush_lock: addr=ffff82d04052c430, lockval=00080007, cpu=0
(XEN) lock:3(49953), block:0(0)
(XEN) Global pci_config_lock: addr=ffff82d04052c420, lockval=22702270,
not locked
(XEN) lock:14(100313), block:0(0)
(XEN) Global lapic_nmi_owner_lock: addr=ffff82d04052bff0,
lockval=00000000, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global map_pgdir_lock: addr=ffff82d04052bfd0, lockval=00050005,
not locked
(XEN) lock:3(386), block:0(0)
(XEN) Global vector_lock: addr=ffff82d04052bfc0, lockval=00260026, not
locked
(XEN) lock:1(383), block:0(0)
(XEN) Global irq_ratelimit_lock: addr=ffff82d04052bfb0,
lockval=00000000, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global lock: addr=ffff82d04052bf90, lockval=00000000, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global msix_fixmap_lock: addr=ffff82d04052bf80, lockval=00000000,
not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global ioapic_lock: addr=ffff82d04052bf60, lockval=003a003a, not
locked
(XEN) lock:0(0), block:0(0)
(XEN) Global i8259A_lock: addr=ffff82d04052bf50, lockval=00210021, not
locked
(XEN) lock:0(0), block:0(0)
(XEN) Global osvw_lock: addr=ffff82d04052bf10, lockval=00000000, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global vpmu_lock: addr=ffff82d04052bf00, lockval=00040004, not locked
(XEN) lock:4(449), block:0(0)
(XEN) Global mtrr_mutex: addr=ffff82d04052bcf0, lockval=00000000, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global set_atomicity_lock: addr=ffff82d04052bce0,
lockval=00040004, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global microcode_mutex: addr=ffff82d04052bcc0, lockval=00030003,
not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global cmci_discover_lock: addr=ffff82d04052bc90,
lockval=00000000, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global mce_logout_lock: addr=ffff82d04052bc70, lockval=00000000,
not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global processing_lock: addr=ffff82d04052bc50, lockval=00000000,
not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global pm_lock: addr=ffff82d04052bbc0, lockval=00000000, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global apei_iomaps_lock: addr=ffff82d04052bbb0, lockval=00000000,
not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global erst_lock: addr=ffff82d04052bba0, lockval=00000000, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global shared_intremap_lock: addr=ffff82d04052bb20,
lockval=00000000, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global console_lock: addr=ffff82d04052b7d0, lockval=01350135, not
locked
(XEN) lock:154(111298857), block:0(0)
(XEN) Global ratelimit_lock: addr=ffff82d04052b7c0, lockval=00000000,
not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global lock: addr=ffff82d04052b7a0, lockval=00000000, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global sched_free_cpu_lock: addr=ffff82d04052b770,
lockval=00040004, not locked
(XEN) lock:4(2963), block:0(0)
(XEN) Global cpupool_lock: addr=ffff82d04052b750, lockval=00030003, not
locked
(XEN) lock:3(19403), block:0(0)
(XEN) Global sysctl_lock: addr=ffff82d04052b740, lockval=00000000, not
locked
(XEN) lock:0(0), block:0(0)
(XEN) Global domctl_lock: addr=ffff82d04052b730, lockval=00000000, not
locked
(XEN) lock:0(0), block:0(0)
(XEN) Global pool_list_lock: addr=ffff82d04052b720, lockval=00010001,
not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global xenoprof_lock: addr=ffff82d04052b710, lockval=00000000, not
locked
(XEN) lock:0(0), block:0(0)
(XEN) Global pmu_owner_lock: addr=ffff82d04052b700, lockval=00000000,
not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global vm_lock: addr=ffff82d04052b6f0, lockval=00130013, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global virtual_region_lock: addr=ffff82d04052b6e0,
lockval=00010001, not locked
(XEN) lock:1(198), block:0(0)
(XEN) Global wc_lock: addr=ffff82d04052b6d0, lockval=00020002, not locked
(XEN) lock:1(770), block:0(0)
(XEN) Global tasklet_lock: addr=ffff82d04052b6c0, lockval=03790379, not
locked
(XEN) lock:612(52925), block:0(0)
(XEN) Global symbols_mutex: addr=ffff82d04052b6b0, lockval=00000000, not
locked
(XEN) lock:0(0), block:0(0)
(XEN) Global stopmachine_lock: addr=ffff82d04052b6a0, lockval=00000000,
not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global call_lock: addr=ffff82d04052b680, lockval=00040004, not locked
(XEN) lock:3(27688741), block:0(0)
(XEN) Global lock: addr=ffff82d04052b560, lockval=00000000, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global heap_lock: addr=ffff82d04052b540, lockval=c5d0c5d0, not locked
(XEN) lock:16949(5481440), block:1(34420)
(XEN) Global crash_notes_lock: addr=ffff82d04052b520, lockval=00000000,
not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global lock: addr=ffff82d04052b500, lockval=00000000, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global global_virq_handlers_lock: addr=ffff82d04052b4f0,
lockval=00000000, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global domlist_update_lock: addr=ffff82d04052b4d0,
lockval=00010001, not locked
(XEN) lock:1(105), block:0(0)
(XEN) Global debugtrace_lock: addr=ffff82d04052b4b0, lockval=00030003,
not locked
(XEN) lock:3(3327), block:0(0)
(XEN) Global accounting_lock: addr=ffff82d04052b480, lockval=00000000,
not locked
(XEN) lock:0(0), block:0(0)
(XEN) Global efi_rs_lock: addr=ffff82d04052b020, lockval=00000000, not
locked
(XEN) lock:0(0), block:0(0)
(XEN) Domain 0 event_lock: addr=ffff83027c80f110, lockval=00000000, not
locked
(XEN) lock:0(0), block:0(0)
(XEN) Domain 0 page_alloc_lock: addr=ffff83027c80f028, lockval=05cf05cf,
not locked
(XEN) lock:1487(3914530), block:0(0)
(XEN) Domain 0 domain_lock: addr=ffff83027c80f018, lockval=00000000, not
locked
(XEN) lock:0(0), block:0(0)
(XEN) Domain 32767 page_alloc_lock: addr=ffff83027c875028,
lockval=00000000, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Domain 32767 domain_lock: addr=ffff83027c875018, lockval=00000000,
not locked
(XEN) lock:0(0), block:0(0)
(XEN) Domain 32753 page_alloc_lock: addr=ffff83027c8c8028,
lockval=00000000, not locked
(XEN) lock:0(0), block:0(0)
(XEN) Domain 32753 domain_lock: addr=ffff83027c8c8018, lockval=00000000,
not locked
(XEN) lock:0(0), block:0(0)
(XEN) Domain 32754 page_alloc_lock: addr=ffff83027c8c9028,
lockval=18001800, not locked
(XEN) lock:6144(101574), block:0(0)
(XEN) Domain 32754 domain_lock: addr=ffff83027c8c9018, lockval=00000000,
not locked
(XEN) lock:0(0), block:0(0)
(XEN) debugtrace_dump() global buffer starting
1 cpupool_create(pool=0,sched=0)
2 Created cpupool 0 with scheduler SMP Credit Scheduler rev2 (credit2)
3 cpupool_add_domain(dom=0,pool=0) n_dom 1 rc 0
(XEN) wrap: 0
(XEN) debugtrace_dump() global buffer finished
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 1:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=0000]
(XEN) Faulting linear address: 0000000000000000
(XEN) ****************************************
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 10/22] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
2021-01-28 13:41 ` Julien Grall
@ 2021-01-28 13:53 ` Jan Beulich
2021-01-28 14:21 ` Julien Grall
2021-01-28 14:05 ` Oleksandr
1 sibling, 1 reply; 93+ messages in thread
From: Jan Beulich @ 2021-01-28 13:53 UTC (permalink / raw)
To: Julien Grall, Oleksandr Tyshchenko
Cc: Oleksandr Tyshchenko, Paul Durrant, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Stefano Stabellini,
Jun Nakajima, Kevin Tian, Julien Grall, xen-devel
On 28.01.2021 14:41, Julien Grall wrote:
> Hi Oleksandr,
>
> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> The IOREQ is a common feature now and these fields will be used
>> on Arm as is. Move them to common struct vcpu as a part of new
>> struct vcpu_io and drop duplicating "io" prefixes. Also move
>> enum hvm_io_completion to xen/sched.h and remove "hvm" prefixes.
>>
>> This patch completely removes layering violation in the common code.
>>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> Reviewed-by: Julien Grall <jgrall@amazon.com>
>> Reviewed-by: Paul Durrant <paul@xen.org>
>> Acked-by: Jan Beulich <jbeulich@suse.com>
>> CC: Julien Grall <julien.grall@arm.com>
>> [On Arm only]
>> Tested-by: Wei Chen <Wei.Chen@arm.com>
>
> I seem to have trouble running Xen on x86 platform after this patch is
> applied (see trace below).
>
> The bisector pointed out to this patch but I can't quite figure out why
> this is breaking.
>
> Does this ring any bell to someone?
Memory overwriting / corruption? This ...
> (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
> (XEN) ----[ Xen-4.15-unstable x86_64 debug=n gcov=y Tainted: C ]----
> (XEN) CPU: 1
> (XEN) RIP: e008:[<ffff82d04041c1c7>]
> x86_64/entry.S#restore_all_guest+0x7/0x145
> (XEN) RFLAGS: 0000000000010002 CONTEXT: hypervisor (d0v0)
> (XEN) rax: 00000000000000ff rbx: ffff83027c806000 rcx: ffff82d0406c9a80
> (XEN) rdx: 0000000000000000 rsi: fffffffffffffed9 rdi: 0000000000000001
> (XEN) rbp: ffff83027c887df0 rsp: ffff83027c887ef8 r8: 00000000aaa8946e
> (XEN) r9: 0000000000000002 r10: ffff83027c806040 r11: ffff83027c8cc020
> (XEN) r12: ffff83027c80f000 r13: ffff83027c895000 r14: 0000000000000000
> (XEN) r15: 0000000000000000 cr0: 0000000080050033 cr4: 00000000003426e0
> (XEN) cr3: 0000000273a2d000 cr2: 0000000000000000
> (XEN) fsb: 0000000000000000 gsb: 0000000000000000 gss: 0000000000000000
> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
> (XEN) Xen code around <ffff82d04041c1c7>
> (x86_64/entry.S#restore_all_guest+0x7/0x145):
> (XEN) 00 48 8b 93 98 0d 00 00 <44> 8b 3a 4c 8b 8b 68 0b 00 00 ba ff 7f
... is
restore_all_guest:
ASSERT_INTERRUPTS_DISABLED
/* Stash guest SPEC_CTRL value while we can read struct vcpu. */
mov VCPU_arch_msrs(%rbx), %rdx
mov VCPUMSR_spec_ctrl_raw(%rdx), %r15d
i.e. Dom0/vCPU0's v->arch.msrs got zeroed in an unintended way,
hence %rdx is zero here (%rbx looks at least plausible).
I take it that you double check this isn't an incremental build
issue, i.e. entry.o for some reason not having got rebuilt
despite struct vcpu's layout having changed?
Jan
> 00 00 48
> (XEN) Xen stack trace from rsp=ffff83027c887ef8:
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 ffffffff83a2c000 0000000000000000 0000000000000000
> (XEN) ffffffff82c9e160 000000000000e033 0000000000000200 ffffffff83a4f000
> (XEN) 000000000000e02b 0000000000000000 0000000000000000 00000000ffffffff
> (XEN) 0000000000000000 0000e01000000001 ffff83027c806000 000000323c1d9000
> (XEN) 00000000003426e0 0000000000000000 0000000000000000 0000060100000000
> (XEN) 0000000000000000
> (XEN) Xen call trace:
> (XEN) [<ffff82d04041c1c7>] R x86_64/entry.S#restore_all_guest+0x7/0x145
> (XEN)
> (XEN) Pagetable walk from 0000000000000000:
> (XEN) L4[0x000] = 0000000000000000 ffffffffffffffff
> (XEN) Xen lock profile info SHOW (now = 4770175959 total = 4770175959)
> (XEN) Global xenpf_lock: addr=ffff82d04052c4a0, lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global rtc_lock: addr=ffff82d04052c480, lockval=00010001, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global pit_lock: addr=ffff82d04052c470, lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global platform_timer_lock: addr=ffff82d04052c460,
> lockval=000e000e, not locked
> (XEN) lock:6(57390), block:0(0)
> (XEN) Global sync_lock: addr=ffff82d04052c440, lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global lock: addr=ffff82d04052c450, lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global flush_lock: addr=ffff82d04052c430, lockval=00080007, cpu=0
> (XEN) lock:3(49953), block:0(0)
> (XEN) Global pci_config_lock: addr=ffff82d04052c420, lockval=22702270,
> not locked
> (XEN) lock:14(100313), block:0(0)
> (XEN) Global lapic_nmi_owner_lock: addr=ffff82d04052bff0,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global map_pgdir_lock: addr=ffff82d04052bfd0, lockval=00050005,
> not locked
> (XEN) lock:3(386), block:0(0)
> (XEN) Global vector_lock: addr=ffff82d04052bfc0, lockval=00260026, not
> locked
> (XEN) lock:1(383), block:0(0)
> (XEN) Global irq_ratelimit_lock: addr=ffff82d04052bfb0,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global lock: addr=ffff82d04052bf90, lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global msix_fixmap_lock: addr=ffff82d04052bf80, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global ioapic_lock: addr=ffff82d04052bf60, lockval=003a003a, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global i8259A_lock: addr=ffff82d04052bf50, lockval=00210021, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global osvw_lock: addr=ffff82d04052bf10, lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global vpmu_lock: addr=ffff82d04052bf00, lockval=00040004, not locked
> (XEN) lock:4(449), block:0(0)
> (XEN) Global mtrr_mutex: addr=ffff82d04052bcf0, lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global set_atomicity_lock: addr=ffff82d04052bce0,
> lockval=00040004, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global microcode_mutex: addr=ffff82d04052bcc0, lockval=00030003,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global cmci_discover_lock: addr=ffff82d04052bc90,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global mce_logout_lock: addr=ffff82d04052bc70, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global processing_lock: addr=ffff82d04052bc50, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global pm_lock: addr=ffff82d04052bbc0, lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global apei_iomaps_lock: addr=ffff82d04052bbb0, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global erst_lock: addr=ffff82d04052bba0, lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global shared_intremap_lock: addr=ffff82d04052bb20,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global console_lock: addr=ffff82d04052b7d0, lockval=01350135, not
> locked
> (XEN) lock:154(111298857), block:0(0)
> (XEN) Global ratelimit_lock: addr=ffff82d04052b7c0, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global lock: addr=ffff82d04052b7a0, lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global sched_free_cpu_lock: addr=ffff82d04052b770,
> lockval=00040004, not locked
> (XEN) lock:4(2963), block:0(0)
> (XEN) Global cpupool_lock: addr=ffff82d04052b750, lockval=00030003, not
> locked
> (XEN) lock:3(19403), block:0(0)
> (XEN) Global sysctl_lock: addr=ffff82d04052b740, lockval=00000000, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global domctl_lock: addr=ffff82d04052b730, lockval=00000000, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global pool_list_lock: addr=ffff82d04052b720, lockval=00010001,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global xenoprof_lock: addr=ffff82d04052b710, lockval=00000000, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global pmu_owner_lock: addr=ffff82d04052b700, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global vm_lock: addr=ffff82d04052b6f0, lockval=00130013, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global virtual_region_lock: addr=ffff82d04052b6e0,
> lockval=00010001, not locked
> (XEN) lock:1(198), block:0(0)
> (XEN) Global wc_lock: addr=ffff82d04052b6d0, lockval=00020002, not locked
> (XEN) lock:1(770), block:0(0)
> (XEN) Global tasklet_lock: addr=ffff82d04052b6c0, lockval=03790379, not
> locked
> (XEN) lock:612(52925), block:0(0)
> (XEN) Global symbols_mutex: addr=ffff82d04052b6b0, lockval=00000000, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global stopmachine_lock: addr=ffff82d04052b6a0, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global call_lock: addr=ffff82d04052b680, lockval=00040004, not locked
> (XEN) lock:3(27688741), block:0(0)
> (XEN) Global lock: addr=ffff82d04052b560, lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global heap_lock: addr=ffff82d04052b540, lockval=c5d0c5d0, not locked
> (XEN) lock:16949(5481440), block:1(34420)
> (XEN) Global crash_notes_lock: addr=ffff82d04052b520, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global lock: addr=ffff82d04052b500, lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global global_virq_handlers_lock: addr=ffff82d04052b4f0,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global domlist_update_lock: addr=ffff82d04052b4d0,
> lockval=00010001, not locked
> (XEN) lock:1(105), block:0(0)
> (XEN) Global debugtrace_lock: addr=ffff82d04052b4b0, lockval=00030003,
> not locked
> (XEN) lock:3(3327), block:0(0)
> (XEN) Global accounting_lock: addr=ffff82d04052b480, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global efi_rs_lock: addr=ffff82d04052b020, lockval=00000000, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Domain 0 event_lock: addr=ffff83027c80f110, lockval=00000000, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Domain 0 page_alloc_lock: addr=ffff83027c80f028, lockval=05cf05cf,
> not locked
> (XEN) lock:1487(3914530), block:0(0)
> (XEN) Domain 0 domain_lock: addr=ffff83027c80f018, lockval=00000000, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Domain 32767 page_alloc_lock: addr=ffff83027c875028,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Domain 32767 domain_lock: addr=ffff83027c875018, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Domain 32753 page_alloc_lock: addr=ffff83027c8c8028,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Domain 32753 domain_lock: addr=ffff83027c8c8018, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Domain 32754 page_alloc_lock: addr=ffff83027c8c9028,
> lockval=18001800, not locked
> (XEN) lock:6144(101574), block:0(0)
> (XEN) Domain 32754 domain_lock: addr=ffff83027c8c9018, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) debugtrace_dump() global buffer starting
> 1 cpupool_create(pool=0,sched=0)
> 2 Created cpupool 0 with scheduler SMP Credit Scheduler rev2 (credit2)
> 3 cpupool_add_domain(dom=0,pool=0) n_dom 1 rc 0
> (XEN) wrap: 0
> (XEN) debugtrace_dump() global buffer finished
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 1:
> (XEN) FATAL PAGE FAULT
> (XEN) [error_code=0000]
> (XEN) Faulting linear address: 0000000000000000
> (XEN) ****************************************
>
>
> Cheers,
>
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 10/22] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
2021-01-28 13:41 ` Julien Grall
2021-01-28 13:53 ` Jan Beulich
@ 2021-01-28 14:05 ` Oleksandr
1 sibling, 0 replies; 93+ messages in thread
From: Oleksandr @ 2021-01-28 14:05 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Stefano Stabellini,
Jun Nakajima, Kevin Tian, Julien Grall
On 28.01.21 15:41, Julien Grall wrote:
> Hi Oleksandr,
Hi Julien
>
> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> The IOREQ is a common feature now and these fields will be used
>> on Arm as is. Move them to common struct vcpu as a part of new
>> struct vcpu_io and drop duplicating "io" prefixes. Also move
>> enum hvm_io_completion to xen/sched.h and remove "hvm" prefixes.
>>
>> This patch completely removes layering violation in the common code.
>>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> Reviewed-by: Julien Grall <jgrall@amazon.com>
>> Reviewed-by: Paul Durrant <paul@xen.org>
>> Acked-by: Jan Beulich <jbeulich@suse.com>
>> CC: Julien Grall <julien.grall@arm.com>
>> [On Arm only]
>> Tested-by: Wei Chen <Wei.Chen@arm.com>
>
> I seem to have trouble running Xen on x86 platform after this patch is
> applied (see trace below).
First of all thank you for testing! I admit, I didn't physically test on
x86 (but performed various build tests).
Also, in current patch I mostly did (or even all) mechanical changes
which wouldn't change a behavior I think,
what is more that I tried to retain existing x86 behavior as much as I
could not only in current patch,
but among all patches in this series.
>
>
> The bisector pointed out to this patch but I can't quite figure out
> why this is breaking.
>
> Does this ring any bell to someone?
>
> (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch
> input)
> (XEN) ----[ Xen-4.15-unstable x86_64 debug=n gcov=y Tainted: C ]----
> (XEN) CPU: 1
> (XEN) RIP: e008:[<ffff82d04041c1c7>]
> x86_64/entry.S#restore_all_guest+0x7/0x145
> (XEN) RFLAGS: 0000000000010002 CONTEXT: hypervisor (d0v0)
> (XEN) rax: 00000000000000ff rbx: ffff83027c806000 rcx:
> ffff82d0406c9a80
> (XEN) rdx: 0000000000000000 rsi: fffffffffffffed9 rdi:
> 0000000000000001
> (XEN) rbp: ffff83027c887df0 rsp: ffff83027c887ef8 r8:
> 00000000aaa8946e
> (XEN) r9: 0000000000000002 r10: ffff83027c806040 r11:
> ffff83027c8cc020
> (XEN) r12: ffff83027c80f000 r13: ffff83027c895000 r14:
> 0000000000000000
> (XEN) r15: 0000000000000000 cr0: 0000000080050033 cr4:
> 00000000003426e0
> (XEN) cr3: 0000000273a2d000 cr2: 0000000000000000
> (XEN) fsb: 0000000000000000 gsb: 0000000000000000 gss:
> 0000000000000000
> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
> (XEN) Xen code around <ffff82d04041c1c7>
> (x86_64/entry.S#restore_all_guest+0x7/0x145):
> (XEN) 00 48 8b 93 98 0d 00 00 <44> 8b 3a 4c 8b 8b 68 0b 00 00 ba ff
> 7f 00 00 48
> (XEN) Xen stack trace from rsp=ffff83027c887ef8:
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) 0000000000000000 ffffffff83a2c000 0000000000000000
> 0000000000000000
> (XEN) ffffffff82c9e160 000000000000e033 0000000000000200
> ffffffff83a4f000
> (XEN) 000000000000e02b 0000000000000000 0000000000000000
> 00000000ffffffff
> (XEN) 0000000000000000 0000e01000000001 ffff83027c806000
> 000000323c1d9000
> (XEN) 00000000003426e0 0000000000000000 0000000000000000
> 0000060100000000
> (XEN) 0000000000000000
> (XEN) Xen call trace:
> (XEN) [<ffff82d04041c1c7>] R
> x86_64/entry.S#restore_all_guest+0x7/0x145
> (XEN)
> (XEN) Pagetable walk from 0000000000000000:
> (XEN) L4[0x000] = 0000000000000000 ffffffffffffffff
> (XEN) Xen lock profile info SHOW (now = 4770175959 total = 4770175959)
> (XEN) Global xenpf_lock: addr=ffff82d04052c4a0, lockval=00000000, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global rtc_lock: addr=ffff82d04052c480, lockval=00010001, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global pit_lock: addr=ffff82d04052c470, lockval=00000000, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global platform_timer_lock: addr=ffff82d04052c460,
> lockval=000e000e, not locked
> (XEN) lock:6(57390), block:0(0)
> (XEN) Global sync_lock: addr=ffff82d04052c440, lockval=00000000, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global lock: addr=ffff82d04052c450, lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global flush_lock: addr=ffff82d04052c430, lockval=00080007, cpu=0
> (XEN) lock:3(49953), block:0(0)
> (XEN) Global pci_config_lock: addr=ffff82d04052c420, lockval=22702270,
> not locked
> (XEN) lock:14(100313), block:0(0)
> (XEN) Global lapic_nmi_owner_lock: addr=ffff82d04052bff0,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global map_pgdir_lock: addr=ffff82d04052bfd0, lockval=00050005,
> not locked
> (XEN) lock:3(386), block:0(0)
> (XEN) Global vector_lock: addr=ffff82d04052bfc0, lockval=00260026, not
> locked
> (XEN) lock:1(383), block:0(0)
> (XEN) Global irq_ratelimit_lock: addr=ffff82d04052bfb0,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global lock: addr=ffff82d04052bf90, lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global msix_fixmap_lock: addr=ffff82d04052bf80,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global ioapic_lock: addr=ffff82d04052bf60, lockval=003a003a, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global i8259A_lock: addr=ffff82d04052bf50, lockval=00210021, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global osvw_lock: addr=ffff82d04052bf10, lockval=00000000, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global vpmu_lock: addr=ffff82d04052bf00, lockval=00040004, not
> locked
> (XEN) lock:4(449), block:0(0)
> (XEN) Global mtrr_mutex: addr=ffff82d04052bcf0, lockval=00000000, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global set_atomicity_lock: addr=ffff82d04052bce0,
> lockval=00040004, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global microcode_mutex: addr=ffff82d04052bcc0, lockval=00030003,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global cmci_discover_lock: addr=ffff82d04052bc90,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global mce_logout_lock: addr=ffff82d04052bc70, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global processing_lock: addr=ffff82d04052bc50, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global pm_lock: addr=ffff82d04052bbc0, lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global apei_iomaps_lock: addr=ffff82d04052bbb0,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global erst_lock: addr=ffff82d04052bba0, lockval=00000000, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global shared_intremap_lock: addr=ffff82d04052bb20,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global console_lock: addr=ffff82d04052b7d0, lockval=01350135,
> not locked
> (XEN) lock:154(111298857), block:0(0)
> (XEN) Global ratelimit_lock: addr=ffff82d04052b7c0, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global lock: addr=ffff82d04052b7a0, lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global sched_free_cpu_lock: addr=ffff82d04052b770,
> lockval=00040004, not locked
> (XEN) lock:4(2963), block:0(0)
> (XEN) Global cpupool_lock: addr=ffff82d04052b750, lockval=00030003,
> not locked
> (XEN) lock:3(19403), block:0(0)
> (XEN) Global sysctl_lock: addr=ffff82d04052b740, lockval=00000000, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global domctl_lock: addr=ffff82d04052b730, lockval=00000000, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global pool_list_lock: addr=ffff82d04052b720, lockval=00010001,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global xenoprof_lock: addr=ffff82d04052b710, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global pmu_owner_lock: addr=ffff82d04052b700, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global vm_lock: addr=ffff82d04052b6f0, lockval=00130013, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global virtual_region_lock: addr=ffff82d04052b6e0,
> lockval=00010001, not locked
> (XEN) lock:1(198), block:0(0)
> (XEN) Global wc_lock: addr=ffff82d04052b6d0, lockval=00020002, not locked
> (XEN) lock:1(770), block:0(0)
> (XEN) Global tasklet_lock: addr=ffff82d04052b6c0, lockval=03790379,
> not locked
> (XEN) lock:612(52925), block:0(0)
> (XEN) Global symbols_mutex: addr=ffff82d04052b6b0, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global stopmachine_lock: addr=ffff82d04052b6a0,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global call_lock: addr=ffff82d04052b680, lockval=00040004, not
> locked
> (XEN) lock:3(27688741), block:0(0)
> (XEN) Global lock: addr=ffff82d04052b560, lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global heap_lock: addr=ffff82d04052b540, lockval=c5d0c5d0, not
> locked
> (XEN) lock:16949(5481440), block:1(34420)
> (XEN) Global crash_notes_lock: addr=ffff82d04052b520,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global lock: addr=ffff82d04052b500, lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global global_virq_handlers_lock: addr=ffff82d04052b4f0,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global domlist_update_lock: addr=ffff82d04052b4d0,
> lockval=00010001, not locked
> (XEN) lock:1(105), block:0(0)
> (XEN) Global debugtrace_lock: addr=ffff82d04052b4b0, lockval=00030003,
> not locked
> (XEN) lock:3(3327), block:0(0)
> (XEN) Global accounting_lock: addr=ffff82d04052b480, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Global efi_rs_lock: addr=ffff82d04052b020, lockval=00000000, not
> locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Domain 0 event_lock: addr=ffff83027c80f110, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Domain 0 page_alloc_lock: addr=ffff83027c80f028,
> lockval=05cf05cf, not locked
> (XEN) lock:1487(3914530), block:0(0)
> (XEN) Domain 0 domain_lock: addr=ffff83027c80f018, lockval=00000000,
> not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Domain 32767 page_alloc_lock: addr=ffff83027c875028,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Domain 32767 domain_lock: addr=ffff83027c875018,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Domain 32753 page_alloc_lock: addr=ffff83027c8c8028,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Domain 32753 domain_lock: addr=ffff83027c8c8018,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) Domain 32754 page_alloc_lock: addr=ffff83027c8c9028,
> lockval=18001800, not locked
> (XEN) lock:6144(101574), block:0(0)
> (XEN) Domain 32754 domain_lock: addr=ffff83027c8c9018,
> lockval=00000000, not locked
> (XEN) lock:0(0), block:0(0)
> (XEN) debugtrace_dump() global buffer starting
> 1 cpupool_create(pool=0,sched=0)
> 2 Created cpupool 0 with scheduler SMP Credit Scheduler rev2 (credit2)
> 3 cpupool_add_domain(dom=0,pool=0) n_dom 1 rc 0
> (XEN) wrap: 0
> (XEN) debugtrace_dump() global buffer finished
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 1:
> (XEN) FATAL PAGE FAULT
> (XEN) [error_code=0000]
> (XEN) Faulting linear address: 0000000000000000
> (XEN) ****************************************
>
>
> Cheers,
>
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 10/22] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
2021-01-28 13:53 ` Jan Beulich
@ 2021-01-28 14:21 ` Julien Grall
2021-01-28 14:36 ` Jan Beulich
0 siblings, 1 reply; 93+ messages in thread
From: Julien Grall @ 2021-01-28 14:21 UTC (permalink / raw)
To: Jan Beulich, Oleksandr Tyshchenko
Cc: Oleksandr Tyshchenko, Paul Durrant, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Stefano Stabellini,
Jun Nakajima, Kevin Tian, Julien Grall, xen-devel
Hi Jan,
On 28/01/2021 13:53, Jan Beulich wrote:
> On 28.01.2021 14:41, Julien Grall wrote:
>> Hi Oleksandr,
>>
>> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>
>>> The IOREQ is a common feature now and these fields will be used
>>> on Arm as is. Move them to common struct vcpu as a part of new
>>> struct vcpu_io and drop duplicating "io" prefixes. Also move
>>> enum hvm_io_completion to xen/sched.h and remove "hvm" prefixes.
>>>
>>> This patch completely removes layering violation in the common code.
>>>
>>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>> Reviewed-by: Julien Grall <jgrall@amazon.com>
>>> Reviewed-by: Paul Durrant <paul@xen.org>
>>> Acked-by: Jan Beulich <jbeulich@suse.com>
>>> CC: Julien Grall <julien.grall@arm.com>
>>> [On Arm only]
>>> Tested-by: Wei Chen <Wei.Chen@arm.com>
>>
>> I seem to have trouble running Xen on x86 platform after this patch is
>> applied (see trace below).
>>
>> The bisector pointed out to this patch but I can't quite figure out why
>> this is breaking.
>>
>> Does this ring any bell to someone?
>
> Memory overwriting / corruption? This ...
>
>> (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
>> (XEN) ----[ Xen-4.15-unstable x86_64 debug=n gcov=y Tainted: C ]----
>> (XEN) CPU: 1
>> (XEN) RIP: e008:[<ffff82d04041c1c7>]
>> x86_64/entry.S#restore_all_guest+0x7/0x145
>> (XEN) RFLAGS: 0000000000010002 CONTEXT: hypervisor (d0v0)
>> (XEN) rax: 00000000000000ff rbx: ffff83027c806000 rcx: ffff82d0406c9a80
>> (XEN) rdx: 0000000000000000 rsi: fffffffffffffed9 rdi: 0000000000000001
>> (XEN) rbp: ffff83027c887df0 rsp: ffff83027c887ef8 r8: 00000000aaa8946e
>> (XEN) r9: 0000000000000002 r10: ffff83027c806040 r11: ffff83027c8cc020
>> (XEN) r12: ffff83027c80f000 r13: ffff83027c895000 r14: 0000000000000000
>> (XEN) r15: 0000000000000000 cr0: 0000000080050033 cr4: 00000000003426e0
>> (XEN) cr3: 0000000273a2d000 cr2: 0000000000000000
>> (XEN) fsb: 0000000000000000 gsb: 0000000000000000 gss: 0000000000000000
>> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
>> (XEN) Xen code around <ffff82d04041c1c7>
>> (x86_64/entry.S#restore_all_guest+0x7/0x145):
>> (XEN) 00 48 8b 93 98 0d 00 00 <44> 8b 3a 4c 8b 8b 68 0b 00 00 ba ff 7f
>
> ... is
>
> restore_all_guest:
> ASSERT_INTERRUPTS_DISABLED
>
> /* Stash guest SPEC_CTRL value while we can read struct vcpu. */
> mov VCPU_arch_msrs(%rbx), %rdx
> mov VCPUMSR_spec_ctrl_raw(%rdx), %r15d
>
> i.e. Dom0/vCPU0's v->arch.msrs got zeroed in an unintended way,
> hence %rdx is zero here (%rbx looks at least plausible).
>
> I take it that you double check this isn't an incremental build
> issue, i.e. entry.o for some reason not having got rebuilt
> despite struct vcpu's layout having changed?
I was going to reply back on my e-mail with more debugging information.
It seems that this is a build issue as if I clean the repo the error
disappear.
The error happens when I move from staging to a batch with this series
applied without a cleaning the tree. It also happens the other way
around as well.
Removing entry.o or asm-offsets.h before building doesn't help. Any
other idea?
On a side note, it looks like asm-offsets.h doesn't get rebuild when
Kconfig change. I noticed an issue when trying to turn on the perf counters.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2021-01-28 13:39 ` Oleksandr
@ 2021-01-28 14:29 ` Oleksandr
2021-01-28 14:41 ` Julien Grall
0 siblings, 1 reply; 93+ messages in thread
From: Oleksandr @ 2021-01-28 14:29 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, Julien Grall, Stefano Stabellini, Volodymyr Babchuk,
Andrew Cooper, George Dunlap, Ian Jackson, Jan Beulich, Wei Liu,
Oleksandr Tyshchenko
Hi Julien
On 28.01.21 15:39, Oleksandr wrote:
>
> On 28.01.21 13:33, Oleksandr wrote:
>
> Hi Julien
>
>>
>> On 28.01.21 11:40, Julien Grall wrote:
>>
>> Hi Julien
>>
>>> Hi Oleksandr,
>>>
>>> On 27/01/2021 19:20, Oleksandr wrote:
>>> > >>> So I think we may be able to drop the include from
>>> asm/hvm/domain.h
>>>> (this would avoid to include it everywhere...).
>>>>
>>>> I have tried that, but other CUs use definitions from
>>>> public/hvm/dm_op.h, for example:
>>>>
>>>> p2m-pt.c: In function 'p2m_type_to_flags':
>>>> p2m-pt.c:87:33: error: 'XEN_DMOP_IOREQ_MEM_ACCESS_WRITE' undeclared
>>>> (first use in this function)
>>>> if ( p2m->ioreq.flags & XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
>>>> ^
>>>> So, I would prefer to leave it as is, please let me know if you
>>>> think otherwise.
>>>
>>> AFAICT, there is only 2 places (p2m-pt.c and p2m-ept.c) that
>>> requires <public/hvm/dm_op.h> but doesn't directly include it.
>>> Folding the diff below in patch #4 should do the job:
>>
>> ok, will do
>
>
> Just to clarify, you mentioned about patch #4, but shouldn't we make
> these changes in patch #9 which actually tries to sort dm related stuff?
or a least in patch #8 which moves the stuff from asm-x86/hvm/domain.h
to xen/ioreq.h (including the user of XEN_DMOP_IO_RANGE_PCI),
what do you think?
As for proposed changes, can confirm they work.
>
>
>>
>>
>>>
>>> diff --git a/xen/arch/x86/mm/p2m-ept.c b/xen/arch/x86/mm/p2m-ept.c
>>> index 975ab403f235..23d411f01d2f 100644
>>> --- a/xen/arch/x86/mm/p2m-ept.c
>>> +++ b/xen/arch/x86/mm/p2m-ept.c
>>> @@ -17,6 +17,7 @@
>>>
>>> #include <xen/domain_page.h>
>>> #include <xen/sched.h>
>>> +#include <public/hvm/dm_op.h>
>>> #include <asm/altp2m.h>
>>> #include <asm/current.h>
>>> #include <asm/paging.h>
>>> diff --git a/xen/arch/x86/mm/p2m-pt.c b/xen/arch/x86/mm/p2m-pt.c
>>> index c43d5d0413a1..f2afcf49a368 100644
>>> --- a/xen/arch/x86/mm/p2m-pt.c
>>> +++ b/xen/arch/x86/mm/p2m-pt.c
>>> @@ -27,6 +27,7 @@
>>> #include <xen/vm_event.h>
>>> #include <xen/event.h>
>>> #include <xen/trace.h>
>>> +#include <public/hvm/dm_op.h>
>>> #include <public/vm_event.h>
>>> #include <asm/altp2m.h>
>>> #include <asm/domain.h>
>>> diff --git a/xen/include/asm-x86/hvm/domain.h
>>> b/xen/include/asm-x86/hvm/domain.h
>>> index 3b36c2f41fa1..f26c1a2e2d5f 100644
>>> --- a/xen/include/asm-x86/hvm/domain.h
>>> +++ b/xen/include/asm-x86/hvm/domain.h
>>> @@ -28,8 +28,6 @@
>>> #include <asm/hvm/vmx/vmcs.h>
>>> #include <asm/hvm/svm/vmcb.h>
>>>
>>> -#include <public/hvm/dm_op.h>
>>> -
>>> #ifdef CONFIG_MEM_SHARING
>>> struct mem_sharing_domain
>>> {
>>>
>>> You would then need to move the include of <public/hvm/dm_op.h> in
>>> <xen/dm.h> from this patch to patch #9.
>>
>> yes, sure
>>
>>
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 10/22] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
2021-01-28 14:21 ` Julien Grall
@ 2021-01-28 14:36 ` Jan Beulich
2021-01-28 14:49 ` Andrew Cooper
` (2 more replies)
0 siblings, 3 replies; 93+ messages in thread
From: Jan Beulich @ 2021-01-28 14:36 UTC (permalink / raw)
To: Julien Grall
Cc: Oleksandr Tyshchenko, Paul Durrant, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Stefano Stabellini,
Jun Nakajima, Kevin Tian, Julien Grall, xen-devel,
Oleksandr Tyshchenko
On 28.01.2021 15:21, Julien Grall wrote:
> On 28/01/2021 13:53, Jan Beulich wrote:
>> On 28.01.2021 14:41, Julien Grall wrote:
>>> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>>>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>>
>>>> The IOREQ is a common feature now and these fields will be used
>>>> on Arm as is. Move them to common struct vcpu as a part of new
>>>> struct vcpu_io and drop duplicating "io" prefixes. Also move
>>>> enum hvm_io_completion to xen/sched.h and remove "hvm" prefixes.
>>>>
>>>> This patch completely removes layering violation in the common code.
>>>>
>>>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>> Reviewed-by: Julien Grall <jgrall@amazon.com>
>>>> Reviewed-by: Paul Durrant <paul@xen.org>
>>>> Acked-by: Jan Beulich <jbeulich@suse.com>
>>>> CC: Julien Grall <julien.grall@arm.com>
>>>> [On Arm only]
>>>> Tested-by: Wei Chen <Wei.Chen@arm.com>
>>>
>>> I seem to have trouble running Xen on x86 platform after this patch is
>>> applied (see trace below).
>>>
>>> The bisector pointed out to this patch but I can't quite figure out why
>>> this is breaking.
>>>
>>> Does this ring any bell to someone?
>>
>> Memory overwriting / corruption? This ...
>>
>>> (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
>>> (XEN) ----[ Xen-4.15-unstable x86_64 debug=n gcov=y Tainted: C ]----
>>> (XEN) CPU: 1
>>> (XEN) RIP: e008:[<ffff82d04041c1c7>]
>>> x86_64/entry.S#restore_all_guest+0x7/0x145
>>> (XEN) RFLAGS: 0000000000010002 CONTEXT: hypervisor (d0v0)
>>> (XEN) rax: 00000000000000ff rbx: ffff83027c806000 rcx: ffff82d0406c9a80
>>> (XEN) rdx: 0000000000000000 rsi: fffffffffffffed9 rdi: 0000000000000001
>>> (XEN) rbp: ffff83027c887df0 rsp: ffff83027c887ef8 r8: 00000000aaa8946e
>>> (XEN) r9: 0000000000000002 r10: ffff83027c806040 r11: ffff83027c8cc020
>>> (XEN) r12: ffff83027c80f000 r13: ffff83027c895000 r14: 0000000000000000
>>> (XEN) r15: 0000000000000000 cr0: 0000000080050033 cr4: 00000000003426e0
>>> (XEN) cr3: 0000000273a2d000 cr2: 0000000000000000
>>> (XEN) fsb: 0000000000000000 gsb: 0000000000000000 gss: 0000000000000000
>>> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
>>> (XEN) Xen code around <ffff82d04041c1c7>
>>> (x86_64/entry.S#restore_all_guest+0x7/0x145):
>>> (XEN) 00 48 8b 93 98 0d 00 00 <44> 8b 3a 4c 8b 8b 68 0b 00 00 ba ff 7f
>>
>> ... is
>>
>> restore_all_guest:
>> ASSERT_INTERRUPTS_DISABLED
>>
>> /* Stash guest SPEC_CTRL value while we can read struct vcpu. */
>> mov VCPU_arch_msrs(%rbx), %rdx
>> mov VCPUMSR_spec_ctrl_raw(%rdx), %r15d
>>
>> i.e. Dom0/vCPU0's v->arch.msrs got zeroed in an unintended way,
>> hence %rdx is zero here (%rbx looks at least plausible).
>>
>> I take it that you double check this isn't an incremental build
>> issue, i.e. entry.o for some reason not having got rebuilt
>> despite struct vcpu's layout having changed?
>
> I was going to reply back on my e-mail with more debugging information.
> It seems that this is a build issue as if I clean the repo the error
> disappear.
>
> The error happens when I move from staging to a batch with this series
> applied without a cleaning the tree. It also happens the other way
> around as well.
>
> Removing entry.o or asm-offsets.h before building doesn't help. Any
> other idea?
No, I'd need to know how exactly to repro and then try to debug.
> On a side note, it looks like asm-offsets.h doesn't get rebuild when
> Kconfig change. I noticed an issue when trying to turn on the perf counters.
That's bad and needs fixing. Assuming you mean the kconfig change
in fact incurs a change to asm-offsets.h. Otherwise there might
be a move-if-changed somewhere preventing unnecessary rebuilding.
Jan
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
2021-01-27 17:45 ` Oleksandr
@ 2021-01-28 14:37 ` Oleksandr
2021-01-28 15:14 ` Julien Grall
0 siblings, 1 reply; 93+ messages in thread
From: Oleksandr @ 2021-01-28 14:37 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Andrew Cooper, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Tim Deegan,
Daniel De Graaf, Volodymyr Babchuk, Bertrand Marquis, Wei Chen,
Kaly Xin, Artem Mygaiev, Alex Bennée
Hi Julien, all
On 27.01.21 19:45, Oleksandr wrote:
>
> On 27.01.21 19:42, Julien Grall wrote:
>
> Hi
>
>>
>>
>> On 27/01/2021 17:37, Oleksandr wrote:
>>>
>>> On 27.01.21 19:33, Julien Grall wrote:
>>>
>>> Hi Julien
>>>
>>>>
>>>>
>>>> On 27/01/2021 16:50, Oleksandr wrote:
>>>>>
>>>>> On 27.01.21 18:43, Julien Grall wrote:
>>>>>> Hi Oleksandr,
>>>>>
>>>>> Hi Julien
>>>>>
>>>>>
>>>>>>
>>>>>> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>>>>>>> ***
>>>>>>>
>>>>>>> Patch series [8] was rebased on recent "staging branch"
>>>>>>> (5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is
>>>>>>> unmapped) and tested on
>>>>>>> Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio
>>>>>>> disk backend [9]
>>>>>>> running in driver domain and unmodified Linux Guest running on
>>>>>>> existing
>>>>>>> virtio-blk driver (frontend). No issues were observed. Guest
>>>>>>> domain 'reboot/destroy'
>>>>>>> use-cases work properly. Patch series was only build-tested on x86.
>>>>>>>
>>>>>>> Please note, build-test passed for the following modes:
>>>>>>> 1. x86: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y (default)
>>>>>>> 2. x86: #CONFIG_HVM is not set / #CONFIG_IOREQ_SERVER is not set
>>>>>>> 3. Arm64: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
>>>>>>> 4. Arm64: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
>>>>>>> 5. Arm32: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
>>>>>>> 6. Arm32: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
>>>>>>
>>>>>> I thought I woudl give a try to test the code, but I can't find a
>>>>>> way to enable CONFIG_IOREQ_SERVER from the UI.
>>>>>>
>>>>>> Looking at the Kconfig, it looks like CONFIG_IOREQ_SERVER doesn't
>>>>>> have a prompt and is not selected by Arm.
>>>>>>
>>>>>> Can you provide details how this can be built on Arm?
>>>>>
>>>>> Please apply the attached patch to select IOREQ on Arm.
>>>>
>>>> This is roughly what I wrote. I think a user should be able to
>>>> select IOREQ via the menuconfig without any additional patch on top
>>>> of your series.
>>>>
>>>> Can you include a patch that would enable that?
>>>
>>> Yes, do you prefer a separate patch or required changes could be
>>> folded in patch #14?
>>
>> I would do a separate patch as IOREQ only really work after the full
>> series applies.
>
>
> Makes sense, I will do it for V6
Could we please negotiate *the last posting time* for me to able to
prepare and push V6 not later than it?
>
>
>
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2021-01-28 14:29 ` Oleksandr
@ 2021-01-28 14:41 ` Julien Grall
2021-01-28 14:52 ` Oleksandr
0 siblings, 1 reply; 93+ messages in thread
From: Julien Grall @ 2021-01-28 14:41 UTC (permalink / raw)
To: Oleksandr
Cc: xen-devel, Julien Grall, Stefano Stabellini, Volodymyr Babchuk,
Andrew Cooper, George Dunlap, Ian Jackson, Jan Beulich, Wei Liu,
Oleksandr Tyshchenko
On 28/01/2021 14:29, Oleksandr wrote:
>
> Hi Julien
>
>
> On 28.01.21 15:39, Oleksandr wrote:
>>
>> On 28.01.21 13:33, Oleksandr wrote:
>>
>> Hi Julien
>>
>>>
>>> On 28.01.21 11:40, Julien Grall wrote:
>>>
>>> Hi Julien
>>>
>>>> Hi Oleksandr,
>>>>
>>>> On 27/01/2021 19:20, Oleksandr wrote:
>>>> > >>> So I think we may be able to drop the include from
>>>> asm/hvm/domain.h
>>>>> (this would avoid to include it everywhere...).
>>>>>
>>>>> I have tried that, but other CUs use definitions from
>>>>> public/hvm/dm_op.h, for example:
>>>>>
>>>>> p2m-pt.c: In function 'p2m_type_to_flags':
>>>>> p2m-pt.c:87:33: error: 'XEN_DMOP_IOREQ_MEM_ACCESS_WRITE' undeclared
>>>>> (first use in this function)
>>>>> if ( p2m->ioreq.flags & XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
>>>>> ^
>>>>> So, I would prefer to leave it as is, please let me know if you
>>>>> think otherwise.
>>>>
>>>> AFAICT, there is only 2 places (p2m-pt.c and p2m-ept.c) that
>>>> requires <public/hvm/dm_op.h> but doesn't directly include it.
>>>> Folding the diff below in patch #4 should do the job:
>>>
>>> ok, will do
>>
>>
>> Just to clarify, you mentioned about patch #4, but shouldn't we make
>> these changes in patch #9 which actually tries to sort dm related stuff?
>
> or a least in patch #8 which moves the stuff from asm-x86/hvm/domain.h
> to xen/ioreq.h (including the user of XEN_DMOP_IO_RANGE_PCI),
I looked at the header asm-x86/hvm/domain.h after applying patch #4,
there is nothing requiring DMOP from there.
I tried to build it with this series applied up to patch #4 + my diff.
It does build without any issue.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 10/22] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
2021-01-28 14:36 ` Jan Beulich
@ 2021-01-28 14:49 ` Andrew Cooper
2021-01-28 14:51 ` Ian Jackson
2021-01-28 15:01 ` Julien Grall
2 siblings, 0 replies; 93+ messages in thread
From: Andrew Cooper @ 2021-01-28 14:49 UTC (permalink / raw)
To: Jan Beulich, Julien Grall
Cc: Oleksandr Tyshchenko, Paul Durrant, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Stefano Stabellini,
Jun Nakajima, Kevin Tian, Julien Grall, xen-devel,
Oleksandr Tyshchenko
On 28/01/2021 14:36, Jan Beulich wrote:
> On 28.01.2021 15:21, Julien Grall wrote:
>> I was going to reply back on my e-mail with more debugging information.
>> It seems that this is a build issue as if I clean the repo the error
>> disappear.
>>
>> The error happens when I move from staging to a batch with this series
>> applied without a cleaning the tree. It also happens the other way
>> around as well.
>>
>> Removing entry.o or asm-offsets.h before building doesn't help. Any
>> other idea?
> No, I'd need to know how exactly to repro and then try to debug.
>
>> On a side note, it looks like asm-offsets.h doesn't get rebuild when
>> Kconfig change. I noticed an issue when trying to turn on the perf counters.
> That's bad and needs fixing. Assuming you mean the kconfig change
> in fact incurs a change to asm-offsets.h. Otherwise there might
> be a move-if-changed somewhere preventing unnecessary rebuilding.
FYI this is the incremental build breakage I reported in IRC last week.
There is definitely a missing dependency somewhere, but I just far
haven't been able to precisely what.
~Andrew
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 10/22] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
2021-01-28 14:36 ` Jan Beulich
2021-01-28 14:49 ` Andrew Cooper
@ 2021-01-28 14:51 ` Ian Jackson
2021-01-28 14:54 ` Jan Beulich
2021-01-28 15:01 ` Julien Grall
2 siblings, 1 reply; 93+ messages in thread
From: Ian Jackson @ 2021-01-28 14:51 UTC (permalink / raw)
To: Jan Beulich
Cc: Julien Grall, Oleksandr Tyshchenko, Paul Durrant, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Stefano Stabellini, Jun Nakajima,
Kevin Tian, Julien Grall, xen-devel, Oleksandr Tyshchenko
Jan Beulich writes ("Re: [PATCH V5 10/22] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu"):
> On 28.01.2021 15:21, Julien Grall wrote:
> > It seems that this is a build issue as if I clean the repo the error
> > disappear.
> >
> > The error happens when I move from staging to a batch with this series
> > applied without a cleaning the tree. It also happens the other way
> > around as well.
How vexing.
> > Removing entry.o or asm-offsets.h before building doesn't help. Any
> > other idea?
>
> No, I'd need to know how exactly to repro and then try to debug.
IMO this problem is not a blocker for pushing this today or tomorrow.
Unless someone disagrees ?
Ian.
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2021-01-28 14:41 ` Julien Grall
@ 2021-01-28 14:52 ` Oleksandr
2021-01-28 15:08 ` Julien Grall
0 siblings, 1 reply; 93+ messages in thread
From: Oleksandr @ 2021-01-28 14:52 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, Julien Grall, Stefano Stabellini, Volodymyr Babchuk,
Andrew Cooper, George Dunlap, Ian Jackson, Jan Beulich, Wei Liu,
Oleksandr Tyshchenko
On 28.01.21 16:41, Julien Grall wrote:
Hi Julien
>
>
> On 28/01/2021 14:29, Oleksandr wrote:
>>
>> Hi Julien
>>
>>
>> On 28.01.21 15:39, Oleksandr wrote:
>>>
>>> On 28.01.21 13:33, Oleksandr wrote:
>>>
>>> Hi Julien
>>>
>>>>
>>>> On 28.01.21 11:40, Julien Grall wrote:
>>>>
>>>> Hi Julien
>>>>
>>>>> Hi Oleksandr,
>>>>>
>>>>> On 27/01/2021 19:20, Oleksandr wrote:
>>>>> > >>> So I think we may be able to drop the include from
>>>>> asm/hvm/domain.h
>>>>>> (this would avoid to include it everywhere...).
>>>>>>
>>>>>> I have tried that, but other CUs use definitions from
>>>>>> public/hvm/dm_op.h, for example:
>>>>>>
>>>>>> p2m-pt.c: In function 'p2m_type_to_flags':
>>>>>> p2m-pt.c:87:33: error: 'XEN_DMOP_IOREQ_MEM_ACCESS_WRITE'
>>>>>> undeclared (first use in this function)
>>>>>> if ( p2m->ioreq.flags & XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
>>>>>> ^
>>>>>> So, I would prefer to leave it as is, please let me know if you
>>>>>> think otherwise.
>>>>>
>>>>> AFAICT, there is only 2 places (p2m-pt.c and p2m-ept.c) that
>>>>> requires <public/hvm/dm_op.h> but doesn't directly include it.
>>>>> Folding the diff below in patch #4 should do the job:
>>>>
>>>> ok, will do
>>>
>>>
>>> Just to clarify, you mentioned about patch #4, but shouldn't we make
>>> these changes in patch #9 which actually tries to sort dm related
>>> stuff?
>>
>> or a least in patch #8 which moves the stuff from
>> asm-x86/hvm/domain.h to xen/ioreq.h (including the user of
>> XEN_DMOP_IO_RANGE_PCI),
>
> I looked at the header asm-x86/hvm/domain.h after applying patch #4,
> there is nothing requiring DMOP from there.
>
> I tried to build it with this series applied up to patch #4 + my diff.
> It does build without any issue.
Hmm, interesting. I might miss something, but I got an build issue if I
split these changes with patch #4 and build the series up to this patch:
In file included from
/media/b/build/build/tmp/work/x86_64-xt-linux/domd-image-weston/1.0-r0/repo/build/tmp/work/aarch64-poky-linux/xen/4.14.0+gitAUTOINC+2c6e5a8ceb-r0/git/xen/include/asm/domain.h:7:0,
from
/media/b/build/build/tmp/work/x86_64-xt-linux/domd-image-weston/1.0-r0/repo/build/tmp/work/aarch64-poky-linux/xen/4.14.0+gitAUTOINC+2c6e5a8ceb-r0/git/xen/include/xen/domain.h:8,
from
/media/b/build/build/tmp/work/x86_64-xt-linux/domd-image-weston/1.0-r0/repo/build/tmp/work/aarch64-poky-linux/xen/4.14.0+gitAUTOINC+2c6e5a8ceb-r0/git/xen/include/xen/sched.h:11,
from
/media/b/build/build/tmp/work/x86_64-xt-linux/domd-image-weston/1.0-r0/repo/build/tmp/work/aarch64-poky-linux/xen/4.14.0+gitAUTOINC+2c6e5a8ceb-r0/git/xen/include/asm/paging.h:29,
from
/media/b/build/build/tmp/work/x86_64-xt-linux/domd-image-weston/1.0-r0/repo/build/tmp/work/aarch64-poky-linux/xen/4.14.0+gitAUTOINC+2c6e5a8ceb-r0/git/xen/include/asm/guest_access.h:11,
from
/media/b/build/build/tmp/work/x86_64-xt-linux/domd-image-weston/1.0-r0/repo/build/tmp/work/aarch64-poky-linux/xen/4.14.0+gitAUTOINC+2c6e5a8ceb-r0/git/xen/include/xen/guest_access.h:10,
from compat.c:1:
/media/b/build/build/tmp/work/x86_64-xt-linux/domd-image-weston/1.0-r0/repo/build/tmp/work/aarch64-poky-linux/xen/4.14.0+gitAUTOINC+2c6e5a8ceb-r0/git/xen/include/asm/hvm/domain.h:44:28:
error: 'XEN_DMOP_IO_RANGE_PCI' undeclared here (not in a function)
#define NR_IO_RANGE_TYPES (XEN_DMOP_IO_RANGE_PCI + 1)
^
/media/b/build/build/tmp/work/x86_64-xt-linux/domd-image-weston/1.0-r0/repo/build/tmp/work/aarch64-poky-linux/xen/4.14.0+gitAUTOINC+2c6e5a8ceb-r0/git/xen/include/asm/hvm/domain.h:60:35:
note: in expansion of macro 'NR_IO_RANGE_TYPES'
struct rangeset *range[NR_IO_RANGE_TYPES];
^
/media/b/build/build/tmp/work/x86_64-xt-linux/domd-image-weston/1.0-r0/repo/build/tmp/work/aarch64-poky-linux/xen/4.14.0+gitAUTOINC+2c6e5a8ceb-r0/git/xen/Rules.mk:189:
recipe for target 'compat.o' failed
make[3]: *** [compat.o] Error 1
>
> Cheers,
>
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 10/22] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
2021-01-28 14:51 ` Ian Jackson
@ 2021-01-28 14:54 ` Jan Beulich
2021-01-28 16:15 ` Andrew Cooper
0 siblings, 1 reply; 93+ messages in thread
From: Jan Beulich @ 2021-01-28 14:54 UTC (permalink / raw)
To: Ian Jackson
Cc: Julien Grall, Oleksandr Tyshchenko, Paul Durrant, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Stefano Stabellini, Jun Nakajima,
Kevin Tian, Julien Grall, xen-devel, Oleksandr Tyshchenko
On 28.01.2021 15:51, Ian Jackson wrote:
> Jan Beulich writes ("Re: [PATCH V5 10/22] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu"):
>> On 28.01.2021 15:21, Julien Grall wrote:
>>> It seems that this is a build issue as if I clean the repo the error
>>> disappear.
>>>
>>> The error happens when I move from staging to a batch with this series
>>> applied without a cleaning the tree. It also happens the other way
>>> around as well.
>
> How vexing.
>
>>> Removing entry.o or asm-offsets.h before building doesn't help. Any
>>> other idea?
>>
>> No, I'd need to know how exactly to repro and then try to debug.
>
> IMO this problem is not a blocker for pushing this today or tomorrow.
> Unless someone disagrees ?
No, I don't think this is caused by this series, and Andrew's
reply of having noticed the same supports this.
Jan
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 10/22] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
2021-01-28 14:36 ` Jan Beulich
2021-01-28 14:49 ` Andrew Cooper
2021-01-28 14:51 ` Ian Jackson
@ 2021-01-28 15:01 ` Julien Grall
2 siblings, 0 replies; 93+ messages in thread
From: Julien Grall @ 2021-01-28 15:01 UTC (permalink / raw)
To: Jan Beulich
Cc: Oleksandr Tyshchenko, Paul Durrant, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Stefano Stabellini,
Jun Nakajima, Kevin Tian, Julien Grall, xen-devel,
Oleksandr Tyshchenko
Hi Jan,
On 28/01/2021 14:36, Jan Beulich wrote:
> On 28.01.2021 15:21, Julien Grall wrote:
>> On 28/01/2021 13:53, Jan Beulich wrote:
>>> On 28.01.2021 14:41, Julien Grall wrote:
>>>> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>>>>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>>>
>>>>> The IOREQ is a common feature now and these fields will be used
>>>>> on Arm as is. Move them to common struct vcpu as a part of new
>>>>> struct vcpu_io and drop duplicating "io" prefixes. Also move
>>>>> enum hvm_io_completion to xen/sched.h and remove "hvm" prefixes.
>>>>>
>>>>> This patch completely removes layering violation in the common code.
>>>>>
>>>>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>>> Reviewed-by: Julien Grall <jgrall@amazon.com>
>>>>> Reviewed-by: Paul Durrant <paul@xen.org>
>>>>> Acked-by: Jan Beulich <jbeulich@suse.com>
>>>>> CC: Julien Grall <julien.grall@arm.com>
>>>>> [On Arm only]
>>>>> Tested-by: Wei Chen <Wei.Chen@arm.com>
>>>>
>>>> I seem to have trouble running Xen on x86 platform after this patch is
>>>> applied (see trace below).
>>>>
>>>> The bisector pointed out to this patch but I can't quite figure out why
>>>> this is breaking.
>>>>
>>>> Does this ring any bell to someone?
>>>
>>> Memory overwriting / corruption? This ...
>>>
>>>> (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
>>>> (XEN) ----[ Xen-4.15-unstable x86_64 debug=n gcov=y Tainted: C ]----
>>>> (XEN) CPU: 1
>>>> (XEN) RIP: e008:[<ffff82d04041c1c7>]
>>>> x86_64/entry.S#restore_all_guest+0x7/0x145
>>>> (XEN) RFLAGS: 0000000000010002 CONTEXT: hypervisor (d0v0)
>>>> (XEN) rax: 00000000000000ff rbx: ffff83027c806000 rcx: ffff82d0406c9a80
>>>> (XEN) rdx: 0000000000000000 rsi: fffffffffffffed9 rdi: 0000000000000001
>>>> (XEN) rbp: ffff83027c887df0 rsp: ffff83027c887ef8 r8: 00000000aaa8946e
>>>> (XEN) r9: 0000000000000002 r10: ffff83027c806040 r11: ffff83027c8cc020
>>>> (XEN) r12: ffff83027c80f000 r13: ffff83027c895000 r14: 0000000000000000
>>>> (XEN) r15: 0000000000000000 cr0: 0000000080050033 cr4: 00000000003426e0
>>>> (XEN) cr3: 0000000273a2d000 cr2: 0000000000000000
>>>> (XEN) fsb: 0000000000000000 gsb: 0000000000000000 gss: 0000000000000000
>>>> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
>>>> (XEN) Xen code around <ffff82d04041c1c7>
>>>> (x86_64/entry.S#restore_all_guest+0x7/0x145):
>>>> (XEN) 00 48 8b 93 98 0d 00 00 <44> 8b 3a 4c 8b 8b 68 0b 00 00 ba ff 7f
>>>
>>> ... is
>>>
>>> restore_all_guest:
>>> ASSERT_INTERRUPTS_DISABLED
>>>
>>> /* Stash guest SPEC_CTRL value while we can read struct vcpu. */
>>> mov VCPU_arch_msrs(%rbx), %rdx
>>> mov VCPUMSR_spec_ctrl_raw(%rdx), %r15d
>>>
>>> i.e. Dom0/vCPU0's v->arch.msrs got zeroed in an unintended way,
>>> hence %rdx is zero here (%rbx looks at least plausible).
>>>
>>> I take it that you double check this isn't an incremental build
>>> issue, i.e. entry.o for some reason not having got rebuilt
>>> despite struct vcpu's layout having changed?
>>
>> I was going to reply back on my e-mail with more debugging information.
>> It seems that this is a build issue as if I clean the repo the error
>> disappear.
>>
>> The error happens when I move from staging to a batch with this series
>> applied without a cleaning the tree. It also happens the other way
>> around as well.
>>
>> Removing entry.o or asm-offsets.h before building doesn't help. Any
>> other idea?
>
> No, I'd need to know how exactly to repro and then try to debug.
I have tried to remove all the *.o and still the same issues. So it may
be something related to the generation of the headers.
This is not related to this series, so I am not going to spend more time
on this bug today. The way I reproduced it was to use [1] as .config (it
is a random config from gitlab) and then switched between
staging (2b4b33ffe7d6 "libs/foreignmemory: Implement on NetBSD" at the
time of the writing) and the branch ioreq/ioreq_4.14_ml6 from [2].
Just in case it matters this on Ubuntu 18.04.5 with GCC 7.5.0.
Cheers,
[1] https://pastebin.com/S6ELa493
[2] https://github.com/otyshchenko1/xen.git
--
Julien Grall
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2021-01-28 14:52 ` Oleksandr
@ 2021-01-28 15:08 ` Julien Grall
2021-01-28 17:50 ` Oleksandr
0 siblings, 1 reply; 93+ messages in thread
From: Julien Grall @ 2021-01-28 15:08 UTC (permalink / raw)
To: Oleksandr
Cc: xen-devel, Julien Grall, Stefano Stabellini, Volodymyr Babchuk,
Andrew Cooper, George Dunlap, Ian Jackson, Jan Beulich, Wei Liu,
Oleksandr Tyshchenko
On 28/01/2021 14:52, Oleksandr wrote:
>
> On 28.01.21 16:41, Julien Grall wrote:
>> On 28/01/2021 14:29, Oleksandr wrote:
>>> On 28.01.21 15:39, Oleksandr wrote:
>>>>
>>>> On 28.01.21 13:33, Oleksandr wrote:
>>>>
>>>> Hi Julien
>>>>
>>>>>
>>>>> On 28.01.21 11:40, Julien Grall wrote:
>>>>>
>>>>> Hi Julien
>>>>>
>>>>>> Hi Oleksandr,
>>>>>>
>>>>>> On 27/01/2021 19:20, Oleksandr wrote:
>>>>>> > >>> So I think we may be able to drop the include from
>>>>>> asm/hvm/domain.h
>>>>>>> (this would avoid to include it everywhere...).
>>>>>>>
>>>>>>> I have tried that, but other CUs use definitions from
>>>>>>> public/hvm/dm_op.h, for example:
>>>>>>>
>>>>>>> p2m-pt.c: In function 'p2m_type_to_flags':
>>>>>>> p2m-pt.c:87:33: error: 'XEN_DMOP_IOREQ_MEM_ACCESS_WRITE'
>>>>>>> undeclared (first use in this function)
>>>>>>> if ( p2m->ioreq.flags & XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
>>>>>>> ^
>>>>>>> So, I would prefer to leave it as is, please let me know if you
>>>>>>> think otherwise.
>>>>>>
>>>>>> AFAICT, there is only 2 places (p2m-pt.c and p2m-ept.c) that
>>>>>> requires <public/hvm/dm_op.h> but doesn't directly include it.
>>>>>> Folding the diff below in patch #4 should do the job:
>>>>>
>>>>> ok, will do
>>>>
>>>>
>>>> Just to clarify, you mentioned about patch #4, but shouldn't we make
>>>> these changes in patch #9 which actually tries to sort dm related
>>>> stuff?
>>>
>>> or a least in patch #8 which moves the stuff from
>>> asm-x86/hvm/domain.h to xen/ioreq.h (including the user of
>>> XEN_DMOP_IO_RANGE_PCI),
>>
>> I looked at the header asm-x86/hvm/domain.h after applying patch #4,
>> there is nothing requiring DMOP from there.
>>
>> I tried to build it with this series applied up to patch #4 + my diff.
>> It does build without any issue.
>
> Hmm, interesting. I might miss something, but I got an build issue if I
> split these changes with patch #4 and build the series up to this patch:
I think I looked and tried with the wrong commit. :( This was moved in
patch #7 (xen/ioreq: Make x86's hvm_ioreq_(page/vcpu/server) structs
common).
Sorry for the confusion.
Cheers,
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
2021-01-28 14:37 ` Oleksandr
@ 2021-01-28 15:14 ` Julien Grall
2021-01-28 17:55 ` Oleksandr
0 siblings, 1 reply; 93+ messages in thread
From: Julien Grall @ 2021-01-28 15:14 UTC (permalink / raw)
To: Oleksandr
Cc: xen-devel, Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Andrew Cooper, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Tim Deegan,
Daniel De Graaf, Volodymyr Babchuk, Bertrand Marquis, Wei Chen,
Kaly Xin, Artem Mygaiev, Alex Bennée
Hi,
On 28/01/2021 14:37, Oleksandr wrote:
> On 27.01.21 19:45, Oleksandr wrote:
>>
>> On 27.01.21 19:42, Julien Grall wrote:
>>
>> Hi
>>
>>>
>>>
>>> On 27/01/2021 17:37, Oleksandr wrote:
>>>>
>>>> On 27.01.21 19:33, Julien Grall wrote:
>>>>
>>>> Hi Julien
>>>>
>>>>>
>>>>>
>>>>> On 27/01/2021 16:50, Oleksandr wrote:
>>>>>>
>>>>>> On 27.01.21 18:43, Julien Grall wrote:
>>>>>>> Hi Oleksandr,
>>>>>>
>>>>>> Hi Julien
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>>>>>>>> ***
>>>>>>>>
>>>>>>>> Patch series [8] was rebased on recent "staging branch"
>>>>>>>> (5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is
>>>>>>>> unmapped) and tested on
>>>>>>>> Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio
>>>>>>>> disk backend [9]
>>>>>>>> running in driver domain and unmodified Linux Guest running on
>>>>>>>> existing
>>>>>>>> virtio-blk driver (frontend). No issues were observed. Guest
>>>>>>>> domain 'reboot/destroy'
>>>>>>>> use-cases work properly. Patch series was only build-tested on x86.
>>>>>>>>
>>>>>>>> Please note, build-test passed for the following modes:
>>>>>>>> 1. x86: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y (default)
>>>>>>>> 2. x86: #CONFIG_HVM is not set / #CONFIG_IOREQ_SERVER is not set
>>>>>>>> 3. Arm64: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
>>>>>>>> 4. Arm64: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
>>>>>>>> 5. Arm32: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
>>>>>>>> 6. Arm32: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
>>>>>>>
>>>>>>> I thought I woudl give a try to test the code, but I can't find a
>>>>>>> way to enable CONFIG_IOREQ_SERVER from the UI.
>>>>>>>
>>>>>>> Looking at the Kconfig, it looks like CONFIG_IOREQ_SERVER doesn't
>>>>>>> have a prompt and is not selected by Arm.
>>>>>>>
>>>>>>> Can you provide details how this can be built on Arm?
>>>>>>
>>>>>> Please apply the attached patch to select IOREQ on Arm.
>>>>>
>>>>> This is roughly what I wrote. I think a user should be able to
>>>>> select IOREQ via the menuconfig without any additional patch on top
>>>>> of your series.
>>>>>
>>>>> Can you include a patch that would enable that?
>>>>
>>>> Yes, do you prefer a separate patch or required changes could be
>>>> folded in patch #14?
>>>
>>> I would do a separate patch as IOREQ only really work after the full
>>> series applies.
>>
>>
>> Makes sense, I will do it for V6
>
>
> Could we please negotiate *the last posting time* for me to able to
> prepare and push V6 not later than it?
The feature freeze is going to be happen tomorrow evening. I can't give
you as specific time, but it is probably best if you respin v6 by
tomorrow morning. This should give us some slack for any last minutes
issues (if any).
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (22 preceding siblings ...)
2021-01-27 16:43 ` [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Julien Grall
@ 2021-01-28 16:11 ` Julien Grall
2021-01-28 17:24 ` Stefano Stabellini
23 siblings, 1 reply; 93+ messages in thread
From: Julien Grall @ 2021-01-28 16:11 UTC (permalink / raw)
To: Oleksandr Tyshchenko, xen-devel, Andrew Cooper, Ian Jackson
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Tim Deegan,
Daniel De Graaf, Volodymyr Babchuk, Bertrand Marquis, Wei Chen,
Kaly Xin, Artem Mygaiev, Alex Bennée
Hi,
On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
> Patch series [8] was rebased on recent "staging branch"
> (5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is unmapped) and tested on
> Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio disk backend [9]
> running in driver domain and unmodified Linux Guest running on existing
> virtio-blk driver (frontend). No issues were observed. Guest domain 'reboot/destroy'
> use-cases work properly. Patch series was only build-tested on x86.
I have done basic testing with a Linux guest on x86 and I didn't spot
any regression.
Unfortunately, I don't have a Windows VM in hand, so I can't confirm if
there is no regression there. Can anyone give a try?
On a separate topic, Andrew expressed on IRC some concern to expose the
bufioreq interface to other arch than x86. I will let him explain his
view here.
Given that IOREQ will be a tech preview on Arm (this implies the
interface is not stable) on Arm, I think we can defer the discussion
past the freeze.
For now, I would suggest to check if a Device Model is trying to create
an IOREQ server with bufioreq is enabled (see ioreq_server_create()).
This would not alleviate Andrew's concern, however it would at allow us
to judge whether the buffering concept is going to be used on Arm (I can
see some use for the Virtio doorbell).
What do others think?
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 10/22] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
2021-01-28 14:54 ` Jan Beulich
@ 2021-01-28 16:15 ` Andrew Cooper
0 siblings, 0 replies; 93+ messages in thread
From: Andrew Cooper @ 2021-01-28 16:15 UTC (permalink / raw)
To: Jan Beulich, Ian Jackson
Cc: Julien Grall, Oleksandr Tyshchenko, Paul Durrant,
Roger Pau Monné,
Wei Liu, George Dunlap, Stefano Stabellini, Jun Nakajima,
Kevin Tian, Julien Grall, xen-devel, Oleksandr Tyshchenko
On 28/01/2021 14:54, Jan Beulich wrote:
> On 28.01.2021 15:51, Ian Jackson wrote:
>> Jan Beulich writes ("Re: [PATCH V5 10/22] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu"):
>>>> Removing entry.o or asm-offsets.h before building doesn't help. Any
>>>> other idea?
>>> No, I'd need to know how exactly to repro and then try to debug.
>> IMO this problem is not a blocker for pushing this today or tomorrow.
>> Unless someone disagrees ?
> No, I don't think this is caused by this series, and Andrew's
> reply of having noticed the same supports this.
I agree. A bug manifesting in exactly this way is already present in
staging.
~Andrew
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common
2021-01-28 12:49 ` Oleksandr
@ 2021-01-28 17:13 ` Stefano Stabellini
0 siblings, 0 replies; 93+ messages in thread
From: Stefano Stabellini @ 2021-01-28 17:13 UTC (permalink / raw)
To: Oleksandr
Cc: Jan Beulich, Julien Grall, xen-devel, Oleksandr Tyshchenko,
Andrew Cooper, George Dunlap, Ian Jackson, Wei Liu,
Roger Pau Monné,
Paul Durrant, Jun Nakajima, Kevin Tian, Tim Deegan, Julien Grall,
Stefano Stabellini, Julien Grall
[-- Attachment #1: Type: text/plain, Size: 2780 bytes --]
On Thu, 28 Jan 2021, Oleksandr wrote:
> On 28.01.21 13:21, Jan Beulich wrote:
> > On 28.01.2021 12:01, Oleksandr wrote:
> > > On 27.01.21 22:46, Stefano Stabellini wrote:
> > > > On Wed, 27 Jan 2021, Oleksandr wrote:
> > > > > Thank you. I got a request to make a possibility for user to select
> > > > > IOREQ via
> > > > > the menuconfig on Arm. Saying tech preview do you mean that I also
> > > > > need to put
> > > > > it under CONFIG_EXPERT on Arm?
> > > > Normally I would say that unless it actually takes an expert to enable
> > > > the feature, it is better to make it depend on CONFIG_UNSUPPORTED [1].
> > > >
> > > > However, in this case, it might actually take an expert :-)
> > > > Additional patches are still required to enable the feature at the
> > > > toolstack level, and also the user needs to build a userspace ioreq
> > > > server. So in this case I am fine either way. I'll leave it up to you
> > > > and Julien to pick the best one.
> > > >
> > > > Either way, please add "(EXPERT)" or "(UNSUPPORTED)" in the one-line
> > > > kconfig description for ARM if possible.
> > > >
> > > >
> > > > [1] https://marc.info/?l=xen-devel&m=161168780401884
> > > Thank you for the explanation, personally I would prefer EXPERT, but...
> > >
> > > It would be really nice if we could agreed regarding that *common*
> > > Kconfig option before I submit V6 (which would probably avoid me to send
> > > V7 I hope).
> > >
> > > Now I see it as following (please note, it is a subject for new patch in
> > > this series, which should come last):
> > >
> > > +++ b/xen/common/Kconfig
> > > @@ -137,7 +137,13 @@ config HYPFS_CONFIG
> > > want to hide the .config contents from dom0.
> > >
> > > config IOREQ_SERVER
> > > - bool
> > > + bool "IOREQ support" if EXPERT || X86
> > This would make the prompt visible for x86, which we don't
> > want.
>
> Hmm, I missed that point.
>
>
> > But I guess Stefano was anyway after ...
> >
> > > + default X86
> > > + depends on HVM
> > > + ---help---
> > > + Enables generic mechanism for providing emulated devices to
> > > the guests.
> > > +
> > > + If unsure, say Y.
> > config IOREQ_SERVER
> > bool "IOREQ support (EXPERT)" if EXPERT && !X86
> > default X86
> > depends on HVM
> >
> > As the prompt should be invisible this way on x86, the (EXPERT)
> > doesn't need further qualifying, except maybe in the help text
> > (just to avoid this being taken as a possible mistake).
>
> Yes, this works as my initial version *but* is invisible on x86,
>
> what is more that indeed the *EXPERT* tag for IOREQ on x86 will look quite
> odd...
>
> Perfect, thank you.
Yep, perfect
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
2021-01-28 16:11 ` Julien Grall
@ 2021-01-28 17:24 ` Stefano Stabellini
2021-01-28 17:44 ` Julien Grall
0 siblings, 1 reply; 93+ messages in thread
From: Stefano Stabellini @ 2021-01-28 17:24 UTC (permalink / raw)
To: Julien Grall
Cc: Oleksandr Tyshchenko, xen-devel, Andrew Cooper, Ian Jackson,
Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Stefano Stabellini,
Jun Nakajima, Kevin Tian, Tim Deegan, Daniel De Graaf,
Volodymyr Babchuk, Bertrand Marquis, Wei Chen, Kaly Xin,
Artem Mygaiev, Alex Bennée
On Thu, 28 Jan 2021, Julien Grall wrote:
> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
> > Patch series [8] was rebased on recent "staging branch"
> > (5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is unmapped) and
> > tested on
> > Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio disk
> > backend [9]
> > running in driver domain and unmodified Linux Guest running on existing
> > virtio-blk driver (frontend). No issues were observed. Guest domain
> > 'reboot/destroy'
> > use-cases work properly. Patch series was only build-tested on x86.
>
> I have done basic testing with a Linux guest on x86 and I didn't spot any
> regression.
>
> Unfortunately, I don't have a Windows VM in hand, so I can't confirm if there
> is no regression there. Can anyone give a try?
>
> On a separate topic, Andrew expressed on IRC some concern to expose the
> bufioreq interface to other arch than x86. I will let him explain his view
> here.
>
> Given that IOREQ will be a tech preview on Arm (this implies the interface is
> not stable) on Arm, I think we can defer the discussion past the freeze.
Yes, I agree that we can defer the discussion.
> For now, I would suggest to check if a Device Model is trying to create an
> IOREQ server with bufioreq is enabled (see ioreq_server_create()). This would
> not alleviate Andrew's concern, however it would at allow us to judge whether
> the buffering concept is going to be used on Arm (I can see some use for the
> Virtio doorbell).
>
> What do others think?
Difficult to say. We don't have any uses today but who knows in the
future.
Maybe for now the important thing is to clarify that the bufioreq
interface won't be maintain for backward compatibility on ARM, and could
be removed without warnings, at least as long as the whole IOREQ feature
is a tech preview.
In other words, it is not like we are forced to keep bufioreq around
forever on ARM.
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
2021-01-28 17:24 ` Stefano Stabellini
@ 2021-01-28 17:44 ` Julien Grall
2021-01-28 18:10 ` Andrew Cooper
0 siblings, 1 reply; 93+ messages in thread
From: Julien Grall @ 2021-01-28 17:44 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Oleksandr Tyshchenko, xen-devel, Andrew Cooper, Ian Jackson,
Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Jun Nakajima, Kevin Tian,
Tim Deegan, Daniel De Graaf, Volodymyr Babchuk, Bertrand Marquis,
Wei Chen, Kaly Xin, Artem Mygaiev, Alex Bennée
On 28/01/2021 17:24, Stefano Stabellini wrote:
> On Thu, 28 Jan 2021, Julien Grall wrote:
>> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>> > Patch series [8] was rebased on recent "staging branch"
>>> (5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is unmapped) and
>>> tested on
>>> Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio disk
>>> backend [9]
>>> running in driver domain and unmodified Linux Guest running on existing
>>> virtio-blk driver (frontend). No issues were observed. Guest domain
>>> 'reboot/destroy'
>>> use-cases work properly. Patch series was only build-tested on x86.
>>
>> I have done basic testing with a Linux guest on x86 and I didn't spot any
>> regression.
>>
>> Unfortunately, I don't have a Windows VM in hand, so I can't confirm if there
>> is no regression there. Can anyone give a try?
>>
>> On a separate topic, Andrew expressed on IRC some concern to expose the
>> bufioreq interface to other arch than x86. I will let him explain his view
>> here.
>>
>> Given that IOREQ will be a tech preview on Arm (this implies the interface is
>> not stable) on Arm, I think we can defer the discussion past the freeze.
>
> Yes, I agree that we can defer the discussion.
>
>
>> For now, I would suggest to check if a Device Model is trying to create an
>> IOREQ server with bufioreq is enabled (see ioreq_server_create()). This would
>> not alleviate Andrew's concern, however it would at allow us to judge whether
>> the buffering concept is going to be used on Arm (I can see some use for the
>> Virtio doorbell).
>>
>> What do others think?
>
> Difficult to say. We don't have any uses today but who knows in the
> future.
I have some ideas, but Andrew so far objects on using the existing
bufioreq interface for good reasons. It is using guest_cmpxchg() which
is really expensive.
>
> Maybe for now the important thing is to clarify that the bufioreq
> interface won't be maintain for backward compatibility on ARM, and could
> be removed without warnings, at least as long as the whole IOREQ feature
> is a tech preview. >
> In other words, it is not like we are forced to keep bufioreq around
> forever on ARM.
That's correct, we are not forced to use it. But if you only document
it, then there is a fair chance this will split past the "Tech Preview".
Today, there is no caller which requires to send buffered I/O in the Xen
Arm hypervisor. So a Device Model should not need to say "I want to
allocate a page and event channel for the buffered I/O".
The check I suggested serves two purposes:
1) Catch any future upstream user of bufioreq
2) Avoid an interface to be marked supported by mistake
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2021-01-28 15:08 ` Julien Grall
@ 2021-01-28 17:50 ` Oleksandr
0 siblings, 0 replies; 93+ messages in thread
From: Oleksandr @ 2021-01-28 17:50 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, Julien Grall, Stefano Stabellini, Volodymyr Babchuk,
Andrew Cooper, George Dunlap, Ian Jackson, Jan Beulich, Wei Liu,
Oleksandr Tyshchenko
On 28.01.21 17:08, Julien Grall wrote:
Hi Julien
>
>
> On 28/01/2021 14:52, Oleksandr wrote:
>>
>> On 28.01.21 16:41, Julien Grall wrote:
>>> On 28/01/2021 14:29, Oleksandr wrote:
>>>> On 28.01.21 15:39, Oleksandr wrote:
>>>>>
>>>>> On 28.01.21 13:33, Oleksandr wrote:
>>>>>
>>>>> Hi Julien
>>>>>
>>>>>>
>>>>>> On 28.01.21 11:40, Julien Grall wrote:
>>>>>>
>>>>>> Hi Julien
>>>>>>
>>>>>>> Hi Oleksandr,
>>>>>>>
>>>>>>> On 27/01/2021 19:20, Oleksandr wrote:
>>>>>>> > >>> So I think we may be able to drop the include from
>>>>>>> asm/hvm/domain.h
>>>>>>>> (this would avoid to include it everywhere...).
>>>>>>>>
>>>>>>>> I have tried that, but other CUs use definitions from
>>>>>>>> public/hvm/dm_op.h, for example:
>>>>>>>>
>>>>>>>> p2m-pt.c: In function 'p2m_type_to_flags':
>>>>>>>> p2m-pt.c:87:33: error: 'XEN_DMOP_IOREQ_MEM_ACCESS_WRITE'
>>>>>>>> undeclared (first use in this function)
>>>>>>>> if ( p2m->ioreq.flags &
>>>>>>>> XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
>>>>>>>> ^
>>>>>>>> So, I would prefer to leave it as is, please let me know if you
>>>>>>>> think otherwise.
>>>>>>>
>>>>>>> AFAICT, there is only 2 places (p2m-pt.c and p2m-ept.c) that
>>>>>>> requires <public/hvm/dm_op.h> but doesn't directly include it.
>>>>>>> Folding the diff below in patch #4 should do the job:
>>>>>>
>>>>>> ok, will do
>>>>>
>>>>>
>>>>> Just to clarify, you mentioned about patch #4, but shouldn't we
>>>>> make these changes in patch #9 which actually tries to sort dm
>>>>> related stuff?
>>>>
>>>> or a least in patch #8 which moves the stuff from
>>>> asm-x86/hvm/domain.h to xen/ioreq.h (including the user of
>>>> XEN_DMOP_IO_RANGE_PCI),
>>>
>>> I looked at the header asm-x86/hvm/domain.h after applying patch #4,
>>> there is nothing requiring DMOP from there.
>>>
>>> I tried to build it with this series applied up to patch #4 + my
>>> diff. It does build without any issue.
>>
>> Hmm, interesting. I might miss something, but I got an build issue if
>> I split these changes with patch #4 and build the series up to this
>> patch:
>
> I think I looked and tried with the wrong commit. :( This was moved in
> patch #7 (xen/ioreq: Make x86's hvm_ioreq_(page/vcpu/server) structs
> common).
>
> Sorry for the confusion.
No problem.
So I will merge these changes with patch #7 (that already has all
required approvals) with the following justification:
Also there is no need to include public/hvm/dm_op.h by
asm-x86/hvm/domain.h anymore since #define NR_IO_RANGE_TYPES
(which uses XEN_DMOP_IO_RANGE_PCI) gets moved to another location.
Instead include it by 2 places (p2m-pt.c and p2m-ept.c) which
require that header, but don't directly include it so far.
If there are any objections please let me know.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
2021-01-28 15:14 ` Julien Grall
@ 2021-01-28 17:55 ` Oleksandr
0 siblings, 0 replies; 93+ messages in thread
From: Oleksandr @ 2021-01-28 17:55 UTC (permalink / raw)
To: Julien Grall
Cc: xen-devel, Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Andrew Cooper, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Tim Deegan,
Daniel De Graaf, Volodymyr Babchuk, Bertrand Marquis, Wei Chen,
Kaly Xin, Artem Mygaiev, Alex Bennée
On 28.01.21 17:14, Julien Grall wrote:
Hi Julien
> Hi,
>
> On 28/01/2021 14:37, Oleksandr wrote:
>> On 27.01.21 19:45, Oleksandr wrote:
>>>
>>> On 27.01.21 19:42, Julien Grall wrote:
>>>
>>> Hi
>>>
>>>>
>>>>
>>>> On 27/01/2021 17:37, Oleksandr wrote:
>>>>>
>>>>> On 27.01.21 19:33, Julien Grall wrote:
>>>>>
>>>>> Hi Julien
>>>>>
>>>>>>
>>>>>>
>>>>>> On 27/01/2021 16:50, Oleksandr wrote:
>>>>>>>
>>>>>>> On 27.01.21 18:43, Julien Grall wrote:
>>>>>>>> Hi Oleksandr,
>>>>>>>
>>>>>>> Hi Julien
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>>>>>>>>> ***
>>>>>>>>>
>>>>>>>>> Patch series [8] was rebased on recent "staging branch"
>>>>>>>>> (5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is
>>>>>>>>> unmapped) and tested on
>>>>>>>>> Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with
>>>>>>>>> virtio-mmio disk backend [9]
>>>>>>>>> running in driver domain and unmodified Linux Guest running on
>>>>>>>>> existing
>>>>>>>>> virtio-blk driver (frontend). No issues were observed. Guest
>>>>>>>>> domain 'reboot/destroy'
>>>>>>>>> use-cases work properly. Patch series was only build-tested on
>>>>>>>>> x86.
>>>>>>>>>
>>>>>>>>> Please note, build-test passed for the following modes:
>>>>>>>>> 1. x86: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y (default)
>>>>>>>>> 2. x86: #CONFIG_HVM is not set / #CONFIG_IOREQ_SERVER is not set
>>>>>>>>> 3. Arm64: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
>>>>>>>>> 4. Arm64: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set
>>>>>>>>> (default)
>>>>>>>>> 5. Arm32: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
>>>>>>>>> 6. Arm32: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set
>>>>>>>>> (default)
>>>>>>>>
>>>>>>>> I thought I woudl give a try to test the code, but I can't find
>>>>>>>> a way to enable CONFIG_IOREQ_SERVER from the UI.
>>>>>>>>
>>>>>>>> Looking at the Kconfig, it looks like CONFIG_IOREQ_SERVER
>>>>>>>> doesn't have a prompt and is not selected by Arm.
>>>>>>>>
>>>>>>>> Can you provide details how this can be built on Arm?
>>>>>>>
>>>>>>> Please apply the attached patch to select IOREQ on Arm.
>>>>>>
>>>>>> This is roughly what I wrote. I think a user should be able to
>>>>>> select IOREQ via the menuconfig without any additional patch on
>>>>>> top of your series.
>>>>>>
>>>>>> Can you include a patch that would enable that?
>>>>>
>>>>> Yes, do you prefer a separate patch or required changes could be
>>>>> folded in patch #14?
>>>>
>>>> I would do a separate patch as IOREQ only really work after the
>>>> full series applies.
>>>
>>>
>>> Makes sense, I will do it for V6
>>
>>
>> Could we please negotiate *the last posting time* for me to able to
>> prepare and push V6 not later than it?
>
> The feature freeze is going to be happen tomorrow evening. I can't
> give you as specific time, but it is probably best if you respin v6 by
> tomorrow morning. This should give us some slack for any last minutes
> issues (if any).
I got it, will try
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
2021-01-28 17:44 ` Julien Grall
@ 2021-01-28 18:10 ` Andrew Cooper
2021-01-28 18:16 ` Julien Grall
2021-01-28 19:44 ` Oleksandr
0 siblings, 2 replies; 93+ messages in thread
From: Andrew Cooper @ 2021-01-28 18:10 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: Oleksandr Tyshchenko, xen-devel, Ian Jackson,
Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Jun Nakajima, Kevin Tian,
Tim Deegan, Daniel De Graaf, Volodymyr Babchuk, Bertrand Marquis,
Wei Chen, Kaly Xin, Artem Mygaiev, Alex Bennée
On 28/01/2021 17:44, Julien Grall wrote:
>
>
> On 28/01/2021 17:24, Stefano Stabellini wrote:
>> On Thu, 28 Jan 2021, Julien Grall wrote:
>>> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>>> > Patch series [8] was rebased on recent "staging branch"
>>>> (5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is
>>>> unmapped) and
>>>> tested on
>>>> Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio disk
>>>> backend [9]
>>>> running in driver domain and unmodified Linux Guest running on
>>>> existing
>>>> virtio-blk driver (frontend). No issues were observed. Guest domain
>>>> 'reboot/destroy'
>>>> use-cases work properly. Patch series was only build-tested on x86.
>>>
>>> I have done basic testing with a Linux guest on x86 and I didn't
>>> spot any
>>> regression.
>>>
>>> Unfortunately, I don't have a Windows VM in hand, so I can't confirm
>>> if there
>>> is no regression there. Can anyone give a try?
>>>
>>> On a separate topic, Andrew expressed on IRC some concern to expose the
>>> bufioreq interface to other arch than x86. I will let him explain
>>> his view
>>> here.
>>>
>>> Given that IOREQ will be a tech preview on Arm (this implies the
>>> interface is
>>> not stable) on Arm, I think we can defer the discussion past the
>>> freeze.
>>
>> Yes, I agree that we can defer the discussion.
>>
>>
>>> For now, I would suggest to check if a Device Model is trying to
>>> create an
>>> IOREQ server with bufioreq is enabled (see ioreq_server_create()).
>>> This would
>>> not alleviate Andrew's concern, however it would at allow us to
>>> judge whether
>>> the buffering concept is going to be used on Arm (I can see some use
>>> for the
>>> Virtio doorbell).
>>>
>>> What do others think?
>>
>> Difficult to say. We don't have any uses today but who knows in the
>> future.
>
> I have some ideas, but Andrew so far objects on using the existing
> bufioreq interface for good reasons. It is using guest_cmpxchg() which
> is really expensive.
Worse. Patch 5 needs to switch cmpxchg() to guest_cmpxchg() to avoid
reintroducing XSA-295, but doesn't.
>
>>
>> Maybe for now the important thing is to clarify that the bufioreq
>> interface won't be maintain for backward compatibility on ARM, and could
>> be removed without warnings, at least as long as the whole IOREQ feature
>> is a tech preview. >
>> In other words, it is not like we are forced to keep bufioreq around
>> forever on ARM.
>
> That's correct, we are not forced to use it. But if you only document
> it, then there is a fair chance this will split past the "Tech Preview".
>
> Today, there is no caller which requires to send buffered I/O in the
> Xen Arm hypervisor. So a Device Model should not need to say "I want
> to allocate a page and event channel for the buffered I/O".
>
> The check I suggested serves two purposes:
> 1) Catch any future upstream user of bufioreq
> 2) Avoid an interface to be marked supported by mistake
"use bufioreq" is an input to create_ioreq_server. The easiest fix in
the short term is "if ( !IS_ENABLED(CONFIG_X86) && bufioreq ) return
-EINVAL;"
The key problem with the bufioreq interface is that it is a ring with a
non-power-of-2 number of entries. See c/s b7007bc6f9a45 if the
implications of a non-power-of-2 number of entries aren't immediately clear.
~Andrew
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
2021-01-28 18:10 ` Andrew Cooper
@ 2021-01-28 18:16 ` Julien Grall
2021-01-28 18:21 ` Julien Grall
2021-01-28 20:10 ` Andrew Cooper
2021-01-28 19:44 ` Oleksandr
1 sibling, 2 replies; 93+ messages in thread
From: Julien Grall @ 2021-01-28 18:16 UTC (permalink / raw)
To: Andrew Cooper, Stefano Stabellini
Cc: Oleksandr Tyshchenko, xen-devel, Ian Jackson,
Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Jun Nakajima, Kevin Tian,
Tim Deegan, Daniel De Graaf, Volodymyr Babchuk, Bertrand Marquis,
Wei Chen, Kaly Xin, Artem Mygaiev, Alex Bennée
Hi Andrew,
On 28/01/2021 18:10, Andrew Cooper wrote:
> On 28/01/2021 17:44, Julien Grall wrote:
>>
>>
>> On 28/01/2021 17:24, Stefano Stabellini wrote:
>>> On Thu, 28 Jan 2021, Julien Grall wrote:
>>>> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>>>> > Patch series [8] was rebased on recent "staging branch"
>>>>> (5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is
>>>>> unmapped) and
>>>>> tested on
>>>>> Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio disk
>>>>> backend [9]
>>>>> running in driver domain and unmodified Linux Guest running on
>>>>> existing
>>>>> virtio-blk driver (frontend). No issues were observed. Guest domain
>>>>> 'reboot/destroy'
>>>>> use-cases work properly. Patch series was only build-tested on x86.
>>>>
>>>> I have done basic testing with a Linux guest on x86 and I didn't
>>>> spot any
>>>> regression.
>>>>
>>>> Unfortunately, I don't have a Windows VM in hand, so I can't confirm
>>>> if there
>>>> is no regression there. Can anyone give a try?
>>>>
>>>> On a separate topic, Andrew expressed on IRC some concern to expose the
>>>> bufioreq interface to other arch than x86. I will let him explain
>>>> his view
>>>> here.
>>>>
>>>> Given that IOREQ will be a tech preview on Arm (this implies the
>>>> interface is
>>>> not stable) on Arm, I think we can defer the discussion past the
>>>> freeze.
>>>
>>> Yes, I agree that we can defer the discussion.
>>>
>>>
>>>> For now, I would suggest to check if a Device Model is trying to
>>>> create an
>>>> IOREQ server with bufioreq is enabled (see ioreq_server_create()).
>>>> This would
>>>> not alleviate Andrew's concern, however it would at allow us to
>>>> judge whether
>>>> the buffering concept is going to be used on Arm (I can see some use
>>>> for the
>>>> Virtio doorbell).
>>>>
>>>> What do others think?
>>>
>>> Difficult to say. We don't have any uses today but who knows in the
>>> future.
>>
>> I have some ideas, but Andrew so far objects on using the existing
>> bufioreq interface for good reasons. It is using guest_cmpxchg() which
>> is really expensive.
>
> Worse. Patch 5 needs to switch cmpxchg() to guest_cmpxchg() to avoid
> reintroducing XSA-295, but doesn't.
The memory is only shared with the emulator (we don't allow the legacy
interface on Arm). So why do you think it is re-introducing XSA-295?
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
2021-01-28 18:16 ` Julien Grall
@ 2021-01-28 18:21 ` Julien Grall
2021-01-28 20:10 ` Andrew Cooper
1 sibling, 0 replies; 93+ messages in thread
From: Julien Grall @ 2021-01-28 18:21 UTC (permalink / raw)
To: Andrew Cooper, Stefano Stabellini
Cc: Oleksandr Tyshchenko, xen-devel, Ian Jackson,
Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Jun Nakajima, Kevin Tian,
Tim Deegan, Daniel De Graaf, Volodymyr Babchuk, Bertrand Marquis,
Wei Chen, Kaly Xin, Artem Mygaiev, Alex Bennée
On 28/01/2021 18:16, Julien Grall wrote:
> Hi Andrew,
>
> On 28/01/2021 18:10, Andrew Cooper wrote:
>> On 28/01/2021 17:44, Julien Grall wrote:
>>>
>>>
>>> On 28/01/2021 17:24, Stefano Stabellini wrote:
>>>> On Thu, 28 Jan 2021, Julien Grall wrote:
>>>>> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>>>>> > Patch series [8] was rebased on recent "staging branch"
>>>>>> (5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is
>>>>>> unmapped) and
>>>>>> tested on
>>>>>> Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio disk
>>>>>> backend [9]
>>>>>> running in driver domain and unmodified Linux Guest running on
>>>>>> existing
>>>>>> virtio-blk driver (frontend). No issues were observed. Guest domain
>>>>>> 'reboot/destroy'
>>>>>> use-cases work properly. Patch series was only build-tested on x86.
>>>>>
>>>>> I have done basic testing with a Linux guest on x86 and I didn't
>>>>> spot any
>>>>> regression.
>>>>>
>>>>> Unfortunately, I don't have a Windows VM in hand, so I can't confirm
>>>>> if there
>>>>> is no regression there. Can anyone give a try?
>>>>>
>>>>> On a separate topic, Andrew expressed on IRC some concern to expose
>>>>> the
>>>>> bufioreq interface to other arch than x86. I will let him explain
>>>>> his view
>>>>> here.
>>>>>
>>>>> Given that IOREQ will be a tech preview on Arm (this implies the
>>>>> interface is
>>>>> not stable) on Arm, I think we can defer the discussion past the
>>>>> freeze.
>>>>
>>>> Yes, I agree that we can defer the discussion.
>>>>
>>>>
>>>>> For now, I would suggest to check if a Device Model is trying to
>>>>> create an
>>>>> IOREQ server with bufioreq is enabled (see ioreq_server_create()).
>>>>> This would
>>>>> not alleviate Andrew's concern, however it would at allow us to
>>>>> judge whether
>>>>> the buffering concept is going to be used on Arm (I can see some use
>>>>> for the
>>>>> Virtio doorbell).
>>>>>
>>>>> What do others think?
>>>>
>>>> Difficult to say. We don't have any uses today but who knows in the
>>>> future.
>>>
>>> I have some ideas, but Andrew so far objects on using the existing
>>> bufioreq interface for good reasons. It is using guest_cmpxchg() which
>>> is really expensive.
>>
>> Worse. Patch 5 needs to switch cmpxchg() to guest_cmpxchg() to avoid
>> reintroducing XSA-295, but doesn't.
>
> The memory is only shared with the emulator (we don't allow the legacy
> interface on Arm). So why do you think it is re-introducing XSA-295?
Just for clarification, the swithc to guest_cmpxchg() is done as part of
patch #13.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
2021-01-28 18:10 ` Andrew Cooper
2021-01-28 18:16 ` Julien Grall
@ 2021-01-28 19:44 ` Oleksandr
2021-01-29 1:15 ` Oleksandr
1 sibling, 1 reply; 93+ messages in thread
From: Oleksandr @ 2021-01-28 19:44 UTC (permalink / raw)
To: Andrew Cooper
Cc: Julien Grall, Stefano Stabellini, xen-devel, Ian Jackson,
Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Jun Nakajima, Kevin Tian,
Tim Deegan, Daniel De Graaf, Volodymyr Babchuk, Bertrand Marquis,
Wei Chen, Kaly Xin, Artem Mygaiev, Alex Bennée
On 28.01.21 20:10, Andrew Cooper wrote:
Hi Andrew
> On 28/01/2021 17:44, Julien Grall wrote:
>>
>> On 28/01/2021 17:24, Stefano Stabellini wrote:
>>> On Thu, 28 Jan 2021, Julien Grall wrote:
>>>> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>>>> > Patch series [8] was rebased on recent "staging branch"
>>>>> (5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is
>>>>> unmapped) and
>>>>> tested on
>>>>> Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio disk
>>>>> backend [9]
>>>>> running in driver domain and unmodified Linux Guest running on
>>>>> existing
>>>>> virtio-blk driver (frontend). No issues were observed. Guest domain
>>>>> 'reboot/destroy'
>>>>> use-cases work properly. Patch series was only build-tested on x86.
>>>> I have done basic testing with a Linux guest on x86 and I didn't
>>>> spot any
>>>> regression.
>>>>
>>>> Unfortunately, I don't have a Windows VM in hand, so I can't confirm
>>>> if there
>>>> is no regression there. Can anyone give a try?
>>>>
>>>> On a separate topic, Andrew expressed on IRC some concern to expose the
>>>> bufioreq interface to other arch than x86. I will let him explain
>>>> his view
>>>> here.
>>>>
>>>> Given that IOREQ will be a tech preview on Arm (this implies the
>>>> interface is
>>>> not stable) on Arm, I think we can defer the discussion past the
>>>> freeze.
>>> Yes, I agree that we can defer the discussion.
>>>
>>>
>>>> For now, I would suggest to check if a Device Model is trying to
>>>> create an
>>>> IOREQ server with bufioreq is enabled (see ioreq_server_create()).
>>>> This would
>>>> not alleviate Andrew's concern, however it would at allow us to
>>>> judge whether
>>>> the buffering concept is going to be used on Arm (I can see some use
>>>> for the
>>>> Virtio doorbell).
>>>>
>>>> What do others think?
>>> Difficult to say. We don't have any uses today but who knows in the
>>> future.
>> I have some ideas, but Andrew so far objects on using the existing
>> bufioreq interface for good reasons. It is using guest_cmpxchg() which
>> is really expensive.
> Worse. Patch 5 needs to switch cmpxchg() to guest_cmpxchg() to avoid
> reintroducing XSA-295, but doesn't.
>
>>> Maybe for now the important thing is to clarify that the bufioreq
>>> interface won't be maintain for backward compatibility on ARM, and could
>>> be removed without warnings, at least as long as the whole IOREQ feature
>>> is a tech preview. >
>>> In other words, it is not like we are forced to keep bufioreq around
>>> forever on ARM.
>> That's correct, we are not forced to use it. But if you only document
>> it, then there is a fair chance this will split past the "Tech Preview".
>>
>> Today, there is no caller which requires to send buffered I/O in the
>> Xen Arm hypervisor. So a Device Model should not need to say "I want
>> to allocate a page and event channel for the buffered I/O".
>>
>> The check I suggested serves two purposes:
>> 1) Catch any future upstream user of bufioreq
>> 2) Avoid an interface to be marked supported by mistake
> "use bufioreq" is an input to create_ioreq_server. The easiest fix in
> the short term is "if ( !IS_ENABLED(CONFIG_X86) && bufioreq ) return
> -EINVAL;"
OK, will take into the account for V6 as a separate patch
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 09/22] xen/ioreq: Make x86's IOREQ related dm-op handling common
2021-01-28 13:18 ` Jan Beulich
@ 2021-01-28 20:01 ` Oleksandr
0 siblings, 0 replies; 93+ messages in thread
From: Oleksandr @ 2021-01-28 20:01 UTC (permalink / raw)
To: Jan Beulich
Cc: Julien Grall, Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Paul Durrant, Daniel De Graaf,
Oleksandr Tyshchenko, xen-devel
On 28.01.21 15:18, Jan Beulich wrote:
Hi Jan
> On 28.01.2021 13:06, Oleksandr wrote:
>> On 28.01.21 12:52, Jan Beulich wrote:
>>
>> Hi Jan
>>
>>> On 25.01.2021 20:08, Oleksandr Tyshchenko wrote:
>>>> --- /dev/null
>>>> +++ b/xen/include/xen/dm.h
>>>> @@ -0,0 +1,41 @@
>>>> +/*
>>>> + * Copyright (c) 2016 Citrix Systems Inc.
>>>> + *
>>>> + * This program is free software; you can redistribute it and/or modify it
>>>> + * under the terms and conditions of the GNU General Public License,
>>>> + * version 2, as published by the Free Software Foundation.
>>>> + *
>>>> + * This program is distributed in the hope it will be useful, but WITHOUT
>>>> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
>>>> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
>>>> + * more details.
>>>> + *
>>>> + * You should have received a copy of the GNU General Public License along with
>>>> + * this program; If not, see <http://www.gnu.org/licenses/>.
>>>> + */
>>>> +
>>>> +#ifndef __XEN_DM_H__
>>>> +#define __XEN_DM_H__
>>>> +
>>>> +#include <xen/sched.h>
>>>> +
>>>> +struct dmop_args {
>>>> + domid_t domid;
>>>> + unsigned int nr_bufs;
>>>> + /* Reserve enough buf elements for all current hypercalls. */
>>>> + struct xen_dm_op_buf buf[2];
>>> So this is then the patch where the public header needs including,
>>> to satisfy this use of a struct declared there independent of what
>>> xen/sched.h includes. In fact public/xen.h needs including here
>>> as well, I think, for domid_t. Otoh I can't see why you include
>>> xen/sched.h.
>> Yes, xen/sched.h indeed doesn't need to be included here, I mentioned
>> that we could just replace
>>
>> it by xen/types.h (this is the minimum what we need here I think). As I
>> understand public/xen.h is already included by public/hvm/dm_op.h...
> But that's something you better wouldn't depend on anywhere.
> What one public header does or does not include may change over
> time.
ok, will include public/xen.h as well.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
2021-01-28 18:16 ` Julien Grall
2021-01-28 18:21 ` Julien Grall
@ 2021-01-28 20:10 ` Andrew Cooper
2021-01-28 21:19 ` Julien Grall
1 sibling, 1 reply; 93+ messages in thread
From: Andrew Cooper @ 2021-01-28 20:10 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini
Cc: Oleksandr Tyshchenko, xen-devel, Ian Jackson,
Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Jun Nakajima, Kevin Tian,
Tim Deegan, Daniel De Graaf, Volodymyr Babchuk, Bertrand Marquis,
Wei Chen, Kaly Xin, Artem Mygaiev, Alex Bennée
On 28/01/2021 18:16, Julien Grall wrote:
> Hi Andrew,
>
> On 28/01/2021 18:10, Andrew Cooper wrote:
>> On 28/01/2021 17:44, Julien Grall wrote:
>>>
>>>
>>> On 28/01/2021 17:24, Stefano Stabellini wrote:
>>>> On Thu, 28 Jan 2021, Julien Grall wrote:
>>>>> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>>>>> > Patch series [8] was rebased on recent "staging branch"
>>>>>> (5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is
>>>>>> unmapped) and
>>>>>> tested on
>>>>>> Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio
>>>>>> disk
>>>>>> backend [9]
>>>>>> running in driver domain and unmodified Linux Guest running on
>>>>>> existing
>>>>>> virtio-blk driver (frontend). No issues were observed. Guest domain
>>>>>> 'reboot/destroy'
>>>>>> use-cases work properly. Patch series was only build-tested on x86.
>>>>>
>>>>> I have done basic testing with a Linux guest on x86 and I didn't
>>>>> spot any
>>>>> regression.
>>>>>
>>>>> Unfortunately, I don't have a Windows VM in hand, so I can't confirm
>>>>> if there
>>>>> is no regression there. Can anyone give a try?
>>>>>
>>>>> On a separate topic, Andrew expressed on IRC some concern to
>>>>> expose the
>>>>> bufioreq interface to other arch than x86. I will let him explain
>>>>> his view
>>>>> here.
>>>>>
>>>>> Given that IOREQ will be a tech preview on Arm (this implies the
>>>>> interface is
>>>>> not stable) on Arm, I think we can defer the discussion past the
>>>>> freeze.
>>>>
>>>> Yes, I agree that we can defer the discussion.
>>>>
>>>>
>>>>> For now, I would suggest to check if a Device Model is trying to
>>>>> create an
>>>>> IOREQ server with bufioreq is enabled (see ioreq_server_create()).
>>>>> This would
>>>>> not alleviate Andrew's concern, however it would at allow us to
>>>>> judge whether
>>>>> the buffering concept is going to be used on Arm (I can see some use
>>>>> for the
>>>>> Virtio doorbell).
>>>>>
>>>>> What do others think?
>>>>
>>>> Difficult to say. We don't have any uses today but who knows in the
>>>> future.
>>>
>>> I have some ideas, but Andrew so far objects on using the existing
>>> bufioreq interface for good reasons. It is using guest_cmpxchg() which
>>> is really expensive.
>>
>> Worse. Patch 5 needs to switch cmpxchg() to guest_cmpxchg() to avoid
>> reintroducing XSA-295, but doesn't.
>
> The memory is only shared with the emulator (we don't allow the legacy
> interface on Arm).
Excellent.
> So why do you think it is re-introducing XSA-295?
Because the Xen security model considers "stubdomain can DoS Xen" a
security issue.
Yes - I know that right now, it will only be the hardware domain which
can use acquire_resource on ARM, but eventually we'll fix the
refcounting bug, and lifting the "translate && !hwdom" restriction would
be the thing that actually reintroduces XSA-295.
~Andrew
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
2021-01-28 20:10 ` Andrew Cooper
@ 2021-01-28 21:19 ` Julien Grall
0 siblings, 0 replies; 93+ messages in thread
From: Julien Grall @ 2021-01-28 21:19 UTC (permalink / raw)
To: Andrew Cooper
Cc: Stefano Stabellini, Oleksandr Tyshchenko, xen-devel, Ian Jackson,
Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Jun Nakajima, Kevin Tian,
Tim Deegan, Daniel De Graaf, Volodymyr Babchuk, Bertrand Marquis,
Wei Chen, Kaly Xin, Artem Mygaiev, Alex Bennée
On Thu, 28 Jan 2021 at 20:10, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>
> On 28/01/2021 18:16, Julien Grall wrote:
> > Hi Andrew,
> >
> > On 28/01/2021 18:10, Andrew Cooper wrote:
> >> On 28/01/2021 17:44, Julien Grall wrote:
> >>>
> >>>
> >>> On 28/01/2021 17:24, Stefano Stabellini wrote:
> >>>> On Thu, 28 Jan 2021, Julien Grall wrote:
> >>>>> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
> >>>>> > Patch series [8] was rebased on recent "staging branch"
> >>>>>> (5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is
> >>>>>> unmapped) and
> >>>>>> tested on
> >>>>>> Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio
> >>>>>> disk
> >>>>>> backend [9]
> >>>>>> running in driver domain and unmodified Linux Guest running on
> >>>>>> existing
> >>>>>> virtio-blk driver (frontend). No issues were observed. Guest domain
> >>>>>> 'reboot/destroy'
> >>>>>> use-cases work properly. Patch series was only build-tested on x86.
> >>>>>
> >>>>> I have done basic testing with a Linux guest on x86 and I didn't
> >>>>> spot any
> >>>>> regression.
> >>>>>
> >>>>> Unfortunately, I don't have a Windows VM in hand, so I can't confirm
> >>>>> if there
> >>>>> is no regression there. Can anyone give a try?
> >>>>>
> >>>>> On a separate topic, Andrew expressed on IRC some concern to
> >>>>> expose the
> >>>>> bufioreq interface to other arch than x86. I will let him explain
> >>>>> his view
> >>>>> here.
> >>>>>
> >>>>> Given that IOREQ will be a tech preview on Arm (this implies the
> >>>>> interface is
> >>>>> not stable) on Arm, I think we can defer the discussion past the
> >>>>> freeze.
> >>>>
> >>>> Yes, I agree that we can defer the discussion.
> >>>>
> >>>>
> >>>>> For now, I would suggest to check if a Device Model is trying to
> >>>>> create an
> >>>>> IOREQ server with bufioreq is enabled (see ioreq_server_create()).
> >>>>> This would
> >>>>> not alleviate Andrew's concern, however it would at allow us to
> >>>>> judge whether
> >>>>> the buffering concept is going to be used on Arm (I can see some use
> >>>>> for the
> >>>>> Virtio doorbell).
> >>>>>
> >>>>> What do others think?
> >>>>
> >>>> Difficult to say. We don't have any uses today but who knows in the
> >>>> future.
> >>>
> >>> I have some ideas, but Andrew so far objects on using the existing
> >>> bufioreq interface for good reasons. It is using guest_cmpxchg() which
> >>> is really expensive.
> >>
> >> Worse. Patch 5 needs to switch cmpxchg() to guest_cmpxchg() to avoid
> >> reintroducing XSA-295, but doesn't.
> >
> > The memory is only shared with the emulator (we don't allow the legacy
> > interface on Arm).
>
> Excellent.
>
> > So why do you think it is re-introducing XSA-295?
>
> Because the Xen security model considers "stubdomain can DoS Xen" a
> security issue.
>
> Yes - I know that right now, it will only be the hardware domain which
> can use acquire_resource on ARM, but eventually we'll fix the
> refcounting bug, and lifting the "translate && !hwdom" restriction would
> be the thing that actually reintroduces XSA-295.
By refcounting bugs, are you referring to mapping foreign pages in a
domain? If so, on Arm, we always increment the reference counter on
the foreign page during the map request. The reference will be
released when the emulator unmap it. Therefore, we don't need the
restriction "translate && !hwdom" (see patch #16 [1]).
In addition to that, patch #13 [2] is replacing the cmpxchg() with
guest_cmpxchg() to prevent XSA-295 from reappearing.
So unless I am missing something, the two security issues you
mentioned are already covered by this series.
Cheers,
[1] https://lore.kernel.org/xen-devel/1611601709-28361-17-git-send-email-olekstysh@gmail.com/
[2] https://lore.kernel.org/xen-devel/1611601709-28361-14-git-send-email-olekstysh@gmail.com/
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm
2021-01-28 19:44 ` Oleksandr
@ 2021-01-29 1:15 ` Oleksandr
0 siblings, 0 replies; 93+ messages in thread
From: Oleksandr @ 2021-01-29 1:15 UTC (permalink / raw)
To: Andrew Cooper, Julien Grall
Cc: Stefano Stabellini, xen-devel, Ian Jackson, Oleksandr Tyshchenko,
Paul Durrant, Jan Beulich, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Jun Nakajima, Kevin Tian,
Tim Deegan, Daniel De Graaf, Volodymyr Babchuk, Bertrand Marquis,
Wei Chen, Kaly Xin, Artem Mygaiev, Alex Bennée
On 28.01.21 21:44, Oleksandr wrote:
Hi Andrew, Julien
>
> On 28.01.21 20:10, Andrew Cooper wrote:
>
> Hi Andrew
>
>> On 28/01/2021 17:44, Julien Grall wrote:
>>>
>>> On 28/01/2021 17:24, Stefano Stabellini wrote:
>>>> On Thu, 28 Jan 2021, Julien Grall wrote:
>>>>> On 25/01/2021 19:08, Oleksandr Tyshchenko wrote:
>>>>> > Patch series [8] was rebased on recent "staging branch"
>>>>>> (5e31789 tools/ocaml/libs/xb: Do not crash after xenbus is
>>>>>> unmapped) and
>>>>>> tested on
>>>>>> Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio
>>>>>> disk
>>>>>> backend [9]
>>>>>> running in driver domain and unmodified Linux Guest running on
>>>>>> existing
>>>>>> virtio-blk driver (frontend). No issues were observed. Guest domain
>>>>>> 'reboot/destroy'
>>>>>> use-cases work properly. Patch series was only build-tested on x86.
>>>>> I have done basic testing with a Linux guest on x86 and I didn't
>>>>> spot any
>>>>> regression.
>>>>>
>>>>> Unfortunately, I don't have a Windows VM in hand, so I can't confirm
>>>>> if there
>>>>> is no regression there. Can anyone give a try?
>>>>>
>>>>> On a separate topic, Andrew expressed on IRC some concern to
>>>>> expose the
>>>>> bufioreq interface to other arch than x86. I will let him explain
>>>>> his view
>>>>> here.
>>>>>
>>>>> Given that IOREQ will be a tech preview on Arm (this implies the
>>>>> interface is
>>>>> not stable) on Arm, I think we can defer the discussion past the
>>>>> freeze.
>>>> Yes, I agree that we can defer the discussion.
>>>>
>>>>
>>>>> For now, I would suggest to check if a Device Model is trying to
>>>>> create an
>>>>> IOREQ server with bufioreq is enabled (see ioreq_server_create()).
>>>>> This would
>>>>> not alleviate Andrew's concern, however it would at allow us to
>>>>> judge whether
>>>>> the buffering concept is going to be used on Arm (I can see some use
>>>>> for the
>>>>> Virtio doorbell).
>>>>>
>>>>> What do others think?
>>>> Difficult to say. We don't have any uses today but who knows in the
>>>> future.
>>> I have some ideas, but Andrew so far objects on using the existing
>>> bufioreq interface for good reasons. It is using guest_cmpxchg() which
>>> is really expensive.
>> Worse. Patch 5 needs to switch cmpxchg() to guest_cmpxchg() to avoid
>> reintroducing XSA-295, but doesn't.
>>
>>>> Maybe for now the important thing is to clarify that the bufioreq
>>>> interface won't be maintain for backward compatibility on ARM, and
>>>> could
>>>> be removed without warnings, at least as long as the whole IOREQ
>>>> feature
>>>> is a tech preview. >
>>>> In other words, it is not like we are forced to keep bufioreq around
>>>> forever on ARM.
>>> That's correct, we are not forced to use it. But if you only document
>>> it, then there is a fair chance this will split past the "Tech
>>> Preview".
>>>
>>> Today, there is no caller which requires to send buffered I/O in the
>>> Xen Arm hypervisor. So a Device Model should not need to say "I want
>>> to allocate a page and event channel for the buffered I/O".
>>>
>>> The check I suggested serves two purposes:
>>> 1) Catch any future upstream user of bufioreq
>>> 2) Avoid an interface to be marked supported by mistake
>> "use bufioreq" is an input to create_ioreq_server. The easiest fix in
>> the short term is "if ( !IS_ENABLED(CONFIG_X86) && bufioreq ) return
>> -EINVAL;"
>
> OK, will take into the account for V6 as a separate patch
What I would like to say is that the easiest fix works fine, it breaks
virtio backend PoC by rejecting xendevicemodel_create_ioreq_server() call)))
No, buffered I/O is *not* used for the communication *at run-time*, a
device model just requests bufioreq support, gets bufioreq page and
event channel, and that's all without future use of them.
So I have just removed that unneeded at the moment initialization, what
it more that it doesn't match with that check we would like to get in.
>
>
>
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 93+ messages in thread
end of thread, other threads:[~2021-01-29 1:16 UTC | newest]
Thread overview: 93+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-25 19:08 [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 01/22] x86/ioreq: Prepare IOREQ feature for making it common Oleksandr Tyshchenko
2021-01-27 16:48 ` Jan Beulich
2021-01-25 19:08 ` [PATCH V5 02/22] x86/ioreq: Add IOREQ_STATUS_* #define-s and update code for moving Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 03/22] x86/ioreq: Provide out-of-line wrapper for the handle_mmio() Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 04/22] xen/ioreq: Make x86's IOREQ feature common Oleksandr Tyshchenko
2021-01-25 23:13 ` Julien Grall
2021-01-25 23:40 ` Oleksandr
2021-01-27 18:43 ` Julien Grall
2021-01-27 20:22 ` Oleksandr
2021-01-27 20:46 ` Stefano Stabellini
2021-01-28 11:01 ` Oleksandr
2021-01-28 11:21 ` Jan Beulich
2021-01-28 12:49 ` Oleksandr
2021-01-28 17:13 ` Stefano Stabellini
2021-01-27 16:58 ` Jan Beulich
2021-01-27 20:14 ` Oleksandr
2021-01-28 8:06 ` Jan Beulich
2021-01-28 11:16 ` Oleksandr
2021-01-28 11:24 ` Jan Beulich
2021-01-25 19:08 ` [PATCH V5 05/22] xen/ioreq: Make x86's hvm_ioreq_needs_completion() common Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 06/22] xen/ioreq: Make x86's hvm_mmio_first(last)_byte() common Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 07/22] xen/ioreq: Make x86's hvm_ioreq_(page/vcpu/server) structs common Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 08/22] xen/ioreq: Move x86's ioreq_server to struct domain Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 09/22] xen/ioreq: Make x86's IOREQ related dm-op handling common Oleksandr Tyshchenko
2021-01-26 13:31 ` Paul Durrant
2021-01-28 10:52 ` Jan Beulich
2021-01-28 12:06 ` Oleksandr
2021-01-28 13:18 ` Jan Beulich
2021-01-28 20:01 ` Oleksandr
2021-01-25 19:08 ` [PATCH V5 10/22] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu Oleksandr Tyshchenko
2021-01-28 13:41 ` Julien Grall
2021-01-28 13:53 ` Jan Beulich
2021-01-28 14:21 ` Julien Grall
2021-01-28 14:36 ` Jan Beulich
2021-01-28 14:49 ` Andrew Cooper
2021-01-28 14:51 ` Ian Jackson
2021-01-28 14:54 ` Jan Beulich
2021-01-28 16:15 ` Andrew Cooper
2021-01-28 15:01 ` Julien Grall
2021-01-28 14:05 ` Oleksandr
2021-01-25 19:08 ` [PATCH V5 11/22] xen/mm: Make x86's XENMEM_resource_ioreq_server handling common Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 12/22] xen/ioreq: Remove "hvm" prefixes from involved function names Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 13/22] xen/ioreq: Use guest_cmpxchg64() instead of cmpxchg() Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 14/22] arm/ioreq: Introduce arch specific bits for IOREQ/DM features Oleksandr Tyshchenko
2021-01-25 20:19 ` Stefano Stabellini
2021-01-26 9:15 ` Jan Beulich
2021-01-27 9:54 ` Oleksandr
2021-01-27 10:15 ` Jan Beulich
2021-01-27 17:01 ` Julien Grall
2021-01-27 17:04 ` Jan Beulich
2021-01-27 18:33 ` Julien Grall
2021-01-27 19:20 ` Oleksandr
2021-01-28 9:40 ` Julien Grall
2021-01-28 11:33 ` Oleksandr
2021-01-28 13:39 ` Oleksandr
2021-01-28 14:29 ` Oleksandr
2021-01-28 14:41 ` Julien Grall
2021-01-28 14:52 ` Oleksandr
2021-01-28 15:08 ` Julien Grall
2021-01-28 17:50 ` Oleksandr
2021-01-25 19:08 ` [PATCH V5 15/22] xen/arm: Call vcpu_ioreq_handle_completion() in check_for_vcpu_work() Oleksandr Tyshchenko
2021-01-27 14:49 ` Julien Grall
2021-01-27 15:56 ` Oleksandr
2021-01-27 20:34 ` Stefano Stabellini
2021-01-25 19:08 ` [PATCH V5 16/22] xen/mm: Handle properly reference in set_foreign_p2m_entry() on Arm Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 17/22] xen/ioreq: Introduce domain_has_ioreq_server() Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 18/22] xen/dm: Introduce xendevicemodel_set_irq_level DM op Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 19/22] xen/arm: io: Abstract sign-extension Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 20/22] xen/arm: io: Harden sign extension check Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 21/22] xen/ioreq: Make x86's send_invalidate_req() common Oleksandr Tyshchenko
2021-01-25 19:08 ` [PATCH V5 22/22] xen/arm: Add mapcache invalidation handling Oleksandr Tyshchenko
2021-01-28 9:55 ` Julien Grall
2021-01-28 13:12 ` Oleksandr
2021-01-27 16:43 ` [PATCH V5 00/22] IOREQ feature (+ virtio-mmio) on Arm Julien Grall
2021-01-27 16:50 ` Oleksandr
2021-01-27 17:33 ` Julien Grall
2021-01-27 17:37 ` Oleksandr
2021-01-27 17:42 ` Julien Grall
2021-01-27 17:45 ` Oleksandr
2021-01-28 14:37 ` Oleksandr
2021-01-28 15:14 ` Julien Grall
2021-01-28 17:55 ` Oleksandr
2021-01-28 16:11 ` Julien Grall
2021-01-28 17:24 ` Stefano Stabellini
2021-01-28 17:44 ` Julien Grall
2021-01-28 18:10 ` Andrew Cooper
2021-01-28 18:16 ` Julien Grall
2021-01-28 18:21 ` Julien Grall
2021-01-28 20:10 ` Andrew Cooper
2021-01-28 21:19 ` Julien Grall
2021-01-28 19:44 ` Oleksandr
2021-01-29 1:15 ` Oleksandr
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.