* [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm
@ 2020-10-15 16:44 Oleksandr Tyshchenko
2020-10-15 16:44 ` [PATCH V2 01/23] x86/ioreq: Prepare IOREQ feature for making it common Oleksandr Tyshchenko
` (23 more replies)
0 siblings, 24 replies; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Tim Deegan, Daniel De Graaf,
Volodymyr Babchuk, Jun Nakajima, Kevin Tian, Anthony PERARD,
Bertrand Marquis
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Hello all.
The purpose of this patch series is to add IOREQ/DM support to Xen on Arm.
You can find an initial discussion at [1] and RFC/V1 series at [2]/[3].
Xen on Arm requires some implementation to forward guest MMIO access to a device
model in order to implement virtio-mmio backend or even mediator outside of hypervisor.
As Xen on x86 already contains required support this series tries to make it common
and introduce Arm specific bits plus some new functionality. Patch series is based on
Julien's PoC "xen/arm: Add support for Guest IO forwarding to a device emulator".
Besides splitting existing IOREQ/DM support and introducing Arm side, the series
also includes virtio-mmio related changes (last 2 patches for toolstack)
for the reviewers to be able to see how the whole picture could look like.
According to the initial discussion there are a few open questions/concerns
regarding security, performance in VirtIO solution:
1. virtio-mmio vs virtio-pci, SPI vs MSI, different use-cases require different
transport...
2. virtio backend is able to access all guest memory, some kind of protection
is needed: 'virtio-iommu in Xen' vs 'pre-shared-memory & memcpys in guest'
3. interface between toolstack and 'out-of-qemu' virtio backend, avoid using
Xenstore in virtio backend if possible.
4. a lot of 'foreing mapping' could lead to the memory exhaustion, Julien
has some idea regarding that.
Looks like all of them are valid and worth considering, but the first thing
which we need on Arm is a mechanism to forward guest IO to a device emulator,
so let's focus on it in the first place.
***
There are a lot of changes since RFC series, almost all TODOs were resolved on Arm,
Arm code was improved and hardened, common IOREQ/DM code became really arch-agnostic
(without HVM-ism), but one TODO still remains which is "PIO handling" on Arm.
The "PIO handling" TODO is expected to left unaddressed for the current series.
It is not an big issue for now while Xen doesn't have support for vPCI on Arm.
On Arm64 they are only used for PCI IO Bar and we would probably want to expose
them to emulator as PIO access to make a DM completely arch-agnostic. So "PIO handling"
should be implemented when we add support for vPCI.
I left interface untouched in the following patch
"xen/dm: Introduce xendevicemodel_set_irq_level DM op"
since there is still an open discussion what interface to use/what
information to pass to the hypervisor.
Also I decided to drop the following patch:
"[RFC PATCH V1 07/12] A collection of tweaks to be able to run emulator in driver domain"
as I got an advise to write our own policy using FLASK which would cover our use
case (with emulator in driver domain) rather than tweak Xen.
There are two patches on review this series depends on (each involved patch in this series
contains this note as well):
1. https://patchwork.kernel.org/patch/11816689
2. https://patchwork.kernel.org/patch/11803383
Please note, that IOREQ feature is disabled by default within this series.
***
Patch series [4] was rebased on recent "staging branch"
(8a62dee x86/vLAPIC: don't leak regs page from vlapic_init() upon error) and tested on
Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio disk backend (we will
share it later) running in driver domain and unmodified Linux Guest running on existing
virtio-blk driver (frontend). No issues were observed. Guest domain 'reboot/destroy'
use-cases work properly. Patch series was only build-tested on x86.
Please note, build-test passed for the following modes:
1. x86: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y (default)
2. x86: #CONFIG_HVM is not set / #CONFIG_IOREQ_SERVER is not set
3. Arm64: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
4. Arm64: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
5. Arm32: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
6. Arm32: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
***
Any feedback/help would be highly appreciated.
[1] https://lists.xenproject.org/archives/html/xen-devel/2020-07/msg00825.html
[2] https://lists.xenproject.org/archives/html/xen-devel/2020-08/msg00071.html
[3] https://lists.xenproject.org/archives/html/xen-devel/2020-09/msg00732.html
[4] https://github.com/otyshchenko1/xen/commits/ioreq_4.14_ml3
Julien Grall (5):
xen/dm: Make x86's DM feature common
xen/mm: Make x86's XENMEM_resource_ioreq_server handling common
arm/ioreq: Introduce arch specific bits for IOREQ/DM features
xen/dm: Introduce xendevicemodel_set_irq_level DM op
libxl: Introduce basic virtio-mmio support on Arm
Oleksandr Tyshchenko (18):
x86/ioreq: Prepare IOREQ feature for making it common
xen/ioreq: Make x86's IOREQ feature common
xen/ioreq: Make x86's hvm_ioreq_needs_completion() common
xen/ioreq: Provide alias for the handle_mmio()
xen/ioreq: Make x86's hvm_mmio_first(last)_byte() common
xen/ioreq: Make x86's hvm_ioreq_(page/vcpu/server) structs common
xen/ioreq: Move x86's ioreq_gfn(server) to struct domain
xen/ioreq: Introduce ioreq_params to abstract accesses to
arch.hvm.params
xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
xen/ioreq: Remove "hvm" prefixes from involved function names
xen/ioreq: Use guest_cmpxchg64() instead of cmpxchg()
xen/arm: Stick around in leave_hypervisor_to_guest until I/O has
completed
xen/mm: Handle properly reference in set_foreign_p2m_entry() on Arm
xen/ioreq: Introduce domain_has_ioreq_server()
xen/arm: io: Abstract sign-extension
xen/ioreq: Make x86's send_invalidate_req() common
xen/arm: Add mapcache invalidation handling
[RFC] libxl: Add support for virtio-disk configuration
MAINTAINERS | 8 +-
tools/libs/devicemodel/core.c | 18 +
tools/libs/devicemodel/include/xendevicemodel.h | 4 +
tools/libs/devicemodel/libxendevicemodel.map | 1 +
tools/libs/light/Makefile | 1 +
tools/libs/light/libxl_arm.c | 94 +-
tools/libs/light/libxl_create.c | 1 +
tools/libs/light/libxl_internal.h | 1 +
tools/libs/light/libxl_types.idl | 16 +
tools/libs/light/libxl_types_internal.idl | 1 +
tools/libs/light/libxl_virtio_disk.c | 109 ++
tools/xl/Makefile | 2 +-
tools/xl/xl.h | 3 +
tools/xl/xl_cmdtable.c | 15 +
tools/xl/xl_parse.c | 116 ++
tools/xl/xl_virtio_disk.c | 46 +
xen/arch/arm/Makefile | 2 +
xen/arch/arm/dm.c | 89 ++
xen/arch/arm/domain.c | 9 +
xen/arch/arm/hvm.c | 4 +
xen/arch/arm/io.c | 29 +-
xen/arch/arm/ioreq.c | 126 ++
xen/arch/arm/p2m.c | 29 +
xen/arch/arm/traps.c | 58 +-
xen/arch/x86/Kconfig | 1 +
xen/arch/x86/hvm/Makefile | 1 -
xen/arch/x86/hvm/dm.c | 291 +----
xen/arch/x86/hvm/emulate.c | 60 +-
xen/arch/x86/hvm/hvm.c | 24 +-
xen/arch/x86/hvm/hypercall.c | 9 +-
xen/arch/x86/hvm/intercept.c | 5 +-
xen/arch/x86/hvm/io.c | 26 +-
xen/arch/x86/hvm/ioreq.c | 1533 -----------------------
xen/arch/x86/hvm/stdvga.c | 10 +-
xen/arch/x86/hvm/svm/nestedsvm.c | 2 +-
xen/arch/x86/hvm/vmx/realmode.c | 6 +-
xen/arch/x86/hvm/vmx/vvmx.c | 2 +-
xen/arch/x86/mm.c | 46 +-
xen/arch/x86/mm/p2m.c | 13 +-
xen/arch/x86/mm/shadow/common.c | 2 +-
xen/common/Kconfig | 3 +
xen/common/Makefile | 2 +
xen/common/dm.c | 292 +++++
xen/common/ioreq.c | 1443 +++++++++++++++++++++
xen/common/memory.c | 50 +-
xen/include/asm-arm/domain.h | 5 +
xen/include/asm-arm/hvm/ioreq.h | 109 ++
xen/include/asm-arm/mm.h | 8 -
xen/include/asm-arm/mmio.h | 1 +
xen/include/asm-arm/p2m.h | 19 +-
xen/include/asm-arm/paging.h | 4 +
xen/include/asm-arm/traps.h | 24 +
xen/include/asm-x86/hvm/domain.h | 50 +-
xen/include/asm-x86/hvm/emulate.h | 2 +-
xen/include/asm-x86/hvm/io.h | 17 -
xen/include/asm-x86/hvm/ioreq.h | 198 ++-
xen/include/asm-x86/hvm/vcpu.h | 18 -
xen/include/asm-x86/mm.h | 4 -
xen/include/asm-x86/p2m.h | 20 +-
xen/include/public/arch-arm.h | 5 +
xen/include/public/hvm/dm_op.h | 16 +
xen/include/xen/dm.h | 44 +
xen/include/xen/ioreq.h | 143 +++
xen/include/xen/p2m-common.h | 4 +
xen/include/xen/sched.h | 37 +
xen/include/xsm/dummy.h | 4 +-
xen/include/xsm/xsm.h | 6 +-
xen/xsm/dummy.c | 2 +-
xen/xsm/flask/hooks.c | 5 +-
69 files changed, 3223 insertions(+), 2125 deletions(-)
create mode 100644 tools/libs/light/libxl_virtio_disk.c
create mode 100644 tools/xl/xl_virtio_disk.c
create mode 100644 xen/arch/arm/dm.c
create mode 100644 xen/arch/arm/ioreq.c
delete mode 100644 xen/arch/x86/hvm/ioreq.c
create mode 100644 xen/common/dm.c
create mode 100644 xen/common/ioreq.c
create mode 100644 xen/include/asm-arm/hvm/ioreq.h
create mode 100644 xen/include/xen/dm.h
create mode 100644 xen/include/xen/ioreq.h
--
2.7.4
^ permalink raw reply [flat|nested] 109+ messages in thread
* [PATCH V2 01/23] x86/ioreq: Prepare IOREQ feature for making it common
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-10-20 7:13 ` Paul Durrant
2020-11-12 10:58 ` Jan Beulich
2020-10-15 16:44 ` [PATCH V2 02/23] xen/ioreq: Make x86's IOREQ feature common Oleksandr Tyshchenko
` (22 subsequent siblings)
23 siblings, 2 replies; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Julien Grall, Stefano Stabellini, Wei Liu, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
As a lot of x86 code can be re-used on Arm later on, this
patch makes some preparation to x86/hvm/ioreq.c before moving
to the common code. This way we will get a verbatim copy for
a code movement in subsequent patch (arch/x86/hvm/ioreq.c
will be *just* renamed to common/ioreq).
This patch does the following:
1. Introduce *inline* arch_hvm_ioreq_init(), arch_hvm_ioreq_destroy(),
arch_hvm_io_completion(), arch_hvm_destroy_ioreq_server() and
hvm_ioreq_server_get_type_addr() to abstract arch specific materials.
2 Make hvm_map_mem_type_to_ioreq_server() *inline*. It is not going
to be called from the common code.
3. Make get_ioreq_server() global. It is going to be called from
a few places.
4. Add IOREQ_STATUS_* #define-s and update candidates for moving.
5. Re-order #include-s alphabetically.
This support is going to be used on Arm to be able run device
emulator outside of Xen hypervisor.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- new patch, was split from:
"[RFC PATCH V1 01/12] hvm/ioreq: Make x86's IOREQ feature common"
- fold the check of p->type into hvm_get_ioreq_server_range_type()
and make it return success/failure
- remove relocate_portio_handler() call from arch_hvm_ioreq_destroy()
in arch/x86/hvm/ioreq.c
- introduce arch_hvm_destroy_ioreq_server()/arch_handle_hvm_io_completion()
Changes V1 -> V2:
- update patch description
- make arch functions inline and put them into arch header
to achieve a truly rename by the subsequent patch
- return void in arch_hvm_destroy_ioreq_server()
- return bool in arch_hvm_ioreq_destroy()
- bring relocate_portio_handler() back to arch_hvm_ioreq_destroy()
- rename IOREQ_IO* to IOREQ_STATUS*
- remove *handle* from arch_handle_hvm_io_completion()
- re-order #include-s alphabetically
- rename hvm_get_ioreq_server_range_type() to hvm_ioreq_server_get_type_addr()
and add "const" to several arguments
---
xen/arch/x86/hvm/ioreq.c | 153 +++++--------------------------------
xen/include/asm-x86/hvm/ioreq.h | 165 +++++++++++++++++++++++++++++++++++++++-
2 files changed, 184 insertions(+), 134 deletions(-)
diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 1cc27df..d3433d7 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -1,5 +1,5 @@
/*
- * hvm/io.c: hardware virtual machine I/O emulation
+ * ioreq.c: hardware virtual machine I/O emulation
*
* Copyright (c) 2016 Citrix Systems Inc.
*
@@ -17,21 +17,18 @@
*/
#include <xen/ctype.h>
+#include <xen/domain.h>
+#include <xen/event.h>
#include <xen/init.h>
+#include <xen/irq.h>
#include <xen/lib.h>
-#include <xen/trace.h>
+#include <xen/paging.h>
#include <xen/sched.h>
-#include <xen/irq.h>
#include <xen/softirq.h>
-#include <xen/domain.h>
-#include <xen/event.h>
-#include <xen/paging.h>
+#include <xen/trace.h>
#include <xen/vpci.h>
-#include <asm/hvm/emulate.h>
-#include <asm/hvm/hvm.h>
#include <asm/hvm/ioreq.h>
-#include <asm/hvm/vmx/vmx.h>
#include <public/hvm/ioreq.h>
#include <public/hvm/params.h>
@@ -48,8 +45,8 @@ static void set_ioreq_server(struct domain *d, unsigned int id,
#define GET_IOREQ_SERVER(d, id) \
(d)->arch.hvm.ioreq_server.server[id]
-static struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
- unsigned int id)
+struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
+ unsigned int id)
{
if ( id >= MAX_NR_IOREQ_SERVERS )
return NULL;
@@ -209,19 +206,8 @@ bool handle_hvm_io_completion(struct vcpu *v)
return handle_pio(vio->io_req.addr, vio->io_req.size,
vio->io_req.dir);
- case HVMIO_realmode_completion:
- {
- struct hvm_emulate_ctxt ctxt;
-
- hvm_emulate_init_once(&ctxt, NULL, guest_cpu_user_regs());
- vmx_realmode_emulate_one(&ctxt);
- hvm_emulate_writeback(&ctxt);
-
- break;
- }
default:
- ASSERT_UNREACHABLE();
- break;
+ return arch_hvm_io_completion(io_completion);
}
return true;
@@ -855,7 +841,7 @@ int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
domain_pause(d);
- p2m_set_ioreq_server(d, 0, s);
+ arch_hvm_destroy_ioreq_server(s);
hvm_ioreq_server_disable(s);
@@ -1080,54 +1066,6 @@ int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
return rc;
}
-/*
- * Map or unmap an ioreq server to specific memory type. For now, only
- * HVMMEM_ioreq_server is supported, and in the future new types can be
- * introduced, e.g. HVMMEM_ioreq_serverX mapped to ioreq server X. And
- * currently, only write operations are to be forwarded to an ioreq server.
- * Support for the emulation of read operations can be added when an ioreq
- * server has such requirement in the future.
- */
-int hvm_map_mem_type_to_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint32_t flags)
-{
- struct hvm_ioreq_server *s;
- int rc;
-
- if ( type != HVMMEM_ioreq_server )
- return -EINVAL;
-
- if ( flags & ~XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
- return -EINVAL;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- s = get_ioreq_server(d, id);
-
- rc = -ENOENT;
- if ( !s )
- goto out;
-
- rc = -EPERM;
- if ( s->emulator != current->domain )
- goto out;
-
- rc = p2m_set_ioreq_server(d, flags, s);
-
- out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- if ( rc == 0 && flags == 0 )
- {
- struct p2m_domain *p2m = p2m_get_hostp2m(d);
-
- if ( read_atomic(&p2m->ioreq.entry_count) )
- p2m_change_entry_type_global(d, p2m_ioreq_server, p2m_ram_rw);
- }
-
- return rc;
-}
-
int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
bool enabled)
{
@@ -1215,7 +1153,7 @@ void hvm_destroy_all_ioreq_servers(struct domain *d)
struct hvm_ioreq_server *s;
unsigned int id;
- if ( !relocate_portio_handler(d, 0xcf8, 0xcf8, 4) )
+ if ( !arch_hvm_ioreq_destroy(d) )
return;
spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
@@ -1243,50 +1181,13 @@ struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
ioreq_t *p)
{
struct hvm_ioreq_server *s;
- uint32_t cf8;
uint8_t type;
uint64_t addr;
unsigned int id;
- if ( p->type != IOREQ_TYPE_COPY && p->type != IOREQ_TYPE_PIO )
+ if ( hvm_ioreq_server_get_type_addr(d, p, &type, &addr) )
return NULL;
- cf8 = d->arch.hvm.pci_cf8;
-
- if ( p->type == IOREQ_TYPE_PIO &&
- (p->addr & ~3) == 0xcfc &&
- CF8_ENABLED(cf8) )
- {
- uint32_t x86_fam;
- pci_sbdf_t sbdf;
- unsigned int reg;
-
- reg = hvm_pci_decode_addr(cf8, p->addr, &sbdf);
-
- /* PCI config data cycle */
- type = XEN_DMOP_IO_RANGE_PCI;
- addr = ((uint64_t)sbdf.sbdf << 32) | reg;
- /* AMD extended configuration space access? */
- if ( CF8_ADDR_HI(cf8) &&
- d->arch.cpuid->x86_vendor == X86_VENDOR_AMD &&
- (x86_fam = get_cpu_family(
- d->arch.cpuid->basic.raw_fms, NULL, NULL)) >= 0x10 &&
- x86_fam < 0x17 )
- {
- uint64_t msr_val;
-
- if ( !rdmsr_safe(MSR_AMD64_NB_CFG, msr_val) &&
- (msr_val & (1ULL << AMD64_NB_CFG_CF8_EXT_ENABLE_BIT)) )
- addr |= CF8_ADDR_HI(cf8);
- }
- }
- else
- {
- type = (p->type == IOREQ_TYPE_PIO) ?
- XEN_DMOP_IO_RANGE_PORT : XEN_DMOP_IO_RANGE_MEMORY;
- addr = p->addr;
- }
-
FOR_EACH_IOREQ_SERVER(d, id, s)
{
struct rangeset *r;
@@ -1351,7 +1252,7 @@ static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
pg = iorp->va;
if ( !pg )
- return X86EMUL_UNHANDLEABLE;
+ return IOREQ_STATUS_UNHANDLED;
/*
* Return 0 for the cases we can't deal with:
@@ -1381,7 +1282,7 @@ static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
break;
default:
gdprintk(XENLOG_WARNING, "unexpected ioreq size: %u\n", p->size);
- return X86EMUL_UNHANDLEABLE;
+ return IOREQ_STATUS_UNHANDLED;
}
spin_lock(&s->bufioreq_lock);
@@ -1391,7 +1292,7 @@ static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
{
/* The queue is full: send the iopacket through the normal path. */
spin_unlock(&s->bufioreq_lock);
- return X86EMUL_UNHANDLEABLE;
+ return IOREQ_STATUS_UNHANDLED;
}
pg->buf_ioreq[pg->ptrs.write_pointer % IOREQ_BUFFER_SLOT_NUM] = bp;
@@ -1422,7 +1323,7 @@ static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
notify_via_xen_event_channel(d, s->bufioreq_evtchn);
spin_unlock(&s->bufioreq_lock);
- return X86EMUL_OKAY;
+ return IOREQ_STATUS_HANDLED;
}
int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
@@ -1438,7 +1339,7 @@ int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
return hvm_send_buffered_ioreq(s, proto_p);
if ( unlikely(!vcpu_start_shutdown_deferral(curr)) )
- return X86EMUL_RETRY;
+ return IOREQ_STATUS_RETRY;
list_for_each_entry ( sv,
&s->ioreq_vcpu_list,
@@ -1478,11 +1379,11 @@ int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
notify_via_xen_event_channel(d, port);
sv->pending = true;
- return X86EMUL_RETRY;
+ return IOREQ_STATUS_RETRY;
}
}
- return X86EMUL_UNHANDLEABLE;
+ return IOREQ_STATUS_UNHANDLED;
}
unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered)
@@ -1496,30 +1397,18 @@ unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered)
if ( !s->enabled )
continue;
- if ( hvm_send_ioreq(s, p, buffered) == X86EMUL_UNHANDLEABLE )
+ if ( hvm_send_ioreq(s, p, buffered) == IOREQ_STATUS_UNHANDLED )
failed++;
}
return failed;
}
-static int hvm_access_cf8(
- int dir, unsigned int port, unsigned int bytes, uint32_t *val)
-{
- struct domain *d = current->domain;
-
- if ( dir == IOREQ_WRITE && bytes == 4 )
- d->arch.hvm.pci_cf8 = *val;
-
- /* We always need to fall through to the catch all emulator */
- return X86EMUL_UNHANDLEABLE;
-}
-
void hvm_ioreq_init(struct domain *d)
{
spin_lock_init(&d->arch.hvm.ioreq_server.lock);
- register_portio_handler(d, 0xcf8, 4, hvm_access_cf8);
+ arch_hvm_ioreq_init(d);
}
/*
diff --git a/xen/include/asm-x86/hvm/ioreq.h b/xen/include/asm-x86/hvm/ioreq.h
index e2588e9..376e2ef 100644
--- a/xen/include/asm-x86/hvm/ioreq.h
+++ b/xen/include/asm-x86/hvm/ioreq.h
@@ -19,6 +19,165 @@
#ifndef __ASM_X86_HVM_IOREQ_H__
#define __ASM_X86_HVM_IOREQ_H__
+#include <asm/hvm/emulate.h>
+#include <asm/hvm/vmx/vmx.h>
+
+#include <public/hvm/params.h>
+
+struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
+ unsigned int id);
+
+static inline bool arch_hvm_io_completion(enum hvm_io_completion io_completion)
+{
+ switch ( io_completion )
+ {
+ case HVMIO_realmode_completion:
+ {
+ struct hvm_emulate_ctxt ctxt;
+
+ hvm_emulate_init_once(&ctxt, NULL, guest_cpu_user_regs());
+ vmx_realmode_emulate_one(&ctxt);
+ hvm_emulate_writeback(&ctxt);
+
+ break;
+ }
+
+ default:
+ ASSERT_UNREACHABLE();
+ break;
+ }
+
+ return true;
+}
+
+/* Called when target domain is paused */
+static inline void arch_hvm_destroy_ioreq_server(struct hvm_ioreq_server *s)
+{
+ p2m_set_ioreq_server(s->target, 0, s);
+}
+
+/*
+ * Map or unmap an ioreq server to specific memory type. For now, only
+ * HVMMEM_ioreq_server is supported, and in the future new types can be
+ * introduced, e.g. HVMMEM_ioreq_serverX mapped to ioreq server X. And
+ * currently, only write operations are to be forwarded to an ioreq server.
+ * Support for the emulation of read operations can be added when an ioreq
+ * server has such requirement in the future.
+ */
+static inline int hvm_map_mem_type_to_ioreq_server(struct domain *d,
+ ioservid_t id,
+ uint32_t type,
+ uint32_t flags)
+{
+ struct hvm_ioreq_server *s;
+ int rc;
+
+ if ( type != HVMMEM_ioreq_server )
+ return -EINVAL;
+
+ if ( flags & ~XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
+ return -EINVAL;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ s = get_ioreq_server(d, id);
+
+ rc = -ENOENT;
+ if ( !s )
+ goto out;
+
+ rc = -EPERM;
+ if ( s->emulator != current->domain )
+ goto out;
+
+ rc = p2m_set_ioreq_server(d, flags, s);
+
+ out:
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ if ( rc == 0 && flags == 0 )
+ {
+ struct p2m_domain *p2m = p2m_get_hostp2m(d);
+
+ if ( read_atomic(&p2m->ioreq.entry_count) )
+ p2m_change_entry_type_global(d, p2m_ioreq_server, p2m_ram_rw);
+ }
+
+ return rc;
+}
+
+static inline int hvm_ioreq_server_get_type_addr(const struct domain *d,
+ const ioreq_t *p,
+ uint8_t *type,
+ uint64_t *addr)
+{
+ uint32_t cf8 = d->arch.hvm.pci_cf8;
+
+ if ( p->type != IOREQ_TYPE_COPY && p->type != IOREQ_TYPE_PIO )
+ return -EINVAL;
+
+ if ( p->type == IOREQ_TYPE_PIO &&
+ (p->addr & ~3) == 0xcfc &&
+ CF8_ENABLED(cf8) )
+ {
+ uint32_t x86_fam;
+ pci_sbdf_t sbdf;
+ unsigned int reg;
+
+ reg = hvm_pci_decode_addr(cf8, p->addr, &sbdf);
+
+ /* PCI config data cycle */
+ *type = XEN_DMOP_IO_RANGE_PCI;
+ *addr = ((uint64_t)sbdf.sbdf << 32) | reg;
+ /* AMD extended configuration space access? */
+ if ( CF8_ADDR_HI(cf8) &&
+ d->arch.cpuid->x86_vendor == X86_VENDOR_AMD &&
+ (x86_fam = get_cpu_family(
+ d->arch.cpuid->basic.raw_fms, NULL, NULL)) >= 0x10 &&
+ x86_fam < 0x17 )
+ {
+ uint64_t msr_val;
+
+ if ( !rdmsr_safe(MSR_AMD64_NB_CFG, msr_val) &&
+ (msr_val & (1ULL << AMD64_NB_CFG_CF8_EXT_ENABLE_BIT)) )
+ *addr |= CF8_ADDR_HI(cf8);
+ }
+ }
+ else
+ {
+ *type = (p->type == IOREQ_TYPE_PIO) ?
+ XEN_DMOP_IO_RANGE_PORT : XEN_DMOP_IO_RANGE_MEMORY;
+ *addr = p->addr;
+ }
+
+ return 0;
+}
+
+static inline int hvm_access_cf8(
+ int dir, unsigned int port, unsigned int bytes, uint32_t *val)
+{
+ struct domain *d = current->domain;
+
+ if ( dir == IOREQ_WRITE && bytes == 4 )
+ d->arch.hvm.pci_cf8 = *val;
+
+ /* We always need to fall through to the catch all emulator */
+ return X86EMUL_UNHANDLEABLE;
+}
+
+static inline void arch_hvm_ioreq_init(struct domain *d)
+{
+ register_portio_handler(d, 0xcf8, 4, hvm_access_cf8);
+}
+
+static inline bool arch_hvm_ioreq_destroy(struct domain *d)
+{
+ if ( !relocate_portio_handler(d, 0xcf8, 0xcf8, 4) )
+ return false;
+
+ return true;
+}
+
bool hvm_io_pending(struct vcpu *v);
bool handle_hvm_io_completion(struct vcpu *v);
bool is_ioreq_server_page(struct domain *d, const struct page_info *page);
@@ -38,8 +197,6 @@ int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
uint32_t type, uint64_t start,
uint64_t end);
-int hvm_map_mem_type_to_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint32_t flags);
int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
bool enabled);
@@ -55,6 +212,10 @@ unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered);
void hvm_ioreq_init(struct domain *d);
+#define IOREQ_STATUS_HANDLED X86EMUL_OKAY
+#define IOREQ_STATUS_UNHANDLED X86EMUL_UNHANDLEABLE
+#define IOREQ_STATUS_RETRY X86EMUL_RETRY
+
#endif /* __ASM_X86_HVM_IOREQ_H__ */
/*
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 02/23] xen/ioreq: Make x86's IOREQ feature common
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
2020-10-15 16:44 ` [PATCH V2 01/23] x86/ioreq: Prepare IOREQ feature for making it common Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-10-20 7:57 ` Paul Durrant
2020-11-12 11:11 ` Jan Beulich
2020-10-15 16:44 ` [PATCH V2 03/23] xen/ioreq: Make x86's hvm_ioreq_needs_completion() common Oleksandr Tyshchenko
` (21 subsequent siblings)
23 siblings, 2 replies; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Andrew Cooper, George Dunlap, Ian Jackson,
Jan Beulich, Julien Grall, Stefano Stabellini, Wei Liu,
Roger Pau Monné,
Paul Durrant, Tim Deegan, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
As a lot of x86 code can be re-used on Arm later on, this patch
moves previously prepared x86/hvm/ioreq.c to the common code.
The common IOREQ feature is supposed to be built with IOREQ_SERVER
option enabled, which is selected for x86's config HVM for now.
In order to avoid having a gigantic patch here, the subsequent
patches will update remaining bits in the common code step by step:
- Make IOREQ related structs/materials common
- Drop the "hvm" prefixes and infixes
- Remove layering violation by moving corresponding fields
out of *arch.hvm* or abstracting away accesses to them
This support is going to be used on Arm to be able run device
emulator outside of Xen hypervisor.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
***
Please note, this patch depends on the following which is
on review:
https://patchwork.kernel.org/patch/11816689/
***
Changes RFC -> V1:
- was split into three patches:
- x86/ioreq: Prepare IOREQ feature for making it common
- xen/ioreq: Make x86's IOREQ feature common
- xen/ioreq: Make x86's hvm_ioreq_needs_completion() common
- update MAINTAINERS file
- do not use a separate subdir for the IOREQ stuff, move it to:
- xen/common/ioreq.c
- xen/include/xen/ioreq.h
- update x86's files to include xen/ioreq.h
- remove unneeded headers in arch/x86/hvm/ioreq.c
- re-order the headers alphabetically in common/ioreq.c
- update common/ioreq.c according to the newly introduced arch functions:
arch_hvm_destroy_ioreq_server()/arch_handle_hvm_io_completion()
Changes V1 -> V2:
- update patch description
- make everything needed in the previous patch to achieve
a truly rename here
- don't include unnecessary headers from asm-x86/hvm/ioreq.h
and xen/ioreq.h
- use __XEN_IOREQ_H__ instead of __IOREQ_H__
- move get_ioreq_server() to common/ioreq.c
---
MAINTAINERS | 8 +-
xen/arch/x86/Kconfig | 1 +
xen/arch/x86/hvm/Makefile | 1 -
xen/arch/x86/hvm/ioreq.c | 1422 ---------------------------------------
xen/arch/x86/mm.c | 2 +-
xen/arch/x86/mm/shadow/common.c | 2 +-
xen/common/Kconfig | 3 +
xen/common/Makefile | 1 +
xen/common/ioreq.c | 1422 +++++++++++++++++++++++++++++++++++++++
xen/include/asm-x86/hvm/ioreq.h | 39 +-
xen/include/xen/ioreq.h | 71 ++
11 files changed, 1509 insertions(+), 1463 deletions(-)
delete mode 100644 xen/arch/x86/hvm/ioreq.c
create mode 100644 xen/common/ioreq.c
create mode 100644 xen/include/xen/ioreq.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 26c5382..cbb00d6 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -333,6 +333,13 @@ X: xen/drivers/passthrough/vtd/
X: xen/drivers/passthrough/device_tree.c
F: xen/include/xen/iommu.h
+I/O EMULATION (IOREQ)
+M: Paul Durrant <paul@xen.org>
+S: Supported
+F: xen/common/ioreq.c
+F: xen/include/xen/ioreq.h
+F: xen/include/public/hvm/ioreq.h
+
KCONFIG
M: Doug Goldstein <cardoe@cardoe.com>
S: Supported
@@ -549,7 +556,6 @@ F: xen/arch/x86/hvm/ioreq.c
F: xen/include/asm-x86/hvm/emulate.h
F: xen/include/asm-x86/hvm/io.h
F: xen/include/asm-x86/hvm/ioreq.h
-F: xen/include/public/hvm/ioreq.h
X86 MEMORY MANAGEMENT
M: Jan Beulich <jbeulich@suse.com>
diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index 24868aa..abe0fce 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -91,6 +91,7 @@ config PV_LINEAR_PT
config HVM
def_bool !PV_SHIM_EXCLUSIVE
+ select IOREQ_SERVER
prompt "HVM support"
---help---
Interfaces to support HVM domains. HVM domains require hardware
diff --git a/xen/arch/x86/hvm/Makefile b/xen/arch/x86/hvm/Makefile
index 3464191..0c1eff2 100644
--- a/xen/arch/x86/hvm/Makefile
+++ b/xen/arch/x86/hvm/Makefile
@@ -13,7 +13,6 @@ obj-y += hvm.o
obj-y += hypercall.o
obj-y += intercept.o
obj-y += io.o
-obj-y += ioreq.o
obj-y += irq.o
obj-y += monitor.o
obj-y += mtrr.o
diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
deleted file mode 100644
index d3433d7..0000000
--- a/xen/arch/x86/hvm/ioreq.c
+++ /dev/null
@@ -1,1422 +0,0 @@
-/*
- * ioreq.c: hardware virtual machine I/O emulation
- *
- * Copyright (c) 2016 Citrix Systems Inc.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms and conditions of the GNU General Public License,
- * version 2, as published by the Free Software Foundation.
- *
- * This program is distributed in the hope it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
- * more details.
- *
- * You should have received a copy of the GNU General Public License along with
- * this program; If not, see <http://www.gnu.org/licenses/>.
- */
-
-#include <xen/ctype.h>
-#include <xen/domain.h>
-#include <xen/event.h>
-#include <xen/init.h>
-#include <xen/irq.h>
-#include <xen/lib.h>
-#include <xen/paging.h>
-#include <xen/sched.h>
-#include <xen/softirq.h>
-#include <xen/trace.h>
-#include <xen/vpci.h>
-
-#include <asm/hvm/ioreq.h>
-
-#include <public/hvm/ioreq.h>
-#include <public/hvm/params.h>
-
-static void set_ioreq_server(struct domain *d, unsigned int id,
- struct hvm_ioreq_server *s)
-{
- ASSERT(id < MAX_NR_IOREQ_SERVERS);
- ASSERT(!s || !d->arch.hvm.ioreq_server.server[id]);
-
- d->arch.hvm.ioreq_server.server[id] = s;
-}
-
-#define GET_IOREQ_SERVER(d, id) \
- (d)->arch.hvm.ioreq_server.server[id]
-
-struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
- unsigned int id)
-{
- if ( id >= MAX_NR_IOREQ_SERVERS )
- return NULL;
-
- return GET_IOREQ_SERVER(d, id);
-}
-
-/*
- * Iterate over all possible ioreq servers.
- *
- * NOTE: The iteration is backwards such that more recently created
- * ioreq servers are favoured in hvm_select_ioreq_server().
- * This is a semantic that previously existed when ioreq servers
- * were held in a linked list.
- */
-#define FOR_EACH_IOREQ_SERVER(d, id, s) \
- for ( (id) = MAX_NR_IOREQ_SERVERS; (id) != 0; ) \
- if ( !(s = GET_IOREQ_SERVER(d, --(id))) ) \
- continue; \
- else
-
-static ioreq_t *get_ioreq(struct hvm_ioreq_server *s, struct vcpu *v)
-{
- shared_iopage_t *p = s->ioreq.va;
-
- ASSERT((v == current) || !vcpu_runnable(v));
- ASSERT(p != NULL);
-
- return &p->vcpu_ioreq[v->vcpu_id];
-}
-
-static struct hvm_ioreq_vcpu *get_pending_vcpu(const struct vcpu *v,
- struct hvm_ioreq_server **srvp)
-{
- struct domain *d = v->domain;
- struct hvm_ioreq_server *s;
- unsigned int id;
-
- FOR_EACH_IOREQ_SERVER(d, id, s)
- {
- struct hvm_ioreq_vcpu *sv;
-
- list_for_each_entry ( sv,
- &s->ioreq_vcpu_list,
- list_entry )
- {
- if ( sv->vcpu == v && sv->pending )
- {
- if ( srvp )
- *srvp = s;
- return sv;
- }
- }
- }
-
- return NULL;
-}
-
-bool hvm_io_pending(struct vcpu *v)
-{
- return get_pending_vcpu(v, NULL);
-}
-
-static bool hvm_wait_for_io(struct hvm_ioreq_vcpu *sv, ioreq_t *p)
-{
- unsigned int prev_state = STATE_IOREQ_NONE;
- unsigned int state = p->state;
- uint64_t data = ~0;
-
- smp_rmb();
-
- /*
- * The only reason we should see this condition be false is when an
- * emulator dying races with I/O being requested.
- */
- while ( likely(state != STATE_IOREQ_NONE) )
- {
- if ( unlikely(state < prev_state) )
- {
- gdprintk(XENLOG_ERR, "Weird HVM ioreq state transition %u -> %u\n",
- prev_state, state);
- sv->pending = false;
- domain_crash(sv->vcpu->domain);
- return false; /* bail */
- }
-
- switch ( prev_state = state )
- {
- case STATE_IORESP_READY: /* IORESP_READY -> NONE */
- p->state = STATE_IOREQ_NONE;
- data = p->data;
- break;
-
- case STATE_IOREQ_READY: /* IOREQ_{READY,INPROCESS} -> IORESP_READY */
- case STATE_IOREQ_INPROCESS:
- wait_on_xen_event_channel(sv->ioreq_evtchn,
- ({ state = p->state;
- smp_rmb();
- state != prev_state; }));
- continue;
-
- default:
- gdprintk(XENLOG_ERR, "Weird HVM iorequest state %u\n", state);
- sv->pending = false;
- domain_crash(sv->vcpu->domain);
- return false; /* bail */
- }
-
- break;
- }
-
- p = &sv->vcpu->arch.hvm.hvm_io.io_req;
- if ( hvm_ioreq_needs_completion(p) )
- p->data = data;
-
- sv->pending = false;
-
- return true;
-}
-
-bool handle_hvm_io_completion(struct vcpu *v)
-{
- struct domain *d = v->domain;
- struct hvm_vcpu_io *vio = &v->arch.hvm.hvm_io;
- struct hvm_ioreq_server *s;
- struct hvm_ioreq_vcpu *sv;
- enum hvm_io_completion io_completion;
-
- if ( has_vpci(d) && vpci_process_pending(v) )
- {
- raise_softirq(SCHEDULE_SOFTIRQ);
- return false;
- }
-
- sv = get_pending_vcpu(v, &s);
- if ( sv && !hvm_wait_for_io(sv, get_ioreq(s, v)) )
- return false;
-
- vio->io_req.state = hvm_ioreq_needs_completion(&vio->io_req) ?
- STATE_IORESP_READY : STATE_IOREQ_NONE;
-
- msix_write_completion(v);
- vcpu_end_shutdown_deferral(v);
-
- io_completion = vio->io_completion;
- vio->io_completion = HVMIO_no_completion;
-
- switch ( io_completion )
- {
- case HVMIO_no_completion:
- break;
-
- case HVMIO_mmio_completion:
- return handle_mmio();
-
- case HVMIO_pio_completion:
- return handle_pio(vio->io_req.addr, vio->io_req.size,
- vio->io_req.dir);
-
- default:
- return arch_hvm_io_completion(io_completion);
- }
-
- return true;
-}
-
-static gfn_t hvm_alloc_legacy_ioreq_gfn(struct hvm_ioreq_server *s)
-{
- struct domain *d = s->target;
- unsigned int i;
-
- BUILD_BUG_ON(HVM_PARAM_BUFIOREQ_PFN != HVM_PARAM_IOREQ_PFN + 1);
-
- for ( i = HVM_PARAM_IOREQ_PFN; i <= HVM_PARAM_BUFIOREQ_PFN; i++ )
- {
- if ( !test_and_clear_bit(i, &d->arch.hvm.ioreq_gfn.legacy_mask) )
- return _gfn(d->arch.hvm.params[i]);
- }
-
- return INVALID_GFN;
-}
-
-static gfn_t hvm_alloc_ioreq_gfn(struct hvm_ioreq_server *s)
-{
- struct domain *d = s->target;
- unsigned int i;
-
- for ( i = 0; i < sizeof(d->arch.hvm.ioreq_gfn.mask) * 8; i++ )
- {
- if ( test_and_clear_bit(i, &d->arch.hvm.ioreq_gfn.mask) )
- return _gfn(d->arch.hvm.ioreq_gfn.base + i);
- }
-
- /*
- * If we are out of 'normal' GFNs then we may still have a 'legacy'
- * GFN available.
- */
- return hvm_alloc_legacy_ioreq_gfn(s);
-}
-
-static bool hvm_free_legacy_ioreq_gfn(struct hvm_ioreq_server *s,
- gfn_t gfn)
-{
- struct domain *d = s->target;
- unsigned int i;
-
- for ( i = HVM_PARAM_IOREQ_PFN; i <= HVM_PARAM_BUFIOREQ_PFN; i++ )
- {
- if ( gfn_eq(gfn, _gfn(d->arch.hvm.params[i])) )
- break;
- }
- if ( i > HVM_PARAM_BUFIOREQ_PFN )
- return false;
-
- set_bit(i, &d->arch.hvm.ioreq_gfn.legacy_mask);
- return true;
-}
-
-static void hvm_free_ioreq_gfn(struct hvm_ioreq_server *s, gfn_t gfn)
-{
- struct domain *d = s->target;
- unsigned int i = gfn_x(gfn) - d->arch.hvm.ioreq_gfn.base;
-
- ASSERT(!gfn_eq(gfn, INVALID_GFN));
-
- if ( !hvm_free_legacy_ioreq_gfn(s, gfn) )
- {
- ASSERT(i < sizeof(d->arch.hvm.ioreq_gfn.mask) * 8);
- set_bit(i, &d->arch.hvm.ioreq_gfn.mask);
- }
-}
-
-static void hvm_unmap_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
-{
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
-
- if ( gfn_eq(iorp->gfn, INVALID_GFN) )
- return;
-
- destroy_ring_for_helper(&iorp->va, iorp->page);
- iorp->page = NULL;
-
- hvm_free_ioreq_gfn(s, iorp->gfn);
- iorp->gfn = INVALID_GFN;
-}
-
-static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
-{
- struct domain *d = s->target;
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
- int rc;
-
- if ( iorp->page )
- {
- /*
- * If a page has already been allocated (which will happen on
- * demand if hvm_get_ioreq_server_frame() is called), then
- * mapping a guest frame is not permitted.
- */
- if ( gfn_eq(iorp->gfn, INVALID_GFN) )
- return -EPERM;
-
- return 0;
- }
-
- if ( d->is_dying )
- return -EINVAL;
-
- iorp->gfn = hvm_alloc_ioreq_gfn(s);
-
- if ( gfn_eq(iorp->gfn, INVALID_GFN) )
- return -ENOMEM;
-
- rc = prepare_ring_for_helper(d, gfn_x(iorp->gfn), &iorp->page,
- &iorp->va);
-
- if ( rc )
- hvm_unmap_ioreq_gfn(s, buf);
-
- return rc;
-}
-
-static int hvm_alloc_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
-{
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
- struct page_info *page;
-
- if ( iorp->page )
- {
- /*
- * If a guest frame has already been mapped (which may happen
- * on demand if hvm_get_ioreq_server_info() is called), then
- * allocating a page is not permitted.
- */
- if ( !gfn_eq(iorp->gfn, INVALID_GFN) )
- return -EPERM;
-
- return 0;
- }
-
- page = alloc_domheap_page(s->target, MEMF_no_refcount);
-
- if ( !page )
- return -ENOMEM;
-
- if ( !get_page_and_type(page, s->target, PGT_writable_page) )
- {
- /*
- * The domain can't possibly know about this page yet, so failure
- * here is a clear indication of something fishy going on.
- */
- domain_crash(s->emulator);
- return -ENODATA;
- }
-
- iorp->va = __map_domain_page_global(page);
- if ( !iorp->va )
- goto fail;
-
- iorp->page = page;
- clear_page(iorp->va);
- return 0;
-
- fail:
- put_page_alloc_ref(page);
- put_page_and_type(page);
-
- return -ENOMEM;
-}
-
-static void hvm_free_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
-{
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
- struct page_info *page = iorp->page;
-
- if ( !page )
- return;
-
- iorp->page = NULL;
-
- unmap_domain_page_global(iorp->va);
- iorp->va = NULL;
-
- put_page_alloc_ref(page);
- put_page_and_type(page);
-}
-
-bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
-{
- const struct hvm_ioreq_server *s;
- unsigned int id;
- bool found = false;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- FOR_EACH_IOREQ_SERVER(d, id, s)
- {
- if ( (s->ioreq.page == page) || (s->bufioreq.page == page) )
- {
- found = true;
- break;
- }
- }
-
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- return found;
-}
-
-static void hvm_remove_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
-
-{
- struct domain *d = s->target;
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
-
- if ( gfn_eq(iorp->gfn, INVALID_GFN) )
- return;
-
- if ( guest_physmap_remove_page(d, iorp->gfn,
- page_to_mfn(iorp->page), 0) )
- domain_crash(d);
- clear_page(iorp->va);
-}
-
-static int hvm_add_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
-{
- struct domain *d = s->target;
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
- int rc;
-
- if ( gfn_eq(iorp->gfn, INVALID_GFN) )
- return 0;
-
- clear_page(iorp->va);
-
- rc = guest_physmap_add_page(d, iorp->gfn,
- page_to_mfn(iorp->page), 0);
- if ( rc == 0 )
- paging_mark_pfn_dirty(d, _pfn(gfn_x(iorp->gfn)));
-
- return rc;
-}
-
-static void hvm_update_ioreq_evtchn(struct hvm_ioreq_server *s,
- struct hvm_ioreq_vcpu *sv)
-{
- ASSERT(spin_is_locked(&s->lock));
-
- if ( s->ioreq.va != NULL )
- {
- ioreq_t *p = get_ioreq(s, sv->vcpu);
-
- p->vp_eport = sv->ioreq_evtchn;
- }
-}
-
-#define HANDLE_BUFIOREQ(s) \
- ((s)->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF)
-
-static int hvm_ioreq_server_add_vcpu(struct hvm_ioreq_server *s,
- struct vcpu *v)
-{
- struct hvm_ioreq_vcpu *sv;
- int rc;
-
- sv = xzalloc(struct hvm_ioreq_vcpu);
-
- rc = -ENOMEM;
- if ( !sv )
- goto fail1;
-
- spin_lock(&s->lock);
-
- rc = alloc_unbound_xen_event_channel(v->domain, v->vcpu_id,
- s->emulator->domain_id, NULL);
- if ( rc < 0 )
- goto fail2;
-
- sv->ioreq_evtchn = rc;
-
- if ( v->vcpu_id == 0 && HANDLE_BUFIOREQ(s) )
- {
- rc = alloc_unbound_xen_event_channel(v->domain, 0,
- s->emulator->domain_id, NULL);
- if ( rc < 0 )
- goto fail3;
-
- s->bufioreq_evtchn = rc;
- }
-
- sv->vcpu = v;
-
- list_add(&sv->list_entry, &s->ioreq_vcpu_list);
-
- if ( s->enabled )
- hvm_update_ioreq_evtchn(s, sv);
-
- spin_unlock(&s->lock);
- return 0;
-
- fail3:
- free_xen_event_channel(v->domain, sv->ioreq_evtchn);
-
- fail2:
- spin_unlock(&s->lock);
- xfree(sv);
-
- fail1:
- return rc;
-}
-
-static void hvm_ioreq_server_remove_vcpu(struct hvm_ioreq_server *s,
- struct vcpu *v)
-{
- struct hvm_ioreq_vcpu *sv;
-
- spin_lock(&s->lock);
-
- list_for_each_entry ( sv,
- &s->ioreq_vcpu_list,
- list_entry )
- {
- if ( sv->vcpu != v )
- continue;
-
- list_del(&sv->list_entry);
-
- if ( v->vcpu_id == 0 && HANDLE_BUFIOREQ(s) )
- free_xen_event_channel(v->domain, s->bufioreq_evtchn);
-
- free_xen_event_channel(v->domain, sv->ioreq_evtchn);
-
- xfree(sv);
- break;
- }
-
- spin_unlock(&s->lock);
-}
-
-static void hvm_ioreq_server_remove_all_vcpus(struct hvm_ioreq_server *s)
-{
- struct hvm_ioreq_vcpu *sv, *next;
-
- spin_lock(&s->lock);
-
- list_for_each_entry_safe ( sv,
- next,
- &s->ioreq_vcpu_list,
- list_entry )
- {
- struct vcpu *v = sv->vcpu;
-
- list_del(&sv->list_entry);
-
- if ( v->vcpu_id == 0 && HANDLE_BUFIOREQ(s) )
- free_xen_event_channel(v->domain, s->bufioreq_evtchn);
-
- free_xen_event_channel(v->domain, sv->ioreq_evtchn);
-
- xfree(sv);
- }
-
- spin_unlock(&s->lock);
-}
-
-static int hvm_ioreq_server_map_pages(struct hvm_ioreq_server *s)
-{
- int rc;
-
- rc = hvm_map_ioreq_gfn(s, false);
-
- if ( !rc && HANDLE_BUFIOREQ(s) )
- rc = hvm_map_ioreq_gfn(s, true);
-
- if ( rc )
- hvm_unmap_ioreq_gfn(s, false);
-
- return rc;
-}
-
-static void hvm_ioreq_server_unmap_pages(struct hvm_ioreq_server *s)
-{
- hvm_unmap_ioreq_gfn(s, true);
- hvm_unmap_ioreq_gfn(s, false);
-}
-
-static int hvm_ioreq_server_alloc_pages(struct hvm_ioreq_server *s)
-{
- int rc;
-
- rc = hvm_alloc_ioreq_mfn(s, false);
-
- if ( !rc && (s->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF) )
- rc = hvm_alloc_ioreq_mfn(s, true);
-
- if ( rc )
- hvm_free_ioreq_mfn(s, false);
-
- return rc;
-}
-
-static void hvm_ioreq_server_free_pages(struct hvm_ioreq_server *s)
-{
- hvm_free_ioreq_mfn(s, true);
- hvm_free_ioreq_mfn(s, false);
-}
-
-static void hvm_ioreq_server_free_rangesets(struct hvm_ioreq_server *s)
-{
- unsigned int i;
-
- for ( i = 0; i < NR_IO_RANGE_TYPES; i++ )
- rangeset_destroy(s->range[i]);
-}
-
-static int hvm_ioreq_server_alloc_rangesets(struct hvm_ioreq_server *s,
- ioservid_t id)
-{
- unsigned int i;
- int rc;
-
- for ( i = 0; i < NR_IO_RANGE_TYPES; i++ )
- {
- char *name;
-
- rc = asprintf(&name, "ioreq_server %d %s", id,
- (i == XEN_DMOP_IO_RANGE_PORT) ? "port" :
- (i == XEN_DMOP_IO_RANGE_MEMORY) ? "memory" :
- (i == XEN_DMOP_IO_RANGE_PCI) ? "pci" :
- "");
- if ( rc )
- goto fail;
-
- s->range[i] = rangeset_new(s->target, name,
- RANGESETF_prettyprint_hex);
-
- xfree(name);
-
- rc = -ENOMEM;
- if ( !s->range[i] )
- goto fail;
-
- rangeset_limit(s->range[i], MAX_NR_IO_RANGES);
- }
-
- return 0;
-
- fail:
- hvm_ioreq_server_free_rangesets(s);
-
- return rc;
-}
-
-static void hvm_ioreq_server_enable(struct hvm_ioreq_server *s)
-{
- struct hvm_ioreq_vcpu *sv;
-
- spin_lock(&s->lock);
-
- if ( s->enabled )
- goto done;
-
- hvm_remove_ioreq_gfn(s, false);
- hvm_remove_ioreq_gfn(s, true);
-
- s->enabled = true;
-
- list_for_each_entry ( sv,
- &s->ioreq_vcpu_list,
- list_entry )
- hvm_update_ioreq_evtchn(s, sv);
-
- done:
- spin_unlock(&s->lock);
-}
-
-static void hvm_ioreq_server_disable(struct hvm_ioreq_server *s)
-{
- spin_lock(&s->lock);
-
- if ( !s->enabled )
- goto done;
-
- hvm_add_ioreq_gfn(s, true);
- hvm_add_ioreq_gfn(s, false);
-
- s->enabled = false;
-
- done:
- spin_unlock(&s->lock);
-}
-
-static int hvm_ioreq_server_init(struct hvm_ioreq_server *s,
- struct domain *d, int bufioreq_handling,
- ioservid_t id)
-{
- struct domain *currd = current->domain;
- struct vcpu *v;
- int rc;
-
- s->target = d;
-
- get_knownalive_domain(currd);
- s->emulator = currd;
-
- spin_lock_init(&s->lock);
- INIT_LIST_HEAD(&s->ioreq_vcpu_list);
- spin_lock_init(&s->bufioreq_lock);
-
- s->ioreq.gfn = INVALID_GFN;
- s->bufioreq.gfn = INVALID_GFN;
-
- rc = hvm_ioreq_server_alloc_rangesets(s, id);
- if ( rc )
- return rc;
-
- s->bufioreq_handling = bufioreq_handling;
-
- for_each_vcpu ( d, v )
- {
- rc = hvm_ioreq_server_add_vcpu(s, v);
- if ( rc )
- goto fail_add;
- }
-
- return 0;
-
- fail_add:
- hvm_ioreq_server_remove_all_vcpus(s);
- hvm_ioreq_server_unmap_pages(s);
-
- hvm_ioreq_server_free_rangesets(s);
-
- put_domain(s->emulator);
- return rc;
-}
-
-static void hvm_ioreq_server_deinit(struct hvm_ioreq_server *s)
-{
- ASSERT(!s->enabled);
- hvm_ioreq_server_remove_all_vcpus(s);
-
- /*
- * NOTE: It is safe to call both hvm_ioreq_server_unmap_pages() and
- * hvm_ioreq_server_free_pages() in that order.
- * This is because the former will do nothing if the pages
- * are not mapped, leaving the page to be freed by the latter.
- * However if the pages are mapped then the former will set
- * the page_info pointer to NULL, meaning the latter will do
- * nothing.
- */
- hvm_ioreq_server_unmap_pages(s);
- hvm_ioreq_server_free_pages(s);
-
- hvm_ioreq_server_free_rangesets(s);
-
- put_domain(s->emulator);
-}
-
-int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
- ioservid_t *id)
-{
- struct hvm_ioreq_server *s;
- unsigned int i;
- int rc;
-
- if ( bufioreq_handling > HVM_IOREQSRV_BUFIOREQ_ATOMIC )
- return -EINVAL;
-
- s = xzalloc(struct hvm_ioreq_server);
- if ( !s )
- return -ENOMEM;
-
- domain_pause(d);
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- for ( i = 0; i < MAX_NR_IOREQ_SERVERS; i++ )
- {
- if ( !GET_IOREQ_SERVER(d, i) )
- break;
- }
-
- rc = -ENOSPC;
- if ( i >= MAX_NR_IOREQ_SERVERS )
- goto fail;
-
- /*
- * It is safe to call set_ioreq_server() prior to
- * hvm_ioreq_server_init() since the target domain is paused.
- */
- set_ioreq_server(d, i, s);
-
- rc = hvm_ioreq_server_init(s, d, bufioreq_handling, i);
- if ( rc )
- {
- set_ioreq_server(d, i, NULL);
- goto fail;
- }
-
- if ( id )
- *id = i;
-
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
- domain_unpause(d);
-
- return 0;
-
- fail:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
- domain_unpause(d);
-
- xfree(s);
- return rc;
-}
-
-int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
-{
- struct hvm_ioreq_server *s;
- int rc;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- s = get_ioreq_server(d, id);
-
- rc = -ENOENT;
- if ( !s )
- goto out;
-
- rc = -EPERM;
- if ( s->emulator != current->domain )
- goto out;
-
- domain_pause(d);
-
- arch_hvm_destroy_ioreq_server(s);
-
- hvm_ioreq_server_disable(s);
-
- /*
- * It is safe to call hvm_ioreq_server_deinit() prior to
- * set_ioreq_server() since the target domain is paused.
- */
- hvm_ioreq_server_deinit(s);
- set_ioreq_server(d, id, NULL);
-
- domain_unpause(d);
-
- xfree(s);
-
- rc = 0;
-
- out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- return rc;
-}
-
-int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
- unsigned long *ioreq_gfn,
- unsigned long *bufioreq_gfn,
- evtchn_port_t *bufioreq_port)
-{
- struct hvm_ioreq_server *s;
- int rc;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- s = get_ioreq_server(d, id);
-
- rc = -ENOENT;
- if ( !s )
- goto out;
-
- rc = -EPERM;
- if ( s->emulator != current->domain )
- goto out;
-
- if ( ioreq_gfn || bufioreq_gfn )
- {
- rc = hvm_ioreq_server_map_pages(s);
- if ( rc )
- goto out;
- }
-
- if ( ioreq_gfn )
- *ioreq_gfn = gfn_x(s->ioreq.gfn);
-
- if ( HANDLE_BUFIOREQ(s) )
- {
- if ( bufioreq_gfn )
- *bufioreq_gfn = gfn_x(s->bufioreq.gfn);
-
- if ( bufioreq_port )
- *bufioreq_port = s->bufioreq_evtchn;
- }
-
- rc = 0;
-
- out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- return rc;
-}
-
-int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
- unsigned long idx, mfn_t *mfn)
-{
- struct hvm_ioreq_server *s;
- int rc;
-
- ASSERT(is_hvm_domain(d));
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- s = get_ioreq_server(d, id);
-
- rc = -ENOENT;
- if ( !s )
- goto out;
-
- rc = -EPERM;
- if ( s->emulator != current->domain )
- goto out;
-
- rc = hvm_ioreq_server_alloc_pages(s);
- if ( rc )
- goto out;
-
- switch ( idx )
- {
- case XENMEM_resource_ioreq_server_frame_bufioreq:
- rc = -ENOENT;
- if ( !HANDLE_BUFIOREQ(s) )
- goto out;
-
- *mfn = page_to_mfn(s->bufioreq.page);
- rc = 0;
- break;
-
- case XENMEM_resource_ioreq_server_frame_ioreq(0):
- *mfn = page_to_mfn(s->ioreq.page);
- rc = 0;
- break;
-
- default:
- rc = -EINVAL;
- break;
- }
-
- out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- return rc;
-}
-
-int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint64_t start,
- uint64_t end)
-{
- struct hvm_ioreq_server *s;
- struct rangeset *r;
- int rc;
-
- if ( start > end )
- return -EINVAL;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- s = get_ioreq_server(d, id);
-
- rc = -ENOENT;
- if ( !s )
- goto out;
-
- rc = -EPERM;
- if ( s->emulator != current->domain )
- goto out;
-
- switch ( type )
- {
- case XEN_DMOP_IO_RANGE_PORT:
- case XEN_DMOP_IO_RANGE_MEMORY:
- case XEN_DMOP_IO_RANGE_PCI:
- r = s->range[type];
- break;
-
- default:
- r = NULL;
- break;
- }
-
- rc = -EINVAL;
- if ( !r )
- goto out;
-
- rc = -EEXIST;
- if ( rangeset_overlaps_range(r, start, end) )
- goto out;
-
- rc = rangeset_add_range(r, start, end);
-
- out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- return rc;
-}
-
-int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint64_t start,
- uint64_t end)
-{
- struct hvm_ioreq_server *s;
- struct rangeset *r;
- int rc;
-
- if ( start > end )
- return -EINVAL;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- s = get_ioreq_server(d, id);
-
- rc = -ENOENT;
- if ( !s )
- goto out;
-
- rc = -EPERM;
- if ( s->emulator != current->domain )
- goto out;
-
- switch ( type )
- {
- case XEN_DMOP_IO_RANGE_PORT:
- case XEN_DMOP_IO_RANGE_MEMORY:
- case XEN_DMOP_IO_RANGE_PCI:
- r = s->range[type];
- break;
-
- default:
- r = NULL;
- break;
- }
-
- rc = -EINVAL;
- if ( !r )
- goto out;
-
- rc = -ENOENT;
- if ( !rangeset_contains_range(r, start, end) )
- goto out;
-
- rc = rangeset_remove_range(r, start, end);
-
- out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- return rc;
-}
-
-int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
- bool enabled)
-{
- struct hvm_ioreq_server *s;
- int rc;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- s = get_ioreq_server(d, id);
-
- rc = -ENOENT;
- if ( !s )
- goto out;
-
- rc = -EPERM;
- if ( s->emulator != current->domain )
- goto out;
-
- domain_pause(d);
-
- if ( enabled )
- hvm_ioreq_server_enable(s);
- else
- hvm_ioreq_server_disable(s);
-
- domain_unpause(d);
-
- rc = 0;
-
- out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
- return rc;
-}
-
-int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
-{
- struct hvm_ioreq_server *s;
- unsigned int id;
- int rc;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- FOR_EACH_IOREQ_SERVER(d, id, s)
- {
- rc = hvm_ioreq_server_add_vcpu(s, v);
- if ( rc )
- goto fail;
- }
-
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- return 0;
-
- fail:
- while ( ++id != MAX_NR_IOREQ_SERVERS )
- {
- s = GET_IOREQ_SERVER(d, id);
-
- if ( !s )
- continue;
-
- hvm_ioreq_server_remove_vcpu(s, v);
- }
-
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- return rc;
-}
-
-void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v)
-{
- struct hvm_ioreq_server *s;
- unsigned int id;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- FOR_EACH_IOREQ_SERVER(d, id, s)
- hvm_ioreq_server_remove_vcpu(s, v);
-
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-}
-
-void hvm_destroy_all_ioreq_servers(struct domain *d)
-{
- struct hvm_ioreq_server *s;
- unsigned int id;
-
- if ( !arch_hvm_ioreq_destroy(d) )
- return;
-
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
-
- /* No need to domain_pause() as the domain is being torn down */
-
- FOR_EACH_IOREQ_SERVER(d, id, s)
- {
- hvm_ioreq_server_disable(s);
-
- /*
- * It is safe to call hvm_ioreq_server_deinit() prior to
- * set_ioreq_server() since the target domain is being destroyed.
- */
- hvm_ioreq_server_deinit(s);
- set_ioreq_server(d, id, NULL);
-
- xfree(s);
- }
-
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
-}
-
-struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
- ioreq_t *p)
-{
- struct hvm_ioreq_server *s;
- uint8_t type;
- uint64_t addr;
- unsigned int id;
-
- if ( hvm_ioreq_server_get_type_addr(d, p, &type, &addr) )
- return NULL;
-
- FOR_EACH_IOREQ_SERVER(d, id, s)
- {
- struct rangeset *r;
-
- if ( !s->enabled )
- continue;
-
- r = s->range[type];
-
- switch ( type )
- {
- unsigned long start, end;
-
- case XEN_DMOP_IO_RANGE_PORT:
- start = addr;
- end = start + p->size - 1;
- if ( rangeset_contains_range(r, start, end) )
- return s;
-
- break;
-
- case XEN_DMOP_IO_RANGE_MEMORY:
- start = hvm_mmio_first_byte(p);
- end = hvm_mmio_last_byte(p);
-
- if ( rangeset_contains_range(r, start, end) )
- return s;
-
- break;
-
- case XEN_DMOP_IO_RANGE_PCI:
- if ( rangeset_contains_singleton(r, addr >> 32) )
- {
- p->type = IOREQ_TYPE_PCI_CONFIG;
- p->addr = addr;
- return s;
- }
-
- break;
- }
- }
-
- return NULL;
-}
-
-static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
-{
- struct domain *d = current->domain;
- struct hvm_ioreq_page *iorp;
- buffered_iopage_t *pg;
- buf_ioreq_t bp = { .data = p->data,
- .addr = p->addr,
- .type = p->type,
- .dir = p->dir };
- /* Timeoffset sends 64b data, but no address. Use two consecutive slots. */
- int qw = 0;
-
- /* Ensure buffered_iopage fits in a page */
- BUILD_BUG_ON(sizeof(buffered_iopage_t) > PAGE_SIZE);
-
- iorp = &s->bufioreq;
- pg = iorp->va;
-
- if ( !pg )
- return IOREQ_STATUS_UNHANDLED;
-
- /*
- * Return 0 for the cases we can't deal with:
- * - 'addr' is only a 20-bit field, so we cannot address beyond 1MB
- * - we cannot buffer accesses to guest memory buffers, as the guest
- * may expect the memory buffer to be synchronously accessed
- * - the count field is usually used with data_is_ptr and since we don't
- * support data_is_ptr we do not waste space for the count field either
- */
- if ( (p->addr > 0xffffful) || p->data_is_ptr || (p->count != 1) )
- return 0;
-
- switch ( p->size )
- {
- case 1:
- bp.size = 0;
- break;
- case 2:
- bp.size = 1;
- break;
- case 4:
- bp.size = 2;
- break;
- case 8:
- bp.size = 3;
- qw = 1;
- break;
- default:
- gdprintk(XENLOG_WARNING, "unexpected ioreq size: %u\n", p->size);
- return IOREQ_STATUS_UNHANDLED;
- }
-
- spin_lock(&s->bufioreq_lock);
-
- if ( (pg->ptrs.write_pointer - pg->ptrs.read_pointer) >=
- (IOREQ_BUFFER_SLOT_NUM - qw) )
- {
- /* The queue is full: send the iopacket through the normal path. */
- spin_unlock(&s->bufioreq_lock);
- return IOREQ_STATUS_UNHANDLED;
- }
-
- pg->buf_ioreq[pg->ptrs.write_pointer % IOREQ_BUFFER_SLOT_NUM] = bp;
-
- if ( qw )
- {
- bp.data = p->data >> 32;
- pg->buf_ioreq[(pg->ptrs.write_pointer+1) % IOREQ_BUFFER_SLOT_NUM] = bp;
- }
-
- /* Make the ioreq_t visible /before/ write_pointer. */
- smp_wmb();
- pg->ptrs.write_pointer += qw ? 2 : 1;
-
- /* Canonicalize read/write pointers to prevent their overflow. */
- while ( (s->bufioreq_handling == HVM_IOREQSRV_BUFIOREQ_ATOMIC) &&
- qw++ < IOREQ_BUFFER_SLOT_NUM &&
- pg->ptrs.read_pointer >= IOREQ_BUFFER_SLOT_NUM )
- {
- union bufioreq_pointers old = pg->ptrs, new;
- unsigned int n = old.read_pointer / IOREQ_BUFFER_SLOT_NUM;
-
- new.read_pointer = old.read_pointer - n * IOREQ_BUFFER_SLOT_NUM;
- new.write_pointer = old.write_pointer - n * IOREQ_BUFFER_SLOT_NUM;
- cmpxchg(&pg->ptrs.full, old.full, new.full);
- }
-
- notify_via_xen_event_channel(d, s->bufioreq_evtchn);
- spin_unlock(&s->bufioreq_lock);
-
- return IOREQ_STATUS_HANDLED;
-}
-
-int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
- bool buffered)
-{
- struct vcpu *curr = current;
- struct domain *d = curr->domain;
- struct hvm_ioreq_vcpu *sv;
-
- ASSERT(s);
-
- if ( buffered )
- return hvm_send_buffered_ioreq(s, proto_p);
-
- if ( unlikely(!vcpu_start_shutdown_deferral(curr)) )
- return IOREQ_STATUS_RETRY;
-
- list_for_each_entry ( sv,
- &s->ioreq_vcpu_list,
- list_entry )
- {
- if ( sv->vcpu == curr )
- {
- evtchn_port_t port = sv->ioreq_evtchn;
- ioreq_t *p = get_ioreq(s, curr);
-
- if ( unlikely(p->state != STATE_IOREQ_NONE) )
- {
- gprintk(XENLOG_ERR, "device model set bad IO state %d\n",
- p->state);
- break;
- }
-
- if ( unlikely(p->vp_eport != port) )
- {
- gprintk(XENLOG_ERR, "device model set bad event channel %d\n",
- p->vp_eport);
- break;
- }
-
- proto_p->state = STATE_IOREQ_NONE;
- proto_p->vp_eport = port;
- *p = *proto_p;
-
- prepare_wait_on_xen_event_channel(port);
-
- /*
- * Following happens /after/ blocking and setting up ioreq
- * contents. prepare_wait_on_xen_event_channel() is an implicit
- * barrier.
- */
- p->state = STATE_IOREQ_READY;
- notify_via_xen_event_channel(d, port);
-
- sv->pending = true;
- return IOREQ_STATUS_RETRY;
- }
- }
-
- return IOREQ_STATUS_UNHANDLED;
-}
-
-unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered)
-{
- struct domain *d = current->domain;
- struct hvm_ioreq_server *s;
- unsigned int id, failed = 0;
-
- FOR_EACH_IOREQ_SERVER(d, id, s)
- {
- if ( !s->enabled )
- continue;
-
- if ( hvm_send_ioreq(s, p, buffered) == IOREQ_STATUS_UNHANDLED )
- failed++;
- }
-
- return failed;
-}
-
-void hvm_ioreq_init(struct domain *d)
-{
- spin_lock_init(&d->arch.hvm.ioreq_server.lock);
-
- arch_hvm_ioreq_init(d);
-}
-
-/*
- * Local variables:
- * mode: C
- * c-file-style: "BSD"
- * c-basic-offset: 4
- * tab-width: 4
- * indent-tabs-mode: nil
- * End:
- */
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 8c8f054..b5865ae 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -100,6 +100,7 @@
*/
#include <xen/init.h>
+#include <xen/ioreq.h>
#include <xen/kernel.h>
#include <xen/lib.h>
#include <xen/mm.h>
@@ -141,7 +142,6 @@
#include <asm/io_apic.h>
#include <asm/pci.h>
#include <asm/guest.h>
-#include <asm/hvm/ioreq.h>
#include <asm/hvm/grant_table.h>
#include <asm/pv/domain.h>
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index 6182313..3e6c14d 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -20,6 +20,7 @@
* along with this program; If not, see <http://www.gnu.org/licenses/>.
*/
+#include <xen/ioreq.h>
#include <xen/types.h>
#include <xen/mm.h>
#include <xen/trace.h>
@@ -34,7 +35,6 @@
#include <asm/current.h>
#include <asm/flushtlb.h>
#include <asm/shadow.h>
-#include <asm/hvm/ioreq.h>
#include <xen/numa.h>
#include "private.h"
diff --git a/xen/common/Kconfig b/xen/common/Kconfig
index 3e2cf25..c971ded 100644
--- a/xen/common/Kconfig
+++ b/xen/common/Kconfig
@@ -139,6 +139,9 @@ config HYPFS_CONFIG
Disable this option in case you want to spare some memory or you
want to hide the .config contents from dom0.
+config IOREQ_SERVER
+ bool
+
config KEXEC
bool "kexec support"
default y
diff --git a/xen/common/Makefile b/xen/common/Makefile
index b3b60a1..cdb99fb 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -15,6 +15,7 @@ obj-$(CONFIG_GRANT_TABLE) += grant_table.o
obj-y += guestcopy.o
obj-bin-y += gunzip.init.o
obj-$(CONFIG_HYPFS) += hypfs.o
+obj-$(CONFIG_IOREQ_SERVER) += ioreq.o
obj-y += irq.o
obj-y += kernel.o
obj-y += keyhandler.o
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
new file mode 100644
index 0000000..d3433d7
--- /dev/null
+++ b/xen/common/ioreq.c
@@ -0,0 +1,1422 @@
+/*
+ * ioreq.c: hardware virtual machine I/O emulation
+ *
+ * Copyright (c) 2016 Citrix Systems Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/ctype.h>
+#include <xen/domain.h>
+#include <xen/event.h>
+#include <xen/init.h>
+#include <xen/irq.h>
+#include <xen/lib.h>
+#include <xen/paging.h>
+#include <xen/sched.h>
+#include <xen/softirq.h>
+#include <xen/trace.h>
+#include <xen/vpci.h>
+
+#include <asm/hvm/ioreq.h>
+
+#include <public/hvm/ioreq.h>
+#include <public/hvm/params.h>
+
+static void set_ioreq_server(struct domain *d, unsigned int id,
+ struct hvm_ioreq_server *s)
+{
+ ASSERT(id < MAX_NR_IOREQ_SERVERS);
+ ASSERT(!s || !d->arch.hvm.ioreq_server.server[id]);
+
+ d->arch.hvm.ioreq_server.server[id] = s;
+}
+
+#define GET_IOREQ_SERVER(d, id) \
+ (d)->arch.hvm.ioreq_server.server[id]
+
+struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
+ unsigned int id)
+{
+ if ( id >= MAX_NR_IOREQ_SERVERS )
+ return NULL;
+
+ return GET_IOREQ_SERVER(d, id);
+}
+
+/*
+ * Iterate over all possible ioreq servers.
+ *
+ * NOTE: The iteration is backwards such that more recently created
+ * ioreq servers are favoured in hvm_select_ioreq_server().
+ * This is a semantic that previously existed when ioreq servers
+ * were held in a linked list.
+ */
+#define FOR_EACH_IOREQ_SERVER(d, id, s) \
+ for ( (id) = MAX_NR_IOREQ_SERVERS; (id) != 0; ) \
+ if ( !(s = GET_IOREQ_SERVER(d, --(id))) ) \
+ continue; \
+ else
+
+static ioreq_t *get_ioreq(struct hvm_ioreq_server *s, struct vcpu *v)
+{
+ shared_iopage_t *p = s->ioreq.va;
+
+ ASSERT((v == current) || !vcpu_runnable(v));
+ ASSERT(p != NULL);
+
+ return &p->vcpu_ioreq[v->vcpu_id];
+}
+
+static struct hvm_ioreq_vcpu *get_pending_vcpu(const struct vcpu *v,
+ struct hvm_ioreq_server **srvp)
+{
+ struct domain *d = v->domain;
+ struct hvm_ioreq_server *s;
+ unsigned int id;
+
+ FOR_EACH_IOREQ_SERVER(d, id, s)
+ {
+ struct hvm_ioreq_vcpu *sv;
+
+ list_for_each_entry ( sv,
+ &s->ioreq_vcpu_list,
+ list_entry )
+ {
+ if ( sv->vcpu == v && sv->pending )
+ {
+ if ( srvp )
+ *srvp = s;
+ return sv;
+ }
+ }
+ }
+
+ return NULL;
+}
+
+bool hvm_io_pending(struct vcpu *v)
+{
+ return get_pending_vcpu(v, NULL);
+}
+
+static bool hvm_wait_for_io(struct hvm_ioreq_vcpu *sv, ioreq_t *p)
+{
+ unsigned int prev_state = STATE_IOREQ_NONE;
+ unsigned int state = p->state;
+ uint64_t data = ~0;
+
+ smp_rmb();
+
+ /*
+ * The only reason we should see this condition be false is when an
+ * emulator dying races with I/O being requested.
+ */
+ while ( likely(state != STATE_IOREQ_NONE) )
+ {
+ if ( unlikely(state < prev_state) )
+ {
+ gdprintk(XENLOG_ERR, "Weird HVM ioreq state transition %u -> %u\n",
+ prev_state, state);
+ sv->pending = false;
+ domain_crash(sv->vcpu->domain);
+ return false; /* bail */
+ }
+
+ switch ( prev_state = state )
+ {
+ case STATE_IORESP_READY: /* IORESP_READY -> NONE */
+ p->state = STATE_IOREQ_NONE;
+ data = p->data;
+ break;
+
+ case STATE_IOREQ_READY: /* IOREQ_{READY,INPROCESS} -> IORESP_READY */
+ case STATE_IOREQ_INPROCESS:
+ wait_on_xen_event_channel(sv->ioreq_evtchn,
+ ({ state = p->state;
+ smp_rmb();
+ state != prev_state; }));
+ continue;
+
+ default:
+ gdprintk(XENLOG_ERR, "Weird HVM iorequest state %u\n", state);
+ sv->pending = false;
+ domain_crash(sv->vcpu->domain);
+ return false; /* bail */
+ }
+
+ break;
+ }
+
+ p = &sv->vcpu->arch.hvm.hvm_io.io_req;
+ if ( hvm_ioreq_needs_completion(p) )
+ p->data = data;
+
+ sv->pending = false;
+
+ return true;
+}
+
+bool handle_hvm_io_completion(struct vcpu *v)
+{
+ struct domain *d = v->domain;
+ struct hvm_vcpu_io *vio = &v->arch.hvm.hvm_io;
+ struct hvm_ioreq_server *s;
+ struct hvm_ioreq_vcpu *sv;
+ enum hvm_io_completion io_completion;
+
+ if ( has_vpci(d) && vpci_process_pending(v) )
+ {
+ raise_softirq(SCHEDULE_SOFTIRQ);
+ return false;
+ }
+
+ sv = get_pending_vcpu(v, &s);
+ if ( sv && !hvm_wait_for_io(sv, get_ioreq(s, v)) )
+ return false;
+
+ vio->io_req.state = hvm_ioreq_needs_completion(&vio->io_req) ?
+ STATE_IORESP_READY : STATE_IOREQ_NONE;
+
+ msix_write_completion(v);
+ vcpu_end_shutdown_deferral(v);
+
+ io_completion = vio->io_completion;
+ vio->io_completion = HVMIO_no_completion;
+
+ switch ( io_completion )
+ {
+ case HVMIO_no_completion:
+ break;
+
+ case HVMIO_mmio_completion:
+ return handle_mmio();
+
+ case HVMIO_pio_completion:
+ return handle_pio(vio->io_req.addr, vio->io_req.size,
+ vio->io_req.dir);
+
+ default:
+ return arch_hvm_io_completion(io_completion);
+ }
+
+ return true;
+}
+
+static gfn_t hvm_alloc_legacy_ioreq_gfn(struct hvm_ioreq_server *s)
+{
+ struct domain *d = s->target;
+ unsigned int i;
+
+ BUILD_BUG_ON(HVM_PARAM_BUFIOREQ_PFN != HVM_PARAM_IOREQ_PFN + 1);
+
+ for ( i = HVM_PARAM_IOREQ_PFN; i <= HVM_PARAM_BUFIOREQ_PFN; i++ )
+ {
+ if ( !test_and_clear_bit(i, &d->arch.hvm.ioreq_gfn.legacy_mask) )
+ return _gfn(d->arch.hvm.params[i]);
+ }
+
+ return INVALID_GFN;
+}
+
+static gfn_t hvm_alloc_ioreq_gfn(struct hvm_ioreq_server *s)
+{
+ struct domain *d = s->target;
+ unsigned int i;
+
+ for ( i = 0; i < sizeof(d->arch.hvm.ioreq_gfn.mask) * 8; i++ )
+ {
+ if ( test_and_clear_bit(i, &d->arch.hvm.ioreq_gfn.mask) )
+ return _gfn(d->arch.hvm.ioreq_gfn.base + i);
+ }
+
+ /*
+ * If we are out of 'normal' GFNs then we may still have a 'legacy'
+ * GFN available.
+ */
+ return hvm_alloc_legacy_ioreq_gfn(s);
+}
+
+static bool hvm_free_legacy_ioreq_gfn(struct hvm_ioreq_server *s,
+ gfn_t gfn)
+{
+ struct domain *d = s->target;
+ unsigned int i;
+
+ for ( i = HVM_PARAM_IOREQ_PFN; i <= HVM_PARAM_BUFIOREQ_PFN; i++ )
+ {
+ if ( gfn_eq(gfn, _gfn(d->arch.hvm.params[i])) )
+ break;
+ }
+ if ( i > HVM_PARAM_BUFIOREQ_PFN )
+ return false;
+
+ set_bit(i, &d->arch.hvm.ioreq_gfn.legacy_mask);
+ return true;
+}
+
+static void hvm_free_ioreq_gfn(struct hvm_ioreq_server *s, gfn_t gfn)
+{
+ struct domain *d = s->target;
+ unsigned int i = gfn_x(gfn) - d->arch.hvm.ioreq_gfn.base;
+
+ ASSERT(!gfn_eq(gfn, INVALID_GFN));
+
+ if ( !hvm_free_legacy_ioreq_gfn(s, gfn) )
+ {
+ ASSERT(i < sizeof(d->arch.hvm.ioreq_gfn.mask) * 8);
+ set_bit(i, &d->arch.hvm.ioreq_gfn.mask);
+ }
+}
+
+static void hvm_unmap_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
+{
+ struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+
+ if ( gfn_eq(iorp->gfn, INVALID_GFN) )
+ return;
+
+ destroy_ring_for_helper(&iorp->va, iorp->page);
+ iorp->page = NULL;
+
+ hvm_free_ioreq_gfn(s, iorp->gfn);
+ iorp->gfn = INVALID_GFN;
+}
+
+static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
+{
+ struct domain *d = s->target;
+ struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+ int rc;
+
+ if ( iorp->page )
+ {
+ /*
+ * If a page has already been allocated (which will happen on
+ * demand if hvm_get_ioreq_server_frame() is called), then
+ * mapping a guest frame is not permitted.
+ */
+ if ( gfn_eq(iorp->gfn, INVALID_GFN) )
+ return -EPERM;
+
+ return 0;
+ }
+
+ if ( d->is_dying )
+ return -EINVAL;
+
+ iorp->gfn = hvm_alloc_ioreq_gfn(s);
+
+ if ( gfn_eq(iorp->gfn, INVALID_GFN) )
+ return -ENOMEM;
+
+ rc = prepare_ring_for_helper(d, gfn_x(iorp->gfn), &iorp->page,
+ &iorp->va);
+
+ if ( rc )
+ hvm_unmap_ioreq_gfn(s, buf);
+
+ return rc;
+}
+
+static int hvm_alloc_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
+{
+ struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+ struct page_info *page;
+
+ if ( iorp->page )
+ {
+ /*
+ * If a guest frame has already been mapped (which may happen
+ * on demand if hvm_get_ioreq_server_info() is called), then
+ * allocating a page is not permitted.
+ */
+ if ( !gfn_eq(iorp->gfn, INVALID_GFN) )
+ return -EPERM;
+
+ return 0;
+ }
+
+ page = alloc_domheap_page(s->target, MEMF_no_refcount);
+
+ if ( !page )
+ return -ENOMEM;
+
+ if ( !get_page_and_type(page, s->target, PGT_writable_page) )
+ {
+ /*
+ * The domain can't possibly know about this page yet, so failure
+ * here is a clear indication of something fishy going on.
+ */
+ domain_crash(s->emulator);
+ return -ENODATA;
+ }
+
+ iorp->va = __map_domain_page_global(page);
+ if ( !iorp->va )
+ goto fail;
+
+ iorp->page = page;
+ clear_page(iorp->va);
+ return 0;
+
+ fail:
+ put_page_alloc_ref(page);
+ put_page_and_type(page);
+
+ return -ENOMEM;
+}
+
+static void hvm_free_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
+{
+ struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+ struct page_info *page = iorp->page;
+
+ if ( !page )
+ return;
+
+ iorp->page = NULL;
+
+ unmap_domain_page_global(iorp->va);
+ iorp->va = NULL;
+
+ put_page_alloc_ref(page);
+ put_page_and_type(page);
+}
+
+bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
+{
+ const struct hvm_ioreq_server *s;
+ unsigned int id;
+ bool found = false;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ FOR_EACH_IOREQ_SERVER(d, id, s)
+ {
+ if ( (s->ioreq.page == page) || (s->bufioreq.page == page) )
+ {
+ found = true;
+ break;
+ }
+ }
+
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ return found;
+}
+
+static void hvm_remove_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
+
+{
+ struct domain *d = s->target;
+ struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+
+ if ( gfn_eq(iorp->gfn, INVALID_GFN) )
+ return;
+
+ if ( guest_physmap_remove_page(d, iorp->gfn,
+ page_to_mfn(iorp->page), 0) )
+ domain_crash(d);
+ clear_page(iorp->va);
+}
+
+static int hvm_add_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
+{
+ struct domain *d = s->target;
+ struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+ int rc;
+
+ if ( gfn_eq(iorp->gfn, INVALID_GFN) )
+ return 0;
+
+ clear_page(iorp->va);
+
+ rc = guest_physmap_add_page(d, iorp->gfn,
+ page_to_mfn(iorp->page), 0);
+ if ( rc == 0 )
+ paging_mark_pfn_dirty(d, _pfn(gfn_x(iorp->gfn)));
+
+ return rc;
+}
+
+static void hvm_update_ioreq_evtchn(struct hvm_ioreq_server *s,
+ struct hvm_ioreq_vcpu *sv)
+{
+ ASSERT(spin_is_locked(&s->lock));
+
+ if ( s->ioreq.va != NULL )
+ {
+ ioreq_t *p = get_ioreq(s, sv->vcpu);
+
+ p->vp_eport = sv->ioreq_evtchn;
+ }
+}
+
+#define HANDLE_BUFIOREQ(s) \
+ ((s)->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF)
+
+static int hvm_ioreq_server_add_vcpu(struct hvm_ioreq_server *s,
+ struct vcpu *v)
+{
+ struct hvm_ioreq_vcpu *sv;
+ int rc;
+
+ sv = xzalloc(struct hvm_ioreq_vcpu);
+
+ rc = -ENOMEM;
+ if ( !sv )
+ goto fail1;
+
+ spin_lock(&s->lock);
+
+ rc = alloc_unbound_xen_event_channel(v->domain, v->vcpu_id,
+ s->emulator->domain_id, NULL);
+ if ( rc < 0 )
+ goto fail2;
+
+ sv->ioreq_evtchn = rc;
+
+ if ( v->vcpu_id == 0 && HANDLE_BUFIOREQ(s) )
+ {
+ rc = alloc_unbound_xen_event_channel(v->domain, 0,
+ s->emulator->domain_id, NULL);
+ if ( rc < 0 )
+ goto fail3;
+
+ s->bufioreq_evtchn = rc;
+ }
+
+ sv->vcpu = v;
+
+ list_add(&sv->list_entry, &s->ioreq_vcpu_list);
+
+ if ( s->enabled )
+ hvm_update_ioreq_evtchn(s, sv);
+
+ spin_unlock(&s->lock);
+ return 0;
+
+ fail3:
+ free_xen_event_channel(v->domain, sv->ioreq_evtchn);
+
+ fail2:
+ spin_unlock(&s->lock);
+ xfree(sv);
+
+ fail1:
+ return rc;
+}
+
+static void hvm_ioreq_server_remove_vcpu(struct hvm_ioreq_server *s,
+ struct vcpu *v)
+{
+ struct hvm_ioreq_vcpu *sv;
+
+ spin_lock(&s->lock);
+
+ list_for_each_entry ( sv,
+ &s->ioreq_vcpu_list,
+ list_entry )
+ {
+ if ( sv->vcpu != v )
+ continue;
+
+ list_del(&sv->list_entry);
+
+ if ( v->vcpu_id == 0 && HANDLE_BUFIOREQ(s) )
+ free_xen_event_channel(v->domain, s->bufioreq_evtchn);
+
+ free_xen_event_channel(v->domain, sv->ioreq_evtchn);
+
+ xfree(sv);
+ break;
+ }
+
+ spin_unlock(&s->lock);
+}
+
+static void hvm_ioreq_server_remove_all_vcpus(struct hvm_ioreq_server *s)
+{
+ struct hvm_ioreq_vcpu *sv, *next;
+
+ spin_lock(&s->lock);
+
+ list_for_each_entry_safe ( sv,
+ next,
+ &s->ioreq_vcpu_list,
+ list_entry )
+ {
+ struct vcpu *v = sv->vcpu;
+
+ list_del(&sv->list_entry);
+
+ if ( v->vcpu_id == 0 && HANDLE_BUFIOREQ(s) )
+ free_xen_event_channel(v->domain, s->bufioreq_evtchn);
+
+ free_xen_event_channel(v->domain, sv->ioreq_evtchn);
+
+ xfree(sv);
+ }
+
+ spin_unlock(&s->lock);
+}
+
+static int hvm_ioreq_server_map_pages(struct hvm_ioreq_server *s)
+{
+ int rc;
+
+ rc = hvm_map_ioreq_gfn(s, false);
+
+ if ( !rc && HANDLE_BUFIOREQ(s) )
+ rc = hvm_map_ioreq_gfn(s, true);
+
+ if ( rc )
+ hvm_unmap_ioreq_gfn(s, false);
+
+ return rc;
+}
+
+static void hvm_ioreq_server_unmap_pages(struct hvm_ioreq_server *s)
+{
+ hvm_unmap_ioreq_gfn(s, true);
+ hvm_unmap_ioreq_gfn(s, false);
+}
+
+static int hvm_ioreq_server_alloc_pages(struct hvm_ioreq_server *s)
+{
+ int rc;
+
+ rc = hvm_alloc_ioreq_mfn(s, false);
+
+ if ( !rc && (s->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF) )
+ rc = hvm_alloc_ioreq_mfn(s, true);
+
+ if ( rc )
+ hvm_free_ioreq_mfn(s, false);
+
+ return rc;
+}
+
+static void hvm_ioreq_server_free_pages(struct hvm_ioreq_server *s)
+{
+ hvm_free_ioreq_mfn(s, true);
+ hvm_free_ioreq_mfn(s, false);
+}
+
+static void hvm_ioreq_server_free_rangesets(struct hvm_ioreq_server *s)
+{
+ unsigned int i;
+
+ for ( i = 0; i < NR_IO_RANGE_TYPES; i++ )
+ rangeset_destroy(s->range[i]);
+}
+
+static int hvm_ioreq_server_alloc_rangesets(struct hvm_ioreq_server *s,
+ ioservid_t id)
+{
+ unsigned int i;
+ int rc;
+
+ for ( i = 0; i < NR_IO_RANGE_TYPES; i++ )
+ {
+ char *name;
+
+ rc = asprintf(&name, "ioreq_server %d %s", id,
+ (i == XEN_DMOP_IO_RANGE_PORT) ? "port" :
+ (i == XEN_DMOP_IO_RANGE_MEMORY) ? "memory" :
+ (i == XEN_DMOP_IO_RANGE_PCI) ? "pci" :
+ "");
+ if ( rc )
+ goto fail;
+
+ s->range[i] = rangeset_new(s->target, name,
+ RANGESETF_prettyprint_hex);
+
+ xfree(name);
+
+ rc = -ENOMEM;
+ if ( !s->range[i] )
+ goto fail;
+
+ rangeset_limit(s->range[i], MAX_NR_IO_RANGES);
+ }
+
+ return 0;
+
+ fail:
+ hvm_ioreq_server_free_rangesets(s);
+
+ return rc;
+}
+
+static void hvm_ioreq_server_enable(struct hvm_ioreq_server *s)
+{
+ struct hvm_ioreq_vcpu *sv;
+
+ spin_lock(&s->lock);
+
+ if ( s->enabled )
+ goto done;
+
+ hvm_remove_ioreq_gfn(s, false);
+ hvm_remove_ioreq_gfn(s, true);
+
+ s->enabled = true;
+
+ list_for_each_entry ( sv,
+ &s->ioreq_vcpu_list,
+ list_entry )
+ hvm_update_ioreq_evtchn(s, sv);
+
+ done:
+ spin_unlock(&s->lock);
+}
+
+static void hvm_ioreq_server_disable(struct hvm_ioreq_server *s)
+{
+ spin_lock(&s->lock);
+
+ if ( !s->enabled )
+ goto done;
+
+ hvm_add_ioreq_gfn(s, true);
+ hvm_add_ioreq_gfn(s, false);
+
+ s->enabled = false;
+
+ done:
+ spin_unlock(&s->lock);
+}
+
+static int hvm_ioreq_server_init(struct hvm_ioreq_server *s,
+ struct domain *d, int bufioreq_handling,
+ ioservid_t id)
+{
+ struct domain *currd = current->domain;
+ struct vcpu *v;
+ int rc;
+
+ s->target = d;
+
+ get_knownalive_domain(currd);
+ s->emulator = currd;
+
+ spin_lock_init(&s->lock);
+ INIT_LIST_HEAD(&s->ioreq_vcpu_list);
+ spin_lock_init(&s->bufioreq_lock);
+
+ s->ioreq.gfn = INVALID_GFN;
+ s->bufioreq.gfn = INVALID_GFN;
+
+ rc = hvm_ioreq_server_alloc_rangesets(s, id);
+ if ( rc )
+ return rc;
+
+ s->bufioreq_handling = bufioreq_handling;
+
+ for_each_vcpu ( d, v )
+ {
+ rc = hvm_ioreq_server_add_vcpu(s, v);
+ if ( rc )
+ goto fail_add;
+ }
+
+ return 0;
+
+ fail_add:
+ hvm_ioreq_server_remove_all_vcpus(s);
+ hvm_ioreq_server_unmap_pages(s);
+
+ hvm_ioreq_server_free_rangesets(s);
+
+ put_domain(s->emulator);
+ return rc;
+}
+
+static void hvm_ioreq_server_deinit(struct hvm_ioreq_server *s)
+{
+ ASSERT(!s->enabled);
+ hvm_ioreq_server_remove_all_vcpus(s);
+
+ /*
+ * NOTE: It is safe to call both hvm_ioreq_server_unmap_pages() and
+ * hvm_ioreq_server_free_pages() in that order.
+ * This is because the former will do nothing if the pages
+ * are not mapped, leaving the page to be freed by the latter.
+ * However if the pages are mapped then the former will set
+ * the page_info pointer to NULL, meaning the latter will do
+ * nothing.
+ */
+ hvm_ioreq_server_unmap_pages(s);
+ hvm_ioreq_server_free_pages(s);
+
+ hvm_ioreq_server_free_rangesets(s);
+
+ put_domain(s->emulator);
+}
+
+int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
+ ioservid_t *id)
+{
+ struct hvm_ioreq_server *s;
+ unsigned int i;
+ int rc;
+
+ if ( bufioreq_handling > HVM_IOREQSRV_BUFIOREQ_ATOMIC )
+ return -EINVAL;
+
+ s = xzalloc(struct hvm_ioreq_server);
+ if ( !s )
+ return -ENOMEM;
+
+ domain_pause(d);
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ for ( i = 0; i < MAX_NR_IOREQ_SERVERS; i++ )
+ {
+ if ( !GET_IOREQ_SERVER(d, i) )
+ break;
+ }
+
+ rc = -ENOSPC;
+ if ( i >= MAX_NR_IOREQ_SERVERS )
+ goto fail;
+
+ /*
+ * It is safe to call set_ioreq_server() prior to
+ * hvm_ioreq_server_init() since the target domain is paused.
+ */
+ set_ioreq_server(d, i, s);
+
+ rc = hvm_ioreq_server_init(s, d, bufioreq_handling, i);
+ if ( rc )
+ {
+ set_ioreq_server(d, i, NULL);
+ goto fail;
+ }
+
+ if ( id )
+ *id = i;
+
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ domain_unpause(d);
+
+ return 0;
+
+ fail:
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ domain_unpause(d);
+
+ xfree(s);
+ return rc;
+}
+
+int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
+{
+ struct hvm_ioreq_server *s;
+ int rc;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ s = get_ioreq_server(d, id);
+
+ rc = -ENOENT;
+ if ( !s )
+ goto out;
+
+ rc = -EPERM;
+ if ( s->emulator != current->domain )
+ goto out;
+
+ domain_pause(d);
+
+ arch_hvm_destroy_ioreq_server(s);
+
+ hvm_ioreq_server_disable(s);
+
+ /*
+ * It is safe to call hvm_ioreq_server_deinit() prior to
+ * set_ioreq_server() since the target domain is paused.
+ */
+ hvm_ioreq_server_deinit(s);
+ set_ioreq_server(d, id, NULL);
+
+ domain_unpause(d);
+
+ xfree(s);
+
+ rc = 0;
+
+ out:
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ return rc;
+}
+
+int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
+ unsigned long *ioreq_gfn,
+ unsigned long *bufioreq_gfn,
+ evtchn_port_t *bufioreq_port)
+{
+ struct hvm_ioreq_server *s;
+ int rc;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ s = get_ioreq_server(d, id);
+
+ rc = -ENOENT;
+ if ( !s )
+ goto out;
+
+ rc = -EPERM;
+ if ( s->emulator != current->domain )
+ goto out;
+
+ if ( ioreq_gfn || bufioreq_gfn )
+ {
+ rc = hvm_ioreq_server_map_pages(s);
+ if ( rc )
+ goto out;
+ }
+
+ if ( ioreq_gfn )
+ *ioreq_gfn = gfn_x(s->ioreq.gfn);
+
+ if ( HANDLE_BUFIOREQ(s) )
+ {
+ if ( bufioreq_gfn )
+ *bufioreq_gfn = gfn_x(s->bufioreq.gfn);
+
+ if ( bufioreq_port )
+ *bufioreq_port = s->bufioreq_evtchn;
+ }
+
+ rc = 0;
+
+ out:
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ return rc;
+}
+
+int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
+ unsigned long idx, mfn_t *mfn)
+{
+ struct hvm_ioreq_server *s;
+ int rc;
+
+ ASSERT(is_hvm_domain(d));
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ s = get_ioreq_server(d, id);
+
+ rc = -ENOENT;
+ if ( !s )
+ goto out;
+
+ rc = -EPERM;
+ if ( s->emulator != current->domain )
+ goto out;
+
+ rc = hvm_ioreq_server_alloc_pages(s);
+ if ( rc )
+ goto out;
+
+ switch ( idx )
+ {
+ case XENMEM_resource_ioreq_server_frame_bufioreq:
+ rc = -ENOENT;
+ if ( !HANDLE_BUFIOREQ(s) )
+ goto out;
+
+ *mfn = page_to_mfn(s->bufioreq.page);
+ rc = 0;
+ break;
+
+ case XENMEM_resource_ioreq_server_frame_ioreq(0):
+ *mfn = page_to_mfn(s->ioreq.page);
+ rc = 0;
+ break;
+
+ default:
+ rc = -EINVAL;
+ break;
+ }
+
+ out:
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ return rc;
+}
+
+int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
+ uint32_t type, uint64_t start,
+ uint64_t end)
+{
+ struct hvm_ioreq_server *s;
+ struct rangeset *r;
+ int rc;
+
+ if ( start > end )
+ return -EINVAL;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ s = get_ioreq_server(d, id);
+
+ rc = -ENOENT;
+ if ( !s )
+ goto out;
+
+ rc = -EPERM;
+ if ( s->emulator != current->domain )
+ goto out;
+
+ switch ( type )
+ {
+ case XEN_DMOP_IO_RANGE_PORT:
+ case XEN_DMOP_IO_RANGE_MEMORY:
+ case XEN_DMOP_IO_RANGE_PCI:
+ r = s->range[type];
+ break;
+
+ default:
+ r = NULL;
+ break;
+ }
+
+ rc = -EINVAL;
+ if ( !r )
+ goto out;
+
+ rc = -EEXIST;
+ if ( rangeset_overlaps_range(r, start, end) )
+ goto out;
+
+ rc = rangeset_add_range(r, start, end);
+
+ out:
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ return rc;
+}
+
+int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
+ uint32_t type, uint64_t start,
+ uint64_t end)
+{
+ struct hvm_ioreq_server *s;
+ struct rangeset *r;
+ int rc;
+
+ if ( start > end )
+ return -EINVAL;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ s = get_ioreq_server(d, id);
+
+ rc = -ENOENT;
+ if ( !s )
+ goto out;
+
+ rc = -EPERM;
+ if ( s->emulator != current->domain )
+ goto out;
+
+ switch ( type )
+ {
+ case XEN_DMOP_IO_RANGE_PORT:
+ case XEN_DMOP_IO_RANGE_MEMORY:
+ case XEN_DMOP_IO_RANGE_PCI:
+ r = s->range[type];
+ break;
+
+ default:
+ r = NULL;
+ break;
+ }
+
+ rc = -EINVAL;
+ if ( !r )
+ goto out;
+
+ rc = -ENOENT;
+ if ( !rangeset_contains_range(r, start, end) )
+ goto out;
+
+ rc = rangeset_remove_range(r, start, end);
+
+ out:
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ return rc;
+}
+
+int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
+ bool enabled)
+{
+ struct hvm_ioreq_server *s;
+ int rc;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ s = get_ioreq_server(d, id);
+
+ rc = -ENOENT;
+ if ( !s )
+ goto out;
+
+ rc = -EPERM;
+ if ( s->emulator != current->domain )
+ goto out;
+
+ domain_pause(d);
+
+ if ( enabled )
+ hvm_ioreq_server_enable(s);
+ else
+ hvm_ioreq_server_disable(s);
+
+ domain_unpause(d);
+
+ rc = 0;
+
+ out:
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ return rc;
+}
+
+int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
+{
+ struct hvm_ioreq_server *s;
+ unsigned int id;
+ int rc;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ FOR_EACH_IOREQ_SERVER(d, id, s)
+ {
+ rc = hvm_ioreq_server_add_vcpu(s, v);
+ if ( rc )
+ goto fail;
+ }
+
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ return 0;
+
+ fail:
+ while ( ++id != MAX_NR_IOREQ_SERVERS )
+ {
+ s = GET_IOREQ_SERVER(d, id);
+
+ if ( !s )
+ continue;
+
+ hvm_ioreq_server_remove_vcpu(s, v);
+ }
+
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ return rc;
+}
+
+void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v)
+{
+ struct hvm_ioreq_server *s;
+ unsigned int id;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ FOR_EACH_IOREQ_SERVER(d, id, s)
+ hvm_ioreq_server_remove_vcpu(s, v);
+
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+}
+
+void hvm_destroy_all_ioreq_servers(struct domain *d)
+{
+ struct hvm_ioreq_server *s;
+ unsigned int id;
+
+ if ( !arch_hvm_ioreq_destroy(d) )
+ return;
+
+ spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+
+ /* No need to domain_pause() as the domain is being torn down */
+
+ FOR_EACH_IOREQ_SERVER(d, id, s)
+ {
+ hvm_ioreq_server_disable(s);
+
+ /*
+ * It is safe to call hvm_ioreq_server_deinit() prior to
+ * set_ioreq_server() since the target domain is being destroyed.
+ */
+ hvm_ioreq_server_deinit(s);
+ set_ioreq_server(d, id, NULL);
+
+ xfree(s);
+ }
+
+ spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+}
+
+struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
+ ioreq_t *p)
+{
+ struct hvm_ioreq_server *s;
+ uint8_t type;
+ uint64_t addr;
+ unsigned int id;
+
+ if ( hvm_ioreq_server_get_type_addr(d, p, &type, &addr) )
+ return NULL;
+
+ FOR_EACH_IOREQ_SERVER(d, id, s)
+ {
+ struct rangeset *r;
+
+ if ( !s->enabled )
+ continue;
+
+ r = s->range[type];
+
+ switch ( type )
+ {
+ unsigned long start, end;
+
+ case XEN_DMOP_IO_RANGE_PORT:
+ start = addr;
+ end = start + p->size - 1;
+ if ( rangeset_contains_range(r, start, end) )
+ return s;
+
+ break;
+
+ case XEN_DMOP_IO_RANGE_MEMORY:
+ start = hvm_mmio_first_byte(p);
+ end = hvm_mmio_last_byte(p);
+
+ if ( rangeset_contains_range(r, start, end) )
+ return s;
+
+ break;
+
+ case XEN_DMOP_IO_RANGE_PCI:
+ if ( rangeset_contains_singleton(r, addr >> 32) )
+ {
+ p->type = IOREQ_TYPE_PCI_CONFIG;
+ p->addr = addr;
+ return s;
+ }
+
+ break;
+ }
+ }
+
+ return NULL;
+}
+
+static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
+{
+ struct domain *d = current->domain;
+ struct hvm_ioreq_page *iorp;
+ buffered_iopage_t *pg;
+ buf_ioreq_t bp = { .data = p->data,
+ .addr = p->addr,
+ .type = p->type,
+ .dir = p->dir };
+ /* Timeoffset sends 64b data, but no address. Use two consecutive slots. */
+ int qw = 0;
+
+ /* Ensure buffered_iopage fits in a page */
+ BUILD_BUG_ON(sizeof(buffered_iopage_t) > PAGE_SIZE);
+
+ iorp = &s->bufioreq;
+ pg = iorp->va;
+
+ if ( !pg )
+ return IOREQ_STATUS_UNHANDLED;
+
+ /*
+ * Return 0 for the cases we can't deal with:
+ * - 'addr' is only a 20-bit field, so we cannot address beyond 1MB
+ * - we cannot buffer accesses to guest memory buffers, as the guest
+ * may expect the memory buffer to be synchronously accessed
+ * - the count field is usually used with data_is_ptr and since we don't
+ * support data_is_ptr we do not waste space for the count field either
+ */
+ if ( (p->addr > 0xffffful) || p->data_is_ptr || (p->count != 1) )
+ return 0;
+
+ switch ( p->size )
+ {
+ case 1:
+ bp.size = 0;
+ break;
+ case 2:
+ bp.size = 1;
+ break;
+ case 4:
+ bp.size = 2;
+ break;
+ case 8:
+ bp.size = 3;
+ qw = 1;
+ break;
+ default:
+ gdprintk(XENLOG_WARNING, "unexpected ioreq size: %u\n", p->size);
+ return IOREQ_STATUS_UNHANDLED;
+ }
+
+ spin_lock(&s->bufioreq_lock);
+
+ if ( (pg->ptrs.write_pointer - pg->ptrs.read_pointer) >=
+ (IOREQ_BUFFER_SLOT_NUM - qw) )
+ {
+ /* The queue is full: send the iopacket through the normal path. */
+ spin_unlock(&s->bufioreq_lock);
+ return IOREQ_STATUS_UNHANDLED;
+ }
+
+ pg->buf_ioreq[pg->ptrs.write_pointer % IOREQ_BUFFER_SLOT_NUM] = bp;
+
+ if ( qw )
+ {
+ bp.data = p->data >> 32;
+ pg->buf_ioreq[(pg->ptrs.write_pointer+1) % IOREQ_BUFFER_SLOT_NUM] = bp;
+ }
+
+ /* Make the ioreq_t visible /before/ write_pointer. */
+ smp_wmb();
+ pg->ptrs.write_pointer += qw ? 2 : 1;
+
+ /* Canonicalize read/write pointers to prevent their overflow. */
+ while ( (s->bufioreq_handling == HVM_IOREQSRV_BUFIOREQ_ATOMIC) &&
+ qw++ < IOREQ_BUFFER_SLOT_NUM &&
+ pg->ptrs.read_pointer >= IOREQ_BUFFER_SLOT_NUM )
+ {
+ union bufioreq_pointers old = pg->ptrs, new;
+ unsigned int n = old.read_pointer / IOREQ_BUFFER_SLOT_NUM;
+
+ new.read_pointer = old.read_pointer - n * IOREQ_BUFFER_SLOT_NUM;
+ new.write_pointer = old.write_pointer - n * IOREQ_BUFFER_SLOT_NUM;
+ cmpxchg(&pg->ptrs.full, old.full, new.full);
+ }
+
+ notify_via_xen_event_channel(d, s->bufioreq_evtchn);
+ spin_unlock(&s->bufioreq_lock);
+
+ return IOREQ_STATUS_HANDLED;
+}
+
+int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
+ bool buffered)
+{
+ struct vcpu *curr = current;
+ struct domain *d = curr->domain;
+ struct hvm_ioreq_vcpu *sv;
+
+ ASSERT(s);
+
+ if ( buffered )
+ return hvm_send_buffered_ioreq(s, proto_p);
+
+ if ( unlikely(!vcpu_start_shutdown_deferral(curr)) )
+ return IOREQ_STATUS_RETRY;
+
+ list_for_each_entry ( sv,
+ &s->ioreq_vcpu_list,
+ list_entry )
+ {
+ if ( sv->vcpu == curr )
+ {
+ evtchn_port_t port = sv->ioreq_evtchn;
+ ioreq_t *p = get_ioreq(s, curr);
+
+ if ( unlikely(p->state != STATE_IOREQ_NONE) )
+ {
+ gprintk(XENLOG_ERR, "device model set bad IO state %d\n",
+ p->state);
+ break;
+ }
+
+ if ( unlikely(p->vp_eport != port) )
+ {
+ gprintk(XENLOG_ERR, "device model set bad event channel %d\n",
+ p->vp_eport);
+ break;
+ }
+
+ proto_p->state = STATE_IOREQ_NONE;
+ proto_p->vp_eport = port;
+ *p = *proto_p;
+
+ prepare_wait_on_xen_event_channel(port);
+
+ /*
+ * Following happens /after/ blocking and setting up ioreq
+ * contents. prepare_wait_on_xen_event_channel() is an implicit
+ * barrier.
+ */
+ p->state = STATE_IOREQ_READY;
+ notify_via_xen_event_channel(d, port);
+
+ sv->pending = true;
+ return IOREQ_STATUS_RETRY;
+ }
+ }
+
+ return IOREQ_STATUS_UNHANDLED;
+}
+
+unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered)
+{
+ struct domain *d = current->domain;
+ struct hvm_ioreq_server *s;
+ unsigned int id, failed = 0;
+
+ FOR_EACH_IOREQ_SERVER(d, id, s)
+ {
+ if ( !s->enabled )
+ continue;
+
+ if ( hvm_send_ioreq(s, p, buffered) == IOREQ_STATUS_UNHANDLED )
+ failed++;
+ }
+
+ return failed;
+}
+
+void hvm_ioreq_init(struct domain *d)
+{
+ spin_lock_init(&d->arch.hvm.ioreq_server.lock);
+
+ arch_hvm_ioreq_init(d);
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-x86/hvm/ioreq.h b/xen/include/asm-x86/hvm/ioreq.h
index 376e2ef..a3d8faa 100644
--- a/xen/include/asm-x86/hvm/ioreq.h
+++ b/xen/include/asm-x86/hvm/ioreq.h
@@ -19,14 +19,13 @@
#ifndef __ASM_X86_HVM_IOREQ_H__
#define __ASM_X86_HVM_IOREQ_H__
+#include <xen/ioreq.h>
+
#include <asm/hvm/emulate.h>
#include <asm/hvm/vmx/vmx.h>
#include <public/hvm/params.h>
-struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
- unsigned int id);
-
static inline bool arch_hvm_io_completion(enum hvm_io_completion io_completion)
{
switch ( io_completion )
@@ -178,40 +177,6 @@ static inline bool arch_hvm_ioreq_destroy(struct domain *d)
return true;
}
-bool hvm_io_pending(struct vcpu *v);
-bool handle_hvm_io_completion(struct vcpu *v);
-bool is_ioreq_server_page(struct domain *d, const struct page_info *page);
-
-int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
- ioservid_t *id);
-int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id);
-int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
- unsigned long *ioreq_gfn,
- unsigned long *bufioreq_gfn,
- evtchn_port_t *bufioreq_port);
-int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
- unsigned long idx, mfn_t *mfn);
-int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint64_t start,
- uint64_t end);
-int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint64_t start,
- uint64_t end);
-int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
- bool enabled);
-
-int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v);
-void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v);
-void hvm_destroy_all_ioreq_servers(struct domain *d);
-
-struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
- ioreq_t *p);
-int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
- bool buffered);
-unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered);
-
-void hvm_ioreq_init(struct domain *d);
-
#define IOREQ_STATUS_HANDLED X86EMUL_OKAY
#define IOREQ_STATUS_UNHANDLED X86EMUL_UNHANDLEABLE
#define IOREQ_STATUS_RETRY X86EMUL_RETRY
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
new file mode 100644
index 0000000..6db4392
--- /dev/null
+++ b/xen/include/xen/ioreq.h
@@ -0,0 +1,71 @@
+/*
+ * ioreq.h: Hardware virtual machine assist interface definitions.
+ *
+ * Copyright (c) 2016 Citrix Systems Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __XEN_IOREQ_H__
+#define __XEN_IOREQ_H__
+
+#include <xen/sched.h>
+
+struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
+ unsigned int id);
+
+bool hvm_io_pending(struct vcpu *v);
+bool handle_hvm_io_completion(struct vcpu *v);
+bool is_ioreq_server_page(struct domain *d, const struct page_info *page);
+
+int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
+ ioservid_t *id);
+int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id);
+int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
+ unsigned long *ioreq_gfn,
+ unsigned long *bufioreq_gfn,
+ evtchn_port_t *bufioreq_port);
+int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
+ unsigned long idx, mfn_t *mfn);
+int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
+ uint32_t type, uint64_t start,
+ uint64_t end);
+int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
+ uint32_t type, uint64_t start,
+ uint64_t end);
+int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
+ bool enabled);
+
+int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v);
+void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v);
+void hvm_destroy_all_ioreq_servers(struct domain *d);
+
+struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
+ ioreq_t *p);
+int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
+ bool buffered);
+unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered);
+
+void hvm_ioreq_init(struct domain *d);
+
+#endif /* __XEN_IOREQ_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 03/23] xen/ioreq: Make x86's hvm_ioreq_needs_completion() common
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
2020-10-15 16:44 ` [PATCH V2 01/23] x86/ioreq: Prepare IOREQ feature for making it common Oleksandr Tyshchenko
2020-10-15 16:44 ` [PATCH V2 02/23] xen/ioreq: Make x86's IOREQ feature common Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-10-20 7:59 ` Paul Durrant
2020-10-15 16:44 ` [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio() Oleksandr Tyshchenko
` (20 subsequent siblings)
23 siblings, 1 reply; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, Julien Grall, Stefano Stabellini, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
The IOREQ is a common feature now and this helper will be used
on Arm as is. Move it to xen/ioreq.h and remove "hvm" prefix.
Although PIO handling on Arm is not introduced with the current series
(it will be implemented when we add support for vPCI), technically
the PIOs exist on Arm (however they are accessed the same way as MMIO)
and it would be better not to diverge now.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- new patch, was split from:
"[RFC PATCH V1 01/12] hvm/ioreq: Make x86's IOREQ feature common"
Changes V1 -> V2:
- remove "hvm" prefix
---
xen/arch/x86/hvm/emulate.c | 4 ++--
xen/arch/x86/hvm/io.c | 2 +-
xen/common/ioreq.c | 4 ++--
xen/include/asm-x86/hvm/vcpu.h | 7 -------
xen/include/xen/ioreq.h | 7 +++++++
5 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index 24cf85f..5700274 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -336,7 +336,7 @@ static int hvmemul_do_io(
rc = hvm_send_ioreq(s, &p, 0);
if ( rc != X86EMUL_RETRY || currd->is_shutting_down )
vio->io_req.state = STATE_IOREQ_NONE;
- else if ( !hvm_ioreq_needs_completion(&vio->io_req) )
+ else if ( !ioreq_needs_completion(&vio->io_req) )
rc = X86EMUL_OKAY;
}
break;
@@ -2649,7 +2649,7 @@ static int _hvm_emulate_one(struct hvm_emulate_ctxt *hvmemul_ctxt,
if ( rc == X86EMUL_OKAY && vio->mmio_retry )
rc = X86EMUL_RETRY;
- if ( !hvm_ioreq_needs_completion(&vio->io_req) )
+ if ( !ioreq_needs_completion(&vio->io_req) )
completion = HVMIO_no_completion;
else if ( completion == HVMIO_no_completion )
completion = (vio->io_req.type != IOREQ_TYPE_PIO ||
diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
index 3e09d9b..b220d6b 100644
--- a/xen/arch/x86/hvm/io.c
+++ b/xen/arch/x86/hvm/io.c
@@ -135,7 +135,7 @@ bool handle_pio(uint16_t port, unsigned int size, int dir)
rc = hvmemul_do_pio_buffer(port, size, dir, &data);
- if ( hvm_ioreq_needs_completion(&vio->io_req) )
+ if ( ioreq_needs_completion(&vio->io_req) )
vio->io_completion = HVMIO_pio_completion;
switch ( rc )
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index d3433d7..c89df7a 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -159,7 +159,7 @@ static bool hvm_wait_for_io(struct hvm_ioreq_vcpu *sv, ioreq_t *p)
}
p = &sv->vcpu->arch.hvm.hvm_io.io_req;
- if ( hvm_ioreq_needs_completion(p) )
+ if ( ioreq_needs_completion(p) )
p->data = data;
sv->pending = false;
@@ -185,7 +185,7 @@ bool handle_hvm_io_completion(struct vcpu *v)
if ( sv && !hvm_wait_for_io(sv, get_ioreq(s, v)) )
return false;
- vio->io_req.state = hvm_ioreq_needs_completion(&vio->io_req) ?
+ vio->io_req.state = ioreq_needs_completion(&vio->io_req) ?
STATE_IORESP_READY : STATE_IOREQ_NONE;
msix_write_completion(v);
diff --git a/xen/include/asm-x86/hvm/vcpu.h b/xen/include/asm-x86/hvm/vcpu.h
index 5ccd075..6c1feda 100644
--- a/xen/include/asm-x86/hvm/vcpu.h
+++ b/xen/include/asm-x86/hvm/vcpu.h
@@ -91,13 +91,6 @@ struct hvm_vcpu_io {
const struct g2m_ioport *g2m_ioport;
};
-static inline bool hvm_ioreq_needs_completion(const ioreq_t *ioreq)
-{
- return ioreq->state == STATE_IOREQ_READY &&
- !ioreq->data_is_ptr &&
- (ioreq->type != IOREQ_TYPE_PIO || ioreq->dir != IOREQ_WRITE);
-}
-
struct nestedvcpu {
bool_t nv_guestmode; /* vcpu in guestmode? */
void *nv_vvmcx; /* l1 guest virtual VMCB/VMCS */
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
index 6db4392..8e1603c 100644
--- a/xen/include/xen/ioreq.h
+++ b/xen/include/xen/ioreq.h
@@ -24,6 +24,13 @@
struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
unsigned int id);
+static inline bool ioreq_needs_completion(const ioreq_t *ioreq)
+{
+ return ioreq->state == STATE_IOREQ_READY &&
+ !ioreq->data_is_ptr &&
+ (ioreq->type != IOREQ_TYPE_PIO || ioreq->dir != IOREQ_WRITE);
+}
+
bool hvm_io_pending(struct vcpu *v);
bool handle_hvm_io_completion(struct vcpu *v);
bool is_ioreq_server_page(struct domain *d, const struct page_info *page);
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio()
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (2 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 03/23] xen/ioreq: Make x86's hvm_ioreq_needs_completion() common Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-10-20 9:14 ` Paul Durrant
2020-10-15 16:44 ` [PATCH V2 05/23] xen/ioreq: Make x86's hvm_mmio_first(last)_byte() common Oleksandr Tyshchenko
` (19 subsequent siblings)
23 siblings, 1 reply; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, Julien Grall, Stefano Stabellini, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
The IOREQ is a common feature now and Arm will have its own
implementation.
But the name of the function is pretty generic and can be confusing
on Arm (we already have a try_handle_mmio()).
In order not to rename the function (which is used for a varying
set of purposes on x86) globally and get non-confusing variant on Arm
provide an alias ioreq_complete_mmio() to be used on common and
Arm code.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- new patch
Changes V1 -> V2:
- remove "handle"
- add Jan's A-b
---
xen/common/ioreq.c | 2 +-
xen/include/asm-x86/hvm/ioreq.h | 2 ++
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index c89df7a..29ad48e 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -200,7 +200,7 @@ bool handle_hvm_io_completion(struct vcpu *v)
break;
case HVMIO_mmio_completion:
- return handle_mmio();
+ return ioreq_complete_mmio();
case HVMIO_pio_completion:
return handle_pio(vio->io_req.addr, vio->io_req.size,
diff --git a/xen/include/asm-x86/hvm/ioreq.h b/xen/include/asm-x86/hvm/ioreq.h
index a3d8faa..a147856 100644
--- a/xen/include/asm-x86/hvm/ioreq.h
+++ b/xen/include/asm-x86/hvm/ioreq.h
@@ -181,6 +181,8 @@ static inline bool arch_hvm_ioreq_destroy(struct domain *d)
#define IOREQ_STATUS_UNHANDLED X86EMUL_UNHANDLEABLE
#define IOREQ_STATUS_RETRY X86EMUL_RETRY
+#define ioreq_complete_mmio handle_mmio
+
#endif /* __ASM_X86_HVM_IOREQ_H__ */
/*
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 05/23] xen/ioreq: Make x86's hvm_mmio_first(last)_byte() common
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (3 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio() Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-10-20 9:15 ` Paul Durrant
2020-10-15 16:44 ` [PATCH V2 06/23] xen/ioreq: Make x86's hvm_ioreq_(page/vcpu/server) structs common Oleksandr Tyshchenko
` (18 subsequent siblings)
23 siblings, 1 reply; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, Julien Grall, Stefano Stabellini, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
The IOREQ is a common feature now and these helpers will be used
on Arm as is. Move them to xen/ioreq.h and replace "hvm" prefixes
with "ioreq".
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- new patch
Changes V1 -> V2:
- replace "hvm" prefix by "ioreq"
---
xen/arch/x86/hvm/intercept.c | 5 +++--
xen/arch/x86/hvm/stdvga.c | 4 ++--
xen/common/ioreq.c | 4 ++--
xen/include/asm-x86/hvm/io.h | 16 ----------------
xen/include/xen/ioreq.h | 16 ++++++++++++++++
5 files changed, 23 insertions(+), 22 deletions(-)
diff --git a/xen/arch/x86/hvm/intercept.c b/xen/arch/x86/hvm/intercept.c
index cd4c4c1..02ca3b0 100644
--- a/xen/arch/x86/hvm/intercept.c
+++ b/xen/arch/x86/hvm/intercept.c
@@ -17,6 +17,7 @@
* this program; If not, see <http://www.gnu.org/licenses/>.
*/
+#include <xen/ioreq.h>
#include <xen/types.h>
#include <xen/sched.h>
#include <asm/regs.h>
@@ -34,7 +35,7 @@
static bool_t hvm_mmio_accept(const struct hvm_io_handler *handler,
const ioreq_t *p)
{
- paddr_t first = hvm_mmio_first_byte(p), last;
+ paddr_t first = ioreq_mmio_first_byte(p), last;
BUG_ON(handler->type != IOREQ_TYPE_COPY);
@@ -42,7 +43,7 @@ static bool_t hvm_mmio_accept(const struct hvm_io_handler *handler,
return 0;
/* Make sure the handler will accept the whole access. */
- last = hvm_mmio_last_byte(p);
+ last = ioreq_mmio_last_byte(p);
if ( last != first &&
!handler->mmio.ops->check(current, last) )
domain_crash(current->domain);
diff --git a/xen/arch/x86/hvm/stdvga.c b/xen/arch/x86/hvm/stdvga.c
index e267513..e184664 100644
--- a/xen/arch/x86/hvm/stdvga.c
+++ b/xen/arch/x86/hvm/stdvga.c
@@ -524,8 +524,8 @@ static bool_t stdvga_mem_accept(const struct hvm_io_handler *handler,
* deadlock when hvm_mmio_internal() is called from
* hvm_copy_to/from_guest_phys() in hvm_process_io_intercept().
*/
- if ( (hvm_mmio_first_byte(p) < VGA_MEM_BASE) ||
- (hvm_mmio_last_byte(p) >= (VGA_MEM_BASE + VGA_MEM_SIZE)) )
+ if ( (ioreq_mmio_first_byte(p) < VGA_MEM_BASE) ||
+ (ioreq_mmio_last_byte(p) >= (VGA_MEM_BASE + VGA_MEM_SIZE)) )
return 0;
spin_lock(&s->lock);
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index 29ad48e..5fa10b6 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -1210,8 +1210,8 @@ struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
break;
case XEN_DMOP_IO_RANGE_MEMORY:
- start = hvm_mmio_first_byte(p);
- end = hvm_mmio_last_byte(p);
+ start = ioreq_mmio_first_byte(p);
+ end = ioreq_mmio_last_byte(p);
if ( rangeset_contains_range(r, start, end) )
return s;
diff --git a/xen/include/asm-x86/hvm/io.h b/xen/include/asm-x86/hvm/io.h
index 558426b..fb64294 100644
--- a/xen/include/asm-x86/hvm/io.h
+++ b/xen/include/asm-x86/hvm/io.h
@@ -40,22 +40,6 @@ struct hvm_mmio_ops {
hvm_mmio_write_t write;
};
-static inline paddr_t hvm_mmio_first_byte(const ioreq_t *p)
-{
- return unlikely(p->df) ?
- p->addr - (p->count - 1ul) * p->size :
- p->addr;
-}
-
-static inline paddr_t hvm_mmio_last_byte(const ioreq_t *p)
-{
- unsigned long size = p->size;
-
- return unlikely(p->df) ?
- p->addr + size - 1:
- p->addr + (p->count * size) - 1;
-}
-
typedef int (*portio_action_t)(
int dir, unsigned int port, unsigned int bytes, uint32_t *val);
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
index 8e1603c..768ac94 100644
--- a/xen/include/xen/ioreq.h
+++ b/xen/include/xen/ioreq.h
@@ -24,6 +24,22 @@
struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
unsigned int id);
+static inline paddr_t ioreq_mmio_first_byte(const ioreq_t *p)
+{
+ return unlikely(p->df) ?
+ p->addr - (p->count - 1ul) * p->size :
+ p->addr;
+}
+
+static inline paddr_t ioreq_mmio_last_byte(const ioreq_t *p)
+{
+ unsigned long size = p->size;
+
+ return unlikely(p->df) ?
+ p->addr + size - 1:
+ p->addr + (p->count * size) - 1;
+}
+
static inline bool ioreq_needs_completion(const ioreq_t *ioreq)
{
return ioreq->state == STATE_IOREQ_READY &&
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 06/23] xen/ioreq: Make x86's hvm_ioreq_(page/vcpu/server) structs common
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (4 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 05/23] xen/ioreq: Make x86's hvm_mmio_first(last)_byte() common Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-10-15 16:44 ` [PATCH V2 07/23] xen/ioreq: Move x86's ioreq_gfn(server) to struct domain Oleksandr Tyshchenko
` (17 subsequent siblings)
23 siblings, 0 replies; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Julien Grall, Stefano Stabellini,
Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
The IOREQ is a common feature now and these structs will be used
on Arm as is. Move them to xen/ioreq.h and remove "hvm" prefixes.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- new patch
Changes V1 -> V2:
- remove "hvm" prefix
---
xen/arch/x86/hvm/emulate.c | 2 +-
xen/arch/x86/hvm/stdvga.c | 2 +-
xen/arch/x86/mm/p2m.c | 8 +--
xen/common/ioreq.c | 134 +++++++++++++++++++--------------------
xen/include/asm-x86/hvm/domain.h | 36 +----------
xen/include/asm-x86/hvm/ioreq.h | 4 +-
xen/include/asm-x86/p2m.h | 8 +--
xen/include/xen/ioreq.h | 44 +++++++++++--
8 files changed, 119 insertions(+), 119 deletions(-)
diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index 5700274..4746d5a 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -287,7 +287,7 @@ static int hvmemul_do_io(
* However, there's no cheap approach to avoid above situations in xen,
* so the device model side needs to check the incoming ioreq event.
*/
- struct hvm_ioreq_server *s = NULL;
+ struct ioreq_server *s = NULL;
p2m_type_t p2mt = p2m_invalid;
if ( is_mmio )
diff --git a/xen/arch/x86/hvm/stdvga.c b/xen/arch/x86/hvm/stdvga.c
index e184664..bafb3f6 100644
--- a/xen/arch/x86/hvm/stdvga.c
+++ b/xen/arch/x86/hvm/stdvga.c
@@ -466,7 +466,7 @@ static int stdvga_mem_write(const struct hvm_io_handler *handler,
.dir = IOREQ_WRITE,
.data = data,
};
- struct hvm_ioreq_server *srv;
+ struct ioreq_server *srv;
if ( !stdvga_cache_is_enabled(s) || !s->stdvga )
goto done;
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index 928344b..6102771 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -366,7 +366,7 @@ void p2m_memory_type_changed(struct domain *d)
int p2m_set_ioreq_server(struct domain *d,
unsigned int flags,
- struct hvm_ioreq_server *s)
+ struct ioreq_server *s)
{
struct p2m_domain *p2m = p2m_get_hostp2m(d);
int rc;
@@ -414,11 +414,11 @@ int p2m_set_ioreq_server(struct domain *d,
return rc;
}
-struct hvm_ioreq_server *p2m_get_ioreq_server(struct domain *d,
- unsigned int *flags)
+struct ioreq_server *p2m_get_ioreq_server(struct domain *d,
+ unsigned int *flags)
{
struct p2m_domain *p2m = p2m_get_hostp2m(d);
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
spin_lock(&p2m->ioreq.lock);
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index 5fa10b6..1d62d13 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -34,7 +34,7 @@
#include <public/hvm/params.h>
static void set_ioreq_server(struct domain *d, unsigned int id,
- struct hvm_ioreq_server *s)
+ struct ioreq_server *s)
{
ASSERT(id < MAX_NR_IOREQ_SERVERS);
ASSERT(!s || !d->arch.hvm.ioreq_server.server[id]);
@@ -45,8 +45,8 @@ static void set_ioreq_server(struct domain *d, unsigned int id,
#define GET_IOREQ_SERVER(d, id) \
(d)->arch.hvm.ioreq_server.server[id]
-struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
- unsigned int id)
+struct ioreq_server *get_ioreq_server(const struct domain *d,
+ unsigned int id)
{
if ( id >= MAX_NR_IOREQ_SERVERS )
return NULL;
@@ -68,7 +68,7 @@ struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
continue; \
else
-static ioreq_t *get_ioreq(struct hvm_ioreq_server *s, struct vcpu *v)
+static ioreq_t *get_ioreq(struct ioreq_server *s, struct vcpu *v)
{
shared_iopage_t *p = s->ioreq.va;
@@ -78,16 +78,16 @@ static ioreq_t *get_ioreq(struct hvm_ioreq_server *s, struct vcpu *v)
return &p->vcpu_ioreq[v->vcpu_id];
}
-static struct hvm_ioreq_vcpu *get_pending_vcpu(const struct vcpu *v,
- struct hvm_ioreq_server **srvp)
+static struct ioreq_vcpu *get_pending_vcpu(const struct vcpu *v,
+ struct ioreq_server **srvp)
{
struct domain *d = v->domain;
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
unsigned int id;
FOR_EACH_IOREQ_SERVER(d, id, s)
{
- struct hvm_ioreq_vcpu *sv;
+ struct ioreq_vcpu *sv;
list_for_each_entry ( sv,
&s->ioreq_vcpu_list,
@@ -110,7 +110,7 @@ bool hvm_io_pending(struct vcpu *v)
return get_pending_vcpu(v, NULL);
}
-static bool hvm_wait_for_io(struct hvm_ioreq_vcpu *sv, ioreq_t *p)
+static bool hvm_wait_for_io(struct ioreq_vcpu *sv, ioreq_t *p)
{
unsigned int prev_state = STATE_IOREQ_NONE;
unsigned int state = p->state;
@@ -171,8 +171,8 @@ bool handle_hvm_io_completion(struct vcpu *v)
{
struct domain *d = v->domain;
struct hvm_vcpu_io *vio = &v->arch.hvm.hvm_io;
- struct hvm_ioreq_server *s;
- struct hvm_ioreq_vcpu *sv;
+ struct ioreq_server *s;
+ struct ioreq_vcpu *sv;
enum hvm_io_completion io_completion;
if ( has_vpci(d) && vpci_process_pending(v) )
@@ -213,7 +213,7 @@ bool handle_hvm_io_completion(struct vcpu *v)
return true;
}
-static gfn_t hvm_alloc_legacy_ioreq_gfn(struct hvm_ioreq_server *s)
+static gfn_t hvm_alloc_legacy_ioreq_gfn(struct ioreq_server *s)
{
struct domain *d = s->target;
unsigned int i;
@@ -229,7 +229,7 @@ static gfn_t hvm_alloc_legacy_ioreq_gfn(struct hvm_ioreq_server *s)
return INVALID_GFN;
}
-static gfn_t hvm_alloc_ioreq_gfn(struct hvm_ioreq_server *s)
+static gfn_t hvm_alloc_ioreq_gfn(struct ioreq_server *s)
{
struct domain *d = s->target;
unsigned int i;
@@ -247,7 +247,7 @@ static gfn_t hvm_alloc_ioreq_gfn(struct hvm_ioreq_server *s)
return hvm_alloc_legacy_ioreq_gfn(s);
}
-static bool hvm_free_legacy_ioreq_gfn(struct hvm_ioreq_server *s,
+static bool hvm_free_legacy_ioreq_gfn(struct ioreq_server *s,
gfn_t gfn)
{
struct domain *d = s->target;
@@ -265,7 +265,7 @@ static bool hvm_free_legacy_ioreq_gfn(struct hvm_ioreq_server *s,
return true;
}
-static void hvm_free_ioreq_gfn(struct hvm_ioreq_server *s, gfn_t gfn)
+static void hvm_free_ioreq_gfn(struct ioreq_server *s, gfn_t gfn)
{
struct domain *d = s->target;
unsigned int i = gfn_x(gfn) - d->arch.hvm.ioreq_gfn.base;
@@ -279,9 +279,9 @@ static void hvm_free_ioreq_gfn(struct hvm_ioreq_server *s, gfn_t gfn)
}
}
-static void hvm_unmap_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
+static void hvm_unmap_ioreq_gfn(struct ioreq_server *s, bool buf)
{
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+ struct ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
if ( gfn_eq(iorp->gfn, INVALID_GFN) )
return;
@@ -293,10 +293,10 @@ static void hvm_unmap_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
iorp->gfn = INVALID_GFN;
}
-static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
+static int hvm_map_ioreq_gfn(struct ioreq_server *s, bool buf)
{
struct domain *d = s->target;
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+ struct ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
int rc;
if ( iorp->page )
@@ -329,9 +329,9 @@ static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
return rc;
}
-static int hvm_alloc_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
+static int hvm_alloc_ioreq_mfn(struct ioreq_server *s, bool buf)
{
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+ struct ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
struct page_info *page;
if ( iorp->page )
@@ -377,9 +377,9 @@ static int hvm_alloc_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
return -ENOMEM;
}
-static void hvm_free_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
+static void hvm_free_ioreq_mfn(struct ioreq_server *s, bool buf)
{
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+ struct ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
struct page_info *page = iorp->page;
if ( !page )
@@ -396,7 +396,7 @@ static void hvm_free_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
{
- const struct hvm_ioreq_server *s;
+ const struct ioreq_server *s;
unsigned int id;
bool found = false;
@@ -416,11 +416,11 @@ bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
return found;
}
-static void hvm_remove_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
+static void hvm_remove_ioreq_gfn(struct ioreq_server *s, bool buf)
{
struct domain *d = s->target;
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+ struct ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
if ( gfn_eq(iorp->gfn, INVALID_GFN) )
return;
@@ -431,10 +431,10 @@ static void hvm_remove_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
clear_page(iorp->va);
}
-static int hvm_add_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
+static int hvm_add_ioreq_gfn(struct ioreq_server *s, bool buf)
{
struct domain *d = s->target;
- struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
+ struct ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
int rc;
if ( gfn_eq(iorp->gfn, INVALID_GFN) )
@@ -450,8 +450,8 @@ static int hvm_add_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
return rc;
}
-static void hvm_update_ioreq_evtchn(struct hvm_ioreq_server *s,
- struct hvm_ioreq_vcpu *sv)
+static void hvm_update_ioreq_evtchn(struct ioreq_server *s,
+ struct ioreq_vcpu *sv)
{
ASSERT(spin_is_locked(&s->lock));
@@ -466,13 +466,13 @@ static void hvm_update_ioreq_evtchn(struct hvm_ioreq_server *s,
#define HANDLE_BUFIOREQ(s) \
((s)->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF)
-static int hvm_ioreq_server_add_vcpu(struct hvm_ioreq_server *s,
+static int hvm_ioreq_server_add_vcpu(struct ioreq_server *s,
struct vcpu *v)
{
- struct hvm_ioreq_vcpu *sv;
+ struct ioreq_vcpu *sv;
int rc;
- sv = xzalloc(struct hvm_ioreq_vcpu);
+ sv = xzalloc(struct ioreq_vcpu);
rc = -ENOMEM;
if ( !sv )
@@ -518,10 +518,10 @@ static int hvm_ioreq_server_add_vcpu(struct hvm_ioreq_server *s,
return rc;
}
-static void hvm_ioreq_server_remove_vcpu(struct hvm_ioreq_server *s,
+static void hvm_ioreq_server_remove_vcpu(struct ioreq_server *s,
struct vcpu *v)
{
- struct hvm_ioreq_vcpu *sv;
+ struct ioreq_vcpu *sv;
spin_lock(&s->lock);
@@ -546,9 +546,9 @@ static void hvm_ioreq_server_remove_vcpu(struct hvm_ioreq_server *s,
spin_unlock(&s->lock);
}
-static void hvm_ioreq_server_remove_all_vcpus(struct hvm_ioreq_server *s)
+static void hvm_ioreq_server_remove_all_vcpus(struct ioreq_server *s)
{
- struct hvm_ioreq_vcpu *sv, *next;
+ struct ioreq_vcpu *sv, *next;
spin_lock(&s->lock);
@@ -572,7 +572,7 @@ static void hvm_ioreq_server_remove_all_vcpus(struct hvm_ioreq_server *s)
spin_unlock(&s->lock);
}
-static int hvm_ioreq_server_map_pages(struct hvm_ioreq_server *s)
+static int hvm_ioreq_server_map_pages(struct ioreq_server *s)
{
int rc;
@@ -587,13 +587,13 @@ static int hvm_ioreq_server_map_pages(struct hvm_ioreq_server *s)
return rc;
}
-static void hvm_ioreq_server_unmap_pages(struct hvm_ioreq_server *s)
+static void hvm_ioreq_server_unmap_pages(struct ioreq_server *s)
{
hvm_unmap_ioreq_gfn(s, true);
hvm_unmap_ioreq_gfn(s, false);
}
-static int hvm_ioreq_server_alloc_pages(struct hvm_ioreq_server *s)
+static int hvm_ioreq_server_alloc_pages(struct ioreq_server *s)
{
int rc;
@@ -608,13 +608,13 @@ static int hvm_ioreq_server_alloc_pages(struct hvm_ioreq_server *s)
return rc;
}
-static void hvm_ioreq_server_free_pages(struct hvm_ioreq_server *s)
+static void hvm_ioreq_server_free_pages(struct ioreq_server *s)
{
hvm_free_ioreq_mfn(s, true);
hvm_free_ioreq_mfn(s, false);
}
-static void hvm_ioreq_server_free_rangesets(struct hvm_ioreq_server *s)
+static void hvm_ioreq_server_free_rangesets(struct ioreq_server *s)
{
unsigned int i;
@@ -622,7 +622,7 @@ static void hvm_ioreq_server_free_rangesets(struct hvm_ioreq_server *s)
rangeset_destroy(s->range[i]);
}
-static int hvm_ioreq_server_alloc_rangesets(struct hvm_ioreq_server *s,
+static int hvm_ioreq_server_alloc_rangesets(struct ioreq_server *s,
ioservid_t id)
{
unsigned int i;
@@ -660,9 +660,9 @@ static int hvm_ioreq_server_alloc_rangesets(struct hvm_ioreq_server *s,
return rc;
}
-static void hvm_ioreq_server_enable(struct hvm_ioreq_server *s)
+static void hvm_ioreq_server_enable(struct ioreq_server *s)
{
- struct hvm_ioreq_vcpu *sv;
+ struct ioreq_vcpu *sv;
spin_lock(&s->lock);
@@ -683,7 +683,7 @@ static void hvm_ioreq_server_enable(struct hvm_ioreq_server *s)
spin_unlock(&s->lock);
}
-static void hvm_ioreq_server_disable(struct hvm_ioreq_server *s)
+static void hvm_ioreq_server_disable(struct ioreq_server *s)
{
spin_lock(&s->lock);
@@ -699,7 +699,7 @@ static void hvm_ioreq_server_disable(struct hvm_ioreq_server *s)
spin_unlock(&s->lock);
}
-static int hvm_ioreq_server_init(struct hvm_ioreq_server *s,
+static int hvm_ioreq_server_init(struct ioreq_server *s,
struct domain *d, int bufioreq_handling,
ioservid_t id)
{
@@ -744,7 +744,7 @@ static int hvm_ioreq_server_init(struct hvm_ioreq_server *s,
return rc;
}
-static void hvm_ioreq_server_deinit(struct hvm_ioreq_server *s)
+static void hvm_ioreq_server_deinit(struct ioreq_server *s)
{
ASSERT(!s->enabled);
hvm_ioreq_server_remove_all_vcpus(s);
@@ -769,14 +769,14 @@ static void hvm_ioreq_server_deinit(struct hvm_ioreq_server *s)
int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
ioservid_t *id)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
unsigned int i;
int rc;
if ( bufioreq_handling > HVM_IOREQSRV_BUFIOREQ_ATOMIC )
return -EINVAL;
- s = xzalloc(struct hvm_ioreq_server);
+ s = xzalloc(struct ioreq_server);
if ( !s )
return -ENOMEM;
@@ -824,7 +824,7 @@ int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
int rc;
spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
@@ -869,7 +869,7 @@ int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
unsigned long *bufioreq_gfn,
evtchn_port_t *bufioreq_port)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
int rc;
spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
@@ -914,7 +914,7 @@ int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
unsigned long idx, mfn_t *mfn)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
int rc;
ASSERT(is_hvm_domain(d));
@@ -966,7 +966,7 @@ int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
uint32_t type, uint64_t start,
uint64_t end)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
struct rangeset *r;
int rc;
@@ -1018,7 +1018,7 @@ int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
uint32_t type, uint64_t start,
uint64_t end)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
struct rangeset *r;
int rc;
@@ -1069,7 +1069,7 @@ int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
bool enabled)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
int rc;
spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
@@ -1102,7 +1102,7 @@ int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
unsigned int id;
int rc;
@@ -1137,7 +1137,7 @@ int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
unsigned int id;
spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
@@ -1150,7 +1150,7 @@ void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v)
void hvm_destroy_all_ioreq_servers(struct domain *d)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
unsigned int id;
if ( !arch_hvm_ioreq_destroy(d) )
@@ -1177,10 +1177,10 @@ void hvm_destroy_all_ioreq_servers(struct domain *d)
spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
}
-struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
- ioreq_t *p)
+struct ioreq_server *hvm_select_ioreq_server(struct domain *d,
+ ioreq_t *p)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
uint8_t type;
uint64_t addr;
unsigned int id;
@@ -1233,10 +1233,10 @@ struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
return NULL;
}
-static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
+static int hvm_send_buffered_ioreq(struct ioreq_server *s, ioreq_t *p)
{
struct domain *d = current->domain;
- struct hvm_ioreq_page *iorp;
+ struct ioreq_page *iorp;
buffered_iopage_t *pg;
buf_ioreq_t bp = { .data = p->data,
.addr = p->addr,
@@ -1326,12 +1326,12 @@ static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
return IOREQ_STATUS_HANDLED;
}
-int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
+int hvm_send_ioreq(struct ioreq_server *s, ioreq_t *proto_p,
bool buffered)
{
struct vcpu *curr = current;
struct domain *d = curr->domain;
- struct hvm_ioreq_vcpu *sv;
+ struct ioreq_vcpu *sv;
ASSERT(s);
@@ -1389,7 +1389,7 @@ int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered)
{
struct domain *d = current->domain;
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
unsigned int id, failed = 0;
FOR_EACH_IOREQ_SERVER(d, id, s)
diff --git a/xen/include/asm-x86/hvm/domain.h b/xen/include/asm-x86/hvm/domain.h
index 9d247ba..3b36c2f 100644
--- a/xen/include/asm-x86/hvm/domain.h
+++ b/xen/include/asm-x86/hvm/domain.h
@@ -30,40 +30,6 @@
#include <public/hvm/dm_op.h>
-struct hvm_ioreq_page {
- gfn_t gfn;
- struct page_info *page;
- void *va;
-};
-
-struct hvm_ioreq_vcpu {
- struct list_head list_entry;
- struct vcpu *vcpu;
- evtchn_port_t ioreq_evtchn;
- bool pending;
-};
-
-#define NR_IO_RANGE_TYPES (XEN_DMOP_IO_RANGE_PCI + 1)
-#define MAX_NR_IO_RANGES 256
-
-struct hvm_ioreq_server {
- struct domain *target, *emulator;
-
- /* Lock to serialize toolstack modifications */
- spinlock_t lock;
-
- struct hvm_ioreq_page ioreq;
- struct list_head ioreq_vcpu_list;
- struct hvm_ioreq_page bufioreq;
-
- /* Lock to serialize access to buffered ioreq ring */
- spinlock_t bufioreq_lock;
- evtchn_port_t bufioreq_evtchn;
- struct rangeset *range[NR_IO_RANGE_TYPES];
- bool enabled;
- uint8_t bufioreq_handling;
-};
-
#ifdef CONFIG_MEM_SHARING
struct mem_sharing_domain
{
@@ -110,7 +76,7 @@ struct hvm_domain {
/* Lock protects all other values in the sub-struct and the default */
struct {
spinlock_t lock;
- struct hvm_ioreq_server *server[MAX_NR_IOREQ_SERVERS];
+ struct ioreq_server *server[MAX_NR_IOREQ_SERVERS];
} ioreq_server;
/* Cached CF8 for guest PCI config cycles */
diff --git a/xen/include/asm-x86/hvm/ioreq.h b/xen/include/asm-x86/hvm/ioreq.h
index a147856..d2d64a8 100644
--- a/xen/include/asm-x86/hvm/ioreq.h
+++ b/xen/include/asm-x86/hvm/ioreq.h
@@ -50,7 +50,7 @@ static inline bool arch_hvm_io_completion(enum hvm_io_completion io_completion)
}
/* Called when target domain is paused */
-static inline void arch_hvm_destroy_ioreq_server(struct hvm_ioreq_server *s)
+static inline void arch_hvm_destroy_ioreq_server(struct ioreq_server *s)
{
p2m_set_ioreq_server(s->target, 0, s);
}
@@ -68,7 +68,7 @@ static inline int hvm_map_mem_type_to_ioreq_server(struct domain *d,
uint32_t type,
uint32_t flags)
{
- struct hvm_ioreq_server *s;
+ struct ioreq_server *s;
int rc;
if ( type != HVMMEM_ioreq_server )
diff --git a/xen/include/asm-x86/p2m.h b/xen/include/asm-x86/p2m.h
index 8abae34..5f7ba31 100644
--- a/xen/include/asm-x86/p2m.h
+++ b/xen/include/asm-x86/p2m.h
@@ -350,7 +350,7 @@ struct p2m_domain {
* ioreq server who's responsible for the emulation of
* gfns with specific p2m type(for now, p2m_ioreq_server).
*/
- struct hvm_ioreq_server *server;
+ struct ioreq_server *server;
/*
* flags specifies whether read, write or both operations
* are to be emulated by an ioreq server.
@@ -933,9 +933,9 @@ static inline unsigned int p2m_get_iommu_flags(p2m_type_t p2mt, mfn_t mfn)
}
int p2m_set_ioreq_server(struct domain *d, unsigned int flags,
- struct hvm_ioreq_server *s);
-struct hvm_ioreq_server *p2m_get_ioreq_server(struct domain *d,
- unsigned int *flags);
+ struct ioreq_server *s);
+struct ioreq_server *p2m_get_ioreq_server(struct domain *d,
+ unsigned int *flags);
static inline int p2m_entry_modify(struct p2m_domain *p2m, p2m_type_t nt,
p2m_type_t ot, mfn_t nfn, mfn_t ofn,
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
index 768ac94..8451866 100644
--- a/xen/include/xen/ioreq.h
+++ b/xen/include/xen/ioreq.h
@@ -21,8 +21,42 @@
#include <xen/sched.h>
-struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
- unsigned int id);
+struct ioreq_page {
+ gfn_t gfn;
+ struct page_info *page;
+ void *va;
+};
+
+struct ioreq_vcpu {
+ struct list_head list_entry;
+ struct vcpu *vcpu;
+ evtchn_port_t ioreq_evtchn;
+ bool pending;
+};
+
+#define NR_IO_RANGE_TYPES (XEN_DMOP_IO_RANGE_PCI + 1)
+#define MAX_NR_IO_RANGES 256
+
+struct ioreq_server {
+ struct domain *target, *emulator;
+
+ /* Lock to serialize toolstack modifications */
+ spinlock_t lock;
+
+ struct ioreq_page ioreq;
+ struct list_head ioreq_vcpu_list;
+ struct ioreq_page bufioreq;
+
+ /* Lock to serialize access to buffered ioreq ring */
+ spinlock_t bufioreq_lock;
+ evtchn_port_t bufioreq_evtchn;
+ struct rangeset *range[NR_IO_RANGE_TYPES];
+ bool enabled;
+ uint8_t bufioreq_handling;
+};
+
+struct ioreq_server *get_ioreq_server(const struct domain *d,
+ unsigned int id);
static inline paddr_t ioreq_mmio_first_byte(const ioreq_t *p)
{
@@ -73,9 +107,9 @@ int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v);
void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v);
void hvm_destroy_all_ioreq_servers(struct domain *d);
-struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
- ioreq_t *p);
-int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
+struct ioreq_server *hvm_select_ioreq_server(struct domain *d,
+ ioreq_t *p);
+int hvm_send_ioreq(struct ioreq_server *s, ioreq_t *proto_p,
bool buffered);
unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered);
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 07/23] xen/ioreq: Move x86's ioreq_gfn(server) to struct domain
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (5 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 06/23] xen/ioreq: Make x86's hvm_ioreq_(page/vcpu/server) structs common Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-11-12 11:21 ` Jan Beulich
2020-11-18 12:09 ` Oleksandr
2020-10-15 16:44 ` [PATCH V2 08/23] xen/ioreq: Introduce ioreq_params to abstract accesses to arch.hvm.params Oleksandr Tyshchenko
` (16 subsequent siblings)
23 siblings, 2 replies; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Paul Durrant, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
The IOREQ is a common feature now and these structs will be used
on Arm as is. Move them to common struct domain. This also
significantly reduces the layering violation in the common code
(*arch.hvm* usage).
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes V1 -> V2:
- new patch
---
xen/arch/x86/hvm/hvm.c | 12 +++----
xen/common/ioreq.c | 72 ++++++++++++++++++++--------------------
xen/include/asm-x86/hvm/domain.h | 15 ---------
xen/include/asm-x86/hvm/ioreq.h | 4 +--
xen/include/xen/sched.h | 17 ++++++++++
5 files changed, 61 insertions(+), 59 deletions(-)
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 54e32e4..20376ce 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -4218,20 +4218,20 @@ static int hvm_set_param(struct domain *d, uint32_t index, uint64_t value)
rc = -EINVAL;
break;
case HVM_PARAM_IOREQ_SERVER_PFN:
- d->arch.hvm.ioreq_gfn.base = value;
+ d->ioreq_gfn.base = value;
break;
case HVM_PARAM_NR_IOREQ_SERVER_PAGES:
{
unsigned int i;
if ( value == 0 ||
- value > sizeof(d->arch.hvm.ioreq_gfn.mask) * 8 )
+ value > sizeof(d->ioreq_gfn.mask) * 8 )
{
rc = -EINVAL;
break;
}
for ( i = 0; i < value; i++ )
- set_bit(i, &d->arch.hvm.ioreq_gfn.mask);
+ set_bit(i, &d->ioreq_gfn.mask);
break;
}
@@ -4239,11 +4239,11 @@ static int hvm_set_param(struct domain *d, uint32_t index, uint64_t value)
case HVM_PARAM_IOREQ_PFN:
case HVM_PARAM_BUFIOREQ_PFN:
BUILD_BUG_ON(HVM_PARAM_IOREQ_PFN >
- sizeof(d->arch.hvm.ioreq_gfn.legacy_mask) * 8);
+ sizeof(d->ioreq_gfn.legacy_mask) * 8);
BUILD_BUG_ON(HVM_PARAM_BUFIOREQ_PFN >
- sizeof(d->arch.hvm.ioreq_gfn.legacy_mask) * 8);
+ sizeof(d->ioreq_gfn.legacy_mask) * 8);
if ( value )
- set_bit(index, &d->arch.hvm.ioreq_gfn.legacy_mask);
+ set_bit(index, &d->ioreq_gfn.legacy_mask);
break;
case HVM_PARAM_X87_FIP_WIDTH:
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index 1d62d13..7f91bc2 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -37,13 +37,13 @@ static void set_ioreq_server(struct domain *d, unsigned int id,
struct ioreq_server *s)
{
ASSERT(id < MAX_NR_IOREQ_SERVERS);
- ASSERT(!s || !d->arch.hvm.ioreq_server.server[id]);
+ ASSERT(!s || !d->ioreq_server.server[id]);
- d->arch.hvm.ioreq_server.server[id] = s;
+ d->ioreq_server.server[id] = s;
}
#define GET_IOREQ_SERVER(d, id) \
- (d)->arch.hvm.ioreq_server.server[id]
+ (d)->ioreq_server.server[id]
struct ioreq_server *get_ioreq_server(const struct domain *d,
unsigned int id)
@@ -222,7 +222,7 @@ static gfn_t hvm_alloc_legacy_ioreq_gfn(struct ioreq_server *s)
for ( i = HVM_PARAM_IOREQ_PFN; i <= HVM_PARAM_BUFIOREQ_PFN; i++ )
{
- if ( !test_and_clear_bit(i, &d->arch.hvm.ioreq_gfn.legacy_mask) )
+ if ( !test_and_clear_bit(i, &d->ioreq_gfn.legacy_mask) )
return _gfn(d->arch.hvm.params[i]);
}
@@ -234,10 +234,10 @@ static gfn_t hvm_alloc_ioreq_gfn(struct ioreq_server *s)
struct domain *d = s->target;
unsigned int i;
- for ( i = 0; i < sizeof(d->arch.hvm.ioreq_gfn.mask) * 8; i++ )
+ for ( i = 0; i < sizeof(d->ioreq_gfn.mask) * 8; i++ )
{
- if ( test_and_clear_bit(i, &d->arch.hvm.ioreq_gfn.mask) )
- return _gfn(d->arch.hvm.ioreq_gfn.base + i);
+ if ( test_and_clear_bit(i, &d->ioreq_gfn.mask) )
+ return _gfn(d->ioreq_gfn.base + i);
}
/*
@@ -261,21 +261,21 @@ static bool hvm_free_legacy_ioreq_gfn(struct ioreq_server *s,
if ( i > HVM_PARAM_BUFIOREQ_PFN )
return false;
- set_bit(i, &d->arch.hvm.ioreq_gfn.legacy_mask);
+ set_bit(i, &d->ioreq_gfn.legacy_mask);
return true;
}
static void hvm_free_ioreq_gfn(struct ioreq_server *s, gfn_t gfn)
{
struct domain *d = s->target;
- unsigned int i = gfn_x(gfn) - d->arch.hvm.ioreq_gfn.base;
+ unsigned int i = gfn_x(gfn) - d->ioreq_gfn.base;
ASSERT(!gfn_eq(gfn, INVALID_GFN));
if ( !hvm_free_legacy_ioreq_gfn(s, gfn) )
{
- ASSERT(i < sizeof(d->arch.hvm.ioreq_gfn.mask) * 8);
- set_bit(i, &d->arch.hvm.ioreq_gfn.mask);
+ ASSERT(i < sizeof(d->ioreq_gfn.mask) * 8);
+ set_bit(i, &d->ioreq_gfn.mask);
}
}
@@ -400,7 +400,7 @@ bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
unsigned int id;
bool found = false;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
FOR_EACH_IOREQ_SERVER(d, id, s)
{
@@ -411,7 +411,7 @@ bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
}
}
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
return found;
}
@@ -781,7 +781,7 @@ int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
return -ENOMEM;
domain_pause(d);
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
for ( i = 0; i < MAX_NR_IOREQ_SERVERS; i++ )
{
@@ -809,13 +809,13 @@ int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
if ( id )
*id = i;
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
domain_unpause(d);
return 0;
fail:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
domain_unpause(d);
xfree(s);
@@ -827,7 +827,7 @@ int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
struct ioreq_server *s;
int rc;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
s = get_ioreq_server(d, id);
@@ -859,7 +859,7 @@ int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
rc = 0;
out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
return rc;
}
@@ -872,7 +872,7 @@ int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
struct ioreq_server *s;
int rc;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
s = get_ioreq_server(d, id);
@@ -906,7 +906,7 @@ int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
rc = 0;
out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
return rc;
}
@@ -919,7 +919,7 @@ int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
ASSERT(is_hvm_domain(d));
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
s = get_ioreq_server(d, id);
@@ -957,7 +957,7 @@ int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
}
out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
return rc;
}
@@ -973,7 +973,7 @@ int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
if ( start > end )
return -EINVAL;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
s = get_ioreq_server(d, id);
@@ -1009,7 +1009,7 @@ int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
rc = rangeset_add_range(r, start, end);
out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
return rc;
}
@@ -1025,7 +1025,7 @@ int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
if ( start > end )
return -EINVAL;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
s = get_ioreq_server(d, id);
@@ -1061,7 +1061,7 @@ int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
rc = rangeset_remove_range(r, start, end);
out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
return rc;
}
@@ -1072,7 +1072,7 @@ int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
struct ioreq_server *s;
int rc;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
s = get_ioreq_server(d, id);
@@ -1096,7 +1096,7 @@ int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
rc = 0;
out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
return rc;
}
@@ -1106,7 +1106,7 @@ int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
unsigned int id;
int rc;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
FOR_EACH_IOREQ_SERVER(d, id, s)
{
@@ -1115,7 +1115,7 @@ int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
goto fail;
}
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
return 0;
@@ -1130,7 +1130,7 @@ int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
hvm_ioreq_server_remove_vcpu(s, v);
}
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
return rc;
}
@@ -1140,12 +1140,12 @@ void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v)
struct ioreq_server *s;
unsigned int id;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
FOR_EACH_IOREQ_SERVER(d, id, s)
hvm_ioreq_server_remove_vcpu(s, v);
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
}
void hvm_destroy_all_ioreq_servers(struct domain *d)
@@ -1156,7 +1156,7 @@ void hvm_destroy_all_ioreq_servers(struct domain *d)
if ( !arch_hvm_ioreq_destroy(d) )
return;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
/* No need to domain_pause() as the domain is being torn down */
@@ -1174,7 +1174,7 @@ void hvm_destroy_all_ioreq_servers(struct domain *d)
xfree(s);
}
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
}
struct ioreq_server *hvm_select_ioreq_server(struct domain *d,
@@ -1406,7 +1406,7 @@ unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered)
void hvm_ioreq_init(struct domain *d)
{
- spin_lock_init(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_init(&d->ioreq_server.lock);
arch_hvm_ioreq_init(d);
}
diff --git a/xen/include/asm-x86/hvm/domain.h b/xen/include/asm-x86/hvm/domain.h
index 3b36c2f..5d60737 100644
--- a/xen/include/asm-x86/hvm/domain.h
+++ b/xen/include/asm-x86/hvm/domain.h
@@ -63,22 +63,7 @@ struct hvm_pi_ops {
void (*vcpu_block)(struct vcpu *);
};
-#define MAX_NR_IOREQ_SERVERS 8
-
struct hvm_domain {
- /* Guest page range used for non-default ioreq servers */
- struct {
- unsigned long base;
- unsigned long mask; /* indexed by GFN minus base */
- unsigned long legacy_mask; /* indexed by HVM param number */
- } ioreq_gfn;
-
- /* Lock protects all other values in the sub-struct and the default */
- struct {
- spinlock_t lock;
- struct ioreq_server *server[MAX_NR_IOREQ_SERVERS];
- } ioreq_server;
-
/* Cached CF8 for guest PCI config cycles */
uint32_t pci_cf8;
diff --git a/xen/include/asm-x86/hvm/ioreq.h b/xen/include/asm-x86/hvm/ioreq.h
index d2d64a8..0fccac5 100644
--- a/xen/include/asm-x86/hvm/ioreq.h
+++ b/xen/include/asm-x86/hvm/ioreq.h
@@ -77,7 +77,7 @@ static inline int hvm_map_mem_type_to_ioreq_server(struct domain *d,
if ( flags & ~XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
return -EINVAL;
- spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_lock_recursive(&d->ioreq_server.lock);
s = get_ioreq_server(d, id);
@@ -92,7 +92,7 @@ static inline int hvm_map_mem_type_to_ioreq_server(struct domain *d,
rc = p2m_set_ioreq_server(d, flags, s);
out:
- spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
+ spin_unlock_recursive(&d->ioreq_server.lock);
if ( rc == 0 && flags == 0 )
{
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index d8ed83f..78761cd 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -314,6 +314,8 @@ struct sched_unit {
struct evtchn_port_ops;
+#define MAX_NR_IOREQ_SERVERS 8
+
struct domain
{
domid_t domain_id;
@@ -521,6 +523,21 @@ struct domain
/* Argo interdomain communication support */
struct argo_domain *argo;
#endif
+
+#ifdef CONFIG_IOREQ_SERVER
+ /* Guest page range used for non-default ioreq servers */
+ struct {
+ unsigned long base;
+ unsigned long mask;
+ unsigned long legacy_mask; /* indexed by HVM param number */
+ } ioreq_gfn;
+
+ /* Lock protects all other values in the sub-struct and the default */
+ struct {
+ spinlock_t lock;
+ struct ioreq_server *server[MAX_NR_IOREQ_SERVERS];
+ } ioreq_server;
+#endif
};
static inline struct page_list_head *page_to_list(
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 08/23] xen/ioreq: Introduce ioreq_params to abstract accesses to arch.hvm.params
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (6 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 07/23] xen/ioreq: Move x86's ioreq_gfn(server) to struct domain Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-10-20 10:41 ` Paul Durrant
2020-10-15 16:44 ` [PATCH V2 09/23] xen/dm: Make x86's DM feature common Oleksandr Tyshchenko
` (15 subsequent siblings)
23 siblings, 1 reply; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, Julien Grall, Stefano Stabellini, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
We don't want to move HVM params field out of *arch.hvm* in this particular
case as although it stores a few IOREQ params, it is not a (completely)
IOREQ stuff and is specific to the architecture. Instead, abstract
accesses by the proposed macro.
This is a follow up action to reduce layering violation in the common code.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes V1 -> V2:
- new patch
---
xen/common/ioreq.c | 4 ++--
xen/include/asm-x86/hvm/domain.h | 2 ++
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index 7f91bc2..a07f1d7 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -223,7 +223,7 @@ static gfn_t hvm_alloc_legacy_ioreq_gfn(struct ioreq_server *s)
for ( i = HVM_PARAM_IOREQ_PFN; i <= HVM_PARAM_BUFIOREQ_PFN; i++ )
{
if ( !test_and_clear_bit(i, &d->ioreq_gfn.legacy_mask) )
- return _gfn(d->arch.hvm.params[i]);
+ return _gfn(ioreq_params(d, i));
}
return INVALID_GFN;
@@ -255,7 +255,7 @@ static bool hvm_free_legacy_ioreq_gfn(struct ioreq_server *s,
for ( i = HVM_PARAM_IOREQ_PFN; i <= HVM_PARAM_BUFIOREQ_PFN; i++ )
{
- if ( gfn_eq(gfn, _gfn(d->arch.hvm.params[i])) )
+ if ( gfn_eq(gfn, _gfn(ioreq_params(d, i))) )
break;
}
if ( i > HVM_PARAM_BUFIOREQ_PFN )
diff --git a/xen/include/asm-x86/hvm/domain.h b/xen/include/asm-x86/hvm/domain.h
index 5d60737..c3af339 100644
--- a/xen/include/asm-x86/hvm/domain.h
+++ b/xen/include/asm-x86/hvm/domain.h
@@ -63,6 +63,8 @@ struct hvm_pi_ops {
void (*vcpu_block)(struct vcpu *);
};
+#define ioreq_params(d, i) ((d)->arch.hvm.params[i])
+
struct hvm_domain {
/* Cached CF8 for guest PCI config cycles */
uint32_t pci_cf8;
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 09/23] xen/dm: Make x86's DM feature common
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (7 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 08/23] xen/ioreq: Introduce ioreq_params to abstract accesses to arch.hvm.params Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-11-12 11:32 ` Jan Beulich
2020-10-15 16:44 ` [PATCH V2 10/23] xen/mm: Make x86's XENMEM_resource_ioreq_server handling common Oleksandr Tyshchenko
` (14 subsequent siblings)
23 siblings, 1 reply; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Julien Grall, Jan Beulich, Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Daniel De Graaf, Oleksandr Tyshchenko
From: Julien Grall <julien.grall@arm.com>
As a lot of x86 code can be re-used on Arm later on, this patch
splits devicemodel support into common and arch specific parts.
The common DM feature is supposed to be built with IOREQ_SERVER
option enabled (as well as the IOREQ feature), which is selected
for x86's config HVM for now.
Also update XSM code a bit to let DM op be used on Arm.
This support is going to be used on Arm to be able run device
emulator outside of Xen hypervisor.
Signed-off-by: Julien Grall <julien.grall@arm.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- update XSM, related changes were pulled from:
[RFC PATCH V1 04/12] xen/arm: Introduce arch specific bits for IOREQ/DM features
Changes V1 -> V2:
- update the author of a patch
- update patch description
- introduce xen/dm.h and move definitions here
---
xen/arch/x86/hvm/dm.c | 291 ++++--------------------------------------------
xen/common/Makefile | 1 +
xen/common/dm.c | 291 ++++++++++++++++++++++++++++++++++++++++++++++++
xen/include/xen/dm.h | 44 ++++++++
xen/include/xsm/dummy.h | 4 +-
xen/include/xsm/xsm.h | 6 +-
xen/xsm/dummy.c | 2 +-
xen/xsm/flask/hooks.c | 5 +-
8 files changed, 364 insertions(+), 280 deletions(-)
create mode 100644 xen/common/dm.c
create mode 100644 xen/include/xen/dm.h
diff --git a/xen/arch/x86/hvm/dm.c b/xen/arch/x86/hvm/dm.c
index 71f5ca4..35f860a 100644
--- a/xen/arch/x86/hvm/dm.c
+++ b/xen/arch/x86/hvm/dm.c
@@ -16,6 +16,7 @@
#include <xen/event.h>
#include <xen/guest_access.h>
+#include <xen/dm.h>
#include <xen/hypercall.h>
#include <xen/nospec.h>
#include <xen/sched.h>
@@ -29,13 +30,6 @@
#include <public/hvm/hvm_op.h>
-struct dmop_args {
- domid_t domid;
- unsigned int nr_bufs;
- /* Reserve enough buf elements for all current hypercalls. */
- struct xen_dm_op_buf buf[2];
-};
-
static bool _raw_copy_from_guest_buf_offset(void *dst,
const struct dmop_args *args,
unsigned int buf_idx,
@@ -338,148 +332,20 @@ static int inject_event(struct domain *d,
return 0;
}
-static int dm_op(const struct dmop_args *op_args)
+int arch_dm_op(struct xen_dm_op *op, struct domain *d,
+ const struct dmop_args *op_args, bool *const_op)
{
- struct domain *d;
- struct xen_dm_op op;
- bool const_op = true;
long rc;
- size_t offset;
-
- static const uint8_t op_size[] = {
- [XEN_DMOP_create_ioreq_server] = sizeof(struct xen_dm_op_create_ioreq_server),
- [XEN_DMOP_get_ioreq_server_info] = sizeof(struct xen_dm_op_get_ioreq_server_info),
- [XEN_DMOP_map_io_range_to_ioreq_server] = sizeof(struct xen_dm_op_ioreq_server_range),
- [XEN_DMOP_unmap_io_range_from_ioreq_server] = sizeof(struct xen_dm_op_ioreq_server_range),
- [XEN_DMOP_set_ioreq_server_state] = sizeof(struct xen_dm_op_set_ioreq_server_state),
- [XEN_DMOP_destroy_ioreq_server] = sizeof(struct xen_dm_op_destroy_ioreq_server),
- [XEN_DMOP_track_dirty_vram] = sizeof(struct xen_dm_op_track_dirty_vram),
- [XEN_DMOP_set_pci_intx_level] = sizeof(struct xen_dm_op_set_pci_intx_level),
- [XEN_DMOP_set_isa_irq_level] = sizeof(struct xen_dm_op_set_isa_irq_level),
- [XEN_DMOP_set_pci_link_route] = sizeof(struct xen_dm_op_set_pci_link_route),
- [XEN_DMOP_modified_memory] = sizeof(struct xen_dm_op_modified_memory),
- [XEN_DMOP_set_mem_type] = sizeof(struct xen_dm_op_set_mem_type),
- [XEN_DMOP_inject_event] = sizeof(struct xen_dm_op_inject_event),
- [XEN_DMOP_inject_msi] = sizeof(struct xen_dm_op_inject_msi),
- [XEN_DMOP_map_mem_type_to_ioreq_server] = sizeof(struct xen_dm_op_map_mem_type_to_ioreq_server),
- [XEN_DMOP_remote_shutdown] = sizeof(struct xen_dm_op_remote_shutdown),
- [XEN_DMOP_relocate_memory] = sizeof(struct xen_dm_op_relocate_memory),
- [XEN_DMOP_pin_memory_cacheattr] = sizeof(struct xen_dm_op_pin_memory_cacheattr),
- };
-
- rc = rcu_lock_remote_domain_by_id(op_args->domid, &d);
- if ( rc )
- return rc;
-
- if ( !is_hvm_domain(d) )
- goto out;
-
- rc = xsm_dm_op(XSM_DM_PRIV, d);
- if ( rc )
- goto out;
-
- offset = offsetof(struct xen_dm_op, u);
-
- rc = -EFAULT;
- if ( op_args->buf[0].size < offset )
- goto out;
-
- if ( copy_from_guest_offset((void *)&op, op_args->buf[0].h, 0, offset) )
- goto out;
-
- if ( op.op >= ARRAY_SIZE(op_size) )
- {
- rc = -EOPNOTSUPP;
- goto out;
- }
-
- op.op = array_index_nospec(op.op, ARRAY_SIZE(op_size));
-
- if ( op_args->buf[0].size < offset + op_size[op.op] )
- goto out;
-
- if ( copy_from_guest_offset((void *)&op.u, op_args->buf[0].h, offset,
- op_size[op.op]) )
- goto out;
-
- rc = -EINVAL;
- if ( op.pad )
- goto out;
-
- switch ( op.op )
- {
- case XEN_DMOP_create_ioreq_server:
- {
- struct xen_dm_op_create_ioreq_server *data =
- &op.u.create_ioreq_server;
-
- const_op = false;
-
- rc = -EINVAL;
- if ( data->pad[0] || data->pad[1] || data->pad[2] )
- break;
-
- rc = hvm_create_ioreq_server(d, data->handle_bufioreq,
- &data->id);
- break;
- }
- case XEN_DMOP_get_ioreq_server_info:
+ switch ( op->op )
{
- struct xen_dm_op_get_ioreq_server_info *data =
- &op.u.get_ioreq_server_info;
- const uint16_t valid_flags = XEN_DMOP_no_gfns;
-
- const_op = false;
-
- rc = -EINVAL;
- if ( data->flags & ~valid_flags )
- break;
-
- rc = hvm_get_ioreq_server_info(d, data->id,
- (data->flags & XEN_DMOP_no_gfns) ?
- NULL : &data->ioreq_gfn,
- (data->flags & XEN_DMOP_no_gfns) ?
- NULL : &data->bufioreq_gfn,
- &data->bufioreq_port);
- break;
- }
-
- case XEN_DMOP_map_io_range_to_ioreq_server:
- {
- const struct xen_dm_op_ioreq_server_range *data =
- &op.u.map_io_range_to_ioreq_server;
-
- rc = -EINVAL;
- if ( data->pad )
- break;
-
- rc = hvm_map_io_range_to_ioreq_server(d, data->id, data->type,
- data->start, data->end);
- break;
- }
-
- case XEN_DMOP_unmap_io_range_from_ioreq_server:
- {
- const struct xen_dm_op_ioreq_server_range *data =
- &op.u.unmap_io_range_from_ioreq_server;
-
- rc = -EINVAL;
- if ( data->pad )
- break;
-
- rc = hvm_unmap_io_range_from_ioreq_server(d, data->id, data->type,
- data->start, data->end);
- break;
- }
-
case XEN_DMOP_map_mem_type_to_ioreq_server:
{
struct xen_dm_op_map_mem_type_to_ioreq_server *data =
- &op.u.map_mem_type_to_ioreq_server;
+ &op->u.map_mem_type_to_ioreq_server;
unsigned long first_gfn = data->opaque;
- const_op = false;
+ *const_op = false;
rc = -EOPNOTSUPP;
if ( !hap_enabled(d) )
@@ -523,36 +389,10 @@ static int dm_op(const struct dmop_args *op_args)
break;
}
- case XEN_DMOP_set_ioreq_server_state:
- {
- const struct xen_dm_op_set_ioreq_server_state *data =
- &op.u.set_ioreq_server_state;
-
- rc = -EINVAL;
- if ( data->pad )
- break;
-
- rc = hvm_set_ioreq_server_state(d, data->id, !!data->enabled);
- break;
- }
-
- case XEN_DMOP_destroy_ioreq_server:
- {
- const struct xen_dm_op_destroy_ioreq_server *data =
- &op.u.destroy_ioreq_server;
-
- rc = -EINVAL;
- if ( data->pad )
- break;
-
- rc = hvm_destroy_ioreq_server(d, data->id);
- break;
- }
-
case XEN_DMOP_track_dirty_vram:
{
const struct xen_dm_op_track_dirty_vram *data =
- &op.u.track_dirty_vram;
+ &op->u.track_dirty_vram;
rc = -EINVAL;
if ( data->pad )
@@ -568,7 +408,7 @@ static int dm_op(const struct dmop_args *op_args)
case XEN_DMOP_set_pci_intx_level:
{
const struct xen_dm_op_set_pci_intx_level *data =
- &op.u.set_pci_intx_level;
+ &op->u.set_pci_intx_level;
rc = set_pci_intx_level(d, data->domain, data->bus,
data->device, data->intx,
@@ -579,7 +419,7 @@ static int dm_op(const struct dmop_args *op_args)
case XEN_DMOP_set_isa_irq_level:
{
const struct xen_dm_op_set_isa_irq_level *data =
- &op.u.set_isa_irq_level;
+ &op->u.set_isa_irq_level;
rc = set_isa_irq_level(d, data->isa_irq, data->level);
break;
@@ -588,7 +428,7 @@ static int dm_op(const struct dmop_args *op_args)
case XEN_DMOP_set_pci_link_route:
{
const struct xen_dm_op_set_pci_link_route *data =
- &op.u.set_pci_link_route;
+ &op->u.set_pci_link_route;
rc = hvm_set_pci_link_route(d, data->link, data->isa_irq);
break;
@@ -597,19 +437,19 @@ static int dm_op(const struct dmop_args *op_args)
case XEN_DMOP_modified_memory:
{
struct xen_dm_op_modified_memory *data =
- &op.u.modified_memory;
+ &op->u.modified_memory;
rc = modified_memory(d, op_args, data);
- const_op = !rc;
+ *const_op = !rc;
break;
}
case XEN_DMOP_set_mem_type:
{
struct xen_dm_op_set_mem_type *data =
- &op.u.set_mem_type;
+ &op->u.set_mem_type;
- const_op = false;
+ *const_op = false;
rc = -EINVAL;
if ( data->pad )
@@ -622,7 +462,7 @@ static int dm_op(const struct dmop_args *op_args)
case XEN_DMOP_inject_event:
{
const struct xen_dm_op_inject_event *data =
- &op.u.inject_event;
+ &op->u.inject_event;
rc = -EINVAL;
if ( data->pad0 || data->pad1 )
@@ -635,7 +475,7 @@ static int dm_op(const struct dmop_args *op_args)
case XEN_DMOP_inject_msi:
{
const struct xen_dm_op_inject_msi *data =
- &op.u.inject_msi;
+ &op->u.inject_msi;
rc = -EINVAL;
if ( data->pad )
@@ -648,7 +488,7 @@ static int dm_op(const struct dmop_args *op_args)
case XEN_DMOP_remote_shutdown:
{
const struct xen_dm_op_remote_shutdown *data =
- &op.u.remote_shutdown;
+ &op->u.remote_shutdown;
domain_shutdown(d, data->reason);
rc = 0;
@@ -657,7 +497,7 @@ static int dm_op(const struct dmop_args *op_args)
case XEN_DMOP_relocate_memory:
{
- struct xen_dm_op_relocate_memory *data = &op.u.relocate_memory;
+ struct xen_dm_op_relocate_memory *data = &op->u.relocate_memory;
struct xen_add_to_physmap xatp = {
.domid = op_args->domid,
.size = data->size,
@@ -680,7 +520,7 @@ static int dm_op(const struct dmop_args *op_args)
data->size -= rc;
data->src_gfn += rc;
data->dst_gfn += rc;
- const_op = false;
+ *const_op = false;
rc = -ERESTART;
}
break;
@@ -689,7 +529,7 @@ static int dm_op(const struct dmop_args *op_args)
case XEN_DMOP_pin_memory_cacheattr:
{
const struct xen_dm_op_pin_memory_cacheattr *data =
- &op.u.pin_memory_cacheattr;
+ &op->u.pin_memory_cacheattr;
if ( data->pad )
{
@@ -707,97 +547,6 @@ static int dm_op(const struct dmop_args *op_args)
break;
}
- if ( (!rc || rc == -ERESTART) &&
- !const_op && copy_to_guest_offset(op_args->buf[0].h, offset,
- (void *)&op.u, op_size[op.op]) )
- rc = -EFAULT;
-
- out:
- rcu_unlock_domain(d);
-
- return rc;
-}
-
-#include <compat/hvm/dm_op.h>
-
-CHECK_dm_op_create_ioreq_server;
-CHECK_dm_op_get_ioreq_server_info;
-CHECK_dm_op_ioreq_server_range;
-CHECK_dm_op_set_ioreq_server_state;
-CHECK_dm_op_destroy_ioreq_server;
-CHECK_dm_op_track_dirty_vram;
-CHECK_dm_op_set_pci_intx_level;
-CHECK_dm_op_set_isa_irq_level;
-CHECK_dm_op_set_pci_link_route;
-CHECK_dm_op_modified_memory;
-CHECK_dm_op_set_mem_type;
-CHECK_dm_op_inject_event;
-CHECK_dm_op_inject_msi;
-CHECK_dm_op_map_mem_type_to_ioreq_server;
-CHECK_dm_op_remote_shutdown;
-CHECK_dm_op_relocate_memory;
-CHECK_dm_op_pin_memory_cacheattr;
-
-int compat_dm_op(domid_t domid,
- unsigned int nr_bufs,
- XEN_GUEST_HANDLE_PARAM(void) bufs)
-{
- struct dmop_args args;
- unsigned int i;
- int rc;
-
- if ( nr_bufs > ARRAY_SIZE(args.buf) )
- return -E2BIG;
-
- args.domid = domid;
- args.nr_bufs = array_index_nospec(nr_bufs, ARRAY_SIZE(args.buf) + 1);
-
- for ( i = 0; i < args.nr_bufs; i++ )
- {
- struct compat_dm_op_buf cmp;
-
- if ( copy_from_guest_offset(&cmp, bufs, i, 1) )
- return -EFAULT;
-
-#define XLAT_dm_op_buf_HNDL_h(_d_, _s_) \
- guest_from_compat_handle((_d_)->h, (_s_)->h)
-
- XLAT_dm_op_buf(&args.buf[i], &cmp);
-
-#undef XLAT_dm_op_buf_HNDL_h
- }
-
- rc = dm_op(&args);
-
- if ( rc == -ERESTART )
- rc = hypercall_create_continuation(__HYPERVISOR_dm_op, "iih",
- domid, nr_bufs, bufs);
-
- return rc;
-}
-
-long do_dm_op(domid_t domid,
- unsigned int nr_bufs,
- XEN_GUEST_HANDLE_PARAM(xen_dm_op_buf_t) bufs)
-{
- struct dmop_args args;
- int rc;
-
- if ( nr_bufs > ARRAY_SIZE(args.buf) )
- return -E2BIG;
-
- args.domid = domid;
- args.nr_bufs = array_index_nospec(nr_bufs, ARRAY_SIZE(args.buf) + 1);
-
- if ( copy_from_guest_offset(&args.buf[0], bufs, 0, args.nr_bufs) )
- return -EFAULT;
-
- rc = dm_op(&args);
-
- if ( rc == -ERESTART )
- rc = hypercall_create_continuation(__HYPERVISOR_dm_op, "iih",
- domid, nr_bufs, bufs);
-
return rc;
}
diff --git a/xen/common/Makefile b/xen/common/Makefile
index cdb99fb..8c872d3 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -6,6 +6,7 @@ obj-$(CONFIG_CORE_PARKING) += core_parking.o
obj-y += cpu.o
obj-$(CONFIG_DEBUG_TRACE) += debugtrace.o
obj-$(CONFIG_HAS_DEVICE_TREE) += device_tree.o
+obj-$(CONFIG_IOREQ_SERVER) += dm.o
obj-y += domain.o
obj-y += event_2l.o
obj-y += event_channel.o
diff --git a/xen/common/dm.c b/xen/common/dm.c
new file mode 100644
index 0000000..36e01a2
--- /dev/null
+++ b/xen/common/dm.c
@@ -0,0 +1,291 @@
+/*
+ * Copyright (c) 2016 Citrix Systems Inc.
+ * Copyright (c) 2019 Arm ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/guest_access.h>
+#include <xen/dm.h>
+#include <xen/hypercall.h>
+#include <xen/ioreq.h>
+#include <xen/nospec.h>
+
+static int dm_op(const struct dmop_args *op_args)
+{
+ struct domain *d;
+ struct xen_dm_op op;
+ long rc;
+ bool const_op = true;
+ const size_t offset = offsetof(struct xen_dm_op, u);
+
+ static const uint8_t op_size[] = {
+ [XEN_DMOP_create_ioreq_server] = sizeof(struct xen_dm_op_create_ioreq_server),
+ [XEN_DMOP_get_ioreq_server_info] = sizeof(struct xen_dm_op_get_ioreq_server_info),
+ [XEN_DMOP_map_io_range_to_ioreq_server] = sizeof(struct xen_dm_op_ioreq_server_range),
+ [XEN_DMOP_unmap_io_range_from_ioreq_server] = sizeof(struct xen_dm_op_ioreq_server_range),
+ [XEN_DMOP_set_ioreq_server_state] = sizeof(struct xen_dm_op_set_ioreq_server_state),
+ [XEN_DMOP_destroy_ioreq_server] = sizeof(struct xen_dm_op_destroy_ioreq_server),
+ [XEN_DMOP_track_dirty_vram] = sizeof(struct xen_dm_op_track_dirty_vram),
+ [XEN_DMOP_set_pci_intx_level] = sizeof(struct xen_dm_op_set_pci_intx_level),
+ [XEN_DMOP_set_isa_irq_level] = sizeof(struct xen_dm_op_set_isa_irq_level),
+ [XEN_DMOP_set_pci_link_route] = sizeof(struct xen_dm_op_set_pci_link_route),
+ [XEN_DMOP_modified_memory] = sizeof(struct xen_dm_op_modified_memory),
+ [XEN_DMOP_set_mem_type] = sizeof(struct xen_dm_op_set_mem_type),
+ [XEN_DMOP_inject_event] = sizeof(struct xen_dm_op_inject_event),
+ [XEN_DMOP_inject_msi] = sizeof(struct xen_dm_op_inject_msi),
+ [XEN_DMOP_map_mem_type_to_ioreq_server] = sizeof(struct xen_dm_op_map_mem_type_to_ioreq_server),
+ [XEN_DMOP_remote_shutdown] = sizeof(struct xen_dm_op_remote_shutdown),
+ [XEN_DMOP_relocate_memory] = sizeof(struct xen_dm_op_relocate_memory),
+ [XEN_DMOP_pin_memory_cacheattr] = sizeof(struct xen_dm_op_pin_memory_cacheattr),
+ };
+
+ rc = rcu_lock_remote_domain_by_id(op_args->domid, &d);
+ if ( rc )
+ return rc;
+
+ if ( !is_hvm_domain(d) )
+ goto out;
+
+ rc = xsm_dm_op(XSM_DM_PRIV, d);
+ if ( rc )
+ goto out;
+
+ rc = -EFAULT;
+ if ( op_args->buf[0].size < offset )
+ goto out;
+
+ if ( copy_from_guest_offset((void *)&op, op_args->buf[0].h, 0, offset) )
+ goto out;
+
+ if ( op.op >= ARRAY_SIZE(op_size) )
+ {
+ rc = -EOPNOTSUPP;
+ goto out;
+ }
+
+ op.op = array_index_nospec(op.op, ARRAY_SIZE(op_size));
+
+ if ( op_args->buf[0].size < offset + op_size[op.op] )
+ goto out;
+
+ if ( copy_from_guest_offset((void *)&op.u, op_args->buf[0].h, offset,
+ op_size[op.op]) )
+ goto out;
+
+ rc = -EINVAL;
+ if ( op.pad )
+ goto out;
+
+ switch ( op.op )
+ {
+ case XEN_DMOP_create_ioreq_server:
+ {
+ struct xen_dm_op_create_ioreq_server *data =
+ &op.u.create_ioreq_server;
+
+ const_op = false;
+
+ rc = -EINVAL;
+ if ( data->pad[0] || data->pad[1] || data->pad[2] )
+ break;
+
+ rc = hvm_create_ioreq_server(d, data->handle_bufioreq,
+ &data->id);
+ break;
+ }
+
+ case XEN_DMOP_get_ioreq_server_info:
+ {
+ struct xen_dm_op_get_ioreq_server_info *data =
+ &op.u.get_ioreq_server_info;
+ const uint16_t valid_flags = XEN_DMOP_no_gfns;
+
+ const_op = false;
+
+ rc = -EINVAL;
+ if ( data->flags & ~valid_flags )
+ break;
+
+ rc = hvm_get_ioreq_server_info(d, data->id,
+ (data->flags & XEN_DMOP_no_gfns) ?
+ NULL : (unsigned long *)&data->ioreq_gfn,
+ (data->flags & XEN_DMOP_no_gfns) ?
+ NULL : (unsigned long *)&data->bufioreq_gfn,
+ &data->bufioreq_port);
+ break;
+ }
+
+ case XEN_DMOP_map_io_range_to_ioreq_server:
+ {
+ const struct xen_dm_op_ioreq_server_range *data =
+ &op.u.map_io_range_to_ioreq_server;
+
+ rc = -EINVAL;
+ if ( data->pad )
+ break;
+
+ rc = hvm_map_io_range_to_ioreq_server(d, data->id, data->type,
+ data->start, data->end);
+ break;
+ }
+
+ case XEN_DMOP_unmap_io_range_from_ioreq_server:
+ {
+ const struct xen_dm_op_ioreq_server_range *data =
+ &op.u.unmap_io_range_from_ioreq_server;
+
+ rc = -EINVAL;
+ if ( data->pad )
+ break;
+
+ rc = hvm_unmap_io_range_from_ioreq_server(d, data->id, data->type,
+ data->start, data->end);
+ break;
+ }
+
+ case XEN_DMOP_set_ioreq_server_state:
+ {
+ const struct xen_dm_op_set_ioreq_server_state *data =
+ &op.u.set_ioreq_server_state;
+
+ rc = -EINVAL;
+ if ( data->pad )
+ break;
+
+ rc = hvm_set_ioreq_server_state(d, data->id, !!data->enabled);
+ break;
+ }
+
+ case XEN_DMOP_destroy_ioreq_server:
+ {
+ const struct xen_dm_op_destroy_ioreq_server *data =
+ &op.u.destroy_ioreq_server;
+
+ rc = -EINVAL;
+ if ( data->pad )
+ break;
+
+ rc = hvm_destroy_ioreq_server(d, data->id);
+ break;
+ }
+
+ default:
+ rc = arch_dm_op(&op, d, op_args, &const_op);
+ }
+
+ if ( (!rc || rc == -ERESTART) &&
+ !const_op && copy_to_guest_offset(op_args->buf[0].h, offset,
+ (void *)&op.u, op_size[op.op]) )
+ rc = -EFAULT;
+
+ out:
+ rcu_unlock_domain(d);
+
+ return rc;
+}
+
+#ifdef CONFIG_COMPAT
+#include <compat/hvm/dm_op.h>
+
+CHECK_dm_op_create_ioreq_server;
+CHECK_dm_op_get_ioreq_server_info;
+CHECK_dm_op_ioreq_server_range;
+CHECK_dm_op_set_ioreq_server_state;
+CHECK_dm_op_destroy_ioreq_server;
+CHECK_dm_op_track_dirty_vram;
+CHECK_dm_op_set_pci_intx_level;
+CHECK_dm_op_set_isa_irq_level;
+CHECK_dm_op_set_pci_link_route;
+CHECK_dm_op_modified_memory;
+CHECK_dm_op_set_mem_type;
+CHECK_dm_op_inject_event;
+CHECK_dm_op_inject_msi;
+CHECK_dm_op_map_mem_type_to_ioreq_server;
+CHECK_dm_op_remote_shutdown;
+CHECK_dm_op_relocate_memory;
+CHECK_dm_op_pin_memory_cacheattr;
+
+int compat_dm_op(domid_t domid,
+ unsigned int nr_bufs,
+ XEN_GUEST_HANDLE_PARAM(void) bufs)
+{
+ struct dmop_args args;
+ unsigned int i;
+ int rc;
+
+ if ( nr_bufs > ARRAY_SIZE(args.buf) )
+ return -E2BIG;
+
+ args.domid = domid;
+ args.nr_bufs = array_index_nospec(nr_bufs, ARRAY_SIZE(args.buf) + 1);
+
+ for ( i = 0; i < args.nr_bufs; i++ )
+ {
+ struct compat_dm_op_buf cmp;
+
+ if ( copy_from_guest_offset(&cmp, bufs, i, 1) )
+ return -EFAULT;
+
+#define XLAT_dm_op_buf_HNDL_h(_d_, _s_) \
+ guest_from_compat_handle((_d_)->h, (_s_)->h)
+
+ XLAT_dm_op_buf(&args.buf[i], &cmp);
+
+#undef XLAT_dm_op_buf_HNDL_h
+ }
+
+ rc = dm_op(&args);
+
+ if ( rc == -ERESTART )
+ rc = hypercall_create_continuation(__HYPERVISOR_dm_op, "iih",
+ domid, nr_bufs, bufs);
+
+ return rc;
+}
+#endif
+
+long do_dm_op(domid_t domid,
+ unsigned int nr_bufs,
+ XEN_GUEST_HANDLE_PARAM(xen_dm_op_buf_t) bufs)
+{
+ struct dmop_args args;
+ int rc;
+
+ if ( nr_bufs > ARRAY_SIZE(args.buf) )
+ return -E2BIG;
+
+ args.domid = domid;
+ args.nr_bufs = array_index_nospec(nr_bufs, ARRAY_SIZE(args.buf) + 1);
+
+ if ( copy_from_guest_offset(&args.buf[0], bufs, 0, args.nr_bufs) )
+ return -EFAULT;
+
+ rc = dm_op(&args);
+
+ if ( rc == -ERESTART )
+ rc = hypercall_create_continuation(__HYPERVISOR_dm_op, "iih",
+ domid, nr_bufs, bufs);
+
+ return rc;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/xen/dm.h b/xen/include/xen/dm.h
new file mode 100644
index 0000000..ef15edf
--- /dev/null
+++ b/xen/include/xen/dm.h
@@ -0,0 +1,44 @@
+/*
+ * Copyright (c) 2016 Citrix Systems Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __XEN_DM_H__
+#define __XEN_DM_H__
+
+#include <xen/sched.h>
+
+struct dmop_args {
+ domid_t domid;
+ unsigned int nr_bufs;
+ /* Reserve enough buf elements for all current hypercalls. */
+ struct xen_dm_op_buf buf[2];
+};
+
+int arch_dm_op(struct xen_dm_op *op,
+ struct domain *d,
+ const struct dmop_args *op_args,
+ bool *const_op);
+
+#endif /* __XEN_DM_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index 7ae3c40..5c61d8e 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -707,14 +707,14 @@ static XSM_INLINE int xsm_pmu_op (XSM_DEFAULT_ARG struct domain *d, unsigned int
}
}
+#endif /* CONFIG_X86 */
+
static XSM_INLINE int xsm_dm_op(XSM_DEFAULT_ARG struct domain *d)
{
XSM_ASSERT_ACTION(XSM_DM_PRIV);
return xsm_default_action(action, current->domain, d);
}
-#endif /* CONFIG_X86 */
-
#ifdef CONFIG_ARGO
static XSM_INLINE int xsm_argo_enable(const struct domain *d)
{
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 358ec13..517f78a 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -177,8 +177,8 @@ struct xsm_operations {
int (*ioport_permission) (struct domain *d, uint32_t s, uint32_t e, uint8_t allow);
int (*ioport_mapping) (struct domain *d, uint32_t s, uint32_t e, uint8_t allow);
int (*pmu_op) (struct domain *d, unsigned int op);
- int (*dm_op) (struct domain *d);
#endif
+ int (*dm_op) (struct domain *d);
int (*xen_version) (uint32_t cmd);
int (*domain_resource_map) (struct domain *d);
#ifdef CONFIG_ARGO
@@ -683,13 +683,13 @@ static inline int xsm_pmu_op (xsm_default_t def, struct domain *d, unsigned int
return xsm_ops->pmu_op(d, op);
}
+#endif /* CONFIG_X86 */
+
static inline int xsm_dm_op(xsm_default_t def, struct domain *d)
{
return xsm_ops->dm_op(d);
}
-#endif /* CONFIG_X86 */
-
static inline int xsm_xen_version (xsm_default_t def, uint32_t op)
{
return xsm_ops->xen_version(op);
diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c
index 9e09512..8bdffe7 100644
--- a/xen/xsm/dummy.c
+++ b/xen/xsm/dummy.c
@@ -147,8 +147,8 @@ void __init xsm_fixup_ops (struct xsm_operations *ops)
set_to_dummy_if_null(ops, ioport_permission);
set_to_dummy_if_null(ops, ioport_mapping);
set_to_dummy_if_null(ops, pmu_op);
- set_to_dummy_if_null(ops, dm_op);
#endif
+ set_to_dummy_if_null(ops, dm_op);
set_to_dummy_if_null(ops, xen_version);
set_to_dummy_if_null(ops, domain_resource_map);
#ifdef CONFIG_ARGO
diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c
index de050cc..8f3f182 100644
--- a/xen/xsm/flask/hooks.c
+++ b/xen/xsm/flask/hooks.c
@@ -1666,14 +1666,13 @@ static int flask_pmu_op (struct domain *d, unsigned int op)
return -EPERM;
}
}
+#endif /* CONFIG_X86 */
static int flask_dm_op(struct domain *d)
{
return current_has_perm(d, SECCLASS_HVM, HVM__DM);
}
-#endif /* CONFIG_X86 */
-
static int flask_xen_version (uint32_t op)
{
u32 dsid = domain_sid(current->domain);
@@ -1875,8 +1874,8 @@ static struct xsm_operations flask_ops = {
.ioport_permission = flask_ioport_permission,
.ioport_mapping = flask_ioport_mapping,
.pmu_op = flask_pmu_op,
- .dm_op = flask_dm_op,
#endif
+ .dm_op = flask_dm_op,
.xen_version = flask_xen_version,
.domain_resource_map = flask_domain_resource_map,
#ifdef CONFIG_ARGO
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 10/23] xen/mm: Make x86's XENMEM_resource_ioreq_server handling common
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (8 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 09/23] xen/dm: Make x86's DM feature common Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-11-12 11:40 ` Jan Beulich
2020-10-15 16:44 ` [PATCH V2 11/23] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu Oleksandr Tyshchenko
` (13 subsequent siblings)
23 siblings, 1 reply; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Julien Grall, Jan Beulich, Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Volodymyr Babchuk, Oleksandr Tyshchenko
From: Julien Grall <julien.grall@arm.com>
As x86 implementation of XENMEM_resource_ioreq_server can be
re-used on Arm later on, this patch makes it common and removes
arch_acquire_resource as unneeded.
This support is going to be used on Arm to be able run device
emulator outside of Xen hypervisor.
Signed-off-by: Julien Grall <julien.grall@arm.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- no changes
Changes V1 -> V2:
- update the author of a patch
---
xen/arch/x86/mm.c | 44 --------------------------------------------
xen/common/memory.c | 45 +++++++++++++++++++++++++++++++++++++++++++--
xen/include/asm-arm/mm.h | 8 --------
xen/include/asm-x86/mm.h | 4 ----
4 files changed, 43 insertions(+), 58 deletions(-)
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index b5865ae..df7619d 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -4591,50 +4591,6 @@ int xenmem_add_to_physmap_one(
return rc;
}
-int arch_acquire_resource(struct domain *d, unsigned int type,
- unsigned int id, unsigned long frame,
- unsigned int nr_frames, xen_pfn_t mfn_list[])
-{
- int rc;
-
- switch ( type )
- {
-#ifdef CONFIG_HVM
- case XENMEM_resource_ioreq_server:
- {
- ioservid_t ioservid = id;
- unsigned int i;
-
- rc = -EINVAL;
- if ( !is_hvm_domain(d) )
- break;
-
- if ( id != (unsigned int)ioservid )
- break;
-
- rc = 0;
- for ( i = 0; i < nr_frames; i++ )
- {
- mfn_t mfn;
-
- rc = hvm_get_ioreq_server_frame(d, id, frame + i, &mfn);
- if ( rc )
- break;
-
- mfn_list[i] = mfn_x(mfn);
- }
- break;
- }
-#endif
-
- default:
- rc = -EOPNOTSUPP;
- break;
- }
-
- return rc;
-}
-
long arch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
{
int rc;
diff --git a/xen/common/memory.c b/xen/common/memory.c
index 1bab0e8..83d800f 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -30,6 +30,10 @@
#include <public/memory.h>
#include <xsm/xsm.h>
+#ifdef CONFIG_IOREQ_SERVER
+#include <xen/ioreq.h>
+#endif
+
#ifdef CONFIG_X86
#include <asm/guest.h>
#endif
@@ -1045,6 +1049,38 @@ static int acquire_grant_table(struct domain *d, unsigned int id,
return 0;
}
+#ifdef CONFIG_IOREQ_SERVER
+static int acquire_ioreq_server(struct domain *d,
+ unsigned int id,
+ unsigned long frame,
+ unsigned int nr_frames,
+ xen_pfn_t mfn_list[])
+{
+ ioservid_t ioservid = id;
+ unsigned int i;
+ int rc;
+
+ if ( !is_hvm_domain(d) )
+ return -EINVAL;
+
+ if ( id != (unsigned int)ioservid )
+ return -EINVAL;
+
+ for ( i = 0; i < nr_frames; i++ )
+ {
+ mfn_t mfn;
+
+ rc = hvm_get_ioreq_server_frame(d, id, frame + i, &mfn);
+ if ( rc )
+ return rc;
+
+ mfn_list[i] = mfn_x(mfn);
+ }
+
+ return 0;
+}
+#endif
+
static int acquire_resource(
XEN_GUEST_HANDLE_PARAM(xen_mem_acquire_resource_t) arg)
{
@@ -1103,9 +1139,14 @@ static int acquire_resource(
mfn_list);
break;
+#ifdef CONFIG_IOREQ_SERVER
+ case XENMEM_resource_ioreq_server:
+ rc = acquire_ioreq_server(d, xmar.id, xmar.frame, xmar.nr_frames,
+ mfn_list);
+ break;
+#endif
default:
- rc = arch_acquire_resource(d, xmar.type, xmar.id, xmar.frame,
- xmar.nr_frames, mfn_list);
+ rc = -EOPNOTSUPP;
break;
}
diff --git a/xen/include/asm-arm/mm.h b/xen/include/asm-arm/mm.h
index f8ba49b..0b7de31 100644
--- a/xen/include/asm-arm/mm.h
+++ b/xen/include/asm-arm/mm.h
@@ -358,14 +358,6 @@ static inline void put_page_and_type(struct page_info *page)
void clear_and_clean_page(struct page_info *page);
-static inline
-int arch_acquire_resource(struct domain *d, unsigned int type, unsigned int id,
- unsigned long frame, unsigned int nr_frames,
- xen_pfn_t mfn_list[])
-{
- return -EOPNOTSUPP;
-}
-
unsigned int arch_get_dma_bitsize(void);
#endif /* __ARCH_ARM_MM__ */
diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h
index deeba75..859214e 100644
--- a/xen/include/asm-x86/mm.h
+++ b/xen/include/asm-x86/mm.h
@@ -639,8 +639,4 @@ static inline bool arch_mfn_in_directmap(unsigned long mfn)
return mfn <= (virt_to_mfn(eva - 1) + 1);
}
-int arch_acquire_resource(struct domain *d, unsigned int type,
- unsigned int id, unsigned long frame,
- unsigned int nr_frames, xen_pfn_t mfn_list[]);
-
#endif /* __ASM_X86_MM_H__ */
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 11/23] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (9 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 10/23] xen/mm: Make x86's XENMEM_resource_ioreq_server handling common Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-10-20 10:55 ` Paul Durrant
2020-11-11 8:04 ` Jan Beulich
2020-10-15 16:44 ` [PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved function names Oleksandr Tyshchenko
` (12 subsequent siblings)
23 siblings, 2 replies; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
The IOREQ is a common feature now and these fields will be used
on Arm as is. Move them to common struct vcpu as a part of new
struct vcpu_io. Also move enum hvm_io_completion to xen/sched.h
and remove "hvm" prefixes.
This patch completely removes layering violation in the common code.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
***
I was thinking that it may be better to place these two fields
into struct vcpu directly (without intermediate "io" struct).
I think, this way the code which operates with these fields
would become cleaner. Another possible option would be either
to rename "io" struct (I failed to think of a better name) or
to drop(replace?) duplicating "io" prefixes from these fields.
***
Changes V1 -> V2:
- new patch
---
xen/arch/x86/hvm/emulate.c | 50 +++++++++++++++++++--------------------
xen/arch/x86/hvm/hvm.c | 2 +-
xen/arch/x86/hvm/io.c | 6 ++---
xen/arch/x86/hvm/svm/nestedsvm.c | 2 +-
xen/arch/x86/hvm/vmx/realmode.c | 6 ++---
xen/common/ioreq.c | 14 +++++------
xen/include/asm-x86/hvm/emulate.h | 2 +-
xen/include/asm-x86/hvm/ioreq.h | 4 ++--
xen/include/asm-x86/hvm/vcpu.h | 11 ---------
xen/include/xen/sched.h | 17 +++++++++++++
10 files changed, 60 insertions(+), 54 deletions(-)
diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index 4746d5a..f6a4eef 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -142,8 +142,8 @@ void hvmemul_cancel(struct vcpu *v)
{
struct hvm_vcpu_io *vio = &v->arch.hvm.hvm_io;
- vio->io_req.state = STATE_IOREQ_NONE;
- vio->io_completion = HVMIO_no_completion;
+ v->io.io_req.state = STATE_IOREQ_NONE;
+ v->io.io_completion = IO_no_completion;
vio->mmio_cache_count = 0;
vio->mmio_insn_bytes = 0;
vio->mmio_access = (struct npfec){};
@@ -159,7 +159,7 @@ static int hvmemul_do_io(
{
struct vcpu *curr = current;
struct domain *currd = curr->domain;
- struct hvm_vcpu_io *vio = &curr->arch.hvm.hvm_io;
+ struct vcpu_io *vio = &curr->io;
ioreq_t p = {
.type = is_mmio ? IOREQ_TYPE_COPY : IOREQ_TYPE_PIO,
.addr = addr,
@@ -1854,7 +1854,7 @@ static int hvmemul_rep_movs(
* cheaper than multiple round trips through the device model. Yet
* when processing a response we can always re-use the translation.
*/
- (vio->io_req.state == STATE_IORESP_READY ||
+ (curr->io.io_req.state == STATE_IORESP_READY ||
((!df || *reps == 1) &&
PAGE_SIZE - (saddr & ~PAGE_MASK) >= *reps * bytes_per_rep)) )
sgpa = pfn_to_paddr(vio->mmio_gpfn) | (saddr & ~PAGE_MASK);
@@ -1870,7 +1870,7 @@ static int hvmemul_rep_movs(
if ( vio->mmio_access.write_access &&
(vio->mmio_gla == (daddr & PAGE_MASK)) &&
/* See comment above. */
- (vio->io_req.state == STATE_IORESP_READY ||
+ (curr->io.io_req.state == STATE_IORESP_READY ||
((!df || *reps == 1) &&
PAGE_SIZE - (daddr & ~PAGE_MASK) >= *reps * bytes_per_rep)) )
dgpa = pfn_to_paddr(vio->mmio_gpfn) | (daddr & ~PAGE_MASK);
@@ -2007,7 +2007,7 @@ static int hvmemul_rep_stos(
if ( vio->mmio_access.write_access &&
(vio->mmio_gla == (addr & PAGE_MASK)) &&
/* See respective comment in MOVS processing. */
- (vio->io_req.state == STATE_IORESP_READY ||
+ (curr->io.io_req.state == STATE_IORESP_READY ||
((!df || *reps == 1) &&
PAGE_SIZE - (addr & ~PAGE_MASK) >= *reps * bytes_per_rep)) )
gpa = pfn_to_paddr(vio->mmio_gpfn) | (addr & ~PAGE_MASK);
@@ -2613,13 +2613,13 @@ static const struct x86_emulate_ops hvm_emulate_ops_no_write = {
};
/*
- * Note that passing HVMIO_no_completion into this function serves as kind
+ * Note that passing IO_no_completion into this function serves as kind
* of (but not fully) an "auto select completion" indicator. When there's
* no completion needed, the passed in value will be ignored in any case.
*/
static int _hvm_emulate_one(struct hvm_emulate_ctxt *hvmemul_ctxt,
const struct x86_emulate_ops *ops,
- enum hvm_io_completion completion)
+ enum io_completion completion)
{
const struct cpu_user_regs *regs = hvmemul_ctxt->ctxt.regs;
struct vcpu *curr = current;
@@ -2634,11 +2634,11 @@ static int _hvm_emulate_one(struct hvm_emulate_ctxt *hvmemul_ctxt,
*/
if ( vio->cache->num_ents > vio->cache->max_ents )
{
- ASSERT(vio->io_req.state == STATE_IOREQ_NONE);
+ ASSERT(curr->io.io_req.state == STATE_IOREQ_NONE);
vio->cache->num_ents = 0;
}
else
- ASSERT(vio->io_req.state == STATE_IORESP_READY);
+ ASSERT(curr->io.io_req.state == STATE_IORESP_READY);
hvm_emulate_init_per_insn(hvmemul_ctxt, vio->mmio_insn,
vio->mmio_insn_bytes);
@@ -2649,25 +2649,25 @@ static int _hvm_emulate_one(struct hvm_emulate_ctxt *hvmemul_ctxt,
if ( rc == X86EMUL_OKAY && vio->mmio_retry )
rc = X86EMUL_RETRY;
- if ( !ioreq_needs_completion(&vio->io_req) )
- completion = HVMIO_no_completion;
- else if ( completion == HVMIO_no_completion )
- completion = (vio->io_req.type != IOREQ_TYPE_PIO ||
- hvmemul_ctxt->is_mem_access) ? HVMIO_mmio_completion
- : HVMIO_pio_completion;
+ if ( !ioreq_needs_completion(&curr->io.io_req) )
+ completion = IO_no_completion;
+ else if ( completion == IO_no_completion )
+ completion = (curr->io.io_req.type != IOREQ_TYPE_PIO ||
+ hvmemul_ctxt->is_mem_access) ? IO_mmio_completion
+ : IO_pio_completion;
- switch ( vio->io_completion = completion )
+ switch ( curr->io.io_completion = completion )
{
- case HVMIO_no_completion:
- case HVMIO_pio_completion:
+ case IO_no_completion:
+ case IO_pio_completion:
vio->mmio_cache_count = 0;
vio->mmio_insn_bytes = 0;
vio->mmio_access = (struct npfec){};
hvmemul_cache_disable(curr);
break;
- case HVMIO_mmio_completion:
- case HVMIO_realmode_completion:
+ case IO_mmio_completion:
+ case IO_realmode_completion:
BUILD_BUG_ON(sizeof(vio->mmio_insn) < sizeof(hvmemul_ctxt->insn_buf));
vio->mmio_insn_bytes = hvmemul_ctxt->insn_buf_bytes;
memcpy(vio->mmio_insn, hvmemul_ctxt->insn_buf, vio->mmio_insn_bytes);
@@ -2716,7 +2716,7 @@ static int _hvm_emulate_one(struct hvm_emulate_ctxt *hvmemul_ctxt,
int hvm_emulate_one(
struct hvm_emulate_ctxt *hvmemul_ctxt,
- enum hvm_io_completion completion)
+ enum io_completion completion)
{
return _hvm_emulate_one(hvmemul_ctxt, &hvm_emulate_ops, completion);
}
@@ -2754,7 +2754,7 @@ int hvm_emulate_one_mmio(unsigned long mfn, unsigned long gla)
guest_cpu_user_regs());
ctxt.ctxt.data = &mmio_ro_ctxt;
- switch ( rc = _hvm_emulate_one(&ctxt, ops, HVMIO_no_completion) )
+ switch ( rc = _hvm_emulate_one(&ctxt, ops, IO_no_completion) )
{
case X86EMUL_UNHANDLEABLE:
case X86EMUL_UNIMPLEMENTED:
@@ -2782,7 +2782,7 @@ void hvm_emulate_one_vm_event(enum emul_kind kind, unsigned int trapnr,
{
case EMUL_KIND_NOWRITE:
rc = _hvm_emulate_one(&ctx, &hvm_emulate_ops_no_write,
- HVMIO_no_completion);
+ IO_no_completion);
break;
case EMUL_KIND_SET_CONTEXT_INSN: {
struct vcpu *curr = current;
@@ -2803,7 +2803,7 @@ void hvm_emulate_one_vm_event(enum emul_kind kind, unsigned int trapnr,
/* Fall-through */
default:
ctx.set_context = (kind == EMUL_KIND_SET_CONTEXT_DATA);
- rc = hvm_emulate_one(&ctx, HVMIO_no_completion);
+ rc = hvm_emulate_one(&ctx, IO_no_completion);
}
switch ( rc )
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 20376ce..341093b 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -3800,7 +3800,7 @@ void hvm_ud_intercept(struct cpu_user_regs *regs)
return;
}
- switch ( hvm_emulate_one(&ctxt, HVMIO_no_completion) )
+ switch ( hvm_emulate_one(&ctxt, IO_no_completion) )
{
case X86EMUL_UNHANDLEABLE:
case X86EMUL_UNIMPLEMENTED:
diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
index b220d6b..36584de 100644
--- a/xen/arch/x86/hvm/io.c
+++ b/xen/arch/x86/hvm/io.c
@@ -85,7 +85,7 @@ bool hvm_emulate_one_insn(hvm_emulate_validate_t *validate, const char *descr)
hvm_emulate_init_once(&ctxt, validate, guest_cpu_user_regs());
- switch ( rc = hvm_emulate_one(&ctxt, HVMIO_no_completion) )
+ switch ( rc = hvm_emulate_one(&ctxt, IO_no_completion) )
{
case X86EMUL_UNHANDLEABLE:
hvm_dump_emulation_state(XENLOG_G_WARNING, descr, &ctxt, rc);
@@ -122,7 +122,7 @@ bool handle_mmio_with_translation(unsigned long gla, unsigned long gpfn,
bool handle_pio(uint16_t port, unsigned int size, int dir)
{
struct vcpu *curr = current;
- struct hvm_vcpu_io *vio = &curr->arch.hvm.hvm_io;
+ struct vcpu_io *vio = &curr->io;
unsigned int data;
int rc;
@@ -136,7 +136,7 @@ bool handle_pio(uint16_t port, unsigned int size, int dir)
rc = hvmemul_do_pio_buffer(port, size, dir, &data);
if ( ioreq_needs_completion(&vio->io_req) )
- vio->io_completion = HVMIO_pio_completion;
+ vio->io_completion = IO_pio_completion;
switch ( rc )
{
diff --git a/xen/arch/x86/hvm/svm/nestedsvm.c b/xen/arch/x86/hvm/svm/nestedsvm.c
index fcfccf7..787d4a0 100644
--- a/xen/arch/x86/hvm/svm/nestedsvm.c
+++ b/xen/arch/x86/hvm/svm/nestedsvm.c
@@ -1266,7 +1266,7 @@ enum hvm_intblk nsvm_intr_blocked(struct vcpu *v)
* Delay the injection because this would result in delivering
* an interrupt *within* the execution of an instruction.
*/
- if ( v->arch.hvm.hvm_io.io_req.state != STATE_IOREQ_NONE )
+ if ( v->io.io_req.state != STATE_IOREQ_NONE )
return hvm_intblk_shadow;
if ( !nv->nv_vmexit_pending && n2vmcb->exit_int_info.v )
diff --git a/xen/arch/x86/hvm/vmx/realmode.c b/xen/arch/x86/hvm/vmx/realmode.c
index 768f01e..f5832a0 100644
--- a/xen/arch/x86/hvm/vmx/realmode.c
+++ b/xen/arch/x86/hvm/vmx/realmode.c
@@ -101,7 +101,7 @@ void vmx_realmode_emulate_one(struct hvm_emulate_ctxt *hvmemul_ctxt)
perfc_incr(realmode_emulations);
- rc = hvm_emulate_one(hvmemul_ctxt, HVMIO_realmode_completion);
+ rc = hvm_emulate_one(hvmemul_ctxt, IO_realmode_completion);
if ( rc == X86EMUL_UNHANDLEABLE )
{
@@ -188,7 +188,7 @@ void vmx_realmode(struct cpu_user_regs *regs)
vmx_realmode_emulate_one(&hvmemul_ctxt);
- if ( vio->io_req.state != STATE_IOREQ_NONE || vio->mmio_retry )
+ if ( curr->io.io_req.state != STATE_IOREQ_NONE || vio->mmio_retry )
break;
/* Stop emulating unless our segment state is not safe */
@@ -202,7 +202,7 @@ void vmx_realmode(struct cpu_user_regs *regs)
}
/* Need to emulate next time if we've started an IO operation */
- if ( vio->io_req.state != STATE_IOREQ_NONE )
+ if ( curr->io.io_req.state != STATE_IOREQ_NONE )
curr->arch.hvm.vmx.vmx_emulate = 1;
if ( !curr->arch.hvm.vmx.vmx_emulate && !curr->arch.hvm.vmx.vmx_realmode )
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index a07f1d7..57ddaaa 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -158,7 +158,7 @@ static bool hvm_wait_for_io(struct ioreq_vcpu *sv, ioreq_t *p)
break;
}
- p = &sv->vcpu->arch.hvm.hvm_io.io_req;
+ p = &sv->vcpu->io.io_req;
if ( ioreq_needs_completion(p) )
p->data = data;
@@ -170,10 +170,10 @@ static bool hvm_wait_for_io(struct ioreq_vcpu *sv, ioreq_t *p)
bool handle_hvm_io_completion(struct vcpu *v)
{
struct domain *d = v->domain;
- struct hvm_vcpu_io *vio = &v->arch.hvm.hvm_io;
+ struct vcpu_io *vio = &v->io;
struct ioreq_server *s;
struct ioreq_vcpu *sv;
- enum hvm_io_completion io_completion;
+ enum io_completion io_completion;
if ( has_vpci(d) && vpci_process_pending(v) )
{
@@ -192,17 +192,17 @@ bool handle_hvm_io_completion(struct vcpu *v)
vcpu_end_shutdown_deferral(v);
io_completion = vio->io_completion;
- vio->io_completion = HVMIO_no_completion;
+ vio->io_completion = IO_no_completion;
switch ( io_completion )
{
- case HVMIO_no_completion:
+ case IO_no_completion:
break;
- case HVMIO_mmio_completion:
+ case IO_mmio_completion:
return ioreq_complete_mmio();
- case HVMIO_pio_completion:
+ case IO_pio_completion:
return handle_pio(vio->io_req.addr, vio->io_req.size,
vio->io_req.dir);
diff --git a/xen/include/asm-x86/hvm/emulate.h b/xen/include/asm-x86/hvm/emulate.h
index 1620cc7..131cdf4 100644
--- a/xen/include/asm-x86/hvm/emulate.h
+++ b/xen/include/asm-x86/hvm/emulate.h
@@ -65,7 +65,7 @@ bool __nonnull(1, 2) hvm_emulate_one_insn(
const char *descr);
int hvm_emulate_one(
struct hvm_emulate_ctxt *hvmemul_ctxt,
- enum hvm_io_completion completion);
+ enum io_completion completion);
void hvm_emulate_one_vm_event(enum emul_kind kind,
unsigned int trapnr,
unsigned int errcode);
diff --git a/xen/include/asm-x86/hvm/ioreq.h b/xen/include/asm-x86/hvm/ioreq.h
index 0fccac5..5ed977e 100644
--- a/xen/include/asm-x86/hvm/ioreq.h
+++ b/xen/include/asm-x86/hvm/ioreq.h
@@ -26,11 +26,11 @@
#include <public/hvm/params.h>
-static inline bool arch_hvm_io_completion(enum hvm_io_completion io_completion)
+static inline bool arch_hvm_io_completion(enum io_completion io_completion)
{
switch ( io_completion )
{
- case HVMIO_realmode_completion:
+ case IO_realmode_completion:
{
struct hvm_emulate_ctxt ctxt;
diff --git a/xen/include/asm-x86/hvm/vcpu.h b/xen/include/asm-x86/hvm/vcpu.h
index 6c1feda..8adf455 100644
--- a/xen/include/asm-x86/hvm/vcpu.h
+++ b/xen/include/asm-x86/hvm/vcpu.h
@@ -28,13 +28,6 @@
#include <asm/mtrr.h>
#include <public/hvm/ioreq.h>
-enum hvm_io_completion {
- HVMIO_no_completion,
- HVMIO_mmio_completion,
- HVMIO_pio_completion,
- HVMIO_realmode_completion
-};
-
struct hvm_vcpu_asid {
uint64_t generation;
uint32_t asid;
@@ -52,10 +45,6 @@ struct hvm_mmio_cache {
};
struct hvm_vcpu_io {
- /* I/O request in flight to device model. */
- enum hvm_io_completion io_completion;
- ioreq_t io_req;
-
/*
* HVM emulation:
* Linear address @mmio_gla maps to MMIO physical frame @mmio_gpfn.
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 78761cd..f9ce14c 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -143,6 +143,19 @@ void evtchn_destroy_final(struct domain *d); /* from complete_domain_destroy */
struct waitqueue_vcpu;
+enum io_completion {
+ IO_no_completion,
+ IO_mmio_completion,
+ IO_pio_completion,
+ IO_realmode_completion
+};
+
+struct vcpu_io {
+ /* I/O request in flight to device model. */
+ enum io_completion io_completion;
+ ioreq_t io_req;
+};
+
struct vcpu
{
int vcpu_id;
@@ -254,6 +267,10 @@ struct vcpu
struct vpci_vcpu vpci;
struct arch_vcpu arch;
+
+#ifdef CONFIG_IOREQ_SERVER
+ struct vcpu_io io;
+#endif
};
struct sched_unit {
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved function names
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (10 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 11/23] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-11-12 11:45 ` Jan Beulich
2020-10-15 16:44 ` [PATCH V2 13/23] xen/ioreq: Use guest_cmpxchg64() instead of cmpxchg() Oleksandr Tyshchenko
` (11 subsequent siblings)
23 siblings, 1 reply; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
This patch removes "hvm" prefixes and infixes from IOREQ
related function names in the common code.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes V1 -> V2:
- new patch
---
xen/arch/x86/hvm/emulate.c | 6 +-
xen/arch/x86/hvm/hvm.c | 10 +-
xen/arch/x86/hvm/io.c | 6 +-
xen/arch/x86/hvm/stdvga.c | 4 +-
xen/arch/x86/hvm/vmx/vvmx.c | 2 +-
xen/common/dm.c | 28 ++---
xen/common/ioreq.c | 240 ++++++++++++++++++++--------------------
xen/common/memory.c | 2 +-
xen/include/asm-x86/hvm/ioreq.h | 16 +--
xen/include/xen/ioreq.h | 58 +++++-----
10 files changed, 186 insertions(+), 186 deletions(-)
diff --git a/xen/arch/x86/hvm/emulate.c b/xen/arch/x86/hvm/emulate.c
index f6a4eef..54cd493 100644
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -261,7 +261,7 @@ static int hvmemul_do_io(
* an ioreq server that can handle it.
*
* Rules:
- * A> PIO or MMIO accesses run through hvm_select_ioreq_server() to
+ * A> PIO or MMIO accesses run through select_ioreq_server() to
* choose the ioreq server by range. If no server is found, the access
* is ignored.
*
@@ -323,7 +323,7 @@ static int hvmemul_do_io(
}
if ( !s )
- s = hvm_select_ioreq_server(currd, &p);
+ s = select_ioreq_server(currd, &p);
/* If there is no suitable backing DM, just ignore accesses */
if ( !s )
@@ -333,7 +333,7 @@ static int hvmemul_do_io(
}
else
{
- rc = hvm_send_ioreq(s, &p, 0);
+ rc = send_ioreq(s, &p, 0);
if ( rc != X86EMUL_RETRY || currd->is_shutting_down )
vio->io_req.state = STATE_IOREQ_NONE;
else if ( !ioreq_needs_completion(&vio->io_req) )
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 341093b..1e788b5 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -546,7 +546,7 @@ void hvm_do_resume(struct vcpu *v)
pt_restore_timer(v);
- if ( !handle_hvm_io_completion(v) )
+ if ( !handle_io_completion(v) )
return;
if ( unlikely(v->arch.vm_event) )
@@ -677,7 +677,7 @@ int hvm_domain_initialise(struct domain *d)
register_g2m_portio_handler(d);
register_vpci_portio_handler(d);
- hvm_ioreq_init(d);
+ ioreq_init(d);
hvm_init_guest_time(d);
@@ -739,7 +739,7 @@ void hvm_domain_relinquish_resources(struct domain *d)
viridian_domain_deinit(d);
- hvm_destroy_all_ioreq_servers(d);
+ destroy_all_ioreq_servers(d);
msixtbl_pt_cleanup(d);
@@ -1582,7 +1582,7 @@ int hvm_vcpu_initialise(struct vcpu *v)
if ( rc )
goto fail5;
- rc = hvm_all_ioreq_servers_add_vcpu(d, v);
+ rc = all_ioreq_servers_add_vcpu(d, v);
if ( rc != 0 )
goto fail6;
@@ -1618,7 +1618,7 @@ void hvm_vcpu_destroy(struct vcpu *v)
{
viridian_vcpu_deinit(v);
- hvm_all_ioreq_servers_remove_vcpu(v->domain, v);
+ all_ioreq_servers_remove_vcpu(v->domain, v);
if ( hvm_altp2m_supported() )
altp2m_vcpu_destroy(v);
diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
index 36584de..2d03ffe 100644
--- a/xen/arch/x86/hvm/io.c
+++ b/xen/arch/x86/hvm/io.c
@@ -60,7 +60,7 @@ void send_timeoffset_req(unsigned long timeoff)
if ( timeoff == 0 )
return;
- if ( hvm_broadcast_ioreq(&p, true) != 0 )
+ if ( broadcast_ioreq(&p, true) != 0 )
gprintk(XENLOG_ERR, "Unsuccessful timeoffset update\n");
}
@@ -74,7 +74,7 @@ void send_invalidate_req(void)
.data = ~0UL, /* flush all */
};
- if ( hvm_broadcast_ioreq(&p, false) != 0 )
+ if ( broadcast_ioreq(&p, false) != 0 )
gprintk(XENLOG_ERR, "Unsuccessful map-cache invalidate\n");
}
@@ -155,7 +155,7 @@ bool handle_pio(uint16_t port, unsigned int size, int dir)
* We should not advance RIP/EIP if the domain is shutting down or
* if X86EMUL_RETRY has been returned by an internal handler.
*/
- if ( curr->domain->is_shutting_down || !hvm_io_pending(curr) )
+ if ( curr->domain->is_shutting_down || !io_pending(curr) )
return false;
break;
diff --git a/xen/arch/x86/hvm/stdvga.c b/xen/arch/x86/hvm/stdvga.c
index bafb3f6..cb1cc7f 100644
--- a/xen/arch/x86/hvm/stdvga.c
+++ b/xen/arch/x86/hvm/stdvga.c
@@ -507,11 +507,11 @@ static int stdvga_mem_write(const struct hvm_io_handler *handler,
}
done:
- srv = hvm_select_ioreq_server(current->domain, &p);
+ srv = select_ioreq_server(current->domain, &p);
if ( !srv )
return X86EMUL_UNHANDLEABLE;
- return hvm_send_ioreq(srv, &p, 1);
+ return send_ioreq(srv, &p, 1);
}
static bool_t stdvga_mem_accept(const struct hvm_io_handler *handler,
diff --git a/xen/arch/x86/hvm/vmx/vvmx.c b/xen/arch/x86/hvm/vmx/vvmx.c
index 3a37e9e..d5a17f12 100644
--- a/xen/arch/x86/hvm/vmx/vvmx.c
+++ b/xen/arch/x86/hvm/vmx/vvmx.c
@@ -1516,7 +1516,7 @@ void nvmx_switch_guest(void)
* don't want to continue as this setup is not implemented nor supported
* as of right now.
*/
- if ( hvm_io_pending(v) )
+ if ( io_pending(v) )
return;
/*
* a softirq may interrupt us between a virtual vmentry is
diff --git a/xen/common/dm.c b/xen/common/dm.c
index 36e01a2..f3a8353 100644
--- a/xen/common/dm.c
+++ b/xen/common/dm.c
@@ -100,8 +100,8 @@ static int dm_op(const struct dmop_args *op_args)
if ( data->pad[0] || data->pad[1] || data->pad[2] )
break;
- rc = hvm_create_ioreq_server(d, data->handle_bufioreq,
- &data->id);
+ rc = create_ioreq_server(d, data->handle_bufioreq,
+ &data->id);
break;
}
@@ -117,12 +117,12 @@ static int dm_op(const struct dmop_args *op_args)
if ( data->flags & ~valid_flags )
break;
- rc = hvm_get_ioreq_server_info(d, data->id,
- (data->flags & XEN_DMOP_no_gfns) ?
- NULL : (unsigned long *)&data->ioreq_gfn,
- (data->flags & XEN_DMOP_no_gfns) ?
- NULL : (unsigned long *)&data->bufioreq_gfn,
- &data->bufioreq_port);
+ rc = get_ioreq_server_info(d, data->id,
+ (data->flags & XEN_DMOP_no_gfns) ?
+ NULL : (unsigned long *)&data->ioreq_gfn,
+ (data->flags & XEN_DMOP_no_gfns) ?
+ NULL : (unsigned long *)&data->bufioreq_gfn,
+ &data->bufioreq_port);
break;
}
@@ -135,8 +135,8 @@ static int dm_op(const struct dmop_args *op_args)
if ( data->pad )
break;
- rc = hvm_map_io_range_to_ioreq_server(d, data->id, data->type,
- data->start, data->end);
+ rc = map_io_range_to_ioreq_server(d, data->id, data->type,
+ data->start, data->end);
break;
}
@@ -149,8 +149,8 @@ static int dm_op(const struct dmop_args *op_args)
if ( data->pad )
break;
- rc = hvm_unmap_io_range_from_ioreq_server(d, data->id, data->type,
- data->start, data->end);
+ rc = unmap_io_range_from_ioreq_server(d, data->id, data->type,
+ data->start, data->end);
break;
}
@@ -163,7 +163,7 @@ static int dm_op(const struct dmop_args *op_args)
if ( data->pad )
break;
- rc = hvm_set_ioreq_server_state(d, data->id, !!data->enabled);
+ rc = set_ioreq_server_state(d, data->id, !!data->enabled);
break;
}
@@ -176,7 +176,7 @@ static int dm_op(const struct dmop_args *op_args)
if ( data->pad )
break;
- rc = hvm_destroy_ioreq_server(d, data->id);
+ rc = destroy_ioreq_server(d, data->id);
break;
}
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index 57ddaaa..98fffae 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -58,7 +58,7 @@ struct ioreq_server *get_ioreq_server(const struct domain *d,
* Iterate over all possible ioreq servers.
*
* NOTE: The iteration is backwards such that more recently created
- * ioreq servers are favoured in hvm_select_ioreq_server().
+ * ioreq servers are favoured in select_ioreq_server().
* This is a semantic that previously existed when ioreq servers
* were held in a linked list.
*/
@@ -105,12 +105,12 @@ static struct ioreq_vcpu *get_pending_vcpu(const struct vcpu *v,
return NULL;
}
-bool hvm_io_pending(struct vcpu *v)
+bool io_pending(struct vcpu *v)
{
return get_pending_vcpu(v, NULL);
}
-static bool hvm_wait_for_io(struct ioreq_vcpu *sv, ioreq_t *p)
+static bool wait_for_io(struct ioreq_vcpu *sv, ioreq_t *p)
{
unsigned int prev_state = STATE_IOREQ_NONE;
unsigned int state = p->state;
@@ -167,7 +167,7 @@ static bool hvm_wait_for_io(struct ioreq_vcpu *sv, ioreq_t *p)
return true;
}
-bool handle_hvm_io_completion(struct vcpu *v)
+bool handle_io_completion(struct vcpu *v)
{
struct domain *d = v->domain;
struct vcpu_io *vio = &v->io;
@@ -182,7 +182,7 @@ bool handle_hvm_io_completion(struct vcpu *v)
}
sv = get_pending_vcpu(v, &s);
- if ( sv && !hvm_wait_for_io(sv, get_ioreq(s, v)) )
+ if ( sv && !wait_for_io(sv, get_ioreq(s, v)) )
return false;
vio->io_req.state = ioreq_needs_completion(&vio->io_req) ?
@@ -207,13 +207,13 @@ bool handle_hvm_io_completion(struct vcpu *v)
vio->io_req.dir);
default:
- return arch_hvm_io_completion(io_completion);
+ return arch_io_completion(io_completion);
}
return true;
}
-static gfn_t hvm_alloc_legacy_ioreq_gfn(struct ioreq_server *s)
+static gfn_t alloc_legacy_ioreq_gfn(struct ioreq_server *s)
{
struct domain *d = s->target;
unsigned int i;
@@ -229,7 +229,7 @@ static gfn_t hvm_alloc_legacy_ioreq_gfn(struct ioreq_server *s)
return INVALID_GFN;
}
-static gfn_t hvm_alloc_ioreq_gfn(struct ioreq_server *s)
+static gfn_t alloc_ioreq_gfn(struct ioreq_server *s)
{
struct domain *d = s->target;
unsigned int i;
@@ -244,11 +244,11 @@ static gfn_t hvm_alloc_ioreq_gfn(struct ioreq_server *s)
* If we are out of 'normal' GFNs then we may still have a 'legacy'
* GFN available.
*/
- return hvm_alloc_legacy_ioreq_gfn(s);
+ return alloc_legacy_ioreq_gfn(s);
}
-static bool hvm_free_legacy_ioreq_gfn(struct ioreq_server *s,
- gfn_t gfn)
+static bool free_legacy_ioreq_gfn(struct ioreq_server *s,
+ gfn_t gfn)
{
struct domain *d = s->target;
unsigned int i;
@@ -265,21 +265,21 @@ static bool hvm_free_legacy_ioreq_gfn(struct ioreq_server *s,
return true;
}
-static void hvm_free_ioreq_gfn(struct ioreq_server *s, gfn_t gfn)
+static void free_ioreq_gfn(struct ioreq_server *s, gfn_t gfn)
{
struct domain *d = s->target;
unsigned int i = gfn_x(gfn) - d->ioreq_gfn.base;
ASSERT(!gfn_eq(gfn, INVALID_GFN));
- if ( !hvm_free_legacy_ioreq_gfn(s, gfn) )
+ if ( !free_legacy_ioreq_gfn(s, gfn) )
{
ASSERT(i < sizeof(d->ioreq_gfn.mask) * 8);
set_bit(i, &d->ioreq_gfn.mask);
}
}
-static void hvm_unmap_ioreq_gfn(struct ioreq_server *s, bool buf)
+static void unmap_ioreq_gfn(struct ioreq_server *s, bool buf)
{
struct ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
@@ -289,11 +289,11 @@ static void hvm_unmap_ioreq_gfn(struct ioreq_server *s, bool buf)
destroy_ring_for_helper(&iorp->va, iorp->page);
iorp->page = NULL;
- hvm_free_ioreq_gfn(s, iorp->gfn);
+ free_ioreq_gfn(s, iorp->gfn);
iorp->gfn = INVALID_GFN;
}
-static int hvm_map_ioreq_gfn(struct ioreq_server *s, bool buf)
+static int map_ioreq_gfn(struct ioreq_server *s, bool buf)
{
struct domain *d = s->target;
struct ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
@@ -303,7 +303,7 @@ static int hvm_map_ioreq_gfn(struct ioreq_server *s, bool buf)
{
/*
* If a page has already been allocated (which will happen on
- * demand if hvm_get_ioreq_server_frame() is called), then
+ * demand if get_ioreq_server_frame() is called), then
* mapping a guest frame is not permitted.
*/
if ( gfn_eq(iorp->gfn, INVALID_GFN) )
@@ -315,7 +315,7 @@ static int hvm_map_ioreq_gfn(struct ioreq_server *s, bool buf)
if ( d->is_dying )
return -EINVAL;
- iorp->gfn = hvm_alloc_ioreq_gfn(s);
+ iorp->gfn = alloc_ioreq_gfn(s);
if ( gfn_eq(iorp->gfn, INVALID_GFN) )
return -ENOMEM;
@@ -324,12 +324,12 @@ static int hvm_map_ioreq_gfn(struct ioreq_server *s, bool buf)
&iorp->va);
if ( rc )
- hvm_unmap_ioreq_gfn(s, buf);
+ unmap_ioreq_gfn(s, buf);
return rc;
}
-static int hvm_alloc_ioreq_mfn(struct ioreq_server *s, bool buf)
+static int alloc_ioreq_mfn(struct ioreq_server *s, bool buf)
{
struct ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
struct page_info *page;
@@ -338,7 +338,7 @@ static int hvm_alloc_ioreq_mfn(struct ioreq_server *s, bool buf)
{
/*
* If a guest frame has already been mapped (which may happen
- * on demand if hvm_get_ioreq_server_info() is called), then
+ * on demand if get_ioreq_server_info() is called), then
* allocating a page is not permitted.
*/
if ( !gfn_eq(iorp->gfn, INVALID_GFN) )
@@ -377,7 +377,7 @@ static int hvm_alloc_ioreq_mfn(struct ioreq_server *s, bool buf)
return -ENOMEM;
}
-static void hvm_free_ioreq_mfn(struct ioreq_server *s, bool buf)
+static void free_ioreq_mfn(struct ioreq_server *s, bool buf)
{
struct ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
struct page_info *page = iorp->page;
@@ -416,7 +416,7 @@ bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
return found;
}
-static void hvm_remove_ioreq_gfn(struct ioreq_server *s, bool buf)
+static void remove_ioreq_gfn(struct ioreq_server *s, bool buf)
{
struct domain *d = s->target;
@@ -431,7 +431,7 @@ static void hvm_remove_ioreq_gfn(struct ioreq_server *s, bool buf)
clear_page(iorp->va);
}
-static int hvm_add_ioreq_gfn(struct ioreq_server *s, bool buf)
+static int add_ioreq_gfn(struct ioreq_server *s, bool buf)
{
struct domain *d = s->target;
struct ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
@@ -450,8 +450,8 @@ static int hvm_add_ioreq_gfn(struct ioreq_server *s, bool buf)
return rc;
}
-static void hvm_update_ioreq_evtchn(struct ioreq_server *s,
- struct ioreq_vcpu *sv)
+static void update_ioreq_evtchn(struct ioreq_server *s,
+ struct ioreq_vcpu *sv)
{
ASSERT(spin_is_locked(&s->lock));
@@ -466,8 +466,8 @@ static void hvm_update_ioreq_evtchn(struct ioreq_server *s,
#define HANDLE_BUFIOREQ(s) \
((s)->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF)
-static int hvm_ioreq_server_add_vcpu(struct ioreq_server *s,
- struct vcpu *v)
+static int ioreq_server_add_vcpu(struct ioreq_server *s,
+ struct vcpu *v)
{
struct ioreq_vcpu *sv;
int rc;
@@ -502,7 +502,7 @@ static int hvm_ioreq_server_add_vcpu(struct ioreq_server *s,
list_add(&sv->list_entry, &s->ioreq_vcpu_list);
if ( s->enabled )
- hvm_update_ioreq_evtchn(s, sv);
+ update_ioreq_evtchn(s, sv);
spin_unlock(&s->lock);
return 0;
@@ -518,8 +518,8 @@ static int hvm_ioreq_server_add_vcpu(struct ioreq_server *s,
return rc;
}
-static void hvm_ioreq_server_remove_vcpu(struct ioreq_server *s,
- struct vcpu *v)
+static void ioreq_server_remove_vcpu(struct ioreq_server *s,
+ struct vcpu *v)
{
struct ioreq_vcpu *sv;
@@ -546,7 +546,7 @@ static void hvm_ioreq_server_remove_vcpu(struct ioreq_server *s,
spin_unlock(&s->lock);
}
-static void hvm_ioreq_server_remove_all_vcpus(struct ioreq_server *s)
+static void ioreq_server_remove_all_vcpus(struct ioreq_server *s)
{
struct ioreq_vcpu *sv, *next;
@@ -572,49 +572,49 @@ static void hvm_ioreq_server_remove_all_vcpus(struct ioreq_server *s)
spin_unlock(&s->lock);
}
-static int hvm_ioreq_server_map_pages(struct ioreq_server *s)
+static int ioreq_server_map_pages(struct ioreq_server *s)
{
int rc;
- rc = hvm_map_ioreq_gfn(s, false);
+ rc = map_ioreq_gfn(s, false);
if ( !rc && HANDLE_BUFIOREQ(s) )
- rc = hvm_map_ioreq_gfn(s, true);
+ rc = map_ioreq_gfn(s, true);
if ( rc )
- hvm_unmap_ioreq_gfn(s, false);
+ unmap_ioreq_gfn(s, false);
return rc;
}
-static void hvm_ioreq_server_unmap_pages(struct ioreq_server *s)
+static void ioreq_server_unmap_pages(struct ioreq_server *s)
{
- hvm_unmap_ioreq_gfn(s, true);
- hvm_unmap_ioreq_gfn(s, false);
+ unmap_ioreq_gfn(s, true);
+ unmap_ioreq_gfn(s, false);
}
-static int hvm_ioreq_server_alloc_pages(struct ioreq_server *s)
+static int ioreq_server_alloc_pages(struct ioreq_server *s)
{
int rc;
- rc = hvm_alloc_ioreq_mfn(s, false);
+ rc = alloc_ioreq_mfn(s, false);
if ( !rc && (s->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF) )
- rc = hvm_alloc_ioreq_mfn(s, true);
+ rc = alloc_ioreq_mfn(s, true);
if ( rc )
- hvm_free_ioreq_mfn(s, false);
+ free_ioreq_mfn(s, false);
return rc;
}
-static void hvm_ioreq_server_free_pages(struct ioreq_server *s)
+static void ioreq_server_free_pages(struct ioreq_server *s)
{
- hvm_free_ioreq_mfn(s, true);
- hvm_free_ioreq_mfn(s, false);
+ free_ioreq_mfn(s, true);
+ free_ioreq_mfn(s, false);
}
-static void hvm_ioreq_server_free_rangesets(struct ioreq_server *s)
+static void ioreq_server_free_rangesets(struct ioreq_server *s)
{
unsigned int i;
@@ -622,8 +622,8 @@ static void hvm_ioreq_server_free_rangesets(struct ioreq_server *s)
rangeset_destroy(s->range[i]);
}
-static int hvm_ioreq_server_alloc_rangesets(struct ioreq_server *s,
- ioservid_t id)
+static int ioreq_server_alloc_rangesets(struct ioreq_server *s,
+ ioservid_t id)
{
unsigned int i;
int rc;
@@ -655,12 +655,12 @@ static int hvm_ioreq_server_alloc_rangesets(struct ioreq_server *s,
return 0;
fail:
- hvm_ioreq_server_free_rangesets(s);
+ ioreq_server_free_rangesets(s);
return rc;
}
-static void hvm_ioreq_server_enable(struct ioreq_server *s)
+static void ioreq_server_enable(struct ioreq_server *s)
{
struct ioreq_vcpu *sv;
@@ -669,29 +669,29 @@ static void hvm_ioreq_server_enable(struct ioreq_server *s)
if ( s->enabled )
goto done;
- hvm_remove_ioreq_gfn(s, false);
- hvm_remove_ioreq_gfn(s, true);
+ remove_ioreq_gfn(s, false);
+ remove_ioreq_gfn(s, true);
s->enabled = true;
list_for_each_entry ( sv,
&s->ioreq_vcpu_list,
list_entry )
- hvm_update_ioreq_evtchn(s, sv);
+ update_ioreq_evtchn(s, sv);
done:
spin_unlock(&s->lock);
}
-static void hvm_ioreq_server_disable(struct ioreq_server *s)
+static void ioreq_server_disable(struct ioreq_server *s)
{
spin_lock(&s->lock);
if ( !s->enabled )
goto done;
- hvm_add_ioreq_gfn(s, true);
- hvm_add_ioreq_gfn(s, false);
+ add_ioreq_gfn(s, true);
+ add_ioreq_gfn(s, false);
s->enabled = false;
@@ -699,9 +699,9 @@ static void hvm_ioreq_server_disable(struct ioreq_server *s)
spin_unlock(&s->lock);
}
-static int hvm_ioreq_server_init(struct ioreq_server *s,
- struct domain *d, int bufioreq_handling,
- ioservid_t id)
+static int ioreq_server_init(struct ioreq_server *s,
+ struct domain *d, int bufioreq_handling,
+ ioservid_t id)
{
struct domain *currd = current->domain;
struct vcpu *v;
@@ -719,7 +719,7 @@ static int hvm_ioreq_server_init(struct ioreq_server *s,
s->ioreq.gfn = INVALID_GFN;
s->bufioreq.gfn = INVALID_GFN;
- rc = hvm_ioreq_server_alloc_rangesets(s, id);
+ rc = ioreq_server_alloc_rangesets(s, id);
if ( rc )
return rc;
@@ -727,7 +727,7 @@ static int hvm_ioreq_server_init(struct ioreq_server *s,
for_each_vcpu ( d, v )
{
- rc = hvm_ioreq_server_add_vcpu(s, v);
+ rc = ioreq_server_add_vcpu(s, v);
if ( rc )
goto fail_add;
}
@@ -735,39 +735,39 @@ static int hvm_ioreq_server_init(struct ioreq_server *s,
return 0;
fail_add:
- hvm_ioreq_server_remove_all_vcpus(s);
- hvm_ioreq_server_unmap_pages(s);
+ ioreq_server_remove_all_vcpus(s);
+ ioreq_server_unmap_pages(s);
- hvm_ioreq_server_free_rangesets(s);
+ ioreq_server_free_rangesets(s);
put_domain(s->emulator);
return rc;
}
-static void hvm_ioreq_server_deinit(struct ioreq_server *s)
+static void ioreq_server_deinit(struct ioreq_server *s)
{
ASSERT(!s->enabled);
- hvm_ioreq_server_remove_all_vcpus(s);
+ ioreq_server_remove_all_vcpus(s);
/*
- * NOTE: It is safe to call both hvm_ioreq_server_unmap_pages() and
- * hvm_ioreq_server_free_pages() in that order.
+ * NOTE: It is safe to call both ioreq_server_unmap_pages() and
+ * ioreq_server_free_pages() in that order.
* This is because the former will do nothing if the pages
* are not mapped, leaving the page to be freed by the latter.
* However if the pages are mapped then the former will set
* the page_info pointer to NULL, meaning the latter will do
* nothing.
*/
- hvm_ioreq_server_unmap_pages(s);
- hvm_ioreq_server_free_pages(s);
+ ioreq_server_unmap_pages(s);
+ ioreq_server_free_pages(s);
- hvm_ioreq_server_free_rangesets(s);
+ ioreq_server_free_rangesets(s);
put_domain(s->emulator);
}
-int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
- ioservid_t *id)
+int create_ioreq_server(struct domain *d, int bufioreq_handling,
+ ioservid_t *id)
{
struct ioreq_server *s;
unsigned int i;
@@ -795,11 +795,11 @@ int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
/*
* It is safe to call set_ioreq_server() prior to
- * hvm_ioreq_server_init() since the target domain is paused.
+ * ioreq_server_init() since the target domain is paused.
*/
set_ioreq_server(d, i, s);
- rc = hvm_ioreq_server_init(s, d, bufioreq_handling, i);
+ rc = ioreq_server_init(s, d, bufioreq_handling, i);
if ( rc )
{
set_ioreq_server(d, i, NULL);
@@ -822,7 +822,7 @@ int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
return rc;
}
-int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
+int destroy_ioreq_server(struct domain *d, ioservid_t id)
{
struct ioreq_server *s;
int rc;
@@ -841,15 +841,15 @@ int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
domain_pause(d);
- arch_hvm_destroy_ioreq_server(s);
+ arch_destroy_ioreq_server(s);
- hvm_ioreq_server_disable(s);
+ ioreq_server_disable(s);
/*
- * It is safe to call hvm_ioreq_server_deinit() prior to
+ * It is safe to call ioreq_server_deinit() prior to
* set_ioreq_server() since the target domain is paused.
*/
- hvm_ioreq_server_deinit(s);
+ ioreq_server_deinit(s);
set_ioreq_server(d, id, NULL);
domain_unpause(d);
@@ -864,10 +864,10 @@ int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
return rc;
}
-int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
- unsigned long *ioreq_gfn,
- unsigned long *bufioreq_gfn,
- evtchn_port_t *bufioreq_port)
+int get_ioreq_server_info(struct domain *d, ioservid_t id,
+ unsigned long *ioreq_gfn,
+ unsigned long *bufioreq_gfn,
+ evtchn_port_t *bufioreq_port)
{
struct ioreq_server *s;
int rc;
@@ -886,7 +886,7 @@ int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
if ( ioreq_gfn || bufioreq_gfn )
{
- rc = hvm_ioreq_server_map_pages(s);
+ rc = ioreq_server_map_pages(s);
if ( rc )
goto out;
}
@@ -911,8 +911,8 @@ int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
return rc;
}
-int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
- unsigned long idx, mfn_t *mfn)
+int get_ioreq_server_frame(struct domain *d, ioservid_t id,
+ unsigned long idx, mfn_t *mfn)
{
struct ioreq_server *s;
int rc;
@@ -931,7 +931,7 @@ int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
if ( s->emulator != current->domain )
goto out;
- rc = hvm_ioreq_server_alloc_pages(s);
+ rc = ioreq_server_alloc_pages(s);
if ( rc )
goto out;
@@ -962,9 +962,9 @@ int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
return rc;
}
-int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint64_t start,
- uint64_t end)
+int map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
+ uint32_t type, uint64_t start,
+ uint64_t end)
{
struct ioreq_server *s;
struct rangeset *r;
@@ -1014,9 +1014,9 @@ int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
return rc;
}
-int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint64_t start,
- uint64_t end)
+int unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
+ uint32_t type, uint64_t start,
+ uint64_t end)
{
struct ioreq_server *s;
struct rangeset *r;
@@ -1066,8 +1066,8 @@ int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
return rc;
}
-int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
- bool enabled)
+int set_ioreq_server_state(struct domain *d, ioservid_t id,
+ bool enabled)
{
struct ioreq_server *s;
int rc;
@@ -1087,9 +1087,9 @@ int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
domain_pause(d);
if ( enabled )
- hvm_ioreq_server_enable(s);
+ ioreq_server_enable(s);
else
- hvm_ioreq_server_disable(s);
+ ioreq_server_disable(s);
domain_unpause(d);
@@ -1100,7 +1100,7 @@ int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
return rc;
}
-int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
+int all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
{
struct ioreq_server *s;
unsigned int id;
@@ -1110,7 +1110,7 @@ int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
FOR_EACH_IOREQ_SERVER(d, id, s)
{
- rc = hvm_ioreq_server_add_vcpu(s, v);
+ rc = ioreq_server_add_vcpu(s, v);
if ( rc )
goto fail;
}
@@ -1127,7 +1127,7 @@ int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
if ( !s )
continue;
- hvm_ioreq_server_remove_vcpu(s, v);
+ ioreq_server_remove_vcpu(s, v);
}
spin_unlock_recursive(&d->ioreq_server.lock);
@@ -1135,7 +1135,7 @@ int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v)
return rc;
}
-void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v)
+void all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v)
{
struct ioreq_server *s;
unsigned int id;
@@ -1143,17 +1143,17 @@ void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v)
spin_lock_recursive(&d->ioreq_server.lock);
FOR_EACH_IOREQ_SERVER(d, id, s)
- hvm_ioreq_server_remove_vcpu(s, v);
+ ioreq_server_remove_vcpu(s, v);
spin_unlock_recursive(&d->ioreq_server.lock);
}
-void hvm_destroy_all_ioreq_servers(struct domain *d)
+void destroy_all_ioreq_servers(struct domain *d)
{
struct ioreq_server *s;
unsigned int id;
- if ( !arch_hvm_ioreq_destroy(d) )
+ if ( !arch_ioreq_destroy(d) )
return;
spin_lock_recursive(&d->ioreq_server.lock);
@@ -1162,13 +1162,13 @@ void hvm_destroy_all_ioreq_servers(struct domain *d)
FOR_EACH_IOREQ_SERVER(d, id, s)
{
- hvm_ioreq_server_disable(s);
+ ioreq_server_disable(s);
/*
- * It is safe to call hvm_ioreq_server_deinit() prior to
+ * It is safe to call ioreq_server_deinit() prior to
* set_ioreq_server() since the target domain is being destroyed.
*/
- hvm_ioreq_server_deinit(s);
+ ioreq_server_deinit(s);
set_ioreq_server(d, id, NULL);
xfree(s);
@@ -1177,15 +1177,15 @@ void hvm_destroy_all_ioreq_servers(struct domain *d)
spin_unlock_recursive(&d->ioreq_server.lock);
}
-struct ioreq_server *hvm_select_ioreq_server(struct domain *d,
- ioreq_t *p)
+struct ioreq_server *select_ioreq_server(struct domain *d,
+ ioreq_t *p)
{
struct ioreq_server *s;
uint8_t type;
uint64_t addr;
unsigned int id;
- if ( hvm_ioreq_server_get_type_addr(d, p, &type, &addr) )
+ if ( ioreq_server_get_type_addr(d, p, &type, &addr) )
return NULL;
FOR_EACH_IOREQ_SERVER(d, id, s)
@@ -1233,7 +1233,7 @@ struct ioreq_server *hvm_select_ioreq_server(struct domain *d,
return NULL;
}
-static int hvm_send_buffered_ioreq(struct ioreq_server *s, ioreq_t *p)
+static int send_buffered_ioreq(struct ioreq_server *s, ioreq_t *p)
{
struct domain *d = current->domain;
struct ioreq_page *iorp;
@@ -1326,8 +1326,8 @@ static int hvm_send_buffered_ioreq(struct ioreq_server *s, ioreq_t *p)
return IOREQ_STATUS_HANDLED;
}
-int hvm_send_ioreq(struct ioreq_server *s, ioreq_t *proto_p,
- bool buffered)
+int send_ioreq(struct ioreq_server *s, ioreq_t *proto_p,
+ bool buffered)
{
struct vcpu *curr = current;
struct domain *d = curr->domain;
@@ -1336,7 +1336,7 @@ int hvm_send_ioreq(struct ioreq_server *s, ioreq_t *proto_p,
ASSERT(s);
if ( buffered )
- return hvm_send_buffered_ioreq(s, proto_p);
+ return send_buffered_ioreq(s, proto_p);
if ( unlikely(!vcpu_start_shutdown_deferral(curr)) )
return IOREQ_STATUS_RETRY;
@@ -1386,7 +1386,7 @@ int hvm_send_ioreq(struct ioreq_server *s, ioreq_t *proto_p,
return IOREQ_STATUS_UNHANDLED;
}
-unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered)
+unsigned int broadcast_ioreq(ioreq_t *p, bool buffered)
{
struct domain *d = current->domain;
struct ioreq_server *s;
@@ -1397,18 +1397,18 @@ unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered)
if ( !s->enabled )
continue;
- if ( hvm_send_ioreq(s, p, buffered) == IOREQ_STATUS_UNHANDLED )
+ if ( send_ioreq(s, p, buffered) == IOREQ_STATUS_UNHANDLED )
failed++;
}
return failed;
}
-void hvm_ioreq_init(struct domain *d)
+void ioreq_init(struct domain *d)
{
spin_lock_init(&d->ioreq_server.lock);
- arch_hvm_ioreq_init(d);
+ arch_ioreq_init(d);
}
/*
diff --git a/xen/common/memory.c b/xen/common/memory.c
index 83d800f..cf53ca3 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -1070,7 +1070,7 @@ static int acquire_ioreq_server(struct domain *d,
{
mfn_t mfn;
- rc = hvm_get_ioreq_server_frame(d, id, frame + i, &mfn);
+ rc = get_ioreq_server_frame(d, id, frame + i, &mfn);
if ( rc )
return rc;
diff --git a/xen/include/asm-x86/hvm/ioreq.h b/xen/include/asm-x86/hvm/ioreq.h
index 5ed977e..1340441 100644
--- a/xen/include/asm-x86/hvm/ioreq.h
+++ b/xen/include/asm-x86/hvm/ioreq.h
@@ -26,7 +26,7 @@
#include <public/hvm/params.h>
-static inline bool arch_hvm_io_completion(enum io_completion io_completion)
+static inline bool arch_io_completion(enum io_completion io_completion)
{
switch ( io_completion )
{
@@ -50,7 +50,7 @@ static inline bool arch_hvm_io_completion(enum io_completion io_completion)
}
/* Called when target domain is paused */
-static inline void arch_hvm_destroy_ioreq_server(struct ioreq_server *s)
+static inline void arch_destroy_ioreq_server(struct ioreq_server *s)
{
p2m_set_ioreq_server(s->target, 0, s);
}
@@ -105,10 +105,10 @@ static inline int hvm_map_mem_type_to_ioreq_server(struct domain *d,
return rc;
}
-static inline int hvm_ioreq_server_get_type_addr(const struct domain *d,
- const ioreq_t *p,
- uint8_t *type,
- uint64_t *addr)
+static inline int ioreq_server_get_type_addr(const struct domain *d,
+ const ioreq_t *p,
+ uint8_t *type,
+ uint64_t *addr)
{
uint32_t cf8 = d->arch.hvm.pci_cf8;
@@ -164,12 +164,12 @@ static inline int hvm_access_cf8(
return X86EMUL_UNHANDLEABLE;
}
-static inline void arch_hvm_ioreq_init(struct domain *d)
+static inline void arch_ioreq_init(struct domain *d)
{
register_portio_handler(d, 0xcf8, 4, hvm_access_cf8);
}
-static inline bool arch_hvm_ioreq_destroy(struct domain *d)
+static inline bool arch_ioreq_destroy(struct domain *d)
{
if ( !relocate_portio_handler(d, 0xcf8, 0xcf8, 4) )
return false;
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
index 8451866..7b03ab5 100644
--- a/xen/include/xen/ioreq.h
+++ b/xen/include/xen/ioreq.h
@@ -81,39 +81,39 @@ static inline bool ioreq_needs_completion(const ioreq_t *ioreq)
(ioreq->type != IOREQ_TYPE_PIO || ioreq->dir != IOREQ_WRITE);
}
-bool hvm_io_pending(struct vcpu *v);
-bool handle_hvm_io_completion(struct vcpu *v);
+bool io_pending(struct vcpu *v);
+bool handle_io_completion(struct vcpu *v);
bool is_ioreq_server_page(struct domain *d, const struct page_info *page);
-int hvm_create_ioreq_server(struct domain *d, int bufioreq_handling,
- ioservid_t *id);
-int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id);
-int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
- unsigned long *ioreq_gfn,
- unsigned long *bufioreq_gfn,
- evtchn_port_t *bufioreq_port);
-int hvm_get_ioreq_server_frame(struct domain *d, ioservid_t id,
- unsigned long idx, mfn_t *mfn);
-int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
+int create_ioreq_server(struct domain *d, int bufioreq_handling,
+ ioservid_t *id);
+int destroy_ioreq_server(struct domain *d, ioservid_t id);
+int get_ioreq_server_info(struct domain *d, ioservid_t id,
+ unsigned long *ioreq_gfn,
+ unsigned long *bufioreq_gfn,
+ evtchn_port_t *bufioreq_port);
+int get_ioreq_server_frame(struct domain *d, ioservid_t id,
+ unsigned long idx, mfn_t *mfn);
+int map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
+ uint32_t type, uint64_t start,
+ uint64_t end);
+int unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
uint32_t type, uint64_t start,
uint64_t end);
-int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
- uint32_t type, uint64_t start,
- uint64_t end);
-int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
- bool enabled);
-
-int hvm_all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v);
-void hvm_all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v);
-void hvm_destroy_all_ioreq_servers(struct domain *d);
-
-struct ioreq_server *hvm_select_ioreq_server(struct domain *d,
- ioreq_t *p);
-int hvm_send_ioreq(struct ioreq_server *s, ioreq_t *proto_p,
- bool buffered);
-unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered);
-
-void hvm_ioreq_init(struct domain *d);
+int set_ioreq_server_state(struct domain *d, ioservid_t id,
+ bool enabled);
+
+int all_ioreq_servers_add_vcpu(struct domain *d, struct vcpu *v);
+void all_ioreq_servers_remove_vcpu(struct domain *d, struct vcpu *v);
+void destroy_all_ioreq_servers(struct domain *d);
+
+struct ioreq_server *select_ioreq_server(struct domain *d,
+ ioreq_t *p);
+int send_ioreq(struct ioreq_server *s, ioreq_t *proto_p,
+ bool buffered);
+unsigned int broadcast_ioreq(ioreq_t *p, bool buffered);
+
+void ioreq_init(struct domain *d);
#endif /* __XEN_IOREQ_H__ */
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 13/23] xen/ioreq: Use guest_cmpxchg64() instead of cmpxchg()
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (11 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved function names Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-10-15 16:44 ` [PATCH V2 14/23] arm/ioreq: Introduce arch specific bits for IOREQ/DM features Oleksandr Tyshchenko
` (10 subsequent siblings)
23 siblings, 0 replies; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Paul Durrant, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
The cmpxchg() in hvm_send_buffered_ioreq() operates on memory shared
with the emulator domain (and the target domain if the legacy
interface is used).
In order to be on the safe side we need to switch
to guest_cmpxchg64() to prevent a domain to DoS Xen on Arm.
As there is no plan to support the legacy interface on Arm,
we will have a page to be mapped in a single domain at the time,
so we can use s->emulator in guest_cmpxchg64() safely.
Thankfully the only user of the legacy interface is x86 so far
and there is not concern regarding the atomics operations.
Please note, that the legacy interface *must* not be used on Arm
without revisiting the code.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- new patch
Changes V1 -> V2:
- move earlier to avoid breaking arm32 compilation
- add an explanation to commit description and hvm_allow_set_param()
- pass s->emulator
---
xen/arch/arm/hvm.c | 4 ++++
xen/common/ioreq.c | 3 ++-
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/xen/arch/arm/hvm.c b/xen/arch/arm/hvm.c
index 8951b34..9694e5a 100644
--- a/xen/arch/arm/hvm.c
+++ b/xen/arch/arm/hvm.c
@@ -31,6 +31,10 @@
#include <asm/hypercall.h>
+/*
+ * The legacy interface (which involves magic IOREQ pages) *must* not be used
+ * without revisiting the code.
+ */
static int hvm_allow_set_param(const struct domain *d, unsigned int param)
{
switch ( param )
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index 98fffae..8612159 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -28,6 +28,7 @@
#include <xen/trace.h>
#include <xen/vpci.h>
+#include <asm/guest_atomics.h>
#include <asm/hvm/ioreq.h>
#include <public/hvm/ioreq.h>
@@ -1317,7 +1318,7 @@ static int send_buffered_ioreq(struct ioreq_server *s, ioreq_t *p)
new.read_pointer = old.read_pointer - n * IOREQ_BUFFER_SLOT_NUM;
new.write_pointer = old.write_pointer - n * IOREQ_BUFFER_SLOT_NUM;
- cmpxchg(&pg->ptrs.full, old.full, new.full);
+ guest_cmpxchg64(s->emulator, &pg->ptrs.full, old.full, new.full);
}
notify_via_xen_event_channel(d, s->bufioreq_evtchn);
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 14/23] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (12 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 13/23] xen/ioreq: Use guest_cmpxchg64() instead of cmpxchg() Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-11-12 11:48 ` Jan Beulich
2020-10-15 16:44 ` [PATCH V2 15/23] xen/arm: Stick around in leave_hypervisor_to_guest until I/O has completed Oleksandr Tyshchenko
` (9 subsequent siblings)
23 siblings, 1 reply; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Julien Grall, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Paul Durrant, Oleksandr Tyshchenko
From: Julien Grall <julien.grall@arm.com>
This patch adds basic IOREQ/DM support on Arm. The subsequent
patches will improve functionality and add remaining bits.
The IOREQ/DM features are supposed to be built with IOREQ_SERVER
option enabled, which is disabled by default on Arm for now.
Please note, the "PIO handling" TODO is expected to left unaddressed
for the current series. It is not an big issue for now while Xen
doesn't have support for vPCI on Arm. On Arm64 they are only used
for PCI IO Bar and we would probably want to expose them to emulator
as PIO access to make a DM completely arch-agnostic. So "PIO handling"
should be implemented when we add support for vPCI.
Signed-off-by: Julien Grall <julien.grall@arm.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- was split into:
- arm/ioreq: Introduce arch specific bits for IOREQ/DM features
- xen/mm: Handle properly reference in set_foreign_p2m_entry() on Arm
- update patch description
- update asm-arm/hvm/ioreq.h according to the newly introduced arch functions:
- arch_hvm_destroy_ioreq_server()
- arch_handle_hvm_io_completion()
- update arch files to include xen/ioreq.h
- remove HVMOP plumbing
- rewrite a logic to handle properly case when hvm_send_ioreq() returns IO_RETRY
- add a logic to handle properly handle_hvm_io_completion() return value
- rename handle_mmio() to ioreq_handle_complete_mmio()
- move paging_mark_pfn_dirty() to asm-arm/paging.h
- remove forward declaration for hvm_ioreq_server in asm-arm/paging.h
- move try_fwd_ioserv() to ioreq.c, provide stubs if !CONFIG_IOREQ_SERVER
- do not remove #ifdef CONFIG_IOREQ_SERVER in memory.c for guarding xen/ioreq.h
- use gdprintk in try_fwd_ioserv(), remove unneeded prints
- update list of #include-s
- move has_vpci() to asm-arm/domain.h
- add a comment (TODO) to unimplemented yet handle_pio()
- remove hvm_mmio_first(last)_byte() and hvm_ioreq_(page/vcpu/server) structs
from the arch files, they were already moved to the common code
- remove set_foreign_p2m_entry() changes, they will be properly implemented
in the follow-up patch
- select IOREQ_SERVER for Arm instead of Arm64 in Kconfig
- remove x86's realmode and other unneeded stubs from xen/ioreq.h
- clafify ioreq_t p.df usage in try_fwd_ioserv()
- set ioreq_t p.count to 1 in try_fwd_ioserv()
Changes V2 -> V1:
- was split into:
- arm/ioreq: Introduce arch specific bits for IOREQ/DM features
- xen/arm: Stick around in leave_hypervisor_to_guest until I/O has completed
- update the author of a patch
- update patch description
- move a loop in leave_hypervisor_to_guest() to a separate patch
- set IOREQ_SERVER disabled by default
- remove already clarified /* XXX */
- replace BUG() by ASSERT_UNREACHABLE() in handle_pio()
- remove default case for handling the return value of try_handle_mmio()
- remove struct hvm_domain, enum hvm_io_completion, struct hvm_vcpu_io,
struct hvm_vcpu from asm-arm/domain.h, these are common materials now
- update everything according to the recent changes (IOREQ related function
names don't contain "hvm" prefixes/infixes anymore, IOREQ related fields
are part of common struct vcpu/domain now, etc)
---
xen/arch/arm/Makefile | 2 +
xen/arch/arm/dm.c | 34 ++++++++++
xen/arch/arm/domain.c | 9 +++
xen/arch/arm/io.c | 11 +++-
xen/arch/arm/ioreq.c | 141 ++++++++++++++++++++++++++++++++++++++++
xen/arch/arm/traps.c | 13 ++++
xen/common/ioreq.c | 1 +
xen/include/asm-arm/domain.h | 5 ++
xen/include/asm-arm/hvm/ioreq.h | 109 +++++++++++++++++++++++++++++++
xen/include/asm-arm/mmio.h | 1 +
xen/include/asm-arm/paging.h | 4 ++
11 files changed, 329 insertions(+), 1 deletion(-)
create mode 100644 xen/arch/arm/dm.c
create mode 100644 xen/arch/arm/ioreq.c
create mode 100644 xen/include/asm-arm/hvm/ioreq.h
diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile
index 296c5e6..c3ff454 100644
--- a/xen/arch/arm/Makefile
+++ b/xen/arch/arm/Makefile
@@ -13,6 +13,7 @@ obj-y += cpuerrata.o
obj-y += cpufeature.o
obj-y += decode.o
obj-y += device.o
+obj-$(CONFIG_IOREQ_SERVER) += dm.o
obj-y += domain.o
obj-y += domain_build.init.o
obj-y += domctl.o
@@ -27,6 +28,7 @@ obj-y += guest_atomics.o
obj-y += guest_walk.o
obj-y += hvm.o
obj-y += io.o
+obj-$(CONFIG_IOREQ_SERVER) += ioreq.o
obj-y += irq.o
obj-y += kernel.init.o
obj-$(CONFIG_LIVEPATCH) += livepatch.o
diff --git a/xen/arch/arm/dm.c b/xen/arch/arm/dm.c
new file mode 100644
index 0000000..5d3da37
--- /dev/null
+++ b/xen/arch/arm/dm.c
@@ -0,0 +1,34 @@
+/*
+ * Copyright (c) 2019 Arm ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/dm.h>
+#include <xen/hypercall.h>
+
+int arch_dm_op(struct xen_dm_op *op, struct domain *d,
+ const struct dmop_args *op_args, bool *const_op)
+{
+ return -EOPNOTSUPP;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 3b37f89..ba9b1fb 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -15,6 +15,7 @@
#include <xen/guest_access.h>
#include <xen/hypercall.h>
#include <xen/init.h>
+#include <xen/ioreq.h>
#include <xen/lib.h>
#include <xen/livepatch.h>
#include <xen/sched.h>
@@ -694,6 +695,10 @@ int arch_domain_create(struct domain *d,
ASSERT(config != NULL);
+#ifdef CONFIG_IOREQ_SERVER
+ ioreq_init(d);
+#endif
+
/* p2m_init relies on some value initialized by the IOMMU subsystem */
if ( (rc = iommu_domain_init(d, config->iommu_opts)) != 0 )
goto fail;
@@ -1012,6 +1017,10 @@ int domain_relinquish_resources(struct domain *d)
if (ret )
return ret;
+#ifdef CONFIG_IOREQ_SERVER
+ destroy_all_ioreq_servers(d);
+#endif
+
PROGRESS(xen):
ret = relinquish_memory(d, &d->xenpage_list);
if ( ret )
diff --git a/xen/arch/arm/io.c b/xen/arch/arm/io.c
index ae7ef96..f44cfd4 100644
--- a/xen/arch/arm/io.c
+++ b/xen/arch/arm/io.c
@@ -23,6 +23,7 @@
#include <asm/cpuerrata.h>
#include <asm/current.h>
#include <asm/mmio.h>
+#include <asm/hvm/ioreq.h>
#include "decode.h"
@@ -123,7 +124,15 @@ enum io_state try_handle_mmio(struct cpu_user_regs *regs,
handler = find_mmio_handler(v->domain, info.gpa);
if ( !handler )
- return IO_UNHANDLED;
+ {
+ int rc;
+
+ rc = try_fwd_ioserv(regs, v, &info);
+ if ( rc == IO_HANDLED )
+ return handle_ioserv(regs, v);
+
+ return rc;
+ }
/* All the instructions used on emulated MMIO region should be valid */
if ( !dabt.valid )
diff --git a/xen/arch/arm/ioreq.c b/xen/arch/arm/ioreq.c
new file mode 100644
index 0000000..da5ceac
--- /dev/null
+++ b/xen/arch/arm/ioreq.c
@@ -0,0 +1,141 @@
+/*
+ * arm/ioreq.c: hardware virtual machine I/O emulation
+ *
+ * Copyright (c) 2019 Arm ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/domain.h>
+#include <xen/ioreq.h>
+
+#include <asm/traps.h>
+
+#include <public/hvm/ioreq.h>
+
+enum io_state handle_ioserv(struct cpu_user_regs *regs, struct vcpu *v)
+{
+ const union hsr hsr = { .bits = regs->hsr };
+ const struct hsr_dabt dabt = hsr.dabt;
+ /* Code is similar to handle_read */
+ uint8_t size = (1 << dabt.size) * 8;
+ register_t r = v->io.io_req.data;
+
+ /* We are done with the IO */
+ v->io.io_req.state = STATE_IOREQ_NONE;
+
+ if ( dabt.write )
+ return IO_HANDLED;
+
+ /*
+ * Sign extend if required.
+ * Note that we expect the read handler to have zeroed the bits
+ * outside the requested access size.
+ */
+ if ( dabt.sign && (r & (1UL << (size - 1))) )
+ {
+ /*
+ * We are relying on register_t using the same as
+ * an unsigned long in order to keep the 32-bit assembly
+ * code smaller.
+ */
+ BUILD_BUG_ON(sizeof(register_t) != sizeof(unsigned long));
+ r |= (~0UL) << size;
+ }
+
+ set_user_reg(regs, dabt.reg, r);
+
+ return IO_HANDLED;
+}
+
+enum io_state try_fwd_ioserv(struct cpu_user_regs *regs,
+ struct vcpu *v, mmio_info_t *info)
+{
+ struct vcpu_io *vio = &v->io;
+ ioreq_t p = {
+ .type = IOREQ_TYPE_COPY,
+ .addr = info->gpa,
+ .size = 1 << info->dabt.size,
+ .count = 1,
+ .dir = !info->dabt.write,
+ /*
+ * On x86, df is used by 'rep' instruction to tell the direction
+ * to iterate (forward or backward).
+ * On Arm, all the accesses to MMIO region will do a single
+ * memory access. So for now, we can safely always set to 0.
+ */
+ .df = 0,
+ .data = get_user_reg(regs, info->dabt.reg),
+ .state = STATE_IOREQ_READY,
+ };
+ struct ioreq_server *s = NULL;
+ enum io_state rc;
+
+ switch ( vio->io_req.state )
+ {
+ case STATE_IOREQ_NONE:
+ break;
+
+ case STATE_IORESP_READY:
+ return IO_HANDLED;
+
+ default:
+ gdprintk(XENLOG_ERR, "wrong state %u\n", vio->io_req.state);
+ return IO_ABORT;
+ }
+
+ s = select_ioreq_server(v->domain, &p);
+ if ( !s )
+ return IO_UNHANDLED;
+
+ if ( !info->dabt.valid )
+ return IO_ABORT;
+
+ vio->io_req = p;
+
+ rc = send_ioreq(s, &p, 0);
+ if ( rc != IO_RETRY || v->domain->is_shutting_down )
+ vio->io_req.state = STATE_IOREQ_NONE;
+ else if ( !ioreq_needs_completion(&vio->io_req) )
+ rc = IO_HANDLED;
+ else
+ vio->io_completion = IO_mmio_completion;
+
+ return rc;
+}
+
+bool ioreq_complete_mmio(void)
+{
+ struct vcpu *v = current;
+ struct cpu_user_regs *regs = guest_cpu_user_regs();
+ const union hsr hsr = { .bits = regs->hsr };
+ paddr_t addr = v->io.io_req.addr;
+
+ if ( try_handle_mmio(regs, hsr, addr) == IO_HANDLED )
+ {
+ advance_pc(regs, hsr);
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 8f40d0e..b154837 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -21,6 +21,7 @@
#include <xen/hypercall.h>
#include <xen/init.h>
#include <xen/iocap.h>
+#include <xen/ioreq.h>
#include <xen/irq.h>
#include <xen/lib.h>
#include <xen/mem_access.h>
@@ -1384,6 +1385,9 @@ static arm_hypercall_t arm_hypercall_table[] = {
#ifdef CONFIG_HYPFS
HYPERCALL(hypfs_op, 5),
#endif
+#ifdef CONFIG_IOREQ_SERVER
+ HYPERCALL(dm_op, 3),
+#endif
};
#ifndef NDEBUG
@@ -1955,6 +1959,9 @@ static void do_trap_stage2_abort_guest(struct cpu_user_regs *regs,
case IO_HANDLED:
advance_pc(regs, hsr);
return;
+ case IO_RETRY:
+ /* finish later */
+ return;
case IO_UNHANDLED:
/* IO unhandled, try another way to handle it. */
break;
@@ -2253,6 +2260,12 @@ static void check_for_vcpu_work(void)
{
struct vcpu *v = current;
+#ifdef CONFIG_IOREQ_SERVER
+ local_irq_enable();
+ handle_io_completion(v);
+ local_irq_disable();
+#endif
+
if ( likely(!v->arch.need_flush_to_ram) )
return;
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index 8612159..bcd4961 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -18,6 +18,7 @@
#include <xen/ctype.h>
#include <xen/domain.h>
+#include <xen/domain_page.h>
#include <xen/event.h>
#include <xen/init.h>
#include <xen/irq.h>
diff --git a/xen/include/asm-arm/domain.h b/xen/include/asm-arm/domain.h
index 6819a3b..d4c3da5 100644
--- a/xen/include/asm-arm/domain.h
+++ b/xen/include/asm-arm/domain.h
@@ -10,6 +10,7 @@
#include <asm/gic.h>
#include <asm/vgic.h>
#include <asm/vpl011.h>
+#include <public/hvm/dm_op.h>
#include <public/hvm/params.h>
struct hvm_domain
@@ -17,6 +18,8 @@ struct hvm_domain
uint64_t params[HVM_NR_PARAMS];
};
+#define ioreq_params(d, i) ((d)->arch.hvm.params[i])
+
#ifdef CONFIG_ARM_64
enum domain_type {
DOMAIN_32BIT,
@@ -262,6 +265,8 @@ static inline void arch_vcpu_block(struct vcpu *v) {}
#define arch_vm_assist_valid_mask(d) (1UL << VMASST_TYPE_runstate_update_flag)
+#define has_vpci(d) ({ (void)(d); false; })
+
#endif /* __ASM_DOMAIN_H__ */
/*
diff --git a/xen/include/asm-arm/hvm/ioreq.h b/xen/include/asm-arm/hvm/ioreq.h
new file mode 100644
index 0000000..9f59f23
--- /dev/null
+++ b/xen/include/asm-arm/hvm/ioreq.h
@@ -0,0 +1,109 @@
+/*
+ * hvm.h: Hardware virtual machine assist interface definitions.
+ *
+ * Copyright (c) 2016 Citrix Systems Inc.
+ * Copyright (c) 2019 Arm ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ASM_ARM_HVM_IOREQ_H__
+#define __ASM_ARM_HVM_IOREQ_H__
+
+#include <xen/ioreq.h>
+
+#include <public/hvm/ioreq.h>
+
+#ifdef CONFIG_IOREQ_SERVER
+enum io_state handle_ioserv(struct cpu_user_regs *regs, struct vcpu *v);
+enum io_state try_fwd_ioserv(struct cpu_user_regs *regs,
+ struct vcpu *v, mmio_info_t *info);
+#else
+static inline enum io_state handle_ioserv(struct cpu_user_regs *regs,
+ struct vcpu *v)
+{
+ return IO_UNHANDLED;
+}
+
+static inline enum io_state try_fwd_ioserv(struct cpu_user_regs *regs,
+ struct vcpu *v, mmio_info_t *info)
+{
+ return IO_UNHANDLED;
+}
+#endif
+
+bool ioreq_complete_mmio(void);
+
+static inline bool handle_pio(uint16_t port, unsigned int size, int dir)
+{
+ /*
+ * TODO: For Arm64, the main user will be PCI. So this should be
+ * implemented when we add support for vPCI.
+ */
+ ASSERT_UNREACHABLE();
+ return true;
+}
+
+static inline void arch_destroy_ioreq_server(struct ioreq_server *s)
+{
+}
+
+static inline void msix_write_completion(struct vcpu *v)
+{
+}
+
+static inline bool arch_io_completion(enum io_completion io_completion)
+{
+ ASSERT_UNREACHABLE();
+ return true;
+}
+
+static inline int ioreq_server_get_type_addr(const struct domain *d,
+ const ioreq_t *p,
+ uint8_t *type,
+ uint64_t *addr)
+{
+ if ( p->type != IOREQ_TYPE_COPY && p->type != IOREQ_TYPE_PIO )
+ return -EINVAL;
+
+ *type = (p->type == IOREQ_TYPE_PIO) ?
+ XEN_DMOP_IO_RANGE_PORT : XEN_DMOP_IO_RANGE_MEMORY;
+ *addr = p->addr;
+
+ return 0;
+}
+
+static inline void arch_ioreq_init(struct domain *d)
+{
+}
+
+static inline bool arch_ioreq_destroy(struct domain *d)
+{
+ return true;
+}
+
+#define IOREQ_STATUS_HANDLED IO_HANDLED
+#define IOREQ_STATUS_UNHANDLED IO_UNHANDLED
+#define IOREQ_STATUS_RETRY IO_RETRY
+
+#endif /* __ASM_ARM_HVM_IOREQ_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/asm-arm/mmio.h b/xen/include/asm-arm/mmio.h
index 8dbfb27..7ab873c 100644
--- a/xen/include/asm-arm/mmio.h
+++ b/xen/include/asm-arm/mmio.h
@@ -37,6 +37,7 @@ enum io_state
IO_ABORT, /* The IO was handled by the helper and led to an abort. */
IO_HANDLED, /* The IO was successfully handled by the helper. */
IO_UNHANDLED, /* The IO was not handled by the helper. */
+ IO_RETRY, /* Retry the emulation for some reason */
};
typedef int (*mmio_read_t)(struct vcpu *v, mmio_info_t *info,
diff --git a/xen/include/asm-arm/paging.h b/xen/include/asm-arm/paging.h
index 6d1a000..0550c55 100644
--- a/xen/include/asm-arm/paging.h
+++ b/xen/include/asm-arm/paging.h
@@ -4,6 +4,10 @@
#define paging_mode_translate(d) (1)
#define paging_mode_external(d) (1)
+static inline void paging_mark_pfn_dirty(struct domain *d, pfn_t pfn)
+{
+}
+
#endif /* XEN_PAGING_H */
/*
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 15/23] xen/arm: Stick around in leave_hypervisor_to_guest until I/O has completed
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (13 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 14/23] arm/ioreq: Introduce arch specific bits for IOREQ/DM features Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-10-15 16:44 ` [PATCH V2 16/23] xen/mm: Handle properly reference in set_foreign_p2m_entry() on Arm Oleksandr Tyshchenko
` (8 subsequent siblings)
23 siblings, 0 replies; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
This patch adds proper handling of return value of handle_io_completion()
which involves using a loop in leave_hypervisor_to_guest().
The reason to use an unbounded loop here is the fact that vCPU
shouldn't continue until an I/O has completed. In Xen case, if an I/O
never completes then it most likely means that something went horribly
wrong with the Device Emulator. And it is most likely not safe to
continue. So letting the vCPU to spin forever if I/O never completes
is a safer action than letting it continue and leaving the guest in
unclear state and is the best what we can do for now.
This wouldn't be an issue for Xen as do_softirq() would be called at
every loop. In case of failure, the guest will crash and the vCPU
will be unscheduled.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes V1 -> V2:
- new patch, changes were derived from (+ new explanation):
arm/ioreq: Introduce arch specific bits for IOREQ/DM features
---
xen/arch/arm/traps.c | 31 ++++++++++++++++++++++++++-----
1 file changed, 26 insertions(+), 5 deletions(-)
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index b154837..507c095 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -2256,18 +2256,23 @@ static void check_for_pcpu_work(void)
* Process pending work for the vCPU. Any call should be fast or
* implement preemption.
*/
-static void check_for_vcpu_work(void)
+static bool check_for_vcpu_work(void)
{
struct vcpu *v = current;
#ifdef CONFIG_IOREQ_SERVER
+ bool handled;
+
local_irq_enable();
- handle_io_completion(v);
+ handled = handle_io_completion(v);
local_irq_disable();
+
+ if ( !handled )
+ return true;
#endif
if ( likely(!v->arch.need_flush_to_ram) )
- return;
+ return false;
/*
* Give a chance for the pCPU to process work before handling the vCPU
@@ -2278,6 +2283,8 @@ static void check_for_vcpu_work(void)
local_irq_enable();
p2m_flush_vm(v);
local_irq_disable();
+
+ return false;
}
/*
@@ -2290,8 +2297,22 @@ void leave_hypervisor_to_guest(void)
{
local_irq_disable();
- check_for_vcpu_work();
- check_for_pcpu_work();
+ /*
+ * The reason to use an unbounded loop here is the fact that vCPU
+ * shouldn't continue until an I/O has completed. In Xen case, if an I/O
+ * never completes then it most likely means that something went horribly
+ * wrong with the Device Emulator. And it is most likely not safe to
+ * continue. So letting the vCPU to spin forever if I/O never completes
+ * is a safer action than letting it continue and leaving the guest in
+ * unclear state and is the best what we can do for now.
+ *
+ * This wouldn't be an issue for Xen as do_softirq() would be called at
+ * every loop. In case of failure, the guest will crash and the vCPU
+ * will be unscheduled.
+ */
+ do {
+ check_for_pcpu_work();
+ } while ( check_for_vcpu_work() );
vgic_sync_to_lrs();
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 16/23] xen/mm: Handle properly reference in set_foreign_p2m_entry() on Arm
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (14 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 15/23] xen/arm: Stick around in leave_hypervisor_to_guest until I/O has completed Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-11-12 12:18 ` Jan Beulich
2020-10-15 16:44 ` [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server() Oleksandr Tyshchenko
` (7 subsequent siblings)
23 siblings, 1 reply; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Andrew Cooper, George Dunlap, Ian Jackson,
Jan Beulich, Wei Liu, Roger Pau Monné,
Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
This patch implements reference counting of foreign entries in
in set_foreign_p2m_entry() on Arm. This is a mandatory action if
we want to run emulator (IOREQ server) in other than dom0 domain,
as we can't trust it to do the right thing if it is not running
in dom0. So we need to grab a reference on the page to avoid it
disappearing.
It is valid to always pass "p2m_map_foreign_rw" type to
guest_physmap_add_entry() since the current and foreign domains
would be always different. A case when they are equal would be
rejected by rcu_lock_remote_domain_by_id().
It was tested with IOREQ feature to confirm that all the pages given
to this function belong to a domain, so we can use the same approach
as for XENMAPSPACE_gmfn_foreign handling in xenmem_add_to_physmap_one().
This involves adding an extra parameter for the foreign domain to
set_foreign_p2m_entry() and a helper to indicate whether the arch
supports the reference counting of foreign entries and the restriction
for the hardware domain in the common code can be skipped for it.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- new patch, was split from:
"[RFC PATCH V1 04/12] xen/arm: Introduce arch specific bits for IOREQ/DM features"
- rewrite a logic to handle properly reference in set_foreign_p2m_entry()
instead of treating foreign entries as p2m_ram_rw
Changes V1 -> V2:
- rebase according to the recent changes to acquire_resource()
- update patch description
- introduce arch_refcounts_p2m()
- add an explanation why p2m_map_foreign_rw is valid
- move set_foreign_p2m_entry() to p2m-common.h
- add const to new parameter
---
xen/arch/arm/p2m.c | 21 +++++++++++++++++++++
xen/arch/x86/mm/p2m.c | 5 +++--
xen/common/memory.c | 5 +++--
xen/include/asm-arm/p2m.h | 19 +++++++++----------
xen/include/asm-x86/p2m.h | 12 +++++++++---
xen/include/xen/p2m-common.h | 4 ++++
6 files changed, 49 insertions(+), 17 deletions(-)
diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index 4eeb867..370173c 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -1380,6 +1380,27 @@ int guest_physmap_remove_page(struct domain *d, gfn_t gfn, mfn_t mfn,
return p2m_remove_mapping(d, gfn, (1 << page_order), mfn);
}
+int set_foreign_p2m_entry(struct domain *d, const struct domain *fd,
+ unsigned long gfn, mfn_t mfn)
+{
+ struct page_info *page = mfn_to_page(mfn);
+ int rc;
+
+ if ( !get_page(page, fd) )
+ return -EINVAL;
+
+ /*
+ * It is valid to always use p2m_map_foreign_rw here as if this gets
+ * called that d != fd. A case when d == fd would be rejected by
+ * rcu_lock_remote_domain_by_id() earlier.
+ */
+ rc = guest_physmap_add_entry(d, _gfn(gfn), mfn, 0, p2m_map_foreign_rw);
+ if ( rc )
+ put_page(page);
+
+ return 0;
+}
+
static struct page_info *p2m_allocate_root(void)
{
struct page_info *page;
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index 6102771..8d03ab4 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -1320,7 +1320,8 @@ static int set_typed_p2m_entry(struct domain *d, unsigned long gfn_l,
}
/* Set foreign mfn in the given guest's p2m table. */
-int set_foreign_p2m_entry(struct domain *d, unsigned long gfn, mfn_t mfn)
+int set_foreign_p2m_entry(struct domain *d, const struct domain *fd,
+ unsigned long gfn, mfn_t mfn)
{
return set_typed_p2m_entry(d, gfn, mfn, PAGE_ORDER_4K, p2m_map_foreign,
p2m_get_hostp2m(d)->default_access);
@@ -2620,7 +2621,7 @@ int p2m_add_foreign(struct domain *tdom, unsigned long fgfn,
* will update the m2p table which will result in mfn -> gpfn of dom0
* and not fgfn of domU.
*/
- rc = set_foreign_p2m_entry(tdom, gpfn, mfn);
+ rc = set_foreign_p2m_entry(tdom, fdom, gpfn, mfn);
if ( rc )
gdprintk(XENLOG_WARNING, "set_foreign_p2m_entry failed. "
"gpfn:%lx mfn:%lx fgfn:%lx td:%d fd:%d\n",
diff --git a/xen/common/memory.c b/xen/common/memory.c
index cf53ca3..fb9ea96 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -1099,7 +1099,8 @@ static int acquire_resource(
* reference counted, it is unsafe to allow mapping of
* resource pages unless the caller is the hardware domain.
*/
- if ( paging_mode_translate(currd) && !is_hardware_domain(currd) )
+ if ( paging_mode_translate(currd) && !is_hardware_domain(currd) &&
+ !arch_refcounts_p2m() )
return -EACCES;
if ( copy_from_guest(&xmar, arg, 1) )
@@ -1168,7 +1169,7 @@ static int acquire_resource(
for ( i = 0; !rc && i < xmar.nr_frames; i++ )
{
- rc = set_foreign_p2m_entry(currd, gfn_list[i],
+ rc = set_foreign_p2m_entry(currd, d, gfn_list[i],
_mfn(mfn_list[i]));
/* rc should be -EIO for any iteration other than the first */
if ( rc && i )
diff --git a/xen/include/asm-arm/p2m.h b/xen/include/asm-arm/p2m.h
index 28ca9a8..d11be80 100644
--- a/xen/include/asm-arm/p2m.h
+++ b/xen/include/asm-arm/p2m.h
@@ -161,6 +161,15 @@ typedef enum {
#endif
#include <xen/p2m-common.h>
+static inline bool arch_refcounts_p2m(void)
+{
+ /*
+ * The reference counting of foreign entries in set_foreign_p2m_entry()
+ * is supported on Arm.
+ */
+ return true;
+}
+
static inline
void p2m_altp2m_check(struct vcpu *v, uint16_t idx)
{
@@ -392,16 +401,6 @@ static inline gfn_t gfn_next_boundary(gfn_t gfn, unsigned int order)
return gfn_add(gfn, 1UL << order);
}
-static inline int set_foreign_p2m_entry(struct domain *d, unsigned long gfn,
- mfn_t mfn)
-{
- /*
- * NOTE: If this is implemented then proper reference counting of
- * foreign entries will need to be implemented.
- */
- return -EOPNOTSUPP;
-}
-
/*
* A vCPU has cache enabled only when the MMU is enabled and data cache
* is enabled.
diff --git a/xen/include/asm-x86/p2m.h b/xen/include/asm-x86/p2m.h
index 5f7ba31..6c42022 100644
--- a/xen/include/asm-x86/p2m.h
+++ b/xen/include/asm-x86/p2m.h
@@ -369,6 +369,15 @@ struct p2m_domain {
#endif
#include <xen/p2m-common.h>
+static inline bool arch_refcounts_p2m(void)
+{
+ /*
+ * The reference counting of foreign entries in set_foreign_p2m_entry()
+ * is not supported on x86.
+ */
+ return false;
+}
+
/*
* Updates vCPU's n2pm to match its np2m_base in VMCx12 and returns that np2m.
*/
@@ -634,9 +643,6 @@ int p2m_finish_type_change(struct domain *d,
int p2m_is_logdirty_range(struct p2m_domain *, unsigned long start,
unsigned long end);
-/* Set foreign entry in the p2m table (for priv-mapping) */
-int set_foreign_p2m_entry(struct domain *d, unsigned long gfn, mfn_t mfn);
-
/* Set mmio addresses in the p2m table (for pass-through) */
int set_mmio_p2m_entry(struct domain *d, gfn_t gfn, mfn_t mfn,
unsigned int order);
diff --git a/xen/include/xen/p2m-common.h b/xen/include/xen/p2m-common.h
index 58031a6..b4bc709 100644
--- a/xen/include/xen/p2m-common.h
+++ b/xen/include/xen/p2m-common.h
@@ -3,6 +3,10 @@
#include <xen/mm.h>
+/* Set foreign entry in the p2m table */
+int set_foreign_p2m_entry(struct domain *d, const struct domain *fd,
+ unsigned long gfn, mfn_t mfn);
+
/* Remove a page from a domain's p2m table */
int __must_check
guest_physmap_remove_page(struct domain *d, gfn_t gfn, mfn_t mfn,
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server()
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (15 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 16/23] xen/mm: Handle properly reference in set_foreign_p2m_entry() on Arm Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-10-20 10:51 ` Paul Durrant
2020-10-15 16:44 ` [PATCH V2 18/23] xen/dm: Introduce xendevicemodel_set_irq_level DM op Oleksandr Tyshchenko
` (6 subsequent siblings)
23 siblings, 1 reply; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Andrew Cooper, George Dunlap, Ian Jackson,
Jan Beulich, Wei Liu, Paul Durrant, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
This patch introduces a helper the main purpose of which is to check
if a domain is using IOREQ server(s).
On Arm the current benefit is to avoid calling handle_io_completion()
(which implies iterating over all possible IOREQ servers anyway)
on every return in leave_hypervisor_to_guest() if there is no active
servers for the particular domain.
Also this helper will be used by one of the subsequent patches on Arm.
This involves adding an extra per-domain variable to store the count
of servers in use.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- new patch
Changes V1 -> V2:
- update patch description
- guard helper with CONFIG_IOREQ_SERVER
- remove "hvm" prefix
- modify helper to just return d->arch.hvm.ioreq_server.nr_servers
- put suitable ASSERT()s
- use ASSERT(d->ioreq_server.server[id] ? !s : !!s) in set_ioreq_server()
- remove d->ioreq_server.nr_servers = 0 from hvm_ioreq_init()
---
xen/arch/arm/traps.c | 15 +++++++++------
xen/common/ioreq.c | 7 ++++++-
xen/include/xen/ioreq.h | 14 ++++++++++++++
xen/include/xen/sched.h | 1 +
4 files changed, 30 insertions(+), 7 deletions(-)
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 507c095..a8f5fdf 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -2261,14 +2261,17 @@ static bool check_for_vcpu_work(void)
struct vcpu *v = current;
#ifdef CONFIG_IOREQ_SERVER
- bool handled;
+ if ( domain_has_ioreq_server(v->domain) )
+ {
+ bool handled;
- local_irq_enable();
- handled = handle_io_completion(v);
- local_irq_disable();
+ local_irq_enable();
+ handled = handle_io_completion(v);
+ local_irq_disable();
- if ( !handled )
- return true;
+ if ( !handled )
+ return true;
+ }
#endif
if ( likely(!v->arch.need_flush_to_ram) )
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index bcd4961..a72bc0e 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -39,9 +39,14 @@ static void set_ioreq_server(struct domain *d, unsigned int id,
struct ioreq_server *s)
{
ASSERT(id < MAX_NR_IOREQ_SERVERS);
- ASSERT(!s || !d->ioreq_server.server[id]);
+ ASSERT(d->ioreq_server.server[id] ? !s : !!s);
d->ioreq_server.server[id] = s;
+
+ if ( s )
+ d->ioreq_server.nr_servers++;
+ else
+ d->ioreq_server.nr_servers--;
}
#define GET_IOREQ_SERVER(d, id) \
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
index 7b03ab5..0679fef 100644
--- a/xen/include/xen/ioreq.h
+++ b/xen/include/xen/ioreq.h
@@ -55,6 +55,20 @@ struct ioreq_server {
uint8_t bufioreq_handling;
};
+#ifdef CONFIG_IOREQ_SERVER
+static inline bool domain_has_ioreq_server(const struct domain *d)
+{
+ ASSERT((current->domain == d) || atomic_read(&d->pause_count));
+
+ return d->ioreq_server.nr_servers;
+}
+#else
+static inline bool domain_has_ioreq_server(const struct domain *d)
+{
+ return false;
+}
+#endif
+
struct ioreq_server *get_ioreq_server(const struct domain *d,
unsigned int id);
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index f9ce14c..290cddb 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -553,6 +553,7 @@ struct domain
struct {
spinlock_t lock;
struct ioreq_server *server[MAX_NR_IOREQ_SERVERS];
+ unsigned int nr_servers;
} ioreq_server;
#endif
};
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 18/23] xen/dm: Introduce xendevicemodel_set_irq_level DM op
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (16 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server() Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-10-15 16:44 ` [PATCH V2 19/23] xen/arm: io: Abstract sign-extension Oleksandr Tyshchenko
` (5 subsequent siblings)
23 siblings, 0 replies; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Julien Grall, Ian Jackson, Wei Liu, Andrew Cooper, George Dunlap,
Jan Beulich, Julien Grall, Stefano Stabellini, Volodymyr Babchuk,
Oleksandr Tyshchenko
From: Julien Grall <julien.grall@arm.com>
This patch adds ability to the device emulator to notify otherend
(some entity running in the guest) using a SPI and implements Arm
specific bits for it. Proposed interface allows emulator to set
the logical level of a one of a domain's IRQ lines.
We can't reuse the existing DM op (xen_dm_op_set_isa_irq_level)
to inject an interrupt as the "isa_irq" field is only 8-bit and
able to cover IRQ 0 - 255, whereas we need a wider range (0 - 1020).
Signed-off-by: Julien Grall <julien.grall@arm.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
***
Please note, I left interface untouched since there is still
an open discussion what interface to use/what information to pass
to the hypervisor. The question whether we should abstract away
the state of the line or not.
***
Changes RFC -> V1:
- check incoming parameters in arch_dm_op()
- add explicit padding to struct xen_dm_op_set_irq_level
Changes V1 -> V2:
- update the author of a patch
- update patch description
- check that padding is always 0
- mention that interface is Arm only and only SPIs are
supported for now
- allow to set the logical level of a line for non-allocated
interrupts only
- add xen_dm_op_set_irq_level_t
---
tools/libs/devicemodel/core.c | 18 ++++++++
tools/libs/devicemodel/include/xendevicemodel.h | 4 ++
tools/libs/devicemodel/libxendevicemodel.map | 1 +
xen/arch/arm/dm.c | 57 ++++++++++++++++++++++++-
xen/common/dm.c | 1 +
xen/include/public/hvm/dm_op.h | 16 +++++++
6 files changed, 96 insertions(+), 1 deletion(-)
diff --git a/tools/libs/devicemodel/core.c b/tools/libs/devicemodel/core.c
index 4d40639..30bd79f 100644
--- a/tools/libs/devicemodel/core.c
+++ b/tools/libs/devicemodel/core.c
@@ -430,6 +430,24 @@ int xendevicemodel_set_isa_irq_level(
return xendevicemodel_op(dmod, domid, 1, &op, sizeof(op));
}
+int xendevicemodel_set_irq_level(
+ xendevicemodel_handle *dmod, domid_t domid, uint32_t irq,
+ unsigned int level)
+{
+ struct xen_dm_op op;
+ struct xen_dm_op_set_irq_level *data;
+
+ memset(&op, 0, sizeof(op));
+
+ op.op = XEN_DMOP_set_irq_level;
+ data = &op.u.set_irq_level;
+
+ data->irq = irq;
+ data->level = level;
+
+ return xendevicemodel_op(dmod, domid, 1, &op, sizeof(op));
+}
+
int xendevicemodel_set_pci_link_route(
xendevicemodel_handle *dmod, domid_t domid, uint8_t link, uint8_t irq)
{
diff --git a/tools/libs/devicemodel/include/xendevicemodel.h b/tools/libs/devicemodel/include/xendevicemodel.h
index e877f5c..c06b3c8 100644
--- a/tools/libs/devicemodel/include/xendevicemodel.h
+++ b/tools/libs/devicemodel/include/xendevicemodel.h
@@ -209,6 +209,10 @@ int xendevicemodel_set_isa_irq_level(
xendevicemodel_handle *dmod, domid_t domid, uint8_t irq,
unsigned int level);
+int xendevicemodel_set_irq_level(
+ xendevicemodel_handle *dmod, domid_t domid, unsigned int irq,
+ unsigned int level);
+
/**
* This function maps a PCI INTx line to a an IRQ line.
*
diff --git a/tools/libs/devicemodel/libxendevicemodel.map b/tools/libs/devicemodel/libxendevicemodel.map
index 561c62d..a0c3012 100644
--- a/tools/libs/devicemodel/libxendevicemodel.map
+++ b/tools/libs/devicemodel/libxendevicemodel.map
@@ -32,6 +32,7 @@ VERS_1.2 {
global:
xendevicemodel_relocate_memory;
xendevicemodel_pin_memory_cacheattr;
+ xendevicemodel_set_irq_level;
} VERS_1.1;
VERS_1.3 {
diff --git a/xen/arch/arm/dm.c b/xen/arch/arm/dm.c
index 5d3da37..e4bb233 100644
--- a/xen/arch/arm/dm.c
+++ b/xen/arch/arm/dm.c
@@ -17,10 +17,65 @@
#include <xen/dm.h>
#include <xen/hypercall.h>
+#include <asm/vgic.h>
+
int arch_dm_op(struct xen_dm_op *op, struct domain *d,
const struct dmop_args *op_args, bool *const_op)
{
- return -EOPNOTSUPP;
+ int rc;
+
+ switch ( op->op )
+ {
+ case XEN_DMOP_set_irq_level:
+ {
+ const struct xen_dm_op_set_irq_level *data =
+ &op->u.set_irq_level;
+ unsigned int i;
+
+ /* Only SPIs are supported */
+ if ( (data->irq < NR_LOCAL_IRQS) || (data->irq >= vgic_num_irqs(d)) )
+ {
+ rc = -EINVAL;
+ break;
+ }
+
+ if ( data->level != 0 && data->level != 1 )
+ {
+ rc = -EINVAL;
+ break;
+ }
+
+ /* Check that padding is always 0 */
+ for ( i = 0; i < sizeof(data->pad); i++ )
+ {
+ if ( data->pad[i] )
+ {
+ rc = -EINVAL;
+ break;
+ }
+ }
+
+ /*
+ * Allow to set the logical level of a line for non-allocated
+ * interrupts only.
+ */
+ if ( test_bit(data->irq, d->arch.vgic.allocated_irqs) )
+ {
+ rc = -EINVAL;
+ break;
+ }
+
+ vgic_inject_irq(d, NULL, data->irq, data->level);
+ rc = 0;
+ break;
+ }
+
+ default:
+ rc = -EOPNOTSUPP;
+ break;
+ }
+
+ return rc;
}
/*
diff --git a/xen/common/dm.c b/xen/common/dm.c
index f3a8353..5f23420 100644
--- a/xen/common/dm.c
+++ b/xen/common/dm.c
@@ -48,6 +48,7 @@ static int dm_op(const struct dmop_args *op_args)
[XEN_DMOP_remote_shutdown] = sizeof(struct xen_dm_op_remote_shutdown),
[XEN_DMOP_relocate_memory] = sizeof(struct xen_dm_op_relocate_memory),
[XEN_DMOP_pin_memory_cacheattr] = sizeof(struct xen_dm_op_pin_memory_cacheattr),
+ [XEN_DMOP_set_irq_level] = sizeof(struct xen_dm_op_set_irq_level),
};
rc = rcu_lock_remote_domain_by_id(op_args->domid, &d);
diff --git a/xen/include/public/hvm/dm_op.h b/xen/include/public/hvm/dm_op.h
index 66cae1a..1f70d58 100644
--- a/xen/include/public/hvm/dm_op.h
+++ b/xen/include/public/hvm/dm_op.h
@@ -434,6 +434,21 @@ struct xen_dm_op_pin_memory_cacheattr {
};
typedef struct xen_dm_op_pin_memory_cacheattr xen_dm_op_pin_memory_cacheattr_t;
+/*
+ * XEN_DMOP_set_irq_level: Set the logical level of a one of a domain's
+ * IRQ lines (currently Arm only).
+ * Only SPIs are supported.
+ */
+#define XEN_DMOP_set_irq_level 19
+
+struct xen_dm_op_set_irq_level {
+ uint32_t irq;
+ /* IN - Level: 0 -> deasserted, 1 -> asserted */
+ uint8_t level;
+ uint8_t pad[3];
+};
+typedef struct xen_dm_op_set_irq_level xen_dm_op_set_irq_level_t;
+
struct xen_dm_op {
uint32_t op;
uint32_t pad;
@@ -447,6 +462,7 @@ struct xen_dm_op {
xen_dm_op_track_dirty_vram_t track_dirty_vram;
xen_dm_op_set_pci_intx_level_t set_pci_intx_level;
xen_dm_op_set_isa_irq_level_t set_isa_irq_level;
+ xen_dm_op_set_irq_level_t set_irq_level;
xen_dm_op_set_pci_link_route_t set_pci_link_route;
xen_dm_op_modified_memory_t modified_memory;
xen_dm_op_set_mem_type_t set_mem_type;
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 19/23] xen/arm: io: Abstract sign-extension
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (17 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 18/23] xen/dm: Introduce xendevicemodel_set_irq_level DM op Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-10-15 16:44 ` [PATCH V2 20/23] xen/ioreq: Make x86's send_invalidate_req() common Oleksandr Tyshchenko
` (4 subsequent siblings)
23 siblings, 0 replies; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
In order to avoid code duplication (both handle_read() and
handle_ioserv() contain the same code for the sign-extension)
put this code to a common helper to be used for both.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes V1 -> V2:
- new patch
---
xen/arch/arm/io.c | 18 ++----------------
xen/arch/arm/ioreq.c | 17 +----------------
xen/include/asm-arm/traps.h | 24 ++++++++++++++++++++++++
3 files changed, 27 insertions(+), 32 deletions(-)
diff --git a/xen/arch/arm/io.c b/xen/arch/arm/io.c
index f44cfd4..8d6ec6c 100644
--- a/xen/arch/arm/io.c
+++ b/xen/arch/arm/io.c
@@ -23,6 +23,7 @@
#include <asm/cpuerrata.h>
#include <asm/current.h>
#include <asm/mmio.h>
+#include <asm/traps.h>
#include <asm/hvm/ioreq.h>
#include "decode.h"
@@ -39,26 +40,11 @@ static enum io_state handle_read(const struct mmio_handler *handler,
* setting r).
*/
register_t r = 0;
- uint8_t size = (1 << dabt.size) * 8;
if ( !handler->ops->read(v, info, &r, handler->priv) )
return IO_ABORT;
- /*
- * Sign extend if required.
- * Note that we expect the read handler to have zeroed the bits
- * outside the requested access size.
- */
- if ( dabt.sign && (r & (1UL << (size - 1))) )
- {
- /*
- * We are relying on register_t using the same as
- * an unsigned long in order to keep the 32-bit assembly
- * code smaller.
- */
- BUILD_BUG_ON(sizeof(register_t) != sizeof(unsigned long));
- r |= (~0UL) << size;
- }
+ r = sign_extend(dabt, r);
set_user_reg(regs, dabt.reg, r);
diff --git a/xen/arch/arm/ioreq.c b/xen/arch/arm/ioreq.c
index da5ceac..ad17b80 100644
--- a/xen/arch/arm/ioreq.c
+++ b/xen/arch/arm/ioreq.c
@@ -28,7 +28,6 @@ enum io_state handle_ioserv(struct cpu_user_regs *regs, struct vcpu *v)
const union hsr hsr = { .bits = regs->hsr };
const struct hsr_dabt dabt = hsr.dabt;
/* Code is similar to handle_read */
- uint8_t size = (1 << dabt.size) * 8;
register_t r = v->io.io_req.data;
/* We are done with the IO */
@@ -37,21 +36,7 @@ enum io_state handle_ioserv(struct cpu_user_regs *regs, struct vcpu *v)
if ( dabt.write )
return IO_HANDLED;
- /*
- * Sign extend if required.
- * Note that we expect the read handler to have zeroed the bits
- * outside the requested access size.
- */
- if ( dabt.sign && (r & (1UL << (size - 1))) )
- {
- /*
- * We are relying on register_t using the same as
- * an unsigned long in order to keep the 32-bit assembly
- * code smaller.
- */
- BUILD_BUG_ON(sizeof(register_t) != sizeof(unsigned long));
- r |= (~0UL) << size;
- }
+ r = sign_extend(dabt, r);
set_user_reg(regs, dabt.reg, r);
diff --git a/xen/include/asm-arm/traps.h b/xen/include/asm-arm/traps.h
index 997c378..e301c44 100644
--- a/xen/include/asm-arm/traps.h
+++ b/xen/include/asm-arm/traps.h
@@ -83,6 +83,30 @@ static inline bool VABORT_GEN_BY_GUEST(const struct cpu_user_regs *regs)
(unsigned long)abort_guest_exit_end == regs->pc;
}
+/* Check whether the sign extension is required and perform it */
+static inline register_t sign_extend(const struct hsr_dabt dabt, register_t r)
+{
+ uint8_t size = (1 << dabt.size) * 8;
+
+ /*
+ * Sign extend if required.
+ * Note that we expect the read handler to have zeroed the bits
+ * outside the requested access size.
+ */
+ if ( dabt.sign && (r & (1UL << (size - 1))) )
+ {
+ /*
+ * We are relying on register_t using the same as
+ * an unsigned long in order to keep the 32-bit assembly
+ * code smaller.
+ */
+ BUILD_BUG_ON(sizeof(register_t) != sizeof(unsigned long));
+ r |= (~0UL) << size;
+ }
+
+ return r;
+}
+
#endif /* __ASM_ARM_TRAPS__ */
/*
* Local variables:
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 20/23] xen/ioreq: Make x86's send_invalidate_req() common
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (18 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 19/23] xen/arm: io: Abstract sign-extension Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-11-12 11:55 ` Jan Beulich
2020-10-15 16:44 ` [PATCH V2 21/23] xen/arm: Add mapcache invalidation handling Oleksandr Tyshchenko
` (3 subsequent siblings)
23 siblings, 1 reply; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Paul Durrant, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
As the IOREQ is a common feature now and we also need to
invalidate qemu/demu mapcache on Arm when the required condition
occurs this patch moves this function to the common code
(and remames it to send_invalidate_ioreq).
This patch also moves per-domain qemu_mapcache_invalidate
variable out of the arch sub-struct.
The subsequent patch will add mapcache invalidation handling on Arm.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
***
Please note, this patch depends on the following which is
on review:
https://patchwork.kernel.org/patch/11803383/
***
Changes RFC -> V1:
- move send_invalidate_req() to the common code
- update patch subject/description
- move qemu_mapcache_invalidate out of the arch sub-struct,
update checks
- remove #if defined(CONFIG_ARM64) from the common code
Changes V1 -> V2:
- was split into:
- xen/ioreq: Make x86's send_invalidate_req() common
- xen/arm: Add mapcache invalidation handling
- update patch description/subject
- move Arm bits to a separate patch
- don't alter the common code, the flag is set by arch code
- rename send_invalidate_req() to send_invalidate_ioreq()
- guard qemu_mapcache_invalidate with CONFIG_IOREQ_SERVER
- use bool instead of bool_t
- remove blank line blank line between head comment and #include-s
---
xen/arch/x86/hvm/hypercall.c | 9 +++++----
xen/arch/x86/hvm/io.c | 14 --------------
xen/common/ioreq.c | 14 ++++++++++++++
xen/include/asm-x86/hvm/domain.h | 1 -
xen/include/asm-x86/hvm/io.h | 1 -
xen/include/xen/ioreq.h | 1 +
xen/include/xen/sched.h | 2 ++
7 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c
index b6ccaf4..324ff97 100644
--- a/xen/arch/x86/hvm/hypercall.c
+++ b/xen/arch/x86/hvm/hypercall.c
@@ -20,6 +20,7 @@
*/
#include <xen/lib.h>
#include <xen/hypercall.h>
+#include <xen/ioreq.h>
#include <xen/nospec.h>
#include <asm/hvm/emulate.h>
@@ -47,7 +48,7 @@ static long hvm_memory_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
rc = compat_memory_op(cmd, arg);
if ( (cmd & MEMOP_CMD_MASK) == XENMEM_decrease_reservation )
- curr->domain->arch.hvm.qemu_mapcache_invalidate = true;
+ curr->domain->qemu_mapcache_invalidate = true;
return rc;
}
@@ -329,9 +330,9 @@ int hvm_hypercall(struct cpu_user_regs *regs)
if ( curr->hcall_preempted )
return HVM_HCALL_preempted;
- if ( unlikely(currd->arch.hvm.qemu_mapcache_invalidate) &&
- test_and_clear_bool(currd->arch.hvm.qemu_mapcache_invalidate) )
- send_invalidate_req();
+ if ( unlikely(currd->qemu_mapcache_invalidate) &&
+ test_and_clear_bool(currd->qemu_mapcache_invalidate) )
+ send_invalidate_ioreq();
return HVM_HCALL_completed;
}
diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
index 2d03ffe..e51304c 100644
--- a/xen/arch/x86/hvm/io.c
+++ b/xen/arch/x86/hvm/io.c
@@ -64,20 +64,6 @@ void send_timeoffset_req(unsigned long timeoff)
gprintk(XENLOG_ERR, "Unsuccessful timeoffset update\n");
}
-/* Ask ioemu mapcache to invalidate mappings. */
-void send_invalidate_req(void)
-{
- ioreq_t p = {
- .type = IOREQ_TYPE_INVALIDATE,
- .size = 4,
- .dir = IOREQ_WRITE,
- .data = ~0UL, /* flush all */
- };
-
- if ( broadcast_ioreq(&p, false) != 0 )
- gprintk(XENLOG_ERR, "Unsuccessful map-cache invalidate\n");
-}
-
bool hvm_emulate_one_insn(hvm_emulate_validate_t *validate, const char *descr)
{
struct hvm_emulate_ctxt ctxt;
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index a72bc0e..2203cf0 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -35,6 +35,20 @@
#include <public/hvm/ioreq.h>
#include <public/hvm/params.h>
+/* Ask ioemu mapcache to invalidate mappings. */
+void send_invalidate_ioreq(void)
+{
+ ioreq_t p = {
+ .type = IOREQ_TYPE_INVALIDATE,
+ .size = 4,
+ .dir = IOREQ_WRITE,
+ .data = ~0UL, /* flush all */
+ };
+
+ if ( broadcast_ioreq(&p, false) != 0 )
+ gprintk(XENLOG_ERR, "Unsuccessful map-cache invalidate\n");
+}
+
static void set_ioreq_server(struct domain *d, unsigned int id,
struct ioreq_server *s)
{
diff --git a/xen/include/asm-x86/hvm/domain.h b/xen/include/asm-x86/hvm/domain.h
index c3af339..caab3a9 100644
--- a/xen/include/asm-x86/hvm/domain.h
+++ b/xen/include/asm-x86/hvm/domain.h
@@ -117,7 +117,6 @@ struct hvm_domain {
struct viridian_domain *viridian;
- bool_t qemu_mapcache_invalidate;
bool_t is_s3_suspended;
/*
diff --git a/xen/include/asm-x86/hvm/io.h b/xen/include/asm-x86/hvm/io.h
index fb64294..3da0136 100644
--- a/xen/include/asm-x86/hvm/io.h
+++ b/xen/include/asm-x86/hvm/io.h
@@ -97,7 +97,6 @@ bool relocate_portio_handler(
unsigned int size);
void send_timeoffset_req(unsigned long timeoff);
-void send_invalidate_req(void);
bool handle_mmio_with_translation(unsigned long gla, unsigned long gpfn,
struct npfec);
bool handle_pio(uint16_t port, unsigned int size, int dir);
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
index 0679fef..aad682f 100644
--- a/xen/include/xen/ioreq.h
+++ b/xen/include/xen/ioreq.h
@@ -126,6 +126,7 @@ struct ioreq_server *select_ioreq_server(struct domain *d,
int send_ioreq(struct ioreq_server *s, ioreq_t *proto_p,
bool buffered);
unsigned int broadcast_ioreq(ioreq_t *p, bool buffered);
+void send_invalidate_ioreq(void);
void ioreq_init(struct domain *d);
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 290cddb..1b8c6eb 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -555,6 +555,8 @@ struct domain
struct ioreq_server *server[MAX_NR_IOREQ_SERVERS];
unsigned int nr_servers;
} ioreq_server;
+
+ bool qemu_mapcache_invalidate;
#endif
};
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 21/23] xen/arm: Add mapcache invalidation handling
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (19 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 20/23] xen/ioreq: Make x86's send_invalidate_req() common Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-10-16 6:29 ` Jan Beulich
2020-10-15 16:44 ` [PATCH V2 22/23] libxl: Introduce basic virtio-mmio support on Arm Oleksandr Tyshchenko
` (2 subsequent siblings)
23 siblings, 1 reply; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Julien Grall
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
We need to send mapcache invalidation request to qemu/demu everytime
the page gets removed from a guest.
At the moment, the Arm code doesn't explicitely remove the existing
mapping before inserting the new mapping. Instead, this is done
implicitely by __p2m_set_entry().
So the corresponding flag will be set in __p2m_set_entry() if old entry
is a RAM page *and* the new MFN is different. And the invalidation
request will be sent in do_trap_hypercall() later on.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
CC: Julien Grall <julien.grall@arm.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes V1 -> V2:
- new patch, some changes were derived from (+ new explanation):
xen/ioreq: Make x86's invalidate qemu mapcache handling common
- put setting of the flag into __p2m_set_entry()
- clarify the conditions when the flag should be set
- use domain_has_ioreq_server()
- update do_trap_hypercall() by adding local variable
---
xen/arch/arm/p2m.c | 8 ++++++++
xen/arch/arm/traps.c | 13 ++++++++++---
2 files changed, 18 insertions(+), 3 deletions(-)
diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index 370173c..2693b0c 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -1,6 +1,7 @@
#include <xen/cpu.h>
#include <xen/domain_page.h>
#include <xen/iocap.h>
+#include <xen/ioreq.h>
#include <xen/lib.h>
#include <xen/sched.h>
#include <xen/softirq.h>
@@ -1067,7 +1068,14 @@ static int __p2m_set_entry(struct p2m_domain *p2m,
*/
if ( p2m_is_valid(orig_pte) &&
!mfn_eq(lpae_get_mfn(*entry), lpae_get_mfn(orig_pte)) )
+ {
+#ifdef CONFIG_IOREQ_SERVER
+ if ( domain_has_ioreq_server(p2m->domain) &&
+ (p2m->domain == current->domain) && p2m_is_ram(orig_pte.p2m.type) )
+ p2m->domain->qemu_mapcache_invalidate = true;
+#endif
p2m_free_entry(p2m, orig_pte, level);
+ }
out:
unmap_domain_page(table);
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index a8f5fdf..9eaa342 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -1442,6 +1442,7 @@ static void do_trap_hypercall(struct cpu_user_regs *regs, register_t *nr,
const union hsr hsr)
{
arm_hypercall_fn_t call = NULL;
+ struct vcpu *v = current;
BUILD_BUG_ON(NR_hypercalls < ARRAY_SIZE(arm_hypercall_table) );
@@ -1458,7 +1459,7 @@ static void do_trap_hypercall(struct cpu_user_regs *regs, register_t *nr,
return;
}
- current->hcall_preempted = false;
+ v->hcall_preempted = false;
perfc_incra(hypercalls, *nr);
call = arm_hypercall_table[*nr].fn;
@@ -1471,7 +1472,7 @@ static void do_trap_hypercall(struct cpu_user_regs *regs, register_t *nr,
HYPERCALL_RESULT_REG(regs) = call(HYPERCALL_ARGS(regs));
#ifndef NDEBUG
- if ( !current->hcall_preempted )
+ if ( !v->hcall_preempted )
{
/* Deliberately corrupt parameter regs used by this hypercall. */
switch ( arm_hypercall_table[*nr].nr_args ) {
@@ -1488,8 +1489,14 @@ static void do_trap_hypercall(struct cpu_user_regs *regs, register_t *nr,
#endif
/* Ensure the hypercall trap instruction is re-executed. */
- if ( current->hcall_preempted )
+ if ( v->hcall_preempted )
regs->pc -= 4; /* re-execute 'hvc #XEN_HYPERCALL_TAG' */
+
+#ifdef CONFIG_IOREQ_SERVER
+ if ( unlikely(v->domain->qemu_mapcache_invalidate) &&
+ test_and_clear_bool(v->domain->qemu_mapcache_invalidate) )
+ send_invalidate_ioreq();
+#endif
}
void arch_hypercall_tasklet_result(struct vcpu *v, long res)
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 22/23] libxl: Introduce basic virtio-mmio support on Arm
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (20 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 21/23] xen/arm: Add mapcache invalidation handling Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-10-15 16:44 ` [PATCH V2 23/23] [RFC] libxl: Add support for virtio-disk configuration Oleksandr Tyshchenko
2020-10-29 7:41 ` [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Masami Hiramatsu
23 siblings, 0 replies; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Julien Grall, Ian Jackson, Wei Liu, Anthony PERARD,
Stefano Stabellini, Julien Grall, Volodymyr Babchuk,
Oleksandr Tyshchenko
From: Julien Grall <julien.grall@arm.com>
This patch creates specific device node in the Guest device-tree
with allocated MMIO range and SPI interrupt if specific 'virtio'
property is present in domain config.
Signed-off-by: Julien Grall <julien.grall@arm.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"
Changes RFC -> V1:
- was squashed with:
"[RFC PATCH V1 09/12] libxl: Handle virtio-mmio irq in more correct way"
"[RFC PATCH V1 11/12] libxl: Insert "dma-coherent" property into virtio-mmio device node"
"[RFC PATCH V1 12/12] libxl: Fix duplicate memory node in DT"
- move VirtIO MMIO #define-s to xen/include/public/arch-arm.h
Changes V1 -> V2:
- update the author of a patch
---
tools/libs/light/libxl_arm.c | 58 ++++++++++++++++++++++++++++++++++++++--
tools/libs/light/libxl_types.idl | 1 +
tools/xl/xl_parse.c | 1 +
xen/include/public/arch-arm.h | 5 ++++
4 files changed, 63 insertions(+), 2 deletions(-)
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index 66e8a06..588ee5a 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -26,8 +26,8 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
{
uint32_t nr_spis = 0;
unsigned int i;
- uint32_t vuart_irq;
- bool vuart_enabled = false;
+ uint32_t vuart_irq, virtio_irq;
+ bool vuart_enabled = false, virtio_enabled = false;
/*
* If pl011 vuart is enabled then increment the nr_spis to allow allocation
@@ -39,6 +39,17 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
vuart_enabled = true;
}
+ /*
+ * XXX: Handle properly virtio
+ * A proper solution would be the toolstack to allocate the interrupts
+ * used by each virtio backend and let the backend now which one is used
+ */
+ if (libxl_defbool_val(d_config->b_info.arch_arm.virtio)) {
+ nr_spis += (GUEST_VIRTIO_MMIO_SPI - 32) + 1;
+ virtio_irq = GUEST_VIRTIO_MMIO_SPI;
+ virtio_enabled = true;
+ }
+
for (i = 0; i < d_config->b_info.num_irqs; i++) {
uint32_t irq = d_config->b_info.irqs[i];
uint32_t spi;
@@ -58,6 +69,12 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
return ERROR_FAIL;
}
+ /* The same check as for vpl011 */
+ if (virtio_enabled && irq == virtio_irq) {
+ LOG(ERROR, "Physical IRQ %u conflicting with virtio SPI\n", irq);
+ return ERROR_FAIL;
+ }
+
if (irq < 32)
continue;
@@ -658,6 +675,39 @@ static int make_vpl011_uart_node(libxl__gc *gc, void *fdt,
return 0;
}
+static int make_virtio_mmio_node(libxl__gc *gc, void *fdt,
+ uint64_t base, uint32_t irq)
+{
+ int res;
+ gic_interrupt intr;
+ /* Placeholder for virtio@ + a 64-bit number + \0 */
+ char buf[24];
+
+ snprintf(buf, sizeof(buf), "virtio@%"PRIx64, base);
+ res = fdt_begin_node(fdt, buf);
+ if (res) return res;
+
+ res = fdt_property_compat(gc, fdt, 1, "virtio,mmio");
+ if (res) return res;
+
+ res = fdt_property_regs(gc, fdt, GUEST_ROOT_ADDRESS_CELLS, GUEST_ROOT_SIZE_CELLS,
+ 1, base, GUEST_VIRTIO_MMIO_SIZE);
+ if (res) return res;
+
+ set_interrupt(intr, irq, 0xf, DT_IRQ_TYPE_EDGE_RISING);
+ res = fdt_property_interrupts(gc, fdt, &intr, 1);
+ if (res) return res;
+
+ res = fdt_property(fdt, "dma-coherent", NULL, 0);
+ if (res) return res;
+
+ res = fdt_end_node(fdt);
+ if (res) return res;
+
+ return 0;
+
+}
+
static const struct arch_info *get_arch_info(libxl__gc *gc,
const struct xc_dom_image *dom)
{
@@ -961,6 +1011,9 @@ next_resize:
if (info->tee == LIBXL_TEE_TYPE_OPTEE)
FDT( make_optee_node(gc, fdt) );
+ if (libxl_defbool_val(info->arch_arm.virtio))
+ FDT( make_virtio_mmio_node(gc, fdt, GUEST_VIRTIO_MMIO_BASE, GUEST_VIRTIO_MMIO_SPI) );
+
if (pfdt)
FDT( copy_partial_fdt(gc, fdt, pfdt) );
@@ -1178,6 +1231,7 @@ void libxl__arch_domain_build_info_setdefault(libxl__gc *gc,
{
/* ACPI is disabled by default */
libxl_defbool_setdefault(&b_info->acpi, false);
+ libxl_defbool_setdefault(&b_info->arch_arm.virtio, false);
if (b_info->type != LIBXL_DOMAIN_TYPE_PV)
return;
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 9d3f05f..b054bf9 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -639,6 +639,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
("arch_arm", Struct(None, [("gic_version", libxl_gic_version),
+ ("virtio", libxl_defbool),
("vuart", libxl_vuart_type),
])),
# Alternate p2m is not bound to any architecture or guest type, as it is
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index cae8eb6..10acf22 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -2581,6 +2581,7 @@ skip_usbdev:
}
xlu_cfg_get_defbool(config, "dm_restrict", &b_info->dm_restrict, 0);
+ xlu_cfg_get_defbool(config, "virtio", &b_info->arch_arm.virtio, 0);
if (c_info->type == LIBXL_DOMAIN_TYPE_HVM) {
if (!xlu_cfg_get_string (config, "vga", &buf, 0)) {
diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
index c365b1b..be7595f 100644
--- a/xen/include/public/arch-arm.h
+++ b/xen/include/public/arch-arm.h
@@ -464,6 +464,11 @@ typedef uint64_t xen_callback_t;
#define PSCI_cpu_on 2
#define PSCI_migrate 3
+/* VirtIO MMIO definitions */
+#define GUEST_VIRTIO_MMIO_BASE xen_mk_ullong(0x02000000)
+#define GUEST_VIRTIO_MMIO_SIZE xen_mk_ullong(0x200)
+#define GUEST_VIRTIO_MMIO_SPI 33
+
#endif
#ifndef __ASSEMBLY__
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* [PATCH V2 23/23] [RFC] libxl: Add support for virtio-disk configuration
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (21 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 22/23] libxl: Introduce basic virtio-mmio support on Arm Oleksandr Tyshchenko
@ 2020-10-15 16:44 ` Oleksandr Tyshchenko
2020-11-09 6:45 ` Wei Chen
2020-10-29 7:41 ` [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Masami Hiramatsu
23 siblings, 1 reply; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-15 16:44 UTC (permalink / raw)
To: xen-devel
Cc: Oleksandr Tyshchenko, Ian Jackson, Wei Liu, Anthony PERARD,
Julien Grall, Stefano Stabellini
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
This patch adds basic support for configuring and assisting virtio-disk
backend (emualator) which is intended to run out of Qemu and could be run
in any domain.
Xenstore was chosen as a communication interface for the emulator running
in non-toolstack domain to be able to get configuration either by reading
Xenstore directly or by receiving command line parameters (an updated 'xl devd'
running in the same domain would read Xenstore beforehand and call backend
executable with the required arguments).
An example of domain configuration (two disks are assigned to the guest,
the latter is in readonly mode):
vdisk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3;ro:/dev/mmcblk1p3' ]
Where per-disk Xenstore entries are:
- filename and readonly flag (configured via "vdisk" property)
- base and irq (allocated dynamically)
Besides handling 'visible' params described in configuration file,
patch also allocates virtio-mmio specific ones for each device and
writes them into Xenstore. virtio-mmio params (irq and base) are
unique per guest domain, they allocated at the domain creation time
and passed through to the emulator. Each VirtIO device has at least
one pair of these params.
TODO:
1. An extra "virtio" property could be removed.
2. Update documentation.
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
Changes RFC -> V1:
- no changes
Changes V1 -> V2:
- rebase according to the new location of libxl_virtio_disk.c
Please note, there is a real concern about VirtIO interrupts allocation.
Just copy here what Stefano said in RFC thread.
So, if we end up allocating let's say 6 virtio interrupts for a domain,
the chance of a clash with a physical interrupt of a passthrough device is real.
I am not entirely sure how to solve it, but these are a few ideas:
- choosing virtio interrupts that are less likely to conflict (maybe > 1000)
- make the virtio irq (optionally) configurable so that a user could
override the default irq and specify one that doesn't conflict
- implementing support for virq != pirq (even the xl interface doesn't
allow to specify the virq number for passthrough devices, see "irqs")
---
tools/libs/light/Makefile | 1 +
tools/libs/light/libxl_arm.c | 56 ++++++++++++---
tools/libs/light/libxl_create.c | 1 +
tools/libs/light/libxl_internal.h | 1 +
tools/libs/light/libxl_types.idl | 15 ++++
tools/libs/light/libxl_types_internal.idl | 1 +
tools/libs/light/libxl_virtio_disk.c | 109 ++++++++++++++++++++++++++++
tools/xl/Makefile | 2 +-
tools/xl/xl.h | 3 +
tools/xl/xl_cmdtable.c | 15 ++++
tools/xl/xl_parse.c | 115 ++++++++++++++++++++++++++++++
tools/xl/xl_virtio_disk.c | 46 ++++++++++++
12 files changed, 354 insertions(+), 11 deletions(-)
create mode 100644 tools/libs/light/libxl_virtio_disk.c
create mode 100644 tools/xl/xl_virtio_disk.c
diff --git a/tools/libs/light/Makefile b/tools/libs/light/Makefile
index f58a321..2ee388a 100644
--- a/tools/libs/light/Makefile
+++ b/tools/libs/light/Makefile
@@ -115,6 +115,7 @@ SRCS-y += libxl_genid.c
SRCS-y += _libxl_types.c
SRCS-y += libxl_flask.c
SRCS-y += _libxl_types_internal.c
+SRCS-y += libxl_virtio_disk.c
ifeq ($(CONFIG_LIBNL),y)
CFLAGS_LIBXL += $(LIBNL3_CFLAGS)
diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
index 588ee5a..9eb3022 100644
--- a/tools/libs/light/libxl_arm.c
+++ b/tools/libs/light/libxl_arm.c
@@ -8,6 +8,12 @@
#include <assert.h>
#include <xen/device_tree_defs.h>
+#ifndef container_of
+#define container_of(ptr, type, member) ({ \
+ typeof( ((type *)0)->member ) *__mptr = (ptr); \
+ (type *)( (char *)__mptr - offsetof(type,member) );})
+#endif
+
static const char *gicv_to_string(libxl_gic_version gic_version)
{
switch (gic_version) {
@@ -39,14 +45,32 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
vuart_enabled = true;
}
- /*
- * XXX: Handle properly virtio
- * A proper solution would be the toolstack to allocate the interrupts
- * used by each virtio backend and let the backend now which one is used
- */
if (libxl_defbool_val(d_config->b_info.arch_arm.virtio)) {
- nr_spis += (GUEST_VIRTIO_MMIO_SPI - 32) + 1;
+ uint64_t virtio_base;
+ libxl_device_virtio_disk *virtio_disk;
+
+ virtio_base = GUEST_VIRTIO_MMIO_BASE;
virtio_irq = GUEST_VIRTIO_MMIO_SPI;
+
+ if (!d_config->num_virtio_disks) {
+ LOG(ERROR, "Virtio is enabled, but no Virtio devices present\n");
+ return ERROR_FAIL;
+ }
+ virtio_disk = &d_config->virtio_disks[0];
+
+ for (i = 0; i < virtio_disk->num_disks; i++) {
+ virtio_disk->disks[i].base = virtio_base;
+ virtio_disk->disks[i].irq = virtio_irq;
+
+ LOG(DEBUG, "Allocate Virtio MMIO params: IRQ %u BASE 0x%"PRIx64,
+ virtio_irq, virtio_base);
+
+ virtio_irq ++;
+ virtio_base += GUEST_VIRTIO_MMIO_SIZE;
+ }
+ virtio_irq --;
+
+ nr_spis += (virtio_irq - 32) + 1;
virtio_enabled = true;
}
@@ -70,8 +94,9 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
}
/* The same check as for vpl011 */
- if (virtio_enabled && irq == virtio_irq) {
- LOG(ERROR, "Physical IRQ %u conflicting with virtio SPI\n", irq);
+ if (virtio_enabled &&
+ (irq >= GUEST_VIRTIO_MMIO_SPI && irq <= virtio_irq)) {
+ LOG(ERROR, "Physical IRQ %u conflicting with Virtio IRQ range\n", irq);
return ERROR_FAIL;
}
@@ -1011,8 +1036,19 @@ next_resize:
if (info->tee == LIBXL_TEE_TYPE_OPTEE)
FDT( make_optee_node(gc, fdt) );
- if (libxl_defbool_val(info->arch_arm.virtio))
- FDT( make_virtio_mmio_node(gc, fdt, GUEST_VIRTIO_MMIO_BASE, GUEST_VIRTIO_MMIO_SPI) );
+ if (libxl_defbool_val(info->arch_arm.virtio)) {
+ libxl_domain_config *d_config =
+ container_of(info, libxl_domain_config, b_info);
+ libxl_device_virtio_disk *virtio_disk = &d_config->virtio_disks[0];
+ unsigned int i;
+
+ for (i = 0; i < virtio_disk->num_disks; i++) {
+ uint64_t base = virtio_disk->disks[i].base;
+ uint32_t irq = virtio_disk->disks[i].irq;
+
+ FDT( make_virtio_mmio_node(gc, fdt, base, irq) );
+ }
+ }
if (pfdt)
FDT( copy_partial_fdt(gc, fdt, pfdt) );
diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index 321a13e..8da328d 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -1821,6 +1821,7 @@ const libxl__device_type *device_type_tbl[] = {
&libxl__dtdev_devtype,
&libxl__vdispl_devtype,
&libxl__vsnd_devtype,
+ &libxl__virtio_disk_devtype,
NULL
};
diff --git a/tools/libs/light/libxl_internal.h b/tools/libs/light/libxl_internal.h
index e26cda9..ea497bb 100644
--- a/tools/libs/light/libxl_internal.h
+++ b/tools/libs/light/libxl_internal.h
@@ -4000,6 +4000,7 @@ extern const libxl__device_type libxl__vdispl_devtype;
extern const libxl__device_type libxl__p9_devtype;
extern const libxl__device_type libxl__pvcallsif_devtype;
extern const libxl__device_type libxl__vsnd_devtype;
+extern const libxl__device_type libxl__virtio_disk_devtype;
extern const libxl__device_type *device_type_tbl[];
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index b054bf9..5f8a3ff 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -935,6 +935,20 @@ libxl_device_vsnd = Struct("device_vsnd", [
("pcms", Array(libxl_vsnd_pcm, "num_vsnd_pcms"))
])
+libxl_virtio_disk_param = Struct("virtio_disk_param", [
+ ("filename", string),
+ ("readonly", bool),
+ ("irq", uint32),
+ ("base", uint64),
+ ])
+
+libxl_device_virtio_disk = Struct("device_virtio_disk", [
+ ("backend_domid", libxl_domid),
+ ("backend_domname", string),
+ ("devid", libxl_devid),
+ ("disks", Array(libxl_virtio_disk_param, "num_disks")),
+ ])
+
libxl_domain_config = Struct("domain_config", [
("c_info", libxl_domain_create_info),
("b_info", libxl_domain_build_info),
@@ -951,6 +965,7 @@ libxl_domain_config = Struct("domain_config", [
("pvcallsifs", Array(libxl_device_pvcallsif, "num_pvcallsifs")),
("vdispls", Array(libxl_device_vdispl, "num_vdispls")),
("vsnds", Array(libxl_device_vsnd, "num_vsnds")),
+ ("virtio_disks", Array(libxl_device_virtio_disk, "num_virtio_disks")),
# a channel manifests as a console with a name,
# see docs/misc/channels.txt
("channels", Array(libxl_device_channel, "num_channels")),
diff --git a/tools/libs/light/libxl_types_internal.idl b/tools/libs/light/libxl_types_internal.idl
index 3593e21..8f71980 100644
--- a/tools/libs/light/libxl_types_internal.idl
+++ b/tools/libs/light/libxl_types_internal.idl
@@ -32,6 +32,7 @@ libxl__device_kind = Enumeration("device_kind", [
(14, "PVCALLS"),
(15, "VSND"),
(16, "VINPUT"),
+ (17, "VIRTIO_DISK"),
])
libxl__console_backend = Enumeration("console_backend", [
diff --git a/tools/libs/light/libxl_virtio_disk.c b/tools/libs/light/libxl_virtio_disk.c
new file mode 100644
index 0000000..25e7f1a
--- /dev/null
+++ b/tools/libs/light/libxl_virtio_disk.c
@@ -0,0 +1,109 @@
+/*
+ * Copyright (C) 2020 EPAM Systems Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published
+ * by the Free Software Foundation; version 2.1 only. with the special
+ * exception on linking described in file LICENSE.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ */
+
+#include "libxl_internal.h"
+
+static int libxl__device_virtio_disk_setdefault(libxl__gc *gc, uint32_t domid,
+ libxl_device_virtio_disk *virtio_disk,
+ bool hotplug)
+{
+ return libxl__resolve_domid(gc, virtio_disk->backend_domname,
+ &virtio_disk->backend_domid);
+}
+
+static int libxl__virtio_disk_from_xenstore(libxl__gc *gc, const char *libxl_path,
+ libxl_devid devid,
+ libxl_device_virtio_disk *virtio_disk)
+{
+ const char *be_path;
+ int rc;
+
+ virtio_disk->devid = devid;
+ rc = libxl__xs_read_mandatory(gc, XBT_NULL,
+ GCSPRINTF("%s/backend", libxl_path),
+ &be_path);
+ if (rc) return rc;
+
+ rc = libxl__backendpath_parse_domid(gc, be_path, &virtio_disk->backend_domid);
+ if (rc) return rc;
+
+ return 0;
+}
+
+static void libxl__update_config_virtio_disk(libxl__gc *gc,
+ libxl_device_virtio_disk *dst,
+ libxl_device_virtio_disk *src)
+{
+ dst->devid = src->devid;
+}
+
+static int libxl_device_virtio_disk_compare(libxl_device_virtio_disk *d1,
+ libxl_device_virtio_disk *d2)
+{
+ return COMPARE_DEVID(d1, d2);
+}
+
+static void libxl__device_virtio_disk_add(libxl__egc *egc, uint32_t domid,
+ libxl_device_virtio_disk *virtio_disk,
+ libxl__ao_device *aodev)
+{
+ libxl__device_add_async(egc, domid, &libxl__virtio_disk_devtype, virtio_disk, aodev);
+}
+
+static int libxl__set_xenstore_virtio_disk(libxl__gc *gc, uint32_t domid,
+ libxl_device_virtio_disk *virtio_disk,
+ flexarray_t *back, flexarray_t *front,
+ flexarray_t *ro_front)
+{
+ int rc;
+ unsigned int i;
+
+ for (i = 0; i < virtio_disk->num_disks; i++) {
+ rc = flexarray_append_pair(ro_front, GCSPRINTF("%d/filename", i),
+ GCSPRINTF("%s", virtio_disk->disks[i].filename));
+ if (rc) return rc;
+
+ rc = flexarray_append_pair(ro_front, GCSPRINTF("%d/readonly", i),
+ GCSPRINTF("%d", virtio_disk->disks[i].readonly));
+ if (rc) return rc;
+
+ rc = flexarray_append_pair(ro_front, GCSPRINTF("%d/base", i),
+ GCSPRINTF("%lu", virtio_disk->disks[i].base));
+ if (rc) return rc;
+
+ rc = flexarray_append_pair(ro_front, GCSPRINTF("%d/irq", i),
+ GCSPRINTF("%u", virtio_disk->disks[i].irq));
+ if (rc) return rc;
+ }
+
+ return 0;
+}
+
+static LIBXL_DEFINE_UPDATE_DEVID(virtio_disk)
+static LIBXL_DEFINE_DEVICE_FROM_TYPE(virtio_disk)
+static LIBXL_DEFINE_DEVICES_ADD(virtio_disk)
+
+DEFINE_DEVICE_TYPE_STRUCT(virtio_disk, VIRTIO_DISK,
+ .update_config = (device_update_config_fn_t) libxl__update_config_virtio_disk,
+ .from_xenstore = (device_from_xenstore_fn_t) libxl__virtio_disk_from_xenstore,
+ .set_xenstore_config = (device_set_xenstore_config_fn_t) libxl__set_xenstore_virtio_disk
+);
+
+/*
+ * Local variables:
+ * mode: C
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/tools/xl/Makefile b/tools/xl/Makefile
index bdf67c8..9d8f2aa 100644
--- a/tools/xl/Makefile
+++ b/tools/xl/Makefile
@@ -23,7 +23,7 @@ XL_OBJS += xl_vtpm.o xl_block.o xl_nic.o xl_usb.o
XL_OBJS += xl_sched.o xl_pci.o xl_vcpu.o xl_cdrom.o xl_mem.o
XL_OBJS += xl_info.o xl_console.o xl_misc.o
XL_OBJS += xl_vmcontrol.o xl_saverestore.o xl_migrate.o
-XL_OBJS += xl_vdispl.o xl_vsnd.o xl_vkb.o
+XL_OBJS += xl_vdispl.o xl_vsnd.o xl_vkb.o xl_virtio_disk.o
$(XL_OBJS): CFLAGS += $(CFLAGS_libxentoollog)
$(XL_OBJS): CFLAGS += $(CFLAGS_XL)
diff --git a/tools/xl/xl.h b/tools/xl/xl.h
index 06569c6..3d26f19 100644
--- a/tools/xl/xl.h
+++ b/tools/xl/xl.h
@@ -178,6 +178,9 @@ int main_vsnddetach(int argc, char **argv);
int main_vkbattach(int argc, char **argv);
int main_vkblist(int argc, char **argv);
int main_vkbdetach(int argc, char **argv);
+int main_virtio_diskattach(int argc, char **argv);
+int main_virtio_disklist(int argc, char **argv);
+int main_virtio_diskdetach(int argc, char **argv);
int main_usbctrl_attach(int argc, char **argv);
int main_usbctrl_detach(int argc, char **argv);
int main_usbdev_attach(int argc, char **argv);
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 7da6c1b..745afab 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -435,6 +435,21 @@ struct cmd_spec cmd_table[] = {
"Destroy a domain's virtual sound device",
"<Domain> <DevId>",
},
+ { "virtio-disk-attach",
+ &main_virtio_diskattach, 1, 1,
+ "Create a new virtio block device",
+ " TBD\n"
+ },
+ { "virtio-disk-list",
+ &main_virtio_disklist, 0, 0,
+ "List virtio block devices for a domain",
+ "<Domain(s)>",
+ },
+ { "virtio-disk-detach",
+ &main_virtio_diskdetach, 0, 1,
+ "Destroy a domain's virtio block device",
+ "<Domain> <DevId>",
+ },
{ "uptime",
&main_uptime, 0, 0,
"Print uptime for all/some domains",
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 10acf22..6cf3524 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -1204,6 +1204,120 @@ out:
if (rc) exit(EXIT_FAILURE);
}
+#define MAX_VIRTIO_DISKS 4
+
+static int parse_virtio_disk_config(libxl_device_virtio_disk *virtio_disk, char *token)
+{
+ char *oparg;
+ libxl_string_list disks = NULL;
+ int i, rc;
+
+ if (MATCH_OPTION("backend", token, oparg)) {
+ virtio_disk->backend_domname = strdup(oparg);
+ } else if (MATCH_OPTION("disks", token, oparg)) {
+ split_string_into_string_list(oparg, ";", &disks);
+
+ virtio_disk->num_disks = libxl_string_list_length(&disks);
+ if (virtio_disk->num_disks > MAX_VIRTIO_DISKS) {
+ fprintf(stderr, "vdisk: currently only %d disks are supported",
+ MAX_VIRTIO_DISKS);
+ return 1;
+ }
+ virtio_disk->disks = xcalloc(virtio_disk->num_disks,
+ sizeof(*virtio_disk->disks));
+
+ for(i = 0; i < virtio_disk->num_disks; i++) {
+ char *disk_opt;
+
+ rc = split_string_into_pair(disks[i], ":", &disk_opt,
+ &virtio_disk->disks[i].filename);
+ if (rc) {
+ fprintf(stderr, "vdisk: failed to split \"%s\" into pair\n",
+ disks[i]);
+ goto out;
+ }
+
+ if (!strcmp(disk_opt, "ro"))
+ virtio_disk->disks[i].readonly = 1;
+ else if (!strcmp(disk_opt, "rw"))
+ virtio_disk->disks[i].readonly = 0;
+ else {
+ fprintf(stderr, "vdisk: failed to parse \"%s\" disk option\n",
+ disk_opt);
+ rc = 1;
+ }
+ free(disk_opt);
+
+ if (rc) goto out;
+ }
+ } else {
+ fprintf(stderr, "Unknown string \"%s\" in vdisk spec\n", token);
+ rc = 1; goto out;
+ }
+
+ rc = 0;
+
+out:
+ libxl_string_list_dispose(&disks);
+ return rc;
+}
+
+static void parse_virtio_disk_list(const XLU_Config *config,
+ libxl_domain_config *d_config)
+{
+ XLU_ConfigList *virtio_disks;
+ const char *item;
+ char *buf = NULL;
+ int rc;
+
+ if (!xlu_cfg_get_list (config, "vdisk", &virtio_disks, 0, 0)) {
+ libxl_domain_build_info *b_info = &d_config->b_info;
+ int entry = 0;
+
+ /* XXX Remove an extra property */
+ libxl_defbool_setdefault(&b_info->arch_arm.virtio, false);
+ if (!libxl_defbool_val(b_info->arch_arm.virtio)) {
+ fprintf(stderr, "Virtio device requires Virtio property to be set\n");
+ exit(EXIT_FAILURE);
+ }
+
+ while ((item = xlu_cfg_get_listitem(virtio_disks, entry)) != NULL) {
+ libxl_device_virtio_disk *virtio_disk;
+ char *p;
+
+ virtio_disk = ARRAY_EXTEND_INIT(d_config->virtio_disks,
+ d_config->num_virtio_disks,
+ libxl_device_virtio_disk_init);
+
+ buf = strdup(item);
+
+ p = strtok (buf, ",");
+ while (p != NULL)
+ {
+ while (*p == ' ') p++;
+
+ rc = parse_virtio_disk_config(virtio_disk, p);
+ if (rc) goto out;
+
+ p = strtok (NULL, ",");
+ }
+
+ entry++;
+
+ if (virtio_disk->num_disks == 0) {
+ fprintf(stderr, "At least one virtio disk should be specified\n");
+ rc = 1; goto out;
+ }
+ }
+ }
+
+ rc = 0;
+
+out:
+ free(buf);
+ if (rc) exit(EXIT_FAILURE);
+}
+
void parse_config_data(const char *config_source,
const char *config_data,
int config_len,
@@ -2734,6 +2848,7 @@ skip_usbdev:
}
parse_vkb_list(config, d_config);
+ parse_virtio_disk_list(config, d_config);
xlu_cfg_get_defbool(config, "xend_suspend_evtchn_compat",
&c_info->xend_suspend_evtchn_compat, 0);
diff --git a/tools/xl/xl_virtio_disk.c b/tools/xl/xl_virtio_disk.c
new file mode 100644
index 0000000..808a7da
--- /dev/null
+++ b/tools/xl/xl_virtio_disk.c
@@ -0,0 +1,46 @@
+/*
+ * Copyright (C) 2020 EPAM Systems Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published
+ * by the Free Software Foundation; version 2.1 only. with the special
+ * exception on linking described in file LICENSE.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ */
+
+#include <stdlib.h>
+
+#include <libxl.h>
+#include <libxl_utils.h>
+#include <libxlutil.h>
+
+#include "xl.h"
+#include "xl_utils.h"
+#include "xl_parse.h"
+
+int main_virtio_diskattach(int argc, char **argv)
+{
+ return 0;
+}
+
+int main_virtio_disklist(int argc, char **argv)
+{
+ return 0;
+}
+
+int main_virtio_diskdetach(int argc, char **argv)
+{
+ return 0;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
--
2.7.4
^ permalink raw reply related [flat|nested] 109+ messages in thread
* Re: [PATCH V2 21/23] xen/arm: Add mapcache invalidation handling
2020-10-15 16:44 ` [PATCH V2 21/23] xen/arm: Add mapcache invalidation handling Oleksandr Tyshchenko
@ 2020-10-16 6:29 ` Jan Beulich
2020-10-16 8:41 ` Julien Grall
0 siblings, 1 reply; 109+ messages in thread
From: Jan Beulich @ 2020-10-16 6:29 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: xen-devel, Oleksandr Tyshchenko, Stefano Stabellini,
Julien Grall, Volodymyr Babchuk, Julien Grall
On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
> @@ -1067,7 +1068,14 @@ static int __p2m_set_entry(struct p2m_domain *p2m,
> */
> if ( p2m_is_valid(orig_pte) &&
> !mfn_eq(lpae_get_mfn(*entry), lpae_get_mfn(orig_pte)) )
> + {
> +#ifdef CONFIG_IOREQ_SERVER
> + if ( domain_has_ioreq_server(p2m->domain) &&
> + (p2m->domain == current->domain) && p2m_is_ram(orig_pte.p2m.type) )
> + p2m->domain->qemu_mapcache_invalidate = true;
> +#endif
> p2m_free_entry(p2m, orig_pte, level);
> + }
For all I have to say here, please bear in mind that I don't know
the internals of Arm memory management.
The first odd thing here the merely MFN-based condition. It may
well be that's sufficient, if there's no way to get a "not present"
entry with an MFN matching any valid MFN. (This isn't just with
your addition, but even before.)
Given how p2m_free_entry() works (or is supposed to work in the
long run), is the new code you add guaranteed to only alter leaf
entries? If not, the freeing of page tables needs deferring until
after qemu has dropped its mappings.
And with there being refcounting only for foreign pages, how do
you prevent the freeing of the page just unmapped before qemu has
dropped its possible mapping? On the x86 side this problem is one
of the reasons why PVH Dom0 isn't "supported", yet. At least a
respective code comment would seem advisable, so the issue to be
addressed won't be forgotten.
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -1442,6 +1442,7 @@ static void do_trap_hypercall(struct cpu_user_regs *regs, register_t *nr,
> const union hsr hsr)
> {
> arm_hypercall_fn_t call = NULL;
> + struct vcpu *v = current;
This ought to be named "curr".
Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 21/23] xen/arm: Add mapcache invalidation handling
2020-10-16 6:29 ` Jan Beulich
@ 2020-10-16 8:41 ` Julien Grall
2020-10-16 8:56 ` Jan Beulich
2020-11-11 0:03 ` Oleksandr
0 siblings, 2 replies; 109+ messages in thread
From: Julien Grall @ 2020-10-16 8:41 UTC (permalink / raw)
To: Jan Beulich, Oleksandr Tyshchenko
Cc: xen-devel, Oleksandr Tyshchenko, Stefano Stabellini,
Volodymyr Babchuk, Julien Grall
Hi Jan,
On 16/10/2020 07:29, Jan Beulich wrote:
> On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
>> @@ -1067,7 +1068,14 @@ static int __p2m_set_entry(struct p2m_domain *p2m,
>> */
>> if ( p2m_is_valid(orig_pte) &&
>> !mfn_eq(lpae_get_mfn(*entry), lpae_get_mfn(orig_pte)) )
>> + {
>> +#ifdef CONFIG_IOREQ_SERVER
>> + if ( domain_has_ioreq_server(p2m->domain) &&
>> + (p2m->domain == current->domain) && p2m_is_ram(orig_pte.p2m.type) )
>> + p2m->domain->qemu_mapcache_invalidate = true;
>> +#endif
>> p2m_free_entry(p2m, orig_pte, level);
>> + }
>
> For all I have to say here, please bear in mind that I don't know
> the internals of Arm memory management.
>
> The first odd thing here the merely MFN-based condition. It may
> well be that's sufficient, if there's no way to get a "not present"
> entry with an MFN matching any valid MFN. (This isn't just with
> your addition, but even before.
Invalid entries are always zeroed. So in theory the problem could arise
if MFN 0 used in the guest. It should not be possible on staging, but I
agree this should be fixed.
>
> Given how p2m_free_entry() works (or is supposed to work in the
> long run), is the new code you add guaranteed to only alter leaf
> entries?
This path may also be called with tables. I think we want to move the
check in p2m_free_entry() so we can find the correct leaf type.
> If not, the freeing of page tables needs deferring until
> after qemu has dropped its mappings.
Freeing the page tables doesn't release a page. So may I ask why we
would need to defer it?
>
> And with there being refcounting only for foreign pages, how do
> you prevent the freeing of the page just unmapped before qemu has
> dropped its possible mapping?
QEMU mappings can only be done using the foreign mapping interface. This
means that page reference count will be incremented for each QEMU
mappings. Therefore the page cannot disappear until QEMU dropped the
last reference.
> On the x86 side this problem is one
> of the reasons why PVH Dom0 isn't "supported", yet. At least a
> respective code comment would seem advisable, so the issue to be
> addressed won't be forgotten.
Are you sure? Isn't because you don't take a reference on foreign pages
while mapping it?
Anyway, Arm has supported foreign mapping since its inception. So if
there is a bug, then it should be fixed.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 21/23] xen/arm: Add mapcache invalidation handling
2020-10-16 8:41 ` Julien Grall
@ 2020-10-16 8:56 ` Jan Beulich
2020-11-11 0:03 ` Oleksandr
1 sibling, 0 replies; 109+ messages in thread
From: Jan Beulich @ 2020-10-16 8:56 UTC (permalink / raw)
To: Julien Grall
Cc: Oleksandr Tyshchenko, xen-devel, Oleksandr Tyshchenko,
Stefano Stabellini, Volodymyr Babchuk, Julien Grall
On 16.10.2020 10:41, Julien Grall wrote:
> On 16/10/2020 07:29, Jan Beulich wrote:
>> Given how p2m_free_entry() works (or is supposed to work in the
>> long run), is the new code you add guaranteed to only alter leaf
>> entries?
>
> This path may also be called with tables. I think we want to move the
> check in p2m_free_entry() so we can find the correct leaf type.
>
>> If not, the freeing of page tables needs deferring until
>> after qemu has dropped its mappings.
>
> Freeing the page tables doesn't release a page. So may I ask why we
> would need to defer it?
Oh, sorry - qemu of course doesn't use the same p2m, so the
intermediate page tables are private to the subject guest.
>> And with there being refcounting only for foreign pages, how do
>> you prevent the freeing of the page just unmapped before qemu has
>> dropped its possible mapping?
> QEMU mappings can only be done using the foreign mapping interface. This
> means that page reference count will be incremented for each QEMU
> mappings. Therefore the page cannot disappear until QEMU dropped the
> last reference.
Okay, sorry for the noise then.
Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [PATCH V2 01/23] x86/ioreq: Prepare IOREQ feature for making it common
2020-10-15 16:44 ` [PATCH V2 01/23] x86/ioreq: Prepare IOREQ feature for making it common Oleksandr Tyshchenko
@ 2020-10-20 7:13 ` Paul Durrant
2020-11-04 9:06 ` Oleksandr
2020-11-12 10:58 ` Jan Beulich
1 sibling, 1 reply; 109+ messages in thread
From: Paul Durrant @ 2020-10-20 7:13 UTC (permalink / raw)
To: 'Oleksandr Tyshchenko', xen-devel
Cc: 'Oleksandr Tyshchenko', 'Jan Beulich',
'Andrew Cooper', 'Roger Pau Monné',
'Julien Grall', 'Stefano Stabellini',
'Wei Liu', 'Julien Grall'
> -----Original Message-----
> From: Oleksandr Tyshchenko <olekstysh@gmail.com>
> Sent: 15 October 2020 17:44
> To: xen-devel@lists.xenproject.org
> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Paul Durrant <paul@xen.org>; Jan Beulich
> <jbeulich@suse.com>; Andrew Cooper <andrew.cooper3@citrix.com>; Roger Pau Monné
> <roger.pau@citrix.com>; Julien Grall <julien@xen.org>; Stefano Stabellini <sstabellini@kernel.org>;
> Wei Liu <wl@xen.org>; Julien Grall <julien.grall@arm.com>
> Subject: [PATCH V2 01/23] x86/ioreq: Prepare IOREQ feature for making it common
>
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> As a lot of x86 code can be re-used on Arm later on, this
> patch makes some preparation to x86/hvm/ioreq.c before moving
> to the common code. This way we will get a verbatim copy for
> a code movement in subsequent patch (arch/x86/hvm/ioreq.c
> will be *just* renamed to common/ioreq).
>
> This patch does the following:
> 1. Introduce *inline* arch_hvm_ioreq_init(), arch_hvm_ioreq_destroy(),
> arch_hvm_io_completion(), arch_hvm_destroy_ioreq_server() and
> hvm_ioreq_server_get_type_addr() to abstract arch specific materials.
> 2 Make hvm_map_mem_type_to_ioreq_server() *inline*. It is not going
> to be called from the common code.
> 3. Make get_ioreq_server() global. It is going to be called from
> a few places.
> 4. Add IOREQ_STATUS_* #define-s and update candidates for moving.
> 5. Re-order #include-s alphabetically.
>
> This support is going to be used on Arm to be able run device
> emulator outside of Xen hypervisor.
>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> CC: Julien Grall <julien.grall@arm.com>
>
> ---
> Please note, this is a split/cleanup/hardening of Julien's PoC:
> "Add support for Guest IO forwarding to a device emulator"
>
> Changes RFC -> V1:
> - new patch, was split from:
> "[RFC PATCH V1 01/12] hvm/ioreq: Make x86's IOREQ feature common"
> - fold the check of p->type into hvm_get_ioreq_server_range_type()
> and make it return success/failure
> - remove relocate_portio_handler() call from arch_hvm_ioreq_destroy()
> in arch/x86/hvm/ioreq.c
> - introduce arch_hvm_destroy_ioreq_server()/arch_handle_hvm_io_completion()
>
> Changes V1 -> V2:
> - update patch description
> - make arch functions inline and put them into arch header
> to achieve a truly rename by the subsequent patch
> - return void in arch_hvm_destroy_ioreq_server()
> - return bool in arch_hvm_ioreq_destroy()
> - bring relocate_portio_handler() back to arch_hvm_ioreq_destroy()
> - rename IOREQ_IO* to IOREQ_STATUS*
> - remove *handle* from arch_handle_hvm_io_completion()
> - re-order #include-s alphabetically
> - rename hvm_get_ioreq_server_range_type() to hvm_ioreq_server_get_type_addr()
> and add "const" to several arguments
> ---
> xen/arch/x86/hvm/ioreq.c | 153 +++++--------------------------------
> xen/include/asm-x86/hvm/ioreq.h | 165 +++++++++++++++++++++++++++++++++++++++-
> 2 files changed, 184 insertions(+), 134 deletions(-)
>
> diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
> index 1cc27df..d3433d7 100644
> --- a/xen/arch/x86/hvm/ioreq.c
> +++ b/xen/arch/x86/hvm/ioreq.c
> @@ -1,5 +1,5 @@
> /*
> - * hvm/io.c: hardware virtual machine I/O emulation
> + * ioreq.c: hardware virtual machine I/O emulation
> *
> * Copyright (c) 2016 Citrix Systems Inc.
> *
> @@ -17,21 +17,18 @@
> */
>
> #include <xen/ctype.h>
> +#include <xen/domain.h>
> +#include <xen/event.h>
> #include <xen/init.h>
> +#include <xen/irq.h>
> #include <xen/lib.h>
> -#include <xen/trace.h>
> +#include <xen/paging.h>
> #include <xen/sched.h>
> -#include <xen/irq.h>
> #include <xen/softirq.h>
> -#include <xen/domain.h>
> -#include <xen/event.h>
> -#include <xen/paging.h>
> +#include <xen/trace.h>
> #include <xen/vpci.h>
>
> -#include <asm/hvm/emulate.h>
> -#include <asm/hvm/hvm.h>
> #include <asm/hvm/ioreq.h>
> -#include <asm/hvm/vmx/vmx.h>
>
> #include <public/hvm/ioreq.h>
> #include <public/hvm/params.h>
> @@ -48,8 +45,8 @@ static void set_ioreq_server(struct domain *d, unsigned int id,
> #define GET_IOREQ_SERVER(d, id) \
> (d)->arch.hvm.ioreq_server.server[id]
>
> -static struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
> - unsigned int id)
> +struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
> + unsigned int id)
> {
> if ( id >= MAX_NR_IOREQ_SERVERS )
> return NULL;
> @@ -209,19 +206,8 @@ bool handle_hvm_io_completion(struct vcpu *v)
> return handle_pio(vio->io_req.addr, vio->io_req.size,
> vio->io_req.dir);
>
> - case HVMIO_realmode_completion:
> - {
> - struct hvm_emulate_ctxt ctxt;
> -
> - hvm_emulate_init_once(&ctxt, NULL, guest_cpu_user_regs());
> - vmx_realmode_emulate_one(&ctxt);
> - hvm_emulate_writeback(&ctxt);
> -
> - break;
> - }
> default:
> - ASSERT_UNREACHABLE();
> - break;
> + return arch_hvm_io_completion(io_completion);
> }
>
> return true;
> @@ -855,7 +841,7 @@ int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
>
> domain_pause(d);
>
> - p2m_set_ioreq_server(d, 0, s);
> + arch_hvm_destroy_ioreq_server(s);
>
> hvm_ioreq_server_disable(s);
>
> @@ -1080,54 +1066,6 @@ int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
> return rc;
> }
>
> -/*
> - * Map or unmap an ioreq server to specific memory type. For now, only
> - * HVMMEM_ioreq_server is supported, and in the future new types can be
> - * introduced, e.g. HVMMEM_ioreq_serverX mapped to ioreq server X. And
> - * currently, only write operations are to be forwarded to an ioreq server.
> - * Support for the emulation of read operations can be added when an ioreq
> - * server has such requirement in the future.
> - */
> -int hvm_map_mem_type_to_ioreq_server(struct domain *d, ioservid_t id,
> - uint32_t type, uint32_t flags)
> -{
> - struct hvm_ioreq_server *s;
> - int rc;
> -
> - if ( type != HVMMEM_ioreq_server )
> - return -EINVAL;
> -
> - if ( flags & ~XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
> - return -EINVAL;
> -
> - spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
> -
> - s = get_ioreq_server(d, id);
> -
> - rc = -ENOENT;
> - if ( !s )
> - goto out;
> -
> - rc = -EPERM;
> - if ( s->emulator != current->domain )
> - goto out;
> -
> - rc = p2m_set_ioreq_server(d, flags, s);
> -
> - out:
> - spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
> -
> - if ( rc == 0 && flags == 0 )
> - {
> - struct p2m_domain *p2m = p2m_get_hostp2m(d);
> -
> - if ( read_atomic(&p2m->ioreq.entry_count) )
> - p2m_change_entry_type_global(d, p2m_ioreq_server, p2m_ram_rw);
> - }
> -
> - return rc;
> -}
> -
> int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
> bool enabled)
> {
> @@ -1215,7 +1153,7 @@ void hvm_destroy_all_ioreq_servers(struct domain *d)
> struct hvm_ioreq_server *s;
> unsigned int id;
>
> - if ( !relocate_portio_handler(d, 0xcf8, 0xcf8, 4) )
> + if ( !arch_hvm_ioreq_destroy(d) )
> return;
>
> spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
> @@ -1243,50 +1181,13 @@ struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,
> ioreq_t *p)
> {
> struct hvm_ioreq_server *s;
> - uint32_t cf8;
> uint8_t type;
> uint64_t addr;
> unsigned int id;
>
> - if ( p->type != IOREQ_TYPE_COPY && p->type != IOREQ_TYPE_PIO )
> + if ( hvm_ioreq_server_get_type_addr(d, p, &type, &addr) )
> return NULL;
>
> - cf8 = d->arch.hvm.pci_cf8;
> -
> - if ( p->type == IOREQ_TYPE_PIO &&
> - (p->addr & ~3) == 0xcfc &&
> - CF8_ENABLED(cf8) )
> - {
> - uint32_t x86_fam;
> - pci_sbdf_t sbdf;
> - unsigned int reg;
> -
> - reg = hvm_pci_decode_addr(cf8, p->addr, &sbdf);
> -
> - /* PCI config data cycle */
> - type = XEN_DMOP_IO_RANGE_PCI;
> - addr = ((uint64_t)sbdf.sbdf << 32) | reg;
> - /* AMD extended configuration space access? */
> - if ( CF8_ADDR_HI(cf8) &&
> - d->arch.cpuid->x86_vendor == X86_VENDOR_AMD &&
> - (x86_fam = get_cpu_family(
> - d->arch.cpuid->basic.raw_fms, NULL, NULL)) >= 0x10 &&
> - x86_fam < 0x17 )
> - {
> - uint64_t msr_val;
> -
> - if ( !rdmsr_safe(MSR_AMD64_NB_CFG, msr_val) &&
> - (msr_val & (1ULL << AMD64_NB_CFG_CF8_EXT_ENABLE_BIT)) )
> - addr |= CF8_ADDR_HI(cf8);
> - }
> - }
> - else
> - {
> - type = (p->type == IOREQ_TYPE_PIO) ?
> - XEN_DMOP_IO_RANGE_PORT : XEN_DMOP_IO_RANGE_MEMORY;
> - addr = p->addr;
> - }
> -
> FOR_EACH_IOREQ_SERVER(d, id, s)
> {
> struct rangeset *r;
> @@ -1351,7 +1252,7 @@ static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
> pg = iorp->va;
>
> if ( !pg )
> - return X86EMUL_UNHANDLEABLE;
> + return IOREQ_STATUS_UNHANDLED;
>
> /*
> * Return 0 for the cases we can't deal with:
> @@ -1381,7 +1282,7 @@ static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
> break;
> default:
> gdprintk(XENLOG_WARNING, "unexpected ioreq size: %u\n", p->size);
> - return X86EMUL_UNHANDLEABLE;
> + return IOREQ_STATUS_UNHANDLED;
> }
>
> spin_lock(&s->bufioreq_lock);
> @@ -1391,7 +1292,7 @@ static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
> {
> /* The queue is full: send the iopacket through the normal path. */
> spin_unlock(&s->bufioreq_lock);
> - return X86EMUL_UNHANDLEABLE;
> + return IOREQ_STATUS_UNHANDLED;
> }
>
> pg->buf_ioreq[pg->ptrs.write_pointer % IOREQ_BUFFER_SLOT_NUM] = bp;
> @@ -1422,7 +1323,7 @@ static int hvm_send_buffered_ioreq(struct hvm_ioreq_server *s, ioreq_t *p)
> notify_via_xen_event_channel(d, s->bufioreq_evtchn);
> spin_unlock(&s->bufioreq_lock);
>
> - return X86EMUL_OKAY;
> + return IOREQ_STATUS_HANDLED;
> }
>
> int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
> @@ -1438,7 +1339,7 @@ int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
> return hvm_send_buffered_ioreq(s, proto_p);
>
> if ( unlikely(!vcpu_start_shutdown_deferral(curr)) )
> - return X86EMUL_RETRY;
> + return IOREQ_STATUS_RETRY;
>
> list_for_each_entry ( sv,
> &s->ioreq_vcpu_list,
> @@ -1478,11 +1379,11 @@ int hvm_send_ioreq(struct hvm_ioreq_server *s, ioreq_t *proto_p,
> notify_via_xen_event_channel(d, port);
>
> sv->pending = true;
> - return X86EMUL_RETRY;
> + return IOREQ_STATUS_RETRY;
> }
> }
>
> - return X86EMUL_UNHANDLEABLE;
> + return IOREQ_STATUS_UNHANDLED;
> }
>
> unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered)
> @@ -1496,30 +1397,18 @@ unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered)
> if ( !s->enabled )
> continue;
>
> - if ( hvm_send_ioreq(s, p, buffered) == X86EMUL_UNHANDLEABLE )
> + if ( hvm_send_ioreq(s, p, buffered) == IOREQ_STATUS_UNHANDLED )
> failed++;
> }
>
> return failed;
> }
>
> -static int hvm_access_cf8(
> - int dir, unsigned int port, unsigned int bytes, uint32_t *val)
> -{
> - struct domain *d = current->domain;
> -
> - if ( dir == IOREQ_WRITE && bytes == 4 )
> - d->arch.hvm.pci_cf8 = *val;
> -
> - /* We always need to fall through to the catch all emulator */
> - return X86EMUL_UNHANDLEABLE;
> -}
> -
> void hvm_ioreq_init(struct domain *d)
> {
> spin_lock_init(&d->arch.hvm.ioreq_server.lock);
>
> - register_portio_handler(d, 0xcf8, 4, hvm_access_cf8);
> + arch_hvm_ioreq_init(d);
> }
>
> /*
> diff --git a/xen/include/asm-x86/hvm/ioreq.h b/xen/include/asm-x86/hvm/ioreq.h
> index e2588e9..376e2ef 100644
> --- a/xen/include/asm-x86/hvm/ioreq.h
> +++ b/xen/include/asm-x86/hvm/ioreq.h
> @@ -19,6 +19,165 @@
> #ifndef __ASM_X86_HVM_IOREQ_H__
> #define __ASM_X86_HVM_IOREQ_H__
>
> +#include <asm/hvm/emulate.h>
> +#include <asm/hvm/vmx/vmx.h>
> +
> +#include <public/hvm/params.h>
> +
> +struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
> + unsigned int id);
> +
> +static inline bool arch_hvm_io_completion(enum hvm_io_completion io_completion)
> +{
> + switch ( io_completion )
> + {
> + case HVMIO_realmode_completion:
> + {
> + struct hvm_emulate_ctxt ctxt;
> +
> + hvm_emulate_init_once(&ctxt, NULL, guest_cpu_user_regs());
> + vmx_realmode_emulate_one(&ctxt);
> + hvm_emulate_writeback(&ctxt);
> +
> + break;
> + }
> +
> + default:
> + ASSERT_UNREACHABLE();
> + break;
> + }
> +
> + return true;
> +}
> +
> +/* Called when target domain is paused */
> +static inline void arch_hvm_destroy_ioreq_server(struct hvm_ioreq_server *s)
> +{
> + p2m_set_ioreq_server(s->target, 0, s);
> +}
> +
> +/*
> + * Map or unmap an ioreq server to specific memory type. For now, only
> + * HVMMEM_ioreq_server is supported, and in the future new types can be
> + * introduced, e.g. HVMMEM_ioreq_serverX mapped to ioreq server X. And
> + * currently, only write operations are to be forwarded to an ioreq server.
> + * Support for the emulation of read operations can be added when an ioreq
> + * server has such requirement in the future.
> + */
> +static inline int hvm_map_mem_type_to_ioreq_server(struct domain *d,
> + ioservid_t id,
> + uint32_t type,
> + uint32_t flags)
> +{
> + struct hvm_ioreq_server *s;
> + int rc;
> +
> + if ( type != HVMMEM_ioreq_server )
> + return -EINVAL;
> +
> + if ( flags & ~XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
> + return -EINVAL;
> +
> + spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
> +
> + s = get_ioreq_server(d, id);
> +
> + rc = -ENOENT;
> + if ( !s )
> + goto out;
> +
> + rc = -EPERM;
> + if ( s->emulator != current->domain )
> + goto out;
> +
> + rc = p2m_set_ioreq_server(d, flags, s);
> +
> + out:
> + spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
> +
> + if ( rc == 0 && flags == 0 )
> + {
> + struct p2m_domain *p2m = p2m_get_hostp2m(d);
> +
> + if ( read_atomic(&p2m->ioreq.entry_count) )
> + p2m_change_entry_type_global(d, p2m_ioreq_server, p2m_ram_rw);
> + }
> +
> + return rc;
> +}
> +
The above doesn't really feel right to me. It's really an entry point into the ioreq server code and as such I think it ought to be left in the common code. I suggest replacing the p2m_set_ioreq_server() function with an arch specific function (also taking the type) which you can then implement here.
The rest of the patch looks ok.
Paul
> +static inline int hvm_ioreq_server_get_type_addr(const struct domain *d,
> + const ioreq_t *p,
> + uint8_t *type,
> + uint64_t *addr)
> +{
> + uint32_t cf8 = d->arch.hvm.pci_cf8;
> +
> + if ( p->type != IOREQ_TYPE_COPY && p->type != IOREQ_TYPE_PIO )
> + return -EINVAL;
> +
> + if ( p->type == IOREQ_TYPE_PIO &&
> + (p->addr & ~3) == 0xcfc &&
> + CF8_ENABLED(cf8) )
> + {
> + uint32_t x86_fam;
> + pci_sbdf_t sbdf;
> + unsigned int reg;
> +
> + reg = hvm_pci_decode_addr(cf8, p->addr, &sbdf);
> +
> + /* PCI config data cycle */
> + *type = XEN_DMOP_IO_RANGE_PCI;
> + *addr = ((uint64_t)sbdf.sbdf << 32) | reg;
> + /* AMD extended configuration space access? */
> + if ( CF8_ADDR_HI(cf8) &&
> + d->arch.cpuid->x86_vendor == X86_VENDOR_AMD &&
> + (x86_fam = get_cpu_family(
> + d->arch.cpuid->basic.raw_fms, NULL, NULL)) >= 0x10 &&
> + x86_fam < 0x17 )
> + {
> + uint64_t msr_val;
> +
> + if ( !rdmsr_safe(MSR_AMD64_NB_CFG, msr_val) &&
> + (msr_val & (1ULL << AMD64_NB_CFG_CF8_EXT_ENABLE_BIT)) )
> + *addr |= CF8_ADDR_HI(cf8);
> + }
> + }
> + else
> + {
> + *type = (p->type == IOREQ_TYPE_PIO) ?
> + XEN_DMOP_IO_RANGE_PORT : XEN_DMOP_IO_RANGE_MEMORY;
> + *addr = p->addr;
> + }
> +
> + return 0;
> +}
> +
> +static inline int hvm_access_cf8(
> + int dir, unsigned int port, unsigned int bytes, uint32_t *val)
> +{
> + struct domain *d = current->domain;
> +
> + if ( dir == IOREQ_WRITE && bytes == 4 )
> + d->arch.hvm.pci_cf8 = *val;
> +
> + /* We always need to fall through to the catch all emulator */
> + return X86EMUL_UNHANDLEABLE;
> +}
> +
> +static inline void arch_hvm_ioreq_init(struct domain *d)
> +{
> + register_portio_handler(d, 0xcf8, 4, hvm_access_cf8);
> +}
> +
> +static inline bool arch_hvm_ioreq_destroy(struct domain *d)
> +{
> + if ( !relocate_portio_handler(d, 0xcf8, 0xcf8, 4) )
> + return false;
> +
> + return true;
> +}
> +
> bool hvm_io_pending(struct vcpu *v);
> bool handle_hvm_io_completion(struct vcpu *v);
> bool is_ioreq_server_page(struct domain *d, const struct page_info *page);
> @@ -38,8 +197,6 @@ int hvm_map_io_range_to_ioreq_server(struct domain *d, ioservid_t id,
> int hvm_unmap_io_range_from_ioreq_server(struct domain *d, ioservid_t id,
> uint32_t type, uint64_t start,
> uint64_t end);
> -int hvm_map_mem_type_to_ioreq_server(struct domain *d, ioservid_t id,
> - uint32_t type, uint32_t flags);
> int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id,
> bool enabled);
>
> @@ -55,6 +212,10 @@ unsigned int hvm_broadcast_ioreq(ioreq_t *p, bool buffered);
>
> void hvm_ioreq_init(struct domain *d);
>
> +#define IOREQ_STATUS_HANDLED X86EMUL_OKAY
> +#define IOREQ_STATUS_UNHANDLED X86EMUL_UNHANDLEABLE
> +#define IOREQ_STATUS_RETRY X86EMUL_RETRY
> +
> #endif /* __ASM_X86_HVM_IOREQ_H__ */
>
> /*
> --
> 2.7.4
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [PATCH V2 02/23] xen/ioreq: Make x86's IOREQ feature common
2020-10-15 16:44 ` [PATCH V2 02/23] xen/ioreq: Make x86's IOREQ feature common Oleksandr Tyshchenko
@ 2020-10-20 7:57 ` Paul Durrant
2020-11-10 16:45 ` Oleksandr
2020-11-12 11:11 ` Jan Beulich
1 sibling, 1 reply; 109+ messages in thread
From: Paul Durrant @ 2020-10-20 7:57 UTC (permalink / raw)
To: 'Oleksandr Tyshchenko', xen-devel
Cc: 'Oleksandr Tyshchenko', 'Andrew Cooper',
'George Dunlap', 'Ian Jackson',
'Jan Beulich', 'Julien Grall',
'Stefano Stabellini', 'Wei Liu',
'Roger Pau Monné', 'Tim Deegan',
'Julien Grall'
> -----Original Message-----
> From: Oleksandr Tyshchenko <olekstysh@gmail.com>
> Sent: 15 October 2020 17:44
> To: xen-devel@lists.xenproject.org
> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Andrew Cooper <andrew.cooper3@citrix.com>;
> George Dunlap <george.dunlap@citrix.com>; Ian Jackson <iwj@xenproject.org>; Jan Beulich
> <jbeulich@suse.com>; Julien Grall <julien@xen.org>; Stefano Stabellini <sstabellini@kernel.org>; Wei
> Liu <wl@xen.org>; Roger Pau Monné <roger.pau@citrix.com>; Paul Durrant <paul@xen.org>; Tim Deegan
> <tim@xen.org>; Julien Grall <julien.grall@arm.com>
> Subject: [PATCH V2 02/23] xen/ioreq: Make x86's IOREQ feature common
>
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> As a lot of x86 code can be re-used on Arm later on, this patch
> moves previously prepared x86/hvm/ioreq.c to the common code.
>
> The common IOREQ feature is supposed to be built with IOREQ_SERVER
> option enabled, which is selected for x86's config HVM for now.
>
> In order to avoid having a gigantic patch here, the subsequent
> patches will update remaining bits in the common code step by step:
> - Make IOREQ related structs/materials common
> - Drop the "hvm" prefixes and infixes
FWIW you could tackle the naming changes in patch #1.
> - Remove layering violation by moving corresponding fields
> out of *arch.hvm* or abstracting away accesses to them
>
> This support is going to be used on Arm to be able run device
> emulator outside of Xen hypervisor.
>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> CC: Julien Grall <julien.grall@arm.com>
>
> ---
> Please note, this is a split/cleanup/hardening of Julien's PoC:
> "Add support for Guest IO forwarding to a device emulator"
>
> ***
> Please note, this patch depends on the following which is
> on review:
> https://patchwork.kernel.org/patch/11816689/
> ***
>
> Changes RFC -> V1:
> - was split into three patches:
> - x86/ioreq: Prepare IOREQ feature for making it common
> - xen/ioreq: Make x86's IOREQ feature common
> - xen/ioreq: Make x86's hvm_ioreq_needs_completion() common
> - update MAINTAINERS file
> - do not use a separate subdir for the IOREQ stuff, move it to:
> - xen/common/ioreq.c
> - xen/include/xen/ioreq.h
> - update x86's files to include xen/ioreq.h
> - remove unneeded headers in arch/x86/hvm/ioreq.c
> - re-order the headers alphabetically in common/ioreq.c
> - update common/ioreq.c according to the newly introduced arch functions:
> arch_hvm_destroy_ioreq_server()/arch_handle_hvm_io_completion()
>
> Changes V1 -> V2:
> - update patch description
> - make everything needed in the previous patch to achieve
> a truly rename here
> - don't include unnecessary headers from asm-x86/hvm/ioreq.h
> and xen/ioreq.h
> - use __XEN_IOREQ_H__ instead of __IOREQ_H__
> - move get_ioreq_server() to common/ioreq.c
> ---
> MAINTAINERS | 8 +-
> xen/arch/x86/Kconfig | 1 +
> xen/arch/x86/hvm/Makefile | 1 -
> xen/arch/x86/hvm/ioreq.c | 1422 ---------------------------------------
> xen/arch/x86/mm.c | 2 +-
> xen/arch/x86/mm/shadow/common.c | 2 +-
> xen/common/Kconfig | 3 +
> xen/common/Makefile | 1 +
> xen/common/ioreq.c | 1422 +++++++++++++++++++++++++++++++++++++++
> xen/include/asm-x86/hvm/ioreq.h | 39 +-
> xen/include/xen/ioreq.h | 71 ++
> 11 files changed, 1509 insertions(+), 1463 deletions(-)
> delete mode 100644 xen/arch/x86/hvm/ioreq.c
> create mode 100644 xen/common/ioreq.c
> create mode 100644 xen/include/xen/ioreq.h
>
[snip]
> +static void hvm_remove_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
> +
> +{
> + struct domain *d = s->target;
> + struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
> +
> + if ( gfn_eq(iorp->gfn, INVALID_GFN) )
> + return;
> +
> + if ( guest_physmap_remove_page(d, iorp->gfn,
> + page_to_mfn(iorp->page), 0) )
> + domain_crash(d);
> + clear_page(iorp->va);
> +}
> +
> +static int hvm_add_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
> +{
> + struct domain *d = s->target;
> + struct hvm_ioreq_page *iorp = buf ? &s->bufioreq : &s->ioreq;
> + int rc;
> +
> + if ( gfn_eq(iorp->gfn, INVALID_GFN) )
> + return 0;
> +
> + clear_page(iorp->va);
> +
> + rc = guest_physmap_add_page(d, iorp->gfn,
> + page_to_mfn(iorp->page), 0);
> + if ( rc == 0 )
> + paging_mark_pfn_dirty(d, _pfn(gfn_x(iorp->gfn)));
> +
> + return rc;
> +}
> +
The 'legacy' mechanism of mapping magic pages for ioreq servers should remain x86 specific I think that aspect of the code needs to remain behind and not get moved into common code. You could do that in arch specific calls in hvm_ioreq_server_enable/disable() and hvm_get_ioreq_server_info().
Paul
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [PATCH V2 03/23] xen/ioreq: Make x86's hvm_ioreq_needs_completion() common
2020-10-15 16:44 ` [PATCH V2 03/23] xen/ioreq: Make x86's hvm_ioreq_needs_completion() common Oleksandr Tyshchenko
@ 2020-10-20 7:59 ` Paul Durrant
0 siblings, 0 replies; 109+ messages in thread
From: Paul Durrant @ 2020-10-20 7:59 UTC (permalink / raw)
To: 'Oleksandr Tyshchenko', xen-devel
Cc: 'Oleksandr Tyshchenko', 'Jan Beulich',
'Andrew Cooper', 'Roger Pau Monné',
'Wei Liu', 'Julien Grall',
'Stefano Stabellini', 'Julien Grall'
> -----Original Message-----
> From: Oleksandr Tyshchenko <olekstysh@gmail.com>
> Sent: 15 October 2020 17:44
> To: xen-devel@lists.xenproject.org
> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Paul Durrant <paul@xen.org>; Jan Beulich
> <jbeulich@suse.com>; Andrew Cooper <andrew.cooper3@citrix.com>; Roger Pau Monné
> <roger.pau@citrix.com>; Wei Liu <wl@xen.org>; Julien Grall <julien@xen.org>; Stefano Stabellini
> <sstabellini@kernel.org>; Julien Grall <julien.grall@arm.com>
> Subject: [PATCH V2 03/23] xen/ioreq: Make x86's hvm_ioreq_needs_completion() common
>
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> The IOREQ is a common feature now and this helper will be used
> on Arm as is. Move it to xen/ioreq.h and remove "hvm" prefix.
>
> Although PIO handling on Arm is not introduced with the current series
> (it will be implemented when we add support for vPCI), technically
> the PIOs exist on Arm (however they are accessed the same way as MMIO)
> and it would be better not to diverge now.
>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> CC: Julien Grall <julien.grall@arm.com>
Reviewed-by: Paul Durrant <paul@xen.org>
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio()
2020-10-15 16:44 ` [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio() Oleksandr Tyshchenko
@ 2020-10-20 9:14 ` Paul Durrant
2020-10-20 10:05 ` Jan Beulich
0 siblings, 1 reply; 109+ messages in thread
From: Paul Durrant @ 2020-10-20 9:14 UTC (permalink / raw)
To: 'Oleksandr Tyshchenko', xen-devel
Cc: 'Oleksandr Tyshchenko', 'Jan Beulich',
'Andrew Cooper', 'Roger Pau Monné',
'Wei Liu', 'Julien Grall',
'Stefano Stabellini', 'Julien Grall'
> -----Original Message-----
> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of Oleksandr Tyshchenko
> Sent: 15 October 2020 17:44
> To: xen-devel@lists.xenproject.org
> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Paul Durrant <paul@xen.org>; Jan Beulich
> <jbeulich@suse.com>; Andrew Cooper <andrew.cooper3@citrix.com>; Roger Pau Monné
> <roger.pau@citrix.com>; Wei Liu <wl@xen.org>; Julien Grall <julien@xen.org>; Stefano Stabellini
> <sstabellini@kernel.org>; Julien Grall <julien.grall@arm.com>
> Subject: [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio()
>
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> The IOREQ is a common feature now and Arm will have its own
> implementation.
>
> But the name of the function is pretty generic and can be confusing
> on Arm (we already have a try_handle_mmio()).
>
> In order not to rename the function (which is used for a varying
> set of purposes on x86) globally and get non-confusing variant on Arm
> provide an alias ioreq_complete_mmio() to be used on common and
> Arm code.
>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> CC: Julien Grall <julien.grall@arm.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>
>
> ---
> Please note, this is a split/cleanup/hardening of Julien's PoC:
> "Add support for Guest IO forwarding to a device emulator"
>
> Changes RFC -> V1:
> - new patch
>
> Changes V1 -> V2:
> - remove "handle"
> - add Jan's A-b
> ---
> xen/common/ioreq.c | 2 +-
> xen/include/asm-x86/hvm/ioreq.h | 2 ++
> 2 files changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
> index c89df7a..29ad48e 100644
> --- a/xen/common/ioreq.c
> +++ b/xen/common/ioreq.c
> @@ -200,7 +200,7 @@ bool handle_hvm_io_completion(struct vcpu *v)
> break;
>
> case HVMIO_mmio_completion:
> - return handle_mmio();
> + return ioreq_complete_mmio();
>
> case HVMIO_pio_completion:
> return handle_pio(vio->io_req.addr, vio->io_req.size,
> diff --git a/xen/include/asm-x86/hvm/ioreq.h b/xen/include/asm-x86/hvm/ioreq.h
> index a3d8faa..a147856 100644
> --- a/xen/include/asm-x86/hvm/ioreq.h
> +++ b/xen/include/asm-x86/hvm/ioreq.h
> @@ -181,6 +181,8 @@ static inline bool arch_hvm_ioreq_destroy(struct domain *d)
> #define IOREQ_STATUS_UNHANDLED X86EMUL_UNHANDLEABLE
> #define IOREQ_STATUS_RETRY X86EMUL_RETRY
>
> +#define ioreq_complete_mmio handle_mmio
> +
A #define? Really? Can we not have a static inline?
Paul
> #endif /* __ASM_X86_HVM_IOREQ_H__ */
>
> /*
> --
> 2.7.4
>
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [PATCH V2 05/23] xen/ioreq: Make x86's hvm_mmio_first(last)_byte() common
2020-10-15 16:44 ` [PATCH V2 05/23] xen/ioreq: Make x86's hvm_mmio_first(last)_byte() common Oleksandr Tyshchenko
@ 2020-10-20 9:15 ` Paul Durrant
0 siblings, 0 replies; 109+ messages in thread
From: Paul Durrant @ 2020-10-20 9:15 UTC (permalink / raw)
To: 'Oleksandr Tyshchenko', xen-devel
Cc: 'Oleksandr Tyshchenko', 'Jan Beulich',
'Andrew Cooper', 'Roger Pau Monné',
'Wei Liu', 'Julien Grall',
'Stefano Stabellini', 'Julien Grall'
> -----Original Message-----
> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of Oleksandr Tyshchenko
> Sent: 15 October 2020 17:44
> To: xen-devel@lists.xenproject.org
> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Paul Durrant <paul@xen.org>; Jan Beulich
> <jbeulich@suse.com>; Andrew Cooper <andrew.cooper3@citrix.com>; Roger Pau Monné
> <roger.pau@citrix.com>; Wei Liu <wl@xen.org>; Julien Grall <julien@xen.org>; Stefano Stabellini
> <sstabellini@kernel.org>; Julien Grall <julien.grall@arm.com>
> Subject: [PATCH V2 05/23] xen/ioreq: Make x86's hvm_mmio_first(last)_byte() common
>
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> The IOREQ is a common feature now and these helpers will be used
> on Arm as is. Move them to xen/ioreq.h and replace "hvm" prefixes
> with "ioreq".
>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> CC: Julien Grall <julien.grall@arm.com>
>
Reviewed-by: Paul Durrant <paul@xen.org>
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio()
2020-10-20 9:14 ` Paul Durrant
@ 2020-10-20 10:05 ` Jan Beulich
2020-10-20 10:38 ` Paul Durrant
0 siblings, 1 reply; 109+ messages in thread
From: Jan Beulich @ 2020-10-20 10:05 UTC (permalink / raw)
To: paul
Cc: 'Oleksandr Tyshchenko',
xen-devel, 'Oleksandr Tyshchenko',
'Andrew Cooper', 'Roger Pau Monné',
'Wei Liu', 'Julien Grall',
'Stefano Stabellini', 'Julien Grall'
On 20.10.2020 11:14, Paul Durrant wrote:
>> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of Oleksandr Tyshchenko
>> Sent: 15 October 2020 17:44
>>
>> --- a/xen/include/asm-x86/hvm/ioreq.h
>> +++ b/xen/include/asm-x86/hvm/ioreq.h
>> @@ -181,6 +181,8 @@ static inline bool arch_hvm_ioreq_destroy(struct domain *d)
>> #define IOREQ_STATUS_UNHANDLED X86EMUL_UNHANDLEABLE
>> #define IOREQ_STATUS_RETRY X86EMUL_RETRY
>>
>> +#define ioreq_complete_mmio handle_mmio
>> +
>
> A #define? Really? Can we not have a static inline?
I guess this would require further shuffling: handle_mmio() is
an inline function in hvm/emulate.h, and hvm/ioreq.h has no
need to include the former (and imo it also shouldn't have).
Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio()
2020-10-20 10:05 ` Jan Beulich
@ 2020-10-20 10:38 ` Paul Durrant
2020-11-10 19:44 ` Oleksandr
0 siblings, 1 reply; 109+ messages in thread
From: Paul Durrant @ 2020-10-20 10:38 UTC (permalink / raw)
To: 'Jan Beulich'
Cc: 'Oleksandr Tyshchenko',
xen-devel, 'Oleksandr Tyshchenko',
'Andrew Cooper', 'Roger Pau Monné',
'Wei Liu', 'Julien Grall',
'Stefano Stabellini', 'Julien Grall'
> -----Original Message-----
> From: Jan Beulich <jbeulich@suse.com>
> Sent: 20 October 2020 11:05
> To: paul@xen.org
> Cc: 'Oleksandr Tyshchenko' <olekstysh@gmail.com>; xen-devel@lists.xenproject.org; 'Oleksandr
> Tyshchenko' <oleksandr_tyshchenko@epam.com>; 'Andrew Cooper' <andrew.cooper3@citrix.com>; 'Roger Pau
> Monné' <roger.pau@citrix.com>; 'Wei Liu' <wl@xen.org>; 'Julien Grall' <julien@xen.org>; 'Stefano
> Stabellini' <sstabellini@kernel.org>; 'Julien Grall' <julien.grall@arm.com>
> Subject: Re: [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio()
>
> On 20.10.2020 11:14, Paul Durrant wrote:
> >> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of Oleksandr Tyshchenko
> >> Sent: 15 October 2020 17:44
> >>
> >> --- a/xen/include/asm-x86/hvm/ioreq.h
> >> +++ b/xen/include/asm-x86/hvm/ioreq.h
> >> @@ -181,6 +181,8 @@ static inline bool arch_hvm_ioreq_destroy(struct domain *d)
> >> #define IOREQ_STATUS_UNHANDLED X86EMUL_UNHANDLEABLE
> >> #define IOREQ_STATUS_RETRY X86EMUL_RETRY
> >>
> >> +#define ioreq_complete_mmio handle_mmio
> >> +
> >
> > A #define? Really? Can we not have a static inline?
>
> I guess this would require further shuffling: handle_mmio() is
> an inline function in hvm/emulate.h, and hvm/ioreq.h has no
> need to include the former (and imo it also shouldn't have).
>
I see. I think we need an x86 ioreq.c anyway, to deal with the legacy use of magic pages, so it could be dealt with there instead.
Paul
> Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [PATCH V2 08/23] xen/ioreq: Introduce ioreq_params to abstract accesses to arch.hvm.params
2020-10-15 16:44 ` [PATCH V2 08/23] xen/ioreq: Introduce ioreq_params to abstract accesses to arch.hvm.params Oleksandr Tyshchenko
@ 2020-10-20 10:41 ` Paul Durrant
2020-11-10 20:00 ` Oleksandr
0 siblings, 1 reply; 109+ messages in thread
From: Paul Durrant @ 2020-10-20 10:41 UTC (permalink / raw)
To: 'Oleksandr Tyshchenko', xen-devel
Cc: 'Oleksandr Tyshchenko', 'Jan Beulich',
'Andrew Cooper', 'Roger Pau Monné',
'Wei Liu', 'Julien Grall',
'Stefano Stabellini', 'Julien Grall'
> -----Original Message-----
> From: Oleksandr Tyshchenko <olekstysh@gmail.com>
> Sent: 15 October 2020 17:44
> To: xen-devel@lists.xenproject.org
> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Paul Durrant <paul@xen.org>; Jan Beulich
> <jbeulich@suse.com>; Andrew Cooper <andrew.cooper3@citrix.com>; Roger Pau Monné
> <roger.pau@citrix.com>; Wei Liu <wl@xen.org>; Julien Grall <julien@xen.org>; Stefano Stabellini
> <sstabellini@kernel.org>; Julien Grall <julien.grall@arm.com>
> Subject: [PATCH V2 08/23] xen/ioreq: Introduce ioreq_params to abstract accesses to arch.hvm.params
>
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> We don't want to move HVM params field out of *arch.hvm* in this particular
> case as although it stores a few IOREQ params, it is not a (completely)
> IOREQ stuff and is specific to the architecture. Instead, abstract
> accesses by the proposed macro.
>
> This is a follow up action to reduce layering violation in the common code.
>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> CC: Julien Grall <julien.grall@arm.com>
>
Keeping the 'legacy' magic page code under an x86 ioreq.c would avoid the need for this patch.
Paul
> ---
> Please note, this is a split/cleanup/hardening of Julien's PoC:
> "Add support for Guest IO forwarding to a device emulator"
>
> Changes V1 -> V2:
> - new patch
> ---
> xen/common/ioreq.c | 4 ++--
> xen/include/asm-x86/hvm/domain.h | 2 ++
> 2 files changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
> index 7f91bc2..a07f1d7 100644
> --- a/xen/common/ioreq.c
> +++ b/xen/common/ioreq.c
> @@ -223,7 +223,7 @@ static gfn_t hvm_alloc_legacy_ioreq_gfn(struct ioreq_server *s)
> for ( i = HVM_PARAM_IOREQ_PFN; i <= HVM_PARAM_BUFIOREQ_PFN; i++ )
> {
> if ( !test_and_clear_bit(i, &d->ioreq_gfn.legacy_mask) )
> - return _gfn(d->arch.hvm.params[i]);
> + return _gfn(ioreq_params(d, i));
> }
>
> return INVALID_GFN;
> @@ -255,7 +255,7 @@ static bool hvm_free_legacy_ioreq_gfn(struct ioreq_server *s,
>
> for ( i = HVM_PARAM_IOREQ_PFN; i <= HVM_PARAM_BUFIOREQ_PFN; i++ )
> {
> - if ( gfn_eq(gfn, _gfn(d->arch.hvm.params[i])) )
> + if ( gfn_eq(gfn, _gfn(ioreq_params(d, i))) )
> break;
> }
> if ( i > HVM_PARAM_BUFIOREQ_PFN )
> diff --git a/xen/include/asm-x86/hvm/domain.h b/xen/include/asm-x86/hvm/domain.h
> index 5d60737..c3af339 100644
> --- a/xen/include/asm-x86/hvm/domain.h
> +++ b/xen/include/asm-x86/hvm/domain.h
> @@ -63,6 +63,8 @@ struct hvm_pi_ops {
> void (*vcpu_block)(struct vcpu *);
> };
>
> +#define ioreq_params(d, i) ((d)->arch.hvm.params[i])
> +
> struct hvm_domain {
> /* Cached CF8 for guest PCI config cycles */
> uint32_t pci_cf8;
> --
> 2.7.4
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server()
2020-10-15 16:44 ` [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server() Oleksandr Tyshchenko
@ 2020-10-20 10:51 ` Paul Durrant
2020-11-10 20:53 ` Oleksandr
0 siblings, 1 reply; 109+ messages in thread
From: Paul Durrant @ 2020-10-20 10:51 UTC (permalink / raw)
To: 'Oleksandr Tyshchenko', xen-devel
Cc: 'Oleksandr Tyshchenko', 'Stefano Stabellini',
'Julien Grall', 'Volodymyr Babchuk',
'Andrew Cooper', 'George Dunlap',
'Ian Jackson', 'Jan Beulich', 'Wei Liu',
'Julien Grall'
> -----Original Message-----
> From: Oleksandr Tyshchenko <olekstysh@gmail.com>
> Sent: 15 October 2020 17:44
> To: xen-devel@lists.xenproject.org
> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Stefano Stabellini <sstabellini@kernel.org>;
> Julien Grall <julien@xen.org>; Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>; Andrew Cooper
> <andrew.cooper3@citrix.com>; George Dunlap <george.dunlap@citrix.com>; Ian Jackson
> <iwj@xenproject.org>; Jan Beulich <jbeulich@suse.com>; Wei Liu <wl@xen.org>; Paul Durrant
> <paul@xen.org>; Julien Grall <julien.grall@arm.com>
> Subject: [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server()
>
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> This patch introduces a helper the main purpose of which is to check
> if a domain is using IOREQ server(s).
>
> On Arm the current benefit is to avoid calling handle_io_completion()
> (which implies iterating over all possible IOREQ servers anyway)
> on every return in leave_hypervisor_to_guest() if there is no active
> servers for the particular domain.
> Also this helper will be used by one of the subsequent patches on Arm.
>
> This involves adding an extra per-domain variable to store the count
> of servers in use.
>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> CC: Julien Grall <julien.grall@arm.com>
>
> ---
> Please note, this is a split/cleanup/hardening of Julien's PoC:
> "Add support for Guest IO forwarding to a device emulator"
>
> Changes RFC -> V1:
> - new patch
>
> Changes V1 -> V2:
> - update patch description
> - guard helper with CONFIG_IOREQ_SERVER
> - remove "hvm" prefix
> - modify helper to just return d->arch.hvm.ioreq_server.nr_servers
> - put suitable ASSERT()s
> - use ASSERT(d->ioreq_server.server[id] ? !s : !!s) in set_ioreq_server()
> - remove d->ioreq_server.nr_servers = 0 from hvm_ioreq_init()
> ---
> xen/arch/arm/traps.c | 15 +++++++++------
> xen/common/ioreq.c | 7 ++++++-
> xen/include/xen/ioreq.h | 14 ++++++++++++++
> xen/include/xen/sched.h | 1 +
> 4 files changed, 30 insertions(+), 7 deletions(-)
>
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index 507c095..a8f5fdf 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -2261,14 +2261,17 @@ static bool check_for_vcpu_work(void)
> struct vcpu *v = current;
>
> #ifdef CONFIG_IOREQ_SERVER
> - bool handled;
> + if ( domain_has_ioreq_server(v->domain) )
> + {
> + bool handled;
>
> - local_irq_enable();
> - handled = handle_io_completion(v);
> - local_irq_disable();
> + local_irq_enable();
> + handled = handle_io_completion(v);
> + local_irq_disable();
>
> - if ( !handled )
> - return true;
> + if ( !handled )
> + return true;
> + }
> #endif
>
> if ( likely(!v->arch.need_flush_to_ram) )
> diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
> index bcd4961..a72bc0e 100644
> --- a/xen/common/ioreq.c
> +++ b/xen/common/ioreq.c
> @@ -39,9 +39,14 @@ static void set_ioreq_server(struct domain *d, unsigned int id,
> struct ioreq_server *s)
> {
> ASSERT(id < MAX_NR_IOREQ_SERVERS);
> - ASSERT(!s || !d->ioreq_server.server[id]);
> + ASSERT(d->ioreq_server.server[id] ? !s : !!s);
That looks odd. How about ASSERT(!s ^ !d->ioreq_server.server[id])?
Paul
>
> d->ioreq_server.server[id] = s;
> +
> + if ( s )
> + d->ioreq_server.nr_servers++;
> + else
> + d->ioreq_server.nr_servers--;
> }
>
> #define GET_IOREQ_SERVER(d, id) \
> diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
> index 7b03ab5..0679fef 100644
> --- a/xen/include/xen/ioreq.h
> +++ b/xen/include/xen/ioreq.h
> @@ -55,6 +55,20 @@ struct ioreq_server {
> uint8_t bufioreq_handling;
> };
>
> +#ifdef CONFIG_IOREQ_SERVER
> +static inline bool domain_has_ioreq_server(const struct domain *d)
> +{
> + ASSERT((current->domain == d) || atomic_read(&d->pause_count));
> +
This seems like an odd place to put such an assertion.
> + return d->ioreq_server.nr_servers;
> +}
> +#else
> +static inline bool domain_has_ioreq_server(const struct domain *d)
> +{
> + return false;
> +}
> +#endif
> +
Can this be any more compact? E.g.
return IS_ENABLED(CONFIG_IOREQ_SERVER) && d->ioreq_server.nr_servers;
?
> struct ioreq_server *get_ioreq_server(const struct domain *d,
> unsigned int id);
>
> diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
> index f9ce14c..290cddb 100644
> --- a/xen/include/xen/sched.h
> +++ b/xen/include/xen/sched.h
> @@ -553,6 +553,7 @@ struct domain
> struct {
> spinlock_t lock;
> struct ioreq_server *server[MAX_NR_IOREQ_SERVERS];
> + unsigned int nr_servers;
> } ioreq_server;
> #endif
> };
> --
> 2.7.4
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [PATCH V2 11/23] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
2020-10-15 16:44 ` [PATCH V2 11/23] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu Oleksandr Tyshchenko
@ 2020-10-20 10:55 ` Paul Durrant
2020-11-10 20:59 ` Oleksandr
2020-11-11 8:04 ` Jan Beulich
1 sibling, 1 reply; 109+ messages in thread
From: Paul Durrant @ 2020-10-20 10:55 UTC (permalink / raw)
To: 'Oleksandr Tyshchenko', xen-devel
Cc: 'Oleksandr Tyshchenko', 'Jan Beulich',
'Andrew Cooper', 'Roger Pau Monné',
'Wei Liu', 'George Dunlap', 'Ian Jackson',
'Julien Grall', 'Stefano Stabellini',
'Jun Nakajima', 'Kevin Tian',
'Julien Grall'
> -----Original Message-----
> From: Oleksandr Tyshchenko <olekstysh@gmail.com>
> Sent: 15 October 2020 17:44
> To: xen-devel@lists.xenproject.org
> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Paul Durrant <paul@xen.org>; Jan Beulich
> <jbeulich@suse.com>; Andrew Cooper <andrew.cooper3@citrix.com>; Roger Pau Monné
> <roger.pau@citrix.com>; Wei Liu <wl@xen.org>; George Dunlap <george.dunlap@citrix.com>; Ian Jackson
> <iwj@xenproject.org>; Julien Grall <julien@xen.org>; Stefano Stabellini <sstabellini@kernel.org>; Jun
> Nakajima <jun.nakajima@intel.com>; Kevin Tian <kevin.tian@intel.com>; Julien Grall
> <julien.grall@arm.com>
> Subject: [PATCH V2 11/23] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
>
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> The IOREQ is a common feature now and these fields will be used
> on Arm as is. Move them to common struct vcpu as a part of new
> struct vcpu_io. Also move enum hvm_io_completion to xen/sched.h
> and remove "hvm" prefixes.
>
> This patch completely removes layering violation in the common code.
>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> CC: Julien Grall <julien.grall@arm.com>
>
> ---
> Please note, this is a split/cleanup/hardening of Julien's PoC:
> "Add support for Guest IO forwarding to a device emulator"
>
> ***
> I was thinking that it may be better to place these two fields
> into struct vcpu directly (without intermediate "io" struct).
> I think, this way the code which operates with these fields
> would become cleaner. Another possible option would be either
> to rename "io" struct (I failed to think of a better name) or
> to drop(replace?) duplicating "io" prefixes from these fields.
Just drop the 'io_' prefix from the field names.
Paul
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
` (22 preceding siblings ...)
2020-10-15 16:44 ` [PATCH V2 23/23] [RFC] libxl: Add support for virtio-disk configuration Oleksandr Tyshchenko
@ 2020-10-29 7:41 ` Masami Hiramatsu
2020-10-29 18:48 ` Oleksandr Tyshchenko
23 siblings, 1 reply; 109+ messages in thread
From: Masami Hiramatsu @ 2020-10-29 7:41 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: xen-devel, Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Andrew Cooper, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Tim Deegan, Daniel De Graaf,
Volodymyr Babchuk, Jun Nakajima, Kevin Tian, Anthony PERARD,
Bertrand Marquis
Hi Oleksandr,
I would like to try this on my arm64 board.
According to your comments in the patch, I made this config file.
# cat debian.conf
name = "debian"
type = "pvh"
vcpus = 8
memory = 512
kernel = "/opt/agl/vmlinuz-5.9.0-1-arm64"
ramdisk = "/opt/agl/initrd.img-5.9.0-1-arm64"
cmdline= "console=hvc0 earlyprintk=xen root=/dev/xvda1 rw"
disk = [ '/opt/agl/debian.qcow2,qcow2,hda' ]
vif = [ 'mac=00:16:3E:74:3d:76,bridge=xenbr0' ]
virtio = 1
vdisk = [ 'backend=Dom0, disks=ro:/dev/sda1' ]
And tried to boot a DomU, but I got below error.
# xl create -c debian.conf
Parsing config from debian.conf
libxl: error: libxl_create.c:1863:domcreate_attach_devices: Domain
1:unable to add virtio_disk devices
libxl: error: libxl_domain.c:1218:destroy_domid_pci_done: Domain
1:xc_domain_pause failed
libxl: error: libxl_dom.c:39:libxl__domain_type: unable to get domain
type for domid=1
libxl: error: libxl_domain.c:1136:domain_destroy_callback: Domain
1:Unable to destroy guest
libxl: error: libxl_domain.c:1063:domain_destroy_cb: Domain
1:Destruction of domain failed
Could you tell me how can I test it?
Thank you,
2020年10月16日(金) 1:46 Oleksandr Tyshchenko <olekstysh@gmail.com>:
>
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> Hello all.
>
> The purpose of this patch series is to add IOREQ/DM support to Xen on Arm.
> You can find an initial discussion at [1] and RFC/V1 series at [2]/[3].
> Xen on Arm requires some implementation to forward guest MMIO access to a device
> model in order to implement virtio-mmio backend or even mediator outside of hypervisor.
> As Xen on x86 already contains required support this series tries to make it common
> and introduce Arm specific bits plus some new functionality. Patch series is based on
> Julien's PoC "xen/arm: Add support for Guest IO forwarding to a device emulator".
> Besides splitting existing IOREQ/DM support and introducing Arm side, the series
> also includes virtio-mmio related changes (last 2 patches for toolstack)
> for the reviewers to be able to see how the whole picture could look like.
>
> According to the initial discussion there are a few open questions/concerns
> regarding security, performance in VirtIO solution:
> 1. virtio-mmio vs virtio-pci, SPI vs MSI, different use-cases require different
> transport...
> 2. virtio backend is able to access all guest memory, some kind of protection
> is needed: 'virtio-iommu in Xen' vs 'pre-shared-memory & memcpys in guest'
> 3. interface between toolstack and 'out-of-qemu' virtio backend, avoid using
> Xenstore in virtio backend if possible.
> 4. a lot of 'foreing mapping' could lead to the memory exhaustion, Julien
> has some idea regarding that.
>
> Looks like all of them are valid and worth considering, but the first thing
> which we need on Arm is a mechanism to forward guest IO to a device emulator,
> so let's focus on it in the first place.
>
> ***
>
> There are a lot of changes since RFC series, almost all TODOs were resolved on Arm,
> Arm code was improved and hardened, common IOREQ/DM code became really arch-agnostic
> (without HVM-ism), but one TODO still remains which is "PIO handling" on Arm.
> The "PIO handling" TODO is expected to left unaddressed for the current series.
> It is not an big issue for now while Xen doesn't have support for vPCI on Arm.
> On Arm64 they are only used for PCI IO Bar and we would probably want to expose
> them to emulator as PIO access to make a DM completely arch-agnostic. So "PIO handling"
> should be implemented when we add support for vPCI.
>
> I left interface untouched in the following patch
> "xen/dm: Introduce xendevicemodel_set_irq_level DM op"
> since there is still an open discussion what interface to use/what
> information to pass to the hypervisor.
>
> Also I decided to drop the following patch:
> "[RFC PATCH V1 07/12] A collection of tweaks to be able to run emulator in driver domain"
> as I got an advise to write our own policy using FLASK which would cover our use
> case (with emulator in driver domain) rather than tweak Xen.
>
> There are two patches on review this series depends on (each involved patch in this series
> contains this note as well):
> 1. https://patchwork.kernel.org/patch/11816689
> 2. https://patchwork.kernel.org/patch/11803383
>
> Please note, that IOREQ feature is disabled by default within this series.
>
> ***
>
> Patch series [4] was rebased on recent "staging branch"
> (8a62dee x86/vLAPIC: don't leak regs page from vlapic_init() upon error) and tested on
> Renesas Salvator-X board + H3 ES3.0 SoC (Arm64) with virtio-mmio disk backend (we will
> share it later) running in driver domain and unmodified Linux Guest running on existing
> virtio-blk driver (frontend). No issues were observed. Guest domain 'reboot/destroy'
> use-cases work properly. Patch series was only build-tested on x86.
>
> Please note, build-test passed for the following modes:
> 1. x86: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y (default)
> 2. x86: #CONFIG_HVM is not set / #CONFIG_IOREQ_SERVER is not set
> 3. Arm64: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
> 4. Arm64: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
> 5. Arm32: CONFIG_HVM=y / CONFIG_IOREQ_SERVER=y
> 6. Arm32: CONFIG_HVM=y / #CONFIG_IOREQ_SERVER is not set (default)
>
> ***
>
> Any feedback/help would be highly appreciated.
>
> [1] https://lists.xenproject.org/archives/html/xen-devel/2020-07/msg00825.html
> [2] https://lists.xenproject.org/archives/html/xen-devel/2020-08/msg00071.html
> [3] https://lists.xenproject.org/archives/html/xen-devel/2020-09/msg00732.html
> [4] https://github.com/otyshchenko1/xen/commits/ioreq_4.14_ml3
>
> Julien Grall (5):
> xen/dm: Make x86's DM feature common
> xen/mm: Make x86's XENMEM_resource_ioreq_server handling common
> arm/ioreq: Introduce arch specific bits for IOREQ/DM features
> xen/dm: Introduce xendevicemodel_set_irq_level DM op
> libxl: Introduce basic virtio-mmio support on Arm
>
> Oleksandr Tyshchenko (18):
> x86/ioreq: Prepare IOREQ feature for making it common
> xen/ioreq: Make x86's IOREQ feature common
> xen/ioreq: Make x86's hvm_ioreq_needs_completion() common
> xen/ioreq: Provide alias for the handle_mmio()
> xen/ioreq: Make x86's hvm_mmio_first(last)_byte() common
> xen/ioreq: Make x86's hvm_ioreq_(page/vcpu/server) structs common
> xen/ioreq: Move x86's ioreq_gfn(server) to struct domain
> xen/ioreq: Introduce ioreq_params to abstract accesses to
> arch.hvm.params
> xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
> xen/ioreq: Remove "hvm" prefixes from involved function names
> xen/ioreq: Use guest_cmpxchg64() instead of cmpxchg()
> xen/arm: Stick around in leave_hypervisor_to_guest until I/O has
> completed
> xen/mm: Handle properly reference in set_foreign_p2m_entry() on Arm
> xen/ioreq: Introduce domain_has_ioreq_server()
> xen/arm: io: Abstract sign-extension
> xen/ioreq: Make x86's send_invalidate_req() common
> xen/arm: Add mapcache invalidation handling
> [RFC] libxl: Add support for virtio-disk configuration
>
> MAINTAINERS | 8 +-
> tools/libs/devicemodel/core.c | 18 +
> tools/libs/devicemodel/include/xendevicemodel.h | 4 +
> tools/libs/devicemodel/libxendevicemodel.map | 1 +
> tools/libs/light/Makefile | 1 +
> tools/libs/light/libxl_arm.c | 94 +-
> tools/libs/light/libxl_create.c | 1 +
> tools/libs/light/libxl_internal.h | 1 +
> tools/libs/light/libxl_types.idl | 16 +
> tools/libs/light/libxl_types_internal.idl | 1 +
> tools/libs/light/libxl_virtio_disk.c | 109 ++
> tools/xl/Makefile | 2 +-
> tools/xl/xl.h | 3 +
> tools/xl/xl_cmdtable.c | 15 +
> tools/xl/xl_parse.c | 116 ++
> tools/xl/xl_virtio_disk.c | 46 +
> xen/arch/arm/Makefile | 2 +
> xen/arch/arm/dm.c | 89 ++
> xen/arch/arm/domain.c | 9 +
> xen/arch/arm/hvm.c | 4 +
> xen/arch/arm/io.c | 29 +-
> xen/arch/arm/ioreq.c | 126 ++
> xen/arch/arm/p2m.c | 29 +
> xen/arch/arm/traps.c | 58 +-
> xen/arch/x86/Kconfig | 1 +
> xen/arch/x86/hvm/Makefile | 1 -
> xen/arch/x86/hvm/dm.c | 291 +----
> xen/arch/x86/hvm/emulate.c | 60 +-
> xen/arch/x86/hvm/hvm.c | 24 +-
> xen/arch/x86/hvm/hypercall.c | 9 +-
> xen/arch/x86/hvm/intercept.c | 5 +-
> xen/arch/x86/hvm/io.c | 26 +-
> xen/arch/x86/hvm/ioreq.c | 1533 -----------------------
> xen/arch/x86/hvm/stdvga.c | 10 +-
> xen/arch/x86/hvm/svm/nestedsvm.c | 2 +-
> xen/arch/x86/hvm/vmx/realmode.c | 6 +-
> xen/arch/x86/hvm/vmx/vvmx.c | 2 +-
> xen/arch/x86/mm.c | 46 +-
> xen/arch/x86/mm/p2m.c | 13 +-
> xen/arch/x86/mm/shadow/common.c | 2 +-
> xen/common/Kconfig | 3 +
> xen/common/Makefile | 2 +
> xen/common/dm.c | 292 +++++
> xen/common/ioreq.c | 1443 +++++++++++++++++++++
> xen/common/memory.c | 50 +-
> xen/include/asm-arm/domain.h | 5 +
> xen/include/asm-arm/hvm/ioreq.h | 109 ++
> xen/include/asm-arm/mm.h | 8 -
> xen/include/asm-arm/mmio.h | 1 +
> xen/include/asm-arm/p2m.h | 19 +-
> xen/include/asm-arm/paging.h | 4 +
> xen/include/asm-arm/traps.h | 24 +
> xen/include/asm-x86/hvm/domain.h | 50 +-
> xen/include/asm-x86/hvm/emulate.h | 2 +-
> xen/include/asm-x86/hvm/io.h | 17 -
> xen/include/asm-x86/hvm/ioreq.h | 198 ++-
> xen/include/asm-x86/hvm/vcpu.h | 18 -
> xen/include/asm-x86/mm.h | 4 -
> xen/include/asm-x86/p2m.h | 20 +-
> xen/include/public/arch-arm.h | 5 +
> xen/include/public/hvm/dm_op.h | 16 +
> xen/include/xen/dm.h | 44 +
> xen/include/xen/ioreq.h | 143 +++
> xen/include/xen/p2m-common.h | 4 +
> xen/include/xen/sched.h | 37 +
> xen/include/xsm/dummy.h | 4 +-
> xen/include/xsm/xsm.h | 6 +-
> xen/xsm/dummy.c | 2 +-
> xen/xsm/flask/hooks.c | 5 +-
> 69 files changed, 3223 insertions(+), 2125 deletions(-)
> create mode 100644 tools/libs/light/libxl_virtio_disk.c
> create mode 100644 tools/xl/xl_virtio_disk.c
> create mode 100644 xen/arch/arm/dm.c
> create mode 100644 xen/arch/arm/ioreq.c
> delete mode 100644 xen/arch/x86/hvm/ioreq.c
> create mode 100644 xen/common/dm.c
> create mode 100644 xen/common/ioreq.c
> create mode 100644 xen/include/asm-arm/hvm/ioreq.h
> create mode 100644 xen/include/xen/dm.h
> create mode 100644 xen/include/xen/ioreq.h
>
> --
> 2.7.4
>
>
--
Masami Hiramatsu
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm
2020-10-29 7:41 ` [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Masami Hiramatsu
@ 2020-10-29 18:48 ` Oleksandr Tyshchenko
2020-10-29 19:53 ` Stefano Stabellini
2020-10-29 20:03 ` Alex Bennée
0 siblings, 2 replies; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-29 18:48 UTC (permalink / raw)
To: Masami Hiramatsu
Cc: xen-devel, Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Andrew Cooper, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Tim Deegan, Daniel De Graaf,
Volodymyr Babchuk, Jun Nakajima, Kevin Tian, Anthony PERARD,
Bertrand Marquis
[-- Attachment #1: Type: text/plain, Size: 1611 bytes --]
On Thu, Oct 29, 2020 at 9:42 AM Masami Hiramatsu <
masami.hiramatsu@linaro.org> wrote:
> Hi Oleksandr,
>
Hi Masami
[sorry for the possible format issue]
>
> I would like to try this on my arm64 board.
>
Glad to hear you are interested in this topic.
>
> According to your comments in the patch, I made this config file.
> # cat debian.conf
> name = "debian"
> type = "pvh"
> vcpus = 8
> memory = 512
> kernel = "/opt/agl/vmlinuz-5.9.0-1-arm64"
> ramdisk = "/opt/agl/initrd.img-5.9.0-1-arm64"
> cmdline= "console=hvc0 earlyprintk=xen root=/dev/xvda1 rw"
> disk = [ '/opt/agl/debian.qcow2,qcow2,hda' ]
> vif = [ 'mac=00:16:3E:74:3d:76,bridge=xenbr0' ]
> virtio = 1
> vdisk = [ 'backend=Dom0, disks=ro:/dev/sda1' ]
>
> And tried to boot a DomU, but I got below error.
>
> # xl create -c debian.conf
> Parsing config from debian.conf
> libxl: error: libxl_create.c:1863:domcreate_attach_devices: Domain
> 1:unable to add virtio_disk devices
> libxl: error: libxl_domain.c:1218:destroy_domid_pci_done: Domain
> 1:xc_domain_pause failed
> libxl: error: libxl_dom.c:39:libxl__domain_type: unable to get domain
> type for domid=1
> libxl: error: libxl_domain.c:1136:domain_destroy_callback: Domain
> 1:Unable to destroy guest
> libxl: error: libxl_domain.c:1063:domain_destroy_cb: Domain
> 1:Destruction of domain failed
>
>
> Could you tell me how can I test it?
>
I assume it is due to the lack of the virtio-disk backend (which I haven't
shared yet as I focused on the IOREQ/DM support on Arm in the first place).
Could you wait a little bit, I am going to share it soon.
--
Regards,
Oleksandr Tyshchenko
[-- Attachment #2: Type: text/html, Size: 2917 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm
2020-10-29 18:48 ` Oleksandr Tyshchenko
@ 2020-10-29 19:53 ` Stefano Stabellini
2020-10-29 21:13 ` Oleksandr Tyshchenko
2020-10-29 20:03 ` Alex Bennée
1 sibling, 1 reply; 109+ messages in thread
From: Stefano Stabellini @ 2020-10-29 19:53 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: Masami Hiramatsu, xen-devel, Oleksandr Tyshchenko, Paul Durrant,
Jan Beulich, Andrew Cooper, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Tim Deegan, Daniel De Graaf,
Volodymyr Babchuk, Jun Nakajima, Kevin Tian, Anthony PERARD,
Bertrand Marquis
[-- Attachment #1: Type: text/plain, Size: 2017 bytes --]
On Thu, 29 Oct 2020, Oleksandr Tyshchenko wrote:
> On Thu, Oct 29, 2020 at 9:42 AM Masami Hiramatsu <masami.hiramatsu@linaro.org> wrote:
> Hi Oleksandr,
>
> Hi Masami
>
> [sorry for the possible format issue]
>
>
> I would like to try this on my arm64 board.
>
> Glad to hear you are interested in this topic.
>
>
> According to your comments in the patch, I made this config file.
> # cat debian.conf
> name = "debian"
> type = "pvh"
> vcpus = 8
> memory = 512
> kernel = "/opt/agl/vmlinuz-5.9.0-1-arm64"
> ramdisk = "/opt/agl/initrd.img-5.9.0-1-arm64"
> cmdline= "console=hvc0 earlyprintk=xen root=/dev/xvda1 rw"
> disk = [ '/opt/agl/debian.qcow2,qcow2,hda' ]
> vif = [ 'mac=00:16:3E:74:3d:76,bridge=xenbr0' ]
> virtio = 1
> vdisk = [ 'backend=Dom0, disks=ro:/dev/sda1' ]
>
> And tried to boot a DomU, but I got below error.
>
> # xl create -c debian.conf
> Parsing config from debian.conf
> libxl: error: libxl_create.c:1863:domcreate_attach_devices: Domain
> 1:unable to add virtio_disk devices
> libxl: error: libxl_domain.c:1218:destroy_domid_pci_done: Domain
> 1:xc_domain_pause failed
> libxl: error: libxl_dom.c:39:libxl__domain_type: unable to get domain
> type for domid=1
> libxl: error: libxl_domain.c:1136:domain_destroy_callback: Domain
> 1:Unable to destroy guest
> libxl: error: libxl_domain.c:1063:domain_destroy_cb: Domain
> 1:Destruction of domain failed
>
>
> Could you tell me how can I test it?
>
>
> I assume it is due to the lack of the virtio-disk backend (which I haven't shared yet as I focused on the IOREQ/DM support on Arm in the
> first place).
> Could you wait a little bit, I am going to share it soon.
Do you have a quick-and-dirty hack you can share in the meantime? Even
just on github as a special branch? It would be very useful to be able
to have a test-driver for the new feature.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm
2020-10-29 18:48 ` Oleksandr Tyshchenko
2020-10-29 19:53 ` Stefano Stabellini
@ 2020-10-29 20:03 ` Alex Bennée
2020-10-29 20:10 ` Stefano Stabellini
2020-10-29 22:06 ` Oleksandr Tyshchenko
1 sibling, 2 replies; 109+ messages in thread
From: Alex Bennée @ 2020-10-29 20:03 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: Masami Hiramatsu, Oleksandr Tyshchenko, Paul Durrant,
Jan Beulich, Andrew Cooper, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Tim Deegan, Daniel De Graaf,
Volodymyr Babchuk, Jun Nakajima, Kevin Tian, Anthony PERARD,
Bertrand Marquis, xen-devel
Oleksandr Tyshchenko <olekstysh@gmail.com> writes:
> On Thu, Oct 29, 2020 at 9:42 AM Masami Hiramatsu <
> masami.hiramatsu@linaro.org> wrote:
>
>> Hi Oleksandr,
>>
> Hi Masami
>
> [sorry for the possible format issue]
>
>
>>
>> I would like to try this on my arm64 board.
>>
> Glad to hear you are interested in this topic.
>
>
>>
>> According to your comments in the patch, I made this config file.
>> # cat debian.conf
>> name = "debian"
>> type = "pvh"
>> vcpus = 8
>> memory = 512
>> kernel = "/opt/agl/vmlinuz-5.9.0-1-arm64"
>> ramdisk = "/opt/agl/initrd.img-5.9.0-1-arm64"
>> cmdline= "console=hvc0 earlyprintk=xen root=/dev/xvda1 rw"
>> disk = [ '/opt/agl/debian.qcow2,qcow2,hda' ]
>> vif = [ 'mac=00:16:3E:74:3d:76,bridge=xenbr0' ]
>> virtio = 1
>> vdisk = [ 'backend=Dom0, disks=ro:/dev/sda1' ]
>>
>> And tried to boot a DomU, but I got below error.
>>
>> # xl create -c debian.conf
>> Parsing config from debian.conf
>> libxl: error: libxl_create.c:1863:domcreate_attach_devices: Domain
>> 1:unable to add virtio_disk devices
>> libxl: error: libxl_domain.c:1218:destroy_domid_pci_done: Domain
>> 1:xc_domain_pause failed
>> libxl: error: libxl_dom.c:39:libxl__domain_type: unable to get domain
>> type for domid=1
>> libxl: error: libxl_domain.c:1136:domain_destroy_callback: Domain
>> 1:Unable to destroy guest
>> libxl: error: libxl_domain.c:1063:domain_destroy_cb: Domain
>> 1:Destruction of domain failed
>>
>>
>> Could you tell me how can I test it?
>>
>
> I assume it is due to the lack of the virtio-disk backend (which I haven't
> shared yet as I focused on the IOREQ/DM support on Arm in the first place).
> Could you wait a little bit, I am going to share it soon.
I assume this is wiring up the required bits of support to handle the
IOREQ requests in QEMU? We are putting together a PoC demo to show
a virtio enabled image (AGL) running on both KVM and Xen hypervisors so
we are keen to see your code as soon as you can share it.
I'm currently preparing a patch series for QEMU which fixes the recent
breakage caused by changes to the build system. As part of that I've
separated CONFIG_XEN and CONFIG_XEN_HVM so it's possible to build
i386-softmmu with unneeded for ARM bits. Hopefully this will allow me to
create a qemu-aarch64-system binary with just the PV related models in
it.
Talking to Stefano it probably makes sense going forward to introduce a
new IOREQ aware machine type for QEMU that doesn't bring in the rest of
the x86 overhead. I was thinking maybe xenvirt?
You've tested with virtio-block but we'd also like to extend this to
other arbitrary virtio devices. I guess we will need some sort of
mechanism to inform the QEMU command line where each device sits in the
virtio-mmio bus so the FDT Xen delivers to the guest matches up with
what QEMU is ready to serve requests for?
--
Alex Bennée
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm
2020-10-29 20:03 ` Alex Bennée
@ 2020-10-29 20:10 ` Stefano Stabellini
2020-10-29 22:06 ` Oleksandr Tyshchenko
1 sibling, 0 replies; 109+ messages in thread
From: Stefano Stabellini @ 2020-10-29 20:10 UTC (permalink / raw)
To: Alex Bennée
Cc: Oleksandr Tyshchenko, Masami Hiramatsu, Oleksandr Tyshchenko,
Paul Durrant, Jan Beulich, Andrew Cooper, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Tim Deegan, Daniel De Graaf,
Volodymyr Babchuk, Jun Nakajima, Kevin Tian, Anthony PERARD,
Bertrand Marquis, xen-devel
[-- Attachment #1: Type: text/plain, Size: 3262 bytes --]
On Thu, 29 Oct 2020, Alex Bennée wrote:
> Oleksandr Tyshchenko <olekstysh@gmail.com> writes:
> > On Thu, Oct 29, 2020 at 9:42 AM Masami Hiramatsu <
> > masami.hiramatsu@linaro.org> wrote:
> >
> >> Hi Oleksandr,
> >>
> > Hi Masami
> >
> > [sorry for the possible format issue]
> >
> >
> >>
> >> I would like to try this on my arm64 board.
> >>
> > Glad to hear you are interested in this topic.
> >
> >
> >>
> >> According to your comments in the patch, I made this config file.
> >> # cat debian.conf
> >> name = "debian"
> >> type = "pvh"
> >> vcpus = 8
> >> memory = 512
> >> kernel = "/opt/agl/vmlinuz-5.9.0-1-arm64"
> >> ramdisk = "/opt/agl/initrd.img-5.9.0-1-arm64"
> >> cmdline= "console=hvc0 earlyprintk=xen root=/dev/xvda1 rw"
> >> disk = [ '/opt/agl/debian.qcow2,qcow2,hda' ]
> >> vif = [ 'mac=00:16:3E:74:3d:76,bridge=xenbr0' ]
> >> virtio = 1
> >> vdisk = [ 'backend=Dom0, disks=ro:/dev/sda1' ]
> >>
> >> And tried to boot a DomU, but I got below error.
> >>
> >> # xl create -c debian.conf
> >> Parsing config from debian.conf
> >> libxl: error: libxl_create.c:1863:domcreate_attach_devices: Domain
> >> 1:unable to add virtio_disk devices
> >> libxl: error: libxl_domain.c:1218:destroy_domid_pci_done: Domain
> >> 1:xc_domain_pause failed
> >> libxl: error: libxl_dom.c:39:libxl__domain_type: unable to get domain
> >> type for domid=1
> >> libxl: error: libxl_domain.c:1136:domain_destroy_callback: Domain
> >> 1:Unable to destroy guest
> >> libxl: error: libxl_domain.c:1063:domain_destroy_cb: Domain
> >> 1:Destruction of domain failed
> >>
> >>
> >> Could you tell me how can I test it?
> >>
> >
> > I assume it is due to the lack of the virtio-disk backend (which I haven't
> > shared yet as I focused on the IOREQ/DM support on Arm in the first place).
> > Could you wait a little bit, I am going to share it soon.
>
> I assume this is wiring up the required bits of support to handle the
> IOREQ requests in QEMU? We are putting together a PoC demo to show
> a virtio enabled image (AGL) running on both KVM and Xen hypervisors so
> we are keen to see your code as soon as you can share it.
>
> I'm currently preparing a patch series for QEMU which fixes the recent
> breakage caused by changes to the build system. As part of that I've
> separated CONFIG_XEN and CONFIG_XEN_HVM so it's possible to build
> i386-softmmu with unneeded for ARM bits. Hopefully this will allow me to
> create a qemu-aarch64-system binary with just the PV related models in
> it.
>
> Talking to Stefano it probably makes sense going forward to introduce a
> new IOREQ aware machine type for QEMU that doesn't bring in the rest of
> the x86 overhead. I was thinking maybe xenvirt?
>
> You've tested with virtio-block but we'd also like to extend this to
> other arbitrary virtio devices. I guess we will need some sort of
> mechanism to inform the QEMU command line where each device sits in the
> virtio-mmio bus so the FDT Xen delivers to the guest matches up with
> what QEMU is ready to serve requests for?
That would be great. As a reference, given that the domU memory mapping
layout is fixed, on x86 we just made sure that QEMU's idea of where the
devices are is the same of Xen's. What you are suggesting would make it
much more flexible and clearer.
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm
2020-10-29 19:53 ` Stefano Stabellini
@ 2020-10-29 21:13 ` Oleksandr Tyshchenko
2020-10-29 21:34 ` Stefano Stabellini
2020-10-30 11:34 ` Masami Hiramatsu
0 siblings, 2 replies; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-29 21:13 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Masami Hiramatsu, xen-devel, Oleksandr Tyshchenko, Paul Durrant,
Jan Beulich, Andrew Cooper, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson, Julien Grall,
Tim Deegan, Daniel De Graaf, Volodymyr Babchuk, Jun Nakajima,
Kevin Tian, Anthony PERARD, Bertrand Marquis
[-- Attachment #1: Type: text/plain, Size: 2431 bytes --]
Hi Stefano
[sorry for the possible format issue]
On Thu, Oct 29, 2020 at 9:53 PM Stefano Stabellini <sstabellini@kernel.org>
wrote:
> On Thu, 29 Oct 2020, Oleksandr Tyshchenko wrote:
> > On Thu, Oct 29, 2020 at 9:42 AM Masami Hiramatsu <
> masami.hiramatsu@linaro.org> wrote:
> > Hi Oleksandr,
> >
> > Hi Masami
> >
> > [sorry for the possible format issue]
> >
> >
> > I would like to try this on my arm64 board.
> >
> > Glad to hear you are interested in this topic.
> >
> >
> > According to your comments in the patch, I made this config file.
> > # cat debian.conf
> > name = "debian"
> > type = "pvh"
> > vcpus = 8
> > memory = 512
> > kernel = "/opt/agl/vmlinuz-5.9.0-1-arm64"
> > ramdisk = "/opt/agl/initrd.img-5.9.0-1-arm64"
> > cmdline= "console=hvc0 earlyprintk=xen root=/dev/xvda1 rw"
> > disk = [ '/opt/agl/debian.qcow2,qcow2,hda' ]
> > vif = [ 'mac=00:16:3E:74:3d:76,bridge=xenbr0' ]
> > virtio = 1
> > vdisk = [ 'backend=Dom0, disks=ro:/dev/sda1' ]
> >
> > And tried to boot a DomU, but I got below error.
> >
> > # xl create -c debian.conf
> > Parsing config from debian.conf
> > libxl: error: libxl_create.c:1863:domcreate_attach_devices: Domain
> > 1:unable to add virtio_disk devices
> > libxl: error: libxl_domain.c:1218:destroy_domid_pci_done: Domain
> > 1:xc_domain_pause failed
> > libxl: error: libxl_dom.c:39:libxl__domain_type: unable to get
> domain
> > type for domid=1
> > libxl: error: libxl_domain.c:1136:domain_destroy_callback: Domain
> > 1:Unable to destroy guest
> > libxl: error: libxl_domain.c:1063:domain_destroy_cb: Domain
> > 1:Destruction of domain failed
> >
> >
> > Could you tell me how can I test it?
> >
> >
> > I assume it is due to the lack of the virtio-disk backend (which I
> haven't shared yet as I focused on the IOREQ/DM support on Arm in the
> > first place).
> > Could you wait a little bit, I am going to share it soon.
>
> Do you have a quick-and-dirty hack you can share in the meantime? Even
> just on github as a special branch? It would be very useful to be able
> to have a test-driver for the new feature.
Well, I will provide a branch on github with our PoC virtio-disk backend by
the end of this week. It will be possible to test this series with it.
--
Regards,
Oleksandr Tyshchenko
[-- Attachment #2: Type: text/html, Size: 3782 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm
2020-10-29 21:13 ` Oleksandr Tyshchenko
@ 2020-10-29 21:34 ` Stefano Stabellini
2020-10-30 11:34 ` Masami Hiramatsu
1 sibling, 0 replies; 109+ messages in thread
From: Stefano Stabellini @ 2020-10-29 21:34 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: Stefano Stabellini, Masami Hiramatsu, xen-devel,
Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson, Julien Grall,
Tim Deegan, Daniel De Graaf, Volodymyr Babchuk, Jun Nakajima,
Kevin Tian, Anthony PERARD, Bertrand Marquis
[-- Attachment #1: Type: text/plain, Size: 2928 bytes --]
On Thu, 29 Oct 2020, Oleksandr Tyshchenko wrote:
> Hi Stefano
>
> [sorry for the possible format issue]
>
> On Thu, Oct 29, 2020 at 9:53 PM Stefano Stabellini <sstabellini@kernel.org> wrote:
> On Thu, 29 Oct 2020, Oleksandr Tyshchenko wrote:
> > On Thu, Oct 29, 2020 at 9:42 AM Masami Hiramatsu <masami.hiramatsu@linaro.org> wrote:
> > Hi Oleksandr,
> >
> > Hi Masami
> >
> > [sorry for the possible format issue]
> >
> >
> > I would like to try this on my arm64 board.
> >
> > Glad to hear you are interested in this topic.
> >
> >
> > According to your comments in the patch, I made this config file.
> > # cat debian.conf
> > name = "debian"
> > type = "pvh"
> > vcpus = 8
> > memory = 512
> > kernel = "/opt/agl/vmlinuz-5.9.0-1-arm64"
> > ramdisk = "/opt/agl/initrd.img-5.9.0-1-arm64"
> > cmdline= "console=hvc0 earlyprintk=xen root=/dev/xvda1 rw"
> > disk = [ '/opt/agl/debian.qcow2,qcow2,hda' ]
> > vif = [ 'mac=00:16:3E:74:3d:76,bridge=xenbr0' ]
> > virtio = 1
> > vdisk = [ 'backend=Dom0, disks=ro:/dev/sda1' ]
> >
> > And tried to boot a DomU, but I got below error.
> >
> > # xl create -c debian.conf
> > Parsing config from debian.conf
> > libxl: error: libxl_create.c:1863:domcreate_attach_devices: Domain
> > 1:unable to add virtio_disk devices
> > libxl: error: libxl_domain.c:1218:destroy_domid_pci_done: Domain
> > 1:xc_domain_pause failed
> > libxl: error: libxl_dom.c:39:libxl__domain_type: unable to get domain
> > type for domid=1
> > libxl: error: libxl_domain.c:1136:domain_destroy_callback: Domain
> > 1:Unable to destroy guest
> > libxl: error: libxl_domain.c:1063:domain_destroy_cb: Domain
> > 1:Destruction of domain failed
> >
> >
> > Could you tell me how can I test it?
> >
> >
> > I assume it is due to the lack of the virtio-disk backend (which I haven't shared yet as I focused on the IOREQ/DM support on
> Arm in the
> > first place).
> > Could you wait a little bit, I am going to share it soon.
>
> Do you have a quick-and-dirty hack you can share in the meantime? Even
> just on github as a special branch? It would be very useful to be able
> to have a test-driver for the new feature.
>
> Well, I will provide a branch on github with our PoC virtio-disk backend by the end of this week. It will be possible to test this series
> with it.
Very good, thank you!
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm
2020-10-29 20:03 ` Alex Bennée
2020-10-29 20:10 ` Stefano Stabellini
@ 2020-10-29 22:06 ` Oleksandr Tyshchenko
1 sibling, 0 replies; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-29 22:06 UTC (permalink / raw)
To: Alex Bennée
Cc: Masami Hiramatsu, Oleksandr Tyshchenko, Paul Durrant,
Jan Beulich, Andrew Cooper, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Tim Deegan, Daniel De Graaf,
Volodymyr Babchuk, Jun Nakajima, Kevin Tian, Anthony PERARD,
Bertrand Marquis, xen-devel
[-- Attachment #1: Type: text/plain, Size: 2162 bytes --]
Hi Alex.
[sorry for the possible format issues]
> I assume this is wiring up the required bits of support to handle the
> IOREQ requests in QEMU?
No, it is not related to QEMU. We don't run QEMU in our system. Being
honest I have never tried to run it)
virtio-disk backend PoC is a completely standalone entity (IOREQ server)
which emulates a virtio-mmio disk device.
It is based on code from DEMU (for IOREQ server purposes) and some code
from kvmtool to implement virtio protocol,
disk operations over underlying H/W and Xenbus code to be able to read
configuration from the Xenstore
(it is configured via domain config file). Last patch in this series
(marked as RFC) actually adds required bits to the libxl code.
> We are putting together a PoC demo to show
> a virtio enabled image (AGL) running on both KVM and Xen hypervisors so
> we are keen to see your code as soon as you can share it.
Thank you. Sure, I will provide a branch with code by the end of this week.
>
> I'm currently preparing a patch series for QEMU which fixes the recent
> breakage caused by changes to the build system. As part of that I've
> separated CONFIG_XEN and CONFIG_XEN_HVM so it's possible to build
> i386-softmmu with unneeded for ARM bits. Hopefully this will allow me to
> create a qemu-aarch64-system binary with just the PV related models in
> it.
>
Does it mean that it will be possible to use QEMU in Xen on Arm just for
"backend provider" purposes?
>
> Talking to Stefano it probably makes sense going forward to introduce a
> new IOREQ aware machine type for QEMU that doesn't bring in the rest of
> the x86 overhead. I was thinking maybe xenvirt?
>
> You've tested with virtio-block but we'd also like to extend this to
> other arbitrary virtio devices. I guess we will need some sort of
> mechanism to inform the QEMU command line where each device sits in the
> virtio-mmio bus so the FDT Xen delivers to the guest matches up with
> what QEMU is ready to serve requests for?
>
I am sorry, I can't provide ideas here, not familiar with QEMU.
But, completely agree, that other virtio devices should be supported.
--
Regards,
Oleksandr Tyshchenko
[-- Attachment #2: Type: text/html, Size: 3495 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm
2020-10-29 21:13 ` Oleksandr Tyshchenko
2020-10-29 21:34 ` Stefano Stabellini
@ 2020-10-30 11:34 ` Masami Hiramatsu
2020-10-31 21:10 ` Oleksandr Tyshchenko
1 sibling, 1 reply; 109+ messages in thread
From: Masami Hiramatsu @ 2020-10-30 11:34 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: Stefano Stabellini, xen-devel, Oleksandr Tyshchenko,
Paul Durrant, Jan Beulich, Andrew Cooper, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson, Julien Grall,
Tim Deegan, Daniel De Graaf, Volodymyr Babchuk, Jun Nakajima,
Kevin Tian, Anthony PERARD, Bertrand Marquis
Hi Oleksandr,
2020年10月30日(金) 6:14 Oleksandr Tyshchenko <olekstysh@gmail.com>:
>
> Hi Stefano
>
> [sorry for the possible format issue]
>
> On Thu, Oct 29, 2020 at 9:53 PM Stefano Stabellini <sstabellini@kernel.org> wrote:
>>
>> On Thu, 29 Oct 2020, Oleksandr Tyshchenko wrote:
>> > On Thu, Oct 29, 2020 at 9:42 AM Masami Hiramatsu <masami.hiramatsu@linaro.org> wrote:
>> > Hi Oleksandr,
>> >
>> > Hi Masami
>> >
>> > [sorry for the possible format issue]
>> >
>> >
>> > I would like to try this on my arm64 board.
>> >
>> > Glad to hear you are interested in this topic.
>> >
>> >
>> > According to your comments in the patch, I made this config file.
>> > # cat debian.conf
>> > name = "debian"
>> > type = "pvh"
>> > vcpus = 8
>> > memory = 512
>> > kernel = "/opt/agl/vmlinuz-5.9.0-1-arm64"
>> > ramdisk = "/opt/agl/initrd.img-5.9.0-1-arm64"
>> > cmdline= "console=hvc0 earlyprintk=xen root=/dev/xvda1 rw"
>> > disk = [ '/opt/agl/debian.qcow2,qcow2,hda' ]
>> > vif = [ 'mac=00:16:3E:74:3d:76,bridge=xenbr0' ]
>> > virtio = 1
>> > vdisk = [ 'backend=Dom0, disks=ro:/dev/sda1' ]
>> >
>> > And tried to boot a DomU, but I got below error.
>> >
>> > # xl create -c debian.conf
>> > Parsing config from debian.conf
>> > libxl: error: libxl_create.c:1863:domcreate_attach_devices: Domain
>> > 1:unable to add virtio_disk devices
>> > libxl: error: libxl_domain.c:1218:destroy_domid_pci_done: Domain
>> > 1:xc_domain_pause failed
>> > libxl: error: libxl_dom.c:39:libxl__domain_type: unable to get domain
>> > type for domid=1
>> > libxl: error: libxl_domain.c:1136:domain_destroy_callback: Domain
>> > 1:Unable to destroy guest
>> > libxl: error: libxl_domain.c:1063:domain_destroy_cb: Domain
>> > 1:Destruction of domain failed
>> >
>> >
>> > Could you tell me how can I test it?
>> >
>> >
>> > I assume it is due to the lack of the virtio-disk backend (which I haven't shared yet as I focused on the IOREQ/DM support on Arm in the
>> > first place).
>> > Could you wait a little bit, I am going to share it soon.
>>
>> Do you have a quick-and-dirty hack you can share in the meantime? Even
>> just on github as a special branch? It would be very useful to be able
>> to have a test-driver for the new feature.
>
> Well, I will provide a branch on github with our PoC virtio-disk backend by the end of this week. It will be possible to test this series with it.
Great! OK I'll be waiting for the PoC backend.
Thank you!
--
Masami Hiramatsu
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm
2020-10-30 11:34 ` Masami Hiramatsu
@ 2020-10-31 21:10 ` Oleksandr Tyshchenko
2020-11-02 7:23 ` Wei Chen
2020-11-03 14:30 ` Masami Hiramatsu
0 siblings, 2 replies; 109+ messages in thread
From: Oleksandr Tyshchenko @ 2020-10-31 21:10 UTC (permalink / raw)
To: Masami Hiramatsu, Alex Bennée
Cc: Stefano Stabellini, xen-devel, Oleksandr Tyshchenko,
Paul Durrant, Jan Beulich, Andrew Cooper, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson, Julien Grall,
Tim Deegan, Daniel De Graaf, Volodymyr Babchuk, Jun Nakajima,
Kevin Tian, Anthony PERARD, Bertrand Marquis
[-- Attachment #1: Type: text/plain, Size: 3973 bytes --]
On Fri, Oct 30, 2020 at 1:34 PM Masami Hiramatsu <
masami.hiramatsu@linaro.org> wrote:
> Hi Oleksandr,
>
Hi Masami, all
[sorry for the possible format issue]
> >> >
> >> > Could you tell me how can I test it?
> >> >
> >> >
> >> > I assume it is due to the lack of the virtio-disk backend (which I
> haven't shared yet as I focused on the IOREQ/DM support on Arm in the
> >> > first place).
> >> > Could you wait a little bit, I am going to share it soon.
> >>
> >> Do you have a quick-and-dirty hack you can share in the meantime? Even
> >> just on github as a special branch? It would be very useful to be able
> >> to have a test-driver for the new feature.
> >
> > Well, I will provide a branch on github with our PoC virtio-disk backend
> by the end of this week. It will be possible to test this series with it.
>
> Great! OK I'll be waiting for the PoC backend.
>
> Thank you!
>
You can find the virtio-disk backend PoC (shared as is) at [1].
Brief description...
The virtio-disk backend PoC is a completely standalone entity (IOREQ
server) which emulates a virtio-mmio disk device.
It is based on code from DEMU [2] (for IOREQ server purposes) and some code
from kvmtool [3] to implement virtio protocol,
disk operations over underlying H/W and Xenbus code to be able to read
configuration from the Xenstore
(it is configured via domain config file). Last patch in this series
(marked as RFC) actually adds required bits to the libxl code.
Some notes...
Backend could be used with current V2 IOREQ series [4] without any
modifications, all what you need is to enable
CONFIG_IOREQ_SERVER on Arm [5], since it is disabled by default within this
series.
Please note that in our system we run backend in DomD (driver domain). I
haven't tested it in Dom0,
since in our system the Dom0 is thin (without any H/W) and only used to
launch VMs, so there is no underlying block H/W.
But, I hope, it is possible to run it in Dom0 as well (at least there is
nothing specific to a particular domain in the backend itself, nothing
hardcoded).
If you are going to run a backend in other than Dom0 domain you need to
write your own policy (FLASK) for the backend (running in that domain)
to be able to issue DM related requests, etc. Only for test purposes you
could use this patch [6] that tweeks Xen dummy policy (not for upstream).
As I mentioned elsewhere you don't need to modify Guest Linux (DomU), just
enable VirtIO related configs.
If I remember correctly, the following would be enough:
CONFIG_BLK_MQ_VIRTIO=y
CONFIG_VIRTIO_BLK=y
CONFIG_VIRTIO=y
CONFIG_VIRTIO_BALLOON=y
CONFIG_VIRTIO_MMIO=y
If I remember correctly, if your Host Linux (Dom0 or DomD) version >= 4.17
you don't need to modify it as well.
Otherwise, you need to cherry-pick "xen/privcmd: add
IOCTL_PRIVCMD_MMAP_RESOURCE" from the upstream to be able
to use the acquire interface for the resource mapping.
We usually build a backend in the context of the Yocto build process and
run it as a systemd service,
but you can also build and run it manually (it should be launched before
DomU creation).
There are no command line options at all. Everything is configured via
domain configuration file:
# This option is mandatory, it shows that VirtIO is going to be used by
guest
virtio=1
# Example of domain configuration (two disks are assigned to the guest, the
latter is in readonly mode):
vdisk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3;ro:/dev/mmcblk1p3' ]
Hope that helps. Feel free to ask questions if any.
[1] https://github.com/xen-troops/virtio-disk/commits/ioreq_v3
[2] https://xenbits.xen.org/gitweb/?p=people/pauldu/demu.git;a=summary
[3] https://git.kernel.org/pub/scm/linux/kernel/git/will/kvmtool.git/
[4] https://github.com/otyshchenko1/xen/commits/ioreq_4.14_ml3
[5]
https://github.com/otyshchenko1/xen/commit/ee221102193f0422a240832edc41d73f6f3da923
[6]
https://github.com/otyshchenko1/xen/commit/be868a63014b7aa6c9731d5692200d7f2f57c611
--
Regards,
Oleksandr Tyshchenko
[-- Attachment #2: Type: text/html, Size: 6215 bytes --]
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm
2020-10-31 21:10 ` Oleksandr Tyshchenko
@ 2020-11-02 7:23 ` Wei Chen
2020-11-02 18:05 ` Oleksandr
2020-11-03 14:30 ` Masami Hiramatsu
1 sibling, 1 reply; 109+ messages in thread
From: Wei Chen @ 2020-11-02 7:23 UTC (permalink / raw)
To: Oleksandr Tyshchenko, Masami Hiramatsu, Alex Bennée
Cc: Stefano Stabellini, xen-devel, Oleksandr Tyshchenko,
Paul Durrant, Jan Beulich, Andrew Cooper, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson, Julien Grall,
Tim Deegan, Daniel De Graaf, Volodymyr Babchuk, Jun Nakajima,
Kevin Tian, Anthony PERARD, Bertrand Marquis
Hi Oleksandr,
Thanks for the sharing of virtio-disk backend. I have tested it on arm FVP_base platform.
We used domain-0 to run virtio disk backend. The backend disk is a loop device.
"virtio_disks": [
{
"backend_domname": "Domain-0",
"devid": 0,
"disks": [
{
"filename": "/dev/loop0"
}
]
}
],
It works fine and I've pasted some logs:
-------------------------------------------
Domain-0 logs:
main: read backend domid 0
(XEN) gnttab_mark_dirty not implemented yet
(XEN) domain_direct_pl011_init for domain#2
main: read frontend domid 2
Info: connected to dom2
demu_seq_next: >XENSTORE_ATTACHED
demu_seq_next: domid = 2
demu_seq_next: filename[0] = /dev/loop0
demu_seq_next: readonly[0] = 0
demu_seq_next: base[0] = 0x2000000
demu_seq_next: irq[0] = 33
demu_seq_next: >XENCTRL_OPEN
demu_seq_next: >XENEVTCHN_OPEN
demu_seq_next: >XENFOREIGNMEMORY_OPEN
demu_seq_next: >XENDEVICEMODEL_OPEN
demu_initialize: 2 vCPU(s)
demu_seq_next: >SERVER_REGISTERED
demu_seq_next: ioservid = 0
demu_seq_next: >RESOURCE_MAPPED
demu_seq_next: shared_iopage = 0xffffae6de000
demu_seq_next: buffered_iopage = 0xffffae6dd000
demu_seq_next: >SERVER_ENABLED
demu_seq_next: >PORT_ARRAY_ALLOCATED
demu_seq_next: >EVTCHN_PORTS_BOUND
demu_seq_next: VCPU0: 3 -> 7
demu_seq_next: VCPU1: 5 -> 8
demu_seq_next: >EVTCHN_BUF_PORT_BOUND
demu_seq_next: 0 -> 9
demu_register_memory_space: 2000000 - 20001ff
Info: (virtio/mmio.c) virtio_mmio_init:290: mailto:virtio-mmio.devices=0x200@0x2000000:33
demu_seq_next: >DEVICE_INITIALIZED
demu_seq_next: >INITIALIZED
IO request not ready
IO request not ready
----------------
Dom-U logs:
[ 0.491037] xen:xen_evtchn: Event-channel device installed
[ 0.493600] Initialising Xen pvcalls frontend driver
[ 0.516807] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[ 0.525565] cacheinfo: Unable to detect cache hierarchy for CPU 0
[ 0.562275] brd: module loaded
[ 0.595300] loop: module loaded
[ 0.683800] virtio_blk virtio0: [vda] 131072 512-byte logical blocks (67.1 MB/64.0 MiB)
[ 0.684000] vda: detected capacity change from 0 to 67108864
/ # dd if=/dev/vda of=/dev/null bs=1M count=64
64+0 records in
64+0 records out
67108864 bytes (64.0MB) copied, 3.196242 seconds, 20.0MB/s
/ # dd if=/dev/zero of=/dev/vda bs=1M count=64
64+0 records in
64+0 records out
67108864 bytes (64.0MB) copied, 3.704594 seconds, 17.3MB/s
---------------------
The read/write seems OK in dom-U. The FVP platform is a emulator, so the performance is no reference.
We will test it on real hardware like N1SDP.
Thanks,
Wei Chen
----------------------------------------------------------------------------------
From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of Oleksandr Tyshchenko
Sent: 2020年11月1日 5:11
To: Masami Hiramatsu <masami.hiramatsu@linaro.org>; Alex Bennée <alex.bennee@linaro.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>; xen-devel <xen-devel@lists.xenproject.org>; Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Paul Durrant <paul@xen.org>; Jan Beulich <jbeulich@suse.com>; Andrew Cooper <andrew.cooper3@citrix.com>; Roger Pau Monné <roger.pau@citrix.com>; Wei Liu <wl@xen.org>; Julien Grall <Julien.Grall@arm.com>; George Dunlap <george.dunlap@citrix.com>; Ian Jackson <iwj@xenproject.org>; Julien Grall <julien@xen.org>; Tim Deegan <tim@xen.org>; Daniel De Graaf <dgdegra@tycho.nsa.gov>; Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>; Jun Nakajima <jun.nakajima@intel.com>; Kevin Tian <kevin.tian@intel.com>; Anthony PERARD <anthony.perard@citrix.com>; Bertrand Marquis <Bertrand.Marquis@arm.com>
Subject: Re: [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm
On Fri, Oct 30, 2020 at 1:34 PM Masami Hiramatsu <mailto:masami.hiramatsu@linaro.org> wrote:
Hi Oleksandr,
Hi Masami, all
[sorry for the possible format issue]
>> >
>> > Could you tell me how can I test it?
>> >
>> >
>> > I assume it is due to the lack of the virtio-disk backend (which I haven't shared yet as I focused on the IOREQ/DM support on Arm in the
>> > first place).
>> > Could you wait a little bit, I am going to share it soon.
>>
>> Do you have a quick-and-dirty hack you can share in the meantime? Even
>> just on github as a special branch? It would be very useful to be able
>> to have a test-driver for the new feature.
>
> Well, I will provide a branch on github with our PoC virtio-disk backend by the end of this week. It will be possible to test this series with it.
Great! OK I'll be waiting for the PoC backend.
Thank you!
You can find the virtio-disk backend PoC (shared as is) at [1].
Brief description...
The virtio-disk backend PoC is a completely standalone entity (IOREQ server) which emulates a virtio-mmio disk device.
It is based on code from DEMU [2] (for IOREQ server purposes) and some code from kvmtool [3] to implement virtio protocol,
disk operations over underlying H/W and Xenbus code to be able to read configuration from the Xenstore
(it is configured via domain config file). Last patch in this series (marked as RFC) actually adds required bits to the libxl code.
Some notes...
Backend could be used with current V2 IOREQ series [4] without any modifications, all what you need is to enable
CONFIG_IOREQ_SERVER on Arm [5], since it is disabled by default within this series.
Please note that in our system we run backend in DomD (driver domain). I haven't tested it in Dom0,
since in our system the Dom0 is thin (without any H/W) and only used to launch VMs, so there is no underlying block H/W.
But, I hope, it is possible to run it in Dom0 as well (at least there is nothing specific to a particular domain in the backend itself, nothing hardcoded).
If you are going to run a backend in other than Dom0 domain you need to write your own policy (FLASK) for the backend (running in that domain)
to be able to issue DM related requests, etc. Only for test purposes you could use this patch [6] that tweeks Xen dummy policy (not for upstream).
As I mentioned elsewhere you don't need to modify Guest Linux (DomU), just enable VirtIO related configs.
If I remember correctly, the following would be enough:
CONFIG_BLK_MQ_VIRTIO=y
CONFIG_VIRTIO_BLK=y
CONFIG_VIRTIO=y
CONFIG_VIRTIO_BALLOON=y
CONFIG_VIRTIO_MMIO=y
If I remember correctly, if your Host Linux (Dom0 or DomD) version >= 4.17 you don't need to modify it as well.
Otherwise, you need to cherry-pick "xen/privcmd: add IOCTL_PRIVCMD_MMAP_RESOURCE" from the upstream to be able
to use the acquire interface for the resource mapping.
We usually build a backend in the context of the Yocto build process and run it as a systemd service,
but you can also build and run it manually (it should be launched before DomU creation).
There are no command line options at all. Everything is configured via domain configuration file:
# This option is mandatory, it shows that VirtIO is going to be used by guest
virtio=1
# Example of domain configuration (two disks are assigned to the guest, the latter is in readonly mode):
vdisk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3;ro:/dev/mmcblk1p3' ]
Hope that helps. Feel free to ask questions if any.
[1] https://github.com/xen-troops/virtio-disk/commits/ioreq_v3
[2] https://xenbits.xen.org/gitweb/?p=people/pauldu/demu.git;a=summary
[3] https://git.kernel.org/pub/scm/linux/kernel/git/will/kvmtool.git/
[4] https://github.com/otyshchenko1/xen/commits/ioreq_4.14_ml3
[5] https://github.com/otyshchenko1/xen/commit/ee221102193f0422a240832edc41d73f6f3da923
[6] https://github.com/otyshchenko1/xen/commit/be868a63014b7aa6c9731d5692200d7f2f57c611
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm
2020-11-02 7:23 ` Wei Chen
@ 2020-11-02 18:05 ` Oleksandr
0 siblings, 0 replies; 109+ messages in thread
From: Oleksandr @ 2020-11-02 18:05 UTC (permalink / raw)
To: Wei Chen
Cc: Masami Hiramatsu, Alex Bennée, Stefano Stabellini,
xen-devel, Oleksandr Tyshchenko, Paul Durrant, Jan Beulich,
Andrew Cooper, Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson, Julien Grall,
Tim Deegan, Daniel De Graaf, Volodymyr Babchuk, Jun Nakajima,
Kevin Tian, Anthony PERARD, Bertrand Marquis
On 02.11.20 09:23, Wei Chen wrote:
> Hi Oleksandr,
Hi Wei.
>
> Thanks for the sharing of virtio-disk backend. I have tested it on arm FVP_base platform.
> We used domain-0 to run virtio disk backend. The backend disk is a loop device.
> "virtio_disks": [
> {
> "backend_domname": "Domain-0",
> "devid": 0,
> "disks": [
> {
> "filename": "/dev/loop0"
> }
> ]
> }
> ],
>
> It works fine and I've pasted some logs:
>
> -------------------------------------------
> Domain-0 logs:
> main: read backend domid 0
> (XEN) gnttab_mark_dirty not implemented yet
> (XEN) domain_direct_pl011_init for domain#2
> main: read frontend domid 2
> Info: connected to dom2
>
> demu_seq_next: >XENSTORE_ATTACHED
> demu_seq_next: domid = 2
> demu_seq_next: filename[0] = /dev/loop0
> demu_seq_next: readonly[0] = 0
> demu_seq_next: base[0] = 0x2000000
> demu_seq_next: irq[0] = 33
> demu_seq_next: >XENCTRL_OPEN
> demu_seq_next: >XENEVTCHN_OPEN
> demu_seq_next: >XENFOREIGNMEMORY_OPEN
> demu_seq_next: >XENDEVICEMODEL_OPEN
> demu_initialize: 2 vCPU(s)
> demu_seq_next: >SERVER_REGISTERED
> demu_seq_next: ioservid = 0
> demu_seq_next: >RESOURCE_MAPPED
> demu_seq_next: shared_iopage = 0xffffae6de000
> demu_seq_next: buffered_iopage = 0xffffae6dd000
> demu_seq_next: >SERVER_ENABLED
> demu_seq_next: >PORT_ARRAY_ALLOCATED
> demu_seq_next: >EVTCHN_PORTS_BOUND
> demu_seq_next: VCPU0: 3 -> 7
> demu_seq_next: VCPU1: 5 -> 8
> demu_seq_next: >EVTCHN_BUF_PORT_BOUND
> demu_seq_next: 0 -> 9
> demu_register_memory_space: 2000000 - 20001ff
> Info: (virtio/mmio.c) virtio_mmio_init:290: mailto:virtio-mmio.devices=0x200@0x2000000:33
> demu_seq_next: >DEVICE_INITIALIZED
> demu_seq_next: >INITIALIZED
> IO request not ready
> IO request not ready
>
> ----------------
> Dom-U logs:
> [ 0.491037] xen:xen_evtchn: Event-channel device installed
> [ 0.493600] Initialising Xen pvcalls frontend driver
> [ 0.516807] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
> [ 0.525565] cacheinfo: Unable to detect cache hierarchy for CPU 0
> [ 0.562275] brd: module loaded
> [ 0.595300] loop: module loaded
> [ 0.683800] virtio_blk virtio0: [vda] 131072 512-byte logical blocks (67.1 MB/64.0 MiB)
> [ 0.684000] vda: detected capacity change from 0 to 67108864
>
>
> / # dd if=/dev/vda of=/dev/null bs=1M count=64
> 64+0 records in
> 64+0 records out
> 67108864 bytes (64.0MB) copied, 3.196242 seconds, 20.0MB/s
> / # dd if=/dev/zero of=/dev/vda bs=1M count=64
> 64+0 records in
> 64+0 records out
> 67108864 bytes (64.0MB) copied, 3.704594 seconds, 17.3MB/s
> ---------------------
>
> The read/write seems OK in dom-U. The FVP platform is a emulator, so the performance is no reference.
> We will test it on real hardware like N1SDP.
This is really a good news. Thank you for testing!
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm
2020-10-31 21:10 ` Oleksandr Tyshchenko
2020-11-02 7:23 ` Wei Chen
@ 2020-11-03 14:30 ` Masami Hiramatsu
2020-11-03 21:09 ` Oleksandr
1 sibling, 1 reply; 109+ messages in thread
From: Masami Hiramatsu @ 2020-11-03 14:30 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: Alex Bennée, Stefano Stabellini, xen-devel,
Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson, Julien Grall,
Tim Deegan, Daniel De Graaf, Volodymyr Babchuk, Jun Nakajima,
Kevin Tian, Anthony PERARD, Bertrand Marquis
Hi Oleksandr,
Thanks for sharing the virtio-disk server, I also tested with a real usb disk.
In config file:
virtio = 1
vdisk = [ 'backend=Domain-0, disks=ro:/dev/sda1' ]
And it can be mounted in DomU
[ 2.892874] virtio_blk virtio0: [vda] 1875382927 512-byte logical
blocks (960 GB/894 GiB)
[ 2.892925] vda: detected capacity change from 0 to 960196058624
...
root@develbox:~# cat /proc/partitions
major minor #blocks name
254 0 937691463 vda
...
root@develbox:~# mount /dev/vda /mnt/
[ 192.260968] EXT4-fs (vda): mounted filesystem with ordered data
mode. Opts: (null)
mount: /mnt: WARNING: source write-protected, mounted read-only.
So "ro" flag also correctly works.
Great!
Thank you!
2020年11月1日(日) 6:10 Oleksandr Tyshchenko <olekstysh@gmail.com>:
>
>
>
> On Fri, Oct 30, 2020 at 1:34 PM Masami Hiramatsu <masami.hiramatsu@linaro.org> wrote:
>>
>> Hi Oleksandr,
>
>
> Hi Masami, all
>
> [sorry for the possible format issue]
>
>>
>> >> >
>> >> > Could you tell me how can I test it?
>> >> >
>> >> >
>> >> > I assume it is due to the lack of the virtio-disk backend (which I haven't shared yet as I focused on the IOREQ/DM support on Arm in the
>> >> > first place).
>> >> > Could you wait a little bit, I am going to share it soon.
>> >>
>> >> Do you have a quick-and-dirty hack you can share in the meantime? Even
>> >> just on github as a special branch? It would be very useful to be able
>> >> to have a test-driver for the new feature.
>> >
>> > Well, I will provide a branch on github with our PoC virtio-disk backend by the end of this week. It will be possible to test this series with it.
>>
>> Great! OK I'll be waiting for the PoC backend.
>>
>> Thank you!
>
>
> You can find the virtio-disk backend PoC (shared as is) at [1].
>
> Brief description...
>
> The virtio-disk backend PoC is a completely standalone entity (IOREQ server) which emulates a virtio-mmio disk device.
> It is based on code from DEMU [2] (for IOREQ server purposes) and some code from kvmtool [3] to implement virtio protocol,
> disk operations over underlying H/W and Xenbus code to be able to read configuration from the Xenstore
> (it is configured via domain config file). Last patch in this series (marked as RFC) actually adds required bits to the libxl code.
>
> Some notes...
>
> Backend could be used with current V2 IOREQ series [4] without any modifications, all what you need is to enable
> CONFIG_IOREQ_SERVER on Arm [5], since it is disabled by default within this series.
>
> Please note that in our system we run backend in DomD (driver domain). I haven't tested it in Dom0,
> since in our system the Dom0 is thin (without any H/W) and only used to launch VMs, so there is no underlying block H/W.
> But, I hope, it is possible to run it in Dom0 as well (at least there is nothing specific to a particular domain in the backend itself, nothing hardcoded).
> If you are going to run a backend in other than Dom0 domain you need to write your own policy (FLASK) for the backend (running in that domain)
> to be able to issue DM related requests, etc. Only for test purposes you could use this patch [6] that tweeks Xen dummy policy (not for upstream).
>
> As I mentioned elsewhere you don't need to modify Guest Linux (DomU), just enable VirtIO related configs.
> If I remember correctly, the following would be enough:
> CONFIG_BLK_MQ_VIRTIO=y
> CONFIG_VIRTIO_BLK=y
> CONFIG_VIRTIO=y
> CONFIG_VIRTIO_BALLOON=y
> CONFIG_VIRTIO_MMIO=y
> If I remember correctly, if your Host Linux (Dom0 or DomD) version >= 4.17 you don't need to modify it as well.
> Otherwise, you need to cherry-pick "xen/privcmd: add IOCTL_PRIVCMD_MMAP_RESOURCE" from the upstream to be able
> to use the acquire interface for the resource mapping.
>
> We usually build a backend in the context of the Yocto build process and run it as a systemd service,
> but you can also build and run it manually (it should be launched before DomU creation).
>
> There are no command line options at all. Everything is configured via domain configuration file:
> # This option is mandatory, it shows that VirtIO is going to be used by guest
> virtio=1
> # Example of domain configuration (two disks are assigned to the guest, the latter is in readonly mode):
> vdisk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3;ro:/dev/mmcblk1p3' ]
>
> Hope that helps. Feel free to ask questions if any.
>
> [1] https://github.com/xen-troops/virtio-disk/commits/ioreq_v3
> [2] https://xenbits.xen.org/gitweb/?p=people/pauldu/demu.git;a=summary
> [3] https://git.kernel.org/pub/scm/linux/kernel/git/will/kvmtool.git/
> [4] https://github.com/otyshchenko1/xen/commits/ioreq_4.14_ml3
> [5] https://github.com/otyshchenko1/xen/commit/ee221102193f0422a240832edc41d73f6f3da923
> [6] https://github.com/otyshchenko1/xen/commit/be868a63014b7aa6c9731d5692200d7f2f57c611
>
> --
> Regards,
>
> Oleksandr Tyshchenko
--
Masami Hiramatsu
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm
2020-11-03 14:30 ` Masami Hiramatsu
@ 2020-11-03 21:09 ` Oleksandr
0 siblings, 0 replies; 109+ messages in thread
From: Oleksandr @ 2020-11-03 21:09 UTC (permalink / raw)
To: Masami Hiramatsu
Cc: Alex Bennée, Stefano Stabellini, xen-devel,
Oleksandr Tyshchenko, Paul Durrant, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, Julien Grall, George Dunlap, Ian Jackson, Julien Grall,
Tim Deegan, Daniel De Graaf, Volodymyr Babchuk, Jun Nakajima,
Kevin Tian, Anthony PERARD, Bertrand Marquis
On 03.11.20 16:30, Masami Hiramatsu wrote:
> Hi Oleksandr,
Hi Masami
>
> Thanks for sharing the virtio-disk server, I also tested with a real usb disk.
>
> In config file:
>
> virtio = 1
> vdisk = [ 'backend=Domain-0, disks=ro:/dev/sda1' ]
>
> And it can be mounted in DomU
>
> [ 2.892874] virtio_blk virtio0: [vda] 1875382927 512-byte logical
> blocks (960 GB/894 GiB)
> [ 2.892925] vda: detected capacity change from 0 to 960196058624
> ...
> root@develbox:~# cat /proc/partitions
> major minor #blocks name
>
> 254 0 937691463 vda
> ...
> root@develbox:~# mount /dev/vda /mnt/
> [ 192.260968] EXT4-fs (vda): mounted filesystem with ordered data
> mode. Opts: (null)
> mount: /mnt: WARNING: source write-protected, mounted read-only.
>
> So "ro" flag also correctly works.
> Great!
>
> Thank you!
Sounds great. Thank you for testing!
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 01/23] x86/ioreq: Prepare IOREQ feature for making it common
2020-10-20 7:13 ` Paul Durrant
@ 2020-11-04 9:06 ` Oleksandr
2020-11-04 9:59 ` Paul Durrant
0 siblings, 1 reply; 109+ messages in thread
From: Oleksandr @ 2020-11-04 9:06 UTC (permalink / raw)
To: paul, xen-devel
Cc: 'Oleksandr Tyshchenko', 'Jan Beulich',
'Andrew Cooper', 'Roger Pau Monné',
'Julien Grall', 'Stefano Stabellini',
'Wei Liu', 'Julien Grall'
On 20.10.20 10:13, Paul Durrant wrote:
Hi Paul.
Sorry for the late response.
>> +
>> +/* Called when target domain is paused */
>> +static inline void arch_hvm_destroy_ioreq_server(struct hvm_ioreq_server *s)
>> +{
>> + p2m_set_ioreq_server(s->target, 0, s);
>> +}
>> +
>> +/*
>> + * Map or unmap an ioreq server to specific memory type. For now, only
>> + * HVMMEM_ioreq_server is supported, and in the future new types can be
>> + * introduced, e.g. HVMMEM_ioreq_serverX mapped to ioreq server X. And
>> + * currently, only write operations are to be forwarded to an ioreq server.
>> + * Support for the emulation of read operations can be added when an ioreq
>> + * server has such requirement in the future.
>> + */
>> +static inline int hvm_map_mem_type_to_ioreq_server(struct domain *d,
>> + ioservid_t id,
>> + uint32_t type,
>> + uint32_t flags)
>> +{
>> + struct hvm_ioreq_server *s;
>> + int rc;
>> +
>> + if ( type != HVMMEM_ioreq_server )
>> + return -EINVAL;
>> +
>> + if ( flags & ~XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
>> + return -EINVAL;
>> +
>> + spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
>> +
>> + s = get_ioreq_server(d, id);
>> +
>> + rc = -ENOENT;
>> + if ( !s )
>> + goto out;
>> +
>> + rc = -EPERM;
>> + if ( s->emulator != current->domain )
>> + goto out;
>> +
>> + rc = p2m_set_ioreq_server(d, flags, s);
>> +
>> + out:
>> + spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
>> +
>> + if ( rc == 0 && flags == 0 )
>> + {
>> + struct p2m_domain *p2m = p2m_get_hostp2m(d);
>> +
>> + if ( read_atomic(&p2m->ioreq.entry_count) )
>> + p2m_change_entry_type_global(d, p2m_ioreq_server, p2m_ram_rw);
>> + }
>> +
>> + return rc;
>> +}
>> +
> The above doesn't really feel right to me. It's really an entry point into the ioreq server code and as such I think it ought to be left in the common code. I suggest replacing the p2m_set_ioreq_server() function with an arch specific function (also taking the type) which you can then implement here.
Agree that it ought to be left in the common code.
However, I am afraid I didn't entirely get you suggestion how this
function could be split. On Arm struct p2m_domain doesn't contain IOREQ
fields (p2m->ioreq.XXX), nor p2m_change_entry_type_global() is used, so
they should be abstracted together with p2m_set_ioreq_server().
So the whole "if ( rc == 0 && flags == 0 )" check should be folded into
arch_p2m_set_ioreq_server implementation on x86? This in turn raises a
question can we put a spin_unlock after.
I am wondering would it be acceptable to replace
hvm_map_mem_type_to_ioreq_server by
arch_hvm_map_mem_type_to_ioreq_server here and have the following in the
common code:
int hvm_map_mem_type_to_ioreq_server(struct domain *d,
ioservid_t id,
uint32_t type,
uint32_t flags)
{
return arch_hvm_map_mem_type_to_ioreq_server(d, id, type, flags);
}
>
> The rest of the patch looks ok.
Thank you.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [PATCH V2 01/23] x86/ioreq: Prepare IOREQ feature for making it common
2020-11-04 9:06 ` Oleksandr
@ 2020-11-04 9:59 ` Paul Durrant
0 siblings, 0 replies; 109+ messages in thread
From: Paul Durrant @ 2020-11-04 9:59 UTC (permalink / raw)
To: 'Oleksandr', xen-devel
Cc: 'Oleksandr Tyshchenko', 'Jan Beulich',
'Andrew Cooper', 'Roger Pau Monné',
'Julien Grall', 'Stefano Stabellini',
'Wei Liu', 'Julien Grall'
> -----Original Message-----
> From: Oleksandr <olekstysh@gmail.com>
> Sent: 04 November 2020 09:06
> To: paul@xen.org; xen-devel@lists.xenproject.org
> Cc: 'Oleksandr Tyshchenko' <oleksandr_tyshchenko@epam.com>; 'Jan Beulich' <jbeulich@suse.com>; 'Andrew
> Cooper' <andrew.cooper3@citrix.com>; 'Roger Pau Monné' <roger.pau@citrix.com>; 'Julien Grall'
> <julien@xen.org>; 'Stefano Stabellini' <sstabellini@kernel.org>; 'Wei Liu' <wl@xen.org>; 'Julien
> Grall' <julien.grall@arm.com>
> Subject: Re: [PATCH V2 01/23] x86/ioreq: Prepare IOREQ feature for making it common
>
>
> On 20.10.20 10:13, Paul Durrant wrote:
>
> Hi Paul.
>
> Sorry for the late response.
>
> >> +
> >> +/* Called when target domain is paused */
> >> +static inline void arch_hvm_destroy_ioreq_server(struct hvm_ioreq_server *s)
> >> +{
> >> + p2m_set_ioreq_server(s->target, 0, s);
> >> +}
> >> +
> >> +/*
> >> + * Map or unmap an ioreq server to specific memory type. For now, only
> >> + * HVMMEM_ioreq_server is supported, and in the future new types can be
> >> + * introduced, e.g. HVMMEM_ioreq_serverX mapped to ioreq server X. And
> >> + * currently, only write operations are to be forwarded to an ioreq server.
> >> + * Support for the emulation of read operations can be added when an ioreq
> >> + * server has such requirement in the future.
> >> + */
> >> +static inline int hvm_map_mem_type_to_ioreq_server(struct domain *d,
> >> + ioservid_t id,
> >> + uint32_t type,
> >> + uint32_t flags)
> >> +{
> >> + struct hvm_ioreq_server *s;
> >> + int rc;
> >> +
> >> + if ( type != HVMMEM_ioreq_server )
> >> + return -EINVAL;
> >> +
> >> + if ( flags & ~XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
> >> + return -EINVAL;
> >> +
> >> + spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
> >> +
> >> + s = get_ioreq_server(d, id);
> >> +
> >> + rc = -ENOENT;
> >> + if ( !s )
> >> + goto out;
> >> +
> >> + rc = -EPERM;
> >> + if ( s->emulator != current->domain )
> >> + goto out;
> >> +
> >> + rc = p2m_set_ioreq_server(d, flags, s);
> >> +
> >> + out:
> >> + spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
> >> +
> >> + if ( rc == 0 && flags == 0 )
> >> + {
> >> + struct p2m_domain *p2m = p2m_get_hostp2m(d);
> >> +
> >> + if ( read_atomic(&p2m->ioreq.entry_count) )
> >> + p2m_change_entry_type_global(d, p2m_ioreq_server, p2m_ram_rw);
> >> + }
> >> +
> >> + return rc;
> >> +}
> >> +
> > The above doesn't really feel right to me. It's really an entry point into the ioreq server code and
> as such I think it ought to be left in the common code. I suggest replacing the p2m_set_ioreq_server()
> function with an arch specific function (also taking the type) which you can then implement here.
>
> Agree that it ought to be left in the common code.
>
> However, I am afraid I didn't entirely get you suggestion how this
> function could be split. On Arm struct p2m_domain doesn't contain IOREQ
> fields (p2m->ioreq.XXX), nor p2m_change_entry_type_global() is used, so
> they should be abstracted together with p2m_set_ioreq_server().
>
> So the whole "if ( rc == 0 && flags == 0 )" check should be folded into
> arch_p2m_set_ioreq_server implementation on x86? This in turn raises a
> question can we put a spin_unlock after.
>
Hi Oleksandr,
I think the code as it stands is really a bit of a layering violation. I don't really see a problem with retaining the ioreq server lock around the call to p2m_change_entry_type_global() so I'd just fold that into p2m_set_ioreq_server().
Paul
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [PATCH V2 23/23] [RFC] libxl: Add support for virtio-disk configuration
2020-10-15 16:44 ` [PATCH V2 23/23] [RFC] libxl: Add support for virtio-disk configuration Oleksandr Tyshchenko
@ 2020-11-09 6:45 ` Wei Chen
2020-11-11 0:53 ` Oleksandr
0 siblings, 1 reply; 109+ messages in thread
From: Wei Chen @ 2020-11-09 6:45 UTC (permalink / raw)
To: Oleksandr Tyshchenko, xen-devel
Cc: Oleksandr Tyshchenko, Ian Jackson, Wei Liu, Anthony PERARD,
Julien Grall, Stefano Stabellini
Hi Oleksandr,
> -----Original Message-----
> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of
> Oleksandr Tyshchenko
> Sent: 2020年10月16日 0:45
> To: xen-devel@lists.xenproject.org
> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Ian Jackson
> <iwj@xenproject.org>; Wei Liu <wl@xen.org>; Anthony PERARD
> <anthony.perard@citrix.com>; Julien Grall <julien@xen.org>; Stefano Stabellini
> <sstabellini@kernel.org>
> Subject: [PATCH V2 23/23] [RFC] libxl: Add support for virtio-disk configuration
>
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> This patch adds basic support for configuring and assisting virtio-disk
> backend (emualator) which is intended to run out of Qemu and could be run
> in any domain.
>
> Xenstore was chosen as a communication interface for the emulator running
> in non-toolstack domain to be able to get configuration either by reading
> Xenstore directly or by receiving command line parameters (an updated 'xl devd'
> running in the same domain would read Xenstore beforehand and call backend
> executable with the required arguments).
>
> An example of domain configuration (two disks are assigned to the guest,
> the latter is in readonly mode):
>
> vdisk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3;ro:/dev/mmcblk1p3' ]
>
Can we keep use the same 'disk' parameter for virtio-disk, but add an option like
"model=virtio-disk"?
For example:
disk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3,model=virtio-disk' ]
Just like what Xen has done for x86 virtio-net.
> Where per-disk Xenstore entries are:
> - filename and readonly flag (configured via "vdisk" property)
> - base and irq (allocated dynamically)
>
> Besides handling 'visible' params described in configuration file,
> patch also allocates virtio-mmio specific ones for each device and
> writes them into Xenstore. virtio-mmio params (irq and base) are
> unique per guest domain, they allocated at the domain creation time
> and passed through to the emulator. Each VirtIO device has at least
> one pair of these params.
>
> TODO:
> 1. An extra "virtio" property could be removed.
> 2. Update documentation.
>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> ---
> Changes RFC -> V1:
> - no changes
>
> Changes V1 -> V2:
> - rebase according to the new location of libxl_virtio_disk.c
>
> Please note, there is a real concern about VirtIO interrupts allocation.
> Just copy here what Stefano said in RFC thread.
>
> So, if we end up allocating let's say 6 virtio interrupts for a domain,
> the chance of a clash with a physical interrupt of a passthrough device is real.
>
> I am not entirely sure how to solve it, but these are a few ideas:
> - choosing virtio interrupts that are less likely to conflict (maybe > 1000)
> - make the virtio irq (optionally) configurable so that a user could
> override the default irq and specify one that doesn't conflict
> - implementing support for virq != pirq (even the xl interface doesn't
> allow to specify the virq number for passthrough devices, see "irqs")
>
> ---
> tools/libs/light/Makefile | 1 +
> tools/libs/light/libxl_arm.c | 56 ++++++++++++---
> tools/libs/light/libxl_create.c | 1 +
> tools/libs/light/libxl_internal.h | 1 +
> tools/libs/light/libxl_types.idl | 15 ++++
> tools/libs/light/libxl_types_internal.idl | 1 +
> tools/libs/light/libxl_virtio_disk.c | 109 ++++++++++++++++++++++++++++
> tools/xl/Makefile | 2 +-
> tools/xl/xl.h | 3 +
> tools/xl/xl_cmdtable.c | 15 ++++
> tools/xl/xl_parse.c | 115 ++++++++++++++++++++++++++++++
> tools/xl/xl_virtio_disk.c | 46 ++++++++++++
> 12 files changed, 354 insertions(+), 11 deletions(-)
> create mode 100644 tools/libs/light/libxl_virtio_disk.c
> create mode 100644 tools/xl/xl_virtio_disk.c
>
> diff --git a/tools/libs/light/Makefile b/tools/libs/light/Makefile
> index f58a321..2ee388a 100644
> --- a/tools/libs/light/Makefile
> +++ b/tools/libs/light/Makefile
> @@ -115,6 +115,7 @@ SRCS-y += libxl_genid.c
> SRCS-y += _libxl_types.c
> SRCS-y += libxl_flask.c
> SRCS-y += _libxl_types_internal.c
> +SRCS-y += libxl_virtio_disk.c
>
> ifeq ($(CONFIG_LIBNL),y)
> CFLAGS_LIBXL += $(LIBNL3_CFLAGS)
> diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c
> index 588ee5a..9eb3022 100644
> --- a/tools/libs/light/libxl_arm.c
> +++ b/tools/libs/light/libxl_arm.c
> @@ -8,6 +8,12 @@
> #include <assert.h>
> #include <xen/device_tree_defs.h>
>
> +#ifndef container_of
> +#define container_of(ptr, type, member) ({ \
> + typeof( ((type *)0)->member ) *__mptr = (ptr); \
> + (type *)( (char *)__mptr - offsetof(type,member) );})
> +#endif
> +
> static const char *gicv_to_string(libxl_gic_version gic_version)
> {
> switch (gic_version) {
> @@ -39,14 +45,32 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
> vuart_enabled = true;
> }
>
> - /*
> - * XXX: Handle properly virtio
> - * A proper solution would be the toolstack to allocate the interrupts
> - * used by each virtio backend and let the backend now which one is used
> - */
> if (libxl_defbool_val(d_config->b_info.arch_arm.virtio)) {
> - nr_spis += (GUEST_VIRTIO_MMIO_SPI - 32) + 1;
> + uint64_t virtio_base;
> + libxl_device_virtio_disk *virtio_disk;
> +
> + virtio_base = GUEST_VIRTIO_MMIO_BASE;
> virtio_irq = GUEST_VIRTIO_MMIO_SPI;
> +
> + if (!d_config->num_virtio_disks) {
> + LOG(ERROR, "Virtio is enabled, but no Virtio devices present\n");
> + return ERROR_FAIL;
> + }
> + virtio_disk = &d_config->virtio_disks[0];
> +
> + for (i = 0; i < virtio_disk->num_disks; i++) {
> + virtio_disk->disks[i].base = virtio_base;
> + virtio_disk->disks[i].irq = virtio_irq;
> +
> + LOG(DEBUG, "Allocate Virtio MMIO params: IRQ %u BASE 0x%"PRIx64,
> + virtio_irq, virtio_base);
> +
> + virtio_irq ++;
> + virtio_base += GUEST_VIRTIO_MMIO_SIZE;
> + }
> + virtio_irq --;
> +
> + nr_spis += (virtio_irq - 32) + 1;
> virtio_enabled = true;
> }
>
> @@ -70,8 +94,9 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
> }
>
> /* The same check as for vpl011 */
> - if (virtio_enabled && irq == virtio_irq) {
> - LOG(ERROR, "Physical IRQ %u conflicting with virtio SPI\n", irq);
> + if (virtio_enabled &&
> + (irq >= GUEST_VIRTIO_MMIO_SPI && irq <= virtio_irq)) {
> + LOG(ERROR, "Physical IRQ %u conflicting with Virtio IRQ range\n", irq);
> return ERROR_FAIL;
> }
>
> @@ -1011,8 +1036,19 @@ next_resize:
> if (info->tee == LIBXL_TEE_TYPE_OPTEE)
> FDT( make_optee_node(gc, fdt) );
>
> - if (libxl_defbool_val(info->arch_arm.virtio))
> - FDT( make_virtio_mmio_node(gc, fdt, GUEST_VIRTIO_MMIO_BASE,
> GUEST_VIRTIO_MMIO_SPI) );
> + if (libxl_defbool_val(info->arch_arm.virtio)) {
> + libxl_domain_config *d_config =
> + container_of(info, libxl_domain_config, b_info);
> + libxl_device_virtio_disk *virtio_disk = &d_config->virtio_disks[0];
> + unsigned int i;
> +
> + for (i = 0; i < virtio_disk->num_disks; i++) {
> + uint64_t base = virtio_disk->disks[i].base;
> + uint32_t irq = virtio_disk->disks[i].irq;
> +
> + FDT( make_virtio_mmio_node(gc, fdt, base, irq) );
> + }
> + }
>
> if (pfdt)
> FDT( copy_partial_fdt(gc, fdt, pfdt) );
> diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
> index 321a13e..8da328d 100644
> --- a/tools/libs/light/libxl_create.c
> +++ b/tools/libs/light/libxl_create.c
> @@ -1821,6 +1821,7 @@ const libxl__device_type *device_type_tbl[] = {
> &libxl__dtdev_devtype,
> &libxl__vdispl_devtype,
> &libxl__vsnd_devtype,
> + &libxl__virtio_disk_devtype,
> NULL
> };
>
> diff --git a/tools/libs/light/libxl_internal.h b/tools/libs/light/libxl_internal.h
> index e26cda9..ea497bb 100644
> --- a/tools/libs/light/libxl_internal.h
> +++ b/tools/libs/light/libxl_internal.h
> @@ -4000,6 +4000,7 @@ extern const libxl__device_type libxl__vdispl_devtype;
> extern const libxl__device_type libxl__p9_devtype;
> extern const libxl__device_type libxl__pvcallsif_devtype;
> extern const libxl__device_type libxl__vsnd_devtype;
> +extern const libxl__device_type libxl__virtio_disk_devtype;
>
> extern const libxl__device_type *device_type_tbl[];
>
> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
> index b054bf9..5f8a3ff 100644
> --- a/tools/libs/light/libxl_types.idl
> +++ b/tools/libs/light/libxl_types.idl
> @@ -935,6 +935,20 @@ libxl_device_vsnd = Struct("device_vsnd", [
> ("pcms", Array(libxl_vsnd_pcm, "num_vsnd_pcms"))
> ])
>
> +libxl_virtio_disk_param = Struct("virtio_disk_param", [
> + ("filename", string),
> + ("readonly", bool),
> + ("irq", uint32),
> + ("base", uint64),
> + ])
> +
> +libxl_device_virtio_disk = Struct("device_virtio_disk", [
> + ("backend_domid", libxl_domid),
> + ("backend_domname", string),
> + ("devid", libxl_devid),
> + ("disks", Array(libxl_virtio_disk_param, "num_disks")),
> + ])
> +
> libxl_domain_config = Struct("domain_config", [
> ("c_info", libxl_domain_create_info),
> ("b_info", libxl_domain_build_info),
> @@ -951,6 +965,7 @@ libxl_domain_config = Struct("domain_config", [
> ("pvcallsifs", Array(libxl_device_pvcallsif, "num_pvcallsifs")),
> ("vdispls", Array(libxl_device_vdispl, "num_vdispls")),
> ("vsnds", Array(libxl_device_vsnd, "num_vsnds")),
> + ("virtio_disks", Array(libxl_device_virtio_disk, "num_virtio_disks")),
> # a channel manifests as a console with a name,
> # see docs/misc/channels.txt
> ("channels", Array(libxl_device_channel, "num_channels")),
> diff --git a/tools/libs/light/libxl_types_internal.idl
> b/tools/libs/light/libxl_types_internal.idl
> index 3593e21..8f71980 100644
> --- a/tools/libs/light/libxl_types_internal.idl
> +++ b/tools/libs/light/libxl_types_internal.idl
> @@ -32,6 +32,7 @@ libxl__device_kind = Enumeration("device_kind", [
> (14, "PVCALLS"),
> (15, "VSND"),
> (16, "VINPUT"),
> + (17, "VIRTIO_DISK"),
> ])
>
> libxl__console_backend = Enumeration("console_backend", [
> diff --git a/tools/libs/light/libxl_virtio_disk.c b/tools/libs/light/libxl_virtio_disk.c
> new file mode 100644
> index 0000000..25e7f1a
> --- /dev/null
> +++ b/tools/libs/light/libxl_virtio_disk.c
> @@ -0,0 +1,109 @@
> +/*
> + * Copyright (C) 2020 EPAM Systems Inc.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU Lesser General Public License as published
> + * by the Free Software Foundation; version 2.1 only. with the special
> + * exception on linking described in file LICENSE.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU Lesser General Public License for more details.
> + */
> +
> +#include "libxl_internal.h"
> +
> +static int libxl__device_virtio_disk_setdefault(libxl__gc *gc, uint32_t domid,
> + libxl_device_virtio_disk *virtio_disk,
> + bool hotplug)
> +{
> + return libxl__resolve_domid(gc, virtio_disk->backend_domname,
> + &virtio_disk->backend_domid);
> +}
> +
> +static int libxl__virtio_disk_from_xenstore(libxl__gc *gc, const char *libxl_path,
> + libxl_devid devid,
> + libxl_device_virtio_disk *virtio_disk)
> +{
> + const char *be_path;
> + int rc;
> +
> + virtio_disk->devid = devid;
> + rc = libxl__xs_read_mandatory(gc, XBT_NULL,
> + GCSPRINTF("%s/backend", libxl_path),
> + &be_path);
> + if (rc) return rc;
> +
> + rc = libxl__backendpath_parse_domid(gc, be_path, &virtio_disk-
> >backend_domid);
> + if (rc) return rc;
> +
> + return 0;
> +}
> +
> +static void libxl__update_config_virtio_disk(libxl__gc *gc,
> + libxl_device_virtio_disk *dst,
> + libxl_device_virtio_disk *src)
> +{
> + dst->devid = src->devid;
> +}
> +
> +static int libxl_device_virtio_disk_compare(libxl_device_virtio_disk *d1,
> + libxl_device_virtio_disk *d2)
> +{
> + return COMPARE_DEVID(d1, d2);
> +}
> +
> +static void libxl__device_virtio_disk_add(libxl__egc *egc, uint32_t domid,
> + libxl_device_virtio_disk *virtio_disk,
> + libxl__ao_device *aodev)
> +{
> + libxl__device_add_async(egc, domid, &libxl__virtio_disk_devtype, virtio_disk,
> aodev);
> +}
> +
> +static int libxl__set_xenstore_virtio_disk(libxl__gc *gc, uint32_t domid,
> + libxl_device_virtio_disk *virtio_disk,
> + flexarray_t *back, flexarray_t *front,
> + flexarray_t *ro_front)
> +{
> + int rc;
> + unsigned int i;
> +
> + for (i = 0; i < virtio_disk->num_disks; i++) {
> + rc = flexarray_append_pair(ro_front, GCSPRINTF("%d/filename", i),
> + GCSPRINTF("%s", virtio_disk->disks[i].filename));
> + if (rc) return rc;
> +
> + rc = flexarray_append_pair(ro_front, GCSPRINTF("%d/readonly", i),
> + GCSPRINTF("%d", virtio_disk->disks[i].readonly));
> + if (rc) return rc;
> +
> + rc = flexarray_append_pair(ro_front, GCSPRINTF("%d/base", i),
> + GCSPRINTF("%lu", virtio_disk->disks[i].base));
> + if (rc) return rc;
> +
> + rc = flexarray_append_pair(ro_front, GCSPRINTF("%d/irq", i),
> + GCSPRINTF("%u", virtio_disk->disks[i].irq));
> + if (rc) return rc;
> + }
> +
> + return 0;
> +}
> +
> +static LIBXL_DEFINE_UPDATE_DEVID(virtio_disk)
> +static LIBXL_DEFINE_DEVICE_FROM_TYPE(virtio_disk)
> +static LIBXL_DEFINE_DEVICES_ADD(virtio_disk)
> +
> +DEFINE_DEVICE_TYPE_STRUCT(virtio_disk, VIRTIO_DISK,
> + .update_config = (device_update_config_fn_t)
> libxl__update_config_virtio_disk,
> + .from_xenstore = (device_from_xenstore_fn_t)
> libxl__virtio_disk_from_xenstore,
> + .set_xenstore_config = (device_set_xenstore_config_fn_t)
> libxl__set_xenstore_virtio_disk
> +);
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> diff --git a/tools/xl/Makefile b/tools/xl/Makefile
> index bdf67c8..9d8f2aa 100644
> --- a/tools/xl/Makefile
> +++ b/tools/xl/Makefile
> @@ -23,7 +23,7 @@ XL_OBJS += xl_vtpm.o xl_block.o xl_nic.o xl_usb.o
> XL_OBJS += xl_sched.o xl_pci.o xl_vcpu.o xl_cdrom.o xl_mem.o
> XL_OBJS += xl_info.o xl_console.o xl_misc.o
> XL_OBJS += xl_vmcontrol.o xl_saverestore.o xl_migrate.o
> -XL_OBJS += xl_vdispl.o xl_vsnd.o xl_vkb.o
> +XL_OBJS += xl_vdispl.o xl_vsnd.o xl_vkb.o xl_virtio_disk.o
>
> $(XL_OBJS): CFLAGS += $(CFLAGS_libxentoollog)
> $(XL_OBJS): CFLAGS += $(CFLAGS_XL)
> diff --git a/tools/xl/xl.h b/tools/xl/xl.h
> index 06569c6..3d26f19 100644
> --- a/tools/xl/xl.h
> +++ b/tools/xl/xl.h
> @@ -178,6 +178,9 @@ int main_vsnddetach(int argc, char **argv);
> int main_vkbattach(int argc, char **argv);
> int main_vkblist(int argc, char **argv);
> int main_vkbdetach(int argc, char **argv);
> +int main_virtio_diskattach(int argc, char **argv);
> +int main_virtio_disklist(int argc, char **argv);
> +int main_virtio_diskdetach(int argc, char **argv);
> int main_usbctrl_attach(int argc, char **argv);
> int main_usbctrl_detach(int argc, char **argv);
> int main_usbdev_attach(int argc, char **argv);
> diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
> index 7da6c1b..745afab 100644
> --- a/tools/xl/xl_cmdtable.c
> +++ b/tools/xl/xl_cmdtable.c
> @@ -435,6 +435,21 @@ struct cmd_spec cmd_table[] = {
> "Destroy a domain's virtual sound device",
> "<Domain> <DevId>",
> },
> + { "virtio-disk-attach",
> + &main_virtio_diskattach, 1, 1,
> + "Create a new virtio block device",
> + " TBD\n"
> + },
> + { "virtio-disk-list",
> + &main_virtio_disklist, 0, 0,
> + "List virtio block devices for a domain",
> + "<Domain(s)>",
> + },
> + { "virtio-disk-detach",
> + &main_virtio_diskdetach, 0, 1,
> + "Destroy a domain's virtio block device",
> + "<Domain> <DevId>",
> + },
> { "uptime",
> &main_uptime, 0, 0,
> "Print uptime for all/some domains",
> diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
> index 10acf22..6cf3524 100644
> --- a/tools/xl/xl_parse.c
> +++ b/tools/xl/xl_parse.c
> @@ -1204,6 +1204,120 @@ out:
> if (rc) exit(EXIT_FAILURE);
> }
>
> +#define MAX_VIRTIO_DISKS 4
> +
> +static int parse_virtio_disk_config(libxl_device_virtio_disk *virtio_disk, char
> *token)
> +{
> + char *oparg;
> + libxl_string_list disks = NULL;
> + int i, rc;
> +
> + if (MATCH_OPTION("backend", token, oparg)) {
> + virtio_disk->backend_domname = strdup(oparg);
> + } else if (MATCH_OPTION("disks", token, oparg)) {
> + split_string_into_string_list(oparg, ";", &disks);
> +
> + virtio_disk->num_disks = libxl_string_list_length(&disks);
> + if (virtio_disk->num_disks > MAX_VIRTIO_DISKS) {
> + fprintf(stderr, "vdisk: currently only %d disks are supported",
> + MAX_VIRTIO_DISKS);
> + return 1;
> + }
> + virtio_disk->disks = xcalloc(virtio_disk->num_disks,
> + sizeof(*virtio_disk->disks));
> +
> + for(i = 0; i < virtio_disk->num_disks; i++) {
> + char *disk_opt;
> +
> + rc = split_string_into_pair(disks[i], ":", &disk_opt,
> + &virtio_disk->disks[i].filename);
> + if (rc) {
> + fprintf(stderr, "vdisk: failed to split \"%s\" into pair\n",
> + disks[i]);
> + goto out;
> + }
> +
> + if (!strcmp(disk_opt, "ro"))
> + virtio_disk->disks[i].readonly = 1;
> + else if (!strcmp(disk_opt, "rw"))
> + virtio_disk->disks[i].readonly = 0;
> + else {
> + fprintf(stderr, "vdisk: failed to parse \"%s\" disk option\n",
> + disk_opt);
> + rc = 1;
> + }
> + free(disk_opt);
> +
> + if (rc) goto out;
> + }
> + } else {
> + fprintf(stderr, "Unknown string \"%s\" in vdisk spec\n", token);
> + rc = 1; goto out;
> + }
> +
> + rc = 0;
> +
> +out:
> + libxl_string_list_dispose(&disks);
> + return rc;
> +}
> +
> +static void parse_virtio_disk_list(const XLU_Config *config,
> + libxl_domain_config *d_config)
> +{
> + XLU_ConfigList *virtio_disks;
> + const char *item;
> + char *buf = NULL;
> + int rc;
> +
> + if (!xlu_cfg_get_list (config, "vdisk", &virtio_disks, 0, 0)) {
> + libxl_domain_build_info *b_info = &d_config->b_info;
> + int entry = 0;
> +
> + /* XXX Remove an extra property */
> + libxl_defbool_setdefault(&b_info->arch_arm.virtio, false);
> + if (!libxl_defbool_val(b_info->arch_arm.virtio)) {
> + fprintf(stderr, "Virtio device requires Virtio property to be set\n");
> + exit(EXIT_FAILURE);
> + }
> +
> + while ((item = xlu_cfg_get_listitem(virtio_disks, entry)) != NULL) {
> + libxl_device_virtio_disk *virtio_disk;
> + char *p;
> +
> + virtio_disk = ARRAY_EXTEND_INIT(d_config->virtio_disks,
> + d_config->num_virtio_disks,
> + libxl_device_virtio_disk_init);
> +
> + buf = strdup(item);
> +
> + p = strtok (buf, ",");
> + while (p != NULL)
> + {
> + while (*p == ' ') p++;
> +
> + rc = parse_virtio_disk_config(virtio_disk, p);
> + if (rc) goto out;
> +
> + p = strtok (NULL, ",");
> + }
> +
> + entry++;
> +
> + if (virtio_disk->num_disks == 0) {
> + fprintf(stderr, "At least one virtio disk should be specified\n");
> + rc = 1; goto out;
> + }
> + }
> + }
> +
> + rc = 0;
> +
> +out:
> + free(buf);
> + if (rc) exit(EXIT_FAILURE);
> +}
> +
> void parse_config_data(const char *config_source,
> const char *config_data,
> int config_len,
> @@ -2734,6 +2848,7 @@ skip_usbdev:
> }
>
> parse_vkb_list(config, d_config);
> + parse_virtio_disk_list(config, d_config);
>
> xlu_cfg_get_defbool(config, "xend_suspend_evtchn_compat",
> &c_info->xend_suspend_evtchn_compat, 0);
> diff --git a/tools/xl/xl_virtio_disk.c b/tools/xl/xl_virtio_disk.c
> new file mode 100644
> index 0000000..808a7da
> --- /dev/null
> +++ b/tools/xl/xl_virtio_disk.c
> @@ -0,0 +1,46 @@
> +/*
> + * Copyright (C) 2020 EPAM Systems Inc.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU Lesser General Public License as published
> + * by the Free Software Foundation; version 2.1 only. with the special
> + * exception on linking described in file LICENSE.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU Lesser General Public License for more details.
> + */
> +
> +#include <stdlib.h>
> +
> +#include <libxl.h>
> +#include <libxl_utils.h>
> +#include <libxlutil.h>
> +
> +#include "xl.h"
> +#include "xl_utils.h"
> +#include "xl_parse.h"
> +
> +int main_virtio_diskattach(int argc, char **argv)
> +{
> + return 0;
> +}
> +
> +int main_virtio_disklist(int argc, char **argv)
> +{
> + return 0;
> +}
> +
> +int main_virtio_diskdetach(int argc, char **argv)
> +{
> + return 0;
> +}
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> --
> 2.7.4
>
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 02/23] xen/ioreq: Make x86's IOREQ feature common
2020-10-20 7:57 ` Paul Durrant
@ 2020-11-10 16:45 ` Oleksandr
2020-11-17 14:47 ` Oleksandr
0 siblings, 1 reply; 109+ messages in thread
From: Oleksandr @ 2020-11-10 16:45 UTC (permalink / raw)
To: paul, xen-devel
Cc: 'Oleksandr Tyshchenko', 'Andrew Cooper',
'George Dunlap', 'Ian Jackson',
'Jan Beulich', 'Julien Grall',
'Stefano Stabellini', 'Wei Liu',
'Roger Pau Monné', 'Tim Deegan',
'Julien Grall'
On 20.10.20 10:57, Paul Durrant wrote:
Hi Paul
Sorry for the late response.
>> -----Original Message-----
>> From: Oleksandr Tyshchenko <olekstysh@gmail.com>
>> Sent: 15 October 2020 17:44
>> To: xen-devel@lists.xenproject.org
>> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Andrew Cooper <andrew.cooper3@citrix.com>;
>> George Dunlap <george.dunlap@citrix.com>; Ian Jackson <iwj@xenproject.org>; Jan Beulich
>> <jbeulich@suse.com>; Julien Grall <julien@xen.org>; Stefano Stabellini <sstabellini@kernel.org>; Wei
>> Liu <wl@xen.org>; Roger Pau Monné <roger.pau@citrix.com>; Paul Durrant <paul@xen.org>; Tim Deegan
>> <tim@xen.org>; Julien Grall <julien.grall@arm.com>
>> Subject: [PATCH V2 02/23] xen/ioreq: Make x86's IOREQ feature common
>>
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> As a lot of x86 code can be re-used on Arm later on, this patch
>> moves previously prepared x86/hvm/ioreq.c to the common code.
>>
>> The common IOREQ feature is supposed to be built with IOREQ_SERVER
>> option enabled, which is selected for x86's config HVM for now.
>>
>> In order to avoid having a gigantic patch here, the subsequent
>> patches will update remaining bits in the common code step by step:
>> - Make IOREQ related structs/materials common
>> - Drop the "hvm" prefixes and infixes
> FWIW you could tackle the naming changes in patch #1.
Unfortunately, there are a lot of places that need touching (in order to
replace/drop hvm prefixes), so I decided to keep that in a separate patch.
From the review comments on previous series I got that renaming could
be done before or after the move, but within
a single series, so I chose the latter.
> The 'legacy' mechanism of mapping magic pages for ioreq servers should remain x86 specific I think that aspect of the code needs to remain behind and not get moved into common code. You could do that in arch specific calls in hvm_ioreq_server_enable/disable() and hvm_get_ioreq_server_info().
Well, if legacy mechanism is not going to be used for other arch and
should remain x86 specific, I will try to investigate what should be
left in x86 code and rework the series.
As a side note, I am afraid, we won't get a 100% code movement (which I
managed to achieve here) for the next version of this patch as we need
arch/x86/hvm/ioreq.c.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio()
2020-10-20 10:38 ` Paul Durrant
@ 2020-11-10 19:44 ` Oleksandr
2020-11-11 7:27 ` Jan Beulich
0 siblings, 1 reply; 109+ messages in thread
From: Oleksandr @ 2020-11-10 19:44 UTC (permalink / raw)
To: paul, 'Jan Beulich'
Cc: xen-devel, 'Oleksandr Tyshchenko',
'Andrew Cooper', 'Roger Pau Monné',
'Wei Liu', 'Julien Grall',
'Stefano Stabellini', 'Julien Grall'
On 20.10.20 13:38, Paul Durrant wrote:
Hi Jan, Paul
Sorry for the late response.
>> -----Original Message-----
>> From: Jan Beulich <jbeulich@suse.com>
>> Sent: 20 October 2020 11:05
>> To: paul@xen.org
>> Cc: 'Oleksandr Tyshchenko' <olekstysh@gmail.com>; xen-devel@lists.xenproject.org; 'Oleksandr
>> Tyshchenko' <oleksandr_tyshchenko@epam.com>; 'Andrew Cooper' <andrew.cooper3@citrix.com>; 'Roger Pau
>> Monné' <roger.pau@citrix.com>; 'Wei Liu' <wl@xen.org>; 'Julien Grall' <julien@xen.org>; 'Stefano
>> Stabellini' <sstabellini@kernel.org>; 'Julien Grall' <julien.grall@arm.com>
>> Subject: Re: [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio()
>>
>> On 20.10.2020 11:14, Paul Durrant wrote:
>>>> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of Oleksandr Tyshchenko
>>>> Sent: 15 October 2020 17:44
>>>>
>>>> --- a/xen/include/asm-x86/hvm/ioreq.h
>>>> +++ b/xen/include/asm-x86/hvm/ioreq.h
>>>> @@ -181,6 +181,8 @@ static inline bool arch_hvm_ioreq_destroy(struct domain *d)
>>>> #define IOREQ_STATUS_UNHANDLED X86EMUL_UNHANDLEABLE
>>>> #define IOREQ_STATUS_RETRY X86EMUL_RETRY
>>>>
>>>> +#define ioreq_complete_mmio handle_mmio
>>>> +
>>> A #define? Really? Can we not have a static inline?
>> I guess this would require further shuffling: handle_mmio() is
>> an inline function in hvm/emulate.h, and hvm/ioreq.h has no
>> need to include the former (and imo it also shouldn't have).
>>
> I see. I think we need an x86 ioreq.c anyway, to deal with the legacy use of magic pages, so it could be dealt with there instead.
I am afraid I don't entirely understand the required changes. Could you
please clarify where the "inline(?)" ioreq_complete_mmio() should
live? I included hvm/emulate.h here not for the "handle_mmio()" reason
only, but for "struct hvm_emulate_ctxt" also (see arch_io_completion()).
But, if we return x86 ioreq.c back I can move arch_io_completion() to it
as well as "non-online" ioreq_complete_mmio().
This will avoid including hvm/emulate.h here. Or I missed something?
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 08/23] xen/ioreq: Introduce ioreq_params to abstract accesses to arch.hvm.params
2020-10-20 10:41 ` Paul Durrant
@ 2020-11-10 20:00 ` Oleksandr
0 siblings, 0 replies; 109+ messages in thread
From: Oleksandr @ 2020-11-10 20:00 UTC (permalink / raw)
To: paul, xen-devel
Cc: 'Oleksandr Tyshchenko', 'Jan Beulich',
'Andrew Cooper', 'Roger Pau Monné',
'Wei Liu', 'Julien Grall',
'Stefano Stabellini', 'Julien Grall'
On 20.10.20 13:41, Paul Durrant wrote:
Hi Paul
Sorry for the late response.
>> -----Original Message-----
>> From: Oleksandr Tyshchenko <olekstysh@gmail.com>
>> Sent: 15 October 2020 17:44
>> To: xen-devel@lists.xenproject.org
>> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Paul Durrant <paul@xen.org>; Jan Beulich
>> <jbeulich@suse.com>; Andrew Cooper <andrew.cooper3@citrix.com>; Roger Pau Monné
>> <roger.pau@citrix.com>; Wei Liu <wl@xen.org>; Julien Grall <julien@xen.org>; Stefano Stabellini
>> <sstabellini@kernel.org>; Julien Grall <julien.grall@arm.com>
>> Subject: [PATCH V2 08/23] xen/ioreq: Introduce ioreq_params to abstract accesses to arch.hvm.params
>>
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> We don't want to move HVM params field out of *arch.hvm* in this particular
>> case as although it stores a few IOREQ params, it is not a (completely)
>> IOREQ stuff and is specific to the architecture. Instead, abstract
>> accesses by the proposed macro.
>>
>> This is a follow up action to reduce layering violation in the common code.
>>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> CC: Julien Grall <julien.grall@arm.com>
>>
> Keeping the 'legacy' magic page code under an x86 ioreq.c would avoid the need for this patch.
In that case, yes, agree.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server()
2020-10-20 10:51 ` Paul Durrant
@ 2020-11-10 20:53 ` Oleksandr
2020-11-11 8:08 ` Jan Beulich
0 siblings, 1 reply; 109+ messages in thread
From: Oleksandr @ 2020-11-10 20:53 UTC (permalink / raw)
To: paul, xen-devel
Cc: 'Oleksandr Tyshchenko', 'Stefano Stabellini',
'Julien Grall', 'Volodymyr Babchuk',
'Andrew Cooper', 'George Dunlap',
'Ian Jackson', 'Jan Beulich', 'Wei Liu',
'Julien Grall'
On 20.10.20 13:51, Paul Durrant wrote:
Hi Paul.
Sorry for the late response.
>
>> -----Original Message-----
>> From: Oleksandr Tyshchenko <olekstysh@gmail.com>
>> Sent: 15 October 2020 17:44
>> To: xen-devel@lists.xenproject.org
>> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Stefano Stabellini <sstabellini@kernel.org>;
>> Julien Grall <julien@xen.org>; Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>; Andrew Cooper
>> <andrew.cooper3@citrix.com>; George Dunlap <george.dunlap@citrix.com>; Ian Jackson
>> <iwj@xenproject.org>; Jan Beulich <jbeulich@suse.com>; Wei Liu <wl@xen.org>; Paul Durrant
>> <paul@xen.org>; Julien Grall <julien.grall@arm.com>
>> Subject: [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server()
>>
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> This patch introduces a helper the main purpose of which is to check
>> if a domain is using IOREQ server(s).
>>
>> On Arm the current benefit is to avoid calling handle_io_completion()
>> (which implies iterating over all possible IOREQ servers anyway)
>> on every return in leave_hypervisor_to_guest() if there is no active
>> servers for the particular domain.
>> Also this helper will be used by one of the subsequent patches on Arm.
>>
>> This involves adding an extra per-domain variable to store the count
>> of servers in use.
>>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> CC: Julien Grall <julien.grall@arm.com>
>>
>> ---
>> Please note, this is a split/cleanup/hardening of Julien's PoC:
>> "Add support for Guest IO forwarding to a device emulator"
>>
>> Changes RFC -> V1:
>> - new patch
>>
>> Changes V1 -> V2:
>> - update patch description
>> - guard helper with CONFIG_IOREQ_SERVER
>> - remove "hvm" prefix
>> - modify helper to just return d->arch.hvm.ioreq_server.nr_servers
>> - put suitable ASSERT()s
>> - use ASSERT(d->ioreq_server.server[id] ? !s : !!s) in set_ioreq_server()
>> - remove d->ioreq_server.nr_servers = 0 from hvm_ioreq_init()
>> ---
>> xen/arch/arm/traps.c | 15 +++++++++------
>> xen/common/ioreq.c | 7 ++++++-
>> xen/include/xen/ioreq.h | 14 ++++++++++++++
>> xen/include/xen/sched.h | 1 +
>> 4 files changed, 30 insertions(+), 7 deletions(-)
>>
>> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
>> index 507c095..a8f5fdf 100644
>> --- a/xen/arch/arm/traps.c
>> +++ b/xen/arch/arm/traps.c
>> @@ -2261,14 +2261,17 @@ static bool check_for_vcpu_work(void)
>> struct vcpu *v = current;
>>
>> #ifdef CONFIG_IOREQ_SERVER
>> - bool handled;
>> + if ( domain_has_ioreq_server(v->domain) )
>> + {
>> + bool handled;
>>
>> - local_irq_enable();
>> - handled = handle_io_completion(v);
>> - local_irq_disable();
>> + local_irq_enable();
>> + handled = handle_io_completion(v);
>> + local_irq_disable();
>>
>> - if ( !handled )
>> - return true;
>> + if ( !handled )
>> + return true;
>> + }
>> #endif
>>
>> if ( likely(!v->arch.need_flush_to_ram) )
>> diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
>> index bcd4961..a72bc0e 100644
>> --- a/xen/common/ioreq.c
>> +++ b/xen/common/ioreq.c
>> @@ -39,9 +39,14 @@ static void set_ioreq_server(struct domain *d, unsigned int id,
>> struct ioreq_server *s)
>> {
>> ASSERT(id < MAX_NR_IOREQ_SERVERS);
>> - ASSERT(!s || !d->ioreq_server.server[id]);
>> + ASSERT(d->ioreq_server.server[id] ? !s : !!s);
> That looks odd. How about ASSERT(!s ^ !d->ioreq_server.server[id])?
ok, looks like it will work.
> Paul
>
>> d->ioreq_server.server[id] = s;
>> +
>> + if ( s )
>> + d->ioreq_server.nr_servers++;
>> + else
>> + d->ioreq_server.nr_servers--;
>> }
>>
>> #define GET_IOREQ_SERVER(d, id) \
>> diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
>> index 7b03ab5..0679fef 100644
>> --- a/xen/include/xen/ioreq.h
>> +++ b/xen/include/xen/ioreq.h
>> @@ -55,6 +55,20 @@ struct ioreq_server {
>> uint8_t bufioreq_handling;
>> };
>>
>> +#ifdef CONFIG_IOREQ_SERVER
>> +static inline bool domain_has_ioreq_server(const struct domain *d)
>> +{
>> + ASSERT((current->domain == d) || atomic_read(&d->pause_count));
>> +
> This seems like an odd place to put such an assertion.
I might miss something or interpreted incorrectly but these asserts are
the result of how I understood the review comment on previous version [1].
I will copy a comment here for the convenience:
"This is safe only when d == current->domain and it's not paused,
or when they're distinct and d is paused. Otherwise the result is
stale before the caller can inspect it. This wants documenting by
at least a comment, but perhaps better by suitable ASSERT()s."
>
>> + return d->ioreq_server.nr_servers;
>> +}
>> +#else
>> +static inline bool domain_has_ioreq_server(const struct domain *d)
>> +{
>> + return false;
>> +}
>> +#endif
>> +
> Can this be any more compact? E.g.
>
> return IS_ENABLED(CONFIG_IOREQ_SERVER) && d->ioreq_server.nr_servers;
>
> ?
I have got a compilation error this way (if CONFIG_IOREQ_SERVER is
disabled):
...xen/4.14.0+gitAUTOINC+ee22110219-r0/git/xen/include/xen/ioreq.h:62:48:
error: ‘const struct domain’ has no member named ‘ioreq_server’
return IS_ENABLED(CONFIG_IOREQ_SERVER) && d->ioreq_server.nr_servers;
^
as domain's ioreq_server struct is guarded by CONFIG_IOREQ_SERVER as well.
[1]
https://patchwork.kernel.org/project/xen-devel/patch/1599769330-17656-12-git-send-email-olekstysh@gmail.com/#23618623
Thank you.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 11/23] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
2020-10-20 10:55 ` Paul Durrant
@ 2020-11-10 20:59 ` Oleksandr
0 siblings, 0 replies; 109+ messages in thread
From: Oleksandr @ 2020-11-10 20:59 UTC (permalink / raw)
To: paul, xen-devel
Cc: 'Oleksandr Tyshchenko', 'Jan Beulich',
'Andrew Cooper', 'Roger Pau Monné',
'Wei Liu', 'George Dunlap', 'Ian Jackson',
'Julien Grall', 'Stefano Stabellini',
'Jun Nakajima', 'Kevin Tian',
'Julien Grall'
On 20.10.20 13:55, Paul Durrant wrote:
Hi Paul.
Sorry for the late response.
>> -----Original Message-----
>> From: Oleksandr Tyshchenko <olekstysh@gmail.com>
>> Sent: 15 October 2020 17:44
>> To: xen-devel@lists.xenproject.org
>> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Paul Durrant <paul@xen.org>; Jan Beulich
>> <jbeulich@suse.com>; Andrew Cooper <andrew.cooper3@citrix.com>; Roger Pau Monné
>> <roger.pau@citrix.com>; Wei Liu <wl@xen.org>; George Dunlap <george.dunlap@citrix.com>; Ian Jackson
>> <iwj@xenproject.org>; Julien Grall <julien@xen.org>; Stefano Stabellini <sstabellini@kernel.org>; Jun
>> Nakajima <jun.nakajima@intel.com>; Kevin Tian <kevin.tian@intel.com>; Julien Grall
>> <julien.grall@arm.com>
>> Subject: [PATCH V2 11/23] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
>>
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> The IOREQ is a common feature now and these fields will be used
>> on Arm as is. Move them to common struct vcpu as a part of new
>> struct vcpu_io. Also move enum hvm_io_completion to xen/sched.h
>> and remove "hvm" prefixes.
>>
>> This patch completely removes layering violation in the common code.
>>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> CC: Julien Grall <julien.grall@arm.com>
>>
>> ---
>> Please note, this is a split/cleanup/hardening of Julien's PoC:
>> "Add support for Guest IO forwarding to a device emulator"
>>
>> ***
>> I was thinking that it may be better to place these two fields
>> into struct vcpu directly (without intermediate "io" struct).
>> I think, this way the code which operates with these fields
>> would become cleaner. Another possible option would be either
>> to rename "io" struct (I failed to think of a better name) or
>> to drop(replace?) duplicating "io" prefixes from these fields.
> Just drop the 'io_' prefix from the field names.
Will drop. This would look like indeed better.
Thank you.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 21/23] xen/arm: Add mapcache invalidation handling
2020-10-16 8:41 ` Julien Grall
2020-10-16 8:56 ` Jan Beulich
@ 2020-11-11 0:03 ` Oleksandr
2020-11-11 19:27 ` Julien Grall
1 sibling, 1 reply; 109+ messages in thread
From: Oleksandr @ 2020-11-11 0:03 UTC (permalink / raw)
To: Julien Grall, Jan Beulich
Cc: xen-devel, Oleksandr Tyshchenko, Stefano Stabellini,
Volodymyr Babchuk, Julien Grall
On 16.10.20 11:41, Julien Grall wrote:
Hi Jan, Julien
Sorry for the late response.
> Hi Jan,
>
> On 16/10/2020 07:29, Jan Beulich wrote:
>> On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
>>> @@ -1067,7 +1068,14 @@ static int __p2m_set_entry(struct p2m_domain
>>> *p2m,
>>> */
>>> if ( p2m_is_valid(orig_pte) &&
>>> !mfn_eq(lpae_get_mfn(*entry), lpae_get_mfn(orig_pte)) )
>>> + {
>>> +#ifdef CONFIG_IOREQ_SERVER
>>> + if ( domain_has_ioreq_server(p2m->domain) &&
>>> + (p2m->domain == current->domain) &&
>>> p2m_is_ram(orig_pte.p2m.type) )
>>> + p2m->domain->qemu_mapcache_invalidate = true;
>>> +#endif
>>> p2m_free_entry(p2m, orig_pte, level);
>>> + }
>>
>> For all I have to say here, please bear in mind that I don't know
>> the internals of Arm memory management.
>>
>> The first odd thing here the merely MFN-based condition. It may
>> well be that's sufficient, if there's no way to get a "not present"
>> entry with an MFN matching any valid MFN. (This isn't just with
>> your addition, but even before.
> Invalid entries are always zeroed. So in theory the problem could
> arise if MFN 0 used in the guest. It should not be possible on
> staging, but I agree this should be fixed.
>
>>
>> Given how p2m_free_entry() works (or is supposed to work in the
>> long run), is the new code you add guaranteed to only alter leaf
>> entries?
>
> This path may also be called with tables. I think we want to move the
> check in p2m_free_entry() so we can find the correct leaf type.
Well, but inside p2m_free_entry() we don't have a new entry in order to
check whether new MFN is actually different. An extra arg only comes to
mind...
[Didn't update call sites yet and didn't tested]
diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index 5bb23df..4001f46 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -739,7 +739,7 @@ static void p2m_put_l3_page(const lpae_t pte)
/* Free lpae sub-tree behind an entry */
static void p2m_free_entry(struct p2m_domain *p2m,
- lpae_t entry, unsigned int level)
+ lpae_t entry, lpae_t new_entry, unsigned int
level)
{
unsigned int i;
lpae_t *table;
@@ -750,17 +750,19 @@ static void p2m_free_entry(struct p2m_domain *p2m,
if ( !p2m_is_valid(entry) )
return;
- /* Nothing to do but updating the stats if the entry is a
super-page. */
- if ( p2m_is_superpage(entry, level) )
+ if ( p2m_is_superpage(entry, level) || (level == 3) )
{
- p2m->stats.mappings[level]--;
- return;
- }
+#ifdef CONFIG_IOREQ_SERVER
+ if ( domain_has_ioreq_server(p2m->domain) &&
+ (p2m->domain == current->domain) &&
+ !mfn_eq(lpae_get_mfn(new_entry), lpae_get_mfn(entry)) &&
+ p2m_is_ram(entry.p2m.type) )
+ p2m->domain->qemu_mapcache_invalidate = true;
+#endif
- if ( level == 3 )
- {
p2m->stats.mappings[level]--;
- p2m_put_l3_page(entry);
+ if ( level == 3 )
+ p2m_put_l3_page(entry);
return;
}
(END)
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply related [flat|nested] 109+ messages in thread
* Re: [PATCH V2 23/23] [RFC] libxl: Add support for virtio-disk configuration
2020-11-09 6:45 ` Wei Chen
@ 2020-11-11 0:53 ` Oleksandr
2020-11-11 4:28 ` Wei Chen
0 siblings, 1 reply; 109+ messages in thread
From: Oleksandr @ 2020-11-11 0:53 UTC (permalink / raw)
To: Wei Chen, xen-devel
Cc: Oleksandr Tyshchenko, Ian Jackson, Wei Liu, Anthony PERARD,
Julien Grall, Stefano Stabellini
On 09.11.20 08:45, Wei Chen wrote:
> Hi Oleksandr,
Hi Wei
>
>> -----Original Message-----
>> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of
>> Oleksandr Tyshchenko
>> Sent: 2020年10月16日 0:45
>> To: xen-devel@lists.xenproject.org
>> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Ian Jackson
>> <iwj@xenproject.org>; Wei Liu <wl@xen.org>; Anthony PERARD
>> <anthony.perard@citrix.com>; Julien Grall <julien@xen.org>; Stefano Stabellini
>> <sstabellini@kernel.org>
>> Subject: [PATCH V2 23/23] [RFC] libxl: Add support for virtio-disk configuration
>>
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> This patch adds basic support for configuring and assisting virtio-disk
>> backend (emualator) which is intended to run out of Qemu and could be run
>> in any domain.
>>
>> Xenstore was chosen as a communication interface for the emulator running
>> in non-toolstack domain to be able to get configuration either by reading
>> Xenstore directly or by receiving command line parameters (an updated 'xl devd'
>> running in the same domain would read Xenstore beforehand and call backend
>> executable with the required arguments).
>>
>> An example of domain configuration (two disks are assigned to the guest,
>> the latter is in readonly mode):
>>
>> vdisk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3;ro:/dev/mmcblk1p3' ]
>>
> Can we keep use the same 'disk' parameter for virtio-disk, but add an option like
> "model=virtio-disk"?
> For example:
> disk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3,model=virtio-disk' ]
> Just like what Xen has done for x86 virtio-net.
I think, this needs an additional investigation. In general I agree with
you that it would be nice to reuse existing 'disk' parameter somehow
rather than introducing new one
for the same purpose (to handle virtual block device(s)).
One note, although both are used for the same purpose they are different
in at least one important option.
For example:
1. disk = [ 'backend=DomD, phy:/dev/mmcblk0p3, xvda1' ]
2. vdisk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3' ]
As you can see existing "disk" parameter contains xvda1, this means that
a new device /dev/xvda1 will appear at the guest side, but "vdisk"
doesn't contain anything similar. So with Xen PV driver (xen-blkfront)
we are able to configure a device name, but with VirtIO solution
(virtio-blk) we aren't (at least I don't know how exactly).
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [PATCH V2 23/23] [RFC] libxl: Add support for virtio-disk configuration
2020-11-11 0:53 ` Oleksandr
@ 2020-11-11 4:28 ` Wei Chen
2020-11-13 17:38 ` Oleksandr
0 siblings, 1 reply; 109+ messages in thread
From: Wei Chen @ 2020-11-11 4:28 UTC (permalink / raw)
To: Oleksandr, xen-devel
Cc: Oleksandr Tyshchenko, Ian Jackson, Wei Liu, Anthony PERARD,
Julien Grall, Stefano Stabellini
Hi Oleksandr,
> -----Original Message-----
> From: Oleksandr <olekstysh@gmail.com>
> Sent: 2020年11月11日 8:53
> To: Wei Chen <Wei.Chen@arm.com>; xen-devel@lists.xenproject.org
> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Ian Jackson
> <iwj@xenproject.org>; Wei Liu <wl@xen.org>; Anthony PERARD
> <anthony.perard@citrix.com>; Julien Grall <julien@xen.org>; Stefano Stabellini
> <sstabellini@kernel.org>
> Subject: Re: [PATCH V2 23/23] [RFC] libxl: Add support for virtio-disk
> configuration
>
>
> On 09.11.20 08:45, Wei Chen wrote:
> > Hi Oleksandr,
>
> Hi Wei
>
>
> >
> >> -----Original Message-----
> >> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of
> >> Oleksandr Tyshchenko
> >> Sent: 2020年10月16日 0:45
> >> To: xen-devel@lists.xenproject.org
> >> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Ian Jackson
> >> <iwj@xenproject.org>; Wei Liu <wl@xen.org>; Anthony PERARD
> >> <anthony.perard@citrix.com>; Julien Grall <julien@xen.org>; Stefano
> Stabellini
> >> <sstabellini@kernel.org>
> >> Subject: [PATCH V2 23/23] [RFC] libxl: Add support for virtio-disk
> configuration
> >>
> >> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> >>
> >> This patch adds basic support for configuring and assisting virtio-disk
> >> backend (emualator) which is intended to run out of Qemu and could be run
> >> in any domain.
> >>
> >> Xenstore was chosen as a communication interface for the emulator running
> >> in non-toolstack domain to be able to get configuration either by reading
> >> Xenstore directly or by receiving command line parameters (an updated 'xl
> devd'
> >> running in the same domain would read Xenstore beforehand and call
> backend
> >> executable with the required arguments).
> >>
> >> An example of domain configuration (two disks are assigned to the guest,
> >> the latter is in readonly mode):
> >>
> >> vdisk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3;ro:/dev/mmcblk1p3' ]
> >>
> > Can we keep use the same 'disk' parameter for virtio-disk, but add an option
> like
> > "model=virtio-disk"?
> > For example:
> > disk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3,model=virtio-disk' ]
> > Just like what Xen has done for x86 virtio-net.
>
> I think, this needs an additional investigation. In general I agree with
> you that it would be nice to reuse existing 'disk' parameter somehow
> rather than introducing new one
> for the same purpose (to handle virtual block device(s)).
>
>
> One note, although both are used for the same purpose they are different
> in at least one important option.
>
> For example:
> 1. disk = [ 'backend=DomD, phy:/dev/mmcblk0p3, xvda1' ]
> 2. vdisk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3' ]
> As you can see existing "disk" parameter contains xvda1, this means that
> a new device /dev/xvda1 will appear at the guest side, but "vdisk"
> doesn't contain anything similar. So with Xen PV driver (xen-blkfront)
Yes, I understand your concerns. But I think specifying a device name
for virtio disk is not a mandatory requirement. Even if we're using physical
disks on bare metal machine, we can't guarantee slot#1 disk is always 'sda'.
So most modern OS are prefer to use blkid to mount filesystem.
> we are able to configure a device name, but with VirtIO solution
> (virtio-blk) we aren't (at least I don't know how exactly).
>
Just my understanding, not exactly accurate:
The virtio-blk could not get VDEV information for a bus like Xen-bus. So the disk
ID is allocated dynamically in bus probe progress. The first probed disk will get
ID 'a'. And then the ID keeps increasing. If we want to specify the disk ID for virtio
disk, one possible way to do this is to construct a reasonable position on bus
(fdt node position of virtio mmio device, PCI Function ID of virtio pci block) for
virtio disk. But it is not always successful, we can't skip 'vda' to specify a virtio
disk as 'vdb'.
Regards,
Wei Chen
>
>
>
>
> --
> Regards,
>
> Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio()
2020-11-10 19:44 ` Oleksandr
@ 2020-11-11 7:27 ` Jan Beulich
2020-11-11 8:09 ` Oleksandr
0 siblings, 1 reply; 109+ messages in thread
From: Jan Beulich @ 2020-11-11 7:27 UTC (permalink / raw)
To: Oleksandr
Cc: xen-devel, 'Oleksandr Tyshchenko',
'Andrew Cooper', 'Roger Pau Monné',
'Wei Liu', 'Julien Grall',
'Stefano Stabellini', 'Julien Grall',
paul
On 10.11.2020 20:44, Oleksandr wrote:
>
> On 20.10.20 13:38, Paul Durrant wrote:
>
> Hi Jan, Paul
>
> Sorry for the late response.
>
>>> -----Original Message-----
>>> From: Jan Beulich <jbeulich@suse.com>
>>> Sent: 20 October 2020 11:05
>>> To: paul@xen.org
>>> Cc: 'Oleksandr Tyshchenko' <olekstysh@gmail.com>; xen-devel@lists.xenproject.org; 'Oleksandr
>>> Tyshchenko' <oleksandr_tyshchenko@epam.com>; 'Andrew Cooper' <andrew.cooper3@citrix.com>; 'Roger Pau
>>> Monné' <roger.pau@citrix.com>; 'Wei Liu' <wl@xen.org>; 'Julien Grall' <julien@xen.org>; 'Stefano
>>> Stabellini' <sstabellini@kernel.org>; 'Julien Grall' <julien.grall@arm.com>
>>> Subject: Re: [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio()
>>>
>>> On 20.10.2020 11:14, Paul Durrant wrote:
>>>>> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of Oleksandr Tyshchenko
>>>>> Sent: 15 October 2020 17:44
>>>>>
>>>>> --- a/xen/include/asm-x86/hvm/ioreq.h
>>>>> +++ b/xen/include/asm-x86/hvm/ioreq.h
>>>>> @@ -181,6 +181,8 @@ static inline bool arch_hvm_ioreq_destroy(struct domain *d)
>>>>> #define IOREQ_STATUS_UNHANDLED X86EMUL_UNHANDLEABLE
>>>>> #define IOREQ_STATUS_RETRY X86EMUL_RETRY
>>>>>
>>>>> +#define ioreq_complete_mmio handle_mmio
>>>>> +
>>>> A #define? Really? Can we not have a static inline?
>>> I guess this would require further shuffling: handle_mmio() is
>>> an inline function in hvm/emulate.h, and hvm/ioreq.h has no
>>> need to include the former (and imo it also shouldn't have).
>>>
>> I see. I think we need an x86 ioreq.c anyway, to deal with the legacy use of magic pages, so it could be dealt with there instead.
> I am afraid I don't entirely understand the required changes. Could you
> please clarify where the "inline(?)" ioreq_complete_mmio() should
> live? I included hvm/emulate.h here not for the "handle_mmio()" reason
> only, but for "struct hvm_emulate_ctxt" also (see arch_io_completion()).
I'm sorry, but in the context of this patch there's no use of any
struct hvm_emulate_ctxt instance. I'm not going to wade through 23
patches to find what you mean.
> But, if we return x86 ioreq.c back I can move arch_io_completion() to it
> as well as "non-online" ioreq_complete_mmio().
> This will avoid including hvm/emulate.h here. Or I missed something?
I suppose an out-of-line function as kind of a last resort solution
is what Paul had in mind. To be honest I'd prefer to avoid the
extra call layer though, if possible.
Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 11/23] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
2020-10-15 16:44 ` [PATCH V2 11/23] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu Oleksandr Tyshchenko
2020-10-20 10:55 ` Paul Durrant
@ 2020-11-11 8:04 ` Jan Beulich
2020-11-11 8:14 ` Oleksandr
1 sibling, 1 reply; 109+ messages in thread
From: Jan Beulich @ 2020-11-11 8:04 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: Oleksandr Tyshchenko, Paul Durrant, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Julien Grall,
xen-devel
On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
> --- a/xen/include/xen/sched.h
> +++ b/xen/include/xen/sched.h
> @@ -143,6 +143,19 @@ void evtchn_destroy_final(struct domain *d); /* from complete_domain_destroy */
>
> struct waitqueue_vcpu;
>
> +enum io_completion {
> + IO_no_completion,
> + IO_mmio_completion,
> + IO_pio_completion,
> + IO_realmode_completion
> +};
May I suggest wrapping at least the last one in #ifdef CONFIG_X86?
Also please add a trailing comma.
Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server()
2020-11-10 20:53 ` Oleksandr
@ 2020-11-11 8:08 ` Jan Beulich
2020-11-11 8:41 ` Oleksandr
0 siblings, 1 reply; 109+ messages in thread
From: Jan Beulich @ 2020-11-11 8:08 UTC (permalink / raw)
To: Oleksandr
Cc: 'Oleksandr Tyshchenko', 'Stefano Stabellini',
'Julien Grall', 'Volodymyr Babchuk',
'Andrew Cooper', 'George Dunlap',
'Ian Jackson', 'Wei Liu', 'Julien Grall',
paul, xen-devel
On 10.11.2020 21:53, Oleksandr wrote:
>
> On 20.10.20 13:51, Paul Durrant wrote:
>
> Hi Paul.
>
> Sorry for the late response.
>
>>
>>> -----Original Message-----
>>> From: Oleksandr Tyshchenko <olekstysh@gmail.com>
>>> Sent: 15 October 2020 17:44
>>> To: xen-devel@lists.xenproject.org
>>> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Stefano Stabellini <sstabellini@kernel.org>;
>>> Julien Grall <julien@xen.org>; Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>; Andrew Cooper
>>> <andrew.cooper3@citrix.com>; George Dunlap <george.dunlap@citrix.com>; Ian Jackson
>>> <iwj@xenproject.org>; Jan Beulich <jbeulich@suse.com>; Wei Liu <wl@xen.org>; Paul Durrant
>>> <paul@xen.org>; Julien Grall <julien.grall@arm.com>
>>> Subject: [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server()
>>>
>>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>
>>> This patch introduces a helper the main purpose of which is to check
>>> if a domain is using IOREQ server(s).
>>>
>>> On Arm the current benefit is to avoid calling handle_io_completion()
>>> (which implies iterating over all possible IOREQ servers anyway)
>>> on every return in leave_hypervisor_to_guest() if there is no active
>>> servers for the particular domain.
>>> Also this helper will be used by one of the subsequent patches on Arm.
>>>
>>> This involves adding an extra per-domain variable to store the count
>>> of servers in use.
>>>
>>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>> CC: Julien Grall <julien.grall@arm.com>
>>>
>>> ---
>>> Please note, this is a split/cleanup/hardening of Julien's PoC:
>>> "Add support for Guest IO forwarding to a device emulator"
>>>
>>> Changes RFC -> V1:
>>> - new patch
>>>
>>> Changes V1 -> V2:
>>> - update patch description
>>> - guard helper with CONFIG_IOREQ_SERVER
>>> - remove "hvm" prefix
>>> - modify helper to just return d->arch.hvm.ioreq_server.nr_servers
>>> - put suitable ASSERT()s
>>> - use ASSERT(d->ioreq_server.server[id] ? !s : !!s) in set_ioreq_server()
>>> - remove d->ioreq_server.nr_servers = 0 from hvm_ioreq_init()
>>> ---
>>> xen/arch/arm/traps.c | 15 +++++++++------
>>> xen/common/ioreq.c | 7 ++++++-
>>> xen/include/xen/ioreq.h | 14 ++++++++++++++
>>> xen/include/xen/sched.h | 1 +
>>> 4 files changed, 30 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
>>> index 507c095..a8f5fdf 100644
>>> --- a/xen/arch/arm/traps.c
>>> +++ b/xen/arch/arm/traps.c
>>> @@ -2261,14 +2261,17 @@ static bool check_for_vcpu_work(void)
>>> struct vcpu *v = current;
>>>
>>> #ifdef CONFIG_IOREQ_SERVER
>>> - bool handled;
>>> + if ( domain_has_ioreq_server(v->domain) )
>>> + {
>>> + bool handled;
>>>
>>> - local_irq_enable();
>>> - handled = handle_io_completion(v);
>>> - local_irq_disable();
>>> + local_irq_enable();
>>> + handled = handle_io_completion(v);
>>> + local_irq_disable();
>>>
>>> - if ( !handled )
>>> - return true;
>>> + if ( !handled )
>>> + return true;
>>> + }
>>> #endif
>>>
>>> if ( likely(!v->arch.need_flush_to_ram) )
>>> diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
>>> index bcd4961..a72bc0e 100644
>>> --- a/xen/common/ioreq.c
>>> +++ b/xen/common/ioreq.c
>>> @@ -39,9 +39,14 @@ static void set_ioreq_server(struct domain *d, unsigned int id,
>>> struct ioreq_server *s)
>>> {
>>> ASSERT(id < MAX_NR_IOREQ_SERVERS);
>>> - ASSERT(!s || !d->ioreq_server.server[id]);
>>> + ASSERT(d->ioreq_server.server[id] ? !s : !!s);
>> That looks odd. How about ASSERT(!s ^ !d->ioreq_server.server[id])?
>
> ok, looks like it will work.
>
>
>> Paul
>>
>>> d->ioreq_server.server[id] = s;
>>> +
>>> + if ( s )
>>> + d->ioreq_server.nr_servers++;
>>> + else
>>> + d->ioreq_server.nr_servers--;
>>> }
>>>
>>> #define GET_IOREQ_SERVER(d, id) \
>>> diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
>>> index 7b03ab5..0679fef 100644
>>> --- a/xen/include/xen/ioreq.h
>>> +++ b/xen/include/xen/ioreq.h
>>> @@ -55,6 +55,20 @@ struct ioreq_server {
>>> uint8_t bufioreq_handling;
>>> };
>>>
>>> +#ifdef CONFIG_IOREQ_SERVER
>>> +static inline bool domain_has_ioreq_server(const struct domain *d)
>>> +{
>>> + ASSERT((current->domain == d) || atomic_read(&d->pause_count));
>>> +
>> This seems like an odd place to put such an assertion.
>
> I might miss something or interpreted incorrectly but these asserts are
> the result of how I understood the review comment on previous version [1].
>
> I will copy a comment here for the convenience:
> "This is safe only when d == current->domain and it's not paused,
> or when they're distinct and d is paused. Otherwise the result is
> stale before the caller can inspect it. This wants documenting by
> at least a comment, but perhaps better by suitable ASSERT()s."
The way his reply was worded, I think Paul was wondering about the
place where you put the assertion, not what you actually assert.
>>> + return d->ioreq_server.nr_servers;
>>> +}
>>> +#else
>>> +static inline bool domain_has_ioreq_server(const struct domain *d)
>>> +{
>>> + return false;
>>> +}
>>> +#endif
>>> +
>> Can this be any more compact? E.g.
>>
>> return IS_ENABLED(CONFIG_IOREQ_SERVER) && d->ioreq_server.nr_servers;
>>
>> ?
> I have got a compilation error this way (if CONFIG_IOREQ_SERVER is
> disabled):
>
> ...xen/4.14.0+gitAUTOINC+ee22110219-r0/git/xen/include/xen/ioreq.h:62:48:
> error: ‘const struct domain’ has no member named ‘ioreq_server’
> return IS_ENABLED(CONFIG_IOREQ_SERVER) && d->ioreq_server.nr_servers;
> ^
> as domain's ioreq_server struct is guarded by CONFIG_IOREQ_SERVER as well.
The #ifdef is unavoidable here, I agree, but it should be inside
the function's body. There's no need to duplicate the rest of it.
Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio()
2020-11-11 7:27 ` Jan Beulich
@ 2020-11-11 8:09 ` Oleksandr
2020-11-11 8:16 ` Jan Beulich
0 siblings, 1 reply; 109+ messages in thread
From: Oleksandr @ 2020-11-11 8:09 UTC (permalink / raw)
To: Jan Beulich
Cc: xen-devel, 'Oleksandr Tyshchenko',
'Andrew Cooper', 'Roger Pau Monné',
'Wei Liu', 'Julien Grall',
'Stefano Stabellini', 'Julien Grall',
paul
On 11.11.20 09:27, Jan Beulich wrote:
Hi Jan
> On 10.11.2020 20:44, Oleksandr wrote:
>> On 20.10.20 13:38, Paul Durrant wrote:
>>
>> Hi Jan, Paul
>>
>> Sorry for the late response.
>>
>>>> -----Original Message-----
>>>> From: Jan Beulich <jbeulich@suse.com>
>>>> Sent: 20 October 2020 11:05
>>>> To: paul@xen.org
>>>> Cc: 'Oleksandr Tyshchenko' <olekstysh@gmail.com>; xen-devel@lists.xenproject.org; 'Oleksandr
>>>> Tyshchenko' <oleksandr_tyshchenko@epam.com>; 'Andrew Cooper' <andrew.cooper3@citrix.com>; 'Roger Pau
>>>> Monné' <roger.pau@citrix.com>; 'Wei Liu' <wl@xen.org>; 'Julien Grall' <julien@xen.org>; 'Stefano
>>>> Stabellini' <sstabellini@kernel.org>; 'Julien Grall' <julien.grall@arm.com>
>>>> Subject: Re: [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio()
>>>>
>>>> On 20.10.2020 11:14, Paul Durrant wrote:
>>>>>> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of Oleksandr Tyshchenko
>>>>>> Sent: 15 October 2020 17:44
>>>>>>
>>>>>> --- a/xen/include/asm-x86/hvm/ioreq.h
>>>>>> +++ b/xen/include/asm-x86/hvm/ioreq.h
>>>>>> @@ -181,6 +181,8 @@ static inline bool arch_hvm_ioreq_destroy(struct domain *d)
>>>>>> #define IOREQ_STATUS_UNHANDLED X86EMUL_UNHANDLEABLE
>>>>>> #define IOREQ_STATUS_RETRY X86EMUL_RETRY
>>>>>>
>>>>>> +#define ioreq_complete_mmio handle_mmio
>>>>>> +
>>>>> A #define? Really? Can we not have a static inline?
>>>> I guess this would require further shuffling: handle_mmio() is
>>>> an inline function in hvm/emulate.h, and hvm/ioreq.h has no
>>>> need to include the former (and imo it also shouldn't have).
>>>>
>>> I see. I think we need an x86 ioreq.c anyway, to deal with the legacy use of magic pages, so it could be dealt with there instead.
>> I am afraid I don't entirely understand the required changes. Could you
>> please clarify where the "inline(?)" ioreq_complete_mmio() should
>> live? I included hvm/emulate.h here not for the "handle_mmio()" reason
>> only, but for "struct hvm_emulate_ctxt" also (see arch_io_completion()).
> I'm sorry, but in the context of this patch there's no use of any
> struct hvm_emulate_ctxt instance. I'm not going to wade through 23
> patches to find what you mean.
Sorry for not being precise here. I meant arch_io_completion() added at [1]
[1]
https://patchwork.kernel.org/project/xen-devel/patch/1602780274-29141-2-git-send-email-olekstysh@gmail.com/
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 11/23] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu
2020-11-11 8:04 ` Jan Beulich
@ 2020-11-11 8:14 ` Oleksandr
0 siblings, 0 replies; 109+ messages in thread
From: Oleksandr @ 2020-11-11 8:14 UTC (permalink / raw)
To: Jan Beulich
Cc: Oleksandr Tyshchenko, Paul Durrant, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Julien Grall,
xen-devel
On 11.11.20 10:04, Jan Beulich wrote:
Hi Jan
> On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
>> --- a/xen/include/xen/sched.h
>> +++ b/xen/include/xen/sched.h
>> @@ -143,6 +143,19 @@ void evtchn_destroy_final(struct domain *d); /* from complete_domain_destroy */
>>
>> struct waitqueue_vcpu;
>>
>> +enum io_completion {
>> + IO_no_completion,
>> + IO_mmio_completion,
>> + IO_pio_completion,
>> + IO_realmode_completion
>> +};
> May I suggest wrapping at least the last one in #ifdef CONFIG_X86?
>
> Also please add a trailing comma.
Yes, will do.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio()
2020-11-11 8:09 ` Oleksandr
@ 2020-11-11 8:16 ` Jan Beulich
0 siblings, 0 replies; 109+ messages in thread
From: Jan Beulich @ 2020-11-11 8:16 UTC (permalink / raw)
To: Oleksandr
Cc: xen-devel, 'Oleksandr Tyshchenko',
'Andrew Cooper', 'Roger Pau Monné',
'Wei Liu', 'Julien Grall',
'Stefano Stabellini', 'Julien Grall',
paul
On 11.11.2020 09:09, Oleksandr wrote:
>
> On 11.11.20 09:27, Jan Beulich wrote:
>
> Hi Jan
>
>> On 10.11.2020 20:44, Oleksandr wrote:
>>> On 20.10.20 13:38, Paul Durrant wrote:
>>>
>>> Hi Jan, Paul
>>>
>>> Sorry for the late response.
>>>
>>>>> -----Original Message-----
>>>>> From: Jan Beulich <jbeulich@suse.com>
>>>>> Sent: 20 October 2020 11:05
>>>>> To: paul@xen.org
>>>>> Cc: 'Oleksandr Tyshchenko' <olekstysh@gmail.com>; xen-devel@lists.xenproject.org; 'Oleksandr
>>>>> Tyshchenko' <oleksandr_tyshchenko@epam.com>; 'Andrew Cooper' <andrew.cooper3@citrix.com>; 'Roger Pau
>>>>> Monné' <roger.pau@citrix.com>; 'Wei Liu' <wl@xen.org>; 'Julien Grall' <julien@xen.org>; 'Stefano
>>>>> Stabellini' <sstabellini@kernel.org>; 'Julien Grall' <julien.grall@arm.com>
>>>>> Subject: Re: [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio()
>>>>>
>>>>> On 20.10.2020 11:14, Paul Durrant wrote:
>>>>>>> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of Oleksandr Tyshchenko
>>>>>>> Sent: 15 October 2020 17:44
>>>>>>>
>>>>>>> --- a/xen/include/asm-x86/hvm/ioreq.h
>>>>>>> +++ b/xen/include/asm-x86/hvm/ioreq.h
>>>>>>> @@ -181,6 +181,8 @@ static inline bool arch_hvm_ioreq_destroy(struct domain *d)
>>>>>>> #define IOREQ_STATUS_UNHANDLED X86EMUL_UNHANDLEABLE
>>>>>>> #define IOREQ_STATUS_RETRY X86EMUL_RETRY
>>>>>>>
>>>>>>> +#define ioreq_complete_mmio handle_mmio
>>>>>>> +
>>>>>> A #define? Really? Can we not have a static inline?
>>>>> I guess this would require further shuffling: handle_mmio() is
>>>>> an inline function in hvm/emulate.h, and hvm/ioreq.h has no
>>>>> need to include the former (and imo it also shouldn't have).
>>>>>
>>>> I see. I think we need an x86 ioreq.c anyway, to deal with the legacy use of magic pages, so it could be dealt with there instead.
>>> I am afraid I don't entirely understand the required changes. Could you
>>> please clarify where the "inline(?)" ioreq_complete_mmio() should
>>> live? I included hvm/emulate.h here not for the "handle_mmio()" reason
>>> only, but for "struct hvm_emulate_ctxt" also (see arch_io_completion()).
>> I'm sorry, but in the context of this patch there's no use of any
>> struct hvm_emulate_ctxt instance. I'm not going to wade through 23
>> patches to find what you mean.
>
> Sorry for not being precise here. I meant arch_io_completion() added at [1]
At least some of the inlines you add there are way too large to be
inline functions, imo. But consensus appears to be now to retain a
per-arch ioreq.c anyway.
Jan
> [1]
> https://patchwork.kernel.org/project/xen-devel/patch/1602780274-29141-2-git-send-email-olekstysh@gmail.com/
>
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server()
2020-11-11 8:08 ` Jan Beulich
@ 2020-11-11 8:41 ` Oleksandr
2020-11-11 13:27 ` Jan Beulich
0 siblings, 1 reply; 109+ messages in thread
From: Oleksandr @ 2020-11-11 8:41 UTC (permalink / raw)
To: Jan Beulich
Cc: 'Oleksandr Tyshchenko', 'Stefano Stabellini',
'Julien Grall', 'Volodymyr Babchuk',
'Andrew Cooper', 'George Dunlap',
'Ian Jackson', 'Wei Liu', 'Julien Grall',
paul, xen-devel
On 11.11.20 10:08, Jan Beulich wrote:
Hi Jan
> On 10.11.2020 21:53, Oleksandr wrote:
>> On 20.10.20 13:51, Paul Durrant wrote:
>>
>> Hi Paul.
>>
>> Sorry for the late response.
>>
>>>> -----Original Message-----
>>>> From: Oleksandr Tyshchenko <olekstysh@gmail.com>
>>>> Sent: 15 October 2020 17:44
>>>> To: xen-devel@lists.xenproject.org
>>>> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Stefano Stabellini <sstabellini@kernel.org>;
>>>> Julien Grall <julien@xen.org>; Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>; Andrew Cooper
>>>> <andrew.cooper3@citrix.com>; George Dunlap <george.dunlap@citrix.com>; Ian Jackson
>>>> <iwj@xenproject.org>; Jan Beulich <jbeulich@suse.com>; Wei Liu <wl@xen.org>; Paul Durrant
>>>> <paul@xen.org>; Julien Grall <julien.grall@arm.com>
>>>> Subject: [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server()
>>>>
>>>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>>
>>>> This patch introduces a helper the main purpose of which is to check
>>>> if a domain is using IOREQ server(s).
>>>>
>>>> On Arm the current benefit is to avoid calling handle_io_completion()
>>>> (which implies iterating over all possible IOREQ servers anyway)
>>>> on every return in leave_hypervisor_to_guest() if there is no active
>>>> servers for the particular domain.
>>>> Also this helper will be used by one of the subsequent patches on Arm.
>>>>
>>>> This involves adding an extra per-domain variable to store the count
>>>> of servers in use.
>>>>
>>>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>> CC: Julien Grall <julien.grall@arm.com>
>>>>
>>>> ---
>>>> Please note, this is a split/cleanup/hardening of Julien's PoC:
>>>> "Add support for Guest IO forwarding to a device emulator"
>>>>
>>>> Changes RFC -> V1:
>>>> - new patch
>>>>
>>>> Changes V1 -> V2:
>>>> - update patch description
>>>> - guard helper with CONFIG_IOREQ_SERVER
>>>> - remove "hvm" prefix
>>>> - modify helper to just return d->arch.hvm.ioreq_server.nr_servers
>>>> - put suitable ASSERT()s
>>>> - use ASSERT(d->ioreq_server.server[id] ? !s : !!s) in set_ioreq_server()
>>>> - remove d->ioreq_server.nr_servers = 0 from hvm_ioreq_init()
>>>> ---
>>>> xen/arch/arm/traps.c | 15 +++++++++------
>>>> xen/common/ioreq.c | 7 ++++++-
>>>> xen/include/xen/ioreq.h | 14 ++++++++++++++
>>>> xen/include/xen/sched.h | 1 +
>>>> 4 files changed, 30 insertions(+), 7 deletions(-)
>>>>
>>>> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
>>>> index 507c095..a8f5fdf 100644
>>>> --- a/xen/arch/arm/traps.c
>>>> +++ b/xen/arch/arm/traps.c
>>>> @@ -2261,14 +2261,17 @@ static bool check_for_vcpu_work(void)
>>>> struct vcpu *v = current;
>>>>
>>>> #ifdef CONFIG_IOREQ_SERVER
>>>> - bool handled;
>>>> + if ( domain_has_ioreq_server(v->domain) )
>>>> + {
>>>> + bool handled;
>>>>
>>>> - local_irq_enable();
>>>> - handled = handle_io_completion(v);
>>>> - local_irq_disable();
>>>> + local_irq_enable();
>>>> + handled = handle_io_completion(v);
>>>> + local_irq_disable();
>>>>
>>>> - if ( !handled )
>>>> - return true;
>>>> + if ( !handled )
>>>> + return true;
>>>> + }
>>>> #endif
>>>>
>>>> if ( likely(!v->arch.need_flush_to_ram) )
>>>> diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
>>>> index bcd4961..a72bc0e 100644
>>>> --- a/xen/common/ioreq.c
>>>> +++ b/xen/common/ioreq.c
>>>> @@ -39,9 +39,14 @@ static void set_ioreq_server(struct domain *d, unsigned int id,
>>>> struct ioreq_server *s)
>>>> {
>>>> ASSERT(id < MAX_NR_IOREQ_SERVERS);
>>>> - ASSERT(!s || !d->ioreq_server.server[id]);
>>>> + ASSERT(d->ioreq_server.server[id] ? !s : !!s);
>>> That looks odd. How about ASSERT(!s ^ !d->ioreq_server.server[id])?
>> ok, looks like it will work.
>>
>>
>>> Paul
>>>
>>>> d->ioreq_server.server[id] = s;
>>>> +
>>>> + if ( s )
>>>> + d->ioreq_server.nr_servers++;
>>>> + else
>>>> + d->ioreq_server.nr_servers--;
>>>> }
>>>>
>>>> #define GET_IOREQ_SERVER(d, id) \
>>>> diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
>>>> index 7b03ab5..0679fef 100644
>>>> --- a/xen/include/xen/ioreq.h
>>>> +++ b/xen/include/xen/ioreq.h
>>>> @@ -55,6 +55,20 @@ struct ioreq_server {
>>>> uint8_t bufioreq_handling;
>>>> };
>>>>
>>>> +#ifdef CONFIG_IOREQ_SERVER
>>>> +static inline bool domain_has_ioreq_server(const struct domain *d)
>>>> +{
>>>> + ASSERT((current->domain == d) || atomic_read(&d->pause_count));
>>>> +
>>> This seems like an odd place to put such an assertion.
>> I might miss something or interpreted incorrectly but these asserts are
>> the result of how I understood the review comment on previous version [1].
>>
>> I will copy a comment here for the convenience:
>> "This is safe only when d == current->domain and it's not paused,
>> or when they're distinct and d is paused. Otherwise the result is
>> stale before the caller can inspect it. This wants documenting by
>> at least a comment, but perhaps better by suitable ASSERT()s."
> The way his reply was worded, I think Paul was wondering about the
> place where you put the assertion, not what you actually assert.
Shall I put the assertion at the call sites of this helper instead?
>
>
>>>> + return d->ioreq_server.nr_servers;
>>>> +}
>>>> +#else
>>>> +static inline bool domain_has_ioreq_server(const struct domain *d)
>>>> +{
>>>> + return false;
>>>> +}
>>>> +#endif
>>>> +
>>> Can this be any more compact? E.g.
>>>
>>> return IS_ENABLED(CONFIG_IOREQ_SERVER) && d->ioreq_server.nr_servers;
>>>
>>> ?
>> I have got a compilation error this way (if CONFIG_IOREQ_SERVER is
>> disabled):
>>
>> ...xen/4.14.0+gitAUTOINC+ee22110219-r0/git/xen/include/xen/ioreq.h:62:48:
>> error: ‘const struct domain’ has no member named ‘ioreq_server’
>> return IS_ENABLED(CONFIG_IOREQ_SERVER) && d->ioreq_server.nr_servers;
>> ^
>> as domain's ioreq_server struct is guarded by CONFIG_IOREQ_SERVER as well.
> The #ifdef is unavoidable here, I agree, but it should be inside
> the function's body. There's no need to duplicate the rest of it.
Got it, will do.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server()
2020-11-11 8:41 ` Oleksandr
@ 2020-11-11 13:27 ` Jan Beulich
2020-11-11 16:28 ` Paul Durrant
2020-11-11 17:31 ` Oleksandr
0 siblings, 2 replies; 109+ messages in thread
From: Jan Beulich @ 2020-11-11 13:27 UTC (permalink / raw)
To: Oleksandr
Cc: 'Oleksandr Tyshchenko', 'Stefano Stabellini',
'Julien Grall', 'Volodymyr Babchuk',
'Andrew Cooper', 'George Dunlap',
'Ian Jackson', 'Wei Liu', 'Julien Grall',
paul, xen-devel
On 11.11.2020 09:41, Oleksandr wrote:
>
> On 11.11.20 10:08, Jan Beulich wrote:
>
> Hi Jan
>
>> On 10.11.2020 21:53, Oleksandr wrote:
>>> On 20.10.20 13:51, Paul Durrant wrote:
>>>
>>> Hi Paul.
>>>
>>> Sorry for the late response.
>>>
>>>>> -----Original Message-----
>>>>> From: Oleksandr Tyshchenko <olekstysh@gmail.com>
>>>>> Sent: 15 October 2020 17:44
>>>>> To: xen-devel@lists.xenproject.org
>>>>> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Stefano Stabellini <sstabellini@kernel.org>;
>>>>> Julien Grall <julien@xen.org>; Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>; Andrew Cooper
>>>>> <andrew.cooper3@citrix.com>; George Dunlap <george.dunlap@citrix.com>; Ian Jackson
>>>>> <iwj@xenproject.org>; Jan Beulich <jbeulich@suse.com>; Wei Liu <wl@xen.org>; Paul Durrant
>>>>> <paul@xen.org>; Julien Grall <julien.grall@arm.com>
>>>>> Subject: [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server()
>>>>>
>>>>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>>>
>>>>> This patch introduces a helper the main purpose of which is to check
>>>>> if a domain is using IOREQ server(s).
>>>>>
>>>>> On Arm the current benefit is to avoid calling handle_io_completion()
>>>>> (which implies iterating over all possible IOREQ servers anyway)
>>>>> on every return in leave_hypervisor_to_guest() if there is no active
>>>>> servers for the particular domain.
>>>>> Also this helper will be used by one of the subsequent patches on Arm.
>>>>>
>>>>> This involves adding an extra per-domain variable to store the count
>>>>> of servers in use.
>>>>>
>>>>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>>> CC: Julien Grall <julien.grall@arm.com>
>>>>>
>>>>> ---
>>>>> Please note, this is a split/cleanup/hardening of Julien's PoC:
>>>>> "Add support for Guest IO forwarding to a device emulator"
>>>>>
>>>>> Changes RFC -> V1:
>>>>> - new patch
>>>>>
>>>>> Changes V1 -> V2:
>>>>> - update patch description
>>>>> - guard helper with CONFIG_IOREQ_SERVER
>>>>> - remove "hvm" prefix
>>>>> - modify helper to just return d->arch.hvm.ioreq_server.nr_servers
>>>>> - put suitable ASSERT()s
>>>>> - use ASSERT(d->ioreq_server.server[id] ? !s : !!s) in set_ioreq_server()
>>>>> - remove d->ioreq_server.nr_servers = 0 from hvm_ioreq_init()
>>>>> ---
>>>>> xen/arch/arm/traps.c | 15 +++++++++------
>>>>> xen/common/ioreq.c | 7 ++++++-
>>>>> xen/include/xen/ioreq.h | 14 ++++++++++++++
>>>>> xen/include/xen/sched.h | 1 +
>>>>> 4 files changed, 30 insertions(+), 7 deletions(-)
>>>>>
>>>>> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
>>>>> index 507c095..a8f5fdf 100644
>>>>> --- a/xen/arch/arm/traps.c
>>>>> +++ b/xen/arch/arm/traps.c
>>>>> @@ -2261,14 +2261,17 @@ static bool check_for_vcpu_work(void)
>>>>> struct vcpu *v = current;
>>>>>
>>>>> #ifdef CONFIG_IOREQ_SERVER
>>>>> - bool handled;
>>>>> + if ( domain_has_ioreq_server(v->domain) )
>>>>> + {
>>>>> + bool handled;
>>>>>
>>>>> - local_irq_enable();
>>>>> - handled = handle_io_completion(v);
>>>>> - local_irq_disable();
>>>>> + local_irq_enable();
>>>>> + handled = handle_io_completion(v);
>>>>> + local_irq_disable();
>>>>>
>>>>> - if ( !handled )
>>>>> - return true;
>>>>> + if ( !handled )
>>>>> + return true;
>>>>> + }
>>>>> #endif
>>>>>
>>>>> if ( likely(!v->arch.need_flush_to_ram) )
>>>>> diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
>>>>> index bcd4961..a72bc0e 100644
>>>>> --- a/xen/common/ioreq.c
>>>>> +++ b/xen/common/ioreq.c
>>>>> @@ -39,9 +39,14 @@ static void set_ioreq_server(struct domain *d, unsigned int id,
>>>>> struct ioreq_server *s)
>>>>> {
>>>>> ASSERT(id < MAX_NR_IOREQ_SERVERS);
>>>>> - ASSERT(!s || !d->ioreq_server.server[id]);
>>>>> + ASSERT(d->ioreq_server.server[id] ? !s : !!s);
>>>> That looks odd. How about ASSERT(!s ^ !d->ioreq_server.server[id])?
>>> ok, looks like it will work.
>>>
>>>
>>>> Paul
>>>>
>>>>> d->ioreq_server.server[id] = s;
>>>>> +
>>>>> + if ( s )
>>>>> + d->ioreq_server.nr_servers++;
>>>>> + else
>>>>> + d->ioreq_server.nr_servers--;
>>>>> }
>>>>>
>>>>> #define GET_IOREQ_SERVER(d, id) \
>>>>> diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
>>>>> index 7b03ab5..0679fef 100644
>>>>> --- a/xen/include/xen/ioreq.h
>>>>> +++ b/xen/include/xen/ioreq.h
>>>>> @@ -55,6 +55,20 @@ struct ioreq_server {
>>>>> uint8_t bufioreq_handling;
>>>>> };
>>>>>
>>>>> +#ifdef CONFIG_IOREQ_SERVER
>>>>> +static inline bool domain_has_ioreq_server(const struct domain *d)
>>>>> +{
>>>>> + ASSERT((current->domain == d) || atomic_read(&d->pause_count));
>>>>> +
>>>> This seems like an odd place to put such an assertion.
>>> I might miss something or interpreted incorrectly but these asserts are
>>> the result of how I understood the review comment on previous version [1].
>>>
>>> I will copy a comment here for the convenience:
>>> "This is safe only when d == current->domain and it's not paused,
>>> or when they're distinct and d is paused. Otherwise the result is
>>> stale before the caller can inspect it. This wants documenting by
>>> at least a comment, but perhaps better by suitable ASSERT()s."
>> The way his reply was worded, I think Paul was wondering about the
>> place where you put the assertion, not what you actually assert.
>
> Shall I put the assertion at the call sites of this helper instead?
Since Paul raised the question, I expect this is a question to him
rather than me?
Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server()
2020-11-11 13:27 ` Jan Beulich
@ 2020-11-11 16:28 ` Paul Durrant
2020-11-11 17:33 ` Oleksandr
2020-11-11 17:31 ` Oleksandr
1 sibling, 1 reply; 109+ messages in thread
From: Paul Durrant @ 2020-11-11 16:28 UTC (permalink / raw)
To: 'Jan Beulich', 'Oleksandr'
Cc: 'Oleksandr Tyshchenko', 'Stefano Stabellini',
'Julien Grall', 'Volodymyr Babchuk',
'Andrew Cooper', 'George Dunlap',
'Ian Jackson', 'Wei Liu', 'Julien Grall',
xen-devel
> -----Original Message-----
> From: Jan Beulich <jbeulich@suse.com>
> Sent: 11 November 2020 13:28
> To: Oleksandr <olekstysh@gmail.com>
> Cc: 'Oleksandr Tyshchenko' <oleksandr_tyshchenko@epam.com>; 'Stefano Stabellini'
> <sstabellini@kernel.org>; 'Julien Grall' <julien@xen.org>; 'Volodymyr Babchuk'
> <Volodymyr_Babchuk@epam.com>; 'Andrew Cooper' <andrew.cooper3@citrix.com>; 'George Dunlap'
> <george.dunlap@citrix.com>; 'Ian Jackson' <iwj@xenproject.org>; 'Wei Liu' <wl@xen.org>; 'Julien Grall'
> <julien.grall@arm.com>; paul@xen.org; xen-devel@lists.xenproject.org
> Subject: Re: [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server()
>
> On 11.11.2020 09:41, Oleksandr wrote:
> >
> > On 11.11.20 10:08, Jan Beulich wrote:
> >
> > Hi Jan
> >
> >> On 10.11.2020 21:53, Oleksandr wrote:
> >>> On 20.10.20 13:51, Paul Durrant wrote:
> >>>
> >>> Hi Paul.
> >>>
> >>> Sorry for the late response.
> >>>
> >>>>> -----Original Message-----
> >>>>> From: Oleksandr Tyshchenko <olekstysh@gmail.com>
> >>>>> Sent: 15 October 2020 17:44
> >>>>> To: xen-devel@lists.xenproject.org
> >>>>> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Stefano Stabellini
> <sstabellini@kernel.org>;
> >>>>> Julien Grall <julien@xen.org>; Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>; Andrew Cooper
> >>>>> <andrew.cooper3@citrix.com>; George Dunlap <george.dunlap@citrix.com>; Ian Jackson
> >>>>> <iwj@xenproject.org>; Jan Beulich <jbeulich@suse.com>; Wei Liu <wl@xen.org>; Paul Durrant
> >>>>> <paul@xen.org>; Julien Grall <julien.grall@arm.com>
> >>>>> Subject: [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server()
> >>>>>
> >>>>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> >>>>>
> >>>>> This patch introduces a helper the main purpose of which is to check
> >>>>> if a domain is using IOREQ server(s).
> >>>>>
> >>>>> On Arm the current benefit is to avoid calling handle_io_completion()
> >>>>> (which implies iterating over all possible IOREQ servers anyway)
> >>>>> on every return in leave_hypervisor_to_guest() if there is no active
> >>>>> servers for the particular domain.
> >>>>> Also this helper will be used by one of the subsequent patches on Arm.
> >>>>>
> >>>>> This involves adding an extra per-domain variable to store the count
> >>>>> of servers in use.
> >>>>>
> >>>>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> >>>>> CC: Julien Grall <julien.grall@arm.com>
> >>>>>
> >>>>> ---
> >>>>> Please note, this is a split/cleanup/hardening of Julien's PoC:
> >>>>> "Add support for Guest IO forwarding to a device emulator"
> >>>>>
> >>>>> Changes RFC -> V1:
> >>>>> - new patch
> >>>>>
> >>>>> Changes V1 -> V2:
> >>>>> - update patch description
> >>>>> - guard helper with CONFIG_IOREQ_SERVER
> >>>>> - remove "hvm" prefix
> >>>>> - modify helper to just return d->arch.hvm.ioreq_server.nr_servers
> >>>>> - put suitable ASSERT()s
> >>>>> - use ASSERT(d->ioreq_server.server[id] ? !s : !!s) in set_ioreq_server()
> >>>>> - remove d->ioreq_server.nr_servers = 0 from hvm_ioreq_init()
> >>>>> ---
> >>>>> xen/arch/arm/traps.c | 15 +++++++++------
> >>>>> xen/common/ioreq.c | 7 ++++++-
> >>>>> xen/include/xen/ioreq.h | 14 ++++++++++++++
> >>>>> xen/include/xen/sched.h | 1 +
> >>>>> 4 files changed, 30 insertions(+), 7 deletions(-)
> >>>>>
> >>>>> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> >>>>> index 507c095..a8f5fdf 100644
> >>>>> --- a/xen/arch/arm/traps.c
> >>>>> +++ b/xen/arch/arm/traps.c
> >>>>> @@ -2261,14 +2261,17 @@ static bool check_for_vcpu_work(void)
> >>>>> struct vcpu *v = current;
> >>>>>
> >>>>> #ifdef CONFIG_IOREQ_SERVER
> >>>>> - bool handled;
> >>>>> + if ( domain_has_ioreq_server(v->domain) )
> >>>>> + {
> >>>>> + bool handled;
> >>>>>
> >>>>> - local_irq_enable();
> >>>>> - handled = handle_io_completion(v);
> >>>>> - local_irq_disable();
> >>>>> + local_irq_enable();
> >>>>> + handled = handle_io_completion(v);
> >>>>> + local_irq_disable();
> >>>>>
> >>>>> - if ( !handled )
> >>>>> - return true;
> >>>>> + if ( !handled )
> >>>>> + return true;
> >>>>> + }
> >>>>> #endif
> >>>>>
> >>>>> if ( likely(!v->arch.need_flush_to_ram) )
> >>>>> diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
> >>>>> index bcd4961..a72bc0e 100644
> >>>>> --- a/xen/common/ioreq.c
> >>>>> +++ b/xen/common/ioreq.c
> >>>>> @@ -39,9 +39,14 @@ static void set_ioreq_server(struct domain *d, unsigned int id,
> >>>>> struct ioreq_server *s)
> >>>>> {
> >>>>> ASSERT(id < MAX_NR_IOREQ_SERVERS);
> >>>>> - ASSERT(!s || !d->ioreq_server.server[id]);
> >>>>> + ASSERT(d->ioreq_server.server[id] ? !s : !!s);
> >>>> That looks odd. How about ASSERT(!s ^ !d->ioreq_server.server[id])?
> >>> ok, looks like it will work.
> >>>
> >>>
> >>>> Paul
> >>>>
> >>>>> d->ioreq_server.server[id] = s;
> >>>>> +
> >>>>> + if ( s )
> >>>>> + d->ioreq_server.nr_servers++;
> >>>>> + else
> >>>>> + d->ioreq_server.nr_servers--;
> >>>>> }
> >>>>>
> >>>>> #define GET_IOREQ_SERVER(d, id) \
> >>>>> diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
> >>>>> index 7b03ab5..0679fef 100644
> >>>>> --- a/xen/include/xen/ioreq.h
> >>>>> +++ b/xen/include/xen/ioreq.h
> >>>>> @@ -55,6 +55,20 @@ struct ioreq_server {
> >>>>> uint8_t bufioreq_handling;
> >>>>> };
> >>>>>
> >>>>> +#ifdef CONFIG_IOREQ_SERVER
> >>>>> +static inline bool domain_has_ioreq_server(const struct domain *d)
> >>>>> +{
> >>>>> + ASSERT((current->domain == d) || atomic_read(&d->pause_count));
> >>>>> +
> >>>> This seems like an odd place to put such an assertion.
> >>> I might miss something or interpreted incorrectly but these asserts are
> >>> the result of how I understood the review comment on previous version [1].
> >>>
> >>> I will copy a comment here for the convenience:
> >>> "This is safe only when d == current->domain and it's not paused,
> >>> or when they're distinct and d is paused. Otherwise the result is
> >>> stale before the caller can inspect it. This wants documenting by
> >>> at least a comment, but perhaps better by suitable ASSERT()s."
> >> The way his reply was worded, I think Paul was wondering about the
> >> place where you put the assertion, not what you actually assert.
> >
> > Shall I put the assertion at the call sites of this helper instead?
>
> Since Paul raised the question, I expect this is a question to him
> rather than me?
If it is indeed a question for me then yes, put the assertion where it is clear why it is needed. domain_has_ioreq_server() is essentially a trivial accessor function; it's not the appropriate place.
Paul
>
> Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server()
2020-11-11 13:27 ` Jan Beulich
2020-11-11 16:28 ` Paul Durrant
@ 2020-11-11 17:31 ` Oleksandr
1 sibling, 0 replies; 109+ messages in thread
From: Oleksandr @ 2020-11-11 17:31 UTC (permalink / raw)
To: Jan Beulich
Cc: 'Oleksandr Tyshchenko', 'Stefano Stabellini',
'Julien Grall', 'Volodymyr Babchuk',
'Andrew Cooper', 'George Dunlap',
'Ian Jackson', 'Wei Liu', 'Julien Grall',
paul, xen-devel
On 11.11.20 15:27, Jan Beulich wrote:
Hi Jan.
>
>>>>>> }
>>>>>>
>>>>>> #define GET_IOREQ_SERVER(d, id) \
>>>>>> diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
>>>>>> index 7b03ab5..0679fef 100644
>>>>>> --- a/xen/include/xen/ioreq.h
>>>>>> +++ b/xen/include/xen/ioreq.h
>>>>>> @@ -55,6 +55,20 @@ struct ioreq_server {
>>>>>> uint8_t bufioreq_handling;
>>>>>> };
>>>>>>
>>>>>> +#ifdef CONFIG_IOREQ_SERVER
>>>>>> +static inline bool domain_has_ioreq_server(const struct domain *d)
>>>>>> +{
>>>>>> + ASSERT((current->domain == d) || atomic_read(&d->pause_count));
>>>>>> +
>>>>> This seems like an odd place to put such an assertion.
>>>> I might miss something or interpreted incorrectly but these asserts are
>>>> the result of how I understood the review comment on previous version [1].
>>>>
>>>> I will copy a comment here for the convenience:
>>>> "This is safe only when d == current->domain and it's not paused,
>>>> or when they're distinct and d is paused. Otherwise the result is
>>>> stale before the caller can inspect it. This wants documenting by
>>>> at least a comment, but perhaps better by suitable ASSERT()s."
>>> The way his reply was worded, I think Paul was wondering about the
>>> place where you put the assertion, not what you actually assert.
>> Shall I put the assertion at the call sites of this helper instead?
> Since Paul raised the question, I expect this is a question to him
> rather than me?
Yes, it was primarily a question to Paul, but I wanted to hear your
opinion as well. Sorry for the confusion.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server()
2020-11-11 16:28 ` Paul Durrant
@ 2020-11-11 17:33 ` Oleksandr
0 siblings, 0 replies; 109+ messages in thread
From: Oleksandr @ 2020-11-11 17:33 UTC (permalink / raw)
To: paul
Cc: 'Jan Beulich', 'Oleksandr Tyshchenko',
'Stefano Stabellini', 'Julien Grall',
'Volodymyr Babchuk', 'Andrew Cooper',
'George Dunlap', 'Ian Jackson', 'Wei Liu',
'Julien Grall',
xen-devel
On 11.11.20 18:28, Paul Durrant wrote:
Hi Paul.
>>
>>>>>>> d->ioreq_server.server[id] = s;
>>>>>>> +
>>>>>>> + if ( s )
>>>>>>> + d->ioreq_server.nr_servers++;
>>>>>>> + else
>>>>>>> + d->ioreq_server.nr_servers--;
>>>>>>> }
>>>>>>>
>>>>>>> #define GET_IOREQ_SERVER(d, id) \
>>>>>>> diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
>>>>>>> index 7b03ab5..0679fef 100644
>>>>>>> --- a/xen/include/xen/ioreq.h
>>>>>>> +++ b/xen/include/xen/ioreq.h
>>>>>>> @@ -55,6 +55,20 @@ struct ioreq_server {
>>>>>>> uint8_t bufioreq_handling;
>>>>>>> };
>>>>>>>
>>>>>>> +#ifdef CONFIG_IOREQ_SERVER
>>>>>>> +static inline bool domain_has_ioreq_server(const struct domain *d)
>>>>>>> +{
>>>>>>> + ASSERT((current->domain == d) || atomic_read(&d->pause_count));
>>>>>>> +
>>>>>> This seems like an odd place to put such an assertion.
>>>>> I might miss something or interpreted incorrectly but these asserts are
>>>>> the result of how I understood the review comment on previous version [1].
>>>>>
>>>>> I will copy a comment here for the convenience:
>>>>> "This is safe only when d == current->domain and it's not paused,
>>>>> or when they're distinct and d is paused. Otherwise the result is
>>>>> stale before the caller can inspect it. This wants documenting by
>>>>> at least a comment, but perhaps better by suitable ASSERT()s."
>>>> The way his reply was worded, I think Paul was wondering about the
>>>> place where you put the assertion, not what you actually assert.
>>> Shall I put the assertion at the call sites of this helper instead?
>> Since Paul raised the question, I expect this is a question to him
>> rather than me?
> If it is indeed a question for me then yes, put the assertion where it is clear why it is needed. domain_has_ioreq_server() is essentially a trivial accessor function; it's not the appropriate place.
Got it. Thank you for the clarification.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 21/23] xen/arm: Add mapcache invalidation handling
2020-11-11 0:03 ` Oleksandr
@ 2020-11-11 19:27 ` Julien Grall
2020-11-11 19:42 ` Oleksandr
0 siblings, 1 reply; 109+ messages in thread
From: Julien Grall @ 2020-11-11 19:27 UTC (permalink / raw)
To: Oleksandr, Jan Beulich
Cc: xen-devel, Oleksandr Tyshchenko, Stefano Stabellini,
Volodymyr Babchuk, Julien Grall
Hi Oleksandr,
On 11/11/2020 00:03, Oleksandr wrote:
>
> On 16.10.20 11:41, Julien Grall wrote:
>> On 16/10/2020 07:29, Jan Beulich wrote:
>>> On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
>>>> @@ -1067,7 +1068,14 @@ static int __p2m_set_entry(struct p2m_domain
>>>> *p2m,
>>>> */
>>>> if ( p2m_is_valid(orig_pte) &&
>>>> !mfn_eq(lpae_get_mfn(*entry), lpae_get_mfn(orig_pte)) )
>>>> + {
>>>> +#ifdef CONFIG_IOREQ_SERVER
>>>> + if ( domain_has_ioreq_server(p2m->domain) &&
>>>> + (p2m->domain == current->domain) &&
>>>> p2m_is_ram(orig_pte.p2m.type) )
>>>> + p2m->domain->qemu_mapcache_invalidate = true;
>>>> +#endif
>>>> p2m_free_entry(p2m, orig_pte, level);
>>>> + }
>>>
>>> For all I have to say here, please bear in mind that I don't know
>>> the internals of Arm memory management.
>>>
>>> The first odd thing here the merely MFN-based condition. It may
>>> well be that's sufficient, if there's no way to get a "not present"
>>> entry with an MFN matching any valid MFN. (This isn't just with
>>> your addition, but even before.
>> Invalid entries are always zeroed. So in theory the problem could
>> arise if MFN 0 used in the guest. It should not be possible on
>> staging, but I agree this should be fixed.
>>
>>>
>>> Given how p2m_free_entry() works (or is supposed to work in the
>>> long run), is the new code you add guaranteed to only alter leaf
>>> entries?
>>
>> This path may also be called with tables. I think we want to move the
>> check in p2m_free_entry() so we can find the correct leaf type.
>
> Well, but inside p2m_free_entry() we don't have a new entry in order to
> check whether new MFN is actually different. An extra arg only comes to
> mind...
Aside the recursive call, there are two users for p2m_free_entry():
- When we fail to shatter a superpage in OOM
- When the entry is replaced by an entry with a different base
I wouldn't be too concerned to send spurious mapcache invalidation in an
error path. So I don't think you need to know the new entry.
What you need to know if the type of the leaf.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 21/23] xen/arm: Add mapcache invalidation handling
2020-11-11 19:27 ` Julien Grall
@ 2020-11-11 19:42 ` Oleksandr
0 siblings, 0 replies; 109+ messages in thread
From: Oleksandr @ 2020-11-11 19:42 UTC (permalink / raw)
To: Julien Grall
Cc: Jan Beulich, xen-devel, Oleksandr Tyshchenko, Stefano Stabellini,
Volodymyr Babchuk, Julien Grall
On 11.11.20 21:27, Julien Grall wrote:
> Hi Oleksandr,
Hi Julien.
>
> On 11/11/2020 00:03, Oleksandr wrote:
>>
>> On 16.10.20 11:41, Julien Grall wrote:
>>> On 16/10/2020 07:29, Jan Beulich wrote:
>>>> On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
>>>>> @@ -1067,7 +1068,14 @@ static int __p2m_set_entry(struct
>>>>> p2m_domain *p2m,
>>>>> */
>>>>> if ( p2m_is_valid(orig_pte) &&
>>>>> !mfn_eq(lpae_get_mfn(*entry), lpae_get_mfn(orig_pte)) )
>>>>> + {
>>>>> +#ifdef CONFIG_IOREQ_SERVER
>>>>> + if ( domain_has_ioreq_server(p2m->domain) &&
>>>>> + (p2m->domain == current->domain) &&
>>>>> p2m_is_ram(orig_pte.p2m.type) )
>>>>> + p2m->domain->qemu_mapcache_invalidate = true;
>>>>> +#endif
>>>>> p2m_free_entry(p2m, orig_pte, level);
>>>>> + }
>>>>
>>>> For all I have to say here, please bear in mind that I don't know
>>>> the internals of Arm memory management.
>>>>
>>>> The first odd thing here the merely MFN-based condition. It may
>>>> well be that's sufficient, if there's no way to get a "not present"
>>>> entry with an MFN matching any valid MFN. (This isn't just with
>>>> your addition, but even before.
>>> Invalid entries are always zeroed. So in theory the problem could
>>> arise if MFN 0 used in the guest. It should not be possible on
>>> staging, but I agree this should be fixed.
>>>
>>>>
>>>> Given how p2m_free_entry() works (or is supposed to work in the
>>>> long run), is the new code you add guaranteed to only alter leaf
>>>> entries?
>>>
>>> This path may also be called with tables. I think we want to move
>>> the check in p2m_free_entry() so we can find the correct leaf type.
>>
>> Well, but inside p2m_free_entry() we don't have a new entry in order
>> to check whether new MFN is actually different. An extra arg only
>> comes to mind...
> Aside the recursive call, there are two users for p2m_free_entry():
> - When we fail to shatter a superpage in OOM
> - When the entry is replaced by an entry with a different base
>
> I wouldn't be too concerned to send spurious mapcache invalidation in
> an error path. So I don't think you need to know the new entry.
Thank you for the clarification, sounds reasonable.
>
> What you need to know if the type of the leaf.
Yes, to check whether it is a RAM page.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 01/23] x86/ioreq: Prepare IOREQ feature for making it common
2020-10-15 16:44 ` [PATCH V2 01/23] x86/ioreq: Prepare IOREQ feature for making it common Oleksandr Tyshchenko
2020-10-20 7:13 ` Paul Durrant
@ 2020-11-12 10:58 ` Jan Beulich
2020-11-13 11:09 ` Oleksandr
1 sibling, 1 reply; 109+ messages in thread
From: Jan Beulich @ 2020-11-12 10:58 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: Oleksandr Tyshchenko, Paul Durrant, Andrew Cooper,
Roger Pau Monné,
Julien Grall, Stefano Stabellini, Wei Liu, Julien Grall,
xen-devel
On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> As a lot of x86 code can be re-used on Arm later on, this
> patch makes some preparation to x86/hvm/ioreq.c before moving
> to the common code. This way we will get a verbatim copy for
> a code movement in subsequent patch (arch/x86/hvm/ioreq.c
> will be *just* renamed to common/ioreq).
>
> This patch does the following:
> 1. Introduce *inline* arch_hvm_ioreq_init(), arch_hvm_ioreq_destroy(),
> arch_hvm_io_completion(), arch_hvm_destroy_ioreq_server() and
> hvm_ioreq_server_get_type_addr() to abstract arch specific materials.
> 2 Make hvm_map_mem_type_to_ioreq_server() *inline*. It is not going
> to be called from the common code.
As already indicated on another sub-thread, I think some of these
are too large to be inline functions. Additionally, considering
their single-use purpose, I don't think they should be placed in
a header consumed by more than the producer and the sole consumer.
> 3. Make get_ioreq_server() global. It is going to be called from
> a few places.
And with this its name ought to change, to fit the general naming
model of global functions of this subsystem.
> 4. Add IOREQ_STATUS_* #define-s and update candidates for moving.
This, it seems to me, could be a separate patch.
> @@ -855,7 +841,7 @@ int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
>
> domain_pause(d);
>
> - p2m_set_ioreq_server(d, 0, s);
> + arch_hvm_destroy_ioreq_server(s);
Iirc there are plans to rename hvm_destroy_ioreq_server() in the
course of the generalization. If so, this arch hook would imo
better be named following the new scheme right away.
> @@ -1215,7 +1153,7 @@ void hvm_destroy_all_ioreq_servers(struct domain *d)
> struct hvm_ioreq_server *s;
> unsigned int id;
>
> - if ( !relocate_portio_handler(d, 0xcf8, 0xcf8, 4) )
> + if ( !arch_hvm_ioreq_destroy(d) )
There's no ioreq being destroyed here, so I think this wants
renaming (and again ideally right away following the planned
new scheme).
> +static inline int hvm_map_mem_type_to_ioreq_server(struct domain *d,
> + ioservid_t id,
> + uint32_t type,
> + uint32_t flags)
> +{
> + struct hvm_ioreq_server *s;
> + int rc;
> +
> + if ( type != HVMMEM_ioreq_server )
> + return -EINVAL;
> +
> + if ( flags & ~XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
> + return -EINVAL;
> +
> + spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
> +
> + s = get_ioreq_server(d, id);
> +
> + rc = -ENOENT;
> + if ( !s )
> + goto out;
> +
> + rc = -EPERM;
> + if ( s->emulator != current->domain )
> + goto out;
> +
> + rc = p2m_set_ioreq_server(d, flags, s);
> +
> + out:
> + spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
> +
> + if ( rc == 0 && flags == 0 )
> + {
> + struct p2m_domain *p2m = p2m_get_hostp2m(d);
I realize I may be asking too much, but would it be possible if,
while moving code, you made simple and likely uncontroversial
adjustments like adding const here? (Such adjustments would be
less desirable to make if they increased the size of the patch,
e.g. if you were touching only nearby code.)
> + if ( read_atomic(&p2m->ioreq.entry_count) )
> + p2m_change_entry_type_global(d, p2m_ioreq_server, p2m_ram_rw);
> + }
> +
> + return rc;
> +}
> +
> +static inline int hvm_ioreq_server_get_type_addr(const struct domain *d,
> + const ioreq_t *p,
> + uint8_t *type,
> + uint64_t *addr)
> +{
> + uint32_t cf8 = d->arch.hvm.pci_cf8;
Similarly, for example, neither this nor ...
> + if ( p->type != IOREQ_TYPE_COPY && p->type != IOREQ_TYPE_PIO )
> + return -EINVAL;
> +
> + if ( p->type == IOREQ_TYPE_PIO &&
> + (p->addr & ~3) == 0xcfc &&
> + CF8_ENABLED(cf8) )
> + {
> + uint32_t x86_fam;
... this really need to use a fixed width type - unsigned int is
going to be quite fine. But since you're only moving this code,
I guess I'm not going to insist.
> +static inline bool arch_hvm_ioreq_destroy(struct domain *d)
> +{
> + if ( !relocate_portio_handler(d, 0xcf8, 0xcf8, 4) )
> + return false;
> +
> + return true;
Any reason this cannot simply be
return relocate_portio_handler(d, 0xcf8, 0xcf8, 4);
?
Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 02/23] xen/ioreq: Make x86's IOREQ feature common
2020-10-15 16:44 ` [PATCH V2 02/23] xen/ioreq: Make x86's IOREQ feature common Oleksandr Tyshchenko
2020-10-20 7:57 ` Paul Durrant
@ 2020-11-12 11:11 ` Jan Beulich
2020-11-13 13:11 ` Oleksandr
1 sibling, 1 reply; 109+ messages in thread
From: Jan Beulich @ 2020-11-12 11:11 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: Oleksandr Tyshchenko, Andrew Cooper, George Dunlap, Ian Jackson,
Julien Grall, Stefano Stabellini, Wei Liu, Roger Pau Monné,
Paul Durrant, Tim Deegan, Julien Grall, xen-devel
On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> As a lot of x86 code can be re-used on Arm later on, this patch
> moves previously prepared x86/hvm/ioreq.c to the common code.
>
> The common IOREQ feature is supposed to be built with IOREQ_SERVER
> option enabled, which is selected for x86's config HVM for now.
>
> In order to avoid having a gigantic patch here, the subsequent
> patches will update remaining bits in the common code step by step:
> - Make IOREQ related structs/materials common
> - Drop the "hvm" prefixes and infixes
> - Remove layering violation by moving corresponding fields
> out of *arch.hvm* or abstracting away accesses to them
>
> This support is going to be used on Arm to be able run device
> emulator outside of Xen hypervisor.
>
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> CC: Julien Grall <julien.grall@arm.com>
>
> ---
> Please note, this is a split/cleanup/hardening of Julien's PoC:
> "Add support for Guest IO forwarding to a device emulator"
>
> ***
> Please note, this patch depends on the following which is
> on review:
> https://patchwork.kernel.org/patch/11816689/
> ***
>
> Changes RFC -> V1:
> - was split into three patches:
> - x86/ioreq: Prepare IOREQ feature for making it common
> - xen/ioreq: Make x86's IOREQ feature common
> - xen/ioreq: Make x86's hvm_ioreq_needs_completion() common
> - update MAINTAINERS file
> - do not use a separate subdir for the IOREQ stuff, move it to:
> - xen/common/ioreq.c
> - xen/include/xen/ioreq.h
> - update x86's files to include xen/ioreq.h
> - remove unneeded headers in arch/x86/hvm/ioreq.c
> - re-order the headers alphabetically in common/ioreq.c
> - update common/ioreq.c according to the newly introduced arch functions:
> arch_hvm_destroy_ioreq_server()/arch_handle_hvm_io_completion()
>
> Changes V1 -> V2:
> - update patch description
> - make everything needed in the previous patch to achieve
> a truly rename here
> - don't include unnecessary headers from asm-x86/hvm/ioreq.h
> and xen/ioreq.h
> - use __XEN_IOREQ_H__ instead of __IOREQ_H__
> - move get_ioreq_server() to common/ioreq.c
> ---
> MAINTAINERS | 8 +-
> xen/arch/x86/Kconfig | 1 +
> xen/arch/x86/hvm/Makefile | 1 -
> xen/arch/x86/hvm/ioreq.c | 1422 ---------------------------------------
> xen/arch/x86/mm.c | 2 +-
> xen/arch/x86/mm/shadow/common.c | 2 +-
> xen/common/Kconfig | 3 +
> xen/common/Makefile | 1 +
> xen/common/ioreq.c | 1422 +++++++++++++++++++++++++++++++++++++++
> xen/include/asm-x86/hvm/ioreq.h | 39 +-
> xen/include/xen/ioreq.h | 71 ++
> 11 files changed, 1509 insertions(+), 1463 deletions(-)
> delete mode 100644 xen/arch/x86/hvm/ioreq.c
> create mode 100644 xen/common/ioreq.c
> create mode 100644 xen/include/xen/ioreq.h
Iirc I've previously asked to make sure the diff here gets created with
git's rename detection enabled, so we wouldn't see 1422 lines deleted
and 1422 lines added, _hoping_ they're all the same (or going through
the extra steps needed to compare old and new code), but instead seeing
just the diff between old and new files (which in the ideal case would
then be empty).
Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 07/23] xen/ioreq: Move x86's ioreq_gfn(server) to struct domain
2020-10-15 16:44 ` [PATCH V2 07/23] xen/ioreq: Move x86's ioreq_gfn(server) to struct domain Oleksandr Tyshchenko
@ 2020-11-12 11:21 ` Jan Beulich
2020-11-13 14:05 ` Oleksandr
2020-11-18 12:09 ` Oleksandr
1 sibling, 1 reply; 109+ messages in thread
From: Jan Beulich @ 2020-11-12 11:21 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: Oleksandr Tyshchenko, Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Paul Durrant, Julien Grall, xen-devel
On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
> --- a/xen/include/asm-x86/hvm/ioreq.h
> +++ b/xen/include/asm-x86/hvm/ioreq.h
> @@ -77,7 +77,7 @@ static inline int hvm_map_mem_type_to_ioreq_server(struct domain *d,
> if ( flags & ~XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
> return -EINVAL;
>
> - spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
> + spin_lock_recursive(&d->ioreq_server.lock);
>
> s = get_ioreq_server(d, id);
>
> @@ -92,7 +92,7 @@ static inline int hvm_map_mem_type_to_ioreq_server(struct domain *d,
> rc = p2m_set_ioreq_server(d, flags, s);
>
> out:
> - spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
> + spin_unlock_recursive(&d->ioreq_server.lock);
>
> if ( rc == 0 && flags == 0 )
> {
Does this build at this point, when !CONFIG_IOREQ_SERVER? Patch 1
moves the code here without guards, and patch 2, when introducing
the Kconfig symbol, doesn't add guards here. I admit I didn't
check further intermediate patches.
Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 09/23] xen/dm: Make x86's DM feature common
2020-10-15 16:44 ` [PATCH V2 09/23] xen/dm: Make x86's DM feature common Oleksandr Tyshchenko
@ 2020-11-12 11:32 ` Jan Beulich
2020-11-13 14:28 ` Oleksandr
0 siblings, 1 reply; 109+ messages in thread
From: Jan Beulich @ 2020-11-12 11:32 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: Julien Grall, Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Daniel De Graaf, Oleksandr Tyshchenko,
xen-devel
On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
> From: Julien Grall <julien.grall@arm.com>
>
> As a lot of x86 code can be re-used on Arm later on, this patch
> splits devicemodel support into common and arch specific parts.
>
> The common DM feature is supposed to be built with IOREQ_SERVER
> option enabled (as well as the IOREQ feature), which is selected
> for x86's config HVM for now.
Did you consider doing it the other way around? It would seem
more natural to have the top level dm-op handling arch-specific
and call into e.g. ioreq_server_dm_op() for otherwise unhandled
ops, just like e.g. do_domctl() calls into iommu_do_domctl()
(indirectly via arch_do_domctl()). I ask because in the long
run I expect the ioreq server sub-ops to only be a small part
of the overall set of dm-ops; already now it's 7 out of 18 if
I got the counting right.
This would then also leave compat_dm_op() in x86 code.
But yes, there are also downsides with this alternative.
Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 10/23] xen/mm: Make x86's XENMEM_resource_ioreq_server handling common
2020-10-15 16:44 ` [PATCH V2 10/23] xen/mm: Make x86's XENMEM_resource_ioreq_server handling common Oleksandr Tyshchenko
@ 2020-11-12 11:40 ` Jan Beulich
2020-11-13 15:00 ` Oleksandr
0 siblings, 1 reply; 109+ messages in thread
From: Jan Beulich @ 2020-11-12 11:40 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: Julien Grall, Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Volodymyr Babchuk, Oleksandr Tyshchenko,
xen-devel
On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
> --- a/xen/common/memory.c
> +++ b/xen/common/memory.c
> @@ -30,6 +30,10 @@
> #include <public/memory.h>
> #include <xsm/xsm.h>
>
> +#ifdef CONFIG_IOREQ_SERVER
> +#include <xen/ioreq.h>
> +#endif
Preferably #ifdef-s would not be needed here. If any, they'd better
live in xen/ioreq.h itself then.
> @@ -1045,6 +1049,38 @@ static int acquire_grant_table(struct domain *d, unsigned int id,
> return 0;
> }
>
> +#ifdef CONFIG_IOREQ_SERVER
To limit the number of #ifdef-s, could this be moved ...
> +static int acquire_ioreq_server(struct domain *d,
> + unsigned int id,
> + unsigned long frame,
> + unsigned int nr_frames,
> + xen_pfn_t mfn_list[])
> +{
... here such that ...
> @@ -1103,9 +1139,14 @@ static int acquire_resource(
> mfn_list);
> break;
>
> +#ifdef CONFIG_IOREQ_SERVER
> + case XENMEM_resource_ioreq_server:
> + rc = acquire_ioreq_server(d, xmar.id, xmar.frame, xmar.nr_frames,
> + mfn_list);
> + break;
> +#endif
... the ones here then can be dropped?
> default:
Also you'll want to a blank line between the new case statement and
the "default:".
Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved function names
2020-10-15 16:44 ` [PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved function names Oleksandr Tyshchenko
@ 2020-11-12 11:45 ` Jan Beulich
2020-11-12 12:14 ` Paul Durrant
2020-11-13 15:53 ` Oleksandr
0 siblings, 2 replies; 109+ messages in thread
From: Jan Beulich @ 2020-11-12 11:45 UTC (permalink / raw)
To: Oleksandr Tyshchenko, Paul Durrant
Cc: Oleksandr Tyshchenko, Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Julien Grall,
xen-devel
On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>
> This patch removes "hvm" prefixes and infixes from IOREQ
> related function names in the common code.
AT least some of the functions touched here would be nice to be
moved to a more consistent new naming scheme right away, to
avoid having to touch all the same places again. I guess ioreq
server functions would be nice to all start with ioreq_server_
and ioreq functions to all start with ioreq_. E.g. ioreq_send()
and ioreq_server_select().
Paul, thoughts?
Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 14/23] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2020-10-15 16:44 ` [PATCH V2 14/23] arm/ioreq: Introduce arch specific bits for IOREQ/DM features Oleksandr Tyshchenko
@ 2020-11-12 11:48 ` Jan Beulich
2020-11-13 15:48 ` Oleksandr
0 siblings, 1 reply; 109+ messages in thread
From: Jan Beulich @ 2020-11-12 11:48 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: Julien Grall, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Paul Durrant, Oleksandr Tyshchenko, xen-devel
On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
> --- a/xen/common/ioreq.c
> +++ b/xen/common/ioreq.c
> @@ -18,6 +18,7 @@
>
> #include <xen/ctype.h>
> #include <xen/domain.h>
> +#include <xen/domain_page.h>
> #include <xen/event.h>
> #include <xen/init.h>
> #include <xen/irq.h>
There preferably wouldn't be a need to touch non-Arm code in this
patch. I suppose the added #include could easily be introduced
e.g. while moving the file?
Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 20/23] xen/ioreq: Make x86's send_invalidate_req() common
2020-10-15 16:44 ` [PATCH V2 20/23] xen/ioreq: Make x86's send_invalidate_req() common Oleksandr Tyshchenko
@ 2020-11-12 11:55 ` Jan Beulich
2020-11-13 16:03 ` Oleksandr
0 siblings, 1 reply; 109+ messages in thread
From: Jan Beulich @ 2020-11-12 11:55 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: Oleksandr Tyshchenko, Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Paul Durrant, Julien Grall, xen-devel
On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
> --- a/xen/include/asm-x86/hvm/io.h
> +++ b/xen/include/asm-x86/hvm/io.h
> @@ -97,7 +97,6 @@ bool relocate_portio_handler(
> unsigned int size);
>
> void send_timeoffset_req(unsigned long timeoff);
> -void send_invalidate_req(void);
> bool handle_mmio_with_translation(unsigned long gla, unsigned long gpfn,
> struct npfec);
> bool handle_pio(uint16_t port, unsigned int size, int dir);
> diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
> index 0679fef..aad682f 100644
> --- a/xen/include/xen/ioreq.h
> +++ b/xen/include/xen/ioreq.h
> @@ -126,6 +126,7 @@ struct ioreq_server *select_ioreq_server(struct domain *d,
> int send_ioreq(struct ioreq_server *s, ioreq_t *proto_p,
> bool buffered);
> unsigned int broadcast_ioreq(ioreq_t *p, bool buffered);
> +void send_invalidate_ioreq(void);
Again while renaming this function anyway could we see about giving
it a suitable and consistent name? Maybe
ioreq_request_mapcache_invalidate() or (to avoid the double "request")
ioreq_signal_mapcache_invalidate()? Maybe even ioreq_server_...().
Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved function names
2020-11-12 11:45 ` Jan Beulich
@ 2020-11-12 12:14 ` Paul Durrant
2020-11-13 15:53 ` Oleksandr
1 sibling, 0 replies; 109+ messages in thread
From: Paul Durrant @ 2020-11-12 12:14 UTC (permalink / raw)
To: 'Jan Beulich', 'Oleksandr Tyshchenko'
Cc: 'Oleksandr Tyshchenko', 'Andrew Cooper',
'Roger Pau Monné', 'Wei Liu',
'George Dunlap', 'Ian Jackson',
'Julien Grall', 'Stefano Stabellini',
'Jun Nakajima', 'Kevin Tian',
'Julien Grall',
xen-devel
> -----Original Message-----
> From: Jan Beulich <jbeulich@suse.com>
> Sent: 12 November 2020 11:45
> To: Oleksandr Tyshchenko <olekstysh@gmail.com>; Paul Durrant <paul@xen.org>
> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Andrew Cooper <andrew.cooper3@citrix.com>;
> Roger Pau Monné <roger.pau@citrix.com>; Wei Liu <wl@xen.org>; George Dunlap
> <george.dunlap@citrix.com>; Ian Jackson <iwj@xenproject.org>; Julien Grall <julien@xen.org>; Stefano
> Stabellini <sstabellini@kernel.org>; Jun Nakajima <jun.nakajima@intel.com>; Kevin Tian
> <kevin.tian@intel.com>; Julien Grall <julien.grall@arm.com>; xen-devel@lists.xenproject.org
> Subject: Re: [PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved function names
>
> On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
> > From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> >
> > This patch removes "hvm" prefixes and infixes from IOREQ
> > related function names in the common code.
>
> AT least some of the functions touched here would be nice to be
> moved to a more consistent new naming scheme right away, to
> avoid having to touch all the same places again. I guess ioreq
> server functions would be nice to all start with ioreq_server_
> and ioreq functions to all start with ioreq_. E.g. ioreq_send()
> and ioreq_server_select().
>
> Paul, thoughts?
>
Yes, that sounds like a good idea. Obviously the code has grown a little organically so some naming cleanup is welcome.
Paul
> Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 16/23] xen/mm: Handle properly reference in set_foreign_p2m_entry() on Arm
2020-10-15 16:44 ` [PATCH V2 16/23] xen/mm: Handle properly reference in set_foreign_p2m_entry() on Arm Oleksandr Tyshchenko
@ 2020-11-12 12:18 ` Jan Beulich
2020-11-13 17:00 ` Oleksandr
0 siblings, 1 reply; 109+ messages in thread
From: Jan Beulich @ 2020-11-12 12:18 UTC (permalink / raw)
To: Oleksandr Tyshchenko
Cc: Oleksandr Tyshchenko, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Andrew Cooper, George Dunlap, Ian Jackson,
Wei Liu, Roger Pau Monné,
Julien Grall, xen-devel
On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
> --- a/xen/arch/arm/p2m.c
> +++ b/xen/arch/arm/p2m.c
> @@ -1380,6 +1380,27 @@ int guest_physmap_remove_page(struct domain *d, gfn_t gfn, mfn_t mfn,
> return p2m_remove_mapping(d, gfn, (1 << page_order), mfn);
> }
>
> +int set_foreign_p2m_entry(struct domain *d, const struct domain *fd,
> + unsigned long gfn, mfn_t mfn)
> +{
> + struct page_info *page = mfn_to_page(mfn);
> + int rc;
> +
> + if ( !get_page(page, fd) )
> + return -EINVAL;
> +
> + /*
> + * It is valid to always use p2m_map_foreign_rw here as if this gets
> + * called that d != fd. A case when d == fd would be rejected by
> + * rcu_lock_remote_domain_by_id() earlier.
> + */
Are you describing things here on the assumption that no new
callers may surface later on? To catch such, I'd recommend
adding at least a respective ASSERT().
> + rc = guest_physmap_add_entry(d, _gfn(gfn), mfn, 0, p2m_map_foreign_rw);
> + if ( rc )
> + put_page(page);
> +
> + return 0;
I can't imagine it's correct to not signal the error to the
caller(s).
> --- a/xen/common/memory.c
> +++ b/xen/common/memory.c
> @@ -1099,7 +1099,8 @@ static int acquire_resource(
> * reference counted, it is unsafe to allow mapping of
> * resource pages unless the caller is the hardware domain.
> */
> - if ( paging_mode_translate(currd) && !is_hardware_domain(currd) )
> + if ( paging_mode_translate(currd) && !is_hardware_domain(currd) &&
> + !arch_refcounts_p2m() )
> return -EACCES;
I guess the hook may want naming differently, as both prior parts of
the condition should be needed only on the x86 side, and there (for
PV) there's no p2m involved in the refcounting. Maybe
arch_acquire_resource_check()? And then the comment wants moving into
the x86 hook as well. You may want to consider leaving a more generic
one here...
Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 01/23] x86/ioreq: Prepare IOREQ feature for making it common
2020-11-12 10:58 ` Jan Beulich
@ 2020-11-13 11:09 ` Oleksandr
2020-11-13 11:20 ` Jan Beulich
0 siblings, 1 reply; 109+ messages in thread
From: Oleksandr @ 2020-11-13 11:09 UTC (permalink / raw)
To: Jan Beulich
Cc: Oleksandr Tyshchenko, Paul Durrant, Andrew Cooper,
Roger Pau Monné,
Julien Grall, Stefano Stabellini, Wei Liu, Julien Grall,
xen-devel
On 12.11.20 12:58, Jan Beulich wrote:
Hi Jan.
> On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> As a lot of x86 code can be re-used on Arm later on, this
>> patch makes some preparation to x86/hvm/ioreq.c before moving
>> to the common code. This way we will get a verbatim copy for
>> a code movement in subsequent patch (arch/x86/hvm/ioreq.c
>> will be *just* renamed to common/ioreq).
>>
>> This patch does the following:
>> 1. Introduce *inline* arch_hvm_ioreq_init(), arch_hvm_ioreq_destroy(),
>> arch_hvm_io_completion(), arch_hvm_destroy_ioreq_server() and
>> hvm_ioreq_server_get_type_addr() to abstract arch specific materials.
>> 2 Make hvm_map_mem_type_to_ioreq_server() *inline*. It is not going
>> to be called from the common code.
> As already indicated on another sub-thread, I think some of these
> are too large to be inline functions. Additionally, considering
> their single-use purpose, I don't think they should be placed in
> a header consumed by more than the producer and the sole consumer.
ok, the only reason I made these inline was to achieve a moving of the
whole x86/hvm/ioreq.c to the common code.
I will move some of them back to ioreq.c.
>
>> 3. Make get_ioreq_server() global. It is going to be called from
>> a few places.
> And with this its name ought to change, to fit the general naming
> model of global functions of this subsystem.
I think, with new requirements (making
hvm_map_mem_type_to_ioreq_server() common) this helper
doesn't need to be global. I will make it static again.
>
>> 4. Add IOREQ_STATUS_* #define-s and update candidates for moving.
> This, it seems to me, could be a separate patch.
Well, will do.
>
>> @@ -855,7 +841,7 @@ int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
>>
>> domain_pause(d);
>>
>> - p2m_set_ioreq_server(d, 0, s);
>> + arch_hvm_destroy_ioreq_server(s);
> Iirc there are plans to rename hvm_destroy_ioreq_server() in the
> course of the generalization. If so, this arch hook would imo
> better be named following the new scheme right away.
Could you please clarify, are you speaking about the plans discussed there
"[PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved
function names"?
Copy text for the convenience:
AT least some of the functions touched here would be nice to be
moved to a more consistent new naming scheme right away, to
avoid having to touch all the same places again. I guess ioreq
server functions would be nice to all start with ioreq_server_
and ioreq functions to all start with ioreq_. E.g. ioreq_send()
and ioreq_server_select().
or some other plans I am not aware of?
What I really want to avoid with IOREQ enabling work is the round-trips
related to naming things, this patch series
became quite big (and consumes som time to rebase and test it) and I
expect it to become bigger.
So the arch_hvm_destroy_ioreq_server() should be
arch_ioreq_server_destroy()?
>
>> @@ -1215,7 +1153,7 @@ void hvm_destroy_all_ioreq_servers(struct domain *d)
>> struct hvm_ioreq_server *s;
>> unsigned int id;
>>
>> - if ( !relocate_portio_handler(d, 0xcf8, 0xcf8, 4) )
>> + if ( !arch_hvm_ioreq_destroy(d) )
> There's no ioreq being destroyed here, so I think this wants
> renaming (and again ideally right away following the planned
> new scheme).
Agree that no ioreq being destroyed here. Probably
ioreq_server_check_for_destroy()?
I couldn't think of a better name.
>
>> +static inline int hvm_map_mem_type_to_ioreq_server(struct domain *d,
>> + ioservid_t id,
>> + uint32_t type,
>> + uint32_t flags)
>> +{
>> + struct hvm_ioreq_server *s;
>> + int rc;
>> +
>> + if ( type != HVMMEM_ioreq_server )
>> + return -EINVAL;
>> +
>> + if ( flags & ~XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
>> + return -EINVAL;
>> +
>> + spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
>> +
>> + s = get_ioreq_server(d, id);
>> +
>> + rc = -ENOENT;
>> + if ( !s )
>> + goto out;
>> +
>> + rc = -EPERM;
>> + if ( s->emulator != current->domain )
>> + goto out;
>> +
>> + rc = p2m_set_ioreq_server(d, flags, s);
>> +
>> + out:
>> + spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
>> +
>> + if ( rc == 0 && flags == 0 )
>> + {
>> + struct p2m_domain *p2m = p2m_get_hostp2m(d);
> I realize I may be asking too much, but would it be possible if,
> while moving code, you made simple and likely uncontroversial
> adjustments like adding const here? (Such adjustments would be
> less desirable to make if they increased the size of the patch,
> e.g. if you were touching only nearby code.)
This function as well as one located below won't be moved to this header
for the next version of patch.
ok, will add const.
>
>> + if ( read_atomic(&p2m->ioreq.entry_count) )
>> + p2m_change_entry_type_global(d, p2m_ioreq_server, p2m_ram_rw);
>> + }
>> +
>> + return rc;
>> +}
>> +
>> +static inline int hvm_ioreq_server_get_type_addr(const struct domain *d,
>> + const ioreq_t *p,
>> + uint8_t *type,
>> + uint64_t *addr)
>> +{
>> + uint32_t cf8 = d->arch.hvm.pci_cf8;
> Similarly, for example, neither this nor ...
>
>> + if ( p->type != IOREQ_TYPE_COPY && p->type != IOREQ_TYPE_PIO )
>> + return -EINVAL;
>> +
>> + if ( p->type == IOREQ_TYPE_PIO &&
>> + (p->addr & ~3) == 0xcfc &&
>> + CF8_ENABLED(cf8) )
>> + {
>> + uint32_t x86_fam;
> ... this really need to use a fixed width type - unsigned int is
> going to be quite fine. But since you're only moving this code,
> I guess I'm not going to insist.
Will use unsigned int.
>
>> +static inline bool arch_hvm_ioreq_destroy(struct domain *d)
>> +{
>> + if ( !relocate_portio_handler(d, 0xcf8, 0xcf8, 4) )
>> + return false;
>> +
>> + return true;
> Any reason this cannot simply be
>
> return relocate_portio_handler(d, 0xcf8, 0xcf8, 4);
Yes, good point.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 01/23] x86/ioreq: Prepare IOREQ feature for making it common
2020-11-13 11:09 ` Oleksandr
@ 2020-11-13 11:20 ` Jan Beulich
2020-11-13 12:45 ` Oleksandr
0 siblings, 1 reply; 109+ messages in thread
From: Jan Beulich @ 2020-11-13 11:20 UTC (permalink / raw)
To: Oleksandr
Cc: Oleksandr Tyshchenko, Paul Durrant, Andrew Cooper,
Roger Pau Monné,
Julien Grall, Stefano Stabellini, Wei Liu, Julien Grall,
xen-devel
On 13.11.2020 12:09, Oleksandr wrote:
> On 12.11.20 12:58, Jan Beulich wrote:
>> On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
>>> @@ -855,7 +841,7 @@ int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
>>>
>>> domain_pause(d);
>>>
>>> - p2m_set_ioreq_server(d, 0, s);
>>> + arch_hvm_destroy_ioreq_server(s);
>> Iirc there are plans to rename hvm_destroy_ioreq_server() in the
>> course of the generalization. If so, this arch hook would imo
>> better be named following the new scheme right away.
> Could you please clarify, are you speaking about the plans discussed there
>
> "[PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved
> function names"?
>
> Copy text for the convenience:
> AT least some of the functions touched here would be nice to be
> moved to a more consistent new naming scheme right away, to
> avoid having to touch all the same places again. I guess ioreq
> server functions would be nice to all start with ioreq_server_
> and ioreq functions to all start with ioreq_. E.g. ioreq_send()
> and ioreq_server_select().
>
> or some other plans I am not aware of?
>
>
> What I really want to avoid with IOREQ enabling work is the round-trips
> related to naming things, this patch series
> became quite big (and consumes som time to rebase and test it) and I
> expect it to become bigger.
>
> So the arch_hvm_destroy_ioreq_server() should be
> arch_ioreq_server_destroy()?
I think so, yes. If you want to avoid doing full patches, how
about you simply list the functions / variables you plan to
rename alongside the intended new names? Would likely be easier
for all involved parties.
>>> @@ -1215,7 +1153,7 @@ void hvm_destroy_all_ioreq_servers(struct domain *d)
>>> struct hvm_ioreq_server *s;
>>> unsigned int id;
>>>
>>> - if ( !relocate_portio_handler(d, 0xcf8, 0xcf8, 4) )
>>> + if ( !arch_hvm_ioreq_destroy(d) )
>> There's no ioreq being destroyed here, so I think this wants
>> renaming (and again ideally right away following the planned
>> new scheme).
> Agree that no ioreq being destroyed here. Probably
> ioreq_server_check_for_destroy()?
> I couldn't think of a better name.
"check" implies no change (and d ought to then be const struct
domain *). With the containing function likely becoming
ioreq_server_destroy_all(), arch_ioreq_server_destroy_all()
would come to mind, or arch_ioreq_server_prepare_destroy_all().
>>> +static inline int hvm_map_mem_type_to_ioreq_server(struct domain *d,
>>> + ioservid_t id,
>>> + uint32_t type,
>>> + uint32_t flags)
>>> +{
>>> + struct hvm_ioreq_server *s;
>>> + int rc;
>>> +
>>> + if ( type != HVMMEM_ioreq_server )
>>> + return -EINVAL;
>>> +
>>> + if ( flags & ~XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
>>> + return -EINVAL;
>>> +
>>> + spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
>>> +
>>> + s = get_ioreq_server(d, id);
>>> +
>>> + rc = -ENOENT;
>>> + if ( !s )
>>> + goto out;
>>> +
>>> + rc = -EPERM;
>>> + if ( s->emulator != current->domain )
>>> + goto out;
>>> +
>>> + rc = p2m_set_ioreq_server(d, flags, s);
>>> +
>>> + out:
>>> + spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
>>> +
>>> + if ( rc == 0 && flags == 0 )
>>> + {
>>> + struct p2m_domain *p2m = p2m_get_hostp2m(d);
>> I realize I may be asking too much, but would it be possible if,
>> while moving code, you made simple and likely uncontroversial
>> adjustments like adding const here? (Such adjustments would be
>> less desirable to make if they increased the size of the patch,
>> e.g. if you were touching only nearby code.)
> This function as well as one located below won't be moved to this header
> for the next version of patch.
>
> ok, will add const.
Well, if you don't move the code, then better keep the diff small
and leave things as they are.
Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 01/23] x86/ioreq: Prepare IOREQ feature for making it common
2020-11-13 11:20 ` Jan Beulich
@ 2020-11-13 12:45 ` Oleksandr
2020-11-13 14:31 ` Jan Beulich
0 siblings, 1 reply; 109+ messages in thread
From: Oleksandr @ 2020-11-13 12:45 UTC (permalink / raw)
To: Jan Beulich
Cc: Oleksandr Tyshchenko, Paul Durrant, Andrew Cooper,
Roger Pau Monné,
Julien Grall, Stefano Stabellini, Wei Liu, Julien Grall,
xen-devel
On 13.11.20 13:20, Jan Beulich wrote:
Hi Jan.
> On 13.11.2020 12:09, Oleksandr wrote:
>> On 12.11.20 12:58, Jan Beulich wrote:
>>> On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
>>>> @@ -855,7 +841,7 @@ int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
>>>>
>>>> domain_pause(d);
>>>>
>>>> - p2m_set_ioreq_server(d, 0, s);
>>>> + arch_hvm_destroy_ioreq_server(s);
>>> Iirc there are plans to rename hvm_destroy_ioreq_server() in the
>>> course of the generalization. If so, this arch hook would imo
>>> better be named following the new scheme right away.
>> Could you please clarify, are you speaking about the plans discussed there
>>
>> "[PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved
>> function names"?
>>
>> Copy text for the convenience:
>> AT least some of the functions touched here would be nice to be
>> moved to a more consistent new naming scheme right away, to
>> avoid having to touch all the same places again. I guess ioreq
>> server functions would be nice to all start with ioreq_server_
>> and ioreq functions to all start with ioreq_. E.g. ioreq_send()
>> and ioreq_server_select().
>>
>> or some other plans I am not aware of?
>>
>>
>> What I really want to avoid with IOREQ enabling work is the round-trips
>> related to naming things, this patch series
>> became quite big (and consumes som time to rebase and test it) and I
>> expect it to become bigger.
>>
>> So the arch_hvm_destroy_ioreq_server() should be
>> arch_ioreq_server_destroy()?
> I think so, yes. If you want to avoid doing full patches, how
> about you simply list the functions / variables you plan to
> rename alongside the intended new names? Would likely be easier
> for all involved parties.
I think it is a good idea. I will prepare a list once I analyze all new
comments to this series.
As I understand that only global IOREQ functions need renaming according
to the new scheme,
local ones can be left as is but without "hvm" prefixes of course?
>
>>>> @@ -1215,7 +1153,7 @@ void hvm_destroy_all_ioreq_servers(struct domain *d)
>>>> struct hvm_ioreq_server *s;
>>>> unsigned int id;
>>>>
>>>> - if ( !relocate_portio_handler(d, 0xcf8, 0xcf8, 4) )
>>>> + if ( !arch_hvm_ioreq_destroy(d) )
>>> There's no ioreq being destroyed here, so I think this wants
>>> renaming (and again ideally right away following the planned
>>> new scheme).
>> Agree that no ioreq being destroyed here. Probably
>> ioreq_server_check_for_destroy()?
>> I couldn't think of a better name.
> "check" implies no change (and d ought to then be const struct
> domain *). With the containing function likely becoming
> ioreq_server_destroy_all(), arch_ioreq_server_destroy_all()
> would come to mind, or arch_ioreq_server_prepare_destroy_all().
ok, agree
>
>>>> +static inline int hvm_map_mem_type_to_ioreq_server(struct domain *d,
>>>> + ioservid_t id,
>>>> + uint32_t type,
>>>> + uint32_t flags)
>>>> +{
>>>> + struct hvm_ioreq_server *s;
>>>> + int rc;
>>>> +
>>>> + if ( type != HVMMEM_ioreq_server )
>>>> + return -EINVAL;
>>>> +
>>>> + if ( flags & ~XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
>>>> + return -EINVAL;
>>>> +
>>>> + spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
>>>> +
>>>> + s = get_ioreq_server(d, id);
>>>> +
>>>> + rc = -ENOENT;
>>>> + if ( !s )
>>>> + goto out;
>>>> +
>>>> + rc = -EPERM;
>>>> + if ( s->emulator != current->domain )
>>>> + goto out;
>>>> +
>>>> + rc = p2m_set_ioreq_server(d, flags, s);
>>>> +
>>>> + out:
>>>> + spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
>>>> +
>>>> + if ( rc == 0 && flags == 0 )
>>>> + {
>>>> + struct p2m_domain *p2m = p2m_get_hostp2m(d);
>>> I realize I may be asking too much, but would it be possible if,
>>> while moving code, you made simple and likely uncontroversial
>>> adjustments like adding const here? (Such adjustments would be
>>> less desirable to make if they increased the size of the patch,
>>> e.g. if you were touching only nearby code.)
>> This function as well as one located below won't be moved to this header
>> for the next version of patch.
>>
>> ok, will add const.
> Well, if you don't move the code, then better keep the diff small
> and leave things as they are.
ok, in case I will have to move that code for any reason, I will take
suggestions into the account.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 02/23] xen/ioreq: Make x86's IOREQ feature common
2020-11-12 11:11 ` Jan Beulich
@ 2020-11-13 13:11 ` Oleksandr
0 siblings, 0 replies; 109+ messages in thread
From: Oleksandr @ 2020-11-13 13:11 UTC (permalink / raw)
To: Jan Beulich
Cc: Oleksandr Tyshchenko, Andrew Cooper, George Dunlap, Ian Jackson,
Julien Grall, Stefano Stabellini, Wei Liu, Roger Pau Monné,
Paul Durrant, Tim Deegan, Julien Grall, xen-devel
On 12.11.20 13:11, Jan Beulich wrote:
Hi Jan
> On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> As a lot of x86 code can be re-used on Arm later on, this patch
>> moves previously prepared x86/hvm/ioreq.c to the common code.
>>
>> The common IOREQ feature is supposed to be built with IOREQ_SERVER
>> option enabled, which is selected for x86's config HVM for now.
>>
>> In order to avoid having a gigantic patch here, the subsequent
>> patches will update remaining bits in the common code step by step:
>> - Make IOREQ related structs/materials common
>> - Drop the "hvm" prefixes and infixes
>> - Remove layering violation by moving corresponding fields
>> out of *arch.hvm* or abstracting away accesses to them
>>
>> This support is going to be used on Arm to be able run device
>> emulator outside of Xen hypervisor.
>>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> CC: Julien Grall <julien.grall@arm.com>
>>
>> ---
>> Please note, this is a split/cleanup/hardening of Julien's PoC:
>> "Add support for Guest IO forwarding to a device emulator"
>>
>> ***
>> Please note, this patch depends on the following which is
>> on review:
>> https://patchwork.kernel.org/patch/11816689/
>> ***
>>
>> Changes RFC -> V1:
>> - was split into three patches:
>> - x86/ioreq: Prepare IOREQ feature for making it common
>> - xen/ioreq: Make x86's IOREQ feature common
>> - xen/ioreq: Make x86's hvm_ioreq_needs_completion() common
>> - update MAINTAINERS file
>> - do not use a separate subdir for the IOREQ stuff, move it to:
>> - xen/common/ioreq.c
>> - xen/include/xen/ioreq.h
>> - update x86's files to include xen/ioreq.h
>> - remove unneeded headers in arch/x86/hvm/ioreq.c
>> - re-order the headers alphabetically in common/ioreq.c
>> - update common/ioreq.c according to the newly introduced arch functions:
>> arch_hvm_destroy_ioreq_server()/arch_handle_hvm_io_completion()
>>
>> Changes V1 -> V2:
>> - update patch description
>> - make everything needed in the previous patch to achieve
>> a truly rename here
>> - don't include unnecessary headers from asm-x86/hvm/ioreq.h
>> and xen/ioreq.h
>> - use __XEN_IOREQ_H__ instead of __IOREQ_H__
>> - move get_ioreq_server() to common/ioreq.c
>> ---
>> MAINTAINERS | 8 +-
>> xen/arch/x86/Kconfig | 1 +
>> xen/arch/x86/hvm/Makefile | 1 -
>> xen/arch/x86/hvm/ioreq.c | 1422 ---------------------------------------
>> xen/arch/x86/mm.c | 2 +-
>> xen/arch/x86/mm/shadow/common.c | 2 +-
>> xen/common/Kconfig | 3 +
>> xen/common/Makefile | 1 +
>> xen/common/ioreq.c | 1422 +++++++++++++++++++++++++++++++++++++++
>> xen/include/asm-x86/hvm/ioreq.h | 39 +-
>> xen/include/xen/ioreq.h | 71 ++
>> 11 files changed, 1509 insertions(+), 1463 deletions(-)
>> delete mode 100644 xen/arch/x86/hvm/ioreq.c
>> create mode 100644 xen/common/ioreq.c
>> create mode 100644 xen/include/xen/ioreq.h
> Iirc I've previously asked to make sure the diff here gets created with
> git's rename detection enabled, so we wouldn't see 1422 lines deleted
> and 1422 lines added, _hoping_ they're all the same (or going through
> the extra steps needed to compare old and new code), but instead seeing
> just the diff between old and new files (which in the ideal case would
> then be empty).
This is my fault, I misread this last time. I have just rechecked this
patch again.
git config diff.renames false
MAINTAINERS | 8 +-
xen/arch/x86/Kconfig | 1 +
xen/arch/x86/hvm/Makefile | 1 -
xen/arch/x86/hvm/ioreq.c | 1544
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
xen/arch/x86/mm.c | 2 +-
xen/arch/x86/mm/shadow/common.c | 2 +-
xen/common/Kconfig | 3 +
xen/common/Makefile | 1 +
xen/common/ioreq.c | 1544
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
xen/include/asm-x86/hvm/ioreq.h | 38 +----
xen/include/xen/ioreq.h | 73 +++++++++
11 files changed, 1633 insertions(+), 1584 deletions(-)
git config diff.renames true
MAINTAINERS | 8 +++++++-
xen/arch/x86/Kconfig | 1 +
xen/arch/x86/hvm/Makefile | 1 -
xen/arch/x86/mm.c | 2 +-
xen/arch/x86/mm/shadow/common.c | 2 +-
xen/common/Kconfig | 3 +++
xen/common/Makefile | 1 +
xen/{arch/x86/hvm => common}/ioreq.c | 0
xen/include/asm-x86/hvm/ioreq.h | 38
++------------------------------------
xen/include/xen/ioreq.h | 73
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
10 files changed, 89 insertions(+), 40 deletions(-)
Although it is not needed for the patch in question anymore (since there
won't be a rename here) I will try to keep that in mind for the future
patches.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 07/23] xen/ioreq: Move x86's ioreq_gfn(server) to struct domain
2020-11-12 11:21 ` Jan Beulich
@ 2020-11-13 14:05 ` Oleksandr
0 siblings, 0 replies; 109+ messages in thread
From: Oleksandr @ 2020-11-13 14:05 UTC (permalink / raw)
To: Jan Beulich
Cc: Oleksandr Tyshchenko, Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Paul Durrant, Julien Grall, xen-devel
On 12.11.20 13:21, Jan Beulich wrote:
Hi Jan
> On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
>> --- a/xen/include/asm-x86/hvm/ioreq.h
>> +++ b/xen/include/asm-x86/hvm/ioreq.h
>> @@ -77,7 +77,7 @@ static inline int hvm_map_mem_type_to_ioreq_server(struct domain *d,
>> if ( flags & ~XEN_DMOP_IOREQ_MEM_ACCESS_WRITE )
>> return -EINVAL;
>>
>> - spin_lock_recursive(&d->arch.hvm.ioreq_server.lock);
>> + spin_lock_recursive(&d->ioreq_server.lock);
>>
>> s = get_ioreq_server(d, id);
>>
>> @@ -92,7 +92,7 @@ static inline int hvm_map_mem_type_to_ioreq_server(struct domain *d,
>> rc = p2m_set_ioreq_server(d, flags, s);
>>
>> out:
>> - spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
>> + spin_unlock_recursive(&d->ioreq_server.lock);
>>
>> if ( rc == 0 && flags == 0 )
>> {
>
> Does this build at this point, when !CONFIG_IOREQ_SERVER? Patch 1
> moves the code here without guards, and patch 2, when introducing
> the Kconfig symbol, doesn't add guards here. I admit I didn't
> check further intermediate patches.
Yes.
I can confirm I checked x86 patch by patch with CONFIG_IOREQ_SERVER, as
for !CONFIG_IOREQ_SERVER I can't recollect to be 100% sure, but likely I
tested also patch by patch.
Anyway, I have just rechecked.
Probably it is because this header isn't in use with
!CONFIG_IOREQ_SERVER since all users are x86/hvm/* and common/ioreq.c
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 09/23] xen/dm: Make x86's DM feature common
2020-11-12 11:32 ` Jan Beulich
@ 2020-11-13 14:28 ` Oleksandr
0 siblings, 0 replies; 109+ messages in thread
From: Oleksandr @ 2020-11-13 14:28 UTC (permalink / raw)
To: Jan Beulich
Cc: Julien Grall, Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Daniel De Graaf, Oleksandr Tyshchenko,
xen-devel
On 12.11.20 13:32, Jan Beulich wrote:
Hi Jan
> On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
>> From: Julien Grall <julien.grall@arm.com>
>>
>> As a lot of x86 code can be re-used on Arm later on, this patch
>> splits devicemodel support into common and arch specific parts.
>>
>> The common DM feature is supposed to be built with IOREQ_SERVER
>> option enabled (as well as the IOREQ feature), which is selected
>> for x86's config HVM for now.
> Did you consider doing it the other way around? It would seem
> more natural to have the top level dm-op handling arch-specific
> and call into e.g. ioreq_server_dm_op() for otherwise unhandled
> ops, just like e.g. do_domctl() calls into iommu_do_domctl()
> (indirectly via arch_do_domctl()). I ask because in the long
> run I expect the ioreq server sub-ops to only be a small part
> of the overall set of dm-ops; already now it's 7 out of 18 if
> I got the counting right.
>
> This would then also leave compat_dm_op() in x86 code.
>
> But yes, there are also downsides with this alternative.
No, I didn't consider. I separated the proposed DM changes from Julien's
patch without modifying the logic.
My changes on top (except rebasing of course) are updating XSM code and
introducing a xen/dm.h for definitions.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 01/23] x86/ioreq: Prepare IOREQ feature for making it common
2020-11-13 12:45 ` Oleksandr
@ 2020-11-13 14:31 ` Jan Beulich
0 siblings, 0 replies; 109+ messages in thread
From: Jan Beulich @ 2020-11-13 14:31 UTC (permalink / raw)
To: Oleksandr
Cc: Oleksandr Tyshchenko, Paul Durrant, Andrew Cooper,
Roger Pau Monné,
Julien Grall, Stefano Stabellini, Wei Liu, Julien Grall,
xen-devel
On 13.11.2020 13:45, Oleksandr wrote:
> On 13.11.20 13:20, Jan Beulich wrote:
>> On 13.11.2020 12:09, Oleksandr wrote:
>>> On 12.11.20 12:58, Jan Beulich wrote:
>>>> On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
>>>>> @@ -855,7 +841,7 @@ int hvm_destroy_ioreq_server(struct domain *d, ioservid_t id)
>>>>>
>>>>> domain_pause(d);
>>>>>
>>>>> - p2m_set_ioreq_server(d, 0, s);
>>>>> + arch_hvm_destroy_ioreq_server(s);
>>>> Iirc there are plans to rename hvm_destroy_ioreq_server() in the
>>>> course of the generalization. If so, this arch hook would imo
>>>> better be named following the new scheme right away.
>>> Could you please clarify, are you speaking about the plans discussed there
>>>
>>> "[PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved
>>> function names"?
>>>
>>> Copy text for the convenience:
>>> AT least some of the functions touched here would be nice to be
>>> moved to a more consistent new naming scheme right away, to
>>> avoid having to touch all the same places again. I guess ioreq
>>> server functions would be nice to all start with ioreq_server_
>>> and ioreq functions to all start with ioreq_. E.g. ioreq_send()
>>> and ioreq_server_select().
>>>
>>> or some other plans I am not aware of?
>>>
>>>
>>> What I really want to avoid with IOREQ enabling work is the round-trips
>>> related to naming things, this patch series
>>> became quite big (and consumes som time to rebase and test it) and I
>>> expect it to become bigger.
>>>
>>> So the arch_hvm_destroy_ioreq_server() should be
>>> arch_ioreq_server_destroy()?
>> I think so, yes. If you want to avoid doing full patches, how
>> about you simply list the functions / variables you plan to
>> rename alongside the intended new names? Would likely be easier
>> for all involved parties.
> I think it is a good idea. I will prepare a list once I analyze all new
> comments to this series.
> As I understand that only global IOREQ functions need renaming according
> to the new scheme,
> local ones can be left as is but without "hvm" prefixes of course?
Please apply common sense: Static ones, if you have to drop a
hvm_ prefix, may in some cases better further be renamed as well,
when their names aren't really in line with their purpose
(anymore). But yes, following a consistent naming model is more
relevant for non-static functions.
Jan
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 10/23] xen/mm: Make x86's XENMEM_resource_ioreq_server handling common
2020-11-12 11:40 ` Jan Beulich
@ 2020-11-13 15:00 ` Oleksandr
0 siblings, 0 replies; 109+ messages in thread
From: Oleksandr @ 2020-11-13 15:00 UTC (permalink / raw)
To: Jan Beulich
Cc: Julien Grall, Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Volodymyr Babchuk, Oleksandr Tyshchenko,
xen-devel
On 12.11.20 13:40, Jan Beulich wrote:
Hi Jan
> On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
>> --- a/xen/common/memory.c
>> +++ b/xen/common/memory.c
>> @@ -30,6 +30,10 @@
>> #include <public/memory.h>
>> #include <xsm/xsm.h>
>>
>> +#ifdef CONFIG_IOREQ_SERVER
>> +#include <xen/ioreq.h>
>> +#endif
> Preferably #ifdef-s would not be needed here. If any, they'd better
> live in xen/ioreq.h itself then.
ok
>
>> @@ -1045,6 +1049,38 @@ static int acquire_grant_table(struct domain *d, unsigned int id,
>> return 0;
>> }
>>
>> +#ifdef CONFIG_IOREQ_SERVER
> To limit the number of #ifdef-s, could this be moved ...
>
>> +static int acquire_ioreq_server(struct domain *d,
>> + unsigned int id,
>> + unsigned long frame,
>> + unsigned int nr_frames,
>> + xen_pfn_t mfn_list[])
>> +{
> ... here such that ...
>
>> @@ -1103,9 +1139,14 @@ static int acquire_resource(
>> mfn_list);
>> break;
>>
>> +#ifdef CONFIG_IOREQ_SERVER
>> + case XENMEM_resource_ioreq_server:
>> + rc = acquire_ioreq_server(d, xmar.id, xmar.frame, xmar.nr_frames,
>> + mfn_list);
>> + break;
>> +#endif
> ... the ones here then can be dropped?
I think yes, that would be better.
>
>> default:
> Also you'll want to a blank line between the new case statement and
> the "default:".
ok
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 14/23] arm/ioreq: Introduce arch specific bits for IOREQ/DM features
2020-11-12 11:48 ` Jan Beulich
@ 2020-11-13 15:48 ` Oleksandr
0 siblings, 0 replies; 109+ messages in thread
From: Oleksandr @ 2020-11-13 15:48 UTC (permalink / raw)
To: Jan Beulich
Cc: Julien Grall, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Paul Durrant, Oleksandr Tyshchenko, xen-devel
On 12.11.20 13:48, Jan Beulich wrote:
Hi Jan
> On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
>> --- a/xen/common/ioreq.c
>> +++ b/xen/common/ioreq.c
>> @@ -18,6 +18,7 @@
>>
>> #include <xen/ctype.h>
>> #include <xen/domain.h>
>> +#include <xen/domain_page.h>
>> #include <xen/event.h>
>> #include <xen/init.h>
>> #include <xen/irq.h>
> There preferably wouldn't be a need to touch non-Arm code in this
> patch. I suppose the added #include could easily be introduced
> e.g. while moving the file?
ok
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved function names
2020-11-12 11:45 ` Jan Beulich
2020-11-12 12:14 ` Paul Durrant
@ 2020-11-13 15:53 ` Oleksandr
2020-11-23 14:39 ` Oleksandr
1 sibling, 1 reply; 109+ messages in thread
From: Oleksandr @ 2020-11-13 15:53 UTC (permalink / raw)
To: Jan Beulich
Cc: Paul Durrant, Oleksandr Tyshchenko, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Julien Grall,
xen-devel
On 12.11.20 13:45, Jan Beulich wrote:
Hi Jan.
> On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> This patch removes "hvm" prefixes and infixes from IOREQ
>> related function names in the common code.
> AT least some of the functions touched here would be nice to be
> moved to a more consistent new naming scheme right away, to
> avoid having to touch all the same places again. I guess ioreq
> server functions would be nice to all start with ioreq_server_
> and ioreq functions to all start with ioreq_. E.g. ioreq_send()
> and ioreq_server_select().
As it was initially discussed at patch #1, ok , I will prepare a list
of the proposed renaming so we could find a common ground I hope.
After that I will make required renaming.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 20/23] xen/ioreq: Make x86's send_invalidate_req() common
2020-11-12 11:55 ` Jan Beulich
@ 2020-11-13 16:03 ` Oleksandr
0 siblings, 0 replies; 109+ messages in thread
From: Oleksandr @ 2020-11-13 16:03 UTC (permalink / raw)
To: Jan Beulich
Cc: Oleksandr Tyshchenko, Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Paul Durrant, Julien Grall, xen-devel
On 12.11.20 13:55, Jan Beulich wrote:
Hi Jan
> On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
>> --- a/xen/include/asm-x86/hvm/io.h
>> +++ b/xen/include/asm-x86/hvm/io.h
>> @@ -97,7 +97,6 @@ bool relocate_portio_handler(
>> unsigned int size);
>>
>> void send_timeoffset_req(unsigned long timeoff);
>> -void send_invalidate_req(void);
>> bool handle_mmio_with_translation(unsigned long gla, unsigned long gpfn,
>> struct npfec);
>> bool handle_pio(uint16_t port, unsigned int size, int dir);
>> diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
>> index 0679fef..aad682f 100644
>> --- a/xen/include/xen/ioreq.h
>> +++ b/xen/include/xen/ioreq.h
>> @@ -126,6 +126,7 @@ struct ioreq_server *select_ioreq_server(struct domain *d,
>> int send_ioreq(struct ioreq_server *s, ioreq_t *proto_p,
>> bool buffered);
>> unsigned int broadcast_ioreq(ioreq_t *p, bool buffered);
>> +void send_invalidate_ioreq(void);
> Again while renaming this function anyway could we see about giving
> it a suitable and consistent name? Maybe
> ioreq_request_mapcache_invalidate() or (to avoid the double "request")
> ioreq_signal_mapcache_invalidate()? Maybe even ioreq_server_...().
iirc send_invalidate_ioreq() was one of the proposed names during V1 review.
But to follow new scheme, I am fine with both the first and second.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 16/23] xen/mm: Handle properly reference in set_foreign_p2m_entry() on Arm
2020-11-12 12:18 ` Jan Beulich
@ 2020-11-13 17:00 ` Oleksandr
0 siblings, 0 replies; 109+ messages in thread
From: Oleksandr @ 2020-11-13 17:00 UTC (permalink / raw)
To: Jan Beulich
Cc: Oleksandr Tyshchenko, Stefano Stabellini, Julien Grall,
Volodymyr Babchuk, Andrew Cooper, George Dunlap, Ian Jackson,
Wei Liu, Roger Pau Monné,
Julien Grall, xen-devel
On 12.11.20 14:18, Jan Beulich wrote:
Hi Jan
> On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
>> --- a/xen/arch/arm/p2m.c
>> +++ b/xen/arch/arm/p2m.c
>> @@ -1380,6 +1380,27 @@ int guest_physmap_remove_page(struct domain *d, gfn_t gfn, mfn_t mfn,
>> return p2m_remove_mapping(d, gfn, (1 << page_order), mfn);
>> }
>>
>> +int set_foreign_p2m_entry(struct domain *d, const struct domain *fd,
>> + unsigned long gfn, mfn_t mfn)
>> +{
>> + struct page_info *page = mfn_to_page(mfn);
>> + int rc;
>> +
>> + if ( !get_page(page, fd) )
>> + return -EINVAL;
>> +
>> + /*
>> + * It is valid to always use p2m_map_foreign_rw here as if this gets
>> + * called that d != fd. A case when d == fd would be rejected by
>> + * rcu_lock_remote_domain_by_id() earlier.
>> + */
> Are you describing things here on the assumption that no new
> callers may surface later on? To catch such, I'd recommend
> adding at least a respective ASSERT().
Agree, will add.
>
>> + rc = guest_physmap_add_entry(d, _gfn(gfn), mfn, 0, p2m_map_foreign_rw);
>> + if ( rc )
>> + put_page(page);
>> +
>> + return 0;
> I can't imagine it's correct to not signal the error to the
> caller(s).
It is not correct, I have just missed to place return rc. Thank you for
the catch.
>
>> --- a/xen/common/memory.c
>> +++ b/xen/common/memory.c
>> @@ -1099,7 +1099,8 @@ static int acquire_resource(
>> * reference counted, it is unsafe to allow mapping of
>> * resource pages unless the caller is the hardware domain.
>> */
>> - if ( paging_mode_translate(currd) && !is_hardware_domain(currd) )
>> + if ( paging_mode_translate(currd) && !is_hardware_domain(currd) &&
>> + !arch_refcounts_p2m() )
>> return -EACCES;
> I guess the hook may want naming differently, as both prior parts of
> the condition should be needed only on the x86 side, and there (for
> PV) there's no p2m involved in the refcounting. Maybe
> arch_acquire_resource_check()? And then the comment wants moving into
> the x86 hook as well. You may want to consider leaving a more generic
> one here...
ok, again, I am fine with the name). Will follow everything suggested
above.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 23/23] [RFC] libxl: Add support for virtio-disk configuration
2020-11-11 4:28 ` Wei Chen
@ 2020-11-13 17:38 ` Oleksandr
0 siblings, 0 replies; 109+ messages in thread
From: Oleksandr @ 2020-11-13 17:38 UTC (permalink / raw)
To: Wei Chen
Cc: xen-devel, Oleksandr Tyshchenko, Ian Jackson, Wei Liu,
Anthony PERARD, Julien Grall, Stefano Stabellini
On 11.11.20 06:28, Wei Chen wrote:
> Hi Oleksandr,
Hi Wei
>>>> An example of domain configuration (two disks are assigned to the guest,
>>>> the latter is in readonly mode):
>>>>
>>>> vdisk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3;ro:/dev/mmcblk1p3' ]
>>>>
>>> Can we keep use the same 'disk' parameter for virtio-disk, but add an option
>> like
>>> "model=virtio-disk"?
>>> For example:
>>> disk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3,model=virtio-disk' ]
>>> Just like what Xen has done for x86 virtio-net.
>> I think, this needs an additional investigation. In general I agree with
>> you that it would be nice to reuse existing 'disk' parameter somehow
>> rather than introducing new one
>> for the same purpose (to handle virtual block device(s)).
>>
>>
>> One note, although both are used for the same purpose they are different
>> in at least one important option.
>>
>> For example:
>> 1. disk = [ 'backend=DomD, phy:/dev/mmcblk0p3, xvda1' ]
>> 2. vdisk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3' ]
>> As you can see existing "disk" parameter contains xvda1, this means that
>> a new device /dev/xvda1 will appear at the guest side, but "vdisk"
>> doesn't contain anything similar. So with Xen PV driver (xen-blkfront)
> Yes, I understand your concerns. But I think specifying a device name
> for virtio disk is not a mandatory requirement. Even if we're using physical
> disks on bare metal machine, we can't guarantee slot#1 disk is always 'sda'.
> So most modern OS are prefer to use blkid to mount filesystem.
>
>> we are able to configure a device name, but with VirtIO solution
>> (virtio-blk) we aren't (at least I don't know how exactly).
>>
> Just my understanding, not exactly accurate:
> The virtio-blk could not get VDEV information for a bus like Xen-bus. So the disk
> ID is allocated dynamically in bus probe progress. The first probed disk will get
> ID 'a'. And then the ID keeps increasing. If we want to specify the disk ID for virtio
> disk, one possible way to do this is to construct a reasonable position on bus
> (fdt node position of virtio mmio device, PCI Function ID of virtio pci block) for
> virtio disk. But it is not always successful, we can't skip 'vda' to specify a virtio
> disk as 'vdb'.
Thank you for the explanation. I got your point regarding using the same
disk parameter and I think it makes sense. Now I am focusing on IOREQ
transport to be available on Arm, and after that (I hope)
I will be able to return to virtio stuff.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 02/23] xen/ioreq: Make x86's IOREQ feature common
2020-11-10 16:45 ` Oleksandr
@ 2020-11-17 14:47 ` Oleksandr
2020-11-17 15:29 ` Paul Durrant
0 siblings, 1 reply; 109+ messages in thread
From: Oleksandr @ 2020-11-17 14:47 UTC (permalink / raw)
To: paul
Cc: xen-devel, 'Oleksandr Tyshchenko',
'Andrew Cooper', 'George Dunlap',
'Ian Jackson', 'Jan Beulich',
'Julien Grall', 'Stefano Stabellini',
'Wei Liu', 'Roger Pau Monné',
'Tim Deegan', 'Julien Grall'
Hi Paul
>
>> The 'legacy' mechanism of mapping magic pages for ioreq servers
>> should remain x86 specific I think that aspect of the code needs to
>> remain behind and not get moved into common code. You could do that
>> in arch specific calls in hvm_ioreq_server_enable/disable() and
>> hvm_get_ioreq_server_info().
> Well, if legacy mechanism is not going to be used for other arch and
> should remain x86 specific, I will try to investigate what should be
> left in x86 code and rework the series.
> As a side note, I am afraid, we won't get a 100% code movement (which
> I managed to achieve here) for the next version of this patch as we
> need arch/x86/hvm/ioreq.c.
I am investigating how to split the code in order to leave the 'legacy'
mechanism x86 specific and have a few questions. Could you please
clarify the following:
1. The split of hvm_ioreq_server_enable/disable() is obvious to me, I
would like to clarify regarding hvm_get_ioreq_server_info().
Is it close to what you had in mind when suggesting the split of
hvm_get_ioreq_server_info() or I just need to abstract
hvm_ioreq_server_map_pages() call only?
(Not completed and non tested)
+/* Called with ioreq_server lock held */
+int arch_ioreq_server_get_info(struct hvm_ioreq_server *s,
+ unsigned long *ioreq_gfn,
+ unsigned long *bufioreq_gfn,
+ evtchn_port_t *bufioreq_port)
+{
+ if ( ioreq_gfn || bufioreq_gfn )
+ {
+ int rc = hvm_ioreq_server_map_pages(s);
+
+ if ( rc )
+ return rc;
+ }
+
+ if ( ioreq_gfn )
+ *ioreq_gfn = gfn_x(s->ioreq.gfn);
+
+ if ( HANDLE_BUFIOREQ(s) )
+ {
+ if ( bufioreq_gfn )
+ *bufioreq_gfn = gfn_x(s->bufioreq.gfn);
+
+ if ( bufioreq_port )
+ *bufioreq_port = s->bufioreq_evtchn;
+ }
+
+ return 0;
+}
+
int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
unsigned long *ioreq_gfn,
unsigned long *bufioreq_gfn,
@@ -916,26 +954,7 @@ int hvm_get_ioreq_server_info(struct domain *d,
ioservid_t id,
if ( s->emulator != current->domain )
goto out;
- if ( ioreq_gfn || bufioreq_gfn )
- {
- rc = hvm_ioreq_server_map_pages(s);
- if ( rc )
- goto out;
- }
-
- if ( ioreq_gfn )
- *ioreq_gfn = gfn_x(s->ioreq.gfn);
-
- if ( HANDLE_BUFIOREQ(s) )
- {
- if ( bufioreq_gfn )
- *bufioreq_gfn = gfn_x(s->bufioreq.gfn);
-
- if ( bufioreq_port )
- *bufioreq_port = s->bufioreq_evtchn;
- }
-
- rc = 0;
+ rc = arch_ioreq_server_get_info(s, ioreq_gfn, bufioreq_gfn,
bufioreq_port);
out:
spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
2. If I understand the code correctly, besides of the above-mentioned
functions the arch specific calls should be in hvm_ioreq_server_init()
and hvm_ioreq_server_deinit() to actually hide
"hvm_ioreq_server_unmap_pages" usage from the common code. I noticed
that the rollback code in hvm_ioreq_server_init() and the whole
hvm_ioreq_server_deinit() have a lot in common except an extra ASSERT()
and hvm_ioreq_server_free_pages() call in the latter. My question is
could we just replace the rollback code by hvm_ioreq_server_deinit()? I
assume an extra hvm_ioreq_server_free_pages() call wouldn't be an issue
here, but I am not sure what to do with the ASSERT, I expect it to be
triggered at such an early stage (so it probably needs moving out of the
hvm_ioreq_server_deinit() (or dropped?) as well as comment needs
updating). I am asking, because this way we would get *a single* arch
hook here which would be arch_ioreq_server_deinit() and remove code
duplication a bit.
Something close to this.
(Not completed and non tested)
@@ -761,18 +771,17 @@ static int hvm_ioreq_server_init(struct
hvm_ioreq_server *s,
return 0;
fail_add:
- hvm_ioreq_server_remove_all_vcpus(s);
- hvm_ioreq_server_unmap_pages(s);
-
- hvm_ioreq_server_free_rangesets(s);
-
- put_domain(s->emulator);
+ hvm_ioreq_server_deinit(s);
return rc;
}
+void arch_ioreq_server_deinit(struct hvm_ioreq_server *s)
+{
+ hvm_ioreq_server_unmap_pages(s);
+}
+
static void hvm_ioreq_server_deinit(struct hvm_ioreq_server *s)
{
- ASSERT(!s->enabled);
hvm_ioreq_server_remove_all_vcpus(s);
/*
@@ -784,7 +793,7 @@ static void hvm_ioreq_server_deinit(struct
hvm_ioreq_server *s)
* the page_info pointer to NULL, meaning the latter will do
* nothing.
*/
- hvm_ioreq_server_unmap_pages(s);
+ arch_ioreq_server_deinit(s);
hvm_ioreq_server_free_pages(s);
hvm_ioreq_server_free_rangesets(s);
put_domain(s->emulator);
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [PATCH V2 02/23] xen/ioreq: Make x86's IOREQ feature common
2020-11-17 14:47 ` Oleksandr
@ 2020-11-17 15:29 ` Paul Durrant
2020-11-17 16:29 ` Oleksandr
0 siblings, 1 reply; 109+ messages in thread
From: Paul Durrant @ 2020-11-17 15:29 UTC (permalink / raw)
To: 'Oleksandr'
Cc: xen-devel, 'Oleksandr Tyshchenko',
'Andrew Cooper', 'George Dunlap',
'Ian Jackson', 'Jan Beulich',
'Julien Grall', 'Stefano Stabellini',
'Wei Liu', 'Roger Pau Monné',
'Tim Deegan', 'Julien Grall'
> -----Original Message-----
> From: Oleksandr <olekstysh@gmail.com>
> Sent: 17 November 2020 14:48
> To: paul@xen.org
> Cc: xen-devel@lists.xenproject.org; 'Oleksandr Tyshchenko' <oleksandr_tyshchenko@epam.com>; 'Andrew
> Cooper' <andrew.cooper3@citrix.com>; 'George Dunlap' <george.dunlap@citrix.com>; 'Ian Jackson'
> <iwj@xenproject.org>; 'Jan Beulich' <jbeulich@suse.com>; 'Julien Grall' <julien@xen.org>; 'Stefano
> Stabellini' <sstabellini@kernel.org>; 'Wei Liu' <wl@xen.org>; 'Roger Pau Monné'
> <roger.pau@citrix.com>; 'Tim Deegan' <tim@xen.org>; 'Julien Grall' <julien.grall@arm.com>
> Subject: Re: [PATCH V2 02/23] xen/ioreq: Make x86's IOREQ feature common
>
>
> Hi Paul
>
Hi Oleksandr,
> >
> >> The 'legacy' mechanism of mapping magic pages for ioreq servers
> >> should remain x86 specific I think that aspect of the code needs to
> >> remain behind and not get moved into common code. You could do that
> >> in arch specific calls in hvm_ioreq_server_enable/disable() and
> >> hvm_get_ioreq_server_info().
> > Well, if legacy mechanism is not going to be used for other arch and
> > should remain x86 specific, I will try to investigate what should be
> > left in x86 code and rework the series.
> > As a side note, I am afraid, we won't get a 100% code movement (which
> > I managed to achieve here) for the next version of this patch as we
> > need arch/x86/hvm/ioreq.c.
>
> I am investigating how to split the code in order to leave the 'legacy'
> mechanism x86 specific and have a few questions. Could you please
> clarify the following:
>
> 1. The split of hvm_ioreq_server_enable/disable() is obvious to me, I
> would like to clarify regarding hvm_get_ioreq_server_info().
> Is it close to what you had in mind when suggesting the split of
> hvm_get_ioreq_server_info() or I just need to abstract
> hvm_ioreq_server_map_pages() call only?
I think it is sufficient to just abstract hvm_ioreq_server_map_pages() (and return -EOPNOTSUPP in the Arm case).
The buf ioreq port should be common.
> (Not completed and non tested)
>
> +/* Called with ioreq_server lock held */
> +int arch_ioreq_server_get_info(struct hvm_ioreq_server *s,
> + unsigned long *ioreq_gfn,
> + unsigned long *bufioreq_gfn,
> + evtchn_port_t *bufioreq_port)
> +{
> + if ( ioreq_gfn || bufioreq_gfn )
> + {
> + int rc = hvm_ioreq_server_map_pages(s);
> +
> + if ( rc )
> + return rc;
> + }
> +
> + if ( ioreq_gfn )
> + *ioreq_gfn = gfn_x(s->ioreq.gfn);
> +
> + if ( HANDLE_BUFIOREQ(s) )
> + {
> + if ( bufioreq_gfn )
> + *bufioreq_gfn = gfn_x(s->bufioreq.gfn);
> +
> + if ( bufioreq_port )
> + *bufioreq_port = s->bufioreq_evtchn;
> + }
> +
> + return 0;
> +}
> +
> int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
> unsigned long *ioreq_gfn,
> unsigned long *bufioreq_gfn,
> @@ -916,26 +954,7 @@ int hvm_get_ioreq_server_info(struct domain *d,
> ioservid_t id,
> if ( s->emulator != current->domain )
> goto out;
>
> - if ( ioreq_gfn || bufioreq_gfn )
> - {
> - rc = hvm_ioreq_server_map_pages(s);
> - if ( rc )
> - goto out;
> - }
> -
> - if ( ioreq_gfn )
> - *ioreq_gfn = gfn_x(s->ioreq.gfn);
> -
> - if ( HANDLE_BUFIOREQ(s) )
> - {
> - if ( bufioreq_gfn )
> - *bufioreq_gfn = gfn_x(s->bufioreq.gfn);
> -
> - if ( bufioreq_port )
> - *bufioreq_port = s->bufioreq_evtchn;
> - }
> -
> - rc = 0;
> + rc = arch_ioreq_server_get_info(s, ioreq_gfn, bufioreq_gfn,
> bufioreq_port);
>
> out:
> spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
>
> 2. If I understand the code correctly, besides of the above-mentioned
> functions the arch specific calls should be in hvm_ioreq_server_init()
> and hvm_ioreq_server_deinit() to actually hide
> "hvm_ioreq_server_unmap_pages" usage from the common code. I noticed
> that the rollback code in hvm_ioreq_server_init() and the whole
> hvm_ioreq_server_deinit() have a lot in common except an extra ASSERT()
> and hvm_ioreq_server_free_pages() call in the latter. My question is
> could we just replace the rollback code by hvm_ioreq_server_deinit()? I
> assume an extra hvm_ioreq_server_free_pages() call wouldn't be an issue
> here, but I am not sure what to do with the ASSERT, I expect it to be
> triggered at such an early stage (so it probably needs moving out of the
> hvm_ioreq_server_deinit() (or dropped?) as well as comment needs
> updating). I am asking, because this way we would get *a single* arch
> hook here which would be arch_ioreq_server_deinit() and remove code
> duplication a bit.
I would arch specific init and deinit, even if one of them does nothing... but then I like symmetry :-)
>
> Something close to this.
> (Not completed and non tested)
>
> @@ -761,18 +771,17 @@ static int hvm_ioreq_server_init(struct
> hvm_ioreq_server *s,
> return 0;
>
> fail_add:
> - hvm_ioreq_server_remove_all_vcpus(s);
> - hvm_ioreq_server_unmap_pages(s);
> -
> - hvm_ioreq_server_free_rangesets(s);
> -
> - put_domain(s->emulator);
> + hvm_ioreq_server_deinit(s);
> return rc;
> }
>
> +void arch_ioreq_server_deinit(struct hvm_ioreq_server *s)
> +{
> + hvm_ioreq_server_unmap_pages(s);
> +}
> +
> static void hvm_ioreq_server_deinit(struct hvm_ioreq_server *s)
> {
> - ASSERT(!s->enabled);
I assume this is the ASSERT you're referring to... There's no way we should be deinit-ing an enabled server so that should remain in common code as is.
Paul
> hvm_ioreq_server_remove_all_vcpus(s);
>
> /*
> @@ -784,7 +793,7 @@ static void hvm_ioreq_server_deinit(struct
> hvm_ioreq_server *s)
> * the page_info pointer to NULL, meaning the latter will do
> * nothing.
> */
> - hvm_ioreq_server_unmap_pages(s);
> + arch_ioreq_server_deinit(s);
> hvm_ioreq_server_free_pages(s);
>
> hvm_ioreq_server_free_rangesets(s);
>
> put_domain(s->emulator);
>
>
> --
>
> Regards,
>
> Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 02/23] xen/ioreq: Make x86's IOREQ feature common
2020-11-17 15:29 ` Paul Durrant
@ 2020-11-17 16:29 ` Oleksandr
2020-11-17 19:43 ` Paul Durrant
0 siblings, 1 reply; 109+ messages in thread
From: Oleksandr @ 2020-11-17 16:29 UTC (permalink / raw)
To: paul
Cc: xen-devel, 'Oleksandr Tyshchenko',
'Andrew Cooper', 'George Dunlap',
'Ian Jackson', 'Jan Beulich',
'Julien Grall', 'Stefano Stabellini',
'Wei Liu', 'Roger Pau Monné',
'Tim Deegan', 'Julien Grall'
On 17.11.20 17:29, Paul Durrant wrote:
Hi Paul
Thank you for the prompt answer.
>>>> The 'legacy' mechanism of mapping magic pages for ioreq servers
>>>> should remain x86 specific I think that aspect of the code needs to
>>>> remain behind and not get moved into common code. You could do that
>>>> in arch specific calls in hvm_ioreq_server_enable/disable() and
>>>> hvm_get_ioreq_server_info().
>>> Well, if legacy mechanism is not going to be used for other arch and
>>> should remain x86 specific, I will try to investigate what should be
>>> left in x86 code and rework the series.
>>> As a side note, I am afraid, we won't get a 100% code movement (which
>>> I managed to achieve here) for the next version of this patch as we
>>> need arch/x86/hvm/ioreq.c.
>> I am investigating how to split the code in order to leave the 'legacy'
>> mechanism x86 specific and have a few questions. Could you please
>> clarify the following:
>>
>> 1. The split of hvm_ioreq_server_enable/disable() is obvious to me, I
>> would like to clarify regarding hvm_get_ioreq_server_info().
>> Is it close to what you had in mind when suggesting the split of
>> hvm_get_ioreq_server_info() or I just need to abstract
>> hvm_ioreq_server_map_pages() call only?
> I think it is sufficient to just abstract hvm_ioreq_server_map_pages() (and return -EOPNOTSUPP in the Arm case).
> The buf ioreq port should be common.
ok, will do.
>
>> (Not completed and non tested)
>>
>> +/* Called with ioreq_server lock held */
>> +int arch_ioreq_server_get_info(struct hvm_ioreq_server *s,
>> + unsigned long *ioreq_gfn,
>> + unsigned long *bufioreq_gfn,
>> + evtchn_port_t *bufioreq_port)
>> +{
>> + if ( ioreq_gfn || bufioreq_gfn )
>> + {
>> + int rc = hvm_ioreq_server_map_pages(s);
>> +
>> + if ( rc )
>> + return rc;
>> + }
>> +
>> + if ( ioreq_gfn )
>> + *ioreq_gfn = gfn_x(s->ioreq.gfn);
>> +
>> + if ( HANDLE_BUFIOREQ(s) )
>> + {
>> + if ( bufioreq_gfn )
>> + *bufioreq_gfn = gfn_x(s->bufioreq.gfn);
>> +
>> + if ( bufioreq_port )
>> + *bufioreq_port = s->bufioreq_evtchn;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> int hvm_get_ioreq_server_info(struct domain *d, ioservid_t id,
>> unsigned long *ioreq_gfn,
>> unsigned long *bufioreq_gfn,
>> @@ -916,26 +954,7 @@ int hvm_get_ioreq_server_info(struct domain *d,
>> ioservid_t id,
>> if ( s->emulator != current->domain )
>> goto out;
>>
>> - if ( ioreq_gfn || bufioreq_gfn )
>> - {
>> - rc = hvm_ioreq_server_map_pages(s);
>> - if ( rc )
>> - goto out;
>> - }
>> -
>> - if ( ioreq_gfn )
>> - *ioreq_gfn = gfn_x(s->ioreq.gfn);
>> -
>> - if ( HANDLE_BUFIOREQ(s) )
>> - {
>> - if ( bufioreq_gfn )
>> - *bufioreq_gfn = gfn_x(s->bufioreq.gfn);
>> -
>> - if ( bufioreq_port )
>> - *bufioreq_port = s->bufioreq_evtchn;
>> - }
>> -
>> - rc = 0;
>> + rc = arch_ioreq_server_get_info(s, ioreq_gfn, bufioreq_gfn,
>> bufioreq_port);
>>
>> out:
>> spin_unlock_recursive(&d->arch.hvm.ioreq_server.lock);
>>
>> 2. If I understand the code correctly, besides of the above-mentioned
>> functions the arch specific calls should be in hvm_ioreq_server_init()
>> and hvm_ioreq_server_deinit() to actually hide
>> "hvm_ioreq_server_unmap_pages" usage from the common code. I noticed
>> that the rollback code in hvm_ioreq_server_init() and the whole
>> hvm_ioreq_server_deinit() have a lot in common except an extra ASSERT()
>> and hvm_ioreq_server_free_pages() call in the latter. My question is
>> could we just replace the rollback code by hvm_ioreq_server_deinit()? I
>> assume an extra hvm_ioreq_server_free_pages() call wouldn't be an issue
>> here, but I am not sure what to do with the ASSERT, I expect it to be
>> triggered at such an early stage (so it probably needs moving out of the
>> hvm_ioreq_server_deinit() (or dropped?) as well as comment needs
>> updating). I am asking, because this way we would get *a single* arch
>> hook here which would be arch_ioreq_server_deinit() and remove code
>> duplication a bit.
> I would arch specific init and deinit, even if one of them does nothing... but then I like symmetry :-)
Both hvm_ioreq_server_init() and hvm_ioreq_server_deinit() call "legacy"
hvm_ioreq_server_unmap_pages()
which we want to be abstracted. The only difference between these two
usages is that the former calls it during rollback only (in case of error).
Taking into the account what has been suggested for question #1 could we
just introduce arch_ioreq_server_unmap_pages() to be called from both
init and deinit?
[Not completed not tested]
@@ -762,7 +772,7 @@ static int hvm_ioreq_server_init(struct
hvm_ioreq_server *s,
fail_add:
hvm_ioreq_server_remove_all_vcpus(s);
- hvm_ioreq_server_unmap_pages(s);
+ arch_ioreq_server_unmap_pages(s);
hvm_ioreq_server_free_rangesets(s);
@@ -776,7 +786,7 @@ static void hvm_ioreq_server_deinit(struct
hvm_ioreq_server *s)
hvm_ioreq_server_remove_all_vcpus(s);
/*
- * NOTE: It is safe to call both hvm_ioreq_server_unmap_pages() and
+ * NOTE: It is safe to call both arch_ioreq_server_unmap_pages() and
* hvm_ioreq_server_free_pages() in that order.
* This is because the former will do nothing if the pages
* are not mapped, leaving the page to be freed by the latter.
@@ -784,7 +794,7 @@ static void hvm_ioreq_server_deinit(struct
hvm_ioreq_server *s)
* the page_info pointer to NULL, meaning the latter will do
* nothing.
*/
- hvm_ioreq_server_unmap_pages(s);
+ arch_ioreq_server_unmap_pages(s);
hvm_ioreq_server_free_pages(s);
hvm_ioreq_server_free_rangesets(s);
@@ -918,7 +928,7 @@ int hvm_get_ioreq_server_info(struct domain *d,
ioservid_t id,
if ( ioreq_gfn || bufioreq_gfn )
{
- rc = hvm_ioreq_server_map_pages(s);
+ rc = arch_ioreq_server_map_pages(s);
if ( rc )
goto out;
}
So looks like for leaving legacy mechanism x86 specific we need 4 new
arch callbacks:
- arch_ioreq_server_enable
- arch_ioreq_server_disable
- arch_ioreq_server_map_pages
- arch_ioreq_server_unmap_pages
>
>> Something close to this.
>> (Not completed and non tested)
>>
>> @@ -761,18 +771,17 @@ static int hvm_ioreq_server_init(struct
>> hvm_ioreq_server *s,
>> return 0;
>>
>> fail_add:
>> - hvm_ioreq_server_remove_all_vcpus(s);
>> - hvm_ioreq_server_unmap_pages(s);
>> -
>> - hvm_ioreq_server_free_rangesets(s);
>> -
>> - put_domain(s->emulator);
>> + hvm_ioreq_server_deinit(s);
>> return rc;
>> }
>>
>> +void arch_ioreq_server_deinit(struct hvm_ioreq_server *s)
>> +{
>> + hvm_ioreq_server_unmap_pages(s);
>> +}
>> +
>> static void hvm_ioreq_server_deinit(struct hvm_ioreq_server *s)
>> {
>> - ASSERT(!s->enabled);
> I assume this is the ASSERT you're referring to... There's no way we should be deinit-ing an enabled server so that should remain in common code as is.
ok, I agree.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [PATCH V2 02/23] xen/ioreq: Make x86's IOREQ feature common
2020-11-17 16:29 ` Oleksandr
@ 2020-11-17 19:43 ` Paul Durrant
0 siblings, 0 replies; 109+ messages in thread
From: Paul Durrant @ 2020-11-17 19:43 UTC (permalink / raw)
To: 'Oleksandr'
Cc: xen-devel, 'Oleksandr Tyshchenko',
'Andrew Cooper', 'George Dunlap',
'Ian Jackson', 'Jan Beulich',
'Julien Grall', 'Stefano Stabellini',
'Wei Liu', 'Roger Pau Monné',
'Tim Deegan', 'Julien Grall'
> -----Original Message-----
[snip]
>
> Both hvm_ioreq_server_init() and hvm_ioreq_server_deinit() call "legacy"
> hvm_ioreq_server_unmap_pages()
> which we want to be abstracted. The only difference between these two
> usages is that the former calls it during rollback only (in case of error).
> Taking into the account what has been suggested for question #1 could we
> just introduce arch_ioreq_server_unmap_pages() to be called from both
> init and deinit?
>
That sounds fine, yes.
>
> [Not completed not tested]
>
> @@ -762,7 +772,7 @@ static int hvm_ioreq_server_init(struct
> hvm_ioreq_server *s,
>
> fail_add:
> hvm_ioreq_server_remove_all_vcpus(s);
> - hvm_ioreq_server_unmap_pages(s);
> + arch_ioreq_server_unmap_pages(s);
>
> hvm_ioreq_server_free_rangesets(s);
>
> @@ -776,7 +786,7 @@ static void hvm_ioreq_server_deinit(struct
> hvm_ioreq_server *s)
> hvm_ioreq_server_remove_all_vcpus(s);
>
> /*
> - * NOTE: It is safe to call both hvm_ioreq_server_unmap_pages() and
> + * NOTE: It is safe to call both arch_ioreq_server_unmap_pages() and
> * hvm_ioreq_server_free_pages() in that order.
> * This is because the former will do nothing if the pages
> * are not mapped, leaving the page to be freed by the latter.
> @@ -784,7 +794,7 @@ static void hvm_ioreq_server_deinit(struct
> hvm_ioreq_server *s)
> * the page_info pointer to NULL, meaning the latter will do
> * nothing.
> */
> - hvm_ioreq_server_unmap_pages(s);
> + arch_ioreq_server_unmap_pages(s);
> hvm_ioreq_server_free_pages(s);
>
> hvm_ioreq_server_free_rangesets(s);
> @@ -918,7 +928,7 @@ int hvm_get_ioreq_server_info(struct domain *d,
> ioservid_t id,
>
> if ( ioreq_gfn || bufioreq_gfn )
> {
> - rc = hvm_ioreq_server_map_pages(s);
> + rc = arch_ioreq_server_map_pages(s);
> if ( rc )
> goto out;
> }
>
>
> So looks like for leaving legacy mechanism x86 specific we need 4 new
> arch callbacks:
>
> - arch_ioreq_server_enable
> - arch_ioreq_server_disable
> - arch_ioreq_server_map_pages
> - arch_ioreq_server_unmap_pages
Yes, that looks ok.
Paul
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 07/23] xen/ioreq: Move x86's ioreq_gfn(server) to struct domain
2020-10-15 16:44 ` [PATCH V2 07/23] xen/ioreq: Move x86's ioreq_gfn(server) to struct domain Oleksandr Tyshchenko
2020-11-12 11:21 ` Jan Beulich
@ 2020-11-18 12:09 ` Oleksandr
2020-11-18 12:29 ` Paul Durrant
1 sibling, 1 reply; 109+ messages in thread
From: Oleksandr @ 2020-11-18 12:09 UTC (permalink / raw)
To: Paul Durrant
Cc: xen-devel, Oleksandr Tyshchenko, Jan Beulich, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Julien Grall
Hi Paul.
> --- a/xen/include/xen/sched.h
> +++ b/xen/include/xen/sched.h
> @@ -314,6 +314,8 @@ struct sched_unit {
>
> struct evtchn_port_ops;
>
> +#define MAX_NR_IOREQ_SERVERS 8
> +
> struct domain
> {
> domid_t domain_id;
> @@ -521,6 +523,21 @@ struct domain
> /* Argo interdomain communication support */
> struct argo_domain *argo;
> #endif
> +
> +#ifdef CONFIG_IOREQ_SERVER
> + /* Guest page range used for non-default ioreq servers */
> + struct {
> + unsigned long base;
> + unsigned long mask;
> + unsigned long legacy_mask; /* indexed by HVM param number */
> + } ioreq_gfn;
I assume the whole ioreq_gfn struct doesn't need to be here. According
to the new requirement to leave legacy interface x86 specific,
these fields won't be used in common code anymore. I will move ioreq_gfn
struct back to hvm_domain. Please confirm.
> +
> + /* Lock protects all other values in the sub-struct and the default */
> + struct {
> + spinlock_t lock;
> + struct ioreq_server *server[MAX_NR_IOREQ_SERVERS];
> + } ioreq_server;
> +#endif
> };
>
> static inline struct page_list_head *page_to_list(
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [PATCH V2 07/23] xen/ioreq: Move x86's ioreq_gfn(server) to struct domain
2020-11-18 12:09 ` Oleksandr
@ 2020-11-18 12:29 ` Paul Durrant
0 siblings, 0 replies; 109+ messages in thread
From: Paul Durrant @ 2020-11-18 12:29 UTC (permalink / raw)
To: 'Oleksandr'
Cc: xen-devel, 'Oleksandr Tyshchenko', 'Jan Beulich',
'Andrew Cooper', 'Roger Pau Monné',
'Wei Liu', 'George Dunlap', 'Ian Jackson',
'Julien Grall', 'Stefano Stabellini',
'Julien Grall'
> -----Original Message-----
> From: Oleksandr <olekstysh@gmail.com>
> Sent: 18 November 2020 12:09
> To: Paul Durrant <paul@xen.org>
> Cc: xen-devel@lists.xenproject.org; Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Jan Beulich
> <jbeulich@suse.com>; Andrew Cooper <andrew.cooper3@citrix.com>; Roger Pau Monné
> <roger.pau@citrix.com>; Wei Liu <wl@xen.org>; George Dunlap <george.dunlap@citrix.com>; Ian Jackson
> <iwj@xenproject.org>; Julien Grall <julien@xen.org>; Stefano Stabellini <sstabellini@kernel.org>;
> Julien Grall <julien.grall@arm.com>
> Subject: Re: [PATCH V2 07/23] xen/ioreq: Move x86's ioreq_gfn(server) to struct domain
>
>
> Hi Paul.
>
> > --- a/xen/include/xen/sched.h
> > +++ b/xen/include/xen/sched.h
> > @@ -314,6 +314,8 @@ struct sched_unit {
> >
> > struct evtchn_port_ops;
> >
> > +#define MAX_NR_IOREQ_SERVERS 8
> > +
> > struct domain
> > {
> > domid_t domain_id;
> > @@ -521,6 +523,21 @@ struct domain
> > /* Argo interdomain communication support */
> > struct argo_domain *argo;
> > #endif
> > +
> > +#ifdef CONFIG_IOREQ_SERVER
> > + /* Guest page range used for non-default ioreq servers */
> > + struct {
> > + unsigned long base;
> > + unsigned long mask;
> > + unsigned long legacy_mask; /* indexed by HVM param number */
> > + } ioreq_gfn;
>
> I assume the whole ioreq_gfn struct doesn't need to be here. According
> to the new requirement to leave legacy interface x86 specific,
>
> these fields won't be used in common code anymore. I will move ioreq_gfn
> struct back to hvm_domain. Please confirm.
Yes, leave it where it is in struct hvm_domain.
Paul
>
>
> > +
> > + /* Lock protects all other values in the sub-struct and the default */
> > + struct {
> > + spinlock_t lock;
> > + struct ioreq_server *server[MAX_NR_IOREQ_SERVERS];
> > + } ioreq_server;
> > +#endif
> > };
> >
> > static inline struct page_list_head *page_to_list(
>
> --
> Regards,
>
> Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved function names
2020-11-13 15:53 ` Oleksandr
@ 2020-11-23 14:39 ` Oleksandr
2020-11-23 14:56 ` Jan Beulich
0 siblings, 1 reply; 109+ messages in thread
From: Oleksandr @ 2020-11-23 14:39 UTC (permalink / raw)
To: Jan Beulich
Cc: Paul Durrant, Oleksandr Tyshchenko, Andrew Cooper,
Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Julien Grall,
xen-devel
Hi Jan.
As it was agreed, below the list of proposed renaming (naming) within
current series.
If there are no objections I will follow the proposed renaming. If any
please let me know.
1. Global (existing):
hvm_map_mem_type_to_ioreq_server -> ioreq_server_map_mem_type
hvm_select_ioreq_server -> ioreq_server_select
hvm_send_ioreq -> ioreq_send
hvm_ioreq_init -> ioreq_init
hvm_destroy_all_ioreq_servers -> ioreq_server_destroy_all
hvm_all_ioreq_servers_add_vcpu -> ioreq_server_add_vcpu_all
hvm_all_ioreq_servers_remove_vcpu -> ioreq_server_remove_vcpu_all
hvm_broadcast_ioreq -> ioreq_broadcast
hvm_create_ioreq_server -> ioreq_server_create
hvm_get_ioreq_server_info -> ioreq_server_get_info
hvm_map_io_range_to_ioreq_server -> ioreq_server_map_io_range
hvm_unmap_io_range_from_ioreq_server -> ioreq_server_unmap_io_range
hvm_set_ioreq_server_state -> ioreq_server_set_state
hvm_destroy_ioreq_server -> ioreq_server_destroy
hvm_get_ioreq_server_frame -> ioreq_server_get_frame
hvm_ioreq_needs_completion -> ioreq_needs_completion
hvm_mmio_first_byte -> ioreq_mmio_first_byte
hvm_mmio_last_byte -> ioreq_mmio_last_byte
send_invalidate_req -> ioreq_signal_mapcache_invalidate
handle_hvm_io_completion -> handle_io_completion
hvm_io_pending -> io_pending
2. Global (new):
arch_io_completion
arch_ioreq_server_map_pages
arch_ioreq_server_unmap_pages
arch_ioreq_server_enable
arch_ioreq_server_disable
arch_ioreq_server_destroy
arch_ioreq_server_map_mem_type
arch_ioreq_server_destroy_all
arch_ioreq_server_get_type_addr
arch_ioreq_init
domain_has_ioreq_server
3. Local (existing) in common ioreq.c:
hvm_alloc_ioreq_mfn -> ioreq_alloc_mfn
hvm_free_ioreq_mfn -> ioreq_free_mfn
hvm_update_ioreq_evtchn -> ioreq_update_evtchn
hvm_ioreq_server_add_vcpu -> ioreq_server_add_vcpu
hvm_ioreq_server_remove_vcpu -> ioreq_server_remove_vcpu
hvm_ioreq_server_remove_all_vcpus -> ioreq_server_remove_all_vcpus
hvm_ioreq_server_alloc_pages -> ioreq_server_alloc_pages
hvm_ioreq_server_free_pages -> ioreq_server_free_pages
hvm_ioreq_server_free_rangesets -> ioreq_server_free_rangesets
hvm_ioreq_server_alloc_rangesets -> ioreq_server_alloc_rangesets
hvm_ioreq_server_enable -> ioreq_server_enable
hvm_ioreq_server_disable -> ioreq_server_disable
hvm_ioreq_server_init -> ioreq_server_init
hvm_ioreq_server_deinit -> ioreq_server_deinit
hvm_send_buffered_ioreq -> ioreq_send_buffered
hvm_wait_for_io -> wait_for_io
4. Local (existing) in x86 ioreq.c:
Everything related to legacy interface (hvm_alloc_legacy_ioreq_gfn, etc)
are going
to remain as is.
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved function names
2020-11-23 14:39 ` Oleksandr
@ 2020-11-23 14:56 ` Jan Beulich
2020-11-23 15:47 ` Oleksandr
0 siblings, 1 reply; 109+ messages in thread
From: Jan Beulich @ 2020-11-23 14:56 UTC (permalink / raw)
To: Oleksandr, Paul Durrant
Cc: Oleksandr Tyshchenko, Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Julien Grall,
xen-devel
On 23.11.2020 15:39, Oleksandr wrote:
> As it was agreed, below the list of proposed renaming (naming) within
> current series.
Thanks for compiling this. A couple of suggestions for consideration:
> 1. Global (existing):
> hvm_map_mem_type_to_ioreq_server -> ioreq_server_map_mem_type
> hvm_select_ioreq_server -> ioreq_server_select
> hvm_send_ioreq -> ioreq_send
> hvm_ioreq_init -> ioreq_init
ioreq_domain_init() (or, imo less desirable domain_ioreq_init())?
> hvm_destroy_all_ioreq_servers -> ioreq_server_destroy_all
> hvm_all_ioreq_servers_add_vcpu -> ioreq_server_add_vcpu_all
> hvm_all_ioreq_servers_remove_vcpu -> ioreq_server_remove_vcpu_all
> hvm_broadcast_ioreq -> ioreq_broadcast
> hvm_create_ioreq_server -> ioreq_server_create
> hvm_get_ioreq_server_info -> ioreq_server_get_info
> hvm_map_io_range_to_ioreq_server -> ioreq_server_map_io_range
> hvm_unmap_io_range_from_ioreq_server -> ioreq_server_unmap_io_range
> hvm_set_ioreq_server_state -> ioreq_server_set_state
> hvm_destroy_ioreq_server -> ioreq_server_destroy
> hvm_get_ioreq_server_frame -> ioreq_server_get_frame
> hvm_ioreq_needs_completion -> ioreq_needs_completion
> hvm_mmio_first_byte -> ioreq_mmio_first_byte
> hvm_mmio_last_byte -> ioreq_mmio_last_byte
> send_invalidate_req -> ioreq_signal_mapcache_invalidate
>
> handle_hvm_io_completion -> handle_io_completion
For this one I'm not sure what to suggest, but I'm not overly happy
with the name.
> hvm_io_pending -> io_pending
vcpu_ioreq_pending() or vcpu_any_ioreq_pending()?
> 2. Global (new):
> arch_io_completion
> arch_ioreq_server_map_pages
> arch_ioreq_server_unmap_pages
> arch_ioreq_server_enable
> arch_ioreq_server_disable
> arch_ioreq_server_destroy
> arch_ioreq_server_map_mem_type
> arch_ioreq_server_destroy_all
> arch_ioreq_server_get_type_addr
> arch_ioreq_init
Assuming this is the arch hook of the similarly named function
further up, a similar adjustment may then be wanted here.
> domain_has_ioreq_server
>
>
> 3. Local (existing) in common ioreq.c:
> hvm_alloc_ioreq_mfn -> ioreq_alloc_mfn
> hvm_free_ioreq_mfn -> ioreq_free_mfn
These two are server functions, so should imo be ioreq_server_...().
However, if they're static (as they're now), no distinguishing
prefix is strictly necessary, i.e. alloc_mfn() and free_mfn() may
be fine. The two names may be too short for Paul's taste, though.
Some similar shortening may be possible for some or all of the ones
below here.
Jan
> hvm_update_ioreq_evtchn -> ioreq_update_evtchn
> hvm_ioreq_server_add_vcpu -> ioreq_server_add_vcpu
> hvm_ioreq_server_remove_vcpu -> ioreq_server_remove_vcpu
> hvm_ioreq_server_remove_all_vcpus -> ioreq_server_remove_all_vcpus
> hvm_ioreq_server_alloc_pages -> ioreq_server_alloc_pages
> hvm_ioreq_server_free_pages -> ioreq_server_free_pages
> hvm_ioreq_server_free_rangesets -> ioreq_server_free_rangesets
> hvm_ioreq_server_alloc_rangesets -> ioreq_server_alloc_rangesets
> hvm_ioreq_server_enable -> ioreq_server_enable
> hvm_ioreq_server_disable -> ioreq_server_disable
> hvm_ioreq_server_init -> ioreq_server_init
> hvm_ioreq_server_deinit -> ioreq_server_deinit
> hvm_send_buffered_ioreq -> ioreq_send_buffered
>
> hvm_wait_for_io -> wait_for_io
>
> 4. Local (existing) in x86 ioreq.c:
> Everything related to legacy interface (hvm_alloc_legacy_ioreq_gfn, etc)
> are going
> to remain as is.
>
>
>
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved function names
2020-11-23 14:56 ` Jan Beulich
@ 2020-11-23 15:47 ` Oleksandr
2020-11-23 15:54 ` Paul Durrant
0 siblings, 1 reply; 109+ messages in thread
From: Oleksandr @ 2020-11-23 15:47 UTC (permalink / raw)
To: Jan Beulich, Paul Durrant
Cc: Oleksandr Tyshchenko, Andrew Cooper, Roger Pau Monné,
Wei Liu, George Dunlap, Ian Jackson, Julien Grall,
Stefano Stabellini, Jun Nakajima, Kevin Tian, Julien Grall,
xen-devel
On 23.11.20 16:56, Jan Beulich wrote:
Hi Jan, Paul
> On 23.11.2020 15:39, Oleksandr wrote:
>> As it was agreed, below the list of proposed renaming (naming) within
>> current series.
> Thanks for compiling this. A couple of suggestions for consideration:
>
>> 1. Global (existing):
>> hvm_map_mem_type_to_ioreq_server -> ioreq_server_map_mem_type
>> hvm_select_ioreq_server -> ioreq_server_select
>> hvm_send_ioreq -> ioreq_send
>> hvm_ioreq_init -> ioreq_init
> ioreq_domain_init() (or, imo less desirable domain_ioreq_init())?
On Arm (for example) I see two variants are present:
1. That starts with subsystem:
- tee_domain_init
- iommu_domain_init
2. Where sybsystem in the middle:
- domain_io_init
- domain_vuart_init
- domain_vtimer_init
If there is no rule, but a matter of taste then I would use
ioreq_domain_init(),
so arch_ioreq_init() wants to be arch_ioreq_domain_init().
>
>> hvm_destroy_all_ioreq_servers -> ioreq_server_destroy_all
>> hvm_all_ioreq_servers_add_vcpu -> ioreq_server_add_vcpu_all
>> hvm_all_ioreq_servers_remove_vcpu -> ioreq_server_remove_vcpu_all
>> hvm_broadcast_ioreq -> ioreq_broadcast
>> hvm_create_ioreq_server -> ioreq_server_create
>> hvm_get_ioreq_server_info -> ioreq_server_get_info
>> hvm_map_io_range_to_ioreq_server -> ioreq_server_map_io_range
>> hvm_unmap_io_range_from_ioreq_server -> ioreq_server_unmap_io_range
>> hvm_set_ioreq_server_state -> ioreq_server_set_state
>> hvm_destroy_ioreq_server -> ioreq_server_destroy
>> hvm_get_ioreq_server_frame -> ioreq_server_get_frame
>> hvm_ioreq_needs_completion -> ioreq_needs_completion
>> hvm_mmio_first_byte -> ioreq_mmio_first_byte
>> hvm_mmio_last_byte -> ioreq_mmio_last_byte
>> send_invalidate_req -> ioreq_signal_mapcache_invalidate
>>
>> handle_hvm_io_completion -> handle_io_completion
> For this one I'm not sure what to suggest, but I'm not overly happy
> with the name.
I also failed to find a better name. Probably ioreq_ or vcpu_ioreq_
prefix wants to be added here?
>
>> hvm_io_pending -> io_pending
> vcpu_ioreq_pending() or vcpu_any_ioreq_pending()?
I am fine with vcpu_ioreq_pending()
>
>> 2. Global (new):
>> arch_io_completion
>> arch_ioreq_server_map_pages
>> arch_ioreq_server_unmap_pages
>> arch_ioreq_server_enable
>> arch_ioreq_server_disable
>> arch_ioreq_server_destroy
>> arch_ioreq_server_map_mem_type
>> arch_ioreq_server_destroy_all
>> arch_ioreq_server_get_type_addr
>> arch_ioreq_init
> Assuming this is the arch hook of the similarly named function
> further up, a similar adjustment may then be wanted here.
Yes.
>
>> domain_has_ioreq_server
>>
>>
>> 3. Local (existing) in common ioreq.c:
>> hvm_alloc_ioreq_mfn -> ioreq_alloc_mfn
>> hvm_free_ioreq_mfn -> ioreq_free_mfn
> These two are server functions, so should imo be ioreq_server_...().
ok, but ...
> However, if they're static (as they're now), no distinguishing
> prefix is strictly necessary, i.e. alloc_mfn() and free_mfn() may
> be fine. The two names may be too short for Paul's taste, though.
> Some similar shortening may be possible for some or all of the ones
... In general I would be fine with any option. However, using the
shortening rule for all
we are going to end up with single-word function names (enable, init, etc).
So I would prefer to leave locals as is (but dropping hvm prefixes of
course and
clarify ioreq_server_alloc_mfn/ioreq_server_free_mfn).
Paul, Jan what do you think?
> below here.
>
> Jan
>
>> hvm_update_ioreq_evtchn -> ioreq_update_evtchn
>> hvm_ioreq_server_add_vcpu -> ioreq_server_add_vcpu
>> hvm_ioreq_server_remove_vcpu -> ioreq_server_remove_vcpu
>> hvm_ioreq_server_remove_all_vcpus -> ioreq_server_remove_all_vcpus
>> hvm_ioreq_server_alloc_pages -> ioreq_server_alloc_pages
>> hvm_ioreq_server_free_pages -> ioreq_server_free_pages
>> hvm_ioreq_server_free_rangesets -> ioreq_server_free_rangesets
>> hvm_ioreq_server_alloc_rangesets -> ioreq_server_alloc_rangesets
>> hvm_ioreq_server_enable -> ioreq_server_enable
>> hvm_ioreq_server_disable -> ioreq_server_disable
>> hvm_ioreq_server_init -> ioreq_server_init
>> hvm_ioreq_server_deinit -> ioreq_server_deinit
>> hvm_send_buffered_ioreq -> ioreq_send_buffered
>>
>> hvm_wait_for_io -> wait_for_io
>>
>> 4. Local (existing) in x86 ioreq.c:
>> Everything related to legacy interface (hvm_alloc_legacy_ioreq_gfn, etc)
>> are going
>> to remain as is.
>>
>>
>>
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
* RE: [PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved function names
2020-11-23 15:47 ` Oleksandr
@ 2020-11-23 15:54 ` Paul Durrant
2020-11-23 16:36 ` Oleksandr
0 siblings, 1 reply; 109+ messages in thread
From: Paul Durrant @ 2020-11-23 15:54 UTC (permalink / raw)
To: 'Oleksandr', 'Jan Beulich'
Cc: 'Oleksandr Tyshchenko', 'Andrew Cooper',
'Roger Pau Monné', 'Wei Liu',
'George Dunlap', 'Ian Jackson',
'Julien Grall', 'Stefano Stabellini',
'Jun Nakajima', 'Kevin Tian',
'Julien Grall',
xen-devel
> -----Original Message-----
> From: Oleksandr <olekstysh@gmail.com>
> Sent: 23 November 2020 15:48
> To: Jan Beulich <jbeulich@suse.com>; Paul Durrant <paul@xen.org>
> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Andrew Cooper <andrew.cooper3@citrix.com>;
> Roger Pau Monné <roger.pau@citrix.com>; Wei Liu <wl@xen.org>; George Dunlap
> <george.dunlap@citrix.com>; Ian Jackson <iwj@xenproject.org>; Julien Grall <julien@xen.org>; Stefano
> Stabellini <sstabellini@kernel.org>; Jun Nakajima <jun.nakajima@intel.com>; Kevin Tian
> <kevin.tian@intel.com>; Julien Grall <julien.grall@arm.com>; xen-devel@lists.xenproject.org
> Subject: Re: [PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved function names
>
>
> On 23.11.20 16:56, Jan Beulich wrote:
>
> Hi Jan, Paul
>
> > On 23.11.2020 15:39, Oleksandr wrote:
> >> As it was agreed, below the list of proposed renaming (naming) within
> >> current series.
> > Thanks for compiling this. A couple of suggestions for consideration:
> >
> >> 1. Global (existing):
> >> hvm_map_mem_type_to_ioreq_server -> ioreq_server_map_mem_type
> >> hvm_select_ioreq_server -> ioreq_server_select
> >> hvm_send_ioreq -> ioreq_send
> >> hvm_ioreq_init -> ioreq_init
> > ioreq_domain_init() (or, imo less desirable domain_ioreq_init())?
> On Arm (for example) I see two variants are present:
> 1. That starts with subsystem:
> - tee_domain_init
> - iommu_domain_init
>
>
> 2. Where sybsystem in the middle:
> - domain_io_init
> - domain_vuart_init
> - domain_vtimer_init
>
> If there is no rule, but a matter of taste then I would use
> ioreq_domain_init(),
> so arch_ioreq_init() wants to be arch_ioreq_domain_init().
>
> >
> >> hvm_destroy_all_ioreq_servers -> ioreq_server_destroy_all
> >> hvm_all_ioreq_servers_add_vcpu -> ioreq_server_add_vcpu_all
> >> hvm_all_ioreq_servers_remove_vcpu -> ioreq_server_remove_vcpu_all
> >> hvm_broadcast_ioreq -> ioreq_broadcast
> >> hvm_create_ioreq_server -> ioreq_server_create
> >> hvm_get_ioreq_server_info -> ioreq_server_get_info
> >> hvm_map_io_range_to_ioreq_server -> ioreq_server_map_io_range
> >> hvm_unmap_io_range_from_ioreq_server -> ioreq_server_unmap_io_range
> >> hvm_set_ioreq_server_state -> ioreq_server_set_state
> >> hvm_destroy_ioreq_server -> ioreq_server_destroy
> >> hvm_get_ioreq_server_frame -> ioreq_server_get_frame
> >> hvm_ioreq_needs_completion -> ioreq_needs_completion
> >> hvm_mmio_first_byte -> ioreq_mmio_first_byte
> >> hvm_mmio_last_byte -> ioreq_mmio_last_byte
> >> send_invalidate_req -> ioreq_signal_mapcache_invalidate
> >>
> >> handle_hvm_io_completion -> handle_io_completion
> > For this one I'm not sure what to suggest, but I'm not overly happy
> > with the name.
>
> I also failed to find a better name. Probably ioreq_ or vcpu_ioreq_
> prefix wants to be added here?
>
>
> >
> >> hvm_io_pending -> io_pending
> > vcpu_ioreq_pending() or vcpu_any_ioreq_pending()?
>
> I am fine with vcpu_ioreq_pending()
>
...in which case vcpu_ioreq_handle_completion() seems like a reasonable choice.
>
> >
> >> 2. Global (new):
> >> arch_io_completion
> >> arch_ioreq_server_map_pages
> >> arch_ioreq_server_unmap_pages
> >> arch_ioreq_server_enable
> >> arch_ioreq_server_disable
> >> arch_ioreq_server_destroy
> >> arch_ioreq_server_map_mem_type
> >> arch_ioreq_server_destroy_all
> >> arch_ioreq_server_get_type_addr
> >> arch_ioreq_init
> > Assuming this is the arch hook of the similarly named function
> > further up, a similar adjustment may then be wanted here.
>
> Yes.
>
>
> >
> >> domain_has_ioreq_server
> >>
> >>
> >> 3. Local (existing) in common ioreq.c:
> >> hvm_alloc_ioreq_mfn -> ioreq_alloc_mfn
> >> hvm_free_ioreq_mfn -> ioreq_free_mfn
> > These two are server functions, so should imo be ioreq_server_...().
>
> ok, but ...
>
>
> > However, if they're static (as they're now), no distinguishing
> > prefix is strictly necessary, i.e. alloc_mfn() and free_mfn() may
> > be fine. The two names may be too short for Paul's taste, though.
> > Some similar shortening may be possible for some or all of the ones
>
>
> ... In general I would be fine with any option. However, using the
> shortening rule for all
> we are going to end up with single-word function names (enable, init, etc).
> So I would prefer to leave locals as is (but dropping hvm prefixes of
> course and
> clarify ioreq_server_alloc_mfn/ioreq_server_free_mfn).
>
> Paul, Jan what do you think?
I prefer ioreq_server_alloc_mfn/ioreq_server_free_mfn. The problem with shortening is that function names become ambiguous within the source base and hence harder to find.
Paul
^ permalink raw reply [flat|nested] 109+ messages in thread
* Re: [PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved function names
2020-11-23 15:54 ` Paul Durrant
@ 2020-11-23 16:36 ` Oleksandr
0 siblings, 0 replies; 109+ messages in thread
From: Oleksandr @ 2020-11-23 16:36 UTC (permalink / raw)
To: paul
Cc: 'Jan Beulich', 'Oleksandr Tyshchenko',
'Andrew Cooper', 'Roger Pau Monné',
'Wei Liu', 'George Dunlap', 'Ian Jackson',
'Julien Grall', 'Stefano Stabellini',
'Jun Nakajima', 'Kevin Tian',
'Julien Grall',
xen-devel
On 23.11.20 17:54, Paul Durrant wrote:
Hi Paul
>> -----Original Message-----
>> From: Oleksandr <olekstysh@gmail.com>
>> Sent: 23 November 2020 15:48
>> To: Jan Beulich <jbeulich@suse.com>; Paul Durrant <paul@xen.org>
>> Cc: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>; Andrew Cooper <andrew.cooper3@citrix.com>;
>> Roger Pau Monné <roger.pau@citrix.com>; Wei Liu <wl@xen.org>; George Dunlap
>> <george.dunlap@citrix.com>; Ian Jackson <iwj@xenproject.org>; Julien Grall <julien@xen.org>; Stefano
>> Stabellini <sstabellini@kernel.org>; Jun Nakajima <jun.nakajima@intel.com>; Kevin Tian
>> <kevin.tian@intel.com>; Julien Grall <julien.grall@arm.com>; xen-devel@lists.xenproject.org
>> Subject: Re: [PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved function names
>>
>>
>> On 23.11.20 16:56, Jan Beulich wrote:
>>
>> Hi Jan, Paul
>>
>>> On 23.11.2020 15:39, Oleksandr wrote:
>>>> As it was agreed, below the list of proposed renaming (naming) within
>>>> current series.
>>> Thanks for compiling this. A couple of suggestions for consideration:
>>>
>>>> 1. Global (existing):
>>>> hvm_map_mem_type_to_ioreq_server -> ioreq_server_map_mem_type
>>>> hvm_select_ioreq_server -> ioreq_server_select
>>>> hvm_send_ioreq -> ioreq_send
>>>> hvm_ioreq_init -> ioreq_init
>>> ioreq_domain_init() (or, imo less desirable domain_ioreq_init())?
>> On Arm (for example) I see two variants are present:
>> 1. That starts with subsystem:
>> - tee_domain_init
>> - iommu_domain_init
>>
>>
>> 2. Where sybsystem in the middle:
>> - domain_io_init
>> - domain_vuart_init
>> - domain_vtimer_init
>>
>> If there is no rule, but a matter of taste then I would use
>> ioreq_domain_init(),
>> so arch_ioreq_init() wants to be arch_ioreq_domain_init().
>>
>>>> hvm_destroy_all_ioreq_servers -> ioreq_server_destroy_all
>>>> hvm_all_ioreq_servers_add_vcpu -> ioreq_server_add_vcpu_all
>>>> hvm_all_ioreq_servers_remove_vcpu -> ioreq_server_remove_vcpu_all
>>>> hvm_broadcast_ioreq -> ioreq_broadcast
>>>> hvm_create_ioreq_server -> ioreq_server_create
>>>> hvm_get_ioreq_server_info -> ioreq_server_get_info
>>>> hvm_map_io_range_to_ioreq_server -> ioreq_server_map_io_range
>>>> hvm_unmap_io_range_from_ioreq_server -> ioreq_server_unmap_io_range
>>>> hvm_set_ioreq_server_state -> ioreq_server_set_state
>>>> hvm_destroy_ioreq_server -> ioreq_server_destroy
>>>> hvm_get_ioreq_server_frame -> ioreq_server_get_frame
>>>> hvm_ioreq_needs_completion -> ioreq_needs_completion
>>>> hvm_mmio_first_byte -> ioreq_mmio_first_byte
>>>> hvm_mmio_last_byte -> ioreq_mmio_last_byte
>>>> send_invalidate_req -> ioreq_signal_mapcache_invalidate
>>>>
>>>> handle_hvm_io_completion -> handle_io_completion
>>> For this one I'm not sure what to suggest, but I'm not overly happy
>>> with the name.
>> I also failed to find a better name. Probably ioreq_ or vcpu_ioreq_
>> prefix wants to be added here?
>>
>>
>>>> hvm_io_pending -> io_pending
>>> vcpu_ioreq_pending() or vcpu_any_ioreq_pending()?
>> I am fine with vcpu_ioreq_pending()
>>
> ...in which case vcpu_ioreq_handle_completion() seems like a reasonable choice.
ok, will rename here ...
>
>>>> 2. Global (new):
>>>> arch_io_completion
and here arch_vcpu_ioreq_completion() (without handle in the middle).
>>>> arch_ioreq_server_map_pages
>>>> arch_ioreq_server_unmap_pages
>>>> arch_ioreq_server_enable
>>>> arch_ioreq_server_disable
>>>> arch_ioreq_server_destroy
>>>> arch_ioreq_server_map_mem_type
>>>> arch_ioreq_server_destroy_all
>>>> arch_ioreq_server_get_type_addr
>>>> arch_ioreq_init
>>> Assuming this is the arch hook of the similarly named function
>>> further up, a similar adjustment may then be wanted here.
>> Yes.
>>
>>
>>>> domain_has_ioreq_server
>>>>
>>>>
>>>> 3. Local (existing) in common ioreq.c:
>>>> hvm_alloc_ioreq_mfn -> ioreq_alloc_mfn
>>>> hvm_free_ioreq_mfn -> ioreq_free_mfn
>>> These two are server functions, so should imo be ioreq_server_...().
>> ok, but ...
>>
>>
>>> However, if they're static (as they're now), no distinguishing
>>> prefix is strictly necessary, i.e. alloc_mfn() and free_mfn() may
>>> be fine. The two names may be too short for Paul's taste, though.
>>> Some similar shortening may be possible for some or all of the ones
>>
>> ... In general I would be fine with any option. However, using the
>> shortening rule for all
>> we are going to end up with single-word function names (enable, init, etc).
>> So I would prefer to leave locals as is (but dropping hvm prefixes of
>> course and
>> clarify ioreq_server_alloc_mfn/ioreq_server_free_mfn).
>>
>> Paul, Jan what do you think?
> I prefer ioreq_server_alloc_mfn/ioreq_server_free_mfn. The problem with shortening is that function names become ambiguous within the source base and hence harder to find.
Got it.
Thank you
--
Regards,
Oleksandr Tyshchenko
^ permalink raw reply [flat|nested] 109+ messages in thread
end of thread, other threads:[~2020-11-23 16:36 UTC | newest]
Thread overview: 109+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-15 16:44 [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Oleksandr Tyshchenko
2020-10-15 16:44 ` [PATCH V2 01/23] x86/ioreq: Prepare IOREQ feature for making it common Oleksandr Tyshchenko
2020-10-20 7:13 ` Paul Durrant
2020-11-04 9:06 ` Oleksandr
2020-11-04 9:59 ` Paul Durrant
2020-11-12 10:58 ` Jan Beulich
2020-11-13 11:09 ` Oleksandr
2020-11-13 11:20 ` Jan Beulich
2020-11-13 12:45 ` Oleksandr
2020-11-13 14:31 ` Jan Beulich
2020-10-15 16:44 ` [PATCH V2 02/23] xen/ioreq: Make x86's IOREQ feature common Oleksandr Tyshchenko
2020-10-20 7:57 ` Paul Durrant
2020-11-10 16:45 ` Oleksandr
2020-11-17 14:47 ` Oleksandr
2020-11-17 15:29 ` Paul Durrant
2020-11-17 16:29 ` Oleksandr
2020-11-17 19:43 ` Paul Durrant
2020-11-12 11:11 ` Jan Beulich
2020-11-13 13:11 ` Oleksandr
2020-10-15 16:44 ` [PATCH V2 03/23] xen/ioreq: Make x86's hvm_ioreq_needs_completion() common Oleksandr Tyshchenko
2020-10-20 7:59 ` Paul Durrant
2020-10-15 16:44 ` [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio() Oleksandr Tyshchenko
2020-10-20 9:14 ` Paul Durrant
2020-10-20 10:05 ` Jan Beulich
2020-10-20 10:38 ` Paul Durrant
2020-11-10 19:44 ` Oleksandr
2020-11-11 7:27 ` Jan Beulich
2020-11-11 8:09 ` Oleksandr
2020-11-11 8:16 ` Jan Beulich
2020-10-15 16:44 ` [PATCH V2 05/23] xen/ioreq: Make x86's hvm_mmio_first(last)_byte() common Oleksandr Tyshchenko
2020-10-20 9:15 ` Paul Durrant
2020-10-15 16:44 ` [PATCH V2 06/23] xen/ioreq: Make x86's hvm_ioreq_(page/vcpu/server) structs common Oleksandr Tyshchenko
2020-10-15 16:44 ` [PATCH V2 07/23] xen/ioreq: Move x86's ioreq_gfn(server) to struct domain Oleksandr Tyshchenko
2020-11-12 11:21 ` Jan Beulich
2020-11-13 14:05 ` Oleksandr
2020-11-18 12:09 ` Oleksandr
2020-11-18 12:29 ` Paul Durrant
2020-10-15 16:44 ` [PATCH V2 08/23] xen/ioreq: Introduce ioreq_params to abstract accesses to arch.hvm.params Oleksandr Tyshchenko
2020-10-20 10:41 ` Paul Durrant
2020-11-10 20:00 ` Oleksandr
2020-10-15 16:44 ` [PATCH V2 09/23] xen/dm: Make x86's DM feature common Oleksandr Tyshchenko
2020-11-12 11:32 ` Jan Beulich
2020-11-13 14:28 ` Oleksandr
2020-10-15 16:44 ` [PATCH V2 10/23] xen/mm: Make x86's XENMEM_resource_ioreq_server handling common Oleksandr Tyshchenko
2020-11-12 11:40 ` Jan Beulich
2020-11-13 15:00 ` Oleksandr
2020-10-15 16:44 ` [PATCH V2 11/23] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu Oleksandr Tyshchenko
2020-10-20 10:55 ` Paul Durrant
2020-11-10 20:59 ` Oleksandr
2020-11-11 8:04 ` Jan Beulich
2020-11-11 8:14 ` Oleksandr
2020-10-15 16:44 ` [PATCH V2 12/23] xen/ioreq: Remove "hvm" prefixes from involved function names Oleksandr Tyshchenko
2020-11-12 11:45 ` Jan Beulich
2020-11-12 12:14 ` Paul Durrant
2020-11-13 15:53 ` Oleksandr
2020-11-23 14:39 ` Oleksandr
2020-11-23 14:56 ` Jan Beulich
2020-11-23 15:47 ` Oleksandr
2020-11-23 15:54 ` Paul Durrant
2020-11-23 16:36 ` Oleksandr
2020-10-15 16:44 ` [PATCH V2 13/23] xen/ioreq: Use guest_cmpxchg64() instead of cmpxchg() Oleksandr Tyshchenko
2020-10-15 16:44 ` [PATCH V2 14/23] arm/ioreq: Introduce arch specific bits for IOREQ/DM features Oleksandr Tyshchenko
2020-11-12 11:48 ` Jan Beulich
2020-11-13 15:48 ` Oleksandr
2020-10-15 16:44 ` [PATCH V2 15/23] xen/arm: Stick around in leave_hypervisor_to_guest until I/O has completed Oleksandr Tyshchenko
2020-10-15 16:44 ` [PATCH V2 16/23] xen/mm: Handle properly reference in set_foreign_p2m_entry() on Arm Oleksandr Tyshchenko
2020-11-12 12:18 ` Jan Beulich
2020-11-13 17:00 ` Oleksandr
2020-10-15 16:44 ` [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server() Oleksandr Tyshchenko
2020-10-20 10:51 ` Paul Durrant
2020-11-10 20:53 ` Oleksandr
2020-11-11 8:08 ` Jan Beulich
2020-11-11 8:41 ` Oleksandr
2020-11-11 13:27 ` Jan Beulich
2020-11-11 16:28 ` Paul Durrant
2020-11-11 17:33 ` Oleksandr
2020-11-11 17:31 ` Oleksandr
2020-10-15 16:44 ` [PATCH V2 18/23] xen/dm: Introduce xendevicemodel_set_irq_level DM op Oleksandr Tyshchenko
2020-10-15 16:44 ` [PATCH V2 19/23] xen/arm: io: Abstract sign-extension Oleksandr Tyshchenko
2020-10-15 16:44 ` [PATCH V2 20/23] xen/ioreq: Make x86's send_invalidate_req() common Oleksandr Tyshchenko
2020-11-12 11:55 ` Jan Beulich
2020-11-13 16:03 ` Oleksandr
2020-10-15 16:44 ` [PATCH V2 21/23] xen/arm: Add mapcache invalidation handling Oleksandr Tyshchenko
2020-10-16 6:29 ` Jan Beulich
2020-10-16 8:41 ` Julien Grall
2020-10-16 8:56 ` Jan Beulich
2020-11-11 0:03 ` Oleksandr
2020-11-11 19:27 ` Julien Grall
2020-11-11 19:42 ` Oleksandr
2020-10-15 16:44 ` [PATCH V2 22/23] libxl: Introduce basic virtio-mmio support on Arm Oleksandr Tyshchenko
2020-10-15 16:44 ` [PATCH V2 23/23] [RFC] libxl: Add support for virtio-disk configuration Oleksandr Tyshchenko
2020-11-09 6:45 ` Wei Chen
2020-11-11 0:53 ` Oleksandr
2020-11-11 4:28 ` Wei Chen
2020-11-13 17:38 ` Oleksandr
2020-10-29 7:41 ` [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm Masami Hiramatsu
2020-10-29 18:48 ` Oleksandr Tyshchenko
2020-10-29 19:53 ` Stefano Stabellini
2020-10-29 21:13 ` Oleksandr Tyshchenko
2020-10-29 21:34 ` Stefano Stabellini
2020-10-30 11:34 ` Masami Hiramatsu
2020-10-31 21:10 ` Oleksandr Tyshchenko
2020-11-02 7:23 ` Wei Chen
2020-11-02 18:05 ` Oleksandr
2020-11-03 14:30 ` Masami Hiramatsu
2020-11-03 21:09 ` Oleksandr
2020-10-29 20:03 ` Alex Bennée
2020-10-29 20:10 ` Stefano Stabellini
2020-10-29 22:06 ` Oleksandr Tyshchenko
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).