From: Stefan Hajnoczi <stefanha@redhat.com>
To: Jagannathan Raman <jag.raman@oracle.com>
Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com,
thuth@redhat.com, bleal@redhat.com, swapnil.ingle@nutanix.com,
john.levon@nutanix.com, philmd@redhat.com, qemu-devel@nongnu.org,
wainersm@redhat.com, alex.williamson@redhat.com,
thanos.makatos@nutanix.com, marcandre.lureau@gmail.com,
crosa@redhat.com, pbonzini@redhat.com, alex.bennee@linaro.org
Subject: Re: [PATCH v4 09/14] vfio-user: handle DMA mappings
Date: Thu, 16 Dec 2021 13:24:06 +0000 [thread overview]
Message-ID: <Ybs99rFUKFBIL/Md@stefanha-x1.localdomain> (raw)
In-Reply-To: <2c9baf82a342cfc4ff3d35e017908b9050faf409.1639549843.git.jag.raman@oracle.com>
[-- Attachment #1: Type: text/plain, Size: 3636 bytes --]
On Wed, Dec 15, 2021 at 10:35:33AM -0500, Jagannathan Raman wrote:
> Define and register callbacks to manage the RAM regions used for
> device DMA
>
> Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
> Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
> Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
> hw/remote/vfio-user-obj.c | 48 +++++++++++++++++++++++++++++++++++++++
> hw/remote/trace-events | 2 ++
> 2 files changed, 50 insertions(+)
>
> diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c
> index c6d0c675b7..46f2251a68 100644
> --- a/hw/remote/vfio-user-obj.c
> +++ b/hw/remote/vfio-user-obj.c
> @@ -208,6 +208,47 @@ static ssize_t vfu_object_cfg_access(vfu_ctx_t *vfu_ctx, char * const buf,
> return count;
> }
>
> +static void dma_register(vfu_ctx_t *vfu_ctx, vfu_dma_info_t *info)
> +{
> + MemoryRegion *subregion = NULL;
> + g_autofree char *name = NULL;
> + static unsigned int suffix;
> + struct iovec *iov = &info->iova;
> +
> + if (!info->vaddr) {
> + return;
> + }
> +
> + name = g_strdup_printf("remote-mem-%u", suffix++);
> +
> + subregion = g_new0(MemoryRegion, 1);
> +
> + memory_region_init_ram_ptr(subregion, NULL, name,
> + iov->iov_len, info->vaddr);
> +
> + memory_region_add_subregion(get_system_memory(), (hwaddr)iov->iov_base,
> + subregion);
> +
> + trace_vfu_dma_register((uint64_t)iov->iov_base, iov->iov_len);
> +}
> +
> +static void dma_unregister(vfu_ctx_t *vfu_ctx, vfu_dma_info_t *info)
> +{
> + MemoryRegion *mr = NULL;
> + ram_addr_t offset;
> +
> + mr = memory_region_from_host(info->vaddr, &offset);
> + if (!mr) {
> + return;
> + }
> +
> + memory_region_del_subregion(get_system_memory(), mr);
> +
> + object_unparent((OBJECT(mr)));
> +
> + trace_vfu_dma_unregister((uint64_t)info->iova.iov_base);
> +}
This does not support hot unplug (memory regions pointing to memory
mapped by libvfio-user are left registered). The code should keep a list
(e.g. https://docs.gtk.org/glib/struct.SList.html) of memory regions and
automatically remove them before destroying the vfu context.
It also doesn't support multiple vfio-user server instances running in
the same QEMU process. get_system_memory() is global but the memory
regions provided by vfio-user are per-client (i.e. VM). If multiple VMs
are connected to one vfio-user server process then they conflict.
I don't know the best way to support multiple vfio-user server
instances, it would be straightforward if QEMU supported multiple
MachineStates and didn't use global get_system_memory()/get_io_memory()
APIs. It would be nice to solve that in the future.
Maybe it's too hard to change that, I haven't looked. An alternative is
to make the x-remote machine empty (it doesn't create any devices) and
instead create a new PCI bus, interrupt controller, memory MemoryRegion,
and io MemoryRegion in VfuObject. Stop using get_system_memory() and
instead use the per-VfuObject memory MemoryRegion.
In either of those approaches it's probably necessary to specify the PCI
bus ID in --device and device_add so it's clear which vfio-user server
the PCI device is associated with.
The multiple vfio-user server instance limitation doesn't need to be
solved now, but I wanted to share some ideas around it. Maybe someone
has better ideas or is aware of limitations preventing what I described.
Stefan
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
next prev parent reply other threads:[~2021-12-16 13:39 UTC|newest]
Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-12-15 15:35 [PATCH v4 00/14] vfio-user server in QEMU Jagannathan Raman
2021-12-15 15:35 ` [PATCH v4 01/14] configure, meson: override C compiler for cmake Jagannathan Raman
2021-12-15 15:35 ` [PATCH v4 02/14] tests/avocado: Specify target VM argument to helper routines Jagannathan Raman
2021-12-15 15:54 ` Philippe Mathieu-Daudé
2021-12-15 22:04 ` Beraldo Leal
2021-12-16 21:28 ` Jag Raman
2021-12-15 15:35 ` [PATCH v4 03/14] vfio-user: build library Jagannathan Raman
2021-12-15 15:35 ` [PATCH v4 04/14] vfio-user: define vfio-user-server object Jagannathan Raman
2021-12-16 9:33 ` Stefan Hajnoczi
2021-12-17 2:17 ` Jag Raman
2021-12-16 9:58 ` Stefan Hajnoczi
2021-12-17 2:31 ` Jag Raman
2021-12-17 8:28 ` Stefan Hajnoczi
2021-12-15 15:35 ` [PATCH v4 05/14] vfio-user: instantiate vfio-user context Jagannathan Raman
2021-12-16 9:55 ` Stefan Hajnoczi
2021-12-16 21:32 ` Jag Raman
2021-12-15 15:35 ` [PATCH v4 06/14] vfio-user: find and init PCI device Jagannathan Raman
2021-12-16 10:39 ` Stefan Hajnoczi
2021-12-17 3:12 ` Jag Raman
2021-12-15 15:35 ` [PATCH v4 07/14] vfio-user: run vfio-user context Jagannathan Raman
2021-12-16 11:17 ` Stefan Hajnoczi
2021-12-17 17:59 ` Jag Raman
2021-12-20 8:29 ` Stefan Hajnoczi
2021-12-21 3:04 ` Jag Raman
2022-01-05 10:38 ` Thanos Makatos
2022-01-06 13:35 ` Stefan Hajnoczi
2022-01-10 17:56 ` John Levon
2022-01-11 9:36 ` Stefan Hajnoczi
2022-01-11 13:12 ` Jag Raman
2021-12-15 15:35 ` [PATCH v4 08/14] vfio-user: handle PCI config space accesses Jagannathan Raman
2021-12-16 11:30 ` Stefan Hajnoczi
2021-12-16 11:47 ` John Levon
2021-12-16 16:00 ` Stefan Hajnoczi
2021-12-15 15:35 ` [PATCH v4 09/14] vfio-user: handle DMA mappings Jagannathan Raman
2021-12-16 13:24 ` Stefan Hajnoczi [this message]
2021-12-17 19:11 ` Jag Raman
2021-12-15 15:35 ` [PATCH v4 10/14] vfio-user: handle PCI BAR accesses Jagannathan Raman
2021-12-16 14:10 ` Stefan Hajnoczi
2021-12-17 19:12 ` Jag Raman
2021-12-15 15:35 ` [PATCH v4 11/14] vfio-user: IOMMU support for remote device Jagannathan Raman
2021-12-16 14:40 ` Stefan Hajnoczi
2021-12-17 20:00 ` Jag Raman
2021-12-20 14:36 ` Stefan Hajnoczi
2021-12-21 4:32 ` Jag Raman
2022-01-06 13:10 ` Stefan Hajnoczi
2021-12-15 15:35 ` [PATCH v4 12/14] vfio-user: handle device interrupts Jagannathan Raman
2021-12-16 15:56 ` Stefan Hajnoczi
2021-12-15 15:35 ` [PATCH v4 13/14] vfio-user: register handlers to facilitate migration Jagannathan Raman
2021-12-15 15:35 ` [PATCH v4 14/14] vfio-user: avocado tests for vfio-user Jagannathan Raman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Ybs99rFUKFBIL/Md@stefanha-x1.localdomain \
--to=stefanha@redhat.com \
--cc=alex.bennee@linaro.org \
--cc=alex.williamson@redhat.com \
--cc=bleal@redhat.com \
--cc=crosa@redhat.com \
--cc=elena.ufimtseva@oracle.com \
--cc=jag.raman@oracle.com \
--cc=john.g.johnson@oracle.com \
--cc=john.levon@nutanix.com \
--cc=marcandre.lureau@gmail.com \
--cc=pbonzini@redhat.com \
--cc=philmd@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=swapnil.ingle@nutanix.com \
--cc=thanos.makatos@nutanix.com \
--cc=thuth@redhat.com \
--cc=wainersm@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).