From: Stefan Hajnoczi <stefanha@redhat.com> To: AKASHI Takahiro <takahiro.akashi@linaro.org> Cc: Stefano Stabellini <sstabellini@kernel.org>, Alex Benn??e <alex.bennee@linaro.org>, Stratos Mailing List <stratos-dev@op-lists.linaro.org>, virtio-dev@lists.oasis-open.org, Arnd Bergmann <arnd.bergmann@linaro.org>, Viresh Kumar <viresh.kumar@linaro.org>, Stefano Stabellini <stefano.stabellini@xilinx.com>, Jan Kiszka <jan.kiszka@siemens.com>, Carl van Schaik <cvanscha@qti.qualcomm.com>, pratikp@quicinc.com, Srivatsa Vaddagiri <vatsa@codeaurora.org>, Jean-Philippe Brucker <jean-philippe@linaro.org>, Mathieu Poirier <mathieu.poirier@linaro.org>, Wei.Chen@arm.com, olekstysh@gmail.com, Oleksandr_Tyshchenko@epam.com, Bertrand.Marquis@arm.com, Artem_Mygaiev@epam.com, julien@xen.org, jgross@suse.com, paul@xen.org, xen-devel@lists.xen.org Subject: Re: Enabling hypervisor agnosticism for VirtIO backends Date: Mon, 23 Aug 2021 10:58:46 +0100 [thread overview] Message-ID: <YSNxVjlpCsc+chEC@stefanha-x1.localdomain> (raw) In-Reply-To: <20210823062500.GC40863@laputa> [-- Attachment #1: Type: text/plain, Size: 3995 bytes --] On Mon, Aug 23, 2021 at 03:25:00PM +0900, AKASHI Takahiro wrote: > Hi Stefan, > > On Tue, Aug 17, 2021 at 11:41:01AM +0100, Stefan Hajnoczi wrote: > > On Wed, Aug 04, 2021 at 12:20:01PM -0700, Stefano Stabellini wrote: > > > > Could we consider the kernel internally converting IOREQ messages from > > > > the Xen hypervisor to eventfd events? Would this scale with other kernel > > > > hypercall interfaces? > > > > > > > > So any thoughts on what directions are worth experimenting with? > > > > > > One option we should consider is for each backend to connect to Xen via > > > the IOREQ interface. We could generalize the IOREQ interface and make it > > > hypervisor agnostic. The interface is really trivial and easy to add. > > > The only Xen-specific part is the notification mechanism, which is an > > > event channel. If we replaced the event channel with something else the > > > interface would be generic. See: > > > https://gitlab.com/xen-project/xen/-/blob/staging/xen/include/public/hvm/ioreq.h#L52 > > > > There have been experiments with something kind of similar in KVM > > recently (see struct ioregionfd_cmd): > > https://lore.kernel.org/kvm/dad3d025bcf15ece11d9df0ff685e8ab0a4f2edd.1613828727.git.eafanasova@gmail.com/ > > Do you know the current status of Elena's work? > It was last February that she posted her latest patch > and it has not been merged upstream yet. Elena worked on this during her Outreachy internship. At the moment no one is actively working on the patches. > > > There is also another problem. IOREQ is probably not be the only > > > interface needed. Have a look at > > > https://marc.info/?l=xen-devel&m=162373754705233&w=2. Don't we also need > > > an interface for the backend to inject interrupts into the frontend? And > > > if the backend requires dynamic memory mappings of frontend pages, then > > > we would also need an interface to map/unmap domU pages. > > > > > > These interfaces are a lot more problematic than IOREQ: IOREQ is tiny > > > and self-contained. It is easy to add anywhere. A new interface to > > > inject interrupts or map pages is more difficult to manage because it > > > would require changes scattered across the various emulators. > > > > Something like ioreq is indeed necessary to implement arbitrary devices, > > but if you are willing to restrict yourself to VIRTIO then other > > interfaces are possible too because the VIRTIO device model is different > > from the general purpose x86 PIO/MMIO that Xen's ioreq seems to support. > > Can you please elaborate your thoughts a bit more here? > > It seems to me that trapping MMIOs to configuration space and > forwarding those events to BE (or device emulation) is a quite > straight-forward way to emulate device MMIOs. > Or do you think of something of protocols used in vhost-user? > > # On the contrary, virtio-ivshmem only requires a driver to explicitly > # forward a "write" request of MMIO accesses to BE. But I don't think > # it's your point. See my first reply to this email thread about alternative interfaces for VIRTIO device emulation. The main thing to note was that although the shared memory vring is used by VIRTIO transports today, the device model actually allows transports to implement virtqueues differently (e.g. making it possible to create a VIRTIO over TCP transport without shared memory in the future). It's possible to define a hypercall interface as a new VIRTIO transport that provides higher-level virtqueue operations. Doing this is more work than using vrings though since existing guest driver and device emulation code already supports vrings. I don't know the requirements of Stratos so I can't say if creating a new hypervisor-independent interface (VIRTIO transport) that doesn't rely on shared memory vrings makes sense. I just wanted to raise the idea in case you find that VIRTIO's vrings don't meet your requirements. Stefan [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --]
WARNING: multiple messages have this Message-ID (diff)
From: Stefan Hajnoczi <stefanha@redhat.com> To: AKASHI Takahiro <takahiro.akashi@linaro.org> Cc: Stefano Stabellini <sstabellini@kernel.org>, Alex Benn??e <alex.bennee@linaro.org>, Stratos Mailing List <stratos-dev@op-lists.linaro.org>, virtio-dev@lists.oasis-open.org, Arnd Bergmann <arnd.bergmann@linaro.org>, Viresh Kumar <viresh.kumar@linaro.org>, Stefano Stabellini <stefano.stabellini@xilinx.com>, Jan Kiszka <jan.kiszka@siemens.com>, Carl van Schaik <cvanscha@qti.qualcomm.com>, pratikp@quicinc.com, Srivatsa Vaddagiri <vatsa@codeaurora.org>, Jean-Philippe Brucker <jean-philippe@linaro.org>, Mathieu Poirier <mathieu.poirier@linaro.org>, Wei.Chen@arm.com, olekstysh@gmail.com, Oleksandr_Tyshchenko@epam.com, Bertrand.Marquis@arm.com, Artem_Mygaiev@epam.com, julien@xen.org, jgross@suse.com, paul@xen.org, xen-devel@lists.xen.org Subject: [virtio-dev] Re: Enabling hypervisor agnosticism for VirtIO backends Date: Mon, 23 Aug 2021 10:58:46 +0100 [thread overview] Message-ID: <YSNxVjlpCsc+chEC@stefanha-x1.localdomain> (raw) In-Reply-To: <20210823062500.GC40863@laputa> [-- Attachment #1: Type: text/plain, Size: 3995 bytes --] On Mon, Aug 23, 2021 at 03:25:00PM +0900, AKASHI Takahiro wrote: > Hi Stefan, > > On Tue, Aug 17, 2021 at 11:41:01AM +0100, Stefan Hajnoczi wrote: > > On Wed, Aug 04, 2021 at 12:20:01PM -0700, Stefano Stabellini wrote: > > > > Could we consider the kernel internally converting IOREQ messages from > > > > the Xen hypervisor to eventfd events? Would this scale with other kernel > > > > hypercall interfaces? > > > > > > > > So any thoughts on what directions are worth experimenting with? > > > > > > One option we should consider is for each backend to connect to Xen via > > > the IOREQ interface. We could generalize the IOREQ interface and make it > > > hypervisor agnostic. The interface is really trivial and easy to add. > > > The only Xen-specific part is the notification mechanism, which is an > > > event channel. If we replaced the event channel with something else the > > > interface would be generic. See: > > > https://gitlab.com/xen-project/xen/-/blob/staging/xen/include/public/hvm/ioreq.h#L52 > > > > There have been experiments with something kind of similar in KVM > > recently (see struct ioregionfd_cmd): > > https://lore.kernel.org/kvm/dad3d025bcf15ece11d9df0ff685e8ab0a4f2edd.1613828727.git.eafanasova@gmail.com/ > > Do you know the current status of Elena's work? > It was last February that she posted her latest patch > and it has not been merged upstream yet. Elena worked on this during her Outreachy internship. At the moment no one is actively working on the patches. > > > There is also another problem. IOREQ is probably not be the only > > > interface needed. Have a look at > > > https://marc.info/?l=xen-devel&m=162373754705233&w=2. Don't we also need > > > an interface for the backend to inject interrupts into the frontend? And > > > if the backend requires dynamic memory mappings of frontend pages, then > > > we would also need an interface to map/unmap domU pages. > > > > > > These interfaces are a lot more problematic than IOREQ: IOREQ is tiny > > > and self-contained. It is easy to add anywhere. A new interface to > > > inject interrupts or map pages is more difficult to manage because it > > > would require changes scattered across the various emulators. > > > > Something like ioreq is indeed necessary to implement arbitrary devices, > > but if you are willing to restrict yourself to VIRTIO then other > > interfaces are possible too because the VIRTIO device model is different > > from the general purpose x86 PIO/MMIO that Xen's ioreq seems to support. > > Can you please elaborate your thoughts a bit more here? > > It seems to me that trapping MMIOs to configuration space and > forwarding those events to BE (or device emulation) is a quite > straight-forward way to emulate device MMIOs. > Or do you think of something of protocols used in vhost-user? > > # On the contrary, virtio-ivshmem only requires a driver to explicitly > # forward a "write" request of MMIO accesses to BE. But I don't think > # it's your point. See my first reply to this email thread about alternative interfaces for VIRTIO device emulation. The main thing to note was that although the shared memory vring is used by VIRTIO transports today, the device model actually allows transports to implement virtqueues differently (e.g. making it possible to create a VIRTIO over TCP transport without shared memory in the future). It's possible to define a hypercall interface as a new VIRTIO transport that provides higher-level virtqueue operations. Doing this is more work than using vrings though since existing guest driver and device emulation code already supports vrings. I don't know the requirements of Stratos so I can't say if creating a new hypervisor-independent interface (VIRTIO transport) that doesn't rely on shared memory vrings makes sense. I just wanted to raise the idea in case you find that VIRTIO's vrings don't meet your requirements. Stefan [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --]
next prev parent reply other threads:[~2021-08-23 9:59 UTC|newest] Thread overview: 66+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-08-04 9:04 [virtio-dev] Enabling hypervisor agnosticism for VirtIO backends Alex Bennée 2021-08-04 19:20 ` Stefano Stabellini 2021-08-11 6:27 ` AKASHI Takahiro 2021-08-14 15:37 ` Oleksandr Tyshchenko 2021-08-16 10:04 ` Wei Chen 2021-08-17 8:07 ` AKASHI Takahiro 2021-08-17 8:39 ` Wei Chen 2021-08-18 5:38 ` AKASHI Takahiro 2021-08-18 8:35 ` Wei Chen 2021-08-20 6:41 ` AKASHI Takahiro 2021-08-26 9:40 ` AKASHI Takahiro 2021-08-26 12:10 ` Wei Chen 2021-08-30 19:36 ` Christopher Clark 2021-08-30 19:53 ` Christopher Clark 2021-08-30 19:53 ` [virtio-dev] " Christopher Clark 2021-09-02 7:19 ` AKASHI Takahiro 2021-09-07 0:57 ` Christopher Clark 2021-09-07 0:57 ` [virtio-dev] " Christopher Clark 2021-09-07 11:55 ` AKASHI Takahiro 2021-09-07 18:09 ` Christopher Clark 2021-09-07 18:09 ` [virtio-dev] " Christopher Clark 2021-09-10 3:12 ` AKASHI Takahiro 2021-08-31 6:18 ` AKASHI Takahiro 2021-09-01 11:12 ` Wei Chen 2021-09-01 12:29 ` AKASHI Takahiro 2021-09-01 16:26 ` Oleksandr Tyshchenko 2021-09-02 1:30 ` Wei Chen 2021-09-02 1:50 ` Wei Chen [not found] ` <0100017b33e585a5-06d4248e-b1a7-485e-800c-7ead89e5f916-000000@email.amazonses.com> 2021-08-12 7:55 ` [Stratos-dev] " François Ozog 2021-08-13 5:10 ` AKASHI Takahiro 2021-09-01 8:57 ` Alex Bennée 2021-09-01 8:57 ` [virtio-dev] " Alex Bennée 2021-08-17 10:41 ` Stefan Hajnoczi 2021-08-17 10:41 ` [virtio-dev] " Stefan Hajnoczi 2021-08-23 6:25 ` AKASHI Takahiro 2021-08-23 9:58 ` Stefan Hajnoczi [this message] 2021-08-23 9:58 ` [virtio-dev] " Stefan Hajnoczi 2021-08-25 10:29 ` AKASHI Takahiro 2021-08-25 15:02 ` Stefan Hajnoczi 2021-08-25 15:02 ` [virtio-dev] " Stefan Hajnoczi 2021-09-01 12:53 ` Alex Bennée 2021-09-01 12:53 ` [virtio-dev] " Alex Bennée 2021-09-02 9:12 ` Stefan Hajnoczi 2021-09-02 9:12 ` [virtio-dev] " Stefan Hajnoczi 2021-09-03 8:06 ` AKASHI Takahiro 2021-09-03 9:28 ` Alex Bennée 2021-09-03 9:28 ` [virtio-dev] " Alex Bennée 2021-09-06 2:23 ` AKASHI Takahiro 2021-09-07 2:41 ` [Stratos-dev] " Christopher Clark 2021-09-07 2:41 ` [virtio-dev] " Christopher Clark 2021-09-10 2:50 ` AKASHI Takahiro 2021-09-10 9:35 ` Alex Bennée 2021-09-10 9:35 ` [virtio-dev] " Alex Bennée 2021-09-13 23:51 ` Stefano Stabellini 2021-09-14 6:08 ` [Stratos-dev] " François Ozog 2021-09-14 14:25 ` Alex Bennée 2021-09-14 14:25 ` [virtio-dev] " Alex Bennée 2021-09-14 17:38 ` [Stratos-dev] " Trilok Soni 2021-09-15 3:29 ` Stefano Stabellini 2021-09-15 23:50 ` Trilok Soni 2021-09-16 2:11 ` Stefano Stabellini 2021-08-05 15:48 ` [virtio-dev] " Stefan Hajnoczi 2021-08-19 9:11 ` [virtio-dev] " Matias Ezequiel Vara Larsen [not found] ` <20210820060558.GB13452@laputa> 2021-08-21 14:08 ` Matias Ezequiel Vara Larsen [not found] ` <20210823012029.GB40863@laputa> 2021-10-04 11:33 ` Matias Ezequiel Vara Larsen 2021-09-01 8:43 ` Alex Bennée
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=YSNxVjlpCsc+chEC@stefanha-x1.localdomain \ --to=stefanha@redhat.com \ --cc=Artem_Mygaiev@epam.com \ --cc=Bertrand.Marquis@arm.com \ --cc=Oleksandr_Tyshchenko@epam.com \ --cc=Wei.Chen@arm.com \ --cc=alex.bennee@linaro.org \ --cc=arnd.bergmann@linaro.org \ --cc=cvanscha@qti.qualcomm.com \ --cc=jan.kiszka@siemens.com \ --cc=jean-philippe@linaro.org \ --cc=jgross@suse.com \ --cc=julien@xen.org \ --cc=mathieu.poirier@linaro.org \ --cc=olekstysh@gmail.com \ --cc=paul@xen.org \ --cc=pratikp@quicinc.com \ --cc=sstabellini@kernel.org \ --cc=stefano.stabellini@xilinx.com \ --cc=stratos-dev@op-lists.linaro.org \ --cc=takahiro.akashi@linaro.org \ --cc=vatsa@codeaurora.org \ --cc=viresh.kumar@linaro.org \ --cc=virtio-dev@lists.oasis-open.org \ --cc=xen-devel@lists.xen.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.