Re: [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm

From: Oleksandr Tyshchenko <olekstysh@gmail.com>
To: "Masami Hiramatsu" <masami.hiramatsu@linaro.org>,
	"Alex Bennée" <alex.bennee@linaro.org>
Cc: "Stefano Stabellini" <sstabellini@kernel.org>,
	xen-devel <xen-devel@lists.xenproject.org>,
	"Oleksandr Tyshchenko" <oleksandr_tyshchenko@epam.com>,
	"Paul Durrant" <paul@xen.org>, "Jan Beulich" <jbeulich@suse.com>,
	"Andrew Cooper" <andrew.cooper3@citrix.com>,
	"Roger Pau Monné" <roger.pau@citrix.com>, "Wei Liu" <wl@xen.org>,
	"Julien Grall" <julien.grall@arm.com>,
	"George Dunlap" <george.dunlap@citrix.com>,
	"Ian Jackson" <iwj@xenproject.org>,
	"Julien Grall" <julien@xen.org>, "Tim Deegan" <tim@xen.org>,
	"Daniel De Graaf" <dgdegra@tycho.nsa.gov>,
	"Volodymyr Babchuk" <Volodymyr_Babchuk@epam.com>,
	"Jun Nakajima" <jun.nakajima@intel.com>,
	"Kevin Tian" <kevin.tian@intel.com>,
	"Anthony PERARD" <anthony.perard@citrix.com>,
	"Bertrand Marquis" <bertrand.marquis@arm.com>
Subject: Re: [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm
Date: Sat, 31 Oct 2020 23:10:40 +0200	[thread overview]
Message-ID: <CAPD2p-=2UimQy6VHKw1FgyVi2R94Ux_HFdPYk7=FR3KWSEqiHw@mail.gmail.com> (raw)
In-Reply-To: <CAA93ih3Z-zxQ33gvr2C43i0J5XP3OBgUhTyMcwhe9zVj-uOONA@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 3973 bytes --]

On Fri, Oct 30, 2020 at 1:34 PM Masami Hiramatsu <
masami.hiramatsu@linaro.org> wrote:

> Hi Oleksandr,
>

Hi Masami, all

[sorry for the possible format issue]

> >> >
> >> >       Could you tell me how can I test it?
> >> >
> >> >
> >> > I assume it is due to the lack of the virtio-disk backend (which I
> haven't shared yet as I focused on the IOREQ/DM support on Arm in the
> >> > first place).
> >> > Could you wait a little bit, I am going to share it soon.
> >>
> >> Do you have a quick-and-dirty hack you can share in the meantime? Even
> >> just on github as a special branch? It would be very useful to be able
> >> to have a test-driver for the new feature.
> >
> > Well, I will provide a branch on github with our PoC virtio-disk backend
> by the end of this week. It will be possible to test this series with it.
>
> Great! OK I'll be waiting for the PoC backend.
>
> Thank you!
>

You can find the virtio-disk backend PoC (shared as is) at [1].

Brief description...

The virtio-disk backend PoC is a completely standalone entity (IOREQ
server) which emulates a virtio-mmio disk device.
It is based on code from DEMU [2] (for IOREQ server purposes) and some code
from kvmtool [3] to implement virtio protocol,
disk operations over underlying H/W and Xenbus code to be able to read
configuration from the Xenstore
(it is configured via domain config file). Last patch in this series
(marked as RFC) actually adds required bits to the libxl code.

Some notes...

Backend could be used with current V2 IOREQ series [4] without any
modifications, all what you need is to enable
CONFIG_IOREQ_SERVER on Arm [5], since it is disabled by default within this
series.

Please note that in our system we run backend in DomD (driver domain). I
haven't tested it in Dom0,
since in our system the Dom0 is thin (without any H/W) and only used to
launch VMs, so there is no underlying block H/W.
But, I hope, it is possible to run it in Dom0 as well (at least there is
nothing specific to a particular domain in the backend itself, nothing
hardcoded).
If you are going to run a backend in other than Dom0 domain you need to
write your own policy (FLASK) for the backend (running in that domain)
to be able to issue DM related requests, etc. Only for test purposes you
could use this patch [6] that tweeks Xen dummy policy (not for upstream).

As I mentioned elsewhere you don't need to modify Guest Linux (DomU), just
enable VirtIO related configs.
If I remember correctly, the following would be enough:
CONFIG_BLK_MQ_VIRTIO=y
CONFIG_VIRTIO_BLK=y
CONFIG_VIRTIO=y
CONFIG_VIRTIO_BALLOON=y
CONFIG_VIRTIO_MMIO=y
If I remember correctly, if your Host Linux (Dom0 or DomD) version >= 4.17
you don't need to modify it as well.
Otherwise, you need to cherry-pick "xen/privcmd: add
IOCTL_PRIVCMD_MMAP_RESOURCE" from the upstream to be able
to use the acquire interface for the resource mapping.

We usually build a backend in the context of the Yocto build process and
run it as a systemd service,
but you can also build and run it manually (it should be launched before
DomU creation).

There are no command line options at all. Everything is configured via
domain configuration file:
# This option is mandatory, it shows that VirtIO is going to be used by
guest
virtio=1
# Example of domain configuration (two disks are assigned to the guest, the
latter is in readonly mode):
vdisk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3;ro:/dev/mmcblk1p3' ]

Hope that helps. Feel free to ask questions if any.

[1] https://github.com/xen-troops/virtio-disk/commits/ioreq_v3
[2] https://xenbits.xen.org/gitweb/?p=people/pauldu/demu.git;a=summary
[3] https://git.kernel.org/pub/scm/linux/kernel/git/will/kvmtool.git/
[4] https://github.com/otyshchenko1/xen/commits/ioreq_4.14_ml3
[5]
https://github.com/otyshchenko1/xen/commit/ee221102193f0422a240832edc41d73f6f3da923
[6]
https://github.com/otyshchenko1/xen/commit/be868a63014b7aa6c9731d5692200d7f2f57c611

-- 
Regards,

Oleksandr Tyshchenko

[-- Attachment #2: Type: text/html, Size: 6215 bytes --]