From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A3BCC388F7 for ; Sat, 31 Oct 2020 21:11:31 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 324F5206E3 for ; Sat, 31 Oct 2020 21:11:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ofbIPN4t" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 324F5206E3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.17033.41800 (Exim 4.92) (envelope-from ) id 1kYy9H-0004Ik-Cr; Sat, 31 Oct 2020 21:10:55 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 17033.41800; Sat, 31 Oct 2020 21:10:55 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kYy9H-0004Id-9r; Sat, 31 Oct 2020 21:10:55 +0000 Received: by outflank-mailman (input) for mailman id 17033; Sat, 31 Oct 2020 21:10:54 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kYy9G-0004IY-AW for xen-devel@lists.xenproject.org; Sat, 31 Oct 2020 21:10:54 +0000 Received: from mail-wr1-x42c.google.com (unknown [2a00:1450:4864:20::42c]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id dc467fe9-9d91-4afe-854f-1a8051b7c388; Sat, 31 Oct 2020 21:10:53 +0000 (UTC) Received: by mail-wr1-x42c.google.com with SMTP id a9so10181426wrg.12 for ; Sat, 31 Oct 2020 14:10:52 -0700 (PDT) Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kYy9G-0004IY-AW for xen-devel@lists.xenproject.org; Sat, 31 Oct 2020 21:10:54 +0000 X-Inumbo-ID: dc467fe9-9d91-4afe-854f-1a8051b7c388 Received: from mail-wr1-x42c.google.com (unknown [2a00:1450:4864:20::42c]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id dc467fe9-9d91-4afe-854f-1a8051b7c388; Sat, 31 Oct 2020 21:10:53 +0000 (UTC) Received: by mail-wr1-x42c.google.com with SMTP id a9so10181426wrg.12 for ; Sat, 31 Oct 2020 14:10:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=uLkdRm8luG3Y/TaJy2hbja3zVLCKJ9ijfgMi5tCNaGA=; b=ofbIPN4tff2jsWRZLQ/I5kVvTVlsl0yq7sasoRBjWLOkf/+q1Pd9mc2B61Cuu6otEm bECnyMx81yPgKwny9q9pSbK3F84SxDJ4xQsuiGcL6vz99cD6r6OLZU4nqV0CfTGRlYap vDw8RpQqUFp6l5imV6FwgUhqQs231GzXvnY/YJx4gVeTTzqZmD6xmmFG6tLN2SwLEZhb EW/hcDACuPYD0KgDpPnsg0fYVkJiX+bCtoESTZu9vw36v85jbBiT+zaUZGZDJvCHYNKR caQeXOe3CvNpetLqPuJaUfgXPSx8GyUPkwIQxrQa/gIchww4CJSPdLSKz4UdnHF3C+FB idaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=uLkdRm8luG3Y/TaJy2hbja3zVLCKJ9ijfgMi5tCNaGA=; b=ZMyLjKM8B7g4brqxSMd2+wncXJ8aTaQbq+t9bqJHE+SbgaPWuj+VsOg2h95e8sSP5W pVZmo0FBW87pShYwCVC5XhcZ/oHW59yUicT6JRsXodECuT73yV8WCMEBjDuwM3VS0/5E H3KKQds5hOsRnZge0JpE2vDJsz6SFV4BBZ/nTo9ZBW9TbYp/6peiuJzynWjTsbGWXKHe jigSO+6RvBUB+cY5DVxzdS1KzyLBsnis51GJKAdCXxxVOXmJ6I1c34wRDP5pjvGvRlm6 pERoxTxAx5WivQTS7NL0hEG5225SUvANZyvqwgSy8P2c0yDn6sk03CeWf1jbEgyx+2ge UwpA== X-Gm-Message-State: AOAM5310hf/SCAHEMEO6aYplupbpU86VsslgI9AhIiihbbmUBDTQHDtm eMRtFZCdcjwzyhrKA47msbK0MLjc+IbZr3r4kD8= X-Google-Smtp-Source: ABdhPJyk9AmXhk1tgcBqa9mfjrBGXwM3Cj1CVpcntjIaKgimQRpet4ZicGCzbgxEPBEJLe1Ab+VRDMonM5rEUI7ekdY= X-Received: by 2002:adf:8b92:: with SMTP id o18mr11511603wra.54.1604178651915; Sat, 31 Oct 2020 14:10:51 -0700 (PDT) MIME-Version: 1.0 References: <1602780274-29141-1-git-send-email-olekstysh@gmail.com> In-Reply-To: From: Oleksandr Tyshchenko Date: Sat, 31 Oct 2020 23:10:40 +0200 Message-ID: Subject: Re: [PATCH V2 00/23] IOREQ feature (+ virtio-mmio) on Arm To: Masami Hiramatsu , =?UTF-8?B?QWxleCBCZW5uw6ll?= Cc: Stefano Stabellini , xen-devel , Oleksandr Tyshchenko , Paul Durrant , Jan Beulich , Andrew Cooper , =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= , Wei Liu , Julien Grall , George Dunlap , Ian Jackson , Julien Grall , Tim Deegan , Daniel De Graaf , Volodymyr Babchuk , Jun Nakajima , Kevin Tian , Anthony PERARD , Bertrand Marquis Content-Type: multipart/alternative; boundary="000000000000b42afe05b2fdf2e1" --000000000000b42afe05b2fdf2e1 Content-Type: text/plain; charset="UTF-8" On Fri, Oct 30, 2020 at 1:34 PM Masami Hiramatsu < masami.hiramatsu@linaro.org> wrote: > Hi Oleksandr, > Hi Masami, all [sorry for the possible format issue] > >> > > >> > Could you tell me how can I test it? > >> > > >> > > >> > I assume it is due to the lack of the virtio-disk backend (which I > haven't shared yet as I focused on the IOREQ/DM support on Arm in the > >> > first place). > >> > Could you wait a little bit, I am going to share it soon. > >> > >> Do you have a quick-and-dirty hack you can share in the meantime? Even > >> just on github as a special branch? It would be very useful to be able > >> to have a test-driver for the new feature. > > > > Well, I will provide a branch on github with our PoC virtio-disk backend > by the end of this week. It will be possible to test this series with it. > > Great! OK I'll be waiting for the PoC backend. > > Thank you! > You can find the virtio-disk backend PoC (shared as is) at [1]. Brief description... The virtio-disk backend PoC is a completely standalone entity (IOREQ server) which emulates a virtio-mmio disk device. It is based on code from DEMU [2] (for IOREQ server purposes) and some code from kvmtool [3] to implement virtio protocol, disk operations over underlying H/W and Xenbus code to be able to read configuration from the Xenstore (it is configured via domain config file). Last patch in this series (marked as RFC) actually adds required bits to the libxl code. Some notes... Backend could be used with current V2 IOREQ series [4] without any modifications, all what you need is to enable CONFIG_IOREQ_SERVER on Arm [5], since it is disabled by default within this series. Please note that in our system we run backend in DomD (driver domain). I haven't tested it in Dom0, since in our system the Dom0 is thin (without any H/W) and only used to launch VMs, so there is no underlying block H/W. But, I hope, it is possible to run it in Dom0 as well (at least there is nothing specific to a particular domain in the backend itself, nothing hardcoded). If you are going to run a backend in other than Dom0 domain you need to write your own policy (FLASK) for the backend (running in that domain) to be able to issue DM related requests, etc. Only for test purposes you could use this patch [6] that tweeks Xen dummy policy (not for upstream). As I mentioned elsewhere you don't need to modify Guest Linux (DomU), just enable VirtIO related configs. If I remember correctly, the following would be enough: CONFIG_BLK_MQ_VIRTIO=y CONFIG_VIRTIO_BLK=y CONFIG_VIRTIO=y CONFIG_VIRTIO_BALLOON=y CONFIG_VIRTIO_MMIO=y If I remember correctly, if your Host Linux (Dom0 or DomD) version >= 4.17 you don't need to modify it as well. Otherwise, you need to cherry-pick "xen/privcmd: add IOCTL_PRIVCMD_MMAP_RESOURCE" from the upstream to be able to use the acquire interface for the resource mapping. We usually build a backend in the context of the Yocto build process and run it as a systemd service, but you can also build and run it manually (it should be launched before DomU creation). There are no command line options at all. Everything is configured via domain configuration file: # This option is mandatory, it shows that VirtIO is going to be used by guest virtio=1 # Example of domain configuration (two disks are assigned to the guest, the latter is in readonly mode): vdisk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3;ro:/dev/mmcblk1p3' ] Hope that helps. Feel free to ask questions if any. [1] https://github.com/xen-troops/virtio-disk/commits/ioreq_v3 [2] https://xenbits.xen.org/gitweb/?p=people/pauldu/demu.git;a=summary [3] https://git.kernel.org/pub/scm/linux/kernel/git/will/kvmtool.git/ [4] https://github.com/otyshchenko1/xen/commits/ioreq_4.14_ml3 [5] https://github.com/otyshchenko1/xen/commit/ee221102193f0422a240832edc41d73f6f3da923 [6] https://github.com/otyshchenko1/xen/commit/be868a63014b7aa6c9731d5692200d7f2f57c611 -- Regards, Oleksandr Tyshchenko --000000000000b42afe05b2fdf2e1 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Fri, Oct 30, 2020 at 1:34 PM Masam= i Hiramatsu <masami.hiramatsu@linaro.org> wrote:
Hi Oleksandr,
=C2=A0<= /div>
Hi Masami, all

[sorry for the possible f= ormat issue]
=C2=A0
>> >
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0Could you tell me how can I test it= ?
>> >
>> >
>> > I assume it is due to the lack of the virtio-disk backend (wh= ich I haven't shared yet as I focused on the IOREQ/DM support on Arm in= the
>> > first place).
>> > Could you wait a little bit, I am going to share it soon.
>>
>> Do you have a quick-and-dirty hack you can share in the meantime? = Even
>> just on github as a special branch? It would be very useful to be = able
>> to have a test-driver for the new feature.
>
> Well, I will provide a branch on github with our PoC virtio-disk backe= nd by the end of this week. It will be possible to test this series with it= .

Great! OK I'll be waiting for the PoC backend.

Thank you!

You can find the virtio-disk= backend PoC (shared as is) at [1].=C2=A0

Brief descripti= on...

The virtio-disk backend PoC is a completely standalone = entity (IOREQ server) which emulates a virtio-mmio disk device.
It is ba= sed on code from DEMU [2] (for IOREQ server purposes) and some code from kv= mtool [3] to implement virtio protocol,
disk operations over underlying = H/W and Xenbus code to be able to read configuration from the Xenstore
(= it is configured via domain config file). Last patch in this series (marked= as RFC) actually adds required bits to the libxl code.=C2=A0 =C2=A0

Some notes...

Backend could= be used with current V2 IOREQ series [4] without any modifications, all wh= at you need is to enable
CONFIG_IOREQ_SERVER on Arm [5], since it= is disabled by default within this series.

Please note that in our system = we run backend in DomD (driver domain). I haven't tested it in Dom0,since in our system the Dom0 is thin (without any H/W) and only used to la= unch VMs, so there is no underlying block H/W.
But, I hope, it is possi= ble to run it in Dom0 as well (at least there is nothing specific to a part= icular domain in the backend itself, nothing hardcoded).
If you are going to run a backend in other than Dom0 domain y= ou need to write your own policy (FLASK) for the backend (running in that= =C2=A0domain)
to be able to issue DM re= lated requests, etc. Only for test purposes you could use this patch [6] th= at tweeks Xen dummy policy (not for upstream).
=C2=A0=C2=A0
As I mentioned elsewhe= re you don't need to modify Guest Linux (DomU), just enable VirtIO rela= ted configs.
If I remember correctly, the f= ollowing would be enough:
CONFIG_BLK_MQ_VIR= TIO=3Dy
CONFIG_VIRTIO_BLK=3Dy
CONFIG_VIRTIO=3Dy
CONFIG_VIRTIO_BALL= OON=3Dy
CONFIG_VIRTIO_MMIO=3Dy
If I reme= mber correctly, if your Host Linux (Dom0 or DomD) version >=3D 4.17 you = don't need to modify it as well.
Otherw= ise, you need to cherry-pick "xen/privcmd: add IOCTL_PRIVCMD_MMAP_RESO= URCE" from the upstream to be able
to use the acquire interface for= the resource mapping.

We usually build a backe= nd in the context of the Yocto build process and run it as a systemd servic= e,
but you can also build and run it manually (it should be launc= hed before DomU creation).

There are no comman= d line options at all. Everything is configured via domain configuration fi= le:
# This option is mandatory, it shows that VirtIO is going to be used= by guest
virtio=3D1
# Example of domain configuration (two disks are= assigned to the guest, the latter is in readonly mode):
vdisk =3D [ = 9;backend=3DDomD, disks=3Drw:/dev/mmcblk0p3;ro:/dev/mmcblk1p3' ]

Hope that helps. Feel free to ask questions if any.<= br>
[1]=C2=A0https://github.com/xen-troops/virtio-disk/commits/ioreq_v3<= br>
<= br>
--
Regards,

Oleksandr Tyshchenko
--000000000000b42afe05b2fdf2e1--