From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AACFDC433DF for ; Fri, 17 Jul 2020 14:11:38 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 78AF020717 for ; Fri, 17 Jul 2020 14:11:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="veMcm7MP" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 78AF020717 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jwR52-0005uY-Pp; Fri, 17 Jul 2020 14:11:16 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jwR51-0005uT-GA for xen-devel@lists.xenproject.org; Fri, 17 Jul 2020 14:11:15 +0000 X-Inumbo-ID: 5f7895b0-c837-11ea-bb8b-bc764e2007e4 Received: from mail-wm1-x341.google.com (unknown [2a00:1450:4864:20::341]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 5f7895b0-c837-11ea-bb8b-bc764e2007e4; Fri, 17 Jul 2020 14:11:14 +0000 (UTC) Received: by mail-wm1-x341.google.com with SMTP id q15so15048169wmj.2 for ; Fri, 17 Jul 2020 07:11:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=U2sdQPnze1MLKuOyuvhgbGRrHIX0NCpYikz7yBYrP2s=; b=veMcm7MPqDZno/rNed+Uxve/K3wX3zKeS9iLg0IUHnzbidZF8Y8lL3Jvf04KyqPqlN bjCZma7N628NLHjPxOpY76Uk4zKJDO4T6MfEmpj7zGkoixjr2132cHATW0hRajD9aYr7 DLTdXIyK/pWyJmrBPCx9jrCfz7L4YC3lqgC5NstdaavS0aYwzWD4wSB5GcyQIqsOrWL2 fpnJssjbCSRAlGZsRBK+o3XpiZ9cCEymOLdGr85UdPntq5Y4BPt3mEAYIXmh1CxUc7SZ USJcFAy6oAjD7QXIPVoZk0HmGOlCjOXruD9+PSWA8dRR5Zv3WX2GSEfC3DNPC2sEnYg2 Iawg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=U2sdQPnze1MLKuOyuvhgbGRrHIX0NCpYikz7yBYrP2s=; b=eHPBcEVBGJyAfZi8CJTAG1fhbCUu78PmeRkjne/R7ASdFdalzBcQmuu4TIdXIZUuP/ qfbb/jFqE1NzNsNt/VxKq/bU4bT15S7XpqRTcycNdhzhwsH7LChpLUhl4QVXFdq30fP6 LryrvNoF7J4uSKXY1uV3BckUjWy/7SMAcrh2CVg7kPxYcXgQ/1d/6vvK3bt8PCxsoqVQ HuxwIkisG/tMRvY5/Lb0B7qCJSWwFmhcnGEhSE4MhNeZmCL5MIEnFKdRjNR4QBvZlATf WAV+93OvgtiIzQnjUxdyoAXW+WzgvB44mnJJ3Izt2ptSnCHwCF/m3HFQRE0ooZEZLhh2 416A== X-Gm-Message-State: AOAM530xuNHBogg73pv11z0mGjen47NCwr1329Fhhiv96ll1bjWWdsAc gxpClsD27VT1bB4iVcBHgq+mCxogy/G84QqFoRKGvY2/3nQ= X-Google-Smtp-Source: ABdhPJyDkKUC2h2JgdO2TeX6ielZxqfsAEnsEafrae3Ug4c8EwPNRyINL55G9FaJvOQK9hbdyCZIdB2rrBi0x3O10Wk= X-Received: by 2002:a05:600c:218f:: with SMTP id e15mr8915485wme.63.1594995073104; Fri, 17 Jul 2020 07:11:13 -0700 (PDT) MIME-Version: 1.0 From: Oleksandr Tyshchenko Date: Fri, 17 Jul 2020 17:11:02 +0300 Message-ID: Subject: Virtio in Xen on Arm (based on IOREQ concept) To: xen-devel Content-Type: multipart/alternative; boundary="000000000000c0409f05aaa3ba28" X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Oleksandr Andrushchenko , Stefano Stabellini , Julien Grall , Artem Mygaiev , Bertrand Marquis Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" --000000000000c0409f05aaa3ba28 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hello all. We would like to resume Virtio in Xen on Arm activities. You can find some background at [1] and Virtio specification at [2]. *A few words about importance:* There is an increasing interest, I would even say, the requirement to have flexible, generic and standardized cross-hypervisor solution for I/O virtualization in the automotive and embedded areas. The target is quite clear here. Providing a standardized interface and device models for device para-virtualization in hypervisor environments, Virtio interface allows us to move Guest domain= s among different hypervisor systems without further modification at the Guest side. What is more that Virtio support is available in Linux, Android and many other operating systems and there are a lot of existing Virtio drivers (frontends= ) which could be just reused without reinventing the wheel. Many organisations push Virtio direction as a common interface. To summarize, Virtio support would be the great feature in Xen on Arm in addition to traditional Xen PV drivers for the user to be able to choose which one to use. *A few word about solution:* As it was mentioned at [1], in order to implement virtio-mmio Xen on Arm requires some implementation to forward guest MMIO access to a device model. And as it turned out the Xen on x86 contains most of the pieces to be able to use tha= t transport (via existing IOREQ concept). Julien has already done a big amoun= t of work in his PoC (xen/arm: Add support for Guest IO forwarding to a device emulator). Using that code as a base we managed to create a completely functional PoC with DomU running on virtio block device instead of a traditional Xen PV driver without modifications to DomU Linux. Our work is mostly about rebasing Julien's code on the actual codebase (Xen 4.14-rc4), various tweeks to be able to run emulator (virtio-disk backend) in other than Dom0 domain (in our system we have thin Dom0 and keep all backends in driver domain), misc fixes for our use-cases and tool support for the configuration. Unfortunately, Julien doesn=E2=80=99t have much time to allocate on the wor= k anymore, so we would like to step in and continue. *A few word about the Xen code:* You can find the whole Xen series at [5]. The patches are in RFC state because some actions in the series should be reconsidered and implemented properly. Before submitting the final code for the review the first IOREQ patch (which is quite big) will be split into x86, Arm and common parts. Please note, x86 part wasn=E2=80=99t even build-tested so far and could be broken with that series. Also the series probably wants splitting into adding IOREQ on Arm (should be focused first) and tools support for the virtio-disk (which is going to be the first Virtio driver) configuration before going into the mailing list. What I would like to add here, the IOREQ feature on Arm could be used not only for implementing Virtio, but for other use-cases which require some emulator entity outside Xen such as custom PCI emulator (non-ECAM compatible) for example. *A few word about the backend(s):* One of the main problems with Virtio in Xen on Arm is the absence of =E2=80=9Cready-to-use=E2=80=9D and =E2=80=9Cout-of-Qemu=E2=80=9D Virtio bac= kends (I least am not aware of). We managed to create virtio-disk backend based on demu [3] and kvmtool [4] using that series. It is worth mentioning that although Xenbus/Xenstore is not supposed to be used with native Virtio, that interface was chosen to just pass configuration from toolstack to the backend and notify it about creating/destroying Guest domain (I think it is not bad since backends are usually tied to the hypervisor and can use services provided by hypervisor), the most important thing here is that all Virtio subsystem in the Guest was left unmodified. Backend wants some cleanup and, probably, refactoring. We have a plan to publish it in a while. Our next plan is to start preparing series for the review. Any feedback and would be highly appreciated. [1] https://lists.xenproject.org/archives/html/xen-devel/2019-07/msg01746.html [2] https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/virtio-v1.1-cs01.html [3] https://xenbits.xen.org/gitweb/?p=3Dpeople/pauldu/demu.git;a=3Dsummary [4] https://git.kernel.org/pub/scm/linux/kernel/git/will/kvmtool.git/ [5] https://github.com/xen-troops/xen/commits/ioreq_4.14_ml --=20 Regards, Oleksandr Tyshchenko --000000000000c0409f05aaa3ba28 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello all.

We would like to resume Virtio in Xen on= Arm activities. You can find some
background at [1] and Virtio specific= ation at [2].

*A few words about importance:*
There is an increas= ing interest, I would even say, the requirement to have
flexible, generi= c and standardized cross-hypervisor solution for I/O virtualization
in t= he automotive and embedded areas. The target is quite clear here.
Provid= ing a standardized interface and device models for device para-virtualizati= on
in hypervisor environments, Virtio interface allows us to move Guest = domains
among different hypervisor systems without further modification = at the Guest side.
What is more that Virtio support is available in Linu= x, Android and many other
operating systems and there are a lot of exist= ing Virtio drivers (frontends)
which could be just reused without reinve= nting the wheel. Many organisations push
Virtio direction as a common in= terface. To summarize, Virtio support would be
the great feature in Xen = on Arm in addition to traditional Xen PV drivers for
the user to be able= to choose which one to use.

*A few word about solution:*
As it w= as mentioned at [1], in order to implement virtio-mmio Xen on Arm requires<= br>some implementation to forward guest MMIO access to a device model. And = as it
turned out the Xen on x86 contains most of the pieces to be able t= o use that
transport (via existing IOREQ concept). Julien has already do= ne a big amount
of work in his PoC (xen/arm: Add support for Guest IO fo= rwarding to a device emulator).
Using that code as a base we managed to = create a completely functional PoC with DomU
running on virtio block dev= ice instead of a traditional Xen PV driver without
modifications to DomU= Linux. Our work is mostly about rebasing Julien's code on the actualcodebase (Xen 4.14-rc4), various tweeks to be able to run emulator (virti= o-disk backend)
in other than Dom0 domain (in our system we have thin Do= m0 and keep all backends
in driver domain), misc fixes for our use-cases= and tool support for the configuration.
Unfortunately, Julien doesn=E2= =80=99t have much time to allocate on the work anymore,
so we would like= to step in and continue.

*A few word about the Xen code:*
You ca= n find the whole Xen series at [5]. The patches are in RFC state becausesome actions in the series should be reconsidered and implemented properly= .
Before submitting the final code for the review the first IOREQ patch= (which is quite
big) will be split into x86, Arm and common parts. Plea= se note, x86 part wasn=E2=80=99t
even build-tested so far and could be b= roken with that series. Also the series probably
wants splitting into ad= ding IOREQ on Arm (should be focused first) and tools support
for the vi= rtio-disk (which is going to be the first Virtio driver) configuration befo= re going
into the mailing list.

What I would like to add h= ere, the IOREQ feature on Arm could be used not only
for implemen= ting Virtio, but for other use-cases which require some emulator entity
outside Xen such as custom PCI emulator (non-ECAM compatible) for ex= ample.

*A few word about the backend(s):*
One of the main pr= oblems with Virtio in Xen on Arm is the absence of
=E2=80=9Cready-to-use= =E2=80=9D and =E2=80=9Cout-of-Qemu=E2=80=9D Virtio backends (I least am not= aware of).
We managed to create virtio-disk backend based on demu [3] a= nd kvmtool [4] using
that series. It is worth mentioning that although X= enbus/Xenstore is not supposed
to be used with native Virtio, that inter= face was chosen to just pass configuration from toolstack
to the backend= and notify it about creating/destroying Guest domain (I think it is
not= bad since backends are usually tied to the hypervisor and can use services=
provided by hypervisor), the most important thing here is that all Virt= io subsystem
in the Guest was left unmodified. Backend wants some cleanu= p and, probably,
refactoring. We have a plan to publish it in a while.
--000000000000c0409f05aaa3ba28--