xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
From: Viresh Kumar <viresh.kumar@linaro.org>
To: stratos-dev@op-lists.linaro.org, xen-devel@lists.xen.org
Cc: "Alex Bennée" <alex.bennee@linaro.org>,
	"Stefano Stabellini" <stefano.stabellini@xilinx.com>,
	"Mathieu Poirier" <mathieu.poirier@linaro.com>,
	"Vincent Guittot" <vincent.guittot@linaro.org>,
	"Mike Holmes" <mike.holmes@linaro.org>,
	"Oleksandr Tyshchenko" <olekstysh@gmail.com>,
	"Wei Liu" <wl@xen.org>
Subject: Re: Virtio on Xen with Rust
Date: Thu, 14 Apr 2022 14:53:58 +0530	[thread overview]
Message-ID: <20220414092358.kepxbmnrtycz7mhe@vireshk-i7> (raw)
In-Reply-To: <20220414091538.jijj4lbrkjiby6el@vireshk-i7>

+xen-devel

On 14-04-22, 14:45, Viresh Kumar wrote:
> Hello,
> 
> We verified our hypervisor-agnostic Rust based vhost-user backends with Qemu
> based setup earlier, and there was growing concern if they were truly
> hypervisor-agnostic.
> 
> In order to prove that, we decided to give it a try with Xen, a type-1
> bare-metal hypervisor.
> 
> We are happy to announce that we were able to make progress on that front and
> have a working setup where we can test our existing Rust based backends, like
> I2C, GPIO, RNG (though only I2C is tested as of now) over Xen.
> 
> Key components:
> --------------
> 
> - Xen: https://github.com/vireshk/xen
> 
>   Xen requires MMIO and device specific support in order to populate the
>   required devices at the guest. This tree contains four patches on the top of
>   mainline Xen, two from Oleksandr (mmio/disk) and two from me (I2C).
> 
> - libxen-sys: https://github.com/vireshk/libxen-sys
> 
>   We currently depend on the userspace tools/libraries provided by Xen, like
>   xendevicemodel, xenevtchn, xenforeignmemory, etc. This crates provides Rust
>   wrappers over those calls, generated automatically with help of bindgen
>   utility in Rust, that allow us to use the installed Xen libraries. Though we
>   plan to replace this with Rust based "oxerun" (find below) in longer run.
> 
> - oxerun (WIP): https://gitlab.com/mathieupoirier/oxerun/-/tree/xen-ioctls
> 
>   This is Rust based implementations for Ioctl and hypercalls to Xen. This is WIP
>   and should eventually replace "libxen-sys" crate entirely (which are C based
>   implementation of the same).
> 
> - vhost-device: https://github.com/vireshk/vhost-device
> 
>   These are Rust based vhost-user backends, maintained inside the rust-vmm
>   project. This already contain support for I2C and RNG, while GPIO is under
>   review. These are not required to be modified based on hypervisor and are
>   truly hypervisor-agnostic.
> 
>   Ideally the backends are hypervisor agnostic, as explained earlier, but
>   because of the way Xen maps the guest memory currently, we need a minor update
>   for the backends to work. Xen maps the memory via a kernel file
>   /dev/xen/privcmd, which needs calls to mmap() followed by an ioctl() to make
>   it work. For this a hack has been added to one of the rust-vmm crates,
>   vm-virtio, which is used by vhost-user.
> 
>   https://github.com/vireshk/vm-memory/commit/54b56c4dd7293428edbd7731c4dbe5739a288abd
> 
>   The update to vm-memory is responsible to do ioctl() after the already present
>   mmap().
> 
> - vhost-user-master (WIP): https://github.com/vireshk/vhost-user-master
> 
>   This implements the master side interface of the vhost protocol, and is like
>   the vhost-user-backend (https://github.com/rust-vmm/vhost-user-backend) crate
>   maintained inside the rust-vmm project, which provides similar infrastructure
>   for the backends to use. This shall be hypervisor independent and provide APIs
>   for the hypervisor specific implementations. This will eventually be
>   maintained inside the rust-vmm project and used by all Rust based hypervisors.
> 
> - xen-vhost-master (WIP): https://github.com/vireshk/xen-vhost-master
> 
>   This is the Xen specific implementation and uses the APIs provided by
>   "vhost-user-master", "oxerun" and "libxen-sys" crates for its functioning.
> 
>   This is designed based on the EPAM's "virtio-disk" repository
>   (https://github.com/xen-troops/virtio-disk/) and is pretty much similar to it.
> 
>   One can see the analogy as:
> 
>   Virtio-disk == "Xen-vhost-master" + "vhost-user-master" + "oxerun" + "libxen-sys" + "vhost-device".
> 
> 
> 
> Test setup:
> ----------
> 
> 1. Build Xen:
> 
>   $ ./configure --libdir=/usr/lib --build=x86_64-unknown-linux-gnu --host=aarch64-linux-gnu --disable-docs --disable-golang --disable-ocamltools --with-system-qemu=/root/qemu/build/i386-softmmu/qemu-system-i386;
>   $ make -j9 debball CROSS_COMPILE=aarch64-linux-gnu- XEN_TARGET_ARCH=arm64
> 
> 2. Run Xen via Qemu on X86 machine:
> 
>   $ qemu-system-aarch64 -machine virt,virtualization=on -cpu cortex-a57 -serial mon:stdio \
>         -device virtio-net-pci,netdev=net0 -netdev user,id=net0,hostfwd=tcp::8022-:22 \
>         -device virtio-scsi-pci -drive file=/home/vireshk/virtio/debian-bullseye-arm64.qcow2,index=0,id=hd0,if=none,format=qcow2 -device scsi-hd,drive=hd0 \
>         -display none -m 8192 -smp 8 -kernel /home/vireshk/virtio/xen/xen \
>         -append "dom0_mem=5G,max:5G dom0_max_vcpus=7 loglvl=all guest_loglvl=all" \
>         -device guest-loader,addr=0x46000000,kernel=/home/vireshk/kernel/barm64/arch/arm64/boot/Image,bootargs="root=/dev/sda2 console=hvc0 earlyprintk=xen" \
>         -device ds1338,address=0x20     # This is required to create a virtual I2C based RTC device on Dom0.
> 
>   This should get Dom0 up and running.
> 
> 3. Build rust crates:
> 
>   $ cd /root/
>   $ git clone https://github.com/vireshk/xen-vhost-master
>   $ cd xen-vhost-master
>   $ cargo build
> 
>   $ cd ../
>   $ git clone https://github.com/vireshk/vhost-device
>   $ cd vhost-device
>   $ cargo build
> 
> 4. Setup I2C based RTC device
> 
>   $ echo ds1338 0x20 > /sys/bus/i2c/devices/i2c-0/new_device; echo 0-0020 > /sys/bus/i2c/devices/0-0020/driver/unbind
> 
> 5. Lets run everything now
> 
>   # Start the I2C backend in one terminal (open new terminal with "ssh
>   # root@localhost -p8022"). This tells the I2C backend to hook up to
>   # "/root/vi2c.sock0" socket and wait for the master to start transacting.
>   $ /root/vhost-device/target/debug/vhost-device-i2c -s /root/vi2c.sock -c 1 -l 0:32
> 
>   # Start the xen-vhost-master in another terminal. This provides the path of
>   # the socket to the master side and the device to look from Xen, which is I2C
>   # here.
>   $ /root/xen-vhost-master/target/debug/xen-vhost-master --socket-path /root/vi2c.sock0 --name i2c
> 
>   # Start guest in another terminal, i2c_domu.conf is attached. The guest kernel
>   # should have Virtio related config options enabled, along with i2c-virtio
>   # driver.
>   $ xl create -c  i2c_domu.conf
> 
>   # The guest should boot fine now. Once the guest is up, you can create the I2C
>   # RTC device and use it. Following will create /dev/rtc0 in the guest, which
>   # you can configure with 'hwclock' utility.
> 
>   $ echo ds1338 0x20 > /sys/bus/i2c/devices/i2c-0/new_device
> 
> 
> Hope this helps.
> 
> -- 
> viresh

i2c_domu.conf

> kernel="/root/Image"
> memory=512
> vcpus=2
> command="console=hvc0 earlycon=xenboot"
> name="domu"
> i2c = [ "virtio=true, irq=1, base=1" ]

-- 
viresh


       reply	other threads:[~2022-04-14  9:24 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <20220414091538.jijj4lbrkjiby6el@vireshk-i7>
2022-04-14  9:23 ` Viresh Kumar [this message]
2022-04-14 11:45   ` Virtio on Xen with Rust Wei Liu
2022-04-14 12:07     ` Andrew Cooper
2022-04-14 12:52       ` Wei Liu
2022-04-14 13:36         ` Alex Bennée
2022-04-14 14:10           ` Wei Liu
2022-04-14 15:59             ` Doug Goldstein
2022-04-19  1:10   ` Viresh Kumar
2022-04-28 13:52 ` Oleksandr Tyshchenko
2022-04-29  3:48   ` Viresh Kumar
2022-04-29  3:59     ` Viresh Kumar
2022-04-29 11:14       ` Oleksandr
2022-04-29 10:44     ` Oleksandr
2022-05-05  7:34       ` Viresh Kumar
2022-06-22 11:49   ` Viresh Kumar
2022-06-22 15:05     ` Oleksandr Tyshchenko
2022-06-23  5:48       ` Viresh Kumar
2022-06-23 12:47         ` Oleksandr Tyshchenko
2022-06-24  5:32           ` Viresh Kumar
2022-05-02 11:23 ` Viresh Kumar

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220414092358.kepxbmnrtycz7mhe@vireshk-i7 \
    --to=viresh.kumar@linaro.org \
    --cc=alex.bennee@linaro.org \
    --cc=mathieu.poirier@linaro.com \
    --cc=mike.holmes@linaro.org \
    --cc=olekstysh@gmail.com \
    --cc=stefano.stabellini@xilinx.com \
    --cc=stratos-dev@op-lists.linaro.org \
    --cc=vincent.guittot@linaro.org \
    --cc=wl@xen.org \
    --cc=xen-devel@lists.xen.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).