All of lore.kernel.org
 help / color / mirror / Atom feed
From: Ming Lei <tom.leiming@gmail.com>
To: io-uring@vger.kernel.org, linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org
Cc: Kirill Tkhai <kirill.tkhai@openvz.org>,
	Manuel Bentele <development@manuel-bentele.de>,
	Stefan Hajnoczi <stefanha@redhat.com>
Subject: ublk-qcow2: ublk-qcow2 is available
Date: Fri, 30 Sep 2022 17:24:11 +0800	[thread overview]
Message-ID: <Yza1u1KfKa7ycQm0@T590> (raw)

Hello,

ublk-qcow2 is available now.

So far it provides basic read/write function, and compression and snapshot
aren't supported yet. The target/backend implementation is completely
based on io_uring, and share the same io_uring with ublk IO command
handler, just like what ublk-loop does.

Follows the main motivations of ublk-qcow2:

- building one complicated target from scratch helps libublksrv APIs/functions
  become mature/stable more quickly, since qcow2 is complicated and needs more
  requirement from libublksrv compared with other simple ones(loop, null)

- there are several attempts of implementing qcow2 driver in kernel, such as
  ``qloop`` [2], ``dm-qcow2`` [3] and ``in kernel qcow2(ro)`` [4], so ublk-qcow2
  might useful be for covering requirement in this field

- performance comparison with qemu-nbd, and it was my 1st thought to evaluate
  performance of ublk/io_uring backend by writing one ublk-qcow2 since ublksrv
  is started

- help to abstract common building block or design pattern for writing new ublk
  target/backend

So far it basically passes xfstest(XFS) test by using ublk-qcow2 block
device as TEST_DEV, and kernel building workload is verified too. Also
soft update approach is applied in meta flushing, and meta data
integrity is guaranteed, 'make test T=qcow2/040' covers this kind of
test, and only cluster leak is reported during this test.

The performance data looks much better compared with qemu-nbd, see
details in commit log[1], README[5] and STATUS[6]. And the test covers both
empty image and pre-allocated image, for example of pre-allocated qcow2
image(8GB):

- qemu-nbd (make test T=qcow2/002)
	randwrite(4k): jobs 1, iops 24605
	randread(4k): jobs 1, iops 30938
	randrw(4k): jobs 1, iops read 13981 write 14001
	rw(512k): jobs 1, iops read 724 write 728

- ublk-qcow2 (make test T=qcow2/022)
	randwrite(4k): jobs 1, iops 104481
	randread(4k): jobs 1, iops 114937
	randrw(4k): jobs 1, iops read 53630 write 53577
	rw(512k): jobs 1, iops read 1412 write 1423

Also ublk-qcow2 aligns queue's chunk_sectors limit with qcow2's cluster size,
which is 64KB at default, this way simplifies backend io handling, but
it could be increased to 512K or more proper size for improving sequential
IO perf, just need one coroutine to handle more than one IOs.


[1] https://github.com/ming1/ubdsrv/commit/9faabbec3a92ca83ddae92335c66eabbeff654e7
[2] https://upcommons.upc.edu/bitstream/handle/2099.1/9619/65757.pdf?sequence=1&isAllowed=y
[3] https://lwn.net/Articles/889429/
[4] https://lab.ks.uni-freiburg.de/projects/kernel-qcow2/repository
[5] https://github.com/ming1/ubdsrv/blob/master/qcow2/README.rst
[6] https://github.com/ming1/ubdsrv/blob/master/qcow2/STATUS.rst

Thanks,
Ming

             reply	other threads:[~2022-09-30  9:24 UTC|newest]

Thread overview: 44+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-09-30  9:24 Ming Lei [this message]
2022-10-03 19:53 ` ublk-qcow2: ublk-qcow2 is available Stefan Hajnoczi
2022-10-03 23:57   ` Denis V. Lunev
2022-10-05 15:11     ` Stefan Hajnoczi
2022-10-06 10:26       ` Ming Lei
2022-10-06 13:59         ` Stefan Hajnoczi
2022-10-06 15:09           ` Ming Lei
2022-10-06 18:29             ` Stefan Hajnoczi
2022-10-07 11:21               ` Ming Lei
2022-10-04  9:43   ` Ming Lei
2022-10-04 13:53     ` Stefan Hajnoczi
2022-10-05  4:18       ` Ming Lei
2022-10-05 12:21         ` Stefan Hajnoczi
2022-10-05 12:38           ` Denis V. Lunev
2022-10-06 11:24           ` Ming Lei
2022-10-07 10:04             ` Yongji Xie
2022-10-07 10:51               ` Ming Lei
2022-10-07 11:21                 ` Yongji Xie
2022-10-07 11:23                   ` Ming Lei
2022-10-08  8:43         ` Ziyang Zhang
2022-10-12 14:22           ` Stefan Hajnoczi
2022-10-13  6:48             ` Yongji Xie
2022-10-13 16:02               ` Stefan Hajnoczi
2022-10-14 12:56               ` Ming Lei
2022-10-17 11:11                 ` Yongji Xie
2022-10-18  6:59                   ` Ming Lei
2022-10-18 13:17                     ` Yongji Xie
2022-10-18 14:54                       ` Stefan Hajnoczi
2022-10-19  9:09                         ` Ming Lei
2022-10-24 16:11                           ` Stefan Hajnoczi
2022-10-21  5:33                         ` Yongji Xie
2022-10-21  6:30                           ` Jason Wang
2022-10-25  8:17                             ` Yongji Xie
2022-10-25 12:02                               ` Stefan Hajnoczi
2022-10-28 13:33                                 ` Yongji Xie
2022-11-01  2:36                                 ` Jason Wang
2022-11-02 19:13                                   ` Stefan Hajnoczi
2022-11-04  6:55                                     ` Jason Wang
2022-10-21  6:28                     ` Jason Wang
2022-10-06 10:14       ` Richard W.M. Jones
2022-10-12 14:15         ` Stefan Hajnoczi
2022-10-13  1:50           ` Ming Lei
2022-10-13 16:01             ` Stefan Hajnoczi
2022-10-04  5:43 ` Manuel Bentele

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=Yza1u1KfKa7ycQm0@T590 \
    --to=tom.leiming@gmail.com \
    --cc=development@manuel-bentele.de \
    --cc=io-uring@vger.kernel.org \
    --cc=kirill.tkhai@openvz.org \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=stefanha@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.