linux-doc.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Palmer Dabbelt <palmer@dabbelt.com>
To: dm-devel@redhat.com, agk@redhat.com, snitzer@redhat.com
Cc: corbet@lwn.net, song@kernel.org,
	Palmer Dabbelt <palmerdabbelt@google.com>,
	shuah@kernel.org, linux-doc@vger.kernel.org,
	linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org,
	linux-kselftest@vger.kernel.org, kernel-team@android.com
Subject: [PATCH v1 0/5] dm: dm-user: New target that proxies BIOs to userspace
Date: Thu,  3 Dec 2020 13:58:54 -0800	[thread overview]
Message-ID: <20201203215859.2719888-1-palmer@dabbelt.com> (raw)

This patch set contains dm-user, a device mapper target that proxies incoming
BIOs to userspace via a misc device.  Essentially it's FUSE, but for block
devices.  There's more information in the documentation patch and as a handful
of commends, so I'm just going to avoid duplicating that here.  I don't really
think there's any fundamental functionality that dm-user enables, as one could
use something along the lines of nbd/iscsi, but dm-user does result in
extremely simple userspace daemons -- so simple that when I tried to write a
helper userspace library for dm-user I just ended up with nothing.

I talked about this a bit at Plumbers and was hoping to send patches a bit
earlier on in the process, but got tied up with a few things.  As a result this
is actually quite far along: it's at the point where we're starting to run this
on real devices as part of an updated Android OTA update flow, where we're
using this to provide an Android-specific compressed backing store for
dm-snap-persistent.  The bulk of that project is scattered throughout the
various Android trees, so there are kselftests and a (somewhat bare for now)
Documentation entry with the intent of making this a self-contained
contribution.  There's a lot to the Android userspace daemon, but it doesn't
interact with dm-user in a very complex manner.

This is still in a somewhat early stage, but it's at the point where things
largely function.  I'm certainly not ready to commit to the user ABI
implemented here and there are a bunch of FIXMEs scattered throughout the code,
but I do think that it's far along enough to begin a more concrete discussion
of where folks would like to go with something like this.  While I'd intending
on sorting that stuff out, I'd like to at least get a feel for whether this is
a path worth pursuing before spending a bunch more time on it.

I haven't done much in the way of performance analysis for dm-user.  Earlier on
I did some simple throughput tests and found that dm-user/ext4 was faster than
half the speed of tmpfs, which is way out of the realm of being an issue for
our use case (decompressing blocks out of a phone's storage).  The design of
dm-user does preclude an extremely high performance implementation, where I
assume one would want an explicit ring buffer and zero copy, but I feel like
users who want that degree of performance are probably better served writing a
proper kernel driver.  I wouldn't be opposed to pushing on performance (ideally
without a major design change), but for now I feel like time is better spent
fortifying the user ABI and fixing the various issues with the implementation.

The patches follow as usual, but in case it's easier I've published a tree as
well:

    git://git.kernel.org/pub/scm/linux/kernel/git/palmer/dm-user.git -b dm-user-v1



             reply	other threads:[~2020-12-03 22:34 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-12-03 21:58 Palmer Dabbelt [this message]
2020-12-03 21:58 ` [PATCH v1 1/5] Documentation: Describe dm-user Palmer Dabbelt
2020-12-03 21:58 ` [PATCH v1 2/5] uapi: Add dm-user structure definition Palmer Dabbelt
2020-12-03 21:58 ` [PATCH v1 3/5] dm: dm-user: New target that proxies BIOs to userspace Palmer Dabbelt
2020-12-03 21:58 ` [PATCH v1 4/5] selftests/dm-user: A handful of tests for dm-user Palmer Dabbelt
2020-12-03 21:58 ` [PATCH v1 5/5] MAINTAINERS: Add myself as a dm-user maintainer Palmer Dabbelt
2020-12-04 10:33 ` [PATCH v1 0/5] dm: dm-user: New target that proxies BIOs to userspace Christoph Hellwig
2020-12-07 18:55   ` Palmer Dabbelt
2020-12-10  3:38     ` [dm-devel] " Bart Van Assche
2020-12-10 17:03       ` Josef Bacik
2020-12-15  3:00         ` Palmer Dabbelt
2020-12-16 18:24           ` Vitaly Mayatskih
2020-12-17  6:55             ` Palmer Dabbelt
2020-12-22 13:32           ` Christoph Hellwig
2020-12-22 14:36             ` Mike Snitzer
2020-12-22 20:38               ` Palmer Dabbelt
2020-12-23  7:48                 ` Christoph Hellwig
2020-12-23 16:59                   ` Bart Van Assche
2020-12-22 20:31             ` [dm-devel] " Palmer Dabbelt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201203215859.2719888-1-palmer@dabbelt.com \
    --to=palmer@dabbelt.com \
    --cc=agk@redhat.com \
    --cc=corbet@lwn.net \
    --cc=dm-devel@redhat.com \
    --cc=kernel-team@android.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=linux-raid@vger.kernel.org \
    --cc=palmerdabbelt@google.com \
    --cc=shuah@kernel.org \
    --cc=snitzer@redhat.com \
    --cc=song@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).