Re: [PATCH 2/2] btrfs: add ioctl for directly writing compressed data

From: Dave Chinner <david@fromorbit.com>
To: Johannes Thumshirn <jthumshirn@suse.de>
Cc: Omar Sandoval <osandov@osandov.com>,
	linux-btrfs@vger.kernel.org, kernel-team@fb.com,
	linux-fsdevel@vger.kernel.org
Subject: Re: [PATCH 2/2] btrfs: add ioctl for directly writing compressed data
Date: Fri, 6 Sep 2019 17:46:54 +1000	[thread overview]
Message-ID: <20190906074654.GM7777@dread.disaster.area> (raw)
In-Reply-To: <8acbff04-aee0-9f88-b2cd-a85bb7b94df8@suse.de>

On Thu, Sep 05, 2019 at 02:16:37PM +0200, Johannes Thumshirn wrote:
> On 05/09/2019 04:10, Dave Chinner wrote:
> > On Wed, Sep 04, 2019 at 12:13:26PM -0700, Omar Sandoval wrote:
> >> From: Omar Sandoval <osandov@fb.com>
> >>
> >> This adds an API for writing compressed data directly to the filesystem.
> >> The use case that I have in mind is send/receive: currently, when
> >> sending data from one compressed filesystem to another, the sending side
> >> decompresses the data and the receiving side recompresses it before
> >> writing it out. This is wasteful and can be avoided if we can just send
> >> and write compressed extents. The send part will be implemented in a
> >> separate series, as this ioctl can stand alone.
> >>
> >> The interface is essentially pwrite(2) with some extra information:
> >>
> >> - The input buffer contains the compressed data.
> >> - Both the compressed and decompressed sizes of the data are given.
> >> - The compression type (zlib, lzo, or zstd) is given.
> > 
> > So why can't you do this with pwritev2()? Heaps of flags, and
> > use a second iovec to hold the decompressed size of the previous
> > iovec. i.e.
> > 
> > 	iov[0].iov_base = compressed_data;
> > 	iov[0].iov_len = compressed_size;
> > 	iov[1].iov_base = NULL;
> > 	iov[1].iov_len = uncompressed_size;
> > 	pwritev2(fd, iov, 2, offset, RWF_COMPRESSED_ZLIB);
> > 
> > And you don't need to reinvent pwritev() with some whacky ioctl that
> > is bound to be completely screwed up is ways not noticed until
> > someone else tries to use it...
> > 
> > I'd also suggest atht if we are going to be able to write compressed
> > data directly, then we should be able to read them as well directly
> > via preadv2()....
> 
> 
> While I'm with you on this from a design PoV, one question remains:
> What to do with the file systems that do not support compression?

EINVAL.

> Currently there's only a kernel global check for known RWF_* flags in
> kiocb_set_rw_flags().

That's really an implementation detail, there's lots of ways of
doing it, probably a superblock feature flag would be the easiest
way to specify support for a filesystem supporting direct read/write
of compressed data.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com