Re: [PATCH 1/2] Block layer filter - second version

From: Sergei Shtepa <sergei.shtepa@veeam.com>
To: Matthew Wilcox <willy@infradead.org>
Cc: Damien Le Moal <Damien.LeMoal@wdc.com>,
	"axboe@kernel.dk" <axboe@kernel.dk>,
	"viro@zeniv.linux.org.uk" <viro@zeniv.linux.org.uk>,
	"hch@infradead.org" <hch@infradead.org>,
	"darrick.wong@oracle.com" <darrick.wong@oracle.com>,
	"linux-xfs@vger.kernel.org" <linux-xfs@vger.kernel.org>,
	"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
	"rjw@rjwysocki.net" <rjw@rjwysocki.net>,
	"len.brown@intel.com" <len.brown@intel.com>,
	"pavel@ucw.cz" <pavel@ucw.cz>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	Johannes Thumshirn <Johannes.Thumshirn@wdc.com>,
	"ming.lei@redhat.com" <ming.lei@redhat.com>,
	"jack@suse.cz" <jack@suse.cz>, "tj@kernel.org" <tj@kernel.org>,
	"gustavo@embeddedor.com" <gustavo@embeddedor.com>,
	"bvanassche@acm.org" <bvanassche@acm.org>,
	"osandov@fb.com" <osandov@fb.com>,
	"koct9i@gmail.com" <koct9i@gmail.com>,
	"steve@sk2.org" <steve@sk2.org>,
	"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-pm@vger.kernel.org" <linux-pm@vger.kernel.org>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>
Subject: Re: [PATCH 1/2] Block layer filter - second version
Date: Wed, 21 Oct 2020 15:55:55 +0300	[thread overview]
Message-ID: <20201021125555.GE20749@veeam.com> (raw)
In-Reply-To: <20201021114438.GK20115@casper.infradead.org>

The 10/21/2020 14:44, Matthew Wilcox wrote:
> On Wed, Oct 21, 2020 at 09:21:36AM +0000, Damien Le Moal wrote:
> > > + * submit_bio_direct - submit a bio to the block device layer for I/O
> > > + * bypass filter.
> > > + * @bio:  The bio describing the location in memory and on the device.
> > >   *
> > > + * Description:
> 
> You don't need this line.
> 
> > > + *    This is a version of submit_bio() that shall only be used for I/O
> > > + *    that cannot be intercepted by block layer filters.
> > > + *    All file systems and other upper level users of the block layer
> > > + *    should use submit_bio() instead.
> > > + *    Use this function to access the swap partition and directly access
> > > + *    the block device file.
> 
> I don't understand why O_DIRECT gets to bypass the block filter.  Nor do
> I understand why anybody would place a block filter on the swap device.
> But if somebody did place a filter on the swap device, why should swap
> be able to bypass the filter?
> 

I am very happy to hear such a question. You are really trying to
understand the algorithm.

Yes, intercepting the swap partition is absurd. But we can't guarantee
that the filter won't intercept swap.

Swap operation is related to the memory allocation logic. If a swap on
the block device are accessed during memory allocation from filter,
a deadlock occurs. We can allow filters to occasionally shoot off their
feet, especially under high load. But I think it's better not to do it.

"directly access" - it is not O_DIRECT. This means (I think) direct
reading from the device file, like "dd if=/dev/sda1".
As for intercepting direct reading, I don't know how to do the right thing.

The problem here is that in fs/block_dev.c in function __blkdev_direct_IO()
uses the qc - value returned by the submit_bio() function.
This value is used below when calling 
blk_poll(bdev_get_queue(dev), qc, true).
The filter cannot return a meaningful value of the blk_qc_t type when
intercepting a request, because at that time it does not know which queue
the request will fall into.

If function submit_bio() will always return BLK_QC_T_NONE - I think the
algorithm of the __blk dev_direct_IO() will not work correctly.
If we need to intercept direct access to a block device, we need to at
least redo the __blkdev_direct_IO function, getting rid of blk_pool.
I'm not sure it's necessary yet.

-- 
Sergei Shtepa
Veeam Software developer.