From: Ilya Dryomov <idryomov@gmail.com>
To: Al Viro <viro@zeniv.linux.org.uk>
Cc: Jeff Layton <jlayton@redhat.com>, "Yan, Zheng" <zyan@redhat.com>,
Sage Weil <sage@redhat.com>,
Ceph Development <ceph-devel@vger.kernel.org>,
linux-fsdevel <linux-fsdevel@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"Zhu, Caifeng" <zhucaifeng@unissoft-nj.com>
Subject: Re: [PATCH v2] ceph/iov_iter: fix bad iov_iter handling in ceph splice codepaths
Date: Thu, 12 Jan 2017 12:13:31 +0100 [thread overview]
Message-ID: <CAOi1vP_EB5Eq6eGsDzVyYQpwDRYk=3JuTnro_EJRfqTNcYKknQ@mail.gmail.com> (raw)
In-Reply-To: <20170112075946.GU1555@ZenIV.linux.org.uk>
On Thu, Jan 12, 2017 at 8:59 AM, Al Viro <viro@zeniv.linux.org.uk> wrote:
> On Tue, Jan 10, 2017 at 07:57:31AM -0500, Jeff Layton wrote:
>> v2: fix bug in offset handling in iov_iter_pvec_size
>>
>> xfstest generic/095 triggers soft lockups in kcephfs. Basically it uses
>> fio to drive some I/O via vmsplice ane splice. Ceph then ends up trying
>> to access an ITER_BVEC type iov_iter as a ITER_IOVEC one. That causes it
>> to pick up a wrong offset and get stuck in an infinite loop while trying
>> to populate the page array. dio_get_pagev_size has a similar problem.
>>
>> To fix the first problem, add a new iov_iter helper to determine the
>> offset into the page for the current segment and have ceph call that.
>> I would just replace dio_get_pages_alloc with iov_iter_get_pages_alloc,
>> but that will only return a single page at a time for ITER_BVEC and
>> it's better to make larger requests when possible.
>>
>> For the second problem, we simply replace it with a new helper that does
>> what it does, but properly for all iov_iter types.
>>
>> Since we're moving that into generic code, we can also utilize the
>> iterate_all_kinds macro to simplify this. That means that we need to
>> rework the logic a bit since we can't advance to the next vector while
>> checking the current one.
>
> Yecchhh... That really looks like exposing way too low-level stuff instead
> of coming up with saner primitive ;-/
>
> Is page vector + offset in the first page + number of bytes really what
> ceph wants? Would e.g. an array of bio_vec be saner? Because _that_
> would make a lot more natural iov_iter_get_pages_alloc() analogue...
>
> And yes, I realize that you have ->pages wired into the struct ceph_osd_request;
> how painful would it be to have it switched to struct bio_vec array instead?
It would be a significant and wide-reaching change, but I've been
meaning to look into switching to iov_iter for a couple of releases
now. There is a lot of ugly code in net/ceph/messenger.c to hangle
iteration over "page vectors", "page lists" and "bio lists". All of it
predates iov_iter proliferation and is mostly incomplete anyway: IIRC
you can send out of a pagelist but can't recv into a pagelist, etc.
That said, Jeff's patch doesn't look too bad to me...
Thanks,
Ilya
next prev parent reply other threads:[~2017-01-12 11:13 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-01-06 18:23 [PATCH] ceph/iov_iter: fix bad iov_iter handling in ceph splice codepaths Jeff Layton
2017-01-06 20:36 ` Jeff Layton
2017-01-09 23:11 ` Jeff Layton
2017-01-10 12:57 ` [PATCH v2] " Jeff Layton
2017-01-11 2:42 ` Yan, Zheng
2017-01-12 7:59 ` Al Viro
2017-01-12 11:13 ` Ilya Dryomov [this message]
2017-01-12 11:37 ` Al Viro
2017-01-12 11:46 ` Ilya Dryomov
2017-01-12 11:53 ` Al Viro
2017-01-12 12:17 ` Ilya Dryomov
2017-01-12 11:57 ` Jeff Layton
2017-01-12 11:27 ` Jeff Layton
2017-01-12 11:37 ` Ilya Dryomov
2017-01-18 12:14 ` Jeff Layton
2017-01-24 15:00 ` Jeff Layton
2017-01-24 20:51 ` Jeff Layton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAOi1vP_EB5Eq6eGsDzVyYQpwDRYk=3JuTnro_EJRfqTNcYKknQ@mail.gmail.com' \
--to=idryomov@gmail.com \
--cc=ceph-devel@vger.kernel.org \
--cc=jlayton@redhat.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=sage@redhat.com \
--cc=viro@zeniv.linux.org.uk \
--cc=zhucaifeng@unissoft-nj.com \
--cc=zyan@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).