From: bfields@fieldses.org (J. Bruce Fields)
To: Jason L Tibbitts III <tibbs@math.uh.edu>
Cc: linux-nfs@vger.kernel.org, km@cm4all.com, linux-kernel@vger.kernel.org
Subject: Re: Regression in 5.1.20: Reading long directory fails
Date: Wed, 28 Aug 2019 13:46:09 -0400 [thread overview]
Message-ID: <20190828174609.GB29148@fieldses.org> (raw)
In-Reply-To: <ufapnkxkn0x.fsf@epithumia.math.uh.edu>
On Thu, Aug 22, 2019 at 02:39:26PM -0500, Jason L Tibbitts III wrote:
> I now have another user reporting the same failure of readdir on a long
> directory which showed up in 5.1.20 and was traced to
> 3536b79ba75ba44b9ac1a9f1634f2e833bbb735c. I'm not sure what to do to
> get more traction besides reposting and adding some addresses to the CC
> list. If there is any information I can provide which might help to get
> to the bottom of this, please let me know.
>
> To recap:
>
> 5.1.20 introduced a regression reading some large directories. In this
> case, the directory should have 7800 files or so in it:
>
> [root@ld00 ~]# ls -l ~dblecher|wc -l
> ls: reading directory '/home/dblecher': Input/output error
> 1844
> [root@ld00 ~]# cat /proc/version Linux version 5.1.20-300.fc30.x86_64 (mockbuild@bkernel04.phx2.fedoraproject.org) (gcc version 9.1.1 20190503 (Red Hat 9.1.1-1) (GCC)) #1 SMP Fri Jul 26 15:03:11 UTC 2019
>
> (The server is a Centos 7 machine running kernel 3.10.0-957.12.2.el7.x86_64.)
>
> Building a kernel which reverts commit 3536b79ba75ba44b9ac1a9f1634f2e833bbb735c:
> Revert "NFS: readdirplus optimization by cache mechanism" (memleak)
Looks like that's db531db951f950b8 upstream. (Do you know if it's
reproduceable upstream as well?)
> fixes the issue, but of course that revert was fixing a real issue so
> I'm not sure what to do.
>
> I can trivially reproduce this by simply trying to list the problematic
> directories but I'm not sure how to construct such a directory; simply
> creating 10000 files doesn't cause the problem for me.
Maybe it depends on having names of the right length to place some bit
of xdr on a boundary. I wonder if it'd be possible to reproduce just by
varying the name lengths randomly till you hit it.
The fact that the problematic patch fixed a memory leak also makes me
wonder if it might have gone to far and freed something out from under
the readdir code.
> I am willing to
> test patches and can build my own kernels, and I'm happy to provide any
> debugging information you might require. Unfortunately I don't know
> enough to dig in and figure out for myself what's going wrong.
>
> I did file https://bugzilla.redhat.com/show_bug.cgi?id=1740954 just to
> have this in a bug tracker somewhere. I'm happy to file one somewhere
> else if that would help.
No clever debugging ideas off the top of my head, I'm afraid. I might
start by patching the kernel or doing some tracing to figure out exactly
where that EIO is being generated?
--b.
next prev parent reply other threads:[~2019-08-28 17:46 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <ufak1bhyuew.fsf@epithumia.math.uh.edu>
2019-08-22 19:39 ` Regression in 5.1.20: Reading long directory fails Jason L Tibbitts III
2019-08-28 17:46 ` J. Bruce Fields [this message]
2019-08-28 18:29 ` Jason L Tibbitts III
2019-08-28 18:33 ` J. Bruce Fields
2019-09-03 15:49 ` Jason L Tibbitts III
2019-09-03 18:02 ` Wolfgang Walter
2019-09-03 19:06 ` Jason L Tibbitts III
2019-09-03 19:08 ` Chuck Lever
2019-09-03 21:37 ` Wolfgang Walter
2019-09-04 1:50 ` Jason L Tibbitts III
2019-09-06 14:48 ` J. Bruce Fields
2019-09-06 20:47 ` Jason L Tibbitts III
2019-09-06 20:50 ` Chuck Lever
2019-09-08 11:39 ` Benjamin Coddington
2019-09-08 15:19 ` Trond Myklebust
2019-09-08 15:48 ` Chuck Lever
2019-09-08 16:47 ` Trond Myklebust
2019-09-08 16:51 ` Chuck Lever
2019-09-11 16:25 ` Benjamin Coddington
2019-09-11 16:39 ` Chuck Lever
2019-09-11 17:26 ` Benjamin Coddington
2019-09-11 17:27 ` Benjamin Coddington
2019-09-11 17:29 ` Chuck Lever
2019-09-11 17:40 ` Benjamin Coddington
2019-09-11 17:43 ` Chuck Lever
2019-09-11 17:59 ` Benjamin Coddington
2019-09-11 17:50 ` Benjamin Coddington
2019-09-11 17:54 ` Chuck Lever
2019-09-12 12:29 ` Benjamin Coddington
2019-09-12 12:53 ` Trond Myklebust
2019-09-12 13:08 ` Benjamin Coddington
2019-09-12 13:13 ` J. Bruce Fields
2019-09-12 13:25 ` Trond Myklebust
2019-09-12 13:35 ` Benjamin Coddington
2019-09-12 13:14 ` Trond Myklebust
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190828174609.GB29148@fieldses.org \
--to=bfields@fieldses.org \
--cc=km@cm4all.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nfs@vger.kernel.org \
--cc=tibbs@math.uh.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).