From: "Benjamin Coddington" <bcodding@redhat.com>
To: "Trond Myklebust" <trondmy@hammerspace.com>
Cc: linux-nfs@vger.kernel.org
Subject: Re: [PATCH v3 00/17] Readdir enhancements
Date: Sun, 08 Nov 2020 06:05:46 -0500 [thread overview]
Message-ID: <0629C8A7-5E4F-4BC0-8346-B8497FDE81F2@redhat.com> (raw)
In-Reply-To: <60540b203a4a707ba9f16fe77ed0cb02aa547ac9.camel@hammerspace.com>
On 7 Nov 2020, at 9:23, Trond Myklebust wrote:
> On Sat, 2020-11-07 at 07:49 -0500, Benjamin Coddington wrote:
>> On 4 Nov 2020, at 11:16, trondmy@gmail.com wrote:
>>
>>> From: Trond Myklebust <trond.myklebust@hammerspace.com>
>>>
>>> The following patch series performs a number of cleanups on the
>>> readdir
>>> code.
>>> It also adds support for 1MB readdir RPC calls on-the-wire, and
>>> modifies
>>> the caching code to ensure that we cache the entire contents of
>>> that
>>> 1MB call (instead of discarding the data that doesn't fit into a
>>> single
>>> page).
>>>
>>> v2: Fix the handling of the NFSv3/v4 directory verifier
>>> v3: Optimise searching when the readdir cookies are seen to be
>>> ordered
>>
>> Hi Trond, thanks for these.
>>
>> I did a bit of testing with these on 4-core/4G client listing 1.5M
>> files
>> with READDIR. I compared v5.10-rc2 without/with this set.
>>
>> +------+ v5.10.rc-2 +--+ this v3 patch set +
>>> run | time | rpc calls | | time | rpc calls |
>>
>> nfsv3 with dtsize 262144:
>> +------+---------+-----------+--+--------+-----------+
>>> 1 | 81.583 | 14710 | | 53.568 | 215 |
>>> 2 | 81.147 | 14710 | | 50.781 | 215 |
>>> 3 | 81.61 | 14710 | | 50.514 | 215 |
>>> 4 | 82.405 | 14710 | | 50.746 | 215 |
>>> 5 | 82.066 | 14710 | | 50.397 | 215 |
>>> 6 | 82.395 | 14710 | | 50.892 | 215 |
>>> 7 | 81.657 | 14710 | | 50.882 | 215 |
>>> 8 | 81.555 | 14710 | | 50.981 | 215 |
>>> 9 | 81.421 | 14710 | | 50.558 | 215 |
>>> 10 | 81.472 | 14710 | | 50.588 | 215 |
>>
>> nfsv3 with dtsize 1048576:
>> +------+---------+-----------+--+--------+-----------+
>>> 1 | 81.563 | 14710 | | 52.692 | 61 |
>>> 2 | 82.123 | 14710 | | 49.934 | 61 |
>>> 3 | 81.714 | 14710 | | 50.158 | 61 |
>>> 4 | 81.707 | 14710 | | 50.083 | 61 |
>>> 5 | 81.44 | 14710 | | 50.045 | 61 |
>>> 6 | 81.685 | 14710 | | 50.021 | 61 |
>>> 7 | 81.17 | 14710 | | 50.131 | 61 |
>>> 8 | 81.366 | 14710 | | 49.928 | 61 |
>>> 9 | 81.067 | 14710 | | 50.081 | 61 |
>>> 10 | 81.524 | 14710 | | 50.442 | 61 |
>>
>> nfsv4 with dtsize 32768:
>> +------+---------+-----------+--+--------+-----------+
>>> 1 | 99.534 | 14712 | | 79.461 | 331 |
>>> 2 | 98.998 | 14712 | | 79.338 | 331 |
>>> 3 | 99.462 | 14712 | | 81.101 | 331 |
>>> 4 | 99.891 | 14712 | | 78.888 | 331 |
>>> 5 | 99.516 | 14712 | | 81.147 | 331 |
>>> 6 | 98.649 | 14712 | | 83.084 | 331 |
>>> 7 | 101.159 | 14712 | | 80.461 | 331 |
>>> 8 | 100.402 | 14712 | | 79.003 | 331 |
>>> 9 | 98.548 | 14712 | | 80.619 | 331 |
>>> 10 | 97.456 | 14712 | | 81.317 | 331 |
>>
>> nfsv4 with dtsize 1048576:
>> +------+---------+-----------+--+--------+-----------+
>>> 1 | 100.357 | 14712 | | 78.976 | 91 |
>>> 2 | 99.61 | 14712 | | 79.328 | 91 |
>>> 3 | 101.095 | 14712 | | 80.649 | 91 |
>>> 4 | 107.904 | 14712 | | 78.285 | 91 |
>>> 5 | 103.665 | 14712 | | 79.258 | 91 |
>>> 6 | 98.877 | 14712 | | 78.817 | 91 |
>>> 7 | 99.567 | 14712 | | 81.11 | 91 |
>>> 8 | 99.096 | 14712 | | 80.296 | 91 |
>>> 9 | 100.124 | 14712 | | 78.865 | 91 |
>>> 10 | 100.603 | 14712 | | 79.143 | 91 |
>>
>> These look great. Feel free to add either/both of my:
>> Reviewed-by: Benjamin Coddington <bcodding@redhat.com>
>> Tested-by: Benjamin Coddington <bcodding@redhat.com>
>
> Thanks again for testing! I missed this email before sending out v4,
> but since that only adds 2 new patches to the series to deal with
> Dave's v. large changing directory case, I assume I can apply the above
> tags to the rest anyway as they have not changed?
Yes, I'll check those out too.
Ben
next prev parent reply other threads:[~2020-11-08 11:05 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-04 16:16 [PATCH v3 00/17] Readdir enhancements trondmy
2020-11-04 16:16 ` [PATCH v3 01/17] NFS: Ensure contents of struct nfs_open_dir_context are consistent trondmy
2020-11-04 16:16 ` [PATCH v3 02/17] NFS: Clean up readdir struct nfs_cache_array trondmy
2020-11-04 16:16 ` [PATCH v3 03/17] NFS: Clean up nfs_readdir_page_filler() trondmy
2020-11-04 16:16 ` [PATCH v3 04/17] NFS: Clean up directory array handling trondmy
2020-11-04 16:16 ` [PATCH v3 05/17] NFS: Don't discard readdir results trondmy
2020-11-04 16:16 ` [PATCH v3 06/17] NFS: Remove unnecessary kmap in nfs_readdir_xdr_to_array() trondmy
2020-11-04 16:16 ` [PATCH v3 07/17] NFS: Replace kmap() with kmap_atomic() in nfs_readdir_search_array() trondmy
2020-11-04 16:16 ` [PATCH v3 08/17] NFS: Simplify struct nfs_cache_array_entry trondmy
2020-11-04 16:16 ` [PATCH v3 09/17] NFS: Support larger readdir buffers trondmy
2020-11-04 16:16 ` [PATCH v3 10/17] NFS: More readdir cleanups trondmy
2020-11-04 16:16 ` [PATCH v3 11/17] NFS: nfs_do_filldir() does not return a value trondmy
2020-11-04 16:16 ` [PATCH v3 12/17] NFS: Reduce readdir stack usage trondmy
2020-11-04 16:16 ` [PATCH v3 13/17] NFS: Cleanup to remove nfs_readdir_descriptor_t typedef trondmy
2020-11-04 16:16 ` [PATCH v3 14/17] NFS: Allow the NFS generic code to pass in a verifier to readdir trondmy
2020-11-04 16:16 ` [PATCH v3 15/17] NFS: Handle NFS4ERR_NOT_SAME and NFSERR_BADCOOKIE from readdir calls trondmy
2020-11-04 16:16 ` [PATCH v3 16/17] NFS: Improve handling of directory verifiers trondmy
2020-11-04 16:16 ` [PATCH v3 17/17] NFS: Optimisations for monotonically increasing readdir cookies trondmy
2020-11-04 21:01 ` [PATCH v3 16/17] NFS: Improve handling of directory verifiers David Wysochanski
2020-11-04 21:31 ` Trond Myklebust
2020-11-04 21:40 ` David Wysochanski
2020-11-06 13:30 ` [PATCH v3 05/17] NFS: Don't discard readdir results David Wysochanski
2020-11-06 15:05 ` Trond Myklebust
2020-11-06 18:00 ` David Wysochanski
2020-11-07 12:49 ` [PATCH v3 00/17] Readdir enhancements Benjamin Coddington
2020-11-07 14:23 ` Trond Myklebust
2020-11-08 11:05 ` Benjamin Coddington [this message]
2020-11-08 18:15 ` Mkrtchyan, Tigran
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0629C8A7-5E4F-4BC0-8346-B8497FDE81F2@redhat.com \
--to=bcodding@redhat.com \
--cc=linux-nfs@vger.kernel.org \
--cc=trondmy@hammerspace.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.