File create performance between NFSv3 and NFSv4.

From: Pradeep <pradeep.thomas@gmail.com>
To: Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: File create performance between NFSv3 and NFSv4.
Date: Tue, 30 Jun 2020 17:55:15 -0700	[thread overview]
Message-ID: <CAD8zhTBCEbOkA_jFzzDXcd0jiH9ZgUh1KUJAJzhnWFMC+JGjNA@mail.gmail.com> (raw)

Hello,

While testing a performance issue, I noticed that the number of
creates that an NFS client can do over NFSv4 is significantly lower
compared to NFSv3. I'm using the test below:

https://github.com/distributed-system-analysis/smallfile

Command run:
smallfile_cli.py  --operation create --threads 128 --file-size 1
--files 1024 --top <nfs-dir>
[ 128 threads, each doing 1024 file creates of 1K size]

This gives around 1169 creates/sec with NFSv4.1 and 8073 creates/sec
with NFSv3. This is with the exact same client and server. NFS server
is tuned to use 200 threads.

When I looked at tcpdump, I noticed that over NFSv3 multiple parallel
requests are being sent and NFSv4 is pretty much serial. Is there
anything that I can tune to improve NFSv4 performance w.r.t to file
creations?

This also shows up in benchmarks like SpecFS SWBUILD's INIT phase
where millions of files are getting created.

Thanks,
Pradeep