Collapsing I/O size with unstable writes with NFSv3 on 4.15

Collapsing I/O size with unstable writes with NFSv3 on 4.15

[Ted Carpenter]

I'm testing performance of a clustered NFS server using a 4.15 client.
My test setup involves running IOZone to write a 30GB file to the
server over NFSv3. The issue is that we occasionally see bad
performance, especially when unstable writes are enabled on the
server.

Based on observing the network traffic, we've seen the write I/O size
during good runs stay around 1MB (which is how we've mounted) but in
the bad runs we see often see dips in write size (down to even 4KB or
8KB) towards the end of the test.

I'm trying to understand why we see the dip in I/O size from the NFS
client as it seems to be related to the decrease in performance. Any
ideas what could cause it? Also, any suggestions on where I should
look to start learning about the way the Linux NFS client and page
cache handle unstable writes internally?

Thanks,
Ted Carpenter

Re: Collapsing I/O size with unstable writes with NFSv3 on 4.15

[J. Bruce Fields]

On Wed, Mar 20, 2019 at 08:47:32AM -0700, Ted Carpenter wrote:
> I'm testing performance of a clustered NFS server using a 4.15 client.
> My test setup involves running IOZone to write a 30GB file to the
> server over NFSv3. The issue is that we occasionally see bad
> performance, especially when unstable writes are enabled on the
> server.

Which server, and how do you enable unstable writes?

> Based on observing the network traffic, we've seen the write I/O size
> during good runs stay around 1MB (which is how we've mounted) but in
> the bad runs we see often see dips in write size (down to even 4KB or
> 8KB) towards the end of the test.
> 
> I'm trying to understand why we see the dip in I/O size from the NFS
> client as it seems to be related to the decrease in performance. Any
> ideas what could cause it? Also, any suggestions on where I should
> look to start learning about the way the Linux NFS client and page
> cache handle unstable writes internally?

I'm not sure what to recommend.  We might also need some more details
about the performance results you're seeing.

--b.