NFSD high CPU usage

NFSD high CPU usage

[Заболотный Андрей]

Hello!

I'm trying to narrow down a problem I've got, and I'm asking for help from knowledgeable people.
I'm sorry if this is the wrong place to ask, please direct me to the right place if I'm wrong.

I have a simple NAS which is built around a dual-core 300MHz ARM with linux 2.6.31, and has a gigabit Ethernet controller.
It can share a SATA hard drive (I have a 2Tb drive installed) over the network using NFSv3 and SMB.
I did a comparison of the raw download speed from NAS hard disk over the GbE to my PC using NFSv3 and SAMBA.
Basically, I just copy a large enough file (several gigabytes) from the mounted NAS share to /dev/null.

With SAMBA I get about 53Mbyte/second (over 400 Megabits) and NAS CPU usage of a single CPU ARM core is about 26%, the other core is idle.

With NFS I get about 17MByte/second when using one NFSD thread and about 28MByte/second when using two and more NFSD threads. In this case the limiting factor is CPU as I can see a 100% CPU load on one or both cores. I tried using TCP or UDP, played with rsize/wsize but this does not change the results much.

I have the kernel sources from the manufacturer, so my idea is to fix the problem and update the kernel on NAS.

So, my questions are:

1. Are there any changes in kernel NFSD since kernel 2.6.31 that could influence CPU usage? I could try to backport the patches.

2. What could cause this high CPU usage on just sequential reads from a file on a NFS drive? In fs/nfsd/vfs.c I see that the nfsd_read() function will open the file, read some bytes, and close it - on every read operation? Is this the cause of high CPU load? 
Also, I can see that NFSv4 does not do the open/close on every read - is NFSv4 CPU usage lower when issuing many sequential reads?

-- 
Andrew