[BUG] NFS in 2.4.20-pre6+ stalls

[BUG] NFS in 2.4.20-pre6+ stalls

[Richard Gooch]

  Hi, all. Just noticed this with 2.4.20-pre6 (and -pre7): NFS write
sometimes (usually) stalls for minutes at a time. This problem wasn't
there on 2.4.19. I've noticed this when writing a files around 1 MiB
or so (some a bit larger, some a bit smaller). It makes NFS almost
unusable. I've appended the kernel logs which come, at no extra
charge, with the problem. On the server, I see partial files (integral
number of wsize= blocks) during the stall. Eventually, the client
seems to recover and the rest of the file is written. The writing
application is in TASK_INTERRUPTIBLE state.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca
===============================================================================
nfs: task 125 can't get a request slot
nfs: task 126 can't get a request slot
nfs: task 127 can't get a request slot
nfs: task 128 can't get a request slot
nfs: task 129 can't get a request slot
nfs: task 130 can't get a request slot
nfs: task 131 can't get a request slot
nfs: task 132 can't get a request slot
nfs: task 133 can't get a request slot
nfs: task 134 can't get a request slot
nfs: task 135 can't get a request slot
nfs: task 136 can't get a request slot
nfs: task 137 can't get a request slot
nfs: task 138 can't get a request slot
nfs: task 139 can't get a request slot
nfs: task 140 can't get a request slot
nfs: task 141 can't get a request slot
nfs: task 142 can't get a request slot
nfs: task 143 can't get a request slot
nfs: server fileserver OK
nfs: server fileserver OK
nfs: server fileserver OK
nfs: server fileserver OK
nfs: server fileserver OK
nfs: server fileserver OK
nfs: server fileserver OK
nfs: server fileserver OK
nfs: server fileserver OK
nfs: server fileserver OK
nfs: server fileserver OK
nfs: server fileserver OK
nfs: server fileserver OK
nfs: server fileserver OK
nfs: server fileserver OK
nfs: server fileserver OK
nfs: server fileserver OK
nfs: server fileserver OK
nfs: server fileserver OK

Re: [BUG] NFS in 2.4.20-pre6+ stalls

[Alan Cox]

On Mon, 2002-09-16 at 23:54, Richard Gooch wrote:
>   Hi, all. Just noticed this with 2.4.20-pre6 (and -pre7): NFS write
> sometimes (usually) stalls for minutes at a time. This problem wasn't
> there on 2.4.19. I've noticed this when writing a files around 1 MiB
> or so (some a bit larger, some a bit smaller). It makes NFS almost
> unusable. I've appended the kernel logs which come, at no extra

I've reported the same to Marcelo. Its there in a slightly different
form in my case - low bandwidth streaming data shows it up very well.

Re: [BUG] NFS in 2.4.20-pre6+ stalls

[Bill Davidsen]

On 17 Sep 2002, Alan Cox wrote:

> On Mon, 2002-09-16 at 23:54, Richard Gooch wrote:
> >   Hi, all. Just noticed this with 2.4.20-pre6 (and -pre7): NFS write
> > sometimes (usually) stalls for minutes at a time. This problem wasn't
> > there on 2.4.19. I've noticed this when writing a files around 1 MiB
> > or so (some a bit larger, some a bit smaller). It makes NFS almost
> > unusable. I've appended the kernel logs which come, at no extra
> 
> I've reported the same to Marcelo. Its there in a slightly different
> form in my case - low bandwidth streaming data shows it up very well.

This may explain why my favorite Internet radio station is acting up.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

RE: [BUG] NFS in 2.4.20-pre6+ stalls

[Richard Gooch]

Trond Myklebust writes:
> >>>>> " " == Richard Gooch <rgooch@ras.ucalgary.ca> writes:
> 
>      > Monday afternoon (my time) I grabbed the current BK tree and
>      > have been using it since then. The problem has been reduced. I
>      > thought it was fixed, but just as I sent this message, I tried
>      > my little test again, and in one of the three runs I had a 21
>      > second stall. I waited a few minutes between tests.
> 
>      > Is your patch meant for the latest BK tree? Or something older?
> 
> The patch should apply fine to the latest BK tree: there have been no
> recent changes that might affect it.

Well, your patch doesn't make things worse. I tried to reproduce the
problem again, but couldn't. I'm running 2.4.20-pre8 plus your patch,
and will keep an eye on it.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

RE: [BUG] NFS in 2.4.20-pre6+ stalls

[Trond Myklebust]

>>>>> " " == Richard Gooch <rgooch@ras.ucalgary.ca> writes:

     > Monday afternoon (my time) I grabbed the current BK tree and
     > have been using it since then. The problem has been reduced. I
     > thought it was fixed, but just as I sent this message, I tried
     > my little test again, and in one of the three runs I had a 21
     > second stall. I waited a few minutes between tests.

     > Is your patch meant for the latest BK tree? Or something older?

The patch should apply fine to the latest BK tree: there have been no
recent changes that might affect it.

Cheers,
   Trond

RE: [BUG] NFS in 2.4.20-pre6+ stalls

[Richard Gooch]

Trond Myklebust writes:
> >   Hi, all. Just noticed this with 2.4.20-pre6 (and -pre7): NFS write
> > sometimes (usually) stalls for minutes at a time. This problem wasn't
> 
> Richard,
>   Does the appended patch make a difference?

Monday afternoon (my time) I grabbed the current BK tree and have been
using it since then. The problem has been reduced. I thought it was
fixed, but just as I sent this message, I tried my little test again,
and in one of the three runs I had a 21 second stall. I waited a few
minutes between tests.

Is your patch meant for the latest BK tree? Or something older?

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

RE: [BUG] NFS in 2.4.20-pre6+ stalls

[Trond Myklebust]

[-- Attachment #1: Type: text/plain, Size: 216 bytes --]

>   Hi, all. Just noticed this with 2.4.20-pre6 (and -pre7): NFS write
> sometimes (usually) stalls for minutes at a time. This problem wasn't

Richard,
  Does the appended patch make a difference?

Cheers,
  Trond


[-- Attachment #2: resend.dif --]
[-- Type: text/plain, Size: 910 bytes --]

--- linux/net/sunrpc/xprt.c.orig	Fri Aug 30 20:16:17 2002
+++ linux/net/sunrpc/xprt.c	Tue Sep 24 00:08:59 2002
@@ -171,10 +171,10 @@
 
 	if (xprt->snd_task)
 		return;
-	if (!xprt->nocong && RPCXPRT_CONGESTED(xprt))
-		return;
 	task = rpc_wake_up_next(&xprt->resend);
 	if (!task) {
+		if (!xprt->nocong && RPCXPRT_CONGESTED(xprt))
+			return;
 		task = rpc_wake_up_next(&xprt->sending);
 		if (!task)
 			return;
@@ -1013,7 +1013,6 @@
 		}
 		rpc_inc_timeo(&task->tk_client->cl_rtt);
 		xprt_adjust_cwnd(req->rq_xprt, -ETIMEDOUT);
-		__xprt_put_cong(xprt, req);
 	}
 	req->rq_nresend++;
 
@@ -1150,10 +1149,7 @@
 		req->rq_bytes_sent = 0;
 	}
  out_release:
-	spin_lock_bh(&xprt->sock_lock);
-	__xprt_release_write(xprt, task);
-	__xprt_put_cong(xprt, req);
-	spin_unlock_bh(&xprt->sock_lock);
+	xprt_release_write(xprt, task);
 	return;
  out_receive:
 	dprintk("RPC: %4d xmit complete\n", task->tk_pid);