* Writing over NFS causes lots of paging
@ 2001-11-11 10:48 Simon Kirby
2001-11-11 18:29 ` Linus Torvalds
0 siblings, 1 reply; 3+ messages in thread
From: Simon Kirby @ 2001-11-11 10:48 UTC (permalink / raw)
To: linux-kernel
It looks like when writing large amounts of data to NFS where the remote
end is slower than the local end the local end appears to start swapping
out a lot I'm guessing this is because it can read much faster than it
can write.
Also, I see NFS timeouts and thus "I/O error" messages fom cp when it is
mounted with the "soft" option, even with high timeouts. "hard" works
fine, but I didn't want to use it for this mount.
For example:
procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
0 0 0 70316 4128 608 141744 0 0 4172 0 3065 4214 1 12 87
0 0 0 70316 3156 620 142704 0 92 3896 92 2904 4028 1 11 88
0 1 0 70316 4208 672 141604 0 40 2912 40 3248 4715 2 12 86
1 0 0 70316 3228 724 142528 0 296 3284 296 2904 4682 5 17 79
0 1 0 70700 4048 720 141948 0 124 3744 124 2987 4109 1 14 85
1 1 0 70700 3752 724 142376 0 116 4136 116 2927 3985 1 16 83
1 0 0 70956 3816 712 142384 0 180 3964 180 2724 3801 1 15 84
0 0 0 70956 3968 720 142308 0 0 3908 0 3045 4277 2 13 84
0 1 0 71724 3336 724 142984 0 580 3796 580 2837 4985 9 21 69
0 0 0 71724 3924 736 144116 0 588 3776 588 2860 3950 1 14 85
0 1 0 72236 3120 752 146132 0 556 3380 556 2731 3983 1 9 89
0 0 0 73260 3212 752 146140 0 2496 3468 2496 2516 3637 1 14 85
0 0 0 73900 3476 744 145868 0 640 3900 640 2776 3888 0 13 87
0 0 0 74156 3192 736 146212 0 540 4150 540 2916 4010 1 15 83
The copy is still running and almost everything is swapped out now
(140 MB). When the copy started, there was about 30 MB of swap.
NFS client (reading from disk and writing through NFS): 2.4.15pre1
NFS server (writing to disk from NFS): 2.4.15pre2
NFSv3 and knfsd used.
Is there something different with the VM here? Should I try 2.4.15pre2
on the NFS client?
Simon-
[ Stormix Technologies Inc. ][ NetNation Communications Inc. ]
[ sim@stormix.com ][ sim@netnation.com ]
[ Opinions expressed are not necessarily those of my employers. ]
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Writing over NFS causes lots of paging
2001-11-11 10:48 Writing over NFS causes lots of paging Simon Kirby
@ 2001-11-11 18:29 ` Linus Torvalds
2001-11-12 3:36 ` Trond Myklebust
0 siblings, 1 reply; 3+ messages in thread
From: Linus Torvalds @ 2001-11-11 18:29 UTC (permalink / raw)
To: sim, linux-kernel
In article <20011111024855.A5893@netnation.com> you write:
>It looks like when writing large amounts of data to NFS where the remote
>end is slower than the local end the local end appears to start swapping
>out a lot I'm guessing this is because it can read much faster than it
>can write.
No, the real reason for why the NFS write stuff causes page-outs is that
the VM layer does not really understand the notion of writeback pages.
The VM layer has one explicit special case: it knows about the magic in
"page->buffers", and can handle writeback for block-oriented devices
sanely. But any non-buffer-oriented filesystem is "invisible" to the VM
layer, and has to use other tricks to make the VM ignore its pages.
In the case of NFS, it increments the page count and has it's own
private non-VM-visible writeback data structures. This pins the page in
memory, but at the same time, because the VM doesn't understand it, the
VM will end up thinking the page is mapped in user space or something
else, and won't know how to start writeouts.
Quite frankly, I don't rightly know what the real fix is. Making
"page->buffers" be a generic thing (a "void *") along with making the
buffer flushing logic be behind a address space operation is probably
the right thing in the long run.
Linus
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Writing over NFS causes lots of paging
2001-11-11 18:29 ` Linus Torvalds
@ 2001-11-12 3:36 ` Trond Myklebust
0 siblings, 0 replies; 3+ messages in thread
From: Trond Myklebust @ 2001-11-12 3:36 UTC (permalink / raw)
To: Linus Torvalds; +Cc: sim, linux-kernel
>>>>> " " == Linus Torvalds <torvalds@transmeta.com> writes:
> The VM layer has one explicit special case: it knows about the
> magic in "page->buffers", and can handle writeback for
> block-oriented devices sanely. But any non-buffer-oriented
> filesystem is "invisible" to the VM layer, and has to use other
> tricks to make the VM ignore its pages.
<snip>
> Quite frankly, I don't rightly know what the real fix
> is. Making "page->buffers" be a generic thing (a "void *")
> along with making the buffer flushing logic be behind a address
> space operation is probably the right thing in the long run.
That only takes care of writebacks. Don't forget that reading &
readahead can also eat memory if somebody forgets to call lock_page()
(a common problem on 'hard,intr' mounts).
You'll notice that in the NFS updates I sent you the other day, there
is a new function 'nfs_try_to_free_pages()' that provides a rather
generic way of freeing up NFS memory resources. Its sole purpose today
is to ensure that we keep an upper limit of 256 requests per mount.
My (still somewhat vague) plan is to expand that interface some time
during 2.5.x to allow the VM to control and limit the memory usage of
the NFS client - flushing out read and write requests if necessary.
IMHO, the filesystem can often be more efficient at clearing out pages
if we leave the choice of strategy up to it, rather than having the VM
micro-manage exactly which page is to be thrown out first.
For instance, under NFSv3 there is usually a huge advantage to sending
off a COMMIT over any other call, since it can potentially free up a
whole truckload of pages.
Cheers,
Trond
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2001-11-12 3:42 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2001-11-11 10:48 Writing over NFS causes lots of paging Simon Kirby
2001-11-11 18:29 ` Linus Torvalds
2001-11-12 3:36 ` Trond Myklebust
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).