VFS: Busy inodes after unmount on 2 way SMP

VFS: Busy inodes after unmount on 2 way SMP

[Arun Sharma]

We've seen a few "VFS: Busy inodes after unmount. Self-destruct in 5 seconds.  Have a nice day.." messages on a dual processor NFS client. Here's the use case:

- Due to network load issues, the NFS server becomes unreachable for some time
- The automounter tries to expire the mount
- The unmount finds a couple of busy inodes. Putting some debug printks shows that typically two inodes are busy i.e. have inode->i_count == 1. But they don't have any waiters on inode->i_wait. Further, the inodes that are busy are
/mnt/foo     -> autofs mount point
/mnt/foo/bar -> bar is a symbolic link

It's not clear if this is a NFS issue or a autofs issue, but it's seen often with autofs. Are there any known race conditions that have been fixed after 2.4.20 ? The two calls I'm worried about are:

fs/autofs/root.c:305:    d_instantiate(dentry, iget(dir->i_sb,ent->ino));
fs/autofs/root.c:416:    d_instantiate(dentry, iget(dir->i_sb,ino));

    -Arun




-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: VFS: Busy inodes after unmount on 2 way SMP

[H. Peter Anvin]

Arun Sharma wrote:
> 
> We've seen a few "VFS: Busy inodes after unmount. Self-destruct in 5
> seconds.  Have a nice day.." messages on a dual processor NFS client.
> Here's the use case:
> 
> - Due to network load issues, the NFS server becomes unreachable for
> some time
> - The automounter tries to expire the mount
> - The unmount finds a couple of busy inodes. Putting some debug printks
> shows that typically two inodes are busy i.e. have inode->i_count == 1.
> But they don't have any waiters on inode->i_wait. Further, the inodes
> that are busy are
> /mnt/foo     -> autofs mount point
> /mnt/foo/bar -> bar is a symbolic link
> 
> It's not clear if this is a NFS issue or a autofs issue, but it's seen
> often with autofs. Are there any known race conditions that have been
> fixed after 2.4.20 ? The two calls I'm worried about are:
> 
> fs/autofs/root.c:305:    d_instantiate(dentry, iget(dir->i_sb,ent->ino));
> fs/autofs/root.c:416:    d_instantiate(dentry, iget(dir->i_sb,ino));
> 

Which version of autofs are you running?  What version of mount(8)?

It's possible NFS lets you umount a mount which has busy inodes under
certain conditions.  autofs stresses mounting/umounting NFS in ways that
sometimes exposes bugs that otherwise wouldn't be seen.

	-hpa

Re: VFS: Busy inodes after unmount on 2 way SMP

[Ryan Go]

Hello,

We have seen the same message on a box with vanilla 2.4.18 kernel with
rmap12h patch. Mount version is 2.11b and autofs4-4.0.0pre10-10 (SuSe
package). If it matters, we are using only nisplus to get our maps.

I know the above setup is kinda old. We are still having some other issues
moving to a newer release.

	<< ryan

On Thu, 11 Sep 2003, H. Peter Anvin wrote:

> Arun Sharma wrote:
> >
> > We've seen a few "VFS: Busy inodes after unmount. Self-destruct in 5
> > seconds.  Have a nice day.." messages on a dual processor NFS client.
> > Here's the use case:
> >
> > - Due to network load issues, the NFS server becomes unreachable for
> > some time
> > - The automounter tries to expire the mount
> > - The unmount finds a couple of busy inodes. Putting some debug printks
> > shows that typically two inodes are busy i.e. have inode->i_count == 1.
> > But they don't have any waiters on inode->i_wait. Further, the inodes
> > that are busy are
> > /mnt/foo     -> autofs mount point
> > /mnt/foo/bar -> bar is a symbolic link
> >
> > It's not clear if this is a NFS issue or a autofs issue, but it's seen
> > often with autofs. Are there any known race conditions that have been
> > fixed after 2.4.20 ? The two calls I'm worried about are:
> >
> > fs/autofs/root.c:305:    d_instantiate(dentry, iget(dir->i_sb,ent->ino));
> > fs/autofs/root.c:416:    d_instantiate(dentry, iget(dir->i_sb,ino));
> >
>
> Which version of autofs are you running?  What version of mount(8)?
>
> It's possible NFS lets you umount a mount which has busy inodes under
> certain conditions.  autofs stresses mounting/umounting NFS in ways that
> sometimes exposes bugs that otherwise wouldn't be seen.
>
> 	-hpa
>
> _______________________________________________
> autofs mailing list
> autofs@linux.kernel.org
> http://linux.kernel.org/mailman/listinfo/autofs
>

Re: VFS: Busy inodes after unmount on 2 way SMP

[Arun Sharma]

H. Peter Anvin wrote:
> 
> Which version of autofs are you running?  What version of mount(8)?
>

autofs-3.1.7 and mount-2.11g
 
> It's possible NFS lets you umount a mount which has busy inodes under
> certain conditions.

Are you thinking of the MNT_FORCE flag ? I have reasons to believe that it won't help. Mainly because if it was a RPC task that marked the inode busy, it'd have put itself on the waitq of the inode. However I found that waitqueue_active(&inode->i_wait) == 0.

	-Arun

Re: VFS: Busy inodes after unmount on 2 way SMP

[H. Peter Anvin]

Arun Sharma wrote:
> H. Peter Anvin wrote:
> 
>>
>> Which version of autofs are you running?  What version of mount(8)?
>>
> 
> autofs-3.1.7 and mount-2.11g
> 
>> It's possible NFS lets you umount a mount which has busy inodes under
>> certain conditions.
> 
> 
> Are you thinking of the MNT_FORCE flag ? I have reasons to believe that
> it won't help. Mainly because if it was a RPC task that marked the inode
> busy, it'd have put itself on the waitq of the inode. However I found
> that waitqueue_active(&inode->i_wait) == 0.
> 

No, I'm thinking about the possibility that there is a bug.

autofs depends critically on a filesystem *not* allowing itself to be
unmounted when there are busy inodes; however, this is something that
things other than autofs doesn't really stress a whole lot.

	-hpa

Re: VFS: Busy inodes after unmount on 2 way SMP

[Ion Badulescu]

On Thu, 11 Sep 2003 17:11:40 -0700, Arun Sharma <arun.sharma@intel.com> wrote:

> We've seen a few "VFS: Busy inodes after unmount. Self-destruct in 5 seconds.  Have a nice day.." messages on a dual processor NFS client. Here's the use case:
> 
> - Due to network load issues, the NFS server becomes unreachable for some time
> - The automounter tries to expire the mount
> - The unmount finds a couple of busy inodes. Putting some debug printks shows that typically two inodes are busy i.e. have inode->i_count == 1. But they don't have any waiters on inode->i_wait. Further, the inodes that are busy are
> /mnt/foo     -> autofs mount point
> /mnt/foo/bar -> bar is a symbolic link
> 
> It's not clear if this is a NFS issue or a autofs issue, but it's seen often with autofs. Are there any known race conditions that have been fixed after 2.4.20 ? The two calls I'm worried about are:
> 
> fs/autofs/root.c:305:    d_instantiate(dentry, iget(dir->i_sb,ent->ino));
> fs/autofs/root.c:416:    d_instantiate(dentry, iget(dir->i_sb,ino));

I don't know if it's the same issue, but I've seen it many many times 
while testing amd's autofs functionality. It occurs when a non-empty 
autofs filesystem is unmounted, e.g. if it has some leftover 
subdirectories from previous mounts. Therefore I'd say it's quite 
clearly an autofs issue, at least in my case.

Do you see it when shutting down the automounter, or in other cases as 
well?

Ion

-- 
  It is better to keep your mouth shut and be thought a fool,
            than to open it and remove all doubt.


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: VFS: Busy inodes after unmount on 2 way SMP

[Arun Sharma]

Ion Badulescu wrote:
> 
> I don't know if it's the same issue, but I've seen it many many times 
> while testing amd's autofs functionality. It occurs when a non-empty 
> autofs filesystem is unmounted, e.g. if it has some leftover 
> subdirectories from previous mounts. Therefore I'd say it's quite 
> clearly an autofs issue, at least in my case.

Do you have a case that can deterministically reproduce the issue ? 

> Do you see it when shutting down the automounter, or in other cases as 
> well?

Everytime I've seen it, it's during an unmount by /usr/sbin/automount after the mount has expired. It's usually coupled with a temporarily unreachable NFS server.

	-Arun



-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: VFS: Busy inodes after unmount on 2 way SMP

[Ion Badulescu]

On Fri, 12 Sep 2003, Arun Sharma wrote:

> Ion Badulescu wrote:
> > 
> > I don't know if it's the same issue, but I've seen it many many times 
> > while testing amd's autofs functionality. It occurs when a non-empty 
> > autofs filesystem is unmounted, e.g. if it has some leftover 
> > subdirectories from previous mounts. Therefore I'd say it's quite 
> > clearly an autofs issue, at least in my case.
> 
> Do you have a case that can deterministically reproduce the issue ? 

Oh yeah, easily. Kill the automounter while some things are mounted,
unmount everything by hand but do not rmdir the mountpoints. When you
unmount the autofs filesystem itself, you will get the self destruct
message.

> Everytime I've seen it, it's during an unmount by /usr/sbin/automount
> after the mount has expired. It's usually coupled with a temporarily
> unreachable NFS server.

So maybe this is a different issue than the one I am describing...

Ion

-- 
  It is better to keep your mouth shut and be thought a fool,
            than to open it and remove all doubt.



-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: VFS: Busy inodes after unmount on 2 way SMP

[Ion Badulescu]

On Fri, 12 Sep 2003, Arun Sharma wrote:

> Ion Badulescu wrote:
> > 
> > I don't know if it's the same issue, but I've seen it many many times 
> > while testing amd's autofs functionality. It occurs when a non-empty 
> > autofs filesystem is unmounted, e.g. if it has some leftover 
> > subdirectories from previous mounts. Therefore I'd say it's quite 
> > clearly an autofs issue, at least in my case.
> 
> Do you have a case that can deterministically reproduce the issue ? 

Oh yeah, easily. Kill the automounter while some things are mounted,
unmount everything by hand but do not rmdir the mountpoints. When you
unmount the autofs filesystem itself, you will get the self destruct
message.

> Everytime I've seen it, it's during an unmount by /usr/sbin/automount
> after the mount has expired. It's usually coupled with a temporarily
> unreachable NFS server.

So maybe this is a different issue than the one I am describing...

Ion

-- 
  It is better to keep your mouth shut and be thought a fool,
            than to open it and remove all doubt.



-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [NFS] VFS: Busy inodes after unmount on 2 way SMP

[Arun Sharma]

Ion Badulescu wrote:
> On Fri, 12 Sep 2003, Arun Sharma wrote:
> 
> 
>>Ion Badulescu wrote:
>>
>>>I don't know if it's the same issue, but I've seen it many many times 
>>>while testing amd's autofs functionality. It occurs when a non-empty 
>>>autofs filesystem is unmounted, e.g. if it has some leftover 
>>>subdirectories from previous mounts. Therefore I'd say it's quite 
>>>clearly an autofs issue, at least in my case.
>>
>>Do you have a case that can deterministically reproduce the issue ? 
> 
> 
> Oh yeah, easily. Kill the automounter while some things are mounted,
> unmount everything by hand but do not rmdir the mountpoints. When you
> unmount the autofs filesystem itself, you will get the self destruct
> message.
> 

Thanks. This reproduces the problem I'm chasing very well. I think this is clearly an unacceptable kernel bug.

	-Arun

Sep 15 13:58:57 arun-desktop kernel: VFS: Busy inodes after unmount. Self-destruct in 5 seconds.  Have a nice day...

<snip>

Sep 15 14:00:00 arun-desktop kernel: Unable to handle kernel NULL pointer dereference at virtual address 00000004
Sep 15 14:00:00 arun-desktop kernel:  printing eip:
Sep 15 14:00:00 arun-desktop kernel: c015c738
Sep 15 14:00:00 arun-desktop kernel: *pde = 00000000
Sep 15 14:00:00 arun-desktop kernel: Oops: 0000
Sep 15 14:00:00 arun-desktop kernel: ide-cd cdrom emu10k1 ac97_codec sound soundcore nfs lockd sunrpc parport_pc lp parport autofs iptable_filter ip_tables
e100 keybdev mousedev hid input usb-uhc
Sep 15 14:00:00 arun-desktop kernel: CPU:    0
Sep 15 14:00:00 arun-desktop kernel: EIP:    0060:[<c015c738>]    Not tainted
Sep 15 14:00:00 arun-desktop kernel: EFLAGS: 00010246
Sep 15 14:00:00 arun-desktop kernel:
Sep 15 14:00:00 arun-desktop kernel: EIP is at destroy_inode [kernel] 0x28 (2.4.20-8)
Sep 15 14:00:00 arun-desktop kernel: eax: 00000000   ebx: cee44c80   ecx: 00000000   edx: cee44c80
Sep 15 14:00:00 arun-desktop kernel: esi: cb32ee00   edi: cee44c80   ebp: 000003c6   esp: cff87f84
Sep 15 14:00:01 arun-desktop kernel: ds: 0068   es: 0068   ss: 0068
Sep 15 14:00:01 arun-desktop kernel: Process kswapd (pid: 5, stackpage=cff87000)
Sep 15 14:00:01 arun-desktop kernel: Stack: cee44c80 cb32ee00 cb32ee18 c015b563 cee44c80 cb0ff180 cff86000 00000000
Sep 15 14:00:01 arun-desktop kernel:        000001d0 00000000 c015b904 0000057c c013cd5b 00000006 000001d0 cff86000
Sep 15 14:00:01 arun-desktop kernel:        cff86000 00000000 cff86305 00000000 c013d023 000001d0 00000002 000001d0
Sep 15 14:00:01 arun-desktop kernel: Call Trace:   [<c015b563>] prune_dcache [kernel] 0xc3 (0xcff87f90))
Sep 15 14:00:01 arun-desktop kernel: [<c015b904>] shrink_dcache_memory [kernel] 0x24 (0xcff87fac))
Sep 15 14:00:01 arun-desktop kernel: [<c013cd5b>] do_try_to_free_pages_kswapd [kernel] 0x1b (0xcff87fb4))
Sep 15 14:00:01 arun-desktop kernel: [<c013d023>] kswapd [kernel] 0x83 (0xcff87fd4))
Sep 15 14:00:01 arun-desktop kernel: [<c013cfa0>] kswapd [kernel] 0x0 (0xcff87fe4))
Sep 15 14:00:01 arun-desktop kernel: [<c010742d>] kernel_thread_helper [kernel] 0x5 (0xcff87ff0))

Re: Re: [NFS] VFS: Busy inodes after unmount on 2 way SMP

[H. Peter Anvin]

Arun Sharma wrote:
>>
>> Oh yeah, easily. Kill the automounter while some things are mounted,
>> unmount everything by hand but do not rmdir the mountpoints. When you
>> unmount the autofs filesystem itself, you will get the self destruct
>> message.
> 
> Thanks. This reproduces the problem I'm chasing very well. I think this
> is clearly an unacceptable kernel bug.
> 
Indeed.  This is on autofs 3?

	-hpa

Re: Re: [NFS] VFS: Busy inodes after unmount on 2 way SMP

[Arun Sharma]

H. Peter Anvin wrote:
> 
> Indeed.  This is on autofs 3?

Yes.

	-Arun

Re: Re: [NFS] VFS: Busy inodes after unmount on 2 way SMP

[Ion Badulescu]

On Mon, 15 Sep 2003, H. Peter Anvin wrote:

> Arun Sharma wrote:
> >>
> >> Oh yeah, easily. Kill the automounter while some things are mounted,
> >> unmount everything by hand but do not rmdir the mountpoints. When you
> >> unmount the autofs filesystem itself, you will get the self destruct
> >> message.
> > 
> > Thanks. This reproduces the problem I'm chasing very well. I think this
> > is clearly an unacceptable kernel bug.
> > 
> Indeed.  This is on autofs 3?

Both 3 and 4, at least as far as the "self-destruct" message goes.

Ion

-- 
  It is better to keep your mouth shut and be thought a fool,
            than to open it and remove all doubt.

Re: [autofs] Re: VFS: Busy inodes after unmount on 2 way SMP

[Matt C]

Don't know if it helps, but we also run a few hundred autofs4 clients, 
with a mix of 2.4.21 and 2.4.18. The 2.4.21 machines are seeing quite a 
few of these messages, but not on the 2.4.18 machines. We haven't seen any 
of the oopses described below, though.

-matt

On Mon, 15 Sep 2003, Ion Badulescu wrote:

> On Mon, 15 Sep 2003, H. Peter Anvin wrote:
> 
> > Arun Sharma wrote:
> > >>
> > >> Oh yeah, easily. Kill the automounter while some things are mounted,
> > >> unmount everything by hand but do not rmdir the mountpoints. When you
> > >> unmount the autofs filesystem itself, you will get the self destruct
> > >> message.
> > > 
> > > Thanks. This reproduces the problem I'm chasing very well. I think this
> > > is clearly an unacceptable kernel bug.
> > > 
> > Indeed.  This is on autofs 3?
> 
> Both 3 and 4, at least as far as the "self-destruct" message goes.
> 
> Ion
> 
> 



-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [autofs] Re: VFS: Busy inodes after unmount on 2 way SMP

[Matt C]

Don't know if it helps, but we also run a few hundred autofs4 clients, 
with a mix of 2.4.21 and 2.4.18. The 2.4.21 machines are seeing quite a 
few of these messages, but not on the 2.4.18 machines. We haven't seen any 
of the oopses described below, though.

-matt

On Mon, 15 Sep 2003, Ion Badulescu wrote:

> On Mon, 15 Sep 2003, H. Peter Anvin wrote:
> 
> > Arun Sharma wrote:
> > >>
> > >> Oh yeah, easily. Kill the automounter while some things are mounted,
> > >> unmount everything by hand but do not rmdir the mountpoints. When you
> > >> unmount the autofs filesystem itself, you will get the self destruct
> > >> message.
> > > 
> > > Thanks. This reproduces the problem I'm chasing very well. I think this
> > > is clearly an unacceptable kernel bug.
> > > 
> > Indeed.  This is on autofs 3?
> 
> Both 3 and 4, at least as far as the "self-destruct" message goes.
> 
> Ion
> 
> 



-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: Re: [NFS] VFS: Busy inodes after unmount on 2 way SMP

[Arun Sharma]

H. Peter Anvin wrote:
> 
> Indeed.  This is on autofs 3?
> 

Correction: the problem was primarily seen on autofs4. However the oops that I reported yesterday was autofs3. So the problem affects both.

	-Arun

Re: Re: [NFS] VFS: Busy inodes after unmount on 2 way SMP

[H. Peter Anvin]

Arun Sharma wrote:
> H. Peter Anvin wrote:
> 
>>
>> Indeed.  This is on autofs 3?
>>
> 
> Correction: the problem was primarily seen on autofs4. However the oops
> that I reported yesterday was autofs3. So the problem affects both.
> 
>     -Arun

Thanks.  That's useful information.

	-hpa

Re: VFS: Busy inodes after unmount on 2 way SMP

[Ian Kent]

On Thu, 11 Sep 2003, Arun Sharma wrote:

>
> We've seen a few "VFS: Busy inodes after unmount. Self-destruct in 5 seconds.  Have a nice day.." messages on a dual processor NFS client. Here's the use case:
>
> - Due to network load issues, the NFS server becomes unreachable for some time
> - The automounter tries to expire the mount
> - The unmount finds a couple of busy inodes. Putting some debug printks shows that typically two inodes are busy i.e. have inode->i_count == 1. But they don't have any waiters on inode->i_wait. Further, the inodes that are busy are
> /mnt/foo     -> autofs mount point
> /mnt/foo/bar -> bar is a symbolic link
>
> It's not clear if this is a NFS issue or a autofs issue, but it's seen often with autofs. Are there any known race conditions that have been fixed after 2.4.20 ? The two calls I'm worried about are:
>
> fs/autofs/root.c:305:    d_instantiate(dentry, iget(dir->i_sb,ent->ino));
> fs/autofs/root.c:416:    d_instantiate(dentry, iget(dir->i_sb,ino));
>

This is in the autofs (aka v3) kernel module not autofs4.

Please specify which version of the userspace daemon and kernel module?

-- 

   ,-._|\    Ian Kent
  /      \   Perth, Western Australia
  *_.--._/   E-mail: raven@themaw.net
        v    Web: http://themaw.net/

Re: VFS: Busy inodes after unmount on 2 way SMP

[Arun Sharma]

Ian Kent wrote:

>>It's not clear if this is a NFS issue or a autofs issue, but it's seen often with autofs. Are there any known race conditions that have been fixed after 2.4.20 ? The two calls I'm worried about are:
>>
>>fs/autofs/root.c:305:    d_instantiate(dentry, iget(dir->i_sb,ent->ino));
>>fs/autofs/root.c:416:    d_instantiate(dentry, iget(dir->i_sb,ino));
>>
> 
> 
> This is in the autofs (aka v3) kernel module not autofs4.
>

Please ignore the above lines. I corrected myself in an earlier message. The problem was seen on the autofs4 kernel module in 2.4.20 + autofs-3.1.7 user daemon.

	-Arun

Re: VFS: Busy inodes after unmount on 2 way SMP

[H. Peter Anvin]

Arun Sharma wrote:
> Ian Kent wrote:
> 
>>> It's not clear if this is a NFS issue or a autofs issue, but it's 
>>> seen often with autofs. Are there any known race conditions that have 
>>> been fixed after 2.4.20 ? The two calls I'm worried about are:
>>>
>>> fs/autofs/root.c:305:    d_instantiate(dentry, 
>>> iget(dir->i_sb,ent->ino));
>>> fs/autofs/root.c:416:    d_instantiate(dentry, iget(dir->i_sb,ino));
>>>
>> This is in the autofs (aka v3) kernel module not autofs4.
>>
> Please ignore the above lines. I corrected myself in an earlier message. 
> The problem was seen on the autofs4 kernel module in 2.4.20 + 
> autofs-3.1.7 user daemon.
> 

Please don't mix the v3 and v4 stuff (this was a misguided change on 
Linus' part which I still very much disagree with.)

	-hpa

Re: VFS: Busy inodes after unmount on 2 way SMP

[Ian Kent]

On Tue, 16 Sep 2003, Arun Sharma wrote:

>
> Please ignore the above lines. I corrected myself in an earlier message. The problem was seen on the autofs4 kernel module in 2.4.20 + autofs-3.1.7 user daemon.
>

Vanilla 2.4.20?

-- 

   ,-._|\    Ian Kent
  /      \   Perth, Western Australia
  *_.--._/   E-mail: raven@themaw.net
        v    Web: http://themaw.net/

Re: VFS: Busy inodes after unmount on 2 way SMP

[Ian Kent]

On Tue, 16 Sep 2003, H. Peter Anvin wrote:

>
> Please don't mix the v3 and v4 stuff (this was a misguided change on
> Linus' part which I still very much disagree with.)
>

Yes.

It should work but if it doesn't it becomes very difficult to support.

-- 

   ,-._|\    Ian Kent
  /      \   Perth, Western Australia
  *_.--._/   E-mail: raven@themaw.net
        v    Web: http://themaw.net/

Re: VFS: Busy inodes after unmount on 2 way SMP

[Ryan Go]

Hello Ian,

We have also seen the self-destruct message on boxes with autofs4 in
kernel 2.4.18 (w/ rmap12h patch) and autofs4-4.0.0pre10-10 userspace
daemon.

Thanks,
	<< ryan

On Wed, 17 Sep 2003, Ian Kent wrote:

> On Thu, 11 Sep 2003, Arun Sharma wrote:
>
> >
> > We've seen a few "VFS: Busy inodes after unmount. Self-destruct in 5 seconds.  Have a nice day.." messages on a dual processor NFS client. Here's the use case:
> >
> > - Due to network load issues, the NFS server becomes unreachable for some time
> > - The automounter tries to expire the mount
> > - The unmount finds a couple of busy inodes. Putting some debug printks shows that typically two inodes are busy i.e. have inode->i_count == 1. But they don't have any waiters on inode->i_wait. Further, the inodes that are busy are
> > /mnt/foo     -> autofs mount point
> > /mnt/foo/bar -> bar is a symbolic link
> >
> > It's not clear if this is a NFS issue or a autofs issue, but it's seen often with autofs. Are there any known race conditions that have been fixed after 2.4.20 ? The two calls I'm worried about are:
> >
> > fs/autofs/root.c:305:    d_instantiate(dentry, iget(dir->i_sb,ent->ino));
> > fs/autofs/root.c:416:    d_instantiate(dentry, iget(dir->i_sb,ino));
> >
>
> This is in the autofs (aka v3) kernel module not autofs4.
>
> Please specify which version of the userspace daemon and kernel module?
>
> --
>
>    ,-._|\    Ian Kent
>   /      \   Perth, Western Australia
>   *_.--._/   E-mail: raven@themaw.net
>         v    Web: http://themaw.net/
>
> _______________________________________________
> autofs mailing list
> autofs@linux.kernel.org
> http://linux.kernel.org/mailman/listinfo/autofs
>

Re: VFS: Busy inodes after unmount on 2 way SMP

[Arun Sharma]

Ian Kent wrote:
> On Tue, 16 Sep 2003, Arun Sharma wrote:
> 
> 
>> Please ignore the above lines. I corrected myself in an earlier
>> message. The problem was seen on the autofs4 kernel module in
>> 2.4.20 + autofs-3.1.7 user daemon.
> 
> 
> Vanilla 2.4.20?
> 

Yes, vanilla 2.4.20.

	-Arun

Re: VFS: Busy inodes after unmount on 2 way SMP

[H. Peter Anvin]

Okay... let me ask this so I get it straight...

Has anyone seen this (busy inodes after stopping the automounter) using 
autofs v3 kernel module and daemon?

	-hpa

Re: VFS: Busy inodes after unmount on 2 way SMP

[Olaf Hering]

 On Wed, Sep 17, H. Peter Anvin wrote:

> Okay... let me ask this so I get it straight...
> 
> Has anyone seen this (busy inodes after stopping the automounter) using 
> autofs v3 kernel module and daemon?

I'm not sure, but everyone who has seen this should fiddle this patch
into the kernel and see how it goes.
The whole thing started very recently (post 2.4.21) for us.

This patch is untested, any feedback appreciated.

        This patch will hopefully fix the "VFS: Busy inodes after unmount.
        Self-destruct in 5 seconds" oopses generated by autofs+nfs.

diff -u -r linux-2.4.21/fs/nfs/inode.c smp/fs/nfs/inode.c
--- linux-2.4.21/fs/nfs/inode.c 2003-09-17 17:51:53.000000000 +0200
+++ smp/fs/nfs/inode.c  2003-09-17 21:24:57.000000000 +0200
@@ -976,9 +976,19 @@
 
 int nfs_release(struct inode *inode, struct file *filp)
 {
+       struct dentry *dentry;
        struct rpc_cred *cred;
 
        lock_kernel();
+
+       /* Do the silly unlink here as long as we still have a reference   
+        * on the vfsmount. Otherwise the unlink cache may hold a reference
+        * to the parent dentry across umount() calls. */
+       if ((dentry = filp->f_dentry) && (dentry->d_flags & DCACHE_NFSFS_RENAMED)) {
+               /* Just trigger the NFS call */
+               nfs_complete_unlink(dentry);
+       }
+
        cred = nfs_file_cred(filp);
        if (cred)
                put_rpccred(cred);



-- 
USB is for mice, FireWire is for men!

sUse lINUX ag, n√úRNBERG

Re: VFS: Busy inodes after unmount on 2 way SMP

[Ian Kent]

On Wed, 17 Sep 2003, Olaf Hering wrote:

>
> This patch is untested, any feedback appreciated.
>
>         This patch will hopefully fix the "VFS: Busy inodes after unmount.
>         Self-destruct in 5 seconds" oopses generated by autofs+nfs.
>

Could you elaborate on reasoning behind the patch a bit please.


-- 

   ,-._|\    Ian Kent
  /      \   Perth, Western Australia
  *_.--._/   E-mail: raven@themaw.net
        v    Web: http://themaw.net/

Re: [NFS] Re: VFS: Busy inodes after unmount on 2 way SMP

[Ion Badulescu]

On Wed, 17 Sep 2003 13:41:15 -0700, H. Peter Anvin <hpa@zytor.com> wrote:
> Okay... let me ask this so I get it straight...
> 
> Has anyone seen this (busy inodes after stopping the automounter) using 
> autofs v3 kernel module and daemon?

Why do you bring the daemon into the picture, it is irrelevant here. 
Sure it probably does things differently which might avoid the error, 
but the kernel issue exists nonetheless.

To answer your question again: yes, the issue is present in both v3 and
v4 kernel modules. You can always replicate it yourself if you don't
believe me, using the little recipe I posted earlier:

# lsmod | grep autofs
autofs                 11236   4  (autoclean)
# cat /proc/mounts
[snip]
boxter:(pid29327) /bar autofs rw 0 0
boxter:(pid29327) /net autofs rw 0 0
boxter:(pid29327) /src autofs rw 0 0
boxter:(pid29327) /misc autofs rw 0 0
buggy:/n/buggy/misc/homes /misc/homes nfs rw,v3,rsize=8192,wsize=8192,hard,intr,tcp,lock,addr=buggy 0 0
# umount /misc/homes
# umount /misc
# dmesg |tail
VFS: Busy inodes after unmount. Self-destruct in 5 seconds.  Have a nice day...

Ion

-- 
  It is better to keep your mouth shut and be thought a fool,
            than to open it and remove all doubt.

Re: [NFS] Re: VFS: Busy inodes after unmount on 2 way SMP

[H. Peter Anvin]

Ion Badulescu wrote:
> On Wed, 17 Sep 2003 13:41:15 -0700, H. Peter Anvin <hpa@zytor.com> wrote:
> 
>>Okay... let me ask this so I get it straight...
>>
>>Has anyone seen this (busy inodes after stopping the automounter) using 
>>autofs v3 kernel module and daemon?
> 
> Why do you bring the daemon into the picture, it is irrelevant here. 

Because it makes it a lot harder to debug the problem.

> Sure it probably does things differently which might avoid the error, 
> but the kernel issue exists nonetheless.
> 
> To answer your question again: yes, the issue is present in both v3 and
> v4 kernel modules. You can always replicate it yourself if you don't
> believe me, using the little recipe I posted earlier:
> 
> # lsmod | grep autofs
> autofs                 11236   4  (autoclean)
> # cat /proc/mounts
> [snip]
> boxter:(pid29327) /bar autofs rw 0 0
> boxter:(pid29327) /net autofs rw 0 0
> boxter:(pid29327) /src autofs rw 0 0
> boxter:(pid29327) /misc autofs rw 0 0
> buggy:/n/buggy/misc/homes /misc/homes nfs rw,v3,rsize=8192,wsize=8192,hard,intr,tcp,lock,addr=buggy 0 0
> # umount /misc/homes
> # umount /misc
> # dmesg |tail
> VFS: Busy inodes after unmount. Self-destruct in 5 seconds.  Have a nice day...

OK.  Did you try Olaf's NFS patch?

	-hpa

Re: VFS: Busy inodes after unmount on 2 way SMP

[Olaf Hering]

 On Thu, Sep 18, Ian Kent wrote:

> On Wed, 17 Sep 2003, Olaf Hering wrote:
> 
> >
> > This patch is untested, any feedback appreciated.
> >
> >         This patch will hopefully fix the "VFS: Busy inodes after unmount.
> >         Self-destruct in 5 seconds" oopses generated by autofs+nfs.
> >
> 
> Could you elaborate on reasoning behind the patch a bit please.

probably that:

...
Additional Comment #10 From Olaf Kirch 2003-09-16 21:13                                                 
The way I understood it the code is supposed to work like                                                                            
                                                                                                                                                  
 umount(8)      calls umount(2) with MNT_FORCE                                              
 umount(2)      calls NFS to signal all RPC tasks                                                                             
                later notices there are still buys inodes and returns to                                                                              
                user space (-EBUSY)                                                                                                                 
 umount(8)      retries umount(2) call                                                                                                                
                                                                                                                     
If that assumption is incorrect, we indeed to wait for the rpc tasks to                                                                  
complete. Nasty :-(                                                                                                                            
...

-- 
USB is for mice, FireWire is for men!

sUse lINUX ag, n√úRNBERG

Re: Re: [autofs] VFS: Busy inodes after unmount on 2 way SMP

[Trond Myklebust]

>>>>> " " == Olaf Hering <olh@suse.de> writes:

     >  On Wed, Sep 17, H. Peter Anvin wrote:
    >> Okay... let me ask this so I get it straight...
    >>
    >> Has anyone seen this (busy inodes after stopping the
    >> automounter) using autofs v3 kernel module and daemon?

     > I'm not sure, but everyone who has seen this should fiddle this
     > patch into the kernel and see how it goes.  The whole thing
     > started very recently (post 2.4.21) for us.

     > This patch is untested, any feedback appreciated.

...and is indeed wrong... It does the exact opposite of what
sillydelete should do. Instead of causing the last application that
closes the file to perform the sillydelete, you are asking the *first*
application that closes it to do so.
Sillydelete *has* to be tied to dentries. Not files, and not
inodes. It is purely a namespace operation...

So exactly what are you trying to do, and why?

Cheers,
  Trond


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: Re: [autofs] VFS: Busy inodes after unmount on 2 way SMP

[Olaf Kirch]

On Thu, Sep 18, 2003 at 01:52:05AM -0400, Trond Myklebust wrote:
> ...and is indeed wrong... It does the exact opposite of what
> sillydelete should do. Instead of causing the last application that
> closes the file to perform the sillydelete, you are asking the *first*
> application that closes it to do so.

I know. I was not claiming it's perfectly right, what I'm interested
in whether it cures the oopses we're seeing. If that were true, one
could still look for The Right Solution...

> Sillydelete *has* to be tied to dentries. Not files, and not
> inodes. It is purely a namespace operation...
> 
> So exactly what are you trying to do, and why?

We have seen several reports of autofs+nfs leading to oopses, preceded by
"Busy inodes on umount, self-destruct in 5 seconds".
It's somewhat hard to reproduce, so I'm currently trying to come up with
possible patches by looking at the source.

You cannot umount if the vfsmount mnt_count is != 2.  So either something
hosed the mnt_count completely (by doing too many mntput() calls for
instance), or something holds dentry references without bumping mnt_count
along with it. Do you agree?

The only place in the nfs client code that actually does a dget is
the sillydelete stuff in unlink.c. So my idea in generating this patch
was that the nfs_complete_unlink was happening too late, when the file
is already closed and the vfsmount reference is gone.

It's a long shot admittedly...

Have a nice day,
Olaf
-- 
Olaf Kirch     |  Anyone who has had to work with X.509 has probably
okir@suse.de   |  experienced what can best be described as
---------------+  ISO water torture. -- Peter Gutmann


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [autofs] Re: VFS: Busy inodes after unmount on 2 way SMP

[Frank Cusack]

On Mon, Sep 15, 2003 at 06:34:33PM -0700, Matt C wrote:
> On Mon, 15 Sep 2003, Ion Badulescu wrote:
> > On Mon, 15 Sep 2003, H. Peter Anvin wrote:
> > > Arun Sharma wrote:
> > > >>
> > > >> Oh yeah, easily. Kill the automounter while some things are mounted,
> > > >> unmount everything by hand but do not rmdir the mountpoints. When you
> > > >> unmount the autofs filesystem itself, you will get the self destruct
> > > >> message.
> > > > 
> > > > Thanks. This reproduces the problem I'm chasing very well. I think this
> > > > is clearly an unacceptable kernel bug.
> > > > 
> > > Indeed.  This is on autofs 3?
> > 
> > Both 3 and 4, at least as far as the "self-destruct" message goes.
>
> Don't know if it helps, but we also run a few hundred autofs4 clients, 
> with a mix of 2.4.21 and 2.4.18. The 2.4.21 machines are seeing quite a 
> few of these messages, but not on the 2.4.18 machines. We haven't seen any 
> of the oopses described below, though.

Same here, 2.4.22, autofs 3 and 4.  I'll be playing with 2.6.0 next week.

I haven't tried to reproduce it with your method, but I do see the bug;
no oops though.

/fc


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [NFS] Re: VFS: Busy inodes after unmount on 2 way SMP

[Olaf Hering]

 On Thu, Sep 18, Trond Myklebust wrote:

> >>>>> " " == Olaf Hering <olh@suse.de> writes:
> 
>      >  On Wed, Sep 17, H. Peter Anvin wrote:
>     >> Okay... let me ask this so I get it straight...
>     >>
>     >> Has anyone seen this (busy inodes after stopping the
>     >> automounter) using autofs v3 kernel module and daemon?
> 
>      > I'm not sure, but everyone who has seen this should fiddle this
>      > patch into the kernel and see how it goes.  The whole thing
>      > started very recently (post 2.4.21) for us.

After thinking more about the conditions, I got a crash with 2.4.19
client. the codebase was unchanged for a long time. But the server side
was updated to 2.4.21 with lots of nfs patches. Possible that the newer
server side causes trouble for the client.
-- 
USB is for mice, FireWire is for men!

sUse lINUX ag, n√úRNBERG

Re: Re: [autofs] VFS: Busy inodes after unmount on 2 way SMP

[Matt C]

Hi All-

We've been having the 'VFS: Busy inodes after unmount' problems like crazy 
on our desktop and server linux boxes running autofs under 2.4.21 
(vanilla). We're using the autofs-3.1.7 userspace tools, but with the 
autofs4 kernel module.

I applied the patch that was attached earlier in this thread to our kernel 
and have been running it for the past few days on my desktop. These errors 
have completely disappeared, and autofs is behaving as expected so far.

So, while it might be the wrong thing to do, it does fix our problems.

-matt

On 18 Sep 2003, Trond Myklebust wrote:

> >>>>> " " == Olaf Hering <olh@suse.de> writes:
> 
>      >  On Wed, Sep 17, H. Peter Anvin wrote:
>     >> Okay... let me ask this so I get it straight...
>     >>
>     >> Has anyone seen this (busy inodes after stopping the
>     >> automounter) using autofs v3 kernel module and daemon?
> 
>      > I'm not sure, but everyone who has seen this should fiddle this
>      > patch into the kernel and see how it goes.  The whole thing
>      > started very recently (post 2.4.21) for us.
> 
>      > This patch is untested, any feedback appreciated.
> 
> ...and is indeed wrong... It does the exact opposite of what
> sillydelete should do. Instead of causing the last application that
> closes the file to perform the sillydelete, you are asking the *first*
> application that closes it to do so.
> Sillydelete *has* to be tied to dentries. Not files, and not
> inodes. It is purely a namespace operation...
> 
> So exactly what are you trying to do, and why?
> 
> Cheers,
>   Trond
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> NFS maillist  -  NFS@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs
> 



-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: Re: [autofs] VFS: Busy inodes after unmount on 2 way SMP

[Matt C]

Hi All-

We've been having the 'VFS: Busy inodes after unmount' problems like crazy 
on our desktop and server linux boxes running autofs under 2.4.21 
(vanilla). We're using the autofs-3.1.7 userspace tools, but with the 
autofs4 kernel module.

I applied the patch that was attached earlier in this thread to our kernel 
and have been running it for the past few days on my desktop. These errors 
have completely disappeared, and autofs is behaving as expected so far.

So, while it might be the wrong thing to do, it does fix our problems.

-matt

On 18 Sep 2003, Trond Myklebust wrote:

> >>>>> " " == Olaf Hering <olh@suse.de> writes:
> 
>      >  On Wed, Sep 17, H. Peter Anvin wrote:
>     >> Okay... let me ask this so I get it straight...
>     >>
>     >> Has anyone seen this (busy inodes after stopping the
>     >> automounter) using autofs v3 kernel module and daemon?
> 
>      > I'm not sure, but everyone who has seen this should fiddle this
>      > patch into the kernel and see how it goes.  The whole thing
>      > started very recently (post 2.4.21) for us.
> 
>      > This patch is untested, any feedback appreciated.
> 
> ...and is indeed wrong... It does the exact opposite of what
> sillydelete should do. Instead of causing the last application that
> closes the file to perform the sillydelete, you are asking the *first*
> application that closes it to do so.
> Sillydelete *has* to be tied to dentries. Not files, and not
> inodes. It is purely a namespace operation...
> 
> So exactly what are you trying to do, and why?
> 
> Cheers,
>   Trond
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> NFS maillist  -  NFS@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs
> 



-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: Re: [autofs] VFS: Busy inodes after unmount on 2 way SMP

[Trond Myklebust]

>>>>> " " == Matt C <Matt> writes:

     > Hi All- We've been having the 'VFS: Busy inodes after unmount'
     > problems like crazy on our desktop and server linux boxes
     > running autofs under 2.4.21 (vanilla). We're using the
     > autofs-3.1.7 userspace tools, but with the autofs4 kernel
     > module.

     > I applied the patch that was attached earlier in this thread to
     > our kernel and have been running it for the past few days on my
     > desktop. These errors have completely disappeared, and autofs
     > is behaving as expected so far.

     > So, while it might be the wrong thing to do, it does fix our
     > problems.

That changes nothing. It is still not a "fix" that I want to see go
into the main kernel.

The problem here is that sillyrename needs to hold onto the parent
directory's inode until it is done, and yet it cannot know which
vfsmount struct to hold onto because that information is not passed
down by the VFS.

I would rather then have something like the appended patch go in to
which force the last process to close the file to actually wait on
completion (unless someone signals it with a 'kill -9' - in which case
they are knowingly breaking things).

Cheers,
  Trond

diff -u --recursive --new-file linux-2.4.22-03-soft2/fs/nfs/dir.c linux-2.4.22-04-fix_unlink/fs/nfs/dir.c
--- linux-2.4.22-03-soft2/fs/nfs/dir.c	2002-10-15 06:59:27.000000000 +0200
+++ linux-2.4.22-04-fix_unlink/fs/nfs/dir.c	2003-07-03 12:29:28.000000000 +0200
@@ -994,7 +994,7 @@
 	struct inode *old_inode = old_dentry->d_inode;
 	struct inode *new_inode = new_dentry->d_inode;
 	struct dentry *dentry = NULL, *rehash = NULL;
-	int error = -EBUSY;
+	int error;
 
 	/*
 	 * To prevent any new references to the target during the rename,
@@ -1020,6 +1020,12 @@
 	 */
 	if (!new_inode)
 		goto go_ahead;
+	/* If target is a hard link to the source, then noop */
+	error = 0;
+	if (NFS_FILEID(new_inode) == NFS_FILEID(old_inode))
+		goto out;
+
+	error = -EBUSY;
 	if (S_ISDIR(new_inode->i_mode))
 		goto out;
 	else if (atomic_read(&new_dentry->d_count) > 1) {
diff -u --recursive --new-file linux-2.4.22-03-soft2/fs/nfs/unlink.c linux-2.4.22-04-fix_unlink/fs/nfs/unlink.c
--- linux-2.4.22-03-soft2/fs/nfs/unlink.c	2002-08-11 13:34:02.000000000 +0200
+++ linux-2.4.22-04-fix_unlink/fs/nfs/unlink.c	2003-07-03 12:29:28.000000000 +0200
@@ -12,6 +12,7 @@
 #include <linux/sunrpc/sched.h>
 #include <linux/sunrpc/clnt.h>
 #include <linux/nfs_fs.h>
+#include <linux/wait.h>
 
 
 struct nfs_unlinkdata {
@@ -21,6 +22,9 @@
 	struct rpc_task	task;
 	struct rpc_cred	*cred;
 	unsigned int	count;
+
+	wait_queue_head_t waitq;
+	int completed;
 };
 
 static struct nfs_unlinkdata	*nfs_deletes;
@@ -133,6 +137,8 @@
 	put_rpccred(data->cred);
 	data->cred = NULL;
 	dput(dir);
+	data->completed = 1;
+	wake_up(&data->waitq);
 }
 
 /**
@@ -175,6 +181,8 @@
 	nfs_deletes = data;
 	data->count = 1;
 
+	init_waitqueue_head(&data->waitq);
+
 	task = &data->task;
 	rpc_init_task(task, clnt, nfs_async_unlink_done , RPC_TASK_ASYNC);
 	task->tk_calldata = data;
@@ -212,7 +220,10 @@
 	data->count++;
 	nfs_copy_dname(dentry, data);
 	dentry->d_flags &= ~DCACHE_NFSFS_RENAMED;
-	if (data->task.tk_rpcwait == &nfs_delete_queue)
+	if (data->task.tk_rpcwait == &nfs_delete_queue) {
+		struct rpc_clnt *clnt = data->task.tk_client;
 		rpc_wake_up_task(&data->task);
+		nfs_wait_event(clnt, data->waitq, data->completed == 1);
+	}
 	nfs_put_unlinkdata(data);
 }


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: Re: [autofs] VFS: Busy inodes after unmount on 2 way SMP

[Trond Myklebust]

>>>>> " " == Matt C <Matt> writes:

     > Hi All- We've been having the 'VFS: Busy inodes after unmount'
     > problems like crazy on our desktop and server linux boxes
     > running autofs under 2.4.21 (vanilla). We're using the
     > autofs-3.1.7 userspace tools, but with the autofs4 kernel
     > module.

     > I applied the patch that was attached earlier in this thread to
     > our kernel and have been running it for the past few days on my
     > desktop. These errors have completely disappeared, and autofs
     > is behaving as expected so far.

     > So, while it might be the wrong thing to do, it does fix our
     > problems.

That changes nothing. It is still not a "fix" that I want to see go
into the main kernel.

The problem here is that sillyrename needs to hold onto the parent
directory's inode until it is done, and yet it cannot know which
vfsmount struct to hold onto because that information is not passed
down by the VFS.

I would rather then have something like the appended patch go in to
which force the last process to close the file to actually wait on
completion (unless someone signals it with a 'kill -9' - in which case
they are knowingly breaking things).

Cheers,
  Trond

diff -u --recursive --new-file linux-2.4.22-03-soft2/fs/nfs/dir.c linux-2.4.22-04-fix_unlink/fs/nfs/dir.c
--- linux-2.4.22-03-soft2/fs/nfs/dir.c	2002-10-15 06:59:27.000000000 +0200
+++ linux-2.4.22-04-fix_unlink/fs/nfs/dir.c	2003-07-03 12:29:28.000000000 +0200
@@ -994,7 +994,7 @@
 	struct inode *old_inode = old_dentry->d_inode;
 	struct inode *new_inode = new_dentry->d_inode;
 	struct dentry *dentry = NULL, *rehash = NULL;
-	int error = -EBUSY;
+	int error;
 
 	/*
 	 * To prevent any new references to the target during the rename,
@@ -1020,6 +1020,12 @@
 	 */
 	if (!new_inode)
 		goto go_ahead;
+	/* If target is a hard link to the source, then noop */
+	error = 0;
+	if (NFS_FILEID(new_inode) == NFS_FILEID(old_inode))
+		goto out;
+
+	error = -EBUSY;
 	if (S_ISDIR(new_inode->i_mode))
 		goto out;
 	else if (atomic_read(&new_dentry->d_count) > 1) {
diff -u --recursive --new-file linux-2.4.22-03-soft2/fs/nfs/unlink.c linux-2.4.22-04-fix_unlink/fs/nfs/unlink.c
--- linux-2.4.22-03-soft2/fs/nfs/unlink.c	2002-08-11 13:34:02.000000000 +0200
+++ linux-2.4.22-04-fix_unlink/fs/nfs/unlink.c	2003-07-03 12:29:28.000000000 +0200
@@ -12,6 +12,7 @@
 #include <linux/sunrpc/sched.h>
 #include <linux/sunrpc/clnt.h>
 #include <linux/nfs_fs.h>
+#include <linux/wait.h>
 
 
 struct nfs_unlinkdata {
@@ -21,6 +22,9 @@
 	struct rpc_task	task;
 	struct rpc_cred	*cred;
 	unsigned int	count;
+
+	wait_queue_head_t waitq;
+	int completed;
 };
 
 static struct nfs_unlinkdata	*nfs_deletes;
@@ -133,6 +137,8 @@
 	put_rpccred(data->cred);
 	data->cred = NULL;
 	dput(dir);
+	data->completed = 1;
+	wake_up(&data->waitq);
 }
 
 /**
@@ -175,6 +181,8 @@
 	nfs_deletes = data;
 	data->count = 1;
 
+	init_waitqueue_head(&data->waitq);
+
 	task = &data->task;
 	rpc_init_task(task, clnt, nfs_async_unlink_done , RPC_TASK_ASYNC);
 	task->tk_calldata = data;
@@ -212,7 +220,10 @@
 	data->count++;
 	nfs_copy_dname(dentry, data);
 	dentry->d_flags &= ~DCACHE_NFSFS_RENAMED;
-	if (data->task.tk_rpcwait == &nfs_delete_queue)
+	if (data->task.tk_rpcwait == &nfs_delete_queue) {
+		struct rpc_clnt *clnt = data->task.tk_client;
 		rpc_wake_up_task(&data->task);
+		nfs_wait_event(clnt, data->waitq, data->completed == 1);
+	}
 	nfs_put_unlinkdata(data);
 }


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [NFS] Re: VFS: Busy inodes after unmount on 2 way SMP

[Ion Badulescu]

On Thu, 25 Sep 2003 17:24:49 -0700, Trond Myklebust <trond.myklebust@fys.uio.no> wrote:

> I would rather then have something like the appended patch go in to
> which force the last process to close the file to actually wait on
> completion (unless someone signals it with a 'kill -9' - in which case
> they are knowingly breaking things).

Ugh, that's ugly -- and potentially a DoS issue as well. Since you don't
need any privileges to issue the 'kill -9' for your own processes, it   
becomes fairly easy to write some code that exploits this and causes 
kernel crashes across the board...

Not that the current situation is any better, of course.

Perhaps making that wait an uninterruptable wait would be enough to take
care of the problem?

Ion

-- 
  It is better to keep your mouth shut and be thought a fool,
            than to open it and remove all doubt.

Re: Re: [autofs] VFS: Busy inodes after unmount on 2 way SMP

[Trond Myklebust]

>>>>> " " == Ion Badulescu <lists@news.cs.columbia.edu> writes:

     > Ugh, that's ugly -- and potentially a DoS issue as well. Since
     > you don't need any privileges to issue the 'kill -9' for your
     > own processes, it becomes fairly easy to write some code that
     > exploits this and causes kernel crashes across the board...

Then don't set the "intr" flag.

     > Not that the current situation is any better, of course.

     > Perhaps making that wait an uninterruptable wait would be
     > enough to take care of the problem?

People expect the "intr" flag to work if they set it.

Cheers,
  Trond


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: Re: [autofs] VFS: Busy inodes after unmount on 2 way SMP

[Olaf Kirch]

On Fri, Sep 26, 2003 at 03:29:40PM -0700, Trond Myklebust wrote:
>      > Not that the current situation is any better, of course.
> 
>      > Perhaps making that wait an uninterruptable wait would be
>      > enough to take care of the problem?
> 
> People expect the "intr" flag to work if they set it.

In this case, not removing the sillyrenamed file instead
of oopsing may be the preferred action.

Olaf
-- 
Olaf Kirch     |  Anyone who has had to work with X.509 has probably
okir@suse.de   |  experienced what can best be described as
---------------+  ISO water torture. -- Peter Gutmann


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: Re: [autofs] VFS: Busy inodes after unmount on 2 way SMP

[Steve Fosdick]

On 2003.09.27 17:55, Olaf Kirch wrote:

> In this case, not removing the sillyrenamed file instead
> of oopsing may be the preferred action.

Would it be possibly to combine both ideas, i.e. have the last process
wait, but if the user attempt to kill that process and it will be
allowed because intr is set the sillyrenamed file is not removed.

Surely it is reasonable for people who kill -9 things to expect to do
some tidying up, but not to expect a kernel panic.

Steve.


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: Re: [autofs] VFS: Busy inodes after unmount on 2 way SMP

[Frank Cusack]

Does this patch help?

--- linux-2.4.22/fs/namei.c	2003-08-25 04:44:43.000000000 -0700
+++ linux-2.4.22-gg4/fs/namei.c	2003-09-26 00:03:16.000000000 -0700
@@ -893,6 +893,8 @@ static inline int check_sticky(struct in
  *  7. If we were asked to remove a directory and victim isn't one - ENOTDIR.
  *  8. If we were asked to remove a non-directory and victim isn't one - EISDIR.
  *  9. We can't remove a root or mountpoint.
+ * 10. We don't allow removal of NFS sillyrenamed files; it's handled by
+ *     nfs_async_unlink().
  */
 static inline int may_delete(struct inode *dir,struct dentry *victim, int isdir)
 {
@@ -916,6 +918,8 @@ static inline int may_delete(struct inod
 		return -EISDIR;
 	if (IS_DEADDIR(dir))
 		return -ENOENT;
+	if (victim->d_flags & DCACHE_NFSFS_RENAMED)
+		return -EBUSY;
 	return 0;
 }
 
@@ -1484,13 +1488,14 @@ int vfs_unlink(struct inode *dir, struct
 				lock_kernel();
 				error = dir->i_op->unlink(dir, dentry);
 				unlock_kernel();
-				if (!error)
+				if (!error &&
+				    !(dentry->d_flags & DCACHE_NFSFS_RENAMED))
 					d_delete(dentry);
 			}
 		}
 	}
 	up(&dir->i_zombie);
-	if (!error)
+	if (!error && !(dentry->d_flags & DCACHE_NFSFS_RENAMED))
 		inode_dir_notify(dir, DN_DELETE);
 	return error;
 }


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: Re: [autofs] VFS: Busy inodes after unmount on 2 way SMP

[Ion Badulescu]

On Fri, 26 Sep 2003, Trond Myklebust wrote:

>      > Perhaps making that wait an uninterruptable wait would be
>      > enough to take care of the problem?
> 
> People expect the "intr" flag to work if they set it.

People also expect the supported "intr" flag not to cause security 
problems and would gladly take an unkillable process over a kernel oops...

Ion

-- 
  It is better to keep your mouth shut and be thought a fool,
            than to open it and remove all doubt.




-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [NFS] Re: VFS: Busy inodes after unmount on 2 way SMP

[Ion Badulescu]

On Wed, 17 Sep 2003, H. Peter Anvin wrote:

> > Why do you bring the daemon into the picture, it is irrelevant here. 
> 
> Because it makes it a lot harder to debug the problem.

Ok, fair enough.

The daemon version doesn't matter, any daemon will do. All you need is to 
kill -9 automountd while something is mounted, and then unmount everything 
by hand without removing the mountpoint directories.

> > # cat /proc/mounts
> > [snip]
> > boxter:(pid29327) /bar autofs rw 0 0
> > boxter:(pid29327) /net autofs rw 0 0
> > boxter:(pid29327) /src autofs rw 0 0
> > boxter:(pid29327) /misc autofs rw 0 0
> > buggy:/n/buggy/misc/homes /misc/homes nfs rw,v3,rsize=8192,wsize=8192,hard,intr,tcp,lock,addr=buggy 0 0
> > # umount /misc/homes
> > # umount /misc
> > # dmesg |tail
> > VFS: Busy inodes after unmount. Self-destruct in 5 seconds.  Have a nice day...
> 
> OK.  Did you try Olaf's NFS patch?

This is not an NFS problem. The message occurs regardless of what is 
mounted on /misc/homes (NFS, local ext2, bind mount, whatever).

Ion

-- 
  It is better to keep your mouth shut and be thought a fool,
            than to open it and remove all doubt.

Re: Re: [autofs] VFS: Busy inodes after unmount on 2 way SMP

[Trond Myklebust]

>>>>> " " == Ion Badulescu <lists@news.cs.columbia.edu> writes:

    >> People expect the "intr" flag to work if they set it.

     > People also expect the supported "intr" flag not to cause
     > security problems and would gladly take an unkillable process
     > over a kernel oops...


You'd have to be extremely unlucky to kill the process and hit the
window for the Oops. I defy you to come up with an exploit for it.

That said, I agree that a full fix would be preferable. I'm working on
other projects right now, that's why I'm being slow about this issue
(plus the fact that it's not exactly easy to reproduce). I'll get onto
it soon...

Cheers,
  Trond


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [NFS] Re: VFS: Busy inodes after unmount on 2 way SMP

[Ian Kent]

On Mon, 29 Sep 2003, Ion Badulescu wrote:

> On Wed, 17 Sep 2003, H. Peter Anvin wrote:
> 
> > > Why do you bring the daemon into the picture, it is irrelevant here. 
> > 
> > Because it makes it a lot harder to debug the problem.
> 
> Ok, fair enough.
> 
> The daemon version doesn't matter, any daemon will do. All you need is to 
> kill -9 automountd while something is mounted, and then unmount everything 
> by hand without removing the mountpoint directories.
> 

Yes, your'e exactly right. That's expected behaviour as the directory 
dentries still exist and there is no daemon to remove them. Hence the 
message. Seeing this message has always been a signal to me that the 
daemon has a bug.

I have been thinking that implementing a umount_begin routine in the 
super_operations for autofs4 would remedy the problem. I believe that, 
correct me if I'm wrong, the umount must be done with a force option in 
this case, so the umount_begin would be called.

snip ...

> > 
> > OK.  Did you try Olaf's NFS patch?
> 
> This is not an NFS problem. The message occurs regardless of what is 
> mounted on /misc/homes (NFS, local ext2, bind mount, whatever).

I must admit, reading through the thread, I was wondering how it could 
be an NFS problem.

-- 

   ,-._|\    Ian Kent
  /      \   Perth, Western Australia
  *_.--._/   E-mail: raven@themaw.net
        v    Web: http://themaw.net/

Re: Re: [autofs] VFS: Busy inodes after unmount on 2 way SMP

[Olaf Kirch]

[-- Attachment #1: Type: text/plain, Size: 1617 bytes --]

On Mon, Sep 29, 2003 at 10:22:40AM -0700, Trond Myklebust wrote:
> You'd have to be extremely unlucky to kill the process and hit the
> window for the Oops. I defy you to come up with an exploit for it.
> 
> That said, I agree that a full fix would be preferable. I'm working on
> other projects right now, that's why I'm being slow about this issue
> (plus the fact that it's not exactly easy to reproduce). I'll get onto
> it soon...

I've had no luck reproducing it either in a controlled environment.
However the bug is common enough to crash a few of our build machines
over a weekend...

I took your patch, Trond, and modified it a little to not crash if
the user ctrl-c's the unlink. Still not pretty - maybe it should
zap the cached attributes:

@@ -212,7 +222,15 @@
        data->count++;
        nfs_copy_dname(dentry, data);
        dentry->d_flags &= ~DCACHE_NFSFS_RENAMED;
-       if (data->task.tk_rpcwait == &nfs_delete_queue)
+       if (data->task.tk_rpcwait == &nfs_delete_queue) {
+               struct rpc_clnt *clnt = data->task.tk_client;
                rpc_wake_up_task(&data->task);
+               nfs_wait_event(clnt, data->waitq, data->completed == 1);
+               /* This is safe as we hold the BKL */
+               if (!data->completed) {
+                       dput(data->dir);
+                       data->dir = NULL;
+               }
+       }
        nfs_put_unlinkdata(data);
 }

Olaf
-- 
Olaf Kirch     |  Anyone who has had to work with X.509 has probably
okir@suse.de   |  experienced what can best be described as
---------------+  ISO water torture. -- Peter Gutmann

[-- Attachment #2: nfs-autofs-umount-crash --]
[-- Type: text/plain, Size: 2362 bytes --]

diff -ur linux-2.4.21/fs/nfs/dir.c nfs/fs/nfs/dir.c
--- linux-2.4.21/fs/nfs/dir.c	2003-09-29 10:33:41.000000000 +0200
+++ nfs/fs/nfs/dir.c	2003-09-29 12:34:36.000000000 +0200
@@ -1144,7 +1144,7 @@
 	struct inode *old_inode = old_dentry->d_inode;
 	struct inode *new_inode = new_dentry->d_inode;
 	struct dentry *dentry = NULL, *rehash = NULL;
-	int error = -EBUSY;
+	int error;
 
 	/*
 	 * To prevent any new references to the target during the rename,
@@ -1170,6 +1170,12 @@
 	 */
 	if (!new_inode)
 		goto go_ahead;
+	/* If target is a hard link to the source, then noop */
+	error = 0;
+	if (NFS_FILEID(new_inode) == NFS_FILEID(old_inode))
+		goto out;
+
+	error = -EBUSY;
 	if (S_ISDIR(new_inode->i_mode))
 		goto out;
 	else if (atomic_read(&new_dentry->d_count) > 1) {
diff -ur linux-2.4.21/fs/nfs/unlink.c nfs/fs/nfs/unlink.c
--- linux-2.4.21/fs/nfs/unlink.c	2002-11-29 00:53:15.000000000 +0100
+++ nfs/fs/nfs/unlink.c	2003-09-29 13:38:49.000000000 +0200
@@ -12,6 +12,7 @@
 #include <linux/sunrpc/sched.h>
 #include <linux/sunrpc/clnt.h>
 #include <linux/nfs_fs.h>
+#include <linux/wait.h>
 
 
 struct nfs_unlinkdata {
@@ -21,6 +22,9 @@
 	struct rpc_task	task;
 	struct rpc_cred	*cred;
 	unsigned int	count;
+
+	wait_queue_head_t waitq;
+	int completed;
 };
 
 static struct nfs_unlinkdata	*nfs_deletes;
@@ -54,6 +58,8 @@
 		nfs_detach_unlinkdata(data);
 		if (data->name.name != NULL)
 			kfree(data->name.name);
+		if (data->cred)
+			put_rpccred(data->cred);
 		kfree(data);
 	}
 }
@@ -133,6 +139,8 @@
 	put_rpccred(data->cred);
 	data->cred = NULL;
 	dput(dir);
+	data->completed = 1;
+	wake_up(&data->waitq);
 }
 
 /**
@@ -175,6 +183,8 @@
 	nfs_deletes = data;
 	data->count = 1;
 
+	init_waitqueue_head(&data->waitq);
+
 	task = &data->task;
 	rpc_init_task(task, clnt, nfs_async_unlink_done , RPC_TASK_ASYNC);
 	task->tk_calldata = data;
@@ -212,7 +222,15 @@
 	data->count++;
 	nfs_copy_dname(dentry, data);
 	dentry->d_flags &= ~DCACHE_NFSFS_RENAMED;
-	if (data->task.tk_rpcwait == &nfs_delete_queue)
+	if (data->task.tk_rpcwait == &nfs_delete_queue) {
+		struct rpc_clnt *clnt = data->task.tk_client;
 		rpc_wake_up_task(&data->task);
+		nfs_wait_event(clnt, data->waitq, data->completed == 1);
+		/* This is safe as we hold the BKL */
+		if (!data->completed) {
+			dput(data->dir);
+			data->dir = NULL;
+		}
+	}
 	nfs_put_unlinkdata(data);
 }

Re: [NFS] Re: VFS: Busy inodes after unmount on 2 way SMP

[Mike Waychison]

Ian Kent wrote:

>>>OK.  Did you try Olaf's NFS patch?
>>>      
>>>
>>This is not an NFS problem. The message occurs regardless of what is 
>>mounted on /misc/homes (NFS, local ext2, bind mount, whatever).
>>    
>>
>
>I must admit, reading through the thread, I was wondering how it could 
>be an NFS problem.
>
>  
>
Inodes are only initialized once, when they are first allocated from the 
slab.  Any reused inodes will continue having a bad refcount, causing 
similar behaviour on different filesystems.

Can this be reproduced on a filesystem other than NFS for which the 
system has never made an NFS mount?

Mike Waychison

Re: [NFS] Re: VFS: Busy inodes after unmount on 2 way SMP

[Ion Badulescu]

Hi Ian,

On Tue, 30 Sep 2003, Ian Kent wrote:

> On Mon, 29 Sep 2003, Ion Badulescu wrote:
> > The daemon version doesn't matter, any daemon will do. All you need is to 
> > kill -9 automountd while something is mounted, and then unmount everything 
> > by hand without removing the mountpoint directories.
> > 
> 
> Yes, your'e exactly right. That's expected behaviour as the directory 
> dentries still exist and there is no daemon to remove them.

Hmm. Expected behaviour, you say? Maybe, maybe not, it's hard to tell 
what's to be expected here, since the autofs umount semantics are so 
different from a regular filesystems.

Normally a filesystem is allowed to be unmounted if it is not busy, right?
But with autofs it's different, it should be unmounted only if it is
completely empty, with no entries at all -- and I'm not convinced the VFS
layer is up to the task of dealing with this requirement.  Alternatively
the autofs code should be prepared to clean up after itself, but that
might not be possible in all circumstances.

> Hence the message. Seeing this message has always been a signal to me
> that the daemon has a bug.

Oh, you're absolutely right here (although we're talking about different
daemons, I'm using--and expanding, and debugging--amd's autofs support).
:-) Nonetheless, it's a bug in the kernel code that should be remedied.

> I have been thinking that implementing a umount_begin routine in the 
> super_operations for autofs4 would remedy the problem. I believe that, 
> correct me if I'm wrong, the umount must be done with a force option in 
> this case, so the umount_begin would be called.

No, you're right, you need MNT_FORCE for umount_begin to be called.

That, however, would be a reasonable solution if the filesystem also
prevented its own unmounting when it is not empty (by having each entry
hold a reference to the vfsmount in its private data, perhaps, I'm not
sure). Then, inside umount_begin, and only if the filesystem is catatonic,
it would garbage-collect any entries not mounted upon and not busy. That
should work, I think...

Thanks,
Ion

-- 
  It is better to keep your mouth shut and be thought a fool,
            than to open it and remove all doubt.

Re: Re: [autofs] VFS: Busy inodes after unmount on 2 way SMP

[Trond Myklebust]

>>>>> " " == Olaf Kirch <okir@suse.de> writes:

     > On Mon, Sep 29, 2003 at 10:22:40AM -0700, Trond Myklebust

    >> That said, I agree that a full fix would be preferable. I'm
    >> working on other projects right now, that's why I'm being slow
    >> about this issue (plus the fact that it's not exactly easy to
    >> reproduce). I'll get onto it soon...

     > I've had no luck reproducing it either in a controlled
     > environment.  However the bug is common enough to crash a few
     > of our build machines over a weekend...

     > I took your patch, Trond, and modified it a little to not crash
     > if the user ctrl-c's the unlink. Still not pretty - maybe it
     > should zap the cached attributes:

Alternatively, we should perhaps make fsync_super() sync out all dirty
inodes and readaheads, then wait on all pending RPC ops. That would be
cleaner, I think...

Cheers,
  Trond


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [NFS] Re: VFS: Busy inodes after unmount on 2 way SMP

[Ian Kent]

On Tue, 30 Sep 2003, Ion Badulescu wrote:

> Hi Ian,
> 
> On Tue, 30 Sep 2003, Ian Kent wrote:
> 
> > On Mon, 29 Sep 2003, Ion Badulescu wrote:
> > > The daemon version doesn't matter, any daemon will do. All you need is to 
> > > kill -9 automountd while something is mounted, and then unmount everything 
> > > by hand without removing the mountpoint directories.
> > > 
> > 
> > Yes, your'e exactly right. That's expected behaviour as the directory 
> > dentries still exist and there is no daemon to remove them.
> 
> Hmm. Expected behaviour, you say? Maybe, maybe not, it's hard to tell 
> what's to be expected here, since the autofs umount semantics are so 
> different from a regular filesystems.

For autofs v4 anyway possibly v3 also. Cause of the nature of it.

> 
> Normally a filesystem is allowed to be unmounted if it is not busy, right?
> But with autofs it's different, it should be unmounted only if it is
> completely empty, with no entries at all -- and I'm not convinced the VFS
> layer is up to the task of dealing with this requirement.  Alternatively
> the autofs code should be prepared to clean up after itself, but that
> might not be possible in all circumstances.
> 
> > Hence the message. Seeing this message has always been a signal to me
> > that the daemon has a bug.
> 
> Oh, you're absolutely right here (although we're talking about different
> daemons, I'm using--and expanding, and debugging--amd's autofs support).
> :-) Nonetheless, it's a bug in the kernel code that should be remedied.
> 
> > I have been thinking that implementing a umount_begin routine in the 
> > super_operations for autofs4 would remedy the problem. I believe that, 
> > correct me if I'm wrong, the umount must be done with a force option in 
> > this case, so the umount_begin would be called.
> 
> No, you're right, you need MNT_FORCE for umount_begin to be called.
> 
> That, however, would be a reasonable solution if the filesystem also
> prevented its own unmounting when it is not empty (by having each entry
> hold a reference to the vfsmount in its private data, perhaps, I'm not
> sure). Then, inside umount_begin, and only if the filesystem is catatonic,
> it would garbage-collect any entries not mounted upon and not busy. That
> should work, I think...

As you point out I am talking about autofs not amd.

I think it's a lot simpler than it sounds above. Granted if the tree is 
busy with mounts you can probably only give up but much of the time it's 
not. The logic to traverse the filesystem would be similar to what I have 
used in my rewrite of the expire module in autofs4. Check it out at 
ftp://ftp.kernel.org/pub/linux/daemons/autofs/v4 and let me know if 
you think the idea has merit.

-- 

   ,-._|\    Ian Kent
  /      \   Perth, Western Australia
  *_.--._/   E-mail: raven@themaw.net
        v    Web: http://themaw.net/

Re: [NFS] Re: VFS: Busy inodes after unmount on 2 way SMP

[Ian Kent]

On Tue, 30 Sep 2003, Mike Waychison wrote:

> Ian Kent wrote:

> >
> >I must admit, reading through the thread, I was wondering how it could 
> >be an NFS problem.
> >
> >  
> >
> Inodes are only initialized once, when they are first allocated from the 
> slab.  Any reused inodes will continue having a bad refcount, causing 
> similar behaviour on different filesystems.
> 
> Can this be reproduced on a filesystem other than NFS for which the 
> system has never made an NFS mount?
> 

All my usage is NFS so I can't say.


-- 

   ,-._|\    Ian Kent
  /      \   Perth, Western Australia
  *_.--._/   E-mail: raven@themaw.net
        v    Web: http://themaw.net/