races in ipathfs

races in ipathfs

[Al Viro]

	Use of qib_super is seriously racy.  qibfs_add() (and worse,
qibfs_remove()) can happen during qibfs_mount() and qibfs_kill_super().

	1) CPU1: qib_init_one().  The sucker is allocated and placed
on the list.  CPU2: ipathfs is mounted, directory created.  CPU1: finally
gets around to qibfs_add(); by now qib_super is non-NULL and off we go,
trying to create it again.  The worst part is, that code doesn't even
notice that dentry is there and positive; you silently leak the old inode.

	2) CPU1: qib_init_one().  Allocated the sucker.  CPU2: ipathfs
is getting mounted.  Picked the first device off the list, creating
directory for it.  CPU1: inserted new device into the head of the list,
continued working.  Got around to qibfs_add(); qib_super is NULL, so
we do nothing.  CPU2: walked the rest of the list, creating directories
for all devices.  Our device is missed, since we are past that point in
the list.  Worse, shift the timing a bit and it doesn't matter whether
you add to the head or to the tail of the list - if qibfs_add() happens
just before we set qib_super, we are screwed again.

	3) CPU1: qib_remove_one().  CPU2: mount ipathfs is walking that
list and decides to try and create a directory for the device that is
being freed.  Oops...

	4) CPU1: qib_init_one() or qib_remove_one(), doesn't matter which.
CPU2: final umount of ipathfs already got through setting sb->s_root to
NULL but still hadn't set qib_super to the same.  Oops...  And no,
moving that qib_super = NULL; up prior to kill_litter_super() won't
fix the race either, of course.

AFAICS, the older driver (in hw/ipath) has the same problems.

RE: races in ipathfs

[Mike Marciniszyn]

We are currently investigating this.

Thanks for the review on this issue!

Mike

> -----Original Message-----
> From: linux-rdma-owner@vger.kernel.org [mailto:linux-rdma-
> owner@vger.kernel.org] On Behalf Of Al Viro
> Sent: Thursday, January 19, 2012 3:20 PM
> To: Dept_Infinipath
> Cc: linux-rdma@vger.kernel.org; linux-kernel
> Subject: races in ipathfs
>
>       Use of qib_super is seriously racy.  qibfs_add() (and worse,
> qibfs_remove()) can happen during qibfs_mount() and qibfs_kill_super().
>
>       1) CPU1: qib_init_one().  The sucker is allocated and placed
> on the list.  CPU2: ipathfs is mounted, directory created.  CPU1:
> finally
> gets around to qibfs_add(); by now qib_super is non-NULL and off we go,
> trying to create it again.  The worst part is, that code doesn't even
> notice that dentry is there and positive; you silently leak the old
> inode.
>
>       2) CPU1: qib_init_one().  Allocated the sucker.  CPU2: ipathfs
> is getting mounted.  Picked the first device off the list, creating
> directory for it.  CPU1: inserted new device into the head of the list,
> continued working.  Got around to qibfs_add(); qib_super is NULL, so
> we do nothing.  CPU2: walked the rest of the list, creating directories
> for all devices.  Our device is missed, since we are past that point in
> the list.  Worse, shift the timing a bit and it doesn't matter whether
> you add to the head or to the tail of the list - if qibfs_add() happens
> just before we set qib_super, we are screwed again.
>
>       3) CPU1: qib_remove_one().  CPU2: mount ipathfs is walking that
> list and decides to try and create a directory for the device that is
> being freed.  Oops...
>
>       4) CPU1: qib_init_one() or qib_remove_one(), doesn't matter
> which.
> CPU2: final umount of ipathfs already got through setting sb->s_root to
> NULL but still hadn't set qib_super to the same.  Oops...  And no,
> moving that qib_super = NULL; up prior to kill_litter_super() won't
> fix the race either, of course.
>
> AFAICS, the older driver (in hw/ipath) has the same problems.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


This message and any attached documents contain information from QLogic Corporation or its wholly-owned subsidiaries that may be confidential. If you are not the intended recipient, you may not read, copy, distribute, or use this information. If you have received this transmission in error, please notify the sender immediately by reply e-mail and then delete this message.

Re: races in ipathfs

[Al Viro]

On Thu, Jan 19, 2012 at 08:20:04PM +0000, Al Viro wrote:
> 	Use of qib_super is seriously racy.  qibfs_add() (and worse,
> qibfs_remove()) can happen during qibfs_mount() and qibfs_kill_super().

[snip]

FWIW, I've put a completely untested patchset into
git.kernel.org/pub/scm/linux/kernel/git/viro/vfs.git qibfs

It tries to avoid that kind of crap by getting rid of "populate at mount
time" logics - we keep (internal) mount of that sucker for as long as
there are any devices owned by ib_qib, adding them on add_device() and
removing on remove_device().  The only complication is with module
use counts - that fs has to be a separate module, or we'll have ib_qib
impossible to rmmod, because fs keeps its module pinned and any devices
held by ib_qib PCI driver will keep the fs pinned, so we never get
to unregistering said PCI driver.  With skeleton of qibfs (static parts
only) taken to ib_qib_fs.c we avoid that problem - it is what ends up
being pinned down for as long as ib_qib owns any devices, but then it's
pinned down by ib_qib using exports from ib_qib_fs anyway.  And once
ib_qib is removed, all internal references to that vfsmount and superblock
disappear as well...

This stuff is completely untested; I don't have the hardware in question.
It does compile and survive modpost, but that's it.  Please, review and
comment...

Re: races in ipathfs

[Al Viro]

On Sun, Mar 18, 2012 at 06:45:47PM +0000, Al Viro wrote:
> On Thu, Jan 19, 2012 at 08:20:04PM +0000, Al Viro wrote:
> > 	Use of qib_super is seriously racy.  qibfs_add() (and worse,
> > qibfs_remove()) can happen during qibfs_mount() and qibfs_kill_super().
> 
> [snip]
> 
> FWIW, I've put a completely untested patchset into
> git.kernel.org/pub/scm/linux/kernel/git/viro/vfs.git qibfs

Corresponding fix for ipathfs added to the same branch; again, completely
untested.