From: Jason Gunthorpe <jgg@ziepe.ca>
To: Leon Romanovsky <leon@kernel.org>
Cc: Doug Ledford <dledford@redhat.com>,
Leon Romanovsky <leonro@mellanox.com>,
RDMA mailing list <linux-rdma@vger.kernel.org>,
Daniel Jurgens <danielj@mellanox.com>,
Erez Shitrit <erezsh@mellanox.com>,
Maor Gottlieb <maorg@mellanox.com>,
Michael Guralnik <michaelgur@mellanox.com>,
Moni Shoua <monis@mellanox.com>,
Parav Pandit <parav@mellanox.com>,
Sean Hefty <sean.hefty@intel.com>,
Valentine Fatiev <valentinef@mellanox.com>,
Yishai Hadas <yishaih@mellanox.com>,
Yonatan Cohen <yonatanc@mellanox.com>,
Zhu Yanjun <yanjunz@mellanox.com>
Subject: Re: [PATCH rdma-rc 8/9] IB/umad: Fix kernel crash while unloading ib_umad
Date: Thu, 13 Feb 2020 10:28:18 -0400 [thread overview]
Message-ID: <20200213142818.GA16120@ziepe.ca> (raw)
In-Reply-To: <20200212072635.682689-9-leon@kernel.org>
On Wed, Feb 12, 2020 at 09:26:34AM +0200, Leon Romanovsky wrote:
> From: Yonatan Cohen <yonatanc@mellanox.com>
>
> When unloading ib_umad, remove ibdev sys file 1st before
> port removal to prevent kernel oops.
>
> ib_mad's method ibdev_show() might access a umad port
> whoes ibdev field has already been NULLed when rmmod ib_umad
> was issued from another shell.
>
> Consider this scenario
> shell-1 shell-2
> rmmod ib_mod cat /sys/devices/../ibdev
> | |
> ib_umad_kill_port() ibdev_show()
> port->ib_dev = NULL dev_name(port->ib_dev)
>
> kernel stack
> PF: error_code(0x0000) - not-present page
> Oops: 0000 [#1] SMP DEBUG_PAGEALLOC PTI
> RIP: 0010:ibdev_show+0x18/0x50 [ib_umad]
> RSP: 0018:ffffc9000097fe40 EFLAGS: 00010282
> RAX: 0000000000000000 RBX: ffffffffa0441120 RCX: ffff8881df514000
> RDX: ffff8881df514000 RSI: ffffffffa0441120 RDI: ffff8881df1e8870
> RBP: ffffffff81caf000 R08: ffff8881df1e8870 R09: 0000000000000000
> R10: 0000000000001000 R11: 0000000000000003 R12: ffff88822f550b40
> R13: 0000000000000001 R14: ffffc9000097ff08 R15: ffff8882238bad58
> FS: 00007f1437ff3740(0000) GS:ffff888236940000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00000000000004e8 CR3: 00000001e0dfc001 CR4: 00000000001606e0
> Call Trace:
> dev_attr_show+0x15/0x50
> sysfs_kf_seq_show+0xb8/0x1a0
> seq_read+0x12d/0x350
> vfs_read+0x89/0x140
> ksys_read+0x55/0xd0
> do_syscall_64+0x55/0x1b0
> entry_SYSCALL_64_after_hwframe+0x44/0xa9:
>
> Fixes: e9dd5daf884c ("IB/umad: Refactor code to use cdev_device_add()")
This is the wrong fixes line, this ordering change was actually
deliberately done:
commit cf7ad3030271c55a7119a8c2162563e3f6e93879
Author: Parav Pandit <parav@mellanox.com>
Date: Fri Dec 21 16:19:24 2018 +0200
IB/umad: Avoid destroying device while it is accessed
ib_umad_reg_agent2() and ib_umad_reg_agent() access the device name in
dev_notice(), while concurrently, ib_umad_kill_port() can destroy the
device using device_destroy().
cpu-0 cpu-1
----- -----
ib_umad_ioctl()
[...] ib_umad_kill_port()
device_destroy(dev)
ib_umad_reg_agent()
dev_notice(dev)
The mistake in the above was to move the device_dstroy() down, not
split it into device_del() above and put_device() below.
Now that is already split we are OK.
Jason
next prev parent reply other threads:[~2020-02-13 14:28 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-02-12 7:26 [PATCH rdma-rc 0/9] Fixes for v5.6 Leon Romanovsky
2020-02-12 7:26 ` [PATCH rdma-rc 1/9] RDMA/ucma: Mask QPN to be 24 bits according to IBTA Leon Romanovsky
2020-02-12 7:26 ` [PATCH rdma-rc 2/9] RDMA/core: Fix protection fault in get_pkey_idx_qp_list Leon Romanovsky
2020-02-12 8:01 ` Leon Romanovsky
2020-02-12 8:06 ` Leon Romanovsky
2020-02-12 7:26 ` [PATCH rdma-rc 3/9] Revert "RDMA/cma: Simplify rdma_resolve_addr() error flow" Leon Romanovsky
2020-02-13 13:30 ` Jason Gunthorpe
2020-02-14 3:11 ` Parav Pandit
2020-02-14 14:08 ` Jason Gunthorpe
2020-02-14 14:48 ` Parav Pandit
2020-02-19 20:40 ` Jason Gunthorpe
2020-02-12 7:26 ` [PATCH rdma-rc 4/9] IB/ipoib: Fix double free of skb in case of multicast traffic in CM mode Leon Romanovsky
2020-02-13 15:37 ` Jason Gunthorpe
2020-02-13 18:10 ` Leon Romanovsky
2020-02-13 18:26 ` Jason Gunthorpe
2020-02-13 18:36 ` Leon Romanovsky
2020-02-13 19:09 ` Jason Gunthorpe
2020-02-12 7:26 ` [PATCH rdma-rc 5/9] RDMA/core: Add missing list deletion on freeing event queue Leon Romanovsky
2020-02-12 7:26 ` [PATCH rdma-rc 6/9] IB/mlx5: Fix async events cleanup flows Leon Romanovsky
2020-02-12 7:26 ` [PATCH rdma-rc 7/9] RDMA/rxe: Fix soft lockup problem due to using tasklets in softirq Leon Romanovsky
2020-02-12 7:26 ` [PATCH rdma-rc 8/9] IB/umad: Fix kernel crash while unloading ib_umad Leon Romanovsky
2020-02-13 14:28 ` Jason Gunthorpe [this message]
2020-02-13 18:03 ` Leon Romanovsky
2020-02-12 7:26 ` [PATCH rdma-rc 9/9] RDMA/mlx5: Prevent overflow in mmap offset calculations Leon Romanovsky
2020-02-13 18:03 ` [PATCH rdma-rc 0/9] Fixes for v5.6 Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200213142818.GA16120@ziepe.ca \
--to=jgg@ziepe.ca \
--cc=danielj@mellanox.com \
--cc=dledford@redhat.com \
--cc=erezsh@mellanox.com \
--cc=leon@kernel.org \
--cc=leonro@mellanox.com \
--cc=linux-rdma@vger.kernel.org \
--cc=maorg@mellanox.com \
--cc=michaelgur@mellanox.com \
--cc=monis@mellanox.com \
--cc=parav@mellanox.com \
--cc=sean.hefty@intel.com \
--cc=valentinef@mellanox.com \
--cc=yanjunz@mellanox.com \
--cc=yishaih@mellanox.com \
--cc=yonatanc@mellanox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).