From: Bart Van Assche <bvanassche@acm.org>
To: Sagi Grimberg <sagi@grimberg.me>,
Yamin Friedman <yaminf@mellanox.com>,
kernel test robot <rong.a.chen@intel.com>
Cc: Jason Gunthorpe <jgg@nvidia.com>,
Max Gurtovoy <maxg@mellanox.com>,
LKML <linux-kernel@vger.kernel.org>,
Doug Ledford <dledford@redhat.com>,
Jason Gunthorpe <jgg+lists@ziepe.ca>,
linux-rdma@vger.kernel.org, lkp@lists.01.org
Subject: Re: [IB/srpt] c804af2c1d: last_state.test.blktests.exit_code.143
Date: Sun, 23 Aug 2020 14:18:41 -0700 [thread overview]
Message-ID: <128192ad-05ff-fa8e-14fc-479a115311e0@acm.org> (raw)
In-Reply-To: <0c42aeb4-23a5-b9d5-bc17-ef58a04db8e8@grimberg.me>
On 2020-08-03 00:27, Sagi Grimberg wrote:
>
>>>> Greeting,
>>>>
>>>> FYI, we noticed the following commit (built with gcc-9):
>>>>
>>>> commit: c804af2c1d3152c0cf877eeb50d60c2d49ac0cf0 ("IB/srpt: use new shared CQ mechanism")
>>>> https://git.kernel.org/cgit/linux/kernel/git/rdma/rdma.git for-next
>>>>
>>>>
>>>> in testcase: blktests
>>>> with following parameters:
>>>>
>>>> test: srp-group1
>>>> ucode: 0x21
>>>>
>>>>
>>>>
>>>> on test machine: 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 4G memory
>>>>
>>>> caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
>>>>
>>>>
>>>>
>>>>
>>>> If you fix the issue, kindly add following tag
>>>> Reported-by: kernel test robot <rong.a.chen@intel.com>
>>>>
>>>>
>>>> user :notice: [ 44.688140] 2020-08-01 16:10:22 ./check srp/001 srp/002 srp/003 srp/004 srp/005 srp/006 srp/007 srp/008 srp/009 srp/010 srp/011 srp/012 srp/013 srp/015
>>>> user :notice: [ 44.706657] srp/001 (Create and remove LUNs)
>>>> user :notice: [ 44.718405] srp/001 (Create and remove LUNs) [passed]
>>>> user :notice: [ 44.729902] runtime ... 1.972s
>>>> user :notice: [ 99.038748] IPMI BMC is not supported on this machine, skip bmc-watchdog setup!
>>>> user :notice: [ 3699.039790] Sat Aug 1 17:11:22 UTC 2020 detected soft_timeout
>>>> user :notice: [ 3699.060341] kill 960 /usr/bin/time -v -o /tmp/lkp/blktests.time /lkp/lkp/src/tests/blktests
>>> Yamin and Max, can you take a look at this? The SRP tests from the
>>> blktests repository pass reliably with kernel version v5.7 and before.
>>> With label next-20200731 from linux-next however that test triggers the
>>> following hang:
>>
>> I will look into it.
>
> FWIW, I ran into this as well with nvme-rdma, but it also reproduces
> when I revert the shared CQ patch from nvme-rdma. Another data point
> is that my tests passes with siw.
Hi Jason,
The patch below is sufficient to unbreak blktests. I think that the
deadlock while unloading rdma_rxe happens because the RDMA core waits for
all ib_dev references to be dropped before dealloc_driver is called.
The rdma_rxe dealloc_driver implementation drops an ib_dev reference. The
dealloc_driver callback was introduced by commit d0899892edd0
("RDMA/device: Provide APIs from the core code to help unregistration").
Do you agree that this regression has been introduced by commits
d0899892edd0 and c367074b6c37 ("RDMA/rxe: Use driver_unregister and new
unregistration API")?
Thanks,
Bart.
---
drivers/infiniband/core/device.c | 7 +------
1 file changed, 1 insertion(+), 6 deletions(-)
diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
index dca2842a7872..5192f305b253 100644
--- a/drivers/infiniband/core/device.c
+++ b/drivers/infiniband/core/device.c
@@ -1287,13 +1287,8 @@ static void disable_device(struct ib_device *device)
/* Pairs with refcount_set in enable_device */
ib_device_put(device);
- wait_for_completion(&device->unreg_completion);
- /*
- * compat devices must be removed after device refcount drops to zero.
- * Otherwise init_net() may add more compatdevs after removing compat
- * devices and before device is disabled.
- */
+ /* To do: prevent init_net() to add more compat_devs. */
remove_compat_devs(device);
}
next prev parent reply other threads:[~2020-08-23 21:18 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-02 6:09 [IB/srpt] c804af2c1d: last_state.test.blktests.exit_code.143 kernel test robot
2020-08-02 15:05 ` Bart Van Assche
2020-08-03 6:19 ` Yamin Friedman
2020-08-03 7:27 ` Sagi Grimberg
2020-08-23 21:18 ` Bart Van Assche [this message]
2020-08-24 13:30 ` Jason Gunthorpe
2020-09-07 2:58 ` Bart Van Assche
2020-09-08 18:22 ` Jason Gunthorpe
2020-09-09 2:01 ` Bart Van Assche
2020-09-11 22:00 ` Bart Van Assche
2020-09-12 4:46 ` Yi Zhang
2020-09-14 18:26 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=128192ad-05ff-fa8e-14fc-479a115311e0@acm.org \
--to=bvanassche@acm.org \
--cc=dledford@redhat.com \
--cc=jgg+lists@ziepe.ca \
--cc=jgg@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=lkp@lists.01.org \
--cc=maxg@mellanox.com \
--cc=rong.a.chen@intel.com \
--cc=sagi@grimberg.me \
--cc=yaminf@mellanox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).