From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A23CBC433F5 for ; Wed, 23 Feb 2022 04:43:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235685AbiBWEno (ORCPT ); Tue, 22 Feb 2022 23:43:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54764 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237886AbiBWEno (ORCPT ); Tue, 22 Feb 2022 23:43:44 -0500 Received: from out0.migadu.com (out0.migadu.com [94.23.1.103]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F35A66AE9 for ; Tue, 22 Feb 2022 20:43:17 -0800 (PST) Subject: Re: bug report for rxe DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1645591395; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AHdUSkvRI3WVyveWp4jskJTj3UvHdvFbD3jmOkG2lzA=; b=O+EQVBxRN71UT+T4WTV920UjO4DthNNS5N0Bmm6m5bixX6pn2hO/fcN30jBMaLl9pSXU5i WiLqrYSKCJ72+HlTdH62w2WilXIvI+odKqsG4hSS+S4lf6iD7arJvn5m0SJy8mzkksAwt8 maNXVWw1QsGxPdoBl/ALvmrVydv21Yc= To: "Pearson, Robert B" , Zhu Yanjun , Jinpu Wang Cc: Jason Gunthorpe , Bob Pearson , RDMA mailing list References: <20220210073655.42281-1-guoqing.jiang@linux.dev> <473a53b6-9ab2-0d48-a9cf-c84b8dc4c3f3@linux.dev> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Guoqing Jiang Message-ID: <3b6ddb23-6dfa-29e2-27fd-741c1e3e576d@linux.dev> Date: Wed, 23 Feb 2022 12:43:08 +0800 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org On 2/23/22 12:58 AM, Pearson, Robert B wrote: >> After investigation, seems the culprit is commit 647bf13ce944 ("RDMA/rxe: >> Create duplicate mapping tables for FMRs"). The problem is >> mr_check_range returns -EFAULT after find iova and length are not >> valid, so connection between two VMs can't be established. >> >> Revert the commit manually or apply below temporary change, rxe works >> again with rnbd/rtrs though I don't think it is the right thing to do. >> Could experts provide a proper solution? Thanks. >> > This patch fixed failures in blktests and srp which were discussed at length. See e.g. > > https://lore.kernel.org/linux-rdma/20210907163939.GW1200268@ziepe.ca/ Thanks for the link, which reminds me the always_invalidate parameter in rtrs_server. > and related messages. The conclusion was that two mappings were required. One owned by the > driver and one by the 'hardware', i.e. bottom half in the rxe case, allowing updating a new mapping > while the old one is still active and then switching them. > > If this case has iova and length not valid as indicated is there a problem with the test case? And after disable always_invalidate (which is enabled by default), rnbd/rtrs over roce works either. So I suppose there might be potential issue for always_invalidate=Y in rtrs server side since invalidate works for srp IIUC, @Jinpu. Thanks, Guoqing