Re: [PATCH for-rc] RDMA/hns: Fix RNR retransmission issue for HIP08

From: Jason Gunthorpe <jgg@nvidia.com>
To: Wenpeng Liang <liangwenpeng@huawei.com>
Cc: leon@kernel.org, linux-rdma@vger.kernel.org, linuxarm@huawei.com
Subject: Re: [PATCH for-rc] RDMA/hns: Fix RNR retransmission issue for HIP08
Date: Tue, 14 Dec 2021 19:47:03 -0400	[thread overview]
Message-ID: <20211214234703.GA968962@nvidia.com> (raw)
In-Reply-To: <20211209140655.49493-1-liangwenpeng@huawei.com>

On Thu, Dec 09, 2021 at 10:06:55PM +0800, Wenpeng Liang wrote:
> From: Yangyang Li <liyangyang20@huawei.com>
> 
> Due to the discrete nature of the HIP08 timer unit, a requester might
> finish the timeout period sooner, in elapsed real time, than its responder
> does, even when both sides share the identical RNR timeout length included
> in the RNR Nak packet and the responder indeed starts the timing prior to
> the requester. Furthermore, if a 'providential' resend packet arrived
> before the responder's timeout period expired, the responder is certainly
> entitled to drop the packet silently in the light of IB protocol.
> 
> To address this problem, our team made good use of certain hardware facts:
> 1) The timing resolution regards the transmission arrangements is 1
> microsecond, e.g. if cq_period field is set to 3, it would be interpreted
> as 3 microsecond by hardware;
> 2) A QPC field shall inform the hardware how many timing unit (ticks)
> constitutes a full microsecond, which, by default, is 1000;
> 3) It takes 14ns for the processor to handle a packet in the buffer, so the
> RNR timeout length of 10ns would ensure our processing mechanism is
> disabled during the entire timeout period and the packet won't be dropped
> silently;
> 
> To achieve (3), we permanently set the QPC field mentioned in (2) to zero
> which nominally indicates every time tick is equivalent to a microsecond
> in wall-clock time; now, a RNR timeout period at face value of 10 would
> only last 10 ticks, which is 10ns in wall-clock time.
> 
> It's worth noting that we adapt the driver by magnifying certain
> configuration parameters(cq_period, eq_period and ack_timeout)by 1000 given
> the user assumes the configuring timing unit to be microseconds.
> 
> Also, this particular improvisation is only deployed on HIP08 since other
> hardware has already solved this issue.
> 
> Fixes: cfc85f3e4b7f ("RDMA/hns: Add profile support for hip08 driver")
> Signed-off-by: Yangyang Li <liyangyang20@huawei.com>
> Signed-off-by: Wenpeng Liang <liangwenpeng@huawei.com>
> ---
>  drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 64 +++++++++++++++++++---
>  drivers/infiniband/hw/hns/hns_roce_hw_v2.h |  8 +++
>  2 files changed, 65 insertions(+), 7 deletions(-)

Applied to for-rc, thanks

Jason