From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB907C10F27 for ; Fri, 6 Mar 2020 13:34:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7DAD02166E for ; Fri, 6 Mar 2020 13:34:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=cloud.ionos.com header.i=@cloud.ionos.com header.b="CnuwD6IW" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727487AbgCFNda (ORCPT ); Fri, 6 Mar 2020 08:33:30 -0500 Received: from mail-io1-f68.google.com ([209.85.166.68]:42583 "EHLO mail-io1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727485AbgCFNda (ORCPT ); Fri, 6 Mar 2020 08:33:30 -0500 Received: by mail-io1-f68.google.com with SMTP id q128so2056918iof.9 for ; Fri, 06 Mar 2020 05:33:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=eOgNubgvkB+PFv8kOH/Puf/i8J9C6TgnMdeObKC0G2g=; b=CnuwD6IW/Zj53S5DiPzKIfnMKa/2r1EcNlAgSe9glmmT8Dz0m8kk6f5EXlxqU6EzyZ mHLtw+Abz96XpzRUkGL5je/otVL2uYAZoDsTKunKgf/xpx8kjyRo9pcmDFE6mAP8fO9V mueONh6C/wKb/LWPZu4Hdf0Bl8OsiCmwHnErMVBen8fF6arV3fy9gKO/WaH7tUFop3Ho qEQ8aiDu8qPDQJKDRoZitYf6oco+48StXcNdVR3/cfT3qataUpYFk0FklCmKHJEmHu2E BzPulcoOlL4cT/T4WzOjfpizhpWd2VsqDq0YAS9Nk66hdDcBYRDiq5gKi3pLRtKk4Ktn 3Dog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=eOgNubgvkB+PFv8kOH/Puf/i8J9C6TgnMdeObKC0G2g=; b=m9tu+r4BTTDjfGoyyRWp1Ui/mN1AQPpRmugMbP9GL3SGgQ0vdlddar7HrSTqd6VRE6 lzd8ypMgMKgQhp7WorV/oZQSApDti9OvJI+JsQUhEFanQE9dK07BY1lCZ55HMmcHCftd Nc+/WXhTjgtFDR1yN4TYajJN4PFDX0XzLOKVsV1NXVxOJx0TQgxdXX+sC8mLULG9m8RB ojl7/RWeshLyK9cTDa/xR0bDhv8vGVbceYKeTqwa+UctiCexVBJJWSOcet2LYcfX20gr gYBMcAhNzJXcldUi1d+liPVKgv9yb3LbEYMhTE/92fQZzgHHI+iXLRuuiSweOnRhInzH mbng== X-Gm-Message-State: ANhLgQ1jVr8FuiXQmpZ6DfaYR0qkJubDM9mNz8y8ZwtyU1n6b3QwrFfo cgcQ3GyLyJ/uk2hG42IsL6pmrTCEdTB7swQFqzhCEg== X-Google-Smtp-Source: ADFU+vvh8femgnXWgeDBLCi2J965f7nTs4zoQBV4JhSVb97H+FBR5UyAbx/Rxb9RCxt0jHVlP14KqqDS1RaBkaqCqwI= X-Received: by 2002:a05:6638:e8b:: with SMTP id p11mr2935174jas.11.1583501609095; Fri, 06 Mar 2020 05:33:29 -0800 (PST) MIME-Version: 1.0 References: <20200221104721.350-1-jinpuwang@gmail.com> <20200221104721.350-7-jinpuwang@gmail.com> <597777e7-ac5a-5fc4-c1f7-3ffa5876a6f2@acm.org> <20200304164903.GF31668@ziepe.ca> <20200305132748.GH31668@ziepe.ca> <20200305135455.GI31668@ziepe.ca> In-Reply-To: <20200305135455.GI31668@ziepe.ca> From: Jinpu Wang Date: Fri, 6 Mar 2020 14:33:18 +0100 Message-ID: Subject: Re: [PATCH v9 06/25] RDMA/rtrs: client: main functionality To: Jason Gunthorpe Cc: Bart Van Assche , Danil Kipnis , Jack Wang , linux-block@vger.kernel.org, linux-rdma@vger.kernel.org, Jens Axboe , Christoph Hellwig , Sagi Grimberg , Leon Romanovsky , Doug Ledford , Roman Penyaev , Pankaj Gupta Content-Type: text/plain; charset="UTF-8" Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org On Thu, Mar 5, 2020 at 2:54 PM Jason Gunthorpe wrote: > > On Thu, Mar 05, 2020 at 02:37:59PM +0100, Jinpu Wang wrote: > > On Thu, Mar 5, 2020 at 2:27 PM Jason Gunthorpe wrote: > > > > > > On Thu, Mar 05, 2020 at 12:26:01PM +0100, Jinpu Wang wrote: > > > > > > > We have to admit, the code snip is from null_blk, get_tag function, > > > > not invented by us. > > > > the get_cpu/put_cpu was added to get/save the current cpu_id, which > > > > can be removed around the do-while loop., > > > > we only need to raw_smp_processor_id to get current cpu, we use it > > > > later to pick which connection to use. > > > > > > Be careful copying crazy core code into drivers.. > > > > > > > > You have to do something to provably guarantee the send q cannot > > > > > overflow. send q overflow is defined as calling post_send before a > > > > > poll_cq has confirmed space is available for send. > > > > > > > Shouldn't the cq api handle that already, with IB_POLL_SOFTIRQ, > > > > poll cq is done on very softirq run, so send queue space should be reclaimed > > > > fast enough, with IB_POLL_WORKQUEUE, when cq->com_handler get called, > > > > the ib_cq_poll_work will do the poll_cq, together with extra > > > > send_queue size reserved, > > > > the send queue can not overflow! > > > > > > Somehow that doesn't sound like 'provably guarentee' - that is some > > > statistical argument.. > > Could you give an example which meets the "provably guarantee", > > seems most of the driver is based on the cq API. > > You are supposed to directly keep track of completions and not issue > sends until completions are seen. > > Jason ok, got it! Thanks