From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60F2FC433E9 for ; Thu, 18 Feb 2021 16:19:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 02F4364EAD for ; Thu, 18 Feb 2021 16:19:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233433AbhBRQSa (ORCPT ); Thu, 18 Feb 2021 11:18:30 -0500 Received: from smtp-fw-4101.amazon.com ([72.21.198.25]:1919 "EHLO smtp-fw-4101.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232773AbhBRPxS (ORCPT ); Thu, 18 Feb 2021 10:53:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1613663598; x=1645199598; h=subject:to:cc:references:from:message-id:date: mime-version:in-reply-to:content-transfer-encoding; bh=JuVVnetNo4ApOUzoqXLCZptDvIITpAFCtFxvxgOKLc8=; b=mPbQN7mr0rdhPmCRoOt419juMWoeRQ++VFXDGukNiq0UliZUYvO7XzCU +hfpr30TVIxTPQWbwe2sPz6JQwoybj/Hw2bSXmj1fpQ4aI1+nnZaqmjeN LBtFXGDrj25g7GAt1CQWDqKpbGXzpNcEfRwixehoIst2LhQV4S/Latl0X s=; X-IronPort-AV: E=Sophos;i="5.81,187,1610409600"; d="scan'208";a="85047341" Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-2b-a7fdc47a.us-west-2.amazon.com) ([10.43.8.2]) by smtp-border-fw-out-4101.iad4.amazon.com with ESMTP; 18 Feb 2021 15:52:24 +0000 Received: from EX13D19EUB003.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan2.pdx.amazon.com [10.236.137.194]) by email-inbound-relay-2b-a7fdc47a.us-west-2.amazon.com (Postfix) with ESMTPS id DE879C0667; Thu, 18 Feb 2021 15:52:23 +0000 (UTC) Received: from 8c85908914bf.ant.amazon.com (10.43.162.239) by EX13D19EUB003.ant.amazon.com (10.43.166.69) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 18 Feb 2021 15:52:21 +0000 Subject: Re: ibv_req_notify_cq clarification To: Jason Gunthorpe CC: RDMA mailing list References: <20210218125339.GY4718@ziepe.ca> From: Gal Pressman Message-ID: <5287c059-3d8c-93f4-6be4-a6da07ccdb8a@amazon.com> Date: Thu, 18 Feb 2021 17:52:16 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: <20210218125339.GY4718@ziepe.ca> Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [10.43.162.239] X-ClientProxiedBy: EX13D13UWB001.ant.amazon.com (10.43.161.156) To EX13D19EUB003.ant.amazon.com (10.43.166.69) Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org On 18/02/2021 14:53, Jason Gunthorpe wrote: > On Thu, Feb 18, 2021 at 11:13:43AM +0200, Gal Pressman wrote: >> I'm a bit confused about the meaning of the ibv_req_notify_cq() verb: >> "Upon the addition of a new CQ entry (CQE) to cq, a completion event will be >> added to the completion channel associated with the CQ." >> >> What is considered a new CQE in this case? >> The next CQE from the user's perspective, i.e. any new CQE that wasn't consumed >> by the user's poll cq? >> Or any new CQE from the device's perspective? > > new CQE from the device perspective. > >> For example, if at the time of ibv_req_notify_cq() call the CQ has received 100 >> completions, but the user hasn't polled his CQ yet, when should he be notified? >> On the 101 completion or immediately (since there are completions waiting on the >> CQ)? > > 101 completion > > It is only meaningful to call it when the CQ is empty. Thanks, so there's an inherent race between the user's CQ poll and the next arm? Do you know what's the purpose of the consumer index in the arm doorbell that's implemented by many providers?