From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ariel Elior Subject: [PATCH net-next 0/6 v2] qed*: Doorbell overflow recovery Date: Mon, 22 Oct 2018 19:40:39 +0300 Message-ID: <20181022164045.25393-1-Ariel.Elior@cavium.com> Mime-Version: 1.0 Content-Type: text/plain Cc: , Ariel Elior , "Michal Kalderon" , Tomer Tayar To: Return-path: Received: from mail-eopbgr700078.outbound.protection.outlook.com ([40.107.70.78]:7022 "EHLO NAM04-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727943AbeJWBB7 (ORCPT ); Mon, 22 Oct 2018 21:01:59 -0400 Sender: netdev-owner@vger.kernel.org List-ID: Doorbell Overflow If sufficient CPU cores will send doorbells at a sufficiently high rate, they can cause an overflow in the doorbell queue block message fifo. When fill level reaches maximum, the device stops accepting all doorbells from that PF until a recovery procedure has taken place. Doorbell Overflow Recovery The recovery procedure basically means resending the last doorbell for every doorbelling entity. A doorbelling entity is anything which may send doorbells: L2 tx ring, rdma sq/rq/cq, light l2, vf l2 tx ring, spq, etc. This relies on the design assumption that all doorbells are aggregative, so last doorbell carries the information of all previous doorbells. APIs All doorbelling entities need to register with the mechanism before sending doorbells. The registration entails providing the doorbell address the entity would be using, and a virtual address where last doorbell data can be found. Typically fastpath structures already have this construct. Executing the recovery procedure Handling the attentions, iterating over all the registered entities and resending their doorbells, is all handled within qed core module. Relevance All doorbelling entities in all protocols need to register with the mechanism, via the new APIs. Technically this is quite simple (just call the API). Some protocol fastpath implementation may not have the doorbell data stored anywhere (compute it from scratch every time) and will have to add such a place. This is rare and is also better practice (save some cycles on the fastpath). Performance Penalty No performance penalty should incur as a result of this feature. If anything performance can improve by avoiding recalcualtion of doorbell data everytime doorbell is sent (in some flows). Add the database used to register doorbelling entities, and APIs for adding and deleting entries, and logic for traversing the database and doorbelling once on behalf of all entities. Please consider applying to net-next. Thanks, Ariel Signed-off-by: Ariel Elior Signed-off-by: Michal Kalderon Signed-off-by: Tomer Tayar v1->v2: ------- Add support for ARCH=i386 (definition of writeq) Ariel Elior (6): qed: Add doorbell overflow recovery mechanism qed: Use the doorbell overflow recovery mechanism in case of doorbell overflow qed: Register slowpath queue doorbell with doorbell overflow recovery mechanism qed: Register light L2 queues with doorbell overflow recovery mechanism qed: Expose the doorbell overflow recovery mechanism to the protocol drivers qede: Register l2 queues with doorbell overflow recovery mechanism drivers/net/ethernet/qlogic/qed/qed.h | 31 ++- drivers/net/ethernet/qlogic/qed/qed_dev.c | 334 ++++++++++++++++++++++++- drivers/net/ethernet/qlogic/qed/qed_dev_api.h | 28 +++ drivers/net/ethernet/qlogic/qed/qed_int.c | 152 +++++++++-- drivers/net/ethernet/qlogic/qed/qed_int.h | 10 + drivers/net/ethernet/qlogic/qed/qed_ll2.c | 30 ++- drivers/net/ethernet/qlogic/qed/qed_ll2.h | 1 + drivers/net/ethernet/qlogic/qed/qed_main.c | 66 ++++- drivers/net/ethernet/qlogic/qed/qed_reg_addr.h | 50 ++++ drivers/net/ethernet/qlogic/qed/qed_sp.h | 4 +- drivers/net/ethernet/qlogic/qed/qed_spq.c | 22 ++ drivers/net/ethernet/qlogic/qede/qede_main.c | 9 + include/linux/qed/qed_if.h | 42 ++++ 13 files changed, 744 insertions(+), 35 deletions(-) -- 1.8.3.1