From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C4D2C32750 for ; Fri, 2 Aug 2019 10:05:45 +0000 (UTC) Received: from dpdk.org (dpdk.org [92.243.14.124]) by mail.kernel.org (Postfix) with ESMTP id E6B5C2067D for ; Fri, 2 Aug 2019 10:05:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E6B5C2067D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2D5CB1C225; Fri, 2 Aug 2019 12:05:44 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 01E921C223 for ; Fri, 2 Aug 2019 12:05:42 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 02 Aug 2019 03:05:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,337,1559545200"; d="scan'208";a="201613195" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by fmsmga002.fm.intel.com with ESMTP; 02 Aug 2019 03:05:41 -0700 Received: from fmsmsx119.amr.corp.intel.com (10.18.124.207) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.439.0; Fri, 2 Aug 2019 03:05:40 -0700 Received: from cdsmsx151.ccr.corp.intel.com (172.17.4.38) by FMSMSX119.amr.corp.intel.com (10.18.124.207) with Microsoft SMTP Server (TLS) id 14.3.439.0; Fri, 2 Aug 2019 03:05:39 -0700 Received: from cdsmsx104.ccr.corp.intel.com ([169.254.4.87]) by CDSMSX151.ccr.corp.intel.com ([169.254.3.78]) with mapi id 14.03.0439.000; Fri, 2 Aug 2019 18:05:38 +0800 From: "Zhang, Tianfei" To: Jerin Jacob Kollanukkaran , "Xu, Rosen" , "dev@dpdk.org" CC: "Yigit, Ferruh" , "Pei, Andy" , "Lomartire, David" , "Zhang, Qi Z" , "Ye, Xiaolong" Thread-Topic: [dpdk-dev] [PATCH v2 02/12] raw/ifpga_rawdev/base: add irq support Thread-Index: AQHVSNAoOOLkDgUT9UyUlcPfJoZlpabmtTyAgADiRKA= Date: Fri, 2 Aug 2019 10:05:38 +0000 Message-ID: References: <1564556752-19257-2-git-send-email-rosen.xu@intel.com> <1564708727-164887-1-git-send-email-rosen.xu@intel.com> <1564708727-164887-3-git-send-email-rosen.xu@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.17.6.105] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v2 02/12] raw/ifpga_rawdev/base: add irq support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Jerin Jacob Kollanukkaran [mailto:jerinj@marvell.com] > Sent: Friday, August 2, 2019 11:58 AM > To: Xu, Rosen ; dev@dpdk.org > Cc: Yigit, Ferruh ; Zhang, Tianfei > ; Pei, Andy ; Lomartire, > David ; Zhang, Qi Z ; Ye= , > Xiaolong > Subject: RE: [dpdk-dev] [PATCH v2 02/12] raw/ifpga_rawdev/base: add irq > support >=20 > > -----Original Message----- > > From: dev On Behalf Of Rosen Xu > > Sent: Friday, August 2, 2019 6:49 AM > > To: dev@dpdk.org > > Cc: ferruh.yigit@intel.com; tianfei.zhang@intel.com; > > rosen.xu@intel.com; andy.pei@intel.com; david.lomartire@intel.com; > > qi.z.zhang@intel.com; xiaolong.ye@intel.com > > Subject: [dpdk-dev] [PATCH v2 02/12] raw/ifpga_rawdev/base: add irq > > support > > > > From: Tianfei zhang > > > > Add irq support for ifpga FME globle error, port error and uint unit. > > We implmented this feature by vfio interrupt mechanism. > > > > Signed-off-by: Tianfei zhang > > --- > > drivers/raw/ifpga_rawdev/base/ifpga_feature_dev.c | 61 > > +++++++++++++++++++++++ > > drivers/raw/ifpga_rawdev/base/ifpga_fme_error.c | 22 ++++++++ > > drivers/raw/ifpga_rawdev/base/ifpga_port.c | 20 ++++++++ > > drivers/raw/ifpga_rawdev/base/ifpga_port_error.c | 21 ++++++++ > > 4 files changed, 124 insertions(+) > > > > diff --git a/drivers/raw/ifpga_rawdev/base/ifpga_feature_dev.c > > b/drivers/raw/ifpga_rawdev/base/ifpga_feature_dev.c > > index 63c8bcc..6b942e6 100644 > > --- a/drivers/raw/ifpga_rawdev/base/ifpga_feature_dev.c > > +++ b/drivers/raw/ifpga_rawdev/base/ifpga_feature_dev.c > > @@ -3,6 +3,7 @@ > > */ > > > > #include > > +#include > > > > #include "ifpga_feature_dev.h" > > > > @@ -331,3 +332,63 @@ int port_hw_init(struct ifpga_port_hw *port) > > port_hw_uinit(port); > > return ret; > > } > > + > > +/* > > + * FIXME: we should get msix vec count during pci enumeration instead > > +of > > + * below hardcode value. > > + */ > > +#define FPGA_MSIX_VEC_COUNT 20 > > +/* irq set buffer length for interrupt */ #define > > +MSIX_IRQ_SET_BUF_LEN (sizeof(struct vfio_irq_set) + \ > > + sizeof(int) * FPGA_MSIX_VEC_COUNT) > > + > > +/* only support msix for now*/ > > +static int vfio_msix_enable_block(s32 vfio_dev_fd, unsigned int vec_st= art, > > + unsigned int count, s32 *fds) >=20 > Isn't better to use generic EAL function for the same? In our PAC N3000 Card, we have 6 PCIe MSI-X vectors, for example: 0~3 for AFU 4 for Port 6 for FME FME (FPGA Management Engine ) will manage all resources in FPGA, like parti= tion reconfiguration, Power manager, thermal, Error reporting. Port is a bridge between FME and AFU. AFU is the accelerator unit which for customers logic. So, we reserve some MSI-X vectors for end-user/customers to use the AFU, an= d end-user/customers can use the AFU for networking acceleration or other acceleration. The DPDK existing API like rte_intr_enable()->vfio_enable_msix() will bind = all of the vectors at the same time and those vectors will register into on= e evenfd and one interrupt handler function. That cannot satisfy our design. we hope that, each MSI-X vector bind into V= FIO and register the interrupt handler function separately. Because the res= erve vectors like 0~3 vectors for AFU, we don't know what exact usage for the end-user/custom= ers in AFU logic, so it had better let them bind VFIO and register interrup= t handler themselves. One suggestion is we expand the vfio_enable_msix() function, let the caller= to specify the start vector and the numbers of vectors to bind the VFIO. static int vfio_enable_msix(const struct rte_intr_handle *intr_handle, int start, int = count) { ... irq_set->count =3D count; irq_set->start =3D start; ... return 0; }