From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.7 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3E39C432C2 for ; Thu, 26 Sep 2019 08:18:00 +0000 (UTC) Received: from dpdk.org (dpdk.org [92.243.14.124]) by mail.kernel.org (Postfix) with ESMTP id 90062222C0 for ; Thu, 26 Sep 2019 08:18:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 90062222C0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B0AD71BED1; Thu, 26 Sep 2019 10:17:56 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id F008A235 for ; Thu, 26 Sep 2019 10:17:43 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Sep 2019 01:17:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,551,1559545200"; d="scan'208";a="390515394" Received: from dpdk-dipei.sh.intel.com ([10.67.110.224]) by fmsmga006.fm.intel.com with ESMTP; 26 Sep 2019 01:17:42 -0700 From: Andy Pei To: dev@dpdk.org Cc: rosen.xu@intel.com, tianfei.zhang@intel.com, xiaolong.ye@intel.com, ferruh.yigit@intel.com Date: Thu, 26 Sep 2019 16:07:27 +0800 Message-Id: <1569485262-457887-3-git-send-email-andy.pei@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569485262-457887-1-git-send-email-andy.pei@intel.com> References: <1568883774-92149-2-git-send-email-andy.pei@intel.com> <1569485262-457887-1-git-send-email-andy.pei@intel.com> Subject: [dpdk-dev] [PATCH v7 02/17] raw/ifpga/base: add irq support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Tianfei zhang Add irq support for ifpga FME global error, port error and uint unit. We implmented this feature by vfio interrupt mechanism. To build this feature, CONFIG_RTE_EAL_VFIO should be enabled. Signed-off-by: Tianfei zhang Signed-off-by: Andy Pei --- config/common_base | 2 +- config/common_linux | 6 +++ drivers/raw/ifpga/base/ifpga_feature_dev.c | 59 ++++++++++++++++++++++++++++++ drivers/raw/ifpga/base/ifpga_fme_error.c | 19 ++++++++++ drivers/raw/ifpga/base/ifpga_port.c | 18 +++++++++ drivers/raw/ifpga/base/ifpga_port_error.c | 19 ++++++++++ 6 files changed, 122 insertions(+), 1 deletion(-) diff --git a/config/common_base b/config/common_base index 8ef75c2..1bbe012 100644 --- a/config/common_base +++ b/config/common_base @@ -768,7 +768,7 @@ CONFIG_RTE_LIBRTE_PMD_DPAA2_QDMA_RAWDEV=n # # Compile PMD for Intel FPGA raw device # -CONFIG_RTE_LIBRTE_PMD_IFPGA_RAWDEV=y +CONFIG_RTE_LIBRTE_PMD_IFPGA_RAWDEV=n # # Compile PMD for Intel IOAT raw device diff --git a/config/common_linux b/config/common_linux index 6e25255..0dc2d82 100644 --- a/config/common_linux +++ b/config/common_linux @@ -63,3 +63,9 @@ CONFIG_RTE_LIBRTE_ENETC_PMD=y # HINIC PMD driver # CONFIG_RTE_LIBRTE_HINIC_PMD=y + +# +# Compile PMD for Intel FPGA raw device +# To compile, CONFIG_RTE_EAL_VFIO should be enabled. +# +CONFIG_RTE_LIBRTE_PMD_IFPGA_RAWDEV=y diff --git a/drivers/raw/ifpga/base/ifpga_feature_dev.c b/drivers/raw/ifpga/base/ifpga_feature_dev.c index 63c8bcc..0f852a7 100644 --- a/drivers/raw/ifpga/base/ifpga_feature_dev.c +++ b/drivers/raw/ifpga/base/ifpga_feature_dev.c @@ -3,6 +3,7 @@ */ #include +#include #include "ifpga_feature_dev.h" @@ -331,3 +332,61 @@ int port_hw_init(struct ifpga_port_hw *port) port_hw_uinit(port); return ret; } + +#define FPGA_MAX_MSIX_VEC_COUNT 128 +/* irq set buffer length for interrupt */ +#define MSIX_IRQ_SET_BUF_LEN (sizeof(struct vfio_irq_set) + \ + sizeof(int) * FPGA_MAX_MSIX_VEC_COUNT) + +/* only support msix for now*/ +static int vfio_msix_enable_block(s32 vfio_dev_fd, unsigned int vec_start, + unsigned int count, s32 *fds) +{ + char irq_set_buf[MSIX_IRQ_SET_BUF_LEN]; + struct vfio_irq_set *irq_set; + int len, ret; + int *fd_ptr; + + len = sizeof(irq_set_buf); + + irq_set = (struct vfio_irq_set *)irq_set_buf; + irq_set->argsz = len; + /* 0 < irq_set->count < FPGA_MAX_MSIX_VEC_COUNT */ + irq_set->count = count ? + (count > FPGA_MAX_MSIX_VEC_COUNT ? + FPGA_MAX_MSIX_VEC_COUNT : count) : 1; + irq_set->flags = VFIO_IRQ_SET_DATA_EVENTFD | + VFIO_IRQ_SET_ACTION_TRIGGER; + irq_set->index = VFIO_PCI_MSIX_IRQ_INDEX; + irq_set->start = vec_start; + + fd_ptr = (int *)&irq_set->data; + opae_memcpy(fd_ptr, fds, sizeof(int) * count); + + ret = ioctl(vfio_dev_fd, VFIO_DEVICE_SET_IRQS, irq_set); + if (ret) + printf("Error enabling MSI-X interrupts\n"); + + return ret; +} + +int fpga_msix_set_block(struct ifpga_feature *feature, unsigned int start, + unsigned int count, s32 *fds) +{ + struct feature_irq_ctx *ctx = feature->ctx; + unsigned int i; + int ret; + + if (start >= feature->ctx_num || start + count > feature->ctx_num) + return -EINVAL; + + /* assume that each feature has continuous vector space in msix*/ + ret = vfio_msix_enable_block(feature->vfio_dev_fd, + ctx[start].idx, count, fds); + if (!ret) { + for (i = 0; i < count; i++) + ctx[i].eventfd = fds[i]; + } + + return ret; +} diff --git a/drivers/raw/ifpga/base/ifpga_fme_error.c b/drivers/raw/ifpga/base/ifpga_fme_error.c index 3794564..2978c79 100644 --- a/drivers/raw/ifpga/base/ifpga_fme_error.c +++ b/drivers/raw/ifpga/base/ifpga_fme_error.c @@ -373,9 +373,28 @@ static int fme_global_error_set_prop(struct ifpga_feature *feature, return -ENOENT; } +static int fme_global_err_set_irq(struct ifpga_feature *feature, void *irq_set) +{ + struct fpga_fme_err_irq_set *err_irq_set = irq_set; + struct ifpga_fme_hw *fme; + int ret; + + fme = (struct ifpga_fme_hw *)feature->parent; + + if (!(fme->capability & FPGA_FME_CAP_ERR_IRQ)) + return -ENODEV; + + spinlock_lock(&fme->lock); + ret = fpga_msix_set_block(feature, 0, 1, &err_irq_set->evtfd); + spinlock_unlock(&fme->lock); + + return ret; +} + struct ifpga_feature_ops fme_global_err_ops = { .init = fme_global_error_init, .uinit = fme_global_error_uinit, .get_prop = fme_global_error_get_prop, .set_prop = fme_global_error_set_prop, + .set_irq = fme_global_err_set_irq, }; diff --git a/drivers/raw/ifpga/base/ifpga_port.c b/drivers/raw/ifpga/base/ifpga_port.c index 6c41164..c0aaf01 100644 --- a/drivers/raw/ifpga/base/ifpga_port.c +++ b/drivers/raw/ifpga/base/ifpga_port.c @@ -384,9 +384,27 @@ static void port_uint_uinit(struct ifpga_feature *feature) dev_info(NULL, "PORT UINT UInit.\n"); } +static int port_uint_set_irq(struct ifpga_feature *feature, void *irq_set) +{ + struct fpga_uafu_irq_set *uafu_irq_set = irq_set; + struct ifpga_port_hw *port = feature->parent; + int ret; + + if (!(port->capability & FPGA_PORT_CAP_UAFU_IRQ)) + return -ENODEV; + + spinlock_lock(&port->lock); + ret = fpga_msix_set_block(feature, uafu_irq_set->start, + uafu_irq_set->count, uafu_irq_set->evtfds); + spinlock_unlock(&port->lock); + + return ret; +} + struct ifpga_feature_ops ifpga_rawdev_port_uint_ops = { .init = port_uint_init, .uinit = port_uint_uinit, + .set_irq = port_uint_set_irq, }; static int port_afu_init(struct ifpga_feature *feature) diff --git a/drivers/raw/ifpga/base/ifpga_port_error.c b/drivers/raw/ifpga/base/ifpga_port_error.c index 138284e..189f762 100644 --- a/drivers/raw/ifpga/base/ifpga_port_error.c +++ b/drivers/raw/ifpga/base/ifpga_port_error.c @@ -136,9 +136,28 @@ static int port_error_set_prop(struct ifpga_feature *feature, return -ENOENT; } +static int port_error_set_irq(struct ifpga_feature *feature, void *irq_set) +{ + struct fpga_port_err_irq_set *err_irq_set = irq_set; + struct ifpga_port_hw *port; + int ret; + + port = feature->parent; + + if (!(port->capability & FPGA_PORT_CAP_ERR_IRQ)) + return -ENODEV; + + spinlock_lock(&port->lock); + ret = fpga_msix_set_block(feature, 0, 1, &err_irq_set->evtfd); + spinlock_unlock(&port->lock); + + return ret; +} + struct ifpga_feature_ops ifpga_rawdev_port_error_ops = { .init = port_error_init, .uinit = port_error_uinit, .get_prop = port_error_get_prop, .set_prop = port_error_set_prop, + .set_irq = port_error_set_irq, }; -- 1.8.3.1