From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5B1EC432C3 for ; Mon, 25 Nov 2019 09:33:37 +0000 (UTC) Received: from dpdk.org (dpdk.org [92.243.14.124]) by mail.kernel.org (Postfix) with ESMTP id 80C752071E for ; Mon, 25 Nov 2019 09:33:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 80C752071E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5FE264CA7; Mon, 25 Nov 2019 10:33:36 +0100 (CET) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id CD7E04C93 for ; Mon, 25 Nov 2019 10:33:33 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 25 Nov 2019 01:33:32 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.69,241,1571727600"; d="scan'208";a="358792449" Received: from silpixa00399752.ir.intel.com (HELO silpixa00399752.ger.corp.intel.com) ([10.237.223.78]) by orsmga004.jf.intel.com with ESMTP; 25 Nov 2019 01:33:26 -0800 From: Ferruh Yigit To: david.marchand@redhat.com Cc: ajit.khaparde@broadcom.com, alejandro.lucero@netronome.com, anatoly.burakov@intel.com, beilei.xing@intel.com, bruce.richardson@intel.com, dev@dpdk.org, hyonkim@cisco.com, igor.russkikh@aquantia.com, jerinj@marvell.com, jingjing.wu@intel.com, john.mcnamara@intel.com, johndale@cisco.com, kirankumark@marvell.com, konstantin.ananyev@intel.com, marko.kovacevic@intel.com, matan@mellanox.com, ndabilpuram@marvell.com, pavel.belous@aquantia.com, qi.z.zhang@intel.com, qiming.yang@intel.com, rmody@marvell.com, shahafs@mellanox.com, shshaikh@marvell.com, somnath.kotur@broadcom.com, thomas@monjalon.net, viacheslavo@mellanox.com, wenzhuo.lu@intel.com, xiao.w.wang@intel.com, yskoh@mellanox.com Date: Mon, 25 Nov 2019 09:33:13 +0000 Message-Id: <20191125093313.61700-1-ferruh.yigit@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <1563800213-29839-3-git-send-email-david.marchand@redhat.com> References: <1563800213-29839-3-git-send-email-david.marchand@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v4 2/4] eal: fix IOVA mode selection as VA for PCI drivers X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 7/22/2019 1:56 PM, David Marchand wrote: > The incriminated commit broke the use of RTE_PCI_DRV_IOVA_AS_VA which > was intended to mean "driver only supports VA" but had been understood > as "driver supports both PA and VA" by most net drivers and used to let > dpdk processes to run as non root (which do not have access to physical > addresses on recent kernels). > > The check on physical addresses actually closed the gap for those > drivers. We don't need to mark them with RTE_PCI_DRV_IOVA_AS_VA and this > flag can retain its intended meaning. > Document explicitly its meaning. > > We can check that a driver requirement wrt to IOVA mode is fulfilled > before trying to probe a device. > > Finally, document the heuristic used to select the IOVA mode and hope > that we won't break it again. > > Fixes: 703458e19c16 ("bus/pci: consider only usable devices for IOVA mode") > > Signed-off-by: David Marchand > Reviewed-by: Jerin Jacob > Tested-by: Jerin Jacob > Acked-by: Anatoly Burakov <...> > diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c > index d2af472..9794552 100644 > --- a/drivers/bus/pci/pci_common.c > +++ b/drivers/bus/pci/pci_common.c > @@ -169,8 +169,22 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) > * This needs to be before rte_pci_map_device(), as it enables to use > * driver flags for adjusting configuration. > */ > - if (!already_probed) > + if (!already_probed) { > + enum rte_iova_mode dev_iova_mode; > + enum rte_iova_mode iova_mode; > + > + dev_iova_mode = pci_device_iova_mode(dr, dev); > + iova_mode = rte_eal_iova_mode(); > + if (dev_iova_mode != RTE_IOVA_DC && > + dev_iova_mode != iova_mode) { > + RTE_LOG(ERR, EAL, " Expecting '%s' IOVA mode but current mode is '%s', not initializing\n", > + dev_iova_mode == RTE_IOVA_PA ? "PA" : "VA", > + iova_mode == RTE_IOVA_PA ? "PA" : "VA"); > + return -EINVAL; > + } > + OvS reported an error while hotplugging a device. It looks like DPDK application initialized as IOVA=VA, and the new device is bound to 'igb_uio' which forces it to PA, fails on above check. I would like to get your comment on the issue. For the OvS mode, hopefully binding the device to 'vfio-pci' can be a solution, but for the cases we don't have that option, can/should we force the DPDK to PA mode after initialization?