From: "Burakov, Anatoly" <anatoly.burakov@intel.com>
To: David Marchand <david.marchand@redhat.com>, dev@dpdk.org
Cc: benjamin.walker@intel.com, jerinj@marvell.com,
maxime.coquelin@redhat.com, thomas@monjalon.net
Subject: Re: [dpdk-dev] [PATCH v2 3/3] bus/pci: only consider usable devices to select IOVA mode
Date: Wed, 3 Jul 2019 11:45:36 +0100 [thread overview]
Message-ID: <4fabfd5f-2ba9-ff45-59dd-cfd01b8d49d5@intel.com> (raw)
In-Reply-To: <1560505157-9769-4-git-send-email-david.marchand@redhat.com>
On 14-Jun-19 10:39 AM, David Marchand wrote:
> From: Ben Walker <benjamin.walker@intel.com>
>
> When selecting the preferred IOVA mode of the pci bus, the current
> heuristic ("are devices bound?", "are devices bound to UIO?", "are pmd
> drivers supporting IOVA as VA?" etc..) should honor the device
> white/blacklist so that an unwanted device does not impact the decision.
>
> There is no reason to consider a device which has no driver available.
>
> This applies to all OS, so implements this in common code then call a
> OS specific callback.
>
> On Linux side:
> - the VFIO special considerations should be evaluated only if VFIO
> support is built,
> - there is no strong requirement on using VA rather than PA if a driver
> supports VA, so defaulting to DC in such a case.
>
> Signed-off-by: Ben Walker <benjamin.walker@intel.com>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
<snip>
> + const struct rte_pci_device *pdev)
> {
> - struct rte_pci_device *dev = NULL;
> - struct rte_pci_driver *drv = NULL;
> + enum rte_iova_mode iova_mode = RTE_IOVA_DC;
> + static int iommu_no_va = -1;
>
> - FOREACH_DRIVER_ON_PCIBUS(drv) {
> - FOREACH_DEVICE_ON_PCIBUS(dev) {
> - if (!rte_pci_match(drv, dev))
> - continue;
> - /*
> - * just one PCI device needs to be checked out because
> - * the IOMMU hardware is the same for all of them.
> - */
> - return pci_one_device_iommu_support_va(dev);
> + switch (pdev->kdrv) {
> + case RTE_KDRV_VFIO: {
> +#ifdef VFIO_PRESENT
> + static int is_vfio_noiommu_enabled = -1;
> +
> + if (is_vfio_noiommu_enabled == -1) {
> + if (rte_vfio_noiommu_is_enabled() == 1)
> + is_vfio_noiommu_enabled = 1;
> + else
> + is_vfio_noiommu_enabled = 0;
> + }
> + if ((pdrv->drv_flags & RTE_PCI_DRV_IOVA_AS_VA) == 0) {
> + iova_mode = RTE_IOVA_PA;
> + } else if (is_vfio_noiommu_enabled != 0) {
> + RTE_LOG(DEBUG, EAL, "Forcing to 'PA', vfio-noiommu mode configured\n");
> + iova_mode = RTE_IOVA_PA;
> }
> +#endif
> + break;
I'm not too well-versed in bus code, so please excuse my ignorance of
this codebase.
It seems that we would be ignoring drv_flags in case VFIO wasn't
compiled - if the driver has no RTE_PCI_DRV_IOVA_AS_VA flag, i'm pretty
sure we can set IOVA mode to PA without caring about VFIO at all. I
think it would be better to have something like this:
if ((pdrv->drv_flags & RTE_PCI_DRV_IOVA_AS_VA) == 0) {
iova_mode = RTE_IOVA_PA;
break; // early exit
}
#ifdef VFIO_PRESENT
static int is_vfio_noiommu_enabled = -1;
if (is_vfio_noiommu_enabled == -1) {
if (rte_vfio_noiommu_is_enabled() == 1)
is_vfio_noiommu_enabled = 1;
else
is_vfio_noiommu_enabled = 0;
}
if (is_vfio_noiommu_enabled != 0) {
iova_mode = RTE_IOVA_PA;
}
#endif
break;
In fact, could we not check if devices support both flags, and do an
early exit in case they don't?
Something like this, for example:
if ((pdrv->drv_flags & RTE_PCI_DRV_IOVA_AS_VA) == 0) {
return RTE_IOVA_PA; // early exit - device only wants PA
}
// device supports both PA and VA modes, so do more checks
switch (pdev->kdrv) {
...
}
Unless i'm missing something, this would look much simpler and easier to
understand.
--
Thanks,
Anatoly
next prev parent reply other threads:[~2019-07-03 10:45 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-05-30 17:48 [dpdk-dev] eal/pci: Improve automatic selection of IOVA mode Ben Walker
2019-05-30 17:48 ` [dpdk-dev] [PATCH 01/12] eal: Make rte_eal_using_phys_addrs work sooner Ben Walker
2019-05-30 21:29 ` [dpdk-dev] [PATCH v2 " Ben Walker
2019-05-30 21:29 ` [dpdk-dev] [PATCH v2 02/12] eal/pci: Inline several functions into rte_pci_get_iommu_class Ben Walker
2019-05-30 21:29 ` [dpdk-dev] [PATCH v2 03/12] eal/pci: Rework loops in rte_pci_get_iommu_class Ben Walker
2019-05-30 21:29 ` [dpdk-dev] [PATCH v2 04/12] eal/pci: Collapse two " Ben Walker
2019-05-30 21:29 ` [dpdk-dev] [PATCH v2 05/12] eal/pci: Add function pci_ignore_device Ben Walker
2019-05-30 21:29 ` [dpdk-dev] [PATCH v2 06/12] eal/pci: Correctly test whitelist/blacklist in rte_pci_get_iommu_class Ben Walker
2019-05-30 21:29 ` [dpdk-dev] [PATCH v2 07/12] eal/pci: Reverse if check " Ben Walker
2019-05-30 21:29 ` [dpdk-dev] [PATCH v2 08/12] eal/pci: Collapse loops " Ben Walker
2019-05-30 21:29 ` [dpdk-dev] [PATCH v2 09/12] eal/pci: Simplify rte_pci_get_iommu class by using a switch Ben Walker
2019-05-30 21:29 ` [dpdk-dev] [PATCH v2 10/12] eal/pci: Finding a device bound to UIO does not force PA Ben Walker
2019-05-30 21:29 ` [dpdk-dev] [PATCH v2 11/12] eal/pci: rte_pci_get_iommu_class handles no drivers Ben Walker
2019-05-30 21:29 ` [dpdk-dev] [PATCH v2 12/12] eal: If bus can't decide PA or VA, try to access PA Ben Walker
2019-05-30 17:48 ` [dpdk-dev] [PATCH 02/12] eal/pci: Inline several functions into rte_pci_get_iommu_class Ben Walker
2019-05-30 17:57 ` Stephen Hemminger
2019-05-30 18:09 ` Walker, Benjamin
2019-05-30 17:48 ` [dpdk-dev] [PATCH 03/12] eal/pci: Rework loops in rte_pci_get_iommu_class Ben Walker
2019-05-30 17:48 ` [dpdk-dev] [PATCH 04/12] eal/pci: Collapse two " Ben Walker
2019-05-30 17:48 ` [dpdk-dev] [PATCH 05/12] eal/pci: Add function pci_ignore_device Ben Walker
2019-05-30 17:48 ` [dpdk-dev] [PATCH 06/12] eal/pci: Correctly test whitelist/blacklist in rte_pci_get_iommu_class Ben Walker
2019-05-30 17:48 ` [dpdk-dev] [PATCH 07/12] eal/pci: Reverse if check " Ben Walker
2019-05-30 17:48 ` [dpdk-dev] [PATCH 08/12] eal/pci: Collapse loops " Ben Walker
2019-05-30 17:48 ` [dpdk-dev] [PATCH 09/12] eal/pci: Simplify rte_pci_get_iommu class by using a switch Ben Walker
2019-05-30 17:48 ` [dpdk-dev] [PATCH 10/12] eal/pci: Finding a device bound to UIO does not force PA Ben Walker
2019-05-30 17:48 ` [dpdk-dev] [PATCH 11/12] eal/pci: rte_pci_get_iommu_class handles no drivers Ben Walker
2019-05-30 17:48 ` [dpdk-dev] [PATCH 12/12] eal: If bus can't decide PA or VA, try to access PA Ben Walker
2019-06-03 10:48 ` [dpdk-dev] eal/pci: Improve automatic selection of IOVA mode David Marchand
2019-06-03 16:44 ` Walker, Benjamin
2019-06-14 8:42 ` David Marchand
2019-06-14 9:39 ` [dpdk-dev] [PATCH v2 0/3] " David Marchand
2019-06-14 9:39 ` [dpdk-dev] [PATCH v2 1/3] kni: refuse to initialise when IOVA is not PA David Marchand
2019-06-14 9:39 ` [dpdk-dev] [PATCH v2 2/3] eal: compute IOVA mode based on PA availability David Marchand
2019-07-03 10:17 ` Burakov, Anatoly
2019-07-04 7:13 ` David Marchand
2019-06-14 9:39 ` [dpdk-dev] [PATCH v2 3/3] bus/pci: only consider usable devices to select IOVA mode David Marchand
2019-07-03 10:45 ` Burakov, Anatoly [this message]
2019-07-04 9:18 ` David Marchand
2019-07-04 10:43 ` Burakov, Anatoly
2019-07-04 10:47 ` David Marchand
2019-07-04 17:14 ` Stephen Hemminger
2019-07-05 7:58 ` David Marchand
2019-07-05 16:27 ` Stephen Hemminger
2019-07-05 8:26 ` Thomas Monjalon
2019-06-27 17:05 ` [dpdk-dev] [PATCH v2 0/3] Improve automatic selection of " Thomas Monjalon
2019-07-02 14:18 ` Thomas Monjalon
2019-07-05 14:57 ` Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4fabfd5f-2ba9-ff45-59dd-cfd01b8d49d5@intel.com \
--to=anatoly.burakov@intel.com \
--cc=benjamin.walker@intel.com \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=jerinj@marvell.com \
--cc=maxime.coquelin@redhat.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).