DPDK-dev Archive on lore.kernel.org
 help / color / Atom feed
From: Vamsi Krishna Attunuru <vattunuru@marvell.com>
To: Ferruh Yigit <ferruh.yigit@intel.com>, "dev@dpdk.org" <dev@dpdk.org>
Cc: "olivier.matz@6wind.com" <olivier.matz@6wind.com>,
	"arybchenko@solarflare.com" <arybchenko@solarflare.com>,
	"Kiran Kumar Kokkilagadda" <kirankumark@marvell.com>
Subject: Re: [dpdk-dev] [EXT] Re: [PATCH v6 4/4] kernel/linux/kni: add IOVA support in kni module
Date: Fri, 12 Jul 2019 10:38:19 +0000
Message-ID: <CH2PR18MB3381BDB398E5BFEDE55E80B8A6F20@CH2PR18MB3381.namprd18.prod.outlook.com> (raw)
In-Reply-To: <98bf2103-f48f-4baa-0d4a-f03f9e538519@intel.com>




________________________________
From: Ferruh Yigit <ferruh.yigit@intel.com>
Sent: Thursday, July 11, 2019 10:00 PM
To: Vamsi Krishna Attunuru; dev@dpdk.org
Cc: olivier.matz@6wind.com; arybchenko@solarflare.com; Kiran Kumar Kokkilagadda
Subject: [EXT] Re: [PATCH v6 4/4] kernel/linux/kni: add IOVA support in kni module

External Email

----------------------------------------------------------------------
On 6/25/2019 4:57 AM, vattunuru@marvell.com wrote:
> From: Kiran Kumar K <kirankumark@marvell.com>
>
> Patch adds support for kernel module to work in IOVA = VA mode,
> the idea is to get physical address from iova address using
> iommu_iova_to_phys API and later use phys_to_virt API to
> convert the physical address to kernel virtual address.
>
> When compared with IOVA = PA mode, there is no performance
> drop with this approach.
>
> This approach does not work with the kernel versions less
> than 4.4.0 because of API compatibility issues.
>
> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
> Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>

<...>

> @@ -351,15 +354,56 @@ kni_ioctl_create(struct net *net, uint32_t ioctl_num,
>        strncpy(kni->name, dev_info.name, RTE_KNI_NAMESIZE);
>
>        /* Translate user space info into kernel space info */
> -     kni->tx_q = phys_to_virt(dev_info.tx_phys);
> -     kni->rx_q = phys_to_virt(dev_info.rx_phys);
> -     kni->alloc_q = phys_to_virt(dev_info.alloc_phys);
> -     kni->free_q = phys_to_virt(dev_info.free_phys);
> -
> -     kni->req_q = phys_to_virt(dev_info.req_phys);
> -     kni->resp_q = phys_to_virt(dev_info.resp_phys);
> -     kni->sync_va = dev_info.sync_va;
> -     kni->sync_kva = phys_to_virt(dev_info.sync_phys);
> +     if (dev_info.iova_mode) {
> +#if KERNEL_VERSION(4, 4, 0) > LINUX_VERSION_CODE

We have "kni/compat.h" to put the version checks, please use abstracted feature
checks only in the code.
From experience this goes ugly quickly with the addition to distro kernels and
their specific versioning, so better to hide these all from the source code.

And this version requirement needs to be documented in kni doc.

ack

> +             (void)pci;
> +             pr_err("Kernel version is not supported\n");

Can you please include 'iova_mode' condition into the message log, because this
kernel version is supported if user wants to use via 'iova_mode == 0' condition.

ack

> +             return -EINVAL;
> +#else
> +             pci = pci_get_device(dev_info.vendor_id,
> +                                  dev_info.device_id, NULL);
> +             while (pci) {
> +                     if ((pci->bus->number == dev_info.bus) &&
> +                         (PCI_SLOT(pci->devfn) == dev_info.devid) &&
> +                         (PCI_FUNC(pci->devfn) == dev_info.function)) {
> +                             domain = iommu_get_domain_for_dev(&pci->dev);
> +                             break;
> +                     }
> +                     pci = pci_get_device(dev_info.vendor_id,
> +                                          dev_info.device_id, pci);
> +             }

What if 'pci' is NULL here?
In kni it is not required to provide a device at all.

Ack, will add a NULL check.
other point is not clear to me, device info is absolutely required at least
for  IOVA=VA mode, since it requires to procure iommu domain details.
Any thoughts or ways to address this without device.?

<...>

> @@ -186,7 +202,10 @@ kni_fifo_trans_pa2va(struct kni_dev *kni,
>                        return;
>
>                for (i = 0; i < num_rx; i++) {
> -                     kva = pa2kva(kni->pa[i]);
> +                     if (likely(kni->iova_mode == 1))
> +                             kva = iova2kva(kni, kni->pa[i]);
> +                     else
> +                             kva = pa2kva(kni->pa[i]);

To reduce the churn, what about updating the 'pa2kva()' and put the
"(kni->iova_mode == 1)" check there? Does it help? (not only 'pa2kva()' but its
friends also, and if it makes more sense agree to rename the functions)

No, in VA mode, kni->pa[i] points to iova address, pa2kva() of iova address might
crash, hence the if..else check is added.

And btw, why 'likely' case is "kni->iova_mode == 1"?

no specific case other than branch predict, will remove this if it's really harmful to PA mode.

  reply index

Thread overview: 103+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-09-27 10:49 [PATCH] kni: add IOVA va support for kni Kiran Kumar
2018-09-27 10:58 ` Burakov, Anatoly
2018-10-02 17:05 ` Ferruh Yigit
2019-04-01 17:30   ` Jerin Jacob Kollanukkaran
2019-04-01 18:20     ` Ferruh Yigit
2019-04-01  9:51 ` [PATCH v2] " Kiran Kumar Kokkilagadda
2019-04-03 16:29   ` Ferruh Yigit
2019-04-04  5:03     ` [dpdk-dev] [EXT] " Kiran Kumar Kokkilagadda
2019-04-04 11:20       ` Ferruh Yigit
2019-04-04 13:29         ` Burakov, Anatoly
2019-04-04  9:57     ` Burakov, Anatoly
2019-04-04 11:21       ` Ferruh Yigit
2019-04-16  4:55   ` [dpdk-dev] [PATCH v3] " kirankumark
2019-04-19 10:38     ` Thomas Monjalon
2019-04-22  4:39     ` [dpdk-dev] [PATCH v4] " kirankumark
2019-04-22  6:15       ` [dpdk-dev] [PATCH v5] " kirankumark
2019-04-26  9:11         ` Burakov, Anatoly
2019-06-25  3:56         ` [dpdk-dev] [PATCH v6 0/4] add IOVA = VA support in KNI vattunuru
2019-06-25  3:56           ` [dpdk-dev] [PATCH v6 1/4] lib/mempool: skip populating mempool objs that falls on page boundaries vattunuru
2019-06-25  3:56           ` [dpdk-dev] [PATCH v6 2/4] lib/kni: add PCI related information vattunuru
2019-06-25 17:41             ` Stephen Hemminger
2019-06-26  3:48               ` [dpdk-dev] [EXT] " Vamsi Krishna Attunuru
2019-06-26 14:58                 ` Stephen Hemminger
2019-06-27  9:43                   ` Vamsi Krishna Attunuru
2019-07-11 16:22             ` [dpdk-dev] " Ferruh Yigit
2019-07-12 11:02               ` [dpdk-dev] [EXT] " Vamsi Krishna Attunuru
2019-07-12 11:11                 ` Ferruh Yigit
2019-06-25  3:56           ` [dpdk-dev] [PATCH v6 3/4] example/kni: add IOVA support for kni application vattunuru
2019-07-11 16:23             ` Ferruh Yigit
2019-06-25  3:57           ` [dpdk-dev] [PATCH v6 4/4] kernel/linux/kni: add IOVA support in kni module vattunuru
2019-07-11 16:30             ` Ferruh Yigit
2019-07-12 10:38               ` Vamsi Krishna Attunuru [this message]
2019-07-12 11:10                 ` [dpdk-dev] [EXT] " Ferruh Yigit
2019-07-12 12:27                   ` Vamsi Krishna Attunuru
2019-07-12 16:29                   ` Vamsi Krishna Attunuru
2019-07-15 11:26                     ` Ferruh Yigit
2019-07-15 13:06                       ` Vamsi Krishna Attunuru
2019-07-11 16:43             ` [dpdk-dev] " Stephen Hemminger
2019-06-25 10:00           ` [dpdk-dev] [PATCH v6 0/4] add IOVA = VA support in KNI Burakov, Anatoly
2019-06-25 11:15             ` Jerin Jacob Kollanukkaran
2019-06-25 11:30               ` Burakov, Anatoly
2019-06-25 13:38                 ` Burakov, Anatoly
2019-06-27  9:34                   ` Jerin Jacob Kollanukkaran
2019-07-01 13:51                     ` Vamsi Krishna Attunuru
2019-07-04  6:42                       ` Vamsi Krishna Attunuru
2019-07-04  9:48                         ` Jerin Jacob Kollanukkaran
2019-07-11 16:21                           ` Ferruh Yigit
2019-07-17  9:04           ` [dpdk-dev] [PATCH v7 0/4] kni: add IOVA=VA support vattunuru
2019-07-17  9:04             ` [dpdk-dev] [PATCH v7 1/4] mempool: modify mempool populate() to skip objects from page boundaries vattunuru
2019-07-17 13:36               ` Andrew Rybchenko
2019-07-17 13:47                 ` Olivier Matz
2019-07-17 17:31                 ` Vamsi Krishna Attunuru
2019-07-18  9:28                   ` Andrew Rybchenko
2019-07-18 14:16                     ` Vamsi Krishna Attunuru
2019-07-19 13:38                       ` [dpdk-dev] [RFC 0/4] mempool: avoid objects allocations across pages Olivier Matz
2019-07-19 13:38                         ` [dpdk-dev] [RFC 1/4] mempool: clarify default populate function Olivier Matz
2019-07-19 15:42                           ` Andrew Rybchenko
2019-07-19 13:38                         ` [dpdk-dev] [RFC 2/4] mempool: unalign size when calculating required mem amount Olivier Matz
2019-08-07 15:21                           ` [dpdk-dev] ***Spam*** " Andrew Rybchenko
2019-07-19 13:38                         ` [dpdk-dev] [RFC 3/4] mempool: introduce function to get mempool page size Olivier Matz
2019-08-07 15:21                           ` Andrew Rybchenko
2019-07-19 13:38                         ` [dpdk-dev] [RFC 4/4] mempool: prevent objects from being across pages Olivier Matz
2019-07-19 14:03                           ` Burakov, Anatoly
2019-07-19 14:11                           ` Burakov, Anatoly
2019-08-07 15:21                           ` Andrew Rybchenko
2019-07-23  5:37                         ` [dpdk-dev] [RFC 0/4] mempool: avoid objects allocations " Vamsi Krishna Attunuru
2019-08-07 15:21                         ` [dpdk-dev] ***Spam*** " Andrew Rybchenko
2019-07-17  9:04             ` [dpdk-dev] [PATCH v7 2/4] kni: add IOVA = VA support in KNI lib vattunuru
2019-07-17  9:04             ` [dpdk-dev] [PATCH v7 3/4] kni: add IOVA=VA support in KNI module vattunuru
2019-07-17  9:04             ` [dpdk-dev] [PATCH v7 4/4] kni: modify IOVA mode checks to support VA vattunuru
2019-07-23  5:38             ` [dpdk-dev] [PATCH v8 0/5] kni: add IOVA=VA support vattunuru
2019-07-23  5:38               ` [dpdk-dev] [PATCH v8 1/5] mempool: populate mempool with page sized chunks of memory vattunuru
2019-07-23 11:08                 ` Andrew Rybchenko
2019-07-23 12:28                   ` Vamsi Krishna Attunuru
2019-07-23 19:33                     ` Andrew Rybchenko
2019-07-24  7:09                       ` Vamsi Krishna Attunuru
2019-07-24  7:27                         ` Andrew Rybchenko
2019-07-29  6:25                           ` Vamsi Krishna Attunuru
2019-07-23  5:38               ` [dpdk-dev] [PATCH v8 2/5] add IOVA -VA support in KNI lib vattunuru
2019-07-23 10:54                 ` Andrew Rybchenko
2019-07-23  5:38               ` [dpdk-dev] [PATCH v8 3/5] kni: add app specific mempool create & free routine vattunuru
2019-07-23 10:50                 ` Andrew Rybchenko
2019-07-23 11:01                   ` Vamsi Krishna Attunuru
2019-07-23  5:38               ` [dpdk-dev] [PATCH v8 4/5] kni: add IOVA=VA support in KNI module vattunuru
2019-07-23  5:38               ` [dpdk-dev] [PATCH v8 5/5] kni: modify IOVA mode checks to support VA vattunuru
2019-07-24  7:14               ` [dpdk-dev] [PATCH v8 0/5] kni: add IOVA=VA support Vamsi Krishna Attunuru
2019-07-29 12:13               ` [dpdk-dev] [PATCH v9 " vattunuru
2019-07-29 12:13                 ` [dpdk-dev] [PATCH v9 1/5] mempool: populate mempool with the page sized chunks of memory vattunuru
2019-07-29 12:41                   ` Andrew Rybchenko
2019-07-29 13:33                     ` [dpdk-dev] [EXT] " Vamsi Krishna Attunuru
2019-08-16  6:12                   ` [dpdk-dev] [PATCH v10 0/5] kni: add IOVA=VA support vattunuru
2019-08-16  6:12                     ` [dpdk-dev] [PATCH v10 1/5] mempool: populate mempool with the page sized chunks vattunuru
2019-08-16  6:12                     ` [dpdk-dev] [PATCH v10 2/5] kni: add IOVA=VA support in KNI lib vattunuru
2019-08-16  6:12                     ` [dpdk-dev] [PATCH v10 3/5] kni: add app specific mempool create and free routines vattunuru
2019-08-16  6:12                     ` [dpdk-dev] [PATCH v10 4/5] kni: add IOVA=VA support in KNI module vattunuru
2019-08-16  6:12                     ` [dpdk-dev] [PATCH v10 5/5] kni: modify IOVA mode checks to support VA vattunuru
2019-07-29 12:13                 ` [dpdk-dev] [PATCH v9 2/5] kni: add IOVA=VA support in KNI lib vattunuru
2019-07-29 12:24                   ` Igor Ryzhov
2019-07-29 13:22                     ` [dpdk-dev] [EXT] " Vamsi Krishna Attunuru
2019-07-29 12:13                 ` [dpdk-dev] [PATCH v9 3/5] kni: add app specific mempool create & free routine vattunuru
2019-07-29 12:13                 ` [dpdk-dev] [PATCH v9 4/5] kni: add IOVA=VA support in KNI module vattunuru
2019-07-29 12:13                 ` [dpdk-dev] [PATCH v9 5/5] kni: modify IOVA mode checks to support VA vattunuru
2019-04-23  8:56       ` [dpdk-dev] [PATCH v4] kni: add IOVA va support for kni Burakov, Anatoly

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CH2PR18MB3381BDB398E5BFEDE55E80B8A6F20@CH2PR18MB3381.namprd18.prod.outlook.com \
    --to=vattunuru@marvell.com \
    --cc=arybchenko@solarflare.com \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=kirankumark@marvell.com \
    --cc=olivier.matz@6wind.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK-dev Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/dpdk-dev/0 dpdk-dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dpdk-dev dpdk-dev/ https://lore.kernel.org/dpdk-dev \
		dev@dpdk.org dpdk-dev@archiver.kernel.org
	public-inbox-index dpdk-dev


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/ public-inbox