DPDK-dev Archive on lore.kernel.org
 help / color / Atom feed
From: Jerin Jacob Kollanukkaran <jerinj@marvell.com>
To: Ferruh Yigit <ferruh.yigit@intel.com>,
	Vamsi Krishna Attunuru <vattunuru@marvell.com>,
	"dev@dpdk.org" <dev@dpdk.org>
Cc: "olivier.matz@6wind.com" <olivier.matz@6wind.com>,
	"arybchenko@solarflare.com" <arybchenko@solarflare.com>,
	"Burakov, Anatoly" <anatoly.burakov@intel.com>
Subject: Re: [dpdk-dev] [PATCH v6 0/4] add IOVA = VA support in KNI
Date: Fri, 12 Jul 2019 09:17:41 +0000
Message-ID: <BYAPR18MB2424B4AB942F7684DF0916E2C8F20@BYAPR18MB2424.namprd18.prod.outlook.com> (raw)


> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@intel.com>
> Sent: Thursday, July 11, 2019 9:52 PM
> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Vamsi Krishna Attunuru
> <vattunuru@marvell.com>; dev@dpdk.org
> Cc: olivier.matz@6wind.com; arybchenko@solarflare.com; Burakov, Anatoly
> <anatoly.burakov@intel.com>
> Subject: [EXT] Re: [dpdk-dev] [PATCH v6 0/4] add IOVA = VA support in KNI
> 
> External Email
> 
> ----------------------------------------------------------------------
> On 7/4/2019 10:48 AM, Jerin Jacob Kollanukkaran wrote:
> >> From: Vamsi Krishna Attunuru
> >> Sent: Thursday, July 4, 2019 12:13 PM
> >> To: dev@dpdk.org
> >> Cc: ferruh.yigit@intel.com; olivier.matz@6wind.com;
> >> arybchenko@solarflare.com; Jerin Jacob Kollanukkaran
> >> <jerinj@marvell.com>; Burakov, Anatoly <anatoly.burakov@intel.com>
> >> Subject: Re: [dpdk-dev] [PATCH v6 0/4] add IOVA = VA support in KNI
> >>
> >> Hi All,
> >>
> >> Just to summarize, below items have arisen from the initial review.
> >> 1) Can the new mempool flag be made default to all the pools and will
> there be case that new flag functionality would fail  for some page sizes.?
> >
> > If the minimum huge page size is 2MB and normal huge page size is
> > 512MB or 1G. So I think, new flags can be default as skipping the page
> boundaries for Mempool objects has nearly zero overhead. But I leave
> decision to maintainers.
> >
> >> 2) Adding HW device info(pci dev info) to KNI device structure, will it
> break KNI on virtual devices in VA or PA mode.?
> >
> > Iommu_domain will be created only for PCI devices and the system runs
> > in IOVA_VA mode. Virtual devices(IOVA_DC(don't care) or IOVA_PA
> > devices still it works without PCI device structure)
> >
> > It is  a useful feature where KNI can run without root privilege and
> > it is pending for long time. Request to review and close this
> 
> I support the idea to remove 'kni' forcing to the IOVA=PA mode, but also not
> sure about forcing all KNI users to update their code to allocate mempool in a
> very specific way.
> 
> What about giving more control to the user on this?
> 
> Any user want to use IOVA=VA and KNI together can update application to
> justify memory allocation of the KNI and give an explicit "kni iova_mode=1"
> config.

Where this config comes, eal or kni sample app or KNI public API?


> Who want to use existing KNI implementation can continue to use it with
> IOVA=PA mode which is current case, or for this case user may need to force
> the DPDK application to IOVA=PA but at least there is a workaround.
> 
> And kni sample application should have sample for both case, although this
> increases the testing and maintenance cost, I hope we can get support from
> you on the iova_mode=1 usecase.
> 
> What do you think?

IMO, If possible we can avoid extra indirection of new config. In worst case
We can add it. How about following to not have new config

1) Make MEMPOOL_F_NO_PAGE_BOUND  as default
http://patches.dpdk.org/patch/55277/
There is absolutely zero overhead of this flag considering the huge page size are minimum
2MB. Typically 512MB or 1GB.
Any one has any objection?

2) Introduce rte_kni_mempool_create() API in kni lib to abstract the 
Mempool requirement for KNI. This will enable portable KNI applications.

Thoughts?

> 
> 
> 
> >
> >>
> >> Can someone suggest if any changes required to address above issues.
> > ________________________________________
> > From: dev <mailto:dev-bounces@dpdk.org> on behalf of Vamsi Krishna
> > Attunuru <mailto:vattunuru@marvell.com>
> > Sent: Monday, July 1, 2019 7:21:22 PM
> > To: Jerin Jacob Kollanukkaran; Burakov, Anatoly; mailto:dev@dpdk.org
> > Cc: mailto:ferruh.yigit@intel.com; mailto:olivier.matz@6wind.com;
> > mailto:arybchenko@solarflare.com
> > Subject: [EXT] Re: [dpdk-dev] [PATCH v6 0/4] add IOVA = VA support in
> > KNI
> >
> > External Email
> >
> > ----------------------------------------------------------------------
> > ping..
> >
> > ________________________________
> > From: Jerin Jacob Kollanukkaran
> > Sent: Thursday, June 27, 2019 3:04:58 PM
> > To: Burakov, Anatoly; Vamsi Krishna Attunuru; mailto:dev@dpdk.org
> > Cc: mailto:ferruh.yigit@intel.com; mailto:olivier.matz@6wind.com;
> > mailto:arybchenko@solarflare.com
> > Subject: RE: [dpdk-dev] [PATCH v6 0/4] add IOVA = VA support in KNI
> >
> >> -----Original Message-----
> >> From: Burakov, Anatoly <mailto:anatoly.burakov@intel.com>
> >> Sent: Tuesday, June 25, 2019 7:09 PM
> >> To: Jerin Jacob Kollanukkaran <mailto:jerinj@marvell.com>; Vamsi
> >> Krishna Attunuru <mailto:vattunuru@marvell.com>;
> mailto:dev@dpdk.org
> >> Cc: mailto:ferruh.yigit@intel.com; mailto:olivier.matz@6wind.com;
> >> mailto:arybchenko@solarflare.com
> >> Subject: Re: [dpdk-dev] [PATCH v6 0/4] add IOVA = VA support in KNI
> >>
> >> On 25-Jun-19 12:30 PM, Burakov, Anatoly wrote:
> >>> On 25-Jun-19 12:15 PM, Jerin Jacob Kollanukkaran wrote:
> >>>>> -----Original Message-----
> >>>>> From: dev <mailto:dev-bounces@dpdk.org> On Behalf Of Burakov,
> >>>>> Anatoly
> >>>>> Sent: Tuesday, June 25, 2019 3:30 PM
> >>>>> To: Vamsi Krishna Attunuru <mailto:vattunuru@marvell.com>;
> >>>>> mailto:dev@dpdk.org
> >>>>> Cc: mailto:ferruh.yigit@intel.com; mailto:olivier.matz@6wind.com;
> >>>>> mailto:arybchenko@solarflare.com
> >>>>> Subject: Re: [dpdk-dev] [PATCH v6 0/4] add IOVA = VA support in
> >>>>> KNI
> >>>>>
> >>>>> On 25-Jun-19 4:56 AM, mailto:vattunuru@marvell.com wrote:
> >>>>>> From: Vamsi Attunuru <mailto:vattunuru@marvell.com>
> >>>>>>
> >>>>>> ----
> >>>>>> V6 Changes:
> >>>>>> * Added new mempool flag to ensure mbuf memory is not scattered
> >>>>>> across page boundaries.
> >>>>>> * Added KNI kernel module required PCI device information.
> >>>>>> * Modified KNI example application to create mempool with new
> >>>>>> mempool flag.
> >>>>>>
> >>>>> Others can chime in, but my 2 cents: this reduces the usefulness
> >>>>> of KNI because it limits the kinds of mempools one can use them
> >>>>> with, and makes it so that the code that works with every other
> >>>>> PMD requires changes to work with KNI.
> >>>>
> >>>> # One option to make this flag as default only for packet
> >>>> mempool(not allow allocate on page boundary).
> >>>> In real world the overhead will be very minimal considering Huge
> >>>> page size is 1G or 512M # Enable this flag explicitly only IOVA =
> >>>> VA mode in library. Not need to expose to application # I don't
> >>>> think, there needs to be any PMD specific change to make KNI with
> >>>> IOVA = VA mode # No preference on flags to be passed by application
> vs in library.
> >>>> But IMO this change would be
> >>>> needed in mempool support KNI in IOVA = VA mode.
> >>>>
> >>>
> >>> I would be OK to just make it default behavior to not cross page
> >>> boundaries when allocating buffers. This would solve the problem for
> >>> KNI and for any other use case that would rely on PA-contiguous
> >>> buffers in face of IOVA as VA mode.
> >>>
> >>> We could also add a flag to explicitly allow page crossing without
> >>> also making mbufs IOVA-non-contiguous, but i'm not sure if there are
> >>> use cases that would benefit from this.
> >>
> >> On another thought, such a default would break 4K pages in case for
> >> packets bigger than page size (i.e. jumbo frames). Should we care?
> >
> > The hugepage size will not be 4K. Right?
> >
> > Olivier,
> >
> > As a maintainer any thoughts of exposing/not exposing the new mepool
> > flag to Skip the page boundaries?
> >
> > All,
> > Either option is fine, Asking for feedback to processed further?
> >


             reply index

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-12  9:17 Jerin Jacob Kollanukkaran [this message]
2019-07-12  9:58 ` Burakov, Anatoly
  -- strict thread matches above, loose matches on Subject: below --
2019-07-12 11:37 Jerin Jacob Kollanukkaran
2019-07-12 12:09 ` Burakov, Anatoly
2019-07-12 10:26 Jerin Jacob Kollanukkaran
2019-07-12 10:48 ` Burakov, Anatoly
2019-04-22  6:15 [dpdk-dev] [PATCH v5] kni: add IOVA va support for kni kirankumark
2019-06-25  3:56 ` [dpdk-dev] [PATCH v6 0/4] add IOVA = VA support in KNI vattunuru
2019-06-25 10:00   ` Burakov, Anatoly
2019-06-25 11:15     ` Jerin Jacob Kollanukkaran
2019-06-25 11:30       ` Burakov, Anatoly
2019-06-25 13:38         ` Burakov, Anatoly
2019-06-27  9:34           ` Jerin Jacob Kollanukkaran
2019-07-01 13:51             ` Vamsi Krishna Attunuru
2019-07-04  6:42               ` Vamsi Krishna Attunuru
2019-07-04  9:48                 ` Jerin Jacob Kollanukkaran
2019-07-11 16:21                   ` Ferruh Yigit

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=BYAPR18MB2424B4AB942F7684DF0916E2C8F20@BYAPR18MB2424.namprd18.prod.outlook.com \
    --to=jerinj@marvell.com \
    --cc=anatoly.burakov@intel.com \
    --cc=arybchenko@solarflare.com \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=olivier.matz@6wind.com \
    --cc=vattunuru@marvell.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK-dev Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/dpdk-dev/0 dpdk-dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dpdk-dev dpdk-dev/ https://lore.kernel.org/dpdk-dev \
		dev@dpdk.org
	public-inbox-index dpdk-dev

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git