From mboxrd@z Thu Jan  1 00:00:00 1970
From: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>
Subject: Re: [RFCv2] service core concept
Date: Wed, 7 Jun 2017 13:09:40 +0000
Message-ID: <2601191342CEEE43887BDE71AB9772583FB06860@IRSMSX109.ger.corp.intel.com>
References: <E923DB57A917B54B9182A2E928D00FA63BF2D169@IRSMSX102.ger.corp.intel.com>
 <2601191342CEEE43887BDE71AB9772583FB0525F@IRSMSX109.ger.corp.intel.com>
 <E923DB57A917B54B9182A2E928D00FA640C143CF@IRSMSX101.ger.corp.intel.com>
 <20170606145347.GA55760@bricha3-MOBL3.ger.corp.intel.com>
 <2601191342CEEE43887BDE71AB9772583FB06531@IRSMSX109.ger.corp.intel.com>
 <E923DB57A917B54B9182A2E928D00FA640C1A7FA@IRSMSX101.ger.corp.intel.com>
 <2601191342CEEE43887BDE71AB9772583FB06787@IRSMSX109.ger.corp.intel.com>
 <E923DB57A917B54B9182A2E928D00FA640C1ADD2@IRSMSX101.ger.corp.intel.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Cc: "dev@dpdk.org" <dev@dpdk.org>, Thomas Monjalon <thomas@monjalon.net>,
 Jerin Jacob <jerin.jacob@caviumnetworks.com>, "Wiles, Keith"
 <keith.wiles@intel.com>
To: "Van Haaren, Harry" <harry.van.haaren@intel.com>, "Richardson, Bruce"
 <bruce.richardson@intel.com>
Return-path: <dev-bounces@dpdk.org>
Received: from mga04.intel.com (mga04.intel.com [192.55.52.120])
 by dpdk.org (Postfix) with ESMTP id C4BB62BD8
 for <dev@dpdk.org>; Wed,  7 Jun 2017 15:09:44 +0200 (CEST)
In-Reply-To: <E923DB57A917B54B9182A2E928D00FA640C1ADD2@IRSMSX101.ger.corp.intel.com>
Content-Language: en-US
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>



> -----Original Message-----
> From: Van Haaren, Harry
> Sent: Wednesday, June 7, 2017 11:30 AM
> To: Ananyev, Konstantin <konstantin.ananyev@intel.com>; Richardson, Bruce=
 <bruce.richardson@intel.com>
> Cc: dev@dpdk.org; Thomas Monjalon <thomas@monjalon.net>; Jerin Jacob <jer=
in.jacob@caviumnetworks.com>; Wiles, Keith
> <keith.wiles@intel.com>
> Subject: RE: [dpdk-dev] [RFCv2] service core concept
>=20
>=20
>=20
> > -----Original Message-----
> > From: Ananyev, Konstantin
> > Sent: Wednesday, June 7, 2017 10:51 AM
> > To: Van Haaren, Harry <harry.van.haaren@intel.com>; Richardson, Bruce
> > <bruce.richardson@intel.com>
> > Cc: dev@dpdk.org; Thomas Monjalon <thomas@monjalon.net>; Jerin Jacob
> > <jerin.jacob@caviumnetworks.com>; Wiles, Keith <keith.wiles@intel.com>
> > Subject: RE: [dpdk-dev] [RFCv2] service core concept
> >
> >
> >
> > > -----Original Message-----
> > > From: Van Haaren, Harry
> > > Sent: Tuesday, June 6, 2017 4:41 PM
> > > To: Ananyev, Konstantin <konstantin.ananyev@intel.com>; Richardson, B=
ruce
> > <bruce.richardson@intel.com>
> > > Cc: dev@dpdk.org; Thomas Monjalon <thomas@monjalon.net>; Jerin Jacob
> > <jerin.jacob@caviumnetworks.com>; Wiles, Keith
> > > <keith.wiles@intel.com>
> > > Subject: RE: [dpdk-dev] [RFCv2] service core concept
> > >
> > > > From: Ananyev, Konstantin
> > > > Sent: Tuesday, June 6, 2017 4:29 PM
> > > > Subject: RE: [dpdk-dev] [RFCv2] service core concept
> > > >
> > > >
> > > > > From: Richardson, Bruce
> > > > > Sent: Tuesday, June 6, 2017 3:54 PM
> > > > >
> > > > > On Tue, Jun 06, 2017 at 11:25:57AM +0100, Van Haaren, Harry wrote=
:
> > > > > > > From: Ananyev, Konstantin
> > > > > > > Sent: Saturday, June 3, 2017 11:23 AM
> > > > > >
> > > > > > <snip>
> > >
> > > <snip other discussion>
> > >
> > > > > >
> > > > > > There are a number of options here, each with its own merit:
> > > > > >
> > > > > > A) Services/cores config in EAL
> > > > > > Benefit is that service functionality can be transparent to the=
 application. Negative
> > is
> > > > that the complexity is in EAL.
> > > > > >
> > > > > > B) Application configures services/cores
> > > > > > Benefit is no added EAL complexity. Negative is that applicatio=
n code has to configure
> > > > cores (duplicated per application).
> > > > > >
> > > > > >
> > > > > > To answer this question, I think we need to estimate how many a=
pplications would
> > benefit
> > > > from EAL integration and balance that against
> > > > > the "complexity cost" of doing so. I do like the simplicity of op=
tion (B), however if
> > there
> > > > is significant value in total transparency to the
> > > > > application I think (A) is the better choice.
> > > > > >
> > > > > >
> > > > > > Input on A) or B) welcomed! -Harry
> > > > >
> > > > > I'm definitely in favour of having it in EAL. The whole reason fo=
r doing
> > > > > this work is to make it easy for applications to dedicate cores t=
o
> > > > > background tasks - including applications written before this
> > > > > functionality was added. By merging this into EAL, we can have
> > > > > transparency in the app, as we can have the service cores complet=
ely in
> > > > > the background, and the app can call rte_eal_mp_remote_launch() e=
xactly
> > > > > as before, without unexpected failures. If we move this externall=
y, the
> > > > > app needs to be reworked to take account of that fact, and call n=
ew,
> > > > > service-core aware, launch functions instead.
> > > >
> > > > Not sure I understood you here:
> > > > If the app don' plan to use any cores for services, it for sure wil=
l be able to call
> > > > rte_eal_mp_remote_launch() as before (no services running case).
> > >
> > > Correct - EAL behavior remains unchanged if --service-cores=3D0xf is =
not passed
> > >
> > >
> > > > From other side, if the app would like to use services - it would n=
eed to specify
> > > > which service it wants to run, and for each service provide a corem=
ask, even if
> > > > EAL already allocates service cores for it.
> > >
> > > See next paragraph
> > >
> > >
> > > > Or are you talking about the when EAL allocates service cores, and =
then
> > > > PMDs themselves (or EAL again) register their services on that core=
s?
> > >
> > > EAL could provide sane default behavior. For example, round-robin ser=
vices over available
> > service-cores. Multithread-capable services can
> > > be registered on all service cores. Its not a perfect solution for al=
l service-to-core
> > mapping problems, but I'd guess about 80% of cases
> > > would be covered: using a single service with a single service core d=
edicated to it :)
> > >
> > >
> > > > That's probably possible, but how PMD would know which service core=
(s) it allowed to use?
> > >
> > > The PMD shouldn't be deciding - EAL for basic sanity config, or Appli=
cation for advanced
> > usage.
> > >
> > >
> > > > Things might get over-complicated here - in theory there could be m=
ultiple PMDs,
> > > > each of them can have more than one service, running on multiple se=
ts of cores, etc.
> > >
> > > True - the NxM service:core mapping possibility can be huge - the API=
 allows the application
> > the flexibility if that flexibility is really required.
> > > If the flexibility is not required, the round-robin 1:1 service:core =
EAL scheme should cover
> > it?
> >
> > Ok, so if I understand you right: by default EAL will allow each PMD to=
 register it's services
> > on all available service cores?
>=20
> Close, but I don't see the PMD being involved in core mapping. I think of=
 it like this:
>=20
> 1) A PMD registers its service (unaware of number of service cores availa=
ble)
> 2) EAL provides default core to service mappings
> 2.1) Application configures using API for advanced uses (optional)

Ok, thanks for explanation.
I am still not quite happy with the fact that EAL will have a dependency on=
 service lib,
but I see your point and indeed it might be usefull for many cases.
So wouldn't object here.
Konstantin

>=20
>=20
> Worked examples of EAL default core mapping with two services registered:
> - Eventdev SW PMD
> - Ethdev to eventdev RX
>=20
> Example A) With cores >=3D services, the services get one core assigned e=
ach:
> ./dpdk-app --service-cores=3D0x3
> eventdev_sw0            : lcore 0
> ethdev_to_eventdev_rx0  : lcore 1
>=20
> Example B) With more services than cores, the services share the availabl=
e cores:
> ./dpdk-app --service-cores=3D0x1
> eventdev_sw0            : lcore 0
> ethdev_to_eventdev_rx0  : lcore 0
>=20
>=20
> The EAL core-mapping logic round-robins services onto cores. If there are=
 more cores than services, they are not used (and a warning print
> given). If there are more services than cores, they are wrapped back to t=
he first core, and services share the core (example B).
>=20
> Keep in mind this is just for simple use-cases. For complex services to c=
ores mappings the application has the service API to configure it
> precisely as it wishes.