From mboxrd@z Thu Jan  1 00:00:00 1970
From: "Van Haaren, Harry" <harry.van.haaren@intel.com>
Subject: Re: Service lcores and Application lcores
Date: Fri, 30 Jun 2017 10:00:18 +0000
Message-ID: <E923DB57A917B54B9182A2E928D00FA640C344F6@IRSMSX102.ger.corp.intel.com>
References: <E923DB57A917B54B9182A2E928D00FA640C33E88@IRSMSX102.ger.corp.intel.com>
 <20170629155707.GA15724@bricha3-MOBL3.ger.corp.intel.com>
 <20170630044508.GA3735@jerin>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Cc: "dev@dpdk.org" <dev@dpdk.org>, "thomas@monjalon.net"
 <thomas@monjalon.net>, "Wiles, Keith" <keith.wiles@intel.com>
To: Jerin Jacob <jerin.jacob@caviumnetworks.com>, "Richardson, Bruce"
 <bruce.richardson@intel.com>
Return-path: <dev-bounces@dpdk.org>
Received: from mga09.intel.com (mga09.intel.com [134.134.136.24])
 by dpdk.org (Postfix) with ESMTP id 91D0C2BE1
 for <dev@dpdk.org>; Fri, 30 Jun 2017 12:01:02 +0200 (CEST)
In-Reply-To: <20170630044508.GA3735@jerin>
Content-Language: en-US
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

> From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com]
> Sent: Friday, June 30, 2017 5:45 AM
> To: Richardson, Bruce <bruce.richardson@intel.com>
> Cc: Van Haaren, Harry <harry.van.haaren@intel.com>; dev@dpdk.org; thomas@=
monjalon.net;
> Wiles, Keith <keith.wiles@intel.com>
> Subject: Re: Service lcores and Application lcores
>=20
> -----Original Message-----
> > Date: Thu, 29 Jun 2017 16:57:08 +0100
> > From: Bruce Richardson <bruce.richardson@intel.com>
> > To: "Van Haaren, Harry" <harry.van.haaren@intel.com>
> > CC: "dev@dpdk.org" <dev@dpdk.org>, 'Jerin Jacob'
> >  <jerin.jacob@caviumnetworks.com>, "thomas@monjalon.net"
> >  <thomas@monjalon.net>, "Wiles, Keith" <keith.wiles@intel.com>
> > Subject: Re: Service lcores and Application lcores
> > User-Agent: Mutt/1.8.1 (2017-04-11)
> >
> > On Thu, Jun 29, 2017 at 03:36:04PM +0100, Van Haaren, Harry wrote:
> > > Hi All,

<snip>

> > > A proposal for Eventdev, to ensure Service lcores and Application lco=
res play nice;
> > >
> > > 1) Application lcores must not directly call rte_eventdev_schedule()
> > > 2A) Service cores are the proper method to run services
> > > 2B) If an application insists on running a service "manually" on an a=
pp lcore, we
> provide a function for that:
> > >      rte_service_run_from_app_lcore(struct service *srv);
> > >
> > > The above function would allow a pesky app to run services on its own=
 (non-service
> core) lcores, but
> > > does so through the service-core framework, allowing the service-libr=
ary atomic to
> keep access serialized as required for non-multi-thread-safe services.
> > >
> > > The above solution maintains the option of running the eventdev PMD a=
s now (single-
> core dedicated to a single service), while providing correct serializatio=
n by using the
> rte_service_run_from_app_lcore() function. Given the atomic is only used =
when required
> (multiple cores mapped to the service) there should be no performance del=
ta.
> > >
> > > Given that the application should not invoke rte_eventdev_schedule(),=
 we could even
> consider removing it from the Eventdev API. A PMD that requires cycles re=
gisters a
> service, and an application can use a service core or the run_from_app_lc=
ore() function if
> it wishes to invoke that service on an application owned lcore.
> > >
> > >
> > > Opinions?
> >
> > I would be in favour of this proposal, except for the proposed name for
> > the new function. It would be useful for an app to be able to "adopt" a
> > service into it's main loop if so desired. If we do this, I think I'd
>=20
> +1
>=20
> Agree with Harry and Bruce here.
>=20
> I think, The adapter function should take "struct service *" and return
> lcore_function_t so that it can run using exiting rte_eal_remote_launch()


I don't think providing a remote-launch API is actually beneficial. Remote-=
launching a single service
is equivalent to adding that lcore as a service-core, and mapping it to jus=
t that single service.
The advantage of adding it as a service core, is future-proofing for if mor=
e services need to be added
to that core in future, and statistics of the service core infrastructure. =
A convenience API could be
provided to perform the core_add(), service_start(), enable_on_service() an=
d core_start() APIs in one.

Also, the remote_launch API doesn't solve the original problem - what if an=
 application lcore wishes
to run one iteration of a service "manually". The remote_launch style API d=
oes not solve this problem.


Here a much simpler API to run a service... as a counter-proposal :)

/** Runs one iteration of *service* on the calling lcore */
int rte_service_iterate(struct rte_service_spec *service);


The iterate() function can check that the service is start()-ed, check the =
number of mapped-lcores and utilize the atomic to prevent concurrent access=
 to multi-thread unsafe services. By exposing the function-pointer/userdata=
 directly, we lose that.

Thinking about it, a function like rte_service_iterate() is the only functi=
onally correct approach. (Exposing the callback directly brings us back to =
the "application thread without atomic check" problem.)

Thoughts?


> > also support the removal of a dedicated schedule call from the eventdev
> > API, or alternatively, if it is needed by other PMDs, leave it as a
> > no-op in the sw PMD in favour of the service-cores managed function.
>=20
> I would be in favor of removing eventdev schedule and
> RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED capability so that it is completely
> transparent to application whether scheduler runs on HW or SW or "combina=
tion
> of both"


Yep this bit sounds good!