From mboxrd@z Thu Jan 1 00:00:00 1970 From: Adrien Mazarguil Subject: Re: [RFC] Generic flow director/filtering/classification API Date: Thu, 21 Jul 2016 14:47:34 +0200 Message-ID: <20160721124734.GR7621@6wind.com> References: <20160705181646.GO7621@6wind.com> <6A0DE07E22DDAD4C9103DF62FEBC09090348E1A7@shsmsx102.ccr.corp.intel.com> <20160707102650.GU7621@6wind.com> <6A0DE07E22DDAD4C9103DF62FEBC090903492563@shsmsx102.ccr.corp.intel.com> <20160719131219.GK7621@6wind.com> <6A0DE07E22DDAD4C9103DF62FEBC090903492A93@shsmsx102.ccr.corp.intel.com> <20160720104115.GN7621@6wind.com> <6A0DE07E22DDAD4C9103DF62FEBC090903492F92@shsmsx102.ccr.corp.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: "dev@dpdk.org" , Thomas Monjalon , "Zhang, Helin" , "Wu, Jingjing" , Rasesh Mody , Ajit Khaparde , Rahul Lakkireddy , Jan Medala , John Daley , "Chen, Jing D" , "Ananyev, Konstantin" , Matej Vido , Alejandro Lucero , Sony Chacko , Jerin Jacob , "De Lara Guarch, Pablo" , Olga Shern To: "Lu, Wenzhuo" Return-path: Received: from mail-wm0-f50.google.com (mail-wm0-f50.google.com [74.125.82.50]) by dpdk.org (Postfix) with ESMTP id 604862BB9 for ; Thu, 21 Jul 2016 14:47:39 +0200 (CEST) Received: by mail-wm0-f50.google.com with SMTP id q128so20734755wma.1 for ; Thu, 21 Jul 2016 05:47:39 -0700 (PDT) Content-Disposition: inline In-Reply-To: <6A0DE07E22DDAD4C9103DF62FEBC090903492F92@shsmsx102.ccr.corp.intel.com> List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Wenzhuo, It seems that we agree on about everything now, just a few more comments below after snipping the now irrelevant parts. On Thu, Jul 21, 2016 at 03:18:11AM +0000, Lu, Wenzhuo wrote: [...] > > > > > Does it mean PMD should store and maintain all the rules? Why n= ot > > > > > let rte do > > > > that? I think if PMD maintain all the rules, it means every kind = of > > > > NIC should have a copy of code for the rules. But if rte do that, > > > > only one copy of code need to be maintained, right? > > > > > > > > I've considered having rules stored in a common format understood= at > > > > the RTE level and not specific to each PMD and decided that the > > > > opaque rte_flow pointer was a better choice for the following rea= sons: > > > > > > > > - Even though flow rules management is done in the control path, = processing > > > > must be as fast as possible. Letting PMDs store flow rules usin= g their own > > > > internal representation gives them the chance to achieve better > > > > performance. > > > Not quite understand. I think we're talking about maintain the rule= s by SW. I > > don=E2=80=99t think there's something need to be optimized according = to specific NICs. If > > we need to optimize the code, I think we need to consider the CPU, OS= ... and > > some common means. I'm wrong? > >=20 > > Perhaps we were talking about different things, here I was only expla= ining why > > rte_flow (the result of creating a flow rule) should be opaque and fu= lly managed > > by the PMD. More on the SW side of things below. > >=20 > > > > - An opaque context managed by PMDs would probably have to be sto= red > > > > somewhere as well anyway. > > > > > > > > - PMDs may not need to allocate/store anything at all if they exc= lusively > > > > rely on HW state for everything. In my opinion, the generic API= has enough > > > > constraints for this to work and maintain consistency between f= low > > > > rules. Note this is currently how most PMDs implement FDIR and = other > > > > filter types. > > > Yes, the rules are stored by HW. But considering stop/start the dev= ice, the > > rules in HW will lose. we have to store the rules by SW and re-progra= m them > > when restarting the device. > >=20 > > Assume a HW capable of keeping flow rules programmed even during a > > stop/start cycle (e.g. mlx4/mlx5 may be able to do it from DPDK point= of view), > > don't you think it is more efficient to standardize on this behavior = and let PMDs > > restore flow rules for HW that do not support it regardless of whethe= r it would > > be done by RTE or the application (SW)? > Didn=E2=80=99t know that. As some NICs have already had the ability to = keep the rules during a stop/start cycle, maybe it could be a trend :) Well yeah, if you are wondering about that, these PMDs do not have the sa= me definition for port stop/start as lower level PMDs like ixgbe and i40e. I= n the mlx4/mlx5 cases, most control path operations (queue creation, destruction and general management) end up performed by kernel drivers. Stopping a port does not really shut it down as the kernel still manages its own netdevice independently. [...] > > > > - The flow rules format described in this specification (pattern = / actions) > > > > will be used by applications directly, and will be free to arra= nge them in > > > > lists, trees or in any other way if they need to keep flow spec= ifications > > > > around for further processing. > > > Who will create the lists, trees or something else? According to pr= evious > > discussion, I think the APP will program the rules one by one. So if = APP organize > > the rules to lists, trees..., PMD doesn=E2=80=99t know that. > > > And you said " Given that the opaque rte_flow pointer associated wi= th a flow > > rule is to be stored by the application ". I'm lost here. > >=20 > > I guess that's because we're discussing two different things, flow ru= le > > specifications and flow rule objects. Let me sum it up: > >=20 > > - Flow rule specifications are the patterns/actions combinations prov= ided by > > applications to rte_flow_create(). Applications can store those as = needed > > and organize them as they wish (hash, tree, list). Neither PMDs nor= RTE > > will do it for them. > >=20 > > - Flow rule objects (struct rte_flow *) are generated when a flow rul= e is > > created. Applications must keep these around if they want to manipu= late > > them later (i.e. destroy or query existing rules). > Thanks for this clarification. So the specifications can be different w= ith objects, right? The specifications are what the APP wants, the object= s are what the APP really gets. As rte_flow_create can fail. Right? Yes, precisely. Apps are also free to keep specifications around even in = the event of a flow creation failure. I think a generic software fallback wil= l be provided at some point. [...] > > What we seem to not agree about is that you think RTE should be respo= nsible > > for restoring flow rules of devices that lose them when stopped. I th= ink doing so > > is unfair to devices for which it is not the case and not really nice= to applications, > > so my opinion is that the PMD is responsible for restoring flow rules= however it > > wants. It is free to use RTE helpers to keep their track, as long as = it's all managed > > internally. > What I think is RTE can store the flow rules and recreate them after re= starting, in the function like rte_dev_start, so APP knows nothing about = it. But according to the discussing above, I think the design doesn't sup= port it, right? Yes. Right now the design explictly states that PMDs are on their own regarding this (4.3 Behavior). While it could be modified, I really think= it would be less efficient for the reasons stated above. > RTE doesn't store the flow rules objects and event it stores them, ther= e's no way designed to re-program the objects. And also considering some = HW doesn't need to be re-programed. I think it's OK to let PMD maintain = the rules as the re-programing is a NIC specific requirement. Great to finally agree on this point. > > > > Thus from an application point of view, whatever happens when > > > > stopping and restarting a port should not matter. If a flow rule = was > > > > present before, it must still be present afterwards. If the PMD h= ad > > > > to destroy flow rules and re-create them, it does not actually ma= tter if they > > differ slightly at the HW level, as long as: > > > > > > > > - Existing opaque flow rule pointers (rte_flow) are still valid t= o the PMD > > > > and refer to the same rules. > > > > > > > > - The overall behavior of all rules is the same. > > > > > > > > The list of rules you think of (patterns / actions) is maintained= by > > > > applications (not RTE), and only if they need them. RTE would nee= dlessly > > duplicate this. > > > As said before, need more details to understand this. Maybe an exam= ple > > > is better :) > >=20 > > The generic format both RTE and applications might understand is the = one > > described in this API (struct rte_flow_pattern and struct rte_flow_ac= tions). > >=20 > > If we wanted RTE to maintain some sort of per-port state for flow rul= e > > specifications, it would have to be a copy of these structures arrang= ed somehow > > (list or something else). > >=20 > > If we consider that PMDs need to keep a context object associated to = a flow > > rule (the opaque struct rte_flow *), then RTE would most likely have = to store it > > along with the flow specification. > >=20 > > Such a list may not be useful to applications (list lookups take time= ), so they > > would implement their own redundant method. They might also require e= xtra > > room to attach some application context to flow rules. A generic list= cannot plan > > for it. > >=20 > > Applications know what they want to do with flow rules and are respon= sible for > > managing them efficiently with RTE out of the way. > >=20 > > I'm not sure if this answered your question, if not, please describe = a scenario > > where a RTE-managed list of flow rules would be mandatory. > Got your point and agree :) Thanks. --=20 Adrien Mazarguil 6WIND