From mboxrd@z Thu Jan 1 00:00:00 1970 From: Neil Horman Subject: Re: [RFC] Yet another option for DPDK options Date: Fri, 3 Jun 2016 15:18:04 -0400 Message-ID: <20160603191804.GE12627@hmsreliant.think-freely.org> References: <8CE01283-1E89-4302-BE7D-486975B43EF6@intel.com> <20160603174437.GC12627@hmsreliant.think-freely.org> <62A67FEB-AE18-43B1-8D15-27F23D5C8A7D@intel.com> <20160603183819.GD12627@hmsreliant.think-freely.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Arnon Warshavsky , Panu Matilainen , "Richardson, Bruce" , Thomas Monjalon , Yuanhan Liu , "dev@dpdk.org" , "Tan, Jianfeng" , Stephen Hemminger , Christian Ehrhardt , Olivier Matz To: "Wiles, Keith" Return-path: Received: from smtp.tuxdriver.com (charlotte.tuxdriver.com [70.61.120.58]) by dpdk.org (Postfix) with ESMTP id B7CA2568B for ; Fri, 3 Jun 2016 21:18:21 +0200 (CEST) Content-Disposition: inline In-Reply-To: List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Fri, Jun 03, 2016 at 07:07:50PM +0000, Wiles, Keith wrote: > On 6/3/16, 2:00 PM, "dev on behalf of Wiles, Keith" wrote: >=20 > >On 6/3/16, 1:52 PM, "Arnon Warshavsky" > wrote: > > > > > > > >On Fri, Jun 3, 2016 at 9:38 PM, Neil Horman > wrote: > >On Fri, Jun 03, 2016 at 06:29:13PM +0000, Wiles, Keith wrote: > >> > >> On 6/3/16, 12:44 PM, "Neil Horman" > wrote: > >> > >> >On Fri, Jun 03, 2016 at 04:04:14PM +0000, Wiles, Keith wrote: > >> >> Sorry, I deleted all of the text as it was getting a bit long. > >> >> > >> >> Here are my thoughts as of now, which is a combination of many su= ggestions I read from everyone=E2=80=99s emails. I hope this is not too h= ard to understand. > >> >> > >> >> - Break out the current command line options out of the DPDK comm= on code and move into a new lib. > >> >> - At this point I was thinking of keeping the rte_eal_init(args= , argv) API and just have it pass the args/argv to the new lib to create = the data storage. > >> >> - Maybe move the rte_eal_init() API to the new lib or keep i= t in the common eal code. Do not want to go hog wild. > >> >> - The rte_eal_init(args, argv) would then call to the new API r= te_eal_initialize(void), which in turn queries the data storage. (still t= hinking here) > >> >These three items seem to be the exact opposite of my suggestion. = The point of > >> >this change was to segregate the parsing of configuration away from= the > >> >initalization dpdk using that configurtion. By keeping rte_eal_ini= t in such a > >> >way that the command line is directly passed into it, you've not ch= anged that > >> >implicit binding to command line options. > >> > >> Neil, > >> > >> You maybe reading the above wrong or I wrote it wrong, which is a hi= gh possibility. I want to move the command line parsing out of DPDK an in= to a library, but I still believe I need to provide some backward compati= bility for ABI and to reduce the learning curve. The current applications= can still call the rte_eal_init(), which then calls the new lib parser f= or dpdk command line options and then calls rte_eal_initialize() or move = to the new API rte_eal_initialize() preceded by a new library call to par= se the old command line args. At some point we can deprecate the rte_eal_= init() if we think it is reasonable. > >> > >> > > >> >I can understand if you want to keep rte_eal_init as is for ABI pur= poses, but > >> >then you should create an rte_eal_init2(foo), where foo is some han= dle to in > >> >memory parsed configuration, so that applications can preform that = separation. > >> > >> I think you describe what I had planned here. The rte_eal_initialize= () routine is the new rte_eal_init2() API and the rte_eal_init() was only= for backward compatibility was my thinking. I figured the argument to rt= e_eal_initialize() would be something to be decided, but it will mostly l= ikely be some type of pointer to the storage. > >> > >> I hope that clears that up, but let me know. > >> > >yes, that clarifies your thinking, and I agree with it. Thank you! > >Neil > > > >> ++Keith > >> > >> > > >> >Neil > >> > > >> >> - The example apps args needs to be passed to the examples as i= s for now, then we can convert them one at a time if needed. > >> >> > >> >> - I would like to keep the storage of the data separate from the = file parser as they can use the =E2=80=98set=E2=80=99 routines to build t= he data storage up. > >> >> - Keeping them split allows for new parsers to be created, whil= e keeping the data storage from changing. > >> >> - The rte_cfg code could be modified to use the new configuration= if someone wants to take on that task =E2=98=BA > >> >> > >> >> - Next is the data storage and how we can access the data in a cl= ean simple way. > >> >> - I want to have some simple level of hierarchy in the data. > >> >> - Having a string containing at least two levels =E2=80=9Cprima= ry:secondary=E2=80=9D. > >> >> - Primary string is something like =E2=80=9CEAL=E2=80=9D or = =E2=80=9CPktgen=E2=80=9D or =E2=80=9Ctestpmd=E2=80=9D to divide the data = storage into logical major groups. > >> >> - The primary allows us to have groups and then we can ha= ve common secondary strings in different groups if needed. > >> >> - Secondary string can be whatever the developer of that gro= up would like e.g. simple =E2=80=9CEAL:foobar=E2=80=9D, two levels =E2=80= =9Ctestpmd:foo.bar=E2=80=9D > >> >> > >> >> - The secondary string is treated as a single string if it has = a hierarchy or not, but referencing a single value in the data storage. > >> >> - Key value pairs (KVP) or a hashmap data store. > >> >> - The key here is the whole string =E2=80=9CEAL:foobar=E2= =80=9D not just =E2=80=9Cfoobar=E2=80=9D secondary string. > >> >> - If we want to have the two split I am ok with that a= s well meaning the API would be: > >> >> rte_map_get(mapObj, =E2=80=9CEAL=E2=80=9D, =E2=80=9C= foo.bar=E2=80=9D); > >> >> rte_map_set(mapObj, =E2=80=9CEAL=E2=80=9D, =E2=80=9C= foo.bar=E2=80=9D, value); > >> >> - Have the primary as a different section in the data = store, would allow for dumping that section maybe easier, not sure. > >> >> - I am leaning toward > >> >> - Not going to try splitting up the string or parse it as it= is up to the developer to make it unique in the data store. > >> >> - Use a code design to make the strings simple to use without hav= ing typos be a problem. > >> >> - Not sure what the design is yet, but I do not want to have t= o concat two string or split strings in the code. > >> >> > >> >> This is as far as I have gotten and got tired of typing =E2=98=BA > >> >> > >> >> I hope this will satisfy most everyone=E2=80=99s needs for now. > >> >> > >> >> > >> >> Regards, > >> >> Keith > >> >> > >> >> > >> >> > >> > > >> > >> > >> > > > >Keith > >What about the data types of the values? > >I would assume that as a library it can provide the service of typed g= et/set and not leave conversion and validation to the app. > > > >rte_map_get_int(map,section,key) > >rte_map_get_double(...) > >rte_map_get_string(...) > >rte_map_get_bytes(...,destBuff , destBuffSize) //e.g byte array of RSS= key > >This may also allow some basic validity of the configuration file > >Another point I forgot about is default values. > >We sometimes use a notation where the app also specifies a default val= ue in case the configuration did not specify it > > rte_map_get_int(map,section,key , defaultValue ) > >and specify if this was a mandatory that has no default > > rte_map_get_int_crash_if_missing (map,section,key) > > > > > > > > > >/Arnon > > > >Arnon, > > > >Yes, I too was thinking about access type APIs, but had not come to a = full conclusion yet. As long as the API for get/put can return any value,= we can add a layer on top of these primary get/put APIs to do some basic= type checking. This way the developer can add his/her own type checking = APIs or we provide a couple basic types for simple values. >=20 > One more thing. I had not thought about default values as the defaults = are handle directly by the code when an option is not applied. I think it= should be left up to the developer to add default values to the storage = or handle it when an option is not found in the storage. >=20 > If I understand your code above the API would pass in a default value i= f one did not exist in the storage, which I guess is reasonable. Anyone t= hink this is a good idea or not? >=20 I'm not opposed to default values, but it seems to me that if we are spli= tting out a configuration storage library from dpdk, part of the initzliation o= f that library can be installing default values. That is to say, instead of hav= ing the code specific areas assume a default value if none is present in the conf= ig, an init function for the configuration storage library would just populate t= he keystore. That way all the dpdk itself has to do is a key lookup. Neil > > > >Does that make sense? > > > >++Keith > > >=20 >=20 >=20