From mboxrd@z Thu Jan 1 00:00:00 1970 From: Neil Horman Subject: Re: [RFC] Yet another option for DPDK options Date: Thu, 2 Jun 2016 06:41:06 -0400 Message-ID: <20160602104106.GA12923@hmsreliant.think-freely.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Yuanhan Liu , Thomas Monjalon , "dev@dpdk.org" , "Richardson, Bruce" , "Tan, Jianfeng" , Stephen Hemminger , Christian Ehrhardt , Panu Matilainen , Olivier Matz To: "Wiles, Keith" Return-path: Received: from smtp.tuxdriver.com (charlotte.tuxdriver.com [70.61.120.58]) by dpdk.org (Postfix) with ESMTP id 8E2C5379B for ; Thu, 2 Jun 2016 12:41:20 +0200 (CEST) Content-Disposition: inline In-Reply-To: List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Wed, Jun 01, 2016 at 03:00:11PM +0000, Wiles, Keith wrote: > Started from the link below, but did not want to highjack the thread. > http://dpdk.org/ml/archives/dev/2016-June/040021.html >=20 > I was thinking about this problem from a user perspective and command l= ine options are very difficult to manage specifically when you have a lar= ge number of options as we have in dpdk. I see all of these options as a = type of database of information for the DPDK and the application, because= the application command line options are also getting very complex as we= ll. >=20 > I have been looking at a number of different options here and the direc= tion I was thinking was using a file for the options and configurations w= ith the data in a clean format. It could have been a INI file or JSON or = XML, but they all seem to have some problems I do not like. The INI file = is too flat and I wanted a hierarchy in the data, the JSON data is simila= r and XML is just hard to read. I wanted to be able to manage multiple ap= plications and possible system the DPDK/app runs. The problem with the ab= ove formats is they are just data and not easy to make decisions about th= e system and applications at runtime. >=20 > If the =E2=80=9Cdatabase=E2=80=9D of information could be queried by th= e EAL, drivers and application then we do not need to try and create a co= mplex command line. It would be nice to execute a DPDK applications like = this: >=20 > ./some_dpdk_app =E2=80=93config-file dpdk-config-filename >=20 > The dpdk-config-filename could contain a lot of information and be able= to startup multiple different applications. The dpdk-config-file could a= lso include other config files to complete the configuration. The format = of the data in the config file needs to be readable, but allow the user t= o put in new options, needs to be hierarchical in nature and have some si= mple functions to execute if required. >=20 > The solution I was thinking is the file information is really just a fr= agment of a scripting language, which the DPDK application contains this = scripting language interpreter. I was looking at using Lua lua.org as the= scripting language interpreter it is small and easy to understand. Pytho= n and others are very big and require a lot of resources and Lua requires= very few system resources. Also I did not want to have to write a parser= (lex/yacc). The other nice feature of Lua is you can create a sandbox fo= r the code to run in and limit the type of system resources and APIs that= can be accessed by the application and configuration. Lua can be trimmed= down to a fairly small size and builds on just about any system or we ca= n just install Lua on the system without changes from a rpm or deb. >=20 > I use Lua in pktgen at this time and the interface between =E2=80=98C=E2= =80=99 and Lua is very simple and easy. Currently I include Lua in Pktgen= , but I could have just used a system library. >=20 > The data in the config file can be data statements along with some limi= ted code to make some data changes at run time without having to modify t= he real application. Here is a simple config file I wrote: Some of the op= tions do not make sense to be in the file at the same time, but wanted to= see all of the options. The mk_lcore_list() and mk_coremap() are just Lu= a functions we can preload to help convert the simple strings into real d= ata in this case tables of information. The application could be somethin= g like pktgen =3D { map =3D { =E2=80=A6 }, more_options =3D 1, } this all= ows the same file to possible contain many application configurations. Ne= eds a bit more work. >=20 > dpdk_default =3D { > lcore_mask =3D 0xFF00, > lcore_list =3D mk_lcore_list("0-7", 10, "14-16"), > coremap =3D mk_coremap("(0-7)@0,10,(14-16)@1"), > master_lcore =3D 1, > log_level =3D 7, > ranks =3D 4, > channels =3D 2, > memory =3D 512, > socket_mem =3D { 512, 512 }, > huge_dir =3D "/mnt/huge", > base_virtaddr =3D 0, > create_uio_dev =3D true, > vfio_intr =3D "legacy", > xen_dom0 =3D false, > proc_type =3D "auto", > pci_blacklist =3D { > "08:00.0", > "08:00.1", > "09:00.0", > "09:00.1", > "83:00.1", > "87:00.0", > "87:00.1", > "89:00.0", > "89:00.1" > }, > pci_whitelist =3D { > }, > vdev =3D { > eth_pcap0 =3D { iface =3D "eth2" }, > eth_pcap1 =3D { iface =3D "eth3" }, > }, > driver =3D { }, > syslog =3D true, > vmware_tsc_map =3D false, > file_prefix =3D "pg", > huge_unlink =3D true, > no_huge =3D false, > no_pci =3D false, > no_hpet =3D false, > no_shconf =3D false, > } >=20 > pktgen_data =3D { > map =3D { =E2=80=A6 }, > more-data =3D 1, > } >=20 > The EAL, driver, application, =E2=80=A6 would query an API to access th= e data and the application can change his options quickly without modifyi= ng the code. >=20 > Anyway comments are welcome. > =20 > Regards, > Keith I'm not sure why you're focusing no selecting a config file format at all= . Why not just focus on removing the argument parsing from the core rte_eal_ini= t code, instead passing in a configuration struct that is stored and queried per application. Leave the parsing of a config file and population of that c= onfig struct as an exercize to the application developer. That way a given application can use command line options, config files, or whatever metho= d they choose, which would be in keeping with traditional application design. For the purposes of the example apps, it would seem that either JSON, YAM= L, or the above Lua format would work just fine. Neil