RE: [PATCH 05/12] gen: add raw packet data API and tests

From: "Van Haaren, Harry" <harry.van.haaren@intel.com>
To: Thomas Monjalon <thomas@monjalon.net>,
	"Randles, Ronan" <ronan.randles@intel.com>
Cc: Jerin Jacob <jerinjacobk@gmail.com>,
	"dev@dpdk.org" <dev@dpdk.org>,
	"Richardson, Bruce" <bruce.richardson@intel.com>
Subject: RE: [PATCH 05/12] gen: add raw packet data API and tests
Date: Mon, 20 Dec 2021 10:21:28 +0000	[thread overview]
Message-ID: <BN0PR11MB571255B36BAEE7F5D24C7054D77B9@BN0PR11MB5712.namprd11.prod.outlook.com> (raw)
In-Reply-To: <6332947.G0QQBjFxQf@thomas>

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Friday, December 17, 2021 4:19 PM
> To: Randles, Ronan <ronan.randles@intel.com>; Van Haaren, Harry
> <harry.van.haaren@intel.com>
> Cc: Jerin Jacob <jerinjacobk@gmail.com>; dev@dpdk.org; Richardson, Bruce
> <bruce.richardson@intel.com>
> Subject: Re: [PATCH 05/12] gen: add raw packet data API and tests
> 
> 17/12/2021 12:40, Van Haaren, Harry:
> > I could ramble on a bit more, but mostly diminishing returns I think...
> > I'll just use this email as a reply to Thomas' tweet;
> > https://twitter.com/tmonjalo/status/1337313985662771201
> 
> My original question was to know available applications,
> not integrating such application in the DPDK repository.
> 
> I may me miss something obvious,
> but I don't understand why trying to add a user app inside DPDK repo.

There are likely a few points-of-view on this particular topic; and I'm glad
you mention it so we can discuss it clearly here.

There are two main parts to this patchset, the first is a packet generation library,
with an easy to use string-based syntax. The *library* is designed to be extended in
future to a range of "useful stuff" to do while generating packets. The packet generation
*application* should have minimal features, and focus on ease-of-use (as suggested below).

In order to test the DPDK code, we need a variety of unit tests, and a sample-application to show
users how to use the library (as well as docs etc). For me, the interesting part is that it is a small
step from a simple sample-app just for testing to a minimal tool for high-rate packet generation. 

I think many users of DPDK first install DPDK, then wish for a tool to generate high traffic rates
to test DPDK, and end up with a usability problem; DPDK does not include a usable packet generator.

To highlight this point; our own DPDK Docs simply ignore the requirement of packet-generation to
actually have packets processed by skeleton: http://doc.dpdk.org/guides/sample_app_ug/skeleton.html 
Our "quick start" on the website uses PCAP vdevs (avoiding the problem)  https://core.dpdk.org/doc/quick-start/
Even searching the entire docs for "generate packet" doesn't give any relevant/useful results:
http://doc.dpdk.org/guides/search.html?q=generate+packet&check_keywords=yes&area=default# 

Users could internet-search & find pktgen, moongen, trex, or similar tools. These tools are fantastic for experienced
developers such as devs on this mailing list - we should *NOT* replicate these complex tools in DPDK itself. However,
building any tool outside of DPDK repo requires more effort; another git-clone, another set of dependencies to install,
perhaps another build-system to get used to. Particularly for people starting out with DPDK (who are likely finding
it difficult to learn the various hugepage/PCI-binding etc), this is yet another problem to solve, or to give up.

So my proposal is as follows; let us add a simple DPDK traffic generator to DPDK. We can define its scope
and its intended use, limiting the scope and capabilities. As before, I do NOT think it a good idea to build a
complex and feature-rich packet generator. I do feel it useful to have an easy-to-use application in DPDK that
is particularly designed for generating specific packets, at specific line-rates, and reports mpps returned.

Thoughts on adding an small scope-limited application to DPDK enabling ease-of-packet-generation for new users?

Regards, -Harry