DPDK-dev Archive on lore.kernel.org
 help / color / Atom feed
From: Jerin Jacob <jerinjacobk@gmail.com>
To: "Mattias Rönnblom" <mattias.ronnblom@ericsson.com>
Cc: David Marchand <david.marchand@redhat.com>,
	 Bruce Richardson <bruce.richardson@intel.com>,
	Jerin Jacob Kollanukkaran <jerinj@marvell.com>,
	 "dev@dpdk.org" <dev@dpdk.org>,
	Thomas Monjalon <thomas@monjalon.net>,
	 Ferruh Yigit <ferruh.yigit@intel.com>,
	Andrew Rybchenko <arybchenko@solarflare.com>,
	 Ajit Khaparde <ajit.khaparde@broadcom.com>,
	Qi Zhang <qi.z.zhang@intel.com>,
	 Xiaolong Ye <xiaolong.ye@intel.com>,
	Raslan Darawsheh <rasland@mellanox.com>,
	 Maxime Coquelin <maxime.coquelin@redhat.com>,
	Tiwei Bie <tiwei.bie@intel.com>,
	 Akhil Goyal <akhil.goyal@nxp.com>,
	Luca Boccassi <bluca@debian.org>,
	 Kevin Traynor <ktraynor@redhat.com>,
	"maintainers@dpdk.org" <maintainers@dpdk.org>,
	 John McNamara <john.mcnamara@intel.com>,
	Marko Kovacevic <marko.kovacevic@intel.com>,
	 Ray Kinsella <mdr@ashroe.eu>, Aaron Conole <aconole@redhat.com>,
	 Michael Santana <maicolgabriel@hotmail.com>,
	Harry van Haaren <harry.van.haaren@intel.com>,
	 Cristian Dumitrescu <cristian.dumitrescu@intel.com>,
	Phil Yang <phil.yang@arm.com>,  Joyce Kong <joyce.kong@arm.com>,
	Jan Viktorin <viktorin@rehivetech.com>,
	 Gavin Hu <gavin.hu@arm.com>,
	David Christensen <drc@linux.vnet.ibm.com>,
	 Konstantin Ananyev <konstantin.ananyev@intel.com>,
	Anatoly Burakov <anatoly.burakov@intel.com>,
	 Harini Ramakrishnan <harini.ramakrishnan@microsoft.com>,
	Omar Cardona <ocardona@microsoft.com>,
	 Anand Rawat <anand.rawat@intel.com>,
	Olivier Matz <olivier.matz@6wind.com>,
	 Gage Eads <gage.eads@intel.com>,
	Adrien Mazarguil <adrien.mazarguil@6wind.com>,
	Nicolas Chautru <nicolas.chautru@intel.com>,
	Declan Doherty <declan.doherty@intel.com>,
	 Fiona Trahe <fiona.trahe@intel.com>,
	Ashish Gupta <ashishg@marvell.com>,
	 Erik Gabriel Carrillo <erik.g.carrillo@intel.com>,
	Abhinandan Gujjar <abhinandan.gujjar@intel.com>,
	 Hemant Agrawal <hemant.agrawal@nxp.com>,
	"Artem V. Andreev" <artem.andreev@oktetlabs.ru>,
	 Nithin Kumar Dabilpuram <ndabilpuram@marvell.com>,
	Vamsi Krishna Attunuru <vattunuru@marvell.com>,
	 Rosen Xu <rosen.xu@intel.com>,
	Sachin Saxena <sachin.saxena@nxp.com>,
	 Stephen Hemminger <sthemmin@microsoft.com>,
	Chas Williams <chas3@att.com>,
	 "John W. Linville" <linville@tuxdriver.com>,
	Prasun Kapoor <pkapoor@marvell.com>,
	 Marcin Wojtas <mw@semihalf.com>,
	Michal Krawczyk <mk@semihalf.com>,
	Guy Tzalik <gtzalik@amazon.com>,
	 Evgeny Schemeilin <evgenys@amazon.com>,
	Igor Chauskin <igorch@amazon.com>,
	Ravi Kumar <ravi1.kumar@amd.com>,
	 Igor Russkikh <igor.russkikh@aquantia.com>,
	Pavel Belous <pavel.belous@aquantia.com>,
	 Shepard Siegel <shepard.siegel@atomicrules.com>,
	Ed Czeck <ed.czeck@atomicrules.com>,
	 John Miller <john.miller@atomicrules.com>,
	Somnath Kotur <somnath.kotur@broadcom.com>,
	 Maciej Czekaj <mczekaj@marvell.com>,
	Shijith Thotton <sthotton@marvell.com>,
	 Srisivasubramanian Srinivasan <srinivasan@marvell.com>,
	Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>,
	 John Daley <johndale@cisco.com>,
	Hyong Youb Kim <hyonkim@cisco.com>,
	 "Wei Hu (Xavier" <xavier.huwei@huawei.com>,
	"Min Hu (Connor" <humin29@huawei.com>,
	 Yisen Zhuang <yisen.zhuang@huawei.com>,
	Ziyang Xuan <xuanziyang2@huawei.com>,
	 Xiaoyun Wang <cloud.wangxiaoyun@huawei.com>,
	Guoyang Zhou <zhouguoyang@huawei.com>,
	 Beilei Xing <beilei.xing@intel.com>,
	Xiao Wang <xiao.w.wang@intel.com>,
	 Jingjing Wu <jingjing.wu@intel.com>,
	Wenzhuo Lu <wenzhuo.lu@intel.com>,
	 Qiming Yang <qiming.yang@intel.com>,
	Tomasz Duszynski <tdu@semihalf.com>,
	 Liron Himi <lironh@marvell.com>, Zyta Szpak <zr@semihalf.com>,
	 Kiran Kumar Kokkilagadda <kirankumark@marvell.com>,
	Matan Azrad <matan@mellanox.com>,
	Shahaf Shuler <shahafs@mellanox.com>,
	Viacheslav Ovsiienko <viacheslavo@mellanox.com>,
	 "K. Y. Srinivasan" <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>,
	Jan Remes <remes@netcope.com>,
	Heinrich Kuhn <heinrich.kuhn@netronome.com>,
	 Jan Gutter <jan.gutter@netronome.com>,
	Gagandeep Singh <g.singh@nxp.com>,
	 Rasesh Mody <rmody@marvell.com>,
	Shahed Shaikh <shshaikh@marvell.com>,
	 Yong Wang <yongwang@vmware.com>,
	Zhihong Wang <zhihong.wang@intel.com>,
	 Steven Webster <steven.webster@windriver.com>,
	Matt Peters <matt.peters@windriver.com>,
	 Keith Wiles <keith.wiles@intel.com>,
	Tetsuya Mukawa <mtetsuyah@gmail.com>,
	 Jasvinder Singh <jasvinder.singh@intel.com>,
	Jakub Grajciar <jgrajcia@cisco.com>,
	 Ruifeng Wang <ruifeng.wang@arm.com>,
	Anoob Joseph <anoobj@marvell.com>,
	 Fan Zhang <roy.fan.zhang@intel.com>,
	Pablo de Lara <pablo.de.lara.guarch@intel.com>,
	 John Griffin <john.griffin@intel.com>,
	Deepak Kumar Jain <deepak.k.jain@intel.com>,
	 Michael Shamis <michaelsh@marvell.com>,
	Nagadheeraj Rottela <rnagadheeraj@marvell.com>,
	 Srikanth Jampala <jsrikanth@marvell.com>,
	Ankur Dwivedi <adwivedi@marvell.com>,
	Jay Zhou <jianjay.zhou@huawei.com>, Lee Daly <lee.daly@intel.com>,
	 Sunila Sahu <ssahu@marvell.com>,
	Nipun Gupta <nipun.gupta@nxp.com>,
	Liang Ma <liang.j.ma@intel.com>,
	 Peter Mccarthy <peter.mccarthy@intel.com>,
	Tianfei zhang <tianfei.zhang@intel.com>,
	 Satha Koteswara Rao Kottidi <skoteshwar@marvell.com>,
	Xiaoyun Li <xiaoyun.li@intel.com>,
	 Bernard Iremonger <bernard.iremonger@intel.com>,
	 Vladimir Medvedkin <vladimir.medvedkin@intel.com>,
	David Hunt <david.hunt@intel.com>,
	 Reshma Pattan <reshma.pattan@intel.com>,
	Byron Marohn <byron.marohn@intel.com>,
	Sameh Gobriel <sameh.gobriel@intel.com>,
	Yipeng Wang <yipeng1.wang@intel.com>,
	 Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>,
	Robert Sanford <rsanford@akamai.com>,
	 Kevin Laatz <kevin.laatz@intel.com>,
	Maryam Tahhan <maryam.tahhan@intel.com>,
	 Ori Kam <orika@mellanox.com>,
	Radu Nicolau <radu.nicolau@intel.com>,
	 Tomasz Kantecki <tomasz.kantecki@intel.com>,
	Sunil Kumar Kori <skori@marvell.com>,
	 Pavan Nikhilesh Bhagavatula <pbhagavatula@marvell.com>,
	 Kirill Rybalchenko <kirill.rybalchenko@intel.com>,
	"Kadam, Pallavi" <pallavi.kadam@intel.com>,
	"dave@barachs.net" <dave@barachs.net>
Subject: Re: [dpdk-dev] [RFC] DPDK Trace support
Date: Mon, 17 Feb 2020 15:53:51 +0530
Message-ID: <CALBAE1MYiR2xA+aV-OaNF-5-==2c_==fSrwK+MR=dLySPT8mfQ@mail.gmail.com> (raw)
In-Reply-To: <8d6a08a0-caad-64b1-b618-66c89c40fa62@ericsson.com>

On Mon, Feb 17, 2020 at 3:05 PM Mattias Rönnblom
<mattias.ronnblom@ericsson.com> wrote:
>
> On 2020-02-15 11:21, Jerin Jacob wrote:
> > On Fri, Jan 17, 2020 at 4:24 PM Jerin Jacob <jerinjacobk@gmail.com> wrote:
> >> On Fri, Jan 17, 2020 at 4:00 PM Mattias Rönnblom
> >> <mattias.ronnblom@ericsson.com> wrote:
> >>>> LTTng kernel tracing only needs kmod support.
> >>>> For the userspace tracing at minium following libraries are required.
> >>>>
> >>>> a) LTTng-UST
> >>>> b) LTTng-tools
> >>>> c) liburcu
> >>>> d) libpopt-dev
> >>> This "DPDK CTF trace emitter" would make DPDK interoperate with, but
> >>> without any build-time dependencies to, LTTng. Correct?
> >> Yes. Native CTF trace emitter without LTTng dependency.
> >>
> >>> Do you have any idea of what the performance benefits one would receive
> >>> from having something DPDK native, compared to just depending on LTTng UST?
> >> I calibrated LTTng cost and pushed the test code to github[1].
> >>
> >> I just started working on the DPDK native CTF emitter.
> >> I am sure it will be less than LTTng as we can leverage hugepage, exploit
> >> dpdk worker thread usage to avoid atomics and use per core variables,
> >> avoid a lot function pointers in fast-path etc
> >> I can share the exact overhead after the PoC.
> > I did the almost feature-complete PoC. The code shared in Github[1]
> > The documentation and code cleanup etc is still pending.
> >
> > [1]
> > https://protect2.fireeye.com/v1/url?k=74b2fae0-283b556d-74b2ba7b-0cc47ad93de2-2bd7c54f29450187&q=1&e=2ef0a614-dea6-4d17-ab4e-a79a7c17ac73&u=https%3A%2F%2Fgithub.com%2Fjerinjacobk%2Fdpdk-trace.git
> >
> > trace overhead data on x86:[2]
> > # 236 cyles with LTTng(>100ns)
> > # 18 cycles(7ns) with Native DPDK CTF emitter.
> >
> > trace overhead data on arm64:
> > #  312  cycles to  1100 cycles with LTTng based on the class of arm64 CPU.
> > #  11 cycles to 13 cycles with Native DPDK CTF emitter based on the
> > class of arm64 CPU.
> >
> > 18 cycles(on x86) vs 11 cycles(on arm64) is due to rdtsc() overhead in
> > x86. It seems  rdtsc takes around 15cycles in x86.
> >
> > # The Native DPDK CTF trace support does not have any dependency on
> > third-party library.
> > The generated output file is compatible with LTTng as both are using
> > CTF trace format.
> >
> > The performance gain comes from:
> > 1) exploit dpdk worker thread usage model to avoid atomics and use per
> > core variables
> > 2) use hugepage,
> > 3) avoid a lot function pointers in fast-path etc
> > 4) avoid unaligned store for arm64 etc
> >
> > Features:
> > - APIs and Features are similar to rte_log dynamic framework
> > API(expect log prints on stdout vs it dumps on trace file)
> > - No specific limit on the events. A string-based event like rte_log
> > for pattern matching
> > - Dynamic enable/disable support.
> > - Instructmention overhead is ~1 cycle. i.e cost of adding the code
> > wth out using trace feature.
> > - Timestamp support for all the events using DPDK rte_rtdsc
> > - No dependency on another library. Cleanroom native implementation of CTF.
> >
> > Functional test case:
> > a) echo "trace_autotest" | sudo ./build/app/test/dpdk-test  -c 0x3
> > --trace-level=8
> >
> > The above command emits the following trace events
> > <code>
> >          uint8_t i;
> >
> >          rte_trace_lib_eal_generic_void();
> >          rte_trace_lib_eal_generic_u64(0x10000000000000);
> >          rte_trace_lib_eal_generic_u32(0x10000000);
> >          rte_trace_lib_eal_generic_u16(0xffee);
> >          rte_trace_lib_eal_generic_u8(0xc);
> >          rte_trace_lib_eal_generic_i64(-1234);
> >          rte_trace_lib_eal_generic_i32(-1234567);
> >          rte_trace_lib_eal_generic_i16(12);
> >          rte_trace_lib_eal_generic_i8(-3);
> >          rte_trace_lib_eal_generic_string("my string");
> >          rte_trace_lib_eal_generic_function(__func__);
> >
> >          for (i = 0; i < 128; i++)
> >                  rte_trace_lib_eal_generic_u8(i);
> > </code>
>
> Is it possible to specify custom types for the events? The equivalent of
> the TRACEPOINT_EVENT() macro in LTTng.

Yes. It is possble to specify the custom event using array of
rte_trace_emit_datatype of rte_trace_emit_string basic blocks.
For example, ethdev configure event would be like.

RTE_TRACE_POINT_DECLARE(__rte_trace_lib_ethdev_configure);

static __rte_always_inline void
rte_trace_lib_ethdev_configure(uint16_t port_id, uint16_t nb_rx_q,
uint16_t nb_tx_q,
                      const struct rte_eth_conf *dev_conf)
{
        rte_trace_emit_begin(&__rte_trace_lib_ethdev_configure);
        rte_trace_emit_datatype(mem, port_id);
        rte_trace_emit_datatype(mem, nb_rx_q);
        rte_trace_emit_datatype(mem, nb_tx_q);
        rte_trace_emit_datatype(mem, dev_conf->link_speeds);
        ..
        ..
}

I tried  avoid usage of complex macro as DPDK community prefers non
macro solutions.

>
> > Install babeltrace package in Linux and point the generated trace file
> > to babel trace. By default trace file created under
> > <user>/dpdk-traces/time_stamp/
> >
> > example:
> > # babeltrace /root/dpdk-traces/rte-2020-02-15-PM-02-56-51 | more
> >
> > [13:27:36.138468807] (+?.?????????) lib.eal.generic.void: { cpu_id =
> > 0, name = "dpdk-test" }, { }
> > [13:27:36.138468851] (+0.000000044) lib.eal.generic.u64: { cpu_id = 0,
> > name = "dpdk-test" }, { in = 4503599627370496 }
> > [13:27:36.138468860] (+0.000000009) lib.eal.generic.u32: { cpu_id = 0,
> > name = "dpdk-test" }, { in = 268435456 }
> > [13:27:36.138468934] (+0.000000074) lib.eal.generic.u16: { cpu_id = 0,
> > name = "dpdk-test" }, { in = 65518 }
> > [13:27:36.138468949] (+0.000000015) lib.eal.generic.u8: { cpu_id = 0,
> > name = "dpdk-test" }, { in = 12 }
> > [13:27:36.138468956] (+0.000000007) lib.eal.generic.i64: { cpu_id = 0,
> > name = "dpdk-test" }, { in = -1234 }
> > [13:27:36.138468963] (+0.000000007) lib.eal.generic.i32: { cpu_id = 0,
> > name = "dpdk-test" }, { in = -1234567 }
> > [13:27:36.138469024] (+0.000000061) lib.eal.generic.i16: { cpu_id = 0,
> > name = "dpdk-test" }, { in = 12 }
> > [13:27:36.138469044] (+0.000000020) lib.eal.generic.i8: { cpu_id = 0,
> > name = "dpdk-test" }, { in = -3 }
> > [13:27:36.138469051] (+0.000000007) lib.eal.generic.string: { cpu_id =
> > 0, name = "dpdk-test" }, { str = "my string" }
> > [13:27:36.138469203] (+0.000000152) lib.eal.generic.func: { cpu_id =
> > 0, name = "dpdk-test" }, { func = "test_trace_points" }
> > [13:27:36.138469239] (+0.000000036) lib.eal.generic.u8: { cpu_id = 0,
> > name = "dpdk-test" }, { in = 0 }
> > [13:27:36.138469246] (+0.000000007) lib.eal.generic.u8: { cpu_id = 0,
> > name = "dpdk-test" }, { in = 1 }
> > [13:27:36.138469252] (+0.000000006) lib.eal.generic.u8: { cpu_id = 0,
> > name = "dpdk-test" }, { in = 2 }
> > [13:27:36.138469262] (+0.000000010) lib.eal.generic.u8: { cpu_id = 0,
> > name = "dpdk-test" }, { in = 3 }
> > [13:27:36.138469269] (+0.000000007) lib.eal.generic.u8: { cpu_id = 0,
> > name = "dpdk-test" }, { in = 4 }
> > [13:27:36.138469276] (+0.000000007) lib.eal.generic.u8: { cpu_id = 0,
> > name = "dpdk-test" }, { in = 5 }
> >
> > # There is a  GUI based trace viewer available in Windows, Linux and Mac.
> > It is called as tracecompass.(https://www.eclipse.org/tracecompass/)
> >
> > The example screenshot and Histogram of above DPDK trace using Tracecompass.
> >
> > https://protect2.fireeye.com/v1/url?k=9cad9dc9-c0243244-9caddd52-0cc47ad93de2-306d4b4409146643&q=1&e=2ef0a614-dea6-4d17-ab4e-a79a7c17ac73&u=https%3A%2F%2Fgithub.com%2Fjerinjacobk%2Fshare%2Fblob%2Fmaster%2Fdpdk_trace.JPG
> >
> >
> > [2] Added performance test case to find the Trace overhead.
> > Command to test trace overhead. It is the overhead of writing
> > zero-argument trace.
> >
> > echo "trace_perf" | sudo ./build/app/test/dpdk-test  -c 0x3 --trace-level=8
> > EAL: Detected 56 lcore(s)
> > EAL: Detected 2 NUMA nodes
> > EAL: Trace dir: /root/dpdk-traces/rte-2020-02-15-PM-03-37-33
> > RTE>>trace_perf
> > Timer running at 2600.00MHz
> > ZERO_ARG: cycles=17.901031 ns=6.885012
> > Test OK
> >
> > [3] The above test is ported to LTTng for finding the LTTng trace
> > overhead. It available at
> > https://protect2.fireeye.com/v1/url?k=7eb42ff5-223d8078-7eb46f6e-0cc47ad93de2-e41c22a09211c207&q=1&e=2ef0a614-dea6-4d17-ab4e-a79a7c17ac73&u=https%3A%2F%2Fgithub.com%2Fjerinjacobk%2Flttng-overhead
> > https://protect2.fireeye.com/v1/url?k=616a430a-3de3ec87-616a0391-0cc47ad93de2-c75160108b40b11b&q=1&e=2ef0a614-dea6-4d17-ab4e-a79a7c17ac73&u=https%3A%2F%2Fgithub.com%2Fjerinjacobk%2Flttng-overhead%2Fblob%2Fmaster%2FREADME
> >
> > File walkthrough:
> >
> > lib/librte_eal/common/include/rte_trace.h - Public API for Trace
> > provider and Trace control
> > lib/librte_eal/common/eal_common_trace.c - main trace implemention
> > lib/librte_eal/common/eal_common_trace_ctf.c - CTF metadata spec implementation
> > lib/librte_eal/common/eal_common_trace_utils.c - command line utils
> > and filesystem operations.
> > lib/librte_eal/common/eal_common_trace_points.c -  trace points for EAL library
> > lib/librte_eal/common/include/rte_trace_eal.h - EAL tracepoint public API.
> > lib/librte_eal/common/eal_trace.h - Private trace header file.
> >
> >
> >> I think, based on the performance we can decide one or another?
> > The above performance data shows a much higher improvement with LTTng.
> >
> > Let me know if anyone have any comments on this and or any suggestion.
> > If there are no comments, I will submit the v1 with after code clean
> > up before the 20.05 proposal deadline.
>

  reply index

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-13 10:40 Jerin Jacob Kollanukkaran
2020-01-13 11:00 ` Ray Kinsella
2020-01-13 12:04   ` [dpdk-dev] [EXT] " Jerin Jacob Kollanukkaran
2020-01-18 15:14   ` [dpdk-dev] " dave
2020-01-20 16:51     ` Stephen Hemminger
2020-01-13 13:05 ` Bruce Richardson
2020-01-13 14:46   ` Jerin Jacob
2020-01-13 14:58     ` Bruce Richardson
2020-01-13 15:13       ` Jerin Jacob
2020-01-13 16:12         ` Bruce Richardson
2020-01-17  4:41           ` Jerin Jacob
2020-01-17  8:04             ` David Marchand
2020-01-17  9:52               ` Jerin Jacob
2020-01-17 10:30                 ` Mattias Rönnblom
2020-01-17 10:54                   ` Jerin Jacob
2020-02-15 10:21                     ` Jerin Jacob
2020-02-17  9:35                       ` Mattias Rönnblom
2020-02-17 10:23                         ` Jerin Jacob [this message]
2020-01-17 10:43                 ` David Marchand
2020-01-17 11:08                   ` Jerin Jacob
2020-01-27 16:12 ` Aaron Conole
2020-01-27 17:23   ` Jerin Jacob
2020-01-20  4:48 Jerin Jacob Kollanukkaran
2020-01-20 12:08 ` Ray Kinsella

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CALBAE1MYiR2xA+aV-OaNF-5-==2c_==fSrwK+MR=dLySPT8mfQ@mail.gmail.com' \
    --to=jerinjacobk@gmail.com \
    --cc=abhinandan.gujjar@intel.com \
    --cc=aconole@redhat.com \
    --cc=adrien.mazarguil@6wind.com \
    --cc=adwivedi@marvell.com \
    --cc=ajit.khaparde@broadcom.com \
    --cc=akhil.goyal@nxp.com \
    --cc=anand.rawat@intel.com \
    --cc=anatoly.burakov@intel.com \
    --cc=anoobj@marvell.com \
    --cc=artem.andreev@oktetlabs.ru \
    --cc=arybchenko@solarflare.com \
    --cc=ashishg@marvell.com \
    --cc=beilei.xing@intel.com \
    --cc=bernard.iremonger@intel.com \
    --cc=bluca@debian.org \
    --cc=bruce.richardson@intel.com \
    --cc=byron.marohn@intel.com \
    --cc=chas3@att.com \
    --cc=cloud.wangxiaoyun@huawei.com \
    --cc=cristian.dumitrescu@intel.com \
    --cc=dave@barachs.net \
    --cc=david.hunt@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=declan.doherty@intel.com \
    --cc=deepak.k.jain@intel.com \
    --cc=dev@dpdk.org \
    --cc=drc@linux.vnet.ibm.com \
    --cc=ed.czeck@atomicrules.com \
    --cc=erik.g.carrillo@intel.com \
    --cc=evgenys@amazon.com \
    --cc=ferruh.yigit@intel.com \
    --cc=fiona.trahe@intel.com \
    --cc=g.singh@nxp.com \
    --cc=gage.eads@intel.com \
    --cc=gavin.hu@arm.com \
    --cc=gtzalik@amazon.com \
    --cc=haiyangz@microsoft.com \
    --cc=harini.ramakrishnan@microsoft.com \
    --cc=harry.van.haaren@intel.com \
    --cc=heinrich.kuhn@netronome.com \
    --cc=hemant.agrawal@nxp.com \
    --cc=honnappa.nagarahalli@arm.com \
    --cc=humin29@huawei.com \
    --cc=hyonkim@cisco.com \
    --cc=igor.russkikh@aquantia.com \
    --cc=igorch@amazon.com \
    --cc=jan.gutter@netronome.com \
    --cc=jasvinder.singh@intel.com \
    --cc=jerinj@marvell.com \
    --cc=jgrajcia@cisco.com \
    --cc=jianjay.zhou@huawei.com \
    --cc=jingjing.wu@intel.com \
    --cc=john.griffin@intel.com \
    --cc=john.mcnamara@intel.com \
    --cc=john.miller@atomicrules.com \
    --cc=johndale@cisco.com \
    --cc=joyce.kong@arm.com \
    --cc=jsrikanth@marvell.com \
    --cc=keith.wiles@intel.com \
    --cc=kevin.laatz@intel.com \
    --cc=kirankumark@marvell.com \
    --cc=kirill.rybalchenko@intel.com \
    --cc=konstantin.ananyev@intel.com \
    --cc=ktraynor@redhat.com \
    --cc=kys@microsoft.com \
    --cc=lee.daly@intel.com \
    --cc=liang.j.ma@intel.com \
    --cc=linville@tuxdriver.com \
    --cc=lironh@marvell.com \
    --cc=maicolgabriel@hotmail.com \
    --cc=maintainers@dpdk.org \
    --cc=marko.kovacevic@intel.com \
    --cc=maryam.tahhan@intel.com \
    --cc=matan@mellanox.com \
    --cc=matt.peters@windriver.com \
    --cc=mattias.ronnblom@ericsson.com \
    --cc=maxime.coquelin@redhat.com \
    --cc=mczekaj@marvell.com \
    --cc=mdr@ashroe.eu \
    --cc=michaelsh@marvell.com \
    --cc=mk@semihalf.com \
    --cc=mtetsuyah@gmail.com \
    --cc=mw@semihalf.com \
    --cc=ndabilpuram@marvell.com \
    --cc=nicolas.chautru@intel.com \
    --cc=nipun.gupta@nxp.com \
    --cc=ocardona@microsoft.com \
    --cc=olivier.matz@6wind.com \
    --cc=orika@mellanox.com \
    --cc=pablo.de.lara.guarch@intel.com \
    --cc=pallavi.kadam@intel.com \
    --cc=pavel.belous@aquantia.com \
    --cc=pbhagavatula@marvell.com \
    --cc=peter.mccarthy@intel.com \
    --cc=phil.yang@arm.com \
    --cc=pkapoor@marvell.com \
    --cc=qi.z.zhang@intel.com \
    --cc=qiming.yang@intel.com \
    --cc=radu.nicolau@intel.com \
    --cc=rahul.lakkireddy@chelsio.com \
    --cc=rasland@mellanox.com \
    --cc=ravi1.kumar@amd.com \
    --cc=remes@netcope.com \
    --cc=reshma.pattan@intel.com \
    --cc=rmody@marvell.com \
    --cc=rnagadheeraj@marvell.com \
    --cc=rosen.xu@intel.com \
    --cc=roy.fan.zhang@intel.com \
    --cc=rsanford@akamai.com \
    --cc=ruifeng.wang@arm.com \
    --cc=sachin.saxena@nxp.com \
    --cc=sameh.gobriel@intel.com \
    --cc=shahafs@mellanox.com \
    --cc=shepard.siegel@atomicrules.com \
    --cc=shshaikh@marvell.com \
    --cc=skori@marvell.com \
    --cc=skoteshwar@marvell.com \
    --cc=somnath.kotur@broadcom.com \
    --cc=srinivasan@marvell.com \
    --cc=ssahu@marvell.com \
    --cc=steven.webster@windriver.com \
    --cc=sthemmin@microsoft.com \
    --cc=sthotton@marvell.com \
    --cc=tdu@semihalf.com \
    --cc=thomas@monjalon.net \
    --cc=tianfei.zhang@intel.com \
    --cc=tiwei.bie@intel.com \
    --cc=tomasz.kantecki@intel.com \
    --cc=vattunuru@marvell.com \
    --cc=viacheslavo@mellanox.com \
    --cc=viktorin@rehivetech.com \
    --cc=vladimir.medvedkin@intel.com \
    --cc=wenzhuo.lu@intel.com \
    --cc=xavier.huwei@huawei.com \
    --cc=xiao.w.wang@intel.com \
    --cc=xiaolong.ye@intel.com \
    --cc=xiaoyun.li@intel.com \
    --cc=xuanziyang2@huawei.com \
    --cc=yipeng1.wang@intel.com \
    --cc=yisen.zhuang@huawei.com \
    --cc=yongwang@vmware.com \
    --cc=zhihong.wang@intel.com \
    --cc=zhouguoyang@huawei.com \
    --cc=zr@semihalf.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK-dev Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/dpdk-dev/0 dpdk-dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dpdk-dev dpdk-dev/ https://lore.kernel.org/dpdk-dev \
		dev@dpdk.org
	public-inbox-index dpdk-dev

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git