dev.dpdk.org archive mirror
 help / color / mirror / Atom feed
From: Pavan Nikhilesh Bhagavatula <pbhagavatula@marvell.com>
To: David Marchand <david.marchand@redhat.com>
Cc: Jerin Jacob Kollanukkaran <jerinj@marvell.com>,
	dev <dev@dpdk.org>,
	"Ananyev, Konstantin" <konstantin.ananyev@intel.com>
Subject: Re: [dpdk-dev] [EXT] Re: [PATCH v4 3/3] examples/l3fwd: use reserved addresses for EM mode
Date: Fri, 5 Nov 2021 07:30:56 +0000	[thread overview]
Message-ID: <PH0PR18MB4086D6FF1A76E58BA84833F0DE8E9@PH0PR18MB4086.namprd18.prod.outlook.com> (raw)
In-Reply-To: <CAJFAV8ytxybikS51z9+0E+EGZjQc5xZgo1yHTFtPZGUAy7qw8w@mail.gmail.com>

>On Sun, Oct 3, 2021 at 10:25 PM <pbhagavatula@marvell.com> wrote:
>>
>> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
>>
>> The l3fwd example should use the reserved IPv4/v6 reserved address
>> ranges defined in RFC5735, RFC5180 and RFC863 discard protocol for
>> the port number in the exact match mode of L3 forwarding.
>>
>> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
>> ---
>>  doc/guides/rel_notes/release_21_11.rst |   6 ++
>>  examples/l3fwd/l3fwd.h                 |   2 +-
>>  examples/l3fwd/l3fwd_em.c              | 142 ++++++++++++-------------
>>  3 files changed, 73 insertions(+), 77 deletions(-)
>>
>> diff --git a/doc/guides/rel_notes/release_21_11.rst
>b/doc/guides/rel_notes/release_21_11.rst
>> index 37dc1a7786..ccdfa47baa 100644
>> --- a/doc/guides/rel_notes/release_21_11.rst
>> +++ b/doc/guides/rel_notes/release_21_11.rst
>> @@ -125,6 +125,12 @@ New Features
>>    * Added tests to validate packets hard expiry.
>>    * Added tests to verify tunnel header verification in IPsec inbound.
>>
>> +* **Increase number of routes configured in l3fwd lpm, em and fib
>lookups.**
>> +
>> +  * Increase number of routes from 8 to 16 for all lookup modes of
>l3fwd, this
>> +    helps in validating SoC with more than 8 ethernet devices using
>l3fwd.
>
>It should be in patch1.
>

Ack.

>
>> +  * Update EM mode to use RFC2544 reserved IP address space.
>> +
>>
>>  Removed Items
>>  -------------
>> diff --git a/examples/l3fwd/l3fwd.h b/examples/l3fwd/l3fwd.h
>> index a808d60247..853a31d4c2 100644
>> --- a/examples/l3fwd/l3fwd.h
>> +++ b/examples/l3fwd/l3fwd.h
>> @@ -54,7 +54,7 @@
>>  /* 32-bit has less address-space for hugepage memory, limit to 1M
>entries */
>>  #define L3FWD_HASH_ENTRIES             (1024*1024*1)
>>  #endif
>> -#define HASH_ENTRY_NUMBER_DEFAULT      4
>> +#define HASH_ENTRY_NUMBER_DEFAULT      16
>>
>>  struct mbuf_table {
>>         uint16_t len;
>> diff --git a/examples/l3fwd/l3fwd_em.c
>b/examples/l3fwd/l3fwd_em.c
>> index 2a8ab6aab5..92998e26b2 100644
>> --- a/examples/l3fwd/l3fwd_em.c
>> +++ b/examples/l3fwd/l3fwd_em.c
>> @@ -99,33 +99,64 @@ struct ipv6_l3fwd_em_route {
>>         uint8_t if_out;
>>  };
>>
>> -static struct ipv4_l3fwd_em_route ipv4_l3fwd_em_route_array[] = {
>> -       {{RTE_IPV4(101, 0, 0, 0), RTE_IPV4(100, 10, 0, 1),  101, 11,
>IPPROTO_TCP}, 0},
>> -       {{RTE_IPV4(201, 0, 0, 0), RTE_IPV4(200, 20, 0, 1),  102, 12,
>IPPROTO_TCP}, 1},
>> -       {{RTE_IPV4(111, 0, 0, 0), RTE_IPV4(100, 30, 0, 1),  101, 11,
>IPPROTO_TCP}, 2},
>> -       {{RTE_IPV4(211, 0, 0, 0), RTE_IPV4(200, 40, 0, 1),  102, 12,
>IPPROTO_TCP}, 3},
>> +/* 198.18.0.0/16 are set aside for RFC2544 benchmarking (RFC5735).
>> + * Use RFC863 Discard Protocol.
>> + */
>> +static const struct ipv4_l3fwd_em_route
>ipv4_l3fwd_em_route_array[] = {
>> +       {{RTE_IPV4(198, 18, 0, 0), RTE_IPV4(198, 18, 0, 1),  9, 9,
>IPPROTO_UDP}, 0},
>> +       {{RTE_IPV4(198, 18, 1, 0), RTE_IPV4(198, 18, 1, 1),  9, 9,
>IPPROTO_UDP}, 1},
>> +       {{RTE_IPV4(198, 18, 2, 0), RTE_IPV4(198, 18, 2, 1),  9, 9,
>IPPROTO_UDP}, 2},
>> +       {{RTE_IPV4(198, 18, 3, 0), RTE_IPV4(198, 18, 3, 1),  9, 9,
>IPPROTO_UDP}, 3},
>> +       {{RTE_IPV4(198, 18, 4, 0), RTE_IPV4(198, 18, 4, 1),  9, 9,
>IPPROTO_UDP}, 4},
>> +       {{RTE_IPV4(198, 18, 5, 0), RTE_IPV4(198, 18, 5, 1),  9, 9,
>IPPROTO_UDP}, 5},
>> +       {{RTE_IPV4(198, 18, 6, 0), RTE_IPV4(198, 18, 6, 1),  9, 9,
>IPPROTO_UDP}, 6},
>> +       {{RTE_IPV4(198, 18, 7, 0), RTE_IPV4(198, 18, 7, 1),  9, 9,
>IPPROTO_UDP}, 7},
>> +       {{RTE_IPV4(198, 18, 8, 0), RTE_IPV4(198, 18, 8, 1),  9, 9,
>IPPROTO_UDP}, 8},
>> +       {{RTE_IPV4(198, 18, 9, 0), RTE_IPV4(198, 18, 9, 1),  9, 9,
>IPPROTO_UDP}, 9},
>> +       {{RTE_IPV4(198, 18, 10, 0), RTE_IPV4(198, 18, 10, 1),  9, 9,
>IPPROTO_UDP}, 10},
>> +       {{RTE_IPV4(198, 18, 11, 0), RTE_IPV4(198, 18, 11, 1),  9, 9,
>IPPROTO_UDP}, 11},
>> +       {{RTE_IPV4(198, 18, 12, 0), RTE_IPV4(198, 18, 12, 1),  9, 9,
>IPPROTO_UDP}, 12},
>> +       {{RTE_IPV4(198, 18, 13, 0), RTE_IPV4(198, 18, 13, 1),  9, 9,
>IPPROTO_UDP}, 13},
>> +       {{RTE_IPV4(198, 18, 14, 0), RTE_IPV4(198, 18, 14, 1),  9, 9,
>IPPROTO_UDP}, 14},
>> +       {{RTE_IPV4(198, 18, 15, 0), RTE_IPV4(198, 18, 15, 1),  9, 9,
>IPPROTO_UDP}, 15},
>>  };
>>
>> -static struct ipv6_l3fwd_em_route ipv6_l3fwd_em_route_array[] = {
>> -       {{
>> -       {0xfe, 0x80, 0, 0, 0, 0, 0, 0, 0x02, 0x1e, 0x67, 0xff, 0xfe, 0, 0, 0},
>> -       {0xfe, 0x80, 0, 0, 0, 0, 0, 0, 0x02, 0x1b, 0x21, 0xff, 0xfe, 0x91, 0x38,
>0x05},
>> -       101, 11, IPPROTO_TCP}, 0},
>> -
>> -       {{
>> -       {0xfe, 0x90, 0, 0, 0, 0, 0, 0, 0x02, 0x1e, 0x67, 0xff, 0xfe, 0, 0, 0},
>> -       {0xfe, 0x90, 0, 0, 0, 0, 0, 0, 0x02, 0x1b, 0x21, 0xff, 0xfe, 0x91, 0x38,
>0x05},
>> -       102, 12, IPPROTO_TCP}, 1},
>> -
>> -       {{
>> -       {0xfe, 0xa0, 0, 0, 0, 0, 0, 0, 0x02, 0x1e, 0x67, 0xff, 0xfe, 0, 0, 0},
>> -       {0xfe, 0xa0, 0, 0, 0, 0, 0, 0, 0x02, 0x1b, 0x21, 0xff, 0xfe, 0x91, 0x38,
>0x05},
>> -       101, 11, IPPROTO_TCP}, 2},
>> -
>> -       {{
>> -       {0xfe, 0xb0, 0, 0, 0, 0, 0, 0, 0x02, 0x1e, 0x67, 0xff, 0xfe, 0, 0, 0},
>> -       {0xfe, 0xb0, 0, 0, 0, 0, 0, 0, 0x02, 0x1b, 0x21, 0xff, 0xfe, 0x91, 0x38,
>0x05},
>> -       102, 12, IPPROTO_TCP}, 3},
>> +/* 2001:0200::/48 is IANA reserved range for IPv6 benchmarking
>(RFC5180).
>> + * Use RFC863 Discard Protocol.
>> + */
>> +static const struct ipv6_l3fwd_em_route
>ipv6_l3fwd_em_route_array[] = {
>> +       {{{32, 1, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
>> +         {32, 1, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1}, 9, 9, IPPROTO_UDP}, 0},
>> +       {{{32, 1, 2, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0},
>> +         {32, 1, 2, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1}, 9, 9, IPPROTO_UDP}, 1},
>> +       {{{32, 1, 2, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0},
>> +         {32, 1, 2, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 1}, 9, 9, IPPROTO_UDP}, 2},
>> +       {{{32, 1, 2, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0},
>> +         {32, 1, 2, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 1}, 9, 9, IPPROTO_UDP}, 3},
>> +       {{{32, 1, 2, 0, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0},
>> +         {32, 1, 2, 0, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 1}, 9, 9, IPPROTO_UDP}, 4},
>> +       {{{32, 1, 2, 0, 0, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0, 0},
>> +         {32, 1, 2, 0, 0, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0, 1}, 9, 9, IPPROTO_UDP}, 5},
>> +       {{{32, 1, 2, 0, 0, 0, 0, 6, 0, 0, 0, 0, 0, 0, 0, 0},
>> +         {32, 1, 2, 0, 0, 0, 0, 6, 0, 0, 0, 0, 0, 0, 0, 1}, 9, 9, IPPROTO_UDP}, 6},
>> +       {{{32, 1, 2, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0, 0, 0},
>> +         {32, 1, 2, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0, 0, 1}, 9, 9, IPPROTO_UDP}, 7},
>> +       {{{32, 1, 2, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 0},
>> +         {32, 1, 2, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 1}, 9, 9, IPPROTO_UDP}, 8},
>> +       {{{32, 1, 2, 0, 0, 0, 0, 9, 0, 0, 0, 0, 0, 0, 0, 0},
>> +         {32, 1, 2, 0, 0, 0, 0, 9, 0, 0, 0, 0, 0, 0, 0, 1}, 9, 9, IPPROTO_UDP}, 9},
>> +       {{{32, 1, 2, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0},
>> +         {32, 1, 2, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 1}, 9, 9, IPPROTO_UDP},
>10},
>> +       {{{32, 1, 2, 0, 0, 0, 0, 11, 0, 0, 0, 0, 0, 0, 0, 0},
>> +         {32, 1, 2, 0, 0, 0, 0, 11, 0, 0, 0, 0, 0, 0, 0, 1}, 9, 9, IPPROTO_UDP},
>11},
>> +       {{{32, 1, 2, 0, 0, 0, 0, 12, 0, 0, 0, 0, 0, 0, 0, 0},
>> +         {32, 1, 2, 0, 0, 0, 0, 12, 0, 0, 0, 0, 0, 0, 0, 1}, 9, 9, IPPROTO_UDP},
>12},
>> +       {{{32, 1, 2, 0, 0, 0, 0, 13, 0, 0, 0, 0, 0, 0, 0, 0},
>> +         {32, 1, 2, 0, 0, 0, 0, 13, 0, 0, 0, 0, 0, 0, 0, 1}, 9, 9, IPPROTO_UDP},
>13},
>> +       {{{32, 1, 2, 0, 0, 0, 0, 14, 0, 0, 0, 0, 0, 0, 0, 0},
>> +         {32, 1, 2, 0, 0, 0, 0, 14, 0, 0, 0, 0, 0, 0, 0, 1}, 9, 9, IPPROTO_UDP},
>14},
>> +       {{{32, 1, 2, 0, 0, 0, 0, 15, 0, 0, 0, 0, 0, 0, 0, 0},
>> +         {32, 1, 2, 0, 0, 0, 0, 15, 0, 0, 0, 0, 0, 0, 0, 1}, 9, 9, IPPROTO_UDP},
>15},
>>  };
>>
>>  struct rte_hash *ipv4_l3fwd_em_lookup_struct[NB_SOCKETS];
>> @@ -405,7 +436,7 @@ populate_ipv6_few_flow_into_table(const
>struct rte_hash *h)
>>                 (uint64_t)IPV6_L3FWD_EM_NUM_ROUTES);
>>  }
>>
>> -#define NUMBER_PORT_USED 4
>> +#define NUMBER_PORT_USED 16
>>  static inline void
>>  populate_ipv4_many_flow_into_table(const struct rte_hash *h,
>>                 unsigned int nr_flow)
>> @@ -416,36 +447,16 @@
>populate_ipv4_many_flow_into_table(const struct rte_hash *h,
>>                                 ALL_32_BITS, ALL_32_BITS} };
>>
>>         for (i = 0; i < nr_flow; i++) {
>> +               uint8_t port = i % NUMBER_PORT_USED;
>>                 struct ipv4_l3fwd_em_route entry;
>>                 union ipv4_5tuple_host newkey;
>>
>> -               uint8_t a = (uint8_t)
>> -                       ((i/NUMBER_PORT_USED)%BYTE_VALUE_MAX);
>> -               uint8_t b = (uint8_t)
>> -
>(((i/NUMBER_PORT_USED)/BYTE_VALUE_MAX)%BYTE_VALUE_MAX);
>> -               uint8_t c = (uint8_t)
>> -
>((i/NUMBER_PORT_USED)/(BYTE_VALUE_MAX*BYTE_VALUE_MAX));
>> +               uint8_t a = (uint8_t)((port + 1) % BYTE_VALUE_MAX);
>>
>>                 /* Create the ipv4 exact match flow */
>>                 memset(&entry, 0, sizeof(entry));
>> -               switch (i & (NUMBER_PORT_USED - 1)) {
>> -               case 0:
>> -                       entry = ipv4_l3fwd_em_route_array[0];
>> -                       entry.key.ip_dst = RTE_IPV4(101, c, b, a);
>> -                       break;
>> -               case 1:
>> -                       entry = ipv4_l3fwd_em_route_array[1];
>> -                       entry.key.ip_dst = RTE_IPV4(201, c, b, a);
>> -                       break;
>> -               case 2:
>> -                       entry = ipv4_l3fwd_em_route_array[2];
>> -                       entry.key.ip_dst = RTE_IPV4(111, c, b, a);
>> -                       break;
>> -               case 3:
>> -                       entry = ipv4_l3fwd_em_route_array[3];
>> -                       entry.key.ip_dst = RTE_IPV4(211, c, b, a);
>> -                       break;
>> -               };
>> +               entry = ipv4_l3fwd_em_route_array[port];
>> +               entry.key.ip_dst = RTE_IPV4(198, 18, port, a);
>>                 convert_ipv4_5tuple(&entry.key, &newkey);
>>                 int32_t ret = rte_hash_add_key(h, (void *) &newkey);
>>
>> @@ -469,35 +480,14 @@
>populate_ipv6_many_flow_into_table(const struct rte_hash *h,
>>         mask2 = (rte_xmm_t){.u32 = {ALL_32_BITS, ALL_32_BITS, 0, 0} };
>>
>>         for (i = 0; i < nr_flow; i++) {
>> +               uint8_t port = i % NUMBER_PORT_USED;
>>                 struct ipv6_l3fwd_em_route entry;
>>                 union ipv6_5tuple_host newkey;
>>
>> -               uint8_t a = (uint8_t)
>> -                       ((i/NUMBER_PORT_USED)%BYTE_VALUE_MAX);
>> -               uint8_t b = (uint8_t)
>> -
>(((i/NUMBER_PORT_USED)/BYTE_VALUE_MAX)%BYTE_VALUE_MAX);
>> -               uint8_t c = (uint8_t)
>> -
>((i/NUMBER_PORT_USED)/(BYTE_VALUE_MAX*BYTE_VALUE_MAX));
>> -
>>                 /* Create the ipv6 exact match flow */
>>                 memset(&entry, 0, sizeof(entry));
>> -               switch (i & (NUMBER_PORT_USED - 1)) {
>> -               case 0:
>> -                       entry = ipv6_l3fwd_em_route_array[0];
>> -                       break;
>> -               case 1:
>> -                       entry = ipv6_l3fwd_em_route_array[1];
>> -                       break;
>> -               case 2:
>> -                       entry = ipv6_l3fwd_em_route_array[2];
>> -                       break;
>> -               case 3:
>> -                       entry = ipv6_l3fwd_em_route_array[3];
>> -                       break;
>> -               };
>> -               entry.key.ip_dst[13] = c;
>> -               entry.key.ip_dst[14] = b;
>> -               entry.key.ip_dst[15] = a;
>> +               entry = ipv6_l3fwd_em_route_array[port];
>> +               entry.key.ip_dst[15] = (port + 1) % BYTE_VALUE_MAX;
>>                 convert_ipv6_5tuple(&entry.key, &newkey);
>>                 int32_t ret = rte_hash_add_key(h, (void *) &newkey);
>>
>> @@ -584,7 +574,7 @@ em_parse_ptype(struct rte_mbuf *m)
>>                 hdr_len = rte_ipv4_hdr_len(ipv4_hdr);
>>                 if (hdr_len == sizeof(struct rte_ipv4_hdr)) {
>>                         packet_type |= RTE_PTYPE_L3_IPV4;
>> -                       if (ipv4_hdr->next_proto_id == IPPROTO_TCP)
>> +                       if (ipv4_hdr->next_proto_id == IPPROTO_UDP)
>
>I don't understand the intention here.

Routes are installed for UDP proto, I think this was an unintended substitution mistake, 
I will fix in next version.

Thanks,
Pavan.

>
>
>>                                 packet_type |= RTE_PTYPE_L4_TCP;
>>                         else if (ipv4_hdr->next_proto_id == IPPROTO_UDP)
>>                                 packet_type |= RTE_PTYPE_L4_UDP;
>> @@ -592,7 +582,7 @@ em_parse_ptype(struct rte_mbuf *m)
>>                         packet_type |= RTE_PTYPE_L3_IPV4_EXT;
>>         } else if (ether_type ==
>rte_cpu_to_be_16(RTE_ETHER_TYPE_IPV6)) {
>>                 ipv6_hdr = (struct rte_ipv6_hdr *)l3;
>> -               if (ipv6_hdr->proto == IPPROTO_TCP)
>> +               if (ipv6_hdr->proto == IPPROTO_UDP)
>>                         packet_type |= RTE_PTYPE_L3_IPV6 |
>RTE_PTYPE_L4_TCP;
>
>Idem.
>
>
>>                 else if (ipv6_hdr->proto == IPPROTO_UDP)
>>                         packet_type |= RTE_PTYPE_L3_IPV6 |
>RTE_PTYPE_L4_UDP;
>> --
>> 2.17.1
>>
>
>
>--
>David Marchand


  reply	other threads:[~2021-11-05  7:31 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-30  5:23 [dpdk-dev] [PATCH 1/2] examples/l3fwd: increase number of routes pbhagavatula
2019-10-30  5:23 ` [dpdk-dev] [PATCH 2/2] examples/l3fwd: print port bdf when initializing routes pbhagavatula
2019-11-08  8:51 ` [dpdk-dev] [PATCH 1/2] examples/l3fwd: increase number of routes David Marchand
2020-08-02 18:16 ` [dpdk-dev] [PATCH v2 1/3] " pbhagavatula
2020-08-02 18:16   ` [dpdk-dev] [PATCH v2 2/3] examples/l3fwd: use IP reserved addresses for EM mode pbhagavatula
2020-10-23 14:11     ` David Marchand
2020-08-02 18:16   ` [dpdk-dev] [PATCH v2 3/3] examples/l3fwd: print port bdf when initializing routes pbhagavatula
2021-10-03 20:11   ` [dpdk-dev] [PATCH 1/3] examples/l3fwd: increase number of routes pbhagavatula
2021-10-03 20:11     ` [dpdk-dev] [PATCH 2/3] examples/l3fwd: print port bdf when adding routes pbhagavatula
2021-11-04 10:28       ` Walsh, Conor
2021-11-05  7:27         ` Pavan Nikhilesh Bhagavatula
2021-10-03 20:11     ` [dpdk-dev] [PATCH 3/3] examples/l3fwd: use reserved addresses for EM mode pbhagavatula
2021-10-03 20:24     ` [dpdk-dev] [PATCH v4 1/3] examples/l3fwd: increase number of routes pbhagavatula
2021-10-03 20:24       ` [dpdk-dev] [PATCH v4 2/3] examples/l3fwd: print port bdf when adding routes pbhagavatula
2021-10-03 20:24       ` [dpdk-dev] [PATCH v4 3/3] examples/l3fwd: use reserved addresses for EM mode pbhagavatula
2021-11-03 20:53         ` David Marchand
2021-11-05  7:30           ` Pavan Nikhilesh Bhagavatula [this message]
2021-11-04 10:28       ` [dpdk-dev] [PATCH v4 1/3] examples/l3fwd: increase number of routes Walsh, Conor
2021-11-05  7:27         ` Pavan Nikhilesh Bhagavatula
2021-11-05  8:38       ` [dpdk-dev] [PATCH v5 " pbhagavatula
2021-11-05  8:38         ` [dpdk-dev] [PATCH v5 2/3] examples/l3fwd: print port bdf when adding routes pbhagavatula
2021-11-05  9:34           ` Walsh, Conor
2021-11-05  8:38         ` [dpdk-dev] [PATCH v5 3/3] examples/l3fwd: use reserved addresses for EM mode pbhagavatula
2021-11-16 10:03           ` David Marchand
2021-11-05  9:28         ` [dpdk-dev] [PATCH v5 1/3] examples/l3fwd: increase number of routes Walsh, Conor
2021-10-04 15:46     ` [dpdk-dev] [PATCH " Stephen Hemminger

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=PH0PR18MB4086D6FF1A76E58BA84833F0DE8E9@PH0PR18MB4086.namprd18.prod.outlook.com \
    --to=pbhagavatula@marvell.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=jerinj@marvell.com \
    --cc=konstantin.ananyev@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).