Re: Could not achieve wire speed for 40GE with any DPDK version on XL710 NIC's

[Pavel Odintsov <pavel.odintsov@gmail.com>]

Thanks for answer, Vladimir! So we need look for x16 NIC if we want
achieve 40GE line rate...

On Wed, Jul 1, 2015 at 3:06 PM, Vladimir Medvedkin <medvedkinv@gmail.com> wrote:
> Hi Pavel,
>
> Looks like you ran into pcie bottleneck. So let's calculate xl710 rx only
> case.
> Assume we have 32byte descriptors (if we want more offload).
> DMA makes one pcie transaction with packet payload, one descriptor writeback
> and one memory request for free descriptors for every 4 packets. For
> Transaction Layer Packet (TLP) there is 30 bytes overhead (4 PHY + 6 DLL +
> 16 header + 4 ECRC). So for 1 rx packet dma sends 30 + 64(packet itself) +
> 30 + 32 (writeback descriptor) + (16 / 4) (read request for new
> descriptors). Note that we do not take into account PCIe ACK/NACK/FC Update
> DLLP. So we have 160 bytes per packet. One lane PCIe 3.0 transmits 1 byte in
> 1 ns, so x8 transmits 8 bytes  in 1 ns. 1 packet transmits in 20 ns.  Thus
> in theory pcie 3.0 x8 may transfer not more than 50mpps.
> Correct me if I'm wrong.
>
> Regards,
> Vladimir
>
>
> 2015-06-29 18:41 GMT+03:00 Pavel Odintsov <pavel.odintsov@gmail.com>:
>>
>> Hello, Andrew!
>>
>> What NIC have you used? Is it XL710?
>>
>> On Mon, Jun 29, 2015 at 6:38 PM, Andrew Theurer <atheurer@redhat.com>
>> wrote:
>> >
>> >
>> > On Mon, Jun 29, 2015 at 10:06 AM, Keunhong Lee <dlrmsghd@gmail.com>
>> > wrote:
>> >>
>> >> I have not used XL710 or i40e.
>> >> I have no opinion for those NICs.
>> >>
>> >> Keunhong.
>> >>
>> >> 2015-06-29 15:59 GMT+09:00 Pavel Odintsov <pavel.odintsov@gmail.com>:
>> >>
>> >> > Hello!
>> >> >
>> >> > Lee, thank you so much for sharing your experience! What do you think
>> >> > about 40GE version of 82599?
>> >> >
>> >> > On Mon, Jun 29, 2015 at 2:35 AM, Keunhong Lee <dlrmsghd@gmail.com>
>> >> > wrote:
>> >> > > DISCLAIMER: This information is not verified. This is truly my
>> >> > > personal
>> >> > > opinion.
>> >> > >
>> >> > > As I know, intel 82599 is the only 10G NIC which supports line rate
>> >> > > with
>> >> > > minimum sized packets (64 byte).
>> >> > > According to our internal tests, Mellanox's 40G NICs even support
>> >> > > less
>> >> > than
>> >> > > 30Mpps.
>> >> > > I think 40 Mpps is the hardware capacity.
>> >
>> >
>> > This is approximately what I see as well.
>> >
>> >>
>> >> > >
>> >> > > Keunhong.
>> >> > >
>> >> > >
>> >> > >
>> >> > > 2015-06-28 19:34 GMT+09:00 Pavel Odintsov
>> >> > > <pavel.odintsov@gmail.com>:
>> >> > >>
>> >> > >> Hello, folks!
>> >> > >>
>> >> > >> We have execute bunch of tests for receive data with Intel XL710
>> >> > >> 40GE
>> >> > >> NIC. We want to achieve wire speed on this platform for traffic
>> >> > >> capture.
>> >> > >>
>> >> > >> But we definitely can't do it. We tried with different versions of
>> >> > >> DPDK: 1.4, 1.6, 1.8, 2.0. And have not success.
>> >> > >>
>> >> > >> We achieved only 40Mpps and could do more.
>> >> > >>
>> >> > >> Could anybody help us with this issue? Looks like this NIC's could
>> >> > >> not
>> >> > >> work on wire speed :(
>> >> > >>
>> >> > >> Platform: Intel Xeon E5 e5 2670 + XL 710.
>> >> > >>
>> >> > >> --
>> >> > >> Sincerely yours, Pavel Odintsov
>> >> > >
>> >> > >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Sincerely yours, Pavel Odintsov
>> >> >
>> >
>> > -Andrew
>> >
>> >
>>
>>
>>
>> --
>> Sincerely yours, Pavel Odintsov
>
>



-- 
Sincerely yours, Pavel Odintsov