Why IVSHMEM was removed since 16.11 ?

Why IVSHMEM was removed since 16.11 ?

[Furong]

The release notes of dpdk-16.11 had shown that IVSHMEM was removed due 
to some design issues.

So, what are these issues?

Thanks!

Re: Why IVSHMEM was removed since 16.11 ?

[Burakov, Anatoly]

> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Furong
> Sent: Tuesday, August 8, 2017 8:26 AM
> To: dev@dpdk.org
> Subject: [dpdk-dev] Why IVSHMEM was removed since 16.11 ?
> 
> The release notes of dpdk-16.11 had shown that IVSHMEM was removed
> due to some design issues.
> 
> So, what are these issues?
> 
> Thanks!
> 

Hi Furong,

There were multiple issues involved. Biggest of all, it required a patch to QEMU that wasn't maintained and wasn't upstream (i.e. vanilla QEMU didn't work with DPDK's implementation of IVSHMEM support). Second, it was basically hacked in to EAL in order to support what it does [1], and the engineering effort to fix all of that isn't worth the benefit it would provide, as no one appeared to be using it heavily enough to object to its deprecation.

Hope this clears things up.

[1] http://dpdk.org/ml/archives/dev/2016-June/040844.html

Thanks,
Anatoly 

Intel 82599ES send packets failed on dpdk-17.05.1 after traffic testing

[zimeiw]

hi,
My test env is dell T430 server, NIC is intel 82599ES.
I integrate my app with dpdk-17.05.1 version. after do some tcp traffic testing, app can receive packets from the 82599 NIC, but 82599 NIC send packets failed.
But my app with dpdk-17.02 works well. 


If any change with ixgbe driver? or need some additional configuration for the NIC?



Thanks.


--

Best Regards,
zimeiw

Re: Intel 82599ES send packets failed on dpdk-17.05.1 after traffic testing

[Ferruh Yigit]

On 8/22/2017 5:45 AM, zimeiw wrote:
> hi,
> My test env is dell T430 server, NIC is intel 82599ES.
> I integrate my app with dpdk-17.05.1 version. after do some tcp traffic testing, app can receive packets from the 82599 NIC, but 82599 NIC send packets failed.
> But my app with dpdk-17.02 works well. 
> 
> 
> If any change with ixgbe driver? or need some additional configuration for the NIC?

tx_pkt_prepare added but it has been added on 17.02 which seems working
for your case.

Just to double check, are you using rte_eth_tx_prepare()?

> 
> 
> 
> Thanks.
> 
> 
> --
> 
> Best Regards,
> zimeiw
> 

Re: Intel 82599ES send packets failed on dpdk-17.05.1 after traffic testing

[zimeiw]

hi,


no use rte_eth_tx_prepare()?


I set txq_flags value as below,  may it cause this issue?


  .txq_flags = ~ETH_TXQ_FLAGS_NOXSUMS 


--

Best Regards,
zimeiw



At 2017-08-22 15:49:56, "Ferruh Yigit" <ferruh.yigit@intel.com> wrote:
>On 8/22/2017 5:45 AM, zimeiw wrote:
>> hi,
>> My test env is dell T430 server, NIC is intel 82599ES.
>> I integrate my app with dpdk-17.05.1 version. after do some tcp traffic testing, app can receive packets from the 82599 NIC, but 82599 NIC send packets failed.
>> But my app with dpdk-17.02 works well. 
>> 
>> 
>> If any change with ixgbe driver? or need some additional configuration for the NIC?
>
>tx_pkt_prepare added but it has been added on 17.02 which seems working
>for your case.
>
>Just to double check, are you using rte_eth_tx_prepare()?
>
>> 
>> 
>> 
>> Thanks.
>> 
>> 
>> --
>> 
>> Best Regards,
>> zimeiw
>> 
>

Re: Intel 82599ES send packets failed on dpdk-17.05.1 after traffic testing

[Ferruh Yigit]

On 8/22/2017 9:30 AM, zimeiw wrote:
> hi,
> 
> no use rte_eth_tx_prepare()?
> 
> I set txq_flags value as below,  may it cause this issue?
> 
>   .txq_flags = ~ETH_TXQ_FLAGS_NOXSUMS 

So you are enabling all csum offloading, for Tx path this requires
packets to be prepared before sending to the NIC.

Can you please test with rte_eth_tx_prepare()? testpmd has the sample usage.

Thanks,
ferruh

> 
> --
> Best Regards,
> zimeiw
> 
> 
> At 2017-08-22 15:49:56, "Ferruh Yigit" <ferruh.yigit@intel.com> wrote:
>>On 8/22/2017 5:45 AM, zimeiw wrote:
>>> hi,
>>> My test env is dell T430 server, NIC is intel 82599ES.
>>> I integrate my app with dpdk-17.05.1 version. after do some tcp traffic testing, app can receive packets from the 82599 NIC, but 82599 NIC send packets failed.
>>> But my app with dpdk-17.02 works well. 
>>> 
>>> 
>>> If any change with ixgbe driver? or need some additional configuration for the NIC?
>>
>>tx_pkt_prepare added but it has been added on 17.02 which seems working
>>for your case.
>>
>>Just to double check, are you using rte_eth_tx_prepare()?
>>
>>> 
>>> 
>>> 
>>> Thanks.
>>> 
>>> 
>>> --
>>> 
>>> Best Regards,
>>> zimeiw
>>> 
>>
> 
> 
> 
>  
> 

Re: Intel 82599ES send packets failed on dpdk-17.05.1 after traffic testing

[zimeiw]

hi,


I have tested below dpdk version, after dpdk-17.05 version, ixgbe nic would send fail. the logs is same as below.


dpdk-17.02    ---- ok
dpdk-17.02.1  ---- ok 


dpdk-17.05  ---- nok 

dpdk-17.05.1  ---- nok 

dpdk-17.08   ---- nok 


PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 tx_tail=63 nb_tx=11
PMD: ixgbe_xmit_cleanup(): TX descriptor   95 is not done(port=0 queue=0)
PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 pktlen=205 tx_first=63 tx_last=63
PMD: ixgbe_xmit_pkts(): Not enough free TX descriptors nb_used=   1 nb_free=   0 (port=0 queue=0)
PMD: ixgbe_xmit_cleanup(): TX descriptor   95 is not done(port=0 queue=0)
tx burst failed, ret 0 , n 16,  Success 
PMD: ixgbe_xmit_cleanup(): TX descriptor   95 is not done(port=0 queue=0)
PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 pktlen=205 tx_first=63 tx_last=63
PMD: ixgbe_xmit_pkts(): Not enough free TX descriptors nb_used=   1 nb_free=   0 (port=0 queue=0)
PMD: ixgbe_xmit_cleanup(): TX descriptor   95 is not done(port=0 queue=0)
tx burst failed, ret 0 , n 16,  Success 
PMD: ixgbe_xmit_cleanup(): TX descriptor   95 is not done(port=0 queue=0)
PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 pktlen=54 tx_first=63 tx_last=63
PMD: ixgbe_xmit_pkts(): Not enough free TX descriptors nb_used=   1 nb_free=   0 (port=0 queue=0)
PMD: ixgbe_xmit_cleanup(): TX descriptor   95 is not done(port=0 queue=0)
tx burst failed, ret 0 , n 5, Success 
PMD: ixgbe_xmit_cleanup(): TX descriptor   95 is not done(port=0 queue=0)



--

Best Regards,
zimeiw



At 2017-08-23 17:04:44, "Ferruh Yigit" <ferruh.yigit@intel.com> wrote:
>On 8/23/2017 6:23 AM, zimeiw wrote:
>> hi,
>> 
>> Any comments?
>> 
>> is it possible NIC haredware issue? or NIC firmware version isn't match
>> dpdk driver?
>
>AFAIK for there isn't this kind of dependency for ixgbe. ixgbe driver
>maintainer cc'ed for more information.
>
>> 
>> --
>> Best Regards,
>> zimeiw
>> 
>> At 2017-08-23 08:52:38, "zimeiw" <zimeiw@163.com> wrote:
>> 
>>     hi,
>> 
>>     "disable tso" logs were printed by my app. 
>> 
>>     dpdk-17.05.1 is download from dpdk website, it is stable release.
>> 
>> 
>>     --
>>     Best Regards,
>>     zimeiw
>> 
>> 
>>     At 2017-08-22 22:23:54, "Ferruh Yigit" <ferruh.yigit@intel.com <mailto:ferruh.yigit@intel.com>> wrote:
>>     >On 8/22/2017 2:51 PM, zimeiw wrote:
>>     >> hi,
>>     >> =====
>>     >> check the dpdk default config,  CONFIG_RTE_ETHDEV_TX_PREPARE_NOOP isn't
>>     >> enable.
>>     >> #
>>     >> # Turn off Tx preparation stage
>>     >> #
>>     >> # Warning: rte_ethdev_tx_prepare() can be safely disabled only if using a
>>     >> # driver which do not implement any Tx preparation.
>>     >> #
>>     >> CONFIG_RTE_ETHDEV_TX_PREPARE_NOOP=n
>>     >> 
>>     >> ====
>>     >> I enable IXGBE debug for dpdk-17.05.1 and dpdk-17.02. the logs are in
>>     >> the attachment.
>>     >> app with dpdk-17.02 works ok.
>>     >> app with dpdk-17.05.1 has below issue. it seems TX descriptor   95 can't
>>     >> be handled by NIC?
>>     >
>>     >This is interesting, no idea what is going on, trying dpdk-17.05 can be
>>     >an option, to be sure if this issue is in the original release or in the
>>     >stable tree?
>>     >
>>     >btw, in 17.05.01, there are "disable tso" logs, do you know where they
>>     >are coming from?
>>     >
>>     >> 
>>     >> 
>>     >> PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 tx_tail=63 nb_tx=11
>>     >> PMD: ixgbe_xmit_cleanup(): TX descriptor   95 is not done(port=0 queue=0)
>>     >
>>     >Descriptor Done bit is not set, HW says descriptor is still in use, so
>>     >cleaning descriptor fails.
>>     >
>>     >> PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 pktlen=205 tx_first=63
>>     >> tx_last=63
>>     >> PMD: ixgbe_xmit_pkts(): Not enough free TX descriptors nb_used=   1
>>     >> nb_free=   0 (port=0 queue=0)
>>     >
>>     >And after a while there is no free descriptor left and Tx fails.
>>     >
>>     >> PMD: ixgbe_xmit_cleanup(): TX descriptor   95 is not done(port=0 queue=0)
>>     >> tx burst failed, ret 0 , n 16, nb_prep 0 , Success 
>>     >> PMD: ixgbe_xmit_cleanup(): TX descriptor   95 is not done(port=0 queue=0)
>>     >> PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 pktlen=205 tx_first=63
>>     >> tx_last=63
>>     >> PMD: ixgbe_xmit_pkts(): Not enough free TX descriptors nb_used=   1
>>     >> nb_free=   0 (port=0 queue=0)
>>     >> PMD: ixgbe_xmit_cleanup(): TX descriptor   95 is not done(port=0 queue=0)
>>     >> tx burst failed, ret 0 , n 16, nb_prep 0 , Success 
>>     >> PMD: ixgbe_xmit_cleanup(): TX descriptor   95 is not done(port=0 queue=0)
>>     >> PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 pktlen=54 tx_first=63
>>     >> tx_last=63
>>     >> PMD: ixgbe_xmit_pkts(): Not enough free TX descriptors nb_used=   1
>>     >> nb_free=   0 (port=0 queue=0)
>>     >> PMD: ixgbe_xmit_cleanup(): TX descriptor   95 is not done(port=0 queue=0)
>>     >> tx burst failed, ret 0 , n 5, nb_prep 0 , Success 
>>     >> 
>>     >> 
>>     >> 
>>     >> --
>>     >> Best Regards,
>>     >> zimeiw
>>     >> 
>>     >> 
>>     >> At 2017-08-22 20:13:57, "Ferruh Yigit" <ferruh.yigit@intel.com <mailto:ferruh.yigit@intel.com>> wrote:
>>     >>>On 8/22/2017 12:00 PM, zimeiw wrote:
>>     >>>> hi,
>>     >>>> 
>>     >>>> i set .txq_flags = 0.
>>     >>>> and change the "send burst" code as below.
>>     >>>> and printf some logs.
>>     >>>> according to the logs, rte_eth_tx_burst() return 0, no error happen in
>>     >>>> rte_eth_tx_burst().
>>     >>>> 
>>     >>>> ====================
>>     >>>>    struct rte_mbuf **m_table;
>>     >>>>     int ret;
>>     >>>>     uint16_t queueid;
>>     >>>>     uint16_t nb_prep;
>>     >>>> 
>>     >>>>     queueid = qconf->tx_queue_id[port];
>>     >>>>     m_table = (struct rte_mbuf **)qconf->tx_mbufs[port].m_table;
>>     >>>> 
>>     >>>>     nb_prep = rte_eth_tx_prepare(port, queueid, m_table, n);
>>     >>>>     if (nb_prep != n)
>>     >>>>         printf("Preparing packet burst %d  to transmit failed: %s\n",
>>     >>>> nb_prep, rte_strerror(rte_errno));
>>     >>>
>>     >>>m->ol_flags should request offloading (PKT_TX_...), otherwise prepare
>>     >>>does not do the work.
>>     >>>
>>     >>>but m->ol_flags is not changed, if this is working with 17.02, issue can
>>     >>>be something else.
>>     >>>
>>     >>>You may try debugging ixgbe PMD to figure out at which point it gives an
>>     >>>error.
>>     >>>
>>     >>>> 
>>     >>>>     ret = rte_eth_tx_burst(port, queueid, m_table, nb_prep);
>>     >>>>     if (unlikely(ret < n))
>>     >>>>     {
>>     >>>>         printf("tx burst failed, ret %d , n %d, nb_prep %d , %s \n",
>>     >>>> ret, n, nb_prep, rte_strerror(rte_errno) );
>>     >>>>         do
>>     >>>>         {
>>     >>>>             rte_pktmbuf_free(m_table[ret]);
>>     >>>>         } while (++ret < n);
>>     >>>>     }
>>     >>>> =======================
>>     >>>> 
>>     >>>> =====log======
>>     >>>> Checking link status done
>>     >>>> Port 0 Link Up - speed 10000 Mbps - full-duplex
>>     >>>> USER8: main loop on lcore 1
>>     >>>> USER8:  -- lcoreid=1 portid=0 rxqueueid=0
>>     >>>> nb ports 1 hz: 1698027100
>>     >>>> tx burst failed, ret 2 , n 16, nb_prep 16 , Success
>>     >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success
>>     >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success
>>     >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success
>>     >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success
>>     >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success
>>     >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success
>>     >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success
>>     >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success
>>     >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success
>>     >>>> 
>>     >>>> ==========NIC firware version =======
>>     >>>> # ethtool -i enp2s0f1
>>     >>>> driver: ixgbe
>>     >>>> version: 4.2.1-k
>>     >>>> firmware-version: 0x61bd0001
>>     >>>> expansion-rom-version:
>>     >>>> bus-info: 0000:02:00.1
>>     >>>> supports-statistics: yes
>>     >>>> supports-test: yes
>>     >>>> supports-eeprom-access: yes
>>     >>>> supports-register-dump: yes
>>     >>>> supports-priv-flags: no
>>     >>>> 
>>     >>>> 
>>     >>>> --
>>     >>>> Best Regards,
>>     >>>> zimeiw
>>     >>>> 
>>     >>>> 
>>     >>>> At 2017-08-22 17:31:56, "Ferruh Yigit" <ferruh.yigit@intel.com <mailto:ferruh.yigit@intel.com>> wrote:
>>     >>>>>On 8/22/2017 9:30 AM, zimeiw wrote:
>>     >>>>>> hi,
>>     >>>>>> 
>>     >>>>>> no use rte_eth_tx_prepare()?
>>     >>>>>> 
>>     >>>>>> I set txq_flags value as below,  may it cause this issue?
>>     >>>>>> 
>>     >>>>>>   .txq_flags = ~ETH_TXQ_FLAGS_NOXSUMS 
>>     >>>>>
>>     >>>>>So you are enabling all csum offloading, for Tx path this requires
>>     >>>>>packets to be prepared before sending to the NIC.
>>     >>>>>
>>     >>>>>Can you please test with rte_eth_tx_prepare()? testpmd has the sample usage.
>>     >>>>>
>>     >>>>>Thanks,
>>     >>>>>ferruh
>>     >>>>>
>>     >>>>>> 
>>     >>>>>> --
>>     >>>>>> Best Regards,
>>     >>>>>> zimeiw
>>     >>>>>> 
>>     >>>>>> 
>>     >>>>>> At 2017-08-22 15:49:56, "Ferruh Yigit" <ferruh.yigit@intel.com <mailto:ferruh.yigit@intel.com>> wrote:
>>     >>>>>>>On 8/22/2017 5:45 AM, zimeiw wrote:
>>     >>>>>>>> hi,
>>     >>>>>>>> My test env is dell T430 server, NIC is intel 82599ES.
>>     >>>>>>>> I integrate my app with dpdk-17.05.1 version. after do some tcp traffic testing, app can receive packets from the 82599 NIC, but 82599 NIC send packets failed.
>>     >>>>>>>> But my app with dpdk-17.02 works well. 
>>     >>>>>>>> 
>>     >>>>>>>> 
>>     >>>>>>>> If any change with ixgbe driver? or need some additional configuration for the NIC?
>>     >>>>>>>
>>     >>>>>>>tx_pkt_prepare added but it has been added on 17.02 which seems working
>>     >>>>>>>for your case.
>>     >>>>>>>
>>     >>>>>>>Just to double check, are you using rte_eth_tx_prepare()?
>>     >>>>>>>
>>     >>>>>>>> 
>>     >>>>>>>> 
>>     >>>>>>>> 
>>     >>>>>>>> Thanks.
>>     >>>>>>>> 
>>     >>>>>>>> 
>>     >>>>>>>> --
>>     >>>>>>>> 
>>     >>>>>>>> Best Regards,
>>     >>>>>>>> zimeiw
>>     >>>>>>>> 
>>     >>>>>>>
>>     >>>>>> 
>>     >>>>>> 
>>     >>>>>> 
>>     >>>>>>  
>>     >>>>>> 
>>     >>>>>
>>     >>>> 
>>     >>>> 
>>     >>>> 
>>     >>>>  
>>     >>>> 
>>     >>>
>>     >> 
>>     >> 
>>     >> 
>>     >>  
>>     >> 
>>     >
>> 
>> 
>> 
>>      
>> 
>> 
>> 
>>  
>> 
>