Mbuf leak issue with IXGBE in vector mod

Mbuf leak issue with IXGBE in vector mod

[Ori Zakin]

Hi,


  1.  When calling rte_eth_dev_stop mbuf pool is depleted.
There appears to be a race condition that occurs when RTE_IXGBE_INC_VECTOR is defined:
ixgbe_reset_rx_queue(struct ixgbe_adapter *adapter, struct ixgbe_rx_queue *rxq)
{
…….

#ifdef RTE_IXGBE_INC_VECTOR
rxq->rxrearm_start = 0;
>>>>>>>>sleep here appears to solve
rxq->rxrearm_nb = 0;
#endif


Behaviour also described here:
http://dpdk.org/ml/archives/users/2016-April/000488.html


  2.  Steps to recreate issue:
     *   rte_mempool_free_count
     *   rte_eth_dev_stop
     *   rte_mempool_free_count - should see a spike in allocated mbufs from mempool.

  3.  2 workarounds that appear to work:
     *   Set CONFIG_RTE_IXGBE_INC_VECTOR=n.
     *   Add sleep in ixgbe_reset_rx_queue


Regards.
Ori Zakin

Re: Mbuf leak issue with IXGBE in vector mod

[Lu, Wenzhuo]

Hi Ori,

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Ori Zakin
> Sent: Thursday, August 4, 2016 11:38 PM
> To: Zhang, Helin; Ananyev, Konstantin; dev@dpdk.org
> Subject: [dpdk-dev] Mbuf leak issue with IXGBE in vector mod
> 
> Hi,
> 
> 
>   1.  When calling rte_eth_dev_stop mbuf pool is depleted.
> There appears to be a race condition that occurs when RTE_IXGBE_INC_VECTOR
> is defined:
> ixgbe_reset_rx_queue(struct ixgbe_adapter *adapter, struct ixgbe_rx_queue
> *rxq) { …….
> 
> #ifdef RTE_IXGBE_INC_VECTOR
> rxq->rxrearm_start = 0;
> >>>>>>>>sleep here appears to solve
> rxq->rxrearm_nb = 0;
> #endif
> 
> 
> Behaviour also described here:
> http://dpdk.org/ml/archives/users/2016-April/000488.html
> 
> 
>   2.  Steps to recreate issue:
>      *   rte_mempool_free_count
>      *   rte_eth_dev_stop
>      *   rte_mempool_free_count - should see a spike in allocated mbufs from
> mempool.
Have you stopped the rx/tx before do this stop/start? I think we'll hit this memory leak problem if the traffic is not stopped. There's an assumption that it’s APP's responsibility to stop rx/tx before operating the ports as we know dpdk is lockless.

> 
>   3.  2 workarounds that appear to work:
>      *   Set CONFIG_RTE_IXGBE_INC_VECTOR=n.
>      *   Add sleep in ixgbe_reset_rx_queue
> 
> 
> Regards.
> Ori Zakin

Re: Mbuf leak issue with IXGBE in vector mod

[Ori Zakin]

Hi,


  1.  I’m seeing 128 mbufs leaked on each stop/start cycle when polling is stopped, you’re right that the leak is more severe when the device is polled while calling rte_eh_dev_stop.
  2.  Maybe the limitations per each function should be moved in the rte_eh_dev documentation from the header to each of the functions so that they’re more accessible when using a specific function.

Thanks,
Ori





On 5 Aug 2016, at 4:00 AM, Lu, Wenzhuo <wenzhuo.lu@intel.com<mailto:wenzhuo.lu@intel.com>> wrote:

Hi Ori,

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Ori Zakin
Sent: Thursday, August 4, 2016 11:38 PM
To: Zhang, Helin; Ananyev, Konstantin; dev@dpdk.org<mailto:dev@dpdk.org>
Subject: [dpdk-dev] Mbuf leak issue with IXGBE in vector mod

Hi,


 1.  When calling rte_eth_dev_stop mbuf pool is depleted.
There appears to be a race condition that occurs when RTE_IXGBE_INC_VECTOR
is defined:
ixgbe_reset_rx_queue(struct ixgbe_adapter *adapter, struct ixgbe_rx_queue
*rxq) { …….

#ifdef RTE_IXGBE_INC_VECTOR
rxq->rxrearm_start = 0;
sleep here appears to solve
rxq->rxrearm_nb = 0;
#endif


Behaviour also described here:
http://dpdk.org/ml/archives/users/2016-April/000488.html


 2.  Steps to recreate issue:
    *   rte_mempool_free_count
    *   rte_eth_dev_stop
    *   rte_mempool_free_count - should see a spike in allocated mbufs from
mempool.
Have you stopped the rx/tx before do this stop/start? I think we'll hit this memory leak problem if the traffic is not stopped. There's an assumption that it’s APP's responsibility to stop rx/tx before operating the ports as we know dpdk is lockless.


 3.  2 workarounds that appear to work:
    *   Set CONFIG_RTE_IXGBE_INC_VECTOR=n.
    *   Add sleep in ixgbe_reset_rx_queue


Regards.
Ori Zakin