Endless errors when starting multiple dpdk applications

Endless errors when starting multiple dpdk applications

[Huimin She]

Hi,

I am trying to run multiple dpdk forwarding applications on VMs (KVM). The settings are as followings
Dpdk version: master
PF1 (port 2 of an Intel I350-T4 NIC) has two VFs: VF1(pci: 0000:01:01.0), VF2 (pci: 0000:01:02.0)
PF2 (port 3 of an Intel I350-T4NIC ) has two VFs: VF3 (pci: 0000:01:03.0), VF4 (pci: 0000:01:04.0)
The VFs are started by: sudo insmod igb.ko max_vfs=0,2,2,0 VMDQ=0,4,4,0

Dpdk-fwd-app-1 runs on the VM between VF1 and VF4 like this:
sudo ./dpdk-app --syslog syslog -c 0x2 -n 2 -m 32  --use-device 01:01.0,01:04.0 --file-prefix 11 --  -q 4

If I only start single application, the packets are successfully forwarded.
However when I start the second dpdk application: Dpdk-fwd-app-2 between VF2 and VF3 like this:
sudo ./dpdk-app --syslog syslog -c 0x4 -n 2 -m 32 --use-device 01:02.0,01:03.0 --file-prefix 22 --  -q 4

Then  endless error messages are printed out:
----------------------------------------------------------------------------------------------
......
CPU-2: CMTL: put_msg: tx buffer is full!
CPU-2: CMTL: shmem_dc_transmit: Tx queue is full,dropping message!
CPU-2: CMTL: Force disconnect!
Error: user_called:1 ecode:0x0000f001 extra:0x00000000
......
--------------------------------------------------------------------------------------------------
I tried to change the parameters for: -c, -n, -q, -m. Still see the errors.

Has any of you seen similar things before?
Is this caused by any missing configurations? (Such as specifying separate TX/RX queues for different VFs, how to do this?)

Thanks a lot.

Best regards,
Huimin She