Re: xdpsock poll syscall CPU 100%

["Eelco Chaudron" <echaudro@redhat.com>]

On 21 Feb 2020, at 9:33, Magnus Karlsson wrote:

> On Fri, Feb 21, 2020 at 9:30 AM Eelco Chaudron <echaudro@redhat.com> 
> wrote:
>>
>>
>>
>> On 20 Feb 2020, at 23:49, William Tu wrote:
>>
>>> Hi,
>>>
>>> I'm trying to save some CPU cycles when there is no packet arrives.
>>> I enable the poll syscall option of xdpsock, by doing
>>>
>>> $ ./xdpsock -r -p -S -i ens16
>>>  sock0@ens160:0 rxdrop xdp-skb poll()
>>>                 pps         pkts        1.00
>>> rx              0           0
>>> tx              0           0
>>>
>>> Since there is no packet coming, I though by calling poll()
>>> system call, the xdpsock process will be blocked and CPU utilization
>>> should be way under 100%. However, I'm still seeing 100%
>>> CPU utilization. Am I understanding this correctly?
>>
>> Hi William, I can remember I saw this in the past two with this code. 
>> It
>> had something to do with the way xdpsock waits for the buffers to be
>> free’ ed by the kernel. What I can remember it had something to do
>> with the veth interfaces also.
>>
>> I do remember that I fixed it in the tutorial for AF_XDP:
>> https://github.com/xdp-project/xdp-tutorial/blob/master/advanced03-AF_XDP/af_xdp_user.c
>
> Eelco,
>
> Do you remember exactly what you had to fix in the xdpsock sample?
> Your tutorial is quite a rewrite so it is hard for me to tell exactly
> which of all the changes that fix this problem. The reason I ask is
> that it would be nice to fix this in the sample too.
>
> Thanks: Magnus

 From an earlier email conversation we had this is where it looped in my 
case:

>>>>> One other thing I noticed, which I need to research is that if I
>>>>> use
>>>>> rx_drop() function from /xdpsock_user.c it loops a lot in:
>>>>>
>>>>>   	while (ret != rcvd) {
>>>>>    		if (ret < 0) {
>>>>> 	  			exit(-1);
>>>>> 		}
>>>>>    		ret = xsk_ring_prod__reserve(&xsk->umem-
>>> fq, rcvd, &idx_fq);
>>>>>
>>>>>    	}
>>>>>
>>>>> As ret return 0, until (it looks like) I send more packets. So 
>>>>> even
>>>>> in the poll() mode, it uses 100% cpu after sending a single 
>>>>> packet.
>>>>> Note this is with the default Fedora Kernel, as I’m working on 
>>>>> this
>>>>> from my laptop. Does this sound familiar? If not I’ll dig into 
>>>>> it
>>>>> once I’m back.
>>>>
>>>> The xdpsock test is a busypolling test, to compare against DPDK
>>>> speeds.  For real use-cases, I think people will want to trade-off
>>>> latency vs. burning CPU.
>>>
>>> I understand the use case, but even with the xdpsock test program, 
>>> if
>>> I send a single packet it’s not received, or at least not when 
>>> it's
>>> sent. It takes 16 (or a multiple of it) before the get
>>> detected/processed. I think it’s because of the
>>> xsk_ring_prod__reserve(), but I’ll try to debug it more today and 
>>> to
>>> understand the APIs better.