Re: xdpsock poll syscall CPU 100%

[Magnus Karlsson <magnus.karlsson@gmail.com>]

On Fri, Feb 21, 2020 at 9:40 AM Eelco Chaudron <echaudro@redhat.com> wrote:
>
>
>
> On 21 Feb 2020, at 9:33, Magnus Karlsson wrote:
>
> > On Fri, Feb 21, 2020 at 9:30 AM Eelco Chaudron <echaudro@redhat.com>
> > wrote:
> >>
> >>
> >>
> >> On 20 Feb 2020, at 23:49, William Tu wrote:
> >>
> >>> Hi,
> >>>
> >>> I'm trying to save some CPU cycles when there is no packet arrives.
> >>> I enable the poll syscall option of xdpsock, by doing
> >>>
> >>> $ ./xdpsock -r -p -S -i ens16
> >>>  sock0@ens160:0 rxdrop xdp-skb poll()
> >>>                 pps         pkts        1.00
> >>> rx              0           0
> >>> tx              0           0
> >>>
> >>> Since there is no packet coming, I though by calling poll()
> >>> system call, the xdpsock process will be blocked and CPU utilization
> >>> should be way under 100%. However, I'm still seeing 100%
> >>> CPU utilization. Am I understanding this correctly?
> >>
> >> Hi William, I can remember I saw this in the past two with this code.
> >> It
> >> had something to do with the way xdpsock waits for the buffers to be
> >> free’ ed by the kernel. What I can remember it had something to do
> >> with the veth interfaces also.
> >>
> >> I do remember that I fixed it in the tutorial for AF_XDP:
> >> https://github.com/xdp-project/xdp-tutorial/blob/master/advanced03-AF_XDP/af_xdp_user.c
> >
> > Eelco,
> >
> > Do you remember exactly what you had to fix in the xdpsock sample?
> > Your tutorial is quite a rewrite so it is hard for me to tell exactly
> > which of all the changes that fix this problem. The reason I ask is
> > that it would be nice to fix this in the sample too.
> >
> > Thanks: Magnus
>
>  From an earlier email conversation we had this is where it looped in my
> case:

Thanks Eelco. Yes, the xdpsock sample is too simplistic in this case.
I will put this on my backlog to fix so that we do not have this
problem in the future. I might take some inspiration from your code
:-). Hope you do not mind.

/Magnus

> >>>>> One other thing I noticed, which I need to research is that if I
> >>>>> use
> >>>>> rx_drop() function from /xdpsock_user.c it loops a lot in:
> >>>>>
> >>>>>           while (ret != rcvd) {
> >>>>>                   if (ret < 0) {
> >>>>>                           exit(-1);
> >>>>>           }
> >>>>>                   ret = xsk_ring_prod__reserve(&xsk->umem-
> >>> fq, rcvd, &idx_fq);
> >>>>>
> >>>>>           }
> >>>>>
> >>>>> As ret return 0, until (it looks like) I send more packets. So
> >>>>> even
> >>>>> in the poll() mode, it uses 100% cpu after sending a single
> >>>>> packet.
> >>>>> Note this is with the default Fedora Kernel, as I’m working on
> >>>>> this
> >>>>> from my laptop. Does this sound familiar? If not I’ll dig into
> >>>>> it
> >>>>> once I’m back.
> >>>>
> >>>> The xdpsock test is a busypolling test, to compare against DPDK
> >>>> speeds.  For real use-cases, I think people will want to trade-off
> >>>> latency vs. burning CPU.
> >>>
> >>> I understand the use case, but even with the xdpsock test program,
> >>> if
> >>> I send a single packet it’s not received, or at least not when
> >>> it's
> >>> sent. It takes 16 (or a multiple of it) before the get
> >>> detected/processed. I think it’s because of the
> >>> xsk_ring_prod__reserve(), but I’ll try to debug it more today and
> >>> to
> >>> understand the APIs better.
>