RE: [PATCH] testsuite: dpdk: Add l2fwd example which is based on DPDK.

["Chen, Hongzhan" <hongzhan.chen@intel.com>]

>-----Original Message-----
>From: Chen, Hongzhan <hongzhan.chen@intel.com>
>Sent: Thursday, February 2, 2023 8:37 AM
>To: Kiszka, Jan <jan.kiszka@siemens.com>; xenomai@lists.linux.dev
>Subject: RE: [PATCH] testsuite: dpdk: Add l2fwd example which is based on
>DPDK.
>
>
>
>>-----Original Message-----
>>From: Jan Kiszka <jan.kiszka@siemens.com>
>>Sent: Tuesday, January 31, 2023 3:27 PM
>>To: Chen, Hongzhan <hongzhan.chen@intel.com>; xenomai@lists.linux.dev
>>Subject: Re: [PATCH] testsuite: dpdk: Add l2fwd example which is based on
>>DPDK.
>>
>>On 31.01.23 06:10, Chen, Hongzhan wrote:
>>>
>>>
>>>> -----Original Message-----
>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>> Sent: Thursday, January 19, 2023 1:44 AM
>>>> To: Chen, Hongzhan <hongzhan.chen@intel.com>;
>>xenomai@lists.linux.dev
>>>> Subject: Re: [PATCH] testsuite: dpdk: Add l2fwd example which is based
>on
>>>> DPDK.
>>>>
>>>> On 18.01.23 14:05, Hongzhan Chen wrote:
>>>>> The L2 Forwarding sample application(l2fwd) is a simple
>>>>> example of packet processing using the Data Plane
>>>>> Development Kit (DPDK).
>>>>
>>>> Can you elaborate a bit on the envisioned use cases here and go more
>>>> into details in the accompanied documentation? The latter only describes
>>>> the "how", not at all the "why" (or "which use cases"). Some words on
>>>> the threading model (or limitations of it) would surely be good as well
>>>> ("how can I integrate this pattern into a real application?").
>>>>
>>>> Regarding the patch itself, you should go over it again and eliminate
>>>> debugging left-overs as well as unrelated changes (there is one at least
>>>> in testsuite/Makefile.am). The way of the integration looks fine to me
>>>> in principle.
>>>>
>>>> BTW, what prevents the usage of the VFIO driver so far?
>>>
>>> I have not looked into and tried VFIO driver so far. Last time you told me
>>that we may need to
>>> develop or port VFIO related drivers if we want to use VFIO-based dpdk in
>>Xenomai but UIO-based
>>> does not such issue. I maybe misunderstood what you guided. If the main
>>difference between uio and vfio is that vfio is capable of programming the
>>platform's IOMMU as described in [1], after vfio init, there is no linux syscall
>>involved during receiving and transmitting packets just as UIO case, we can
>do
>>same thing for VFIO with ported l2fwd.
>>
>>I haven't looked into VFIO myself, but the key question is whether some
>>userspace activity in the primary domain could trigger mapping activity
>>of the driver in the secondary one. If not, VFIO should be fine. If so,
>>and that was the case for some RTnet drivers, we would need to avoid
>>that. I was just wondering if you explored the path already and could
>>share your experience.
>
>According to my test, the patch works [1] on I255 NIC with vfio-pci module &

Sorry , typo, it is on I225

./dpdk-devbind.py -s

Network devices using DPDK-compatible driver
============================================
0000:03:00.0 'Device 15f2' drv=vfio-pci unused=

Regards

Hongzhan Chen

>iommu=pt
>& intel_iommu=on & VT-D enabled in BIOS. There is no mapping
>activities found during receiving and transmitting  packets at least in
>main loop of the l2fwd case after run about 17 hours and  I checked with
>strace log[2].
>
>Regards
>
>Hongzhan Chen
>
>[1]: cat /proc/xenomai/sched/stat
>
>CPU  PID    MSW        CSW        XSC        PF    STAT       %CPU  NAME
>  0  0      0          3          0          0     00218000  100.0  [ROOT/0]
>  1  0      0          72641414   0          0     00218000   99.9  [ROOT/1]
>  2  0      0          0          0          0     00218000  100.0  [ROOT/2]
>  3  0      0          0          0          0     00218000  100.0  [ROOT/3]
>  4  0      0          1          0          0     00218000  100.0  [ROOT/4]
>  5  0      0          2          0          0     00218000  100.0  [ROOT/5]
>  5  935    1          1          5          0     002680c0    0.0  l2fwd
>  1  954    6718       13436      6722       0     00268042    0.0  display-935
>  1  955    1          67171569   67178287   0     00248044    0.1  l2fwd
>
>[2]: Strace log:
>
>13:36:14.266949 futex(0x7ffed97d59b8,
>FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0,
>{tv_sec=1675258579, tv_nsec=0}, FUTEX_BITSET_MATCH_ANY) = 0
>13:36:14.266984 sched_get_priority_min(SCHED_FIFO) = 1
>13:36:14.267004 sched_get_priority_max(SCHED_FIFO) = 99
>13:36:14.267028 mmap(NULL, 69632, PROT_NONE,
>MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7f1e13fef000
>13:36:14.267102 mprotect(0x7f1e13ff0000, 65536, PROT_READ|PROT_WRITE)
>= 0
>13:36:14.267347 clone(child_stack=0x7f1e13ffddf0,
>flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD
>|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CL
>EARTID, parent_tidptr=0x7f1e13fff9d0, tls=0x7f1e13fff700,
>child_tidptr=0x7f1e13fff9d0) = 955
>13:36:14.267380 sched_setaffinity(955, 128, [1]) = 0
>13:36:14.267422 futex(0x7ffed97d59b8,
>FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0,
>{tv_sec=1675258579, tv_nsec=0}, FUTEX_BITSET_MATCH_ANY) = -1 EAGAIN
>(Resource temporarily unavailable)
>13:36:14.267480 rt_sigtimedwait([HUP INT ALRM TERM], {si_signo=SIGINT,
>si_code=SI_KERNEL}, NULL, 8) = 2 (SIGINT)
>08:16:50.286528 getpid()                = 935
>08:16:50.286556 tgkill(935, 954, SIGRTMIN) = 0
>08:16:50.286586 getpid()                = 935
>
>>
>>Jan
>>
>>>
>>> Regards
>>>
>>> Hongzhan Chen
>>>
>>> [1]:
>>https://spdk.io/doc/userspace.html#:~:text=The%20primary%20difference%
>2
>>0between%20uio,User%20Space%20for%20full%20details.
>>>>
>>>> Jan
>>>>
>>>> --
>>>> Siemens AG, Technology
>>>> Competence Center Embedded Linux
>>>
>>
>>--
>>Siemens AG, Technology
>>Competence Center Embedded Linux
>>