Re: ❌ FAIL: Test report for kernel 5.10.0-rc6 (mainline.kernel.org)

[Jianlin Shi <jishi@redhat.com>]

Hi ,

I reported a bug in bugzilla.kernel.org for geneve issue:
https://bugzilla.kernel.org/show_bug.cgi?id=210569

Thanks & Best Regards,
Jianlin Shi


On Tue, Dec 8, 2020 at 9:38 AM Jianlin Shi <jishi@redhat.com> wrote:
>
> Hi ,
>
>
> On Tue, Dec 8, 2020 at 8:25 AM CKI Project <cki-project@redhat.com> wrote:
>>
>>
>> Hello,
>>
>> We ran automated tests on a recent commit from this kernel tree:
>>
>>        Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
>>             Commit: 7059c2c00a21 - Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/dtor/input
>>
>> The results of these automated tests are provided below.
>>
>>     Overall result: FAILED (see details below)
>>              Merge: OK
>>            Compile: OK
>>  Selftests compile: FAILED
>>              Tests: FAILED
>>
>>     Pipeline: https://xci32.lab.eng.rdu2.redhat.com/cki-project/cki-pipeline/-/pipelines/619303
>>
>>     Check out our dashboard including known failures tagged by our bot at
>>       https://datawarehouse.internal.cki-project.org/kcidb/revisions/7748
>>
>> One or more kernel tests failed:
>>
>>     s390x:
>>      ❌ LTP
>>      ❌ Networking tunnel: geneve basic test
>>
>>     ppc64le:
>>      ❌ Boot test
>>      ❌ LTP
>>      ❌ Networking tunnel: geneve basic test
>>
>>     aarch64:
>>      ❌ storage: software RAID testing
>>      ❌ LTP
>>      ❌ Networking tunnel: geneve basic test
>>
>>     x86_64:
>>      ❌ LTP
>>      ❌ Networking tunnel: geneve basic test
>
>
> the basic traffic over geneve would fail, could you help to check. Should we report a bug or not?
>
>
> ip netns add client
>
> ip netns add server
>
>
>
> ip link add veth0_c type veth peer name veth0_s
>
> ip link set veth0_c netns client
>
> ip link set veth0_s netns server
>
>
>
> ip netns exec client ip link set lo up
>
> ip netns exec client ip link set veth0_c up
>
>
>
> ip netns exec server ip link set lo up
>
> ip netns exec server ip link set veth0_s up
>
>
>
>
>
> ip netns exec client ip addr add 2000::1/64 dev veth0_c
>
> ip netns exec client ip addr add 10.10.0.1/24 dev veth0_c
>
>
>
> ip netns exec server ip addr add 2000::2/64 dev veth0_s
>
> ip netns exec server ip addr add 10.10.0.2/24 dev veth0_s
>
>
>
> ip netns exec client ping 10.10.0.2 -c 2
>
> ip netns exec client ping6 2000::2 -c 2
>
>
>
> ip netns exec client ip link add geneve1 type geneve vni 1234 remote 10.10.0.2 ttl 64
>
> ip netns exec server ip link add geneve1 type geneve vni 1234 remote 10.10.0.1 ttl 64
>
>
>
> ip netns exec client ip link set geneve1 up
>
> ip netns exec client ip addr add 1.1.1.1/24 dev geneve1
>
> ip netns exec server ip link set geneve1 up
>
> ip netns exec server ip addr add 1.1.1.2/24 dev geneve1
> ip netns exec client ping 1.1.1.2 -c 3
>
>
>>
>>      ❌ storage: software RAID testing
>>
>> We hope that these logs can help you find the problem quickly. For the full
>> detail on our testing procedures, please scroll to the bottom of this message.
>>
>> Please reply to this email if you have any questions about the tests that we
>> ran or if you have any suggestions on how to make future tests more effective.
>>
>>         ,-.   ,-.
>>        ( C ) ( K )  Continuous
>>         `-',-.`-'   Kernel
>>           ( I )     Integration
>>            `-'
>> ______________________________________________________________________________
>>
>> Compile testing
>> ---------------
>>
>> We compiled the kernel for 4 architectures:
>>
>>     aarch64:
>>       make options: make -j30 INSTALL_MOD_STRIP=1 targz-pkg
>>
>>     ppc64le:
>>       make options: make -j30 INSTALL_MOD_STRIP=1 targz-pkg
>>
>>     s390x:
>>       make options: make -j30 INSTALL_MOD_STRIP=1 targz-pkg
>>
>>     x86_64:
>>       make options: make -j30 INSTALL_MOD_STRIP=1 targz-pkg
>>
>>
>> We built the following selftests:
>>
>>   x86_64:
>>       net: OK
>>       bpf: fail
>>       install and packaging: OK
>>
>> You can find the full log (build-selftests.log) in the artifact storage above.
>>
>>
>> Hardware testing
>> ----------------
>> All the testing jobs are listed here:
>>
>>   https://beaker.engineering.redhat.com/jobs/?jobsearch-0.table=Whiteboard&jobsearch-0.operation=contains&jobsearch-0.value=cki%40gitlab%3A619303
>>
>> We booted each kernel and ran the following tests:
>>
>>   aarch64:
>>     Host 1: https://beaker.engineering.redhat.com/recipes/9156933
>>        ✅ Boot test
>>        ✅ selinux-policy: serge-testsuite
>>        ❌ storage: software RAID testing
>>        ✅ stress: stress-ng
>>         ❌ xfstests - ext4
>>         ✅ xfstests - xfs
>>         ✅ xfstests - btrfs
>>         ❌ IPMI driver test
>>         ✅ IPMItool loop stress test
>>         ✅ Storage blktests
>>         ✅ Storage block - filesystem fio test
>>         ✅ Storage block - queue scheduler test
>>         ✅ Storage nvme - tcp
>>         ✅ Storage: swraid mdadm raid_module test
>>
>>     Host 2: https://beaker.engineering.redhat.com/recipes/9156932
>>        ✅ Boot test
>>        ✅ ACPI table test
>>        ✅ ACPI enabled test
>>        ❌ LTP
>>        ✅ Loopdev Sanity
>>        ✅ Memory: fork_mem
>>        ✅ Memory function: memfd_create
>>        ✅ AMTU (Abstract Machine Test Utility)
>>        ✅ Networking bridge: sanity
>>        ✅ Networking socket: fuzz
>>        ✅ Networking: igmp conformance test
>>        ✅ Networking route: pmtu
>>        ✅ Networking route_func - local
>>        ✅ Networking route_func - forward
>>        ✅ Networking TCP: keepalive test
>>        ✅ Networking UDP: socket
>>        ❌ Networking tunnel: geneve basic test
>>        ✅ Networking tunnel: gre basic
>>        ✅ L2TP basic test
>>        ✅ Networking tunnel: vxlan basic
>>        ✅ Networking ipsec: basic netns - transport
>>        ✅ Networking ipsec: basic netns - tunnel
>>        ✅ Libkcapi AF_ALG test
>>        ✅ pciutils: update pci ids test
>>        ✅ ALSA PCM loopback test
>>        ✅ ALSA Control (mixer) Userspace Element test
>>        ✅ storage: SCSI VPD
>>         ✅ CIFS Connectathon
>>         ✅ POSIX pjd-fstest suites
>>         ✅ Firmware test suite
>>         ✅ jvm - jcstress tests
>>         ✅ Memory function: kaslr
>>         ✅ Ethernet drivers sanity
>>         ✅ Networking firewall: basic netfilter test
>>         ✅ audit: audit testsuite test
>>         ✅ trace: ftrace/tracer
>>         ✅ kdump - kexec_boot
>>
>>   ppc64le:
>>     Host 1: https://beaker.engineering.redhat.com/recipes/9156935
>>        ❌ Boot test
>>        ⚡⚡⚡ selinux-policy: serge-testsuite
>>        ⚡⚡⚡ storage: software RAID testing
>>         ⚡⚡⚡ xfstests - ext4
>>         ⚡⚡⚡ xfstests - xfs
>>         ⚡⚡⚡ xfstests - btrfs
>>         ⚡⚡⚡ IPMI driver test
>>         ⚡⚡⚡ IPMItool loop stress test
>>         ⚡⚡⚡ Storage blktests
>>         ⚡⚡⚡ Storage block - filesystem fio test
>>         ⚡⚡⚡ Storage block - queue scheduler test
>>         ⚡⚡⚡ Storage nvme - tcp
>>         ⚡⚡⚡ Storage: swraid mdadm raid_module test
>>
>>     Host 2: https://beaker.engineering.redhat.com/recipes/9156934
>>        ✅ Boot test
>>        ❌ LTP
>>        ✅ Loopdev Sanity
>>        ✅ Memory: fork_mem
>>        ✅ Memory function: memfd_create
>>        ✅ AMTU (Abstract Machine Test Utility)
>>        ✅ Networking bridge: sanity
>>        ✅ Networking socket: fuzz
>>        ✅ Networking route: pmtu
>>        ✅ Networking route_func - local
>>        ✅ Networking route_func - forward
>>        ✅ Networking TCP: keepalive test
>>        ✅ Networking UDP: socket
>>        ❌ Networking tunnel: geneve basic test
>>        ✅ Networking tunnel: gre basic
>>        ✅ L2TP basic test
>>        ✅ Networking tunnel: vxlan basic
>>        ✅ Networking ipsec: basic netns - tunnel
>>        ✅ Libkcapi AF_ALG test
>>        ✅ pciutils: update pci ids test
>>        ✅ ALSA PCM loopback test
>>        ✅ ALSA Control (mixer) Userspace Element test
>>         ✅ CIFS Connectathon
>>         ✅ POSIX pjd-fstest suites
>>         ✅ jvm - jcstress tests
>>         ✅ Memory function: kaslr
>>         ✅ Ethernet drivers sanity
>>         ✅ Networking firewall: basic netfilter test
>>         ✅ audit: audit testsuite test
>>         ✅ trace: ftrace/tracer
>>
>>   s390x:
>>     Host 1: https://beaker.engineering.redhat.com/recipes/9156940
>>        ✅ Boot test
>>        ✅ selinux-policy: serge-testsuite
>>        ✅ stress: stress-ng
>>         ✅ Storage blktests
>>         ❌ Storage nvme - tcp
>>         ✅ Storage: swraid mdadm raid_module test
>>
>>     Host 2: https://beaker.engineering.redhat.com/recipes/9156939
>>        ✅ Boot test
>>        ❌ LTP
>>        ✅ Loopdev Sanity
>>        ✅ Memory: fork_mem
>>        ✅ Memory function: memfd_create
>>        ✅ AMTU (Abstract Machine Test Utility)
>>        ✅ Networking bridge: sanity
>>        ✅ Networking route: pmtu
>>        ✅ Networking route_func - local
>>        ✅ Networking route_func - forward
>>        ✅ Networking TCP: keepalive test
>>        ✅ Networking UDP: socket
>>        ❌ Networking tunnel: geneve basic test
>>        ✅ Networking tunnel: gre basic
>>        ✅ L2TP basic test
>>        ✅ Networking tunnel: vxlan basic
>>        ✅ Networking ipsec: basic netns - transport
>>        ✅ Networking ipsec: basic netns - tunnel
>>        ✅ Libkcapi AF_ALG test
>>         ✅ CIFS Connectathon
>>         ✅ POSIX pjd-fstest suites
>>         ✅ jvm - jcstress tests
>>         ✅ Memory function: kaslr
>>         ✅ Ethernet drivers sanity
>>         ✅ Networking firewall: basic netfilter test
>>         ❌ audit: audit testsuite test
>>         ✅ trace: ftrace/tracer
>>
>>   x86_64:
>>     Host 1: https://beaker.engineering.redhat.com/recipes/9156936
>>        ✅ Boot test
>>        ✅ ACPI table test
>>        ❌ LTP
>>        ✅ Loopdev Sanity
>>        ✅ Memory: fork_mem
>>        ✅ Memory function: memfd_create
>>        ✅ AMTU (Abstract Machine Test Utility)
>>        ✅ Networking bridge: sanity
>>        ✅ Networking socket: fuzz
>>        ✅ Networking: igmp conformance test
>>        ✅ Networking route: pmtu
>>        ✅ Networking route_func - local
>>        ✅ Networking route_func - forward
>>        ✅ Networking TCP: keepalive test
>>        ✅ Networking UDP: socket
>>        ❌ Networking tunnel: geneve basic test
>>        ✅ Networking tunnel: gre basic
>>        ✅ L2TP basic test
>>        ✅ Networking tunnel: vxlan basic
>>        ✅ Networking ipsec: basic netns - transport
>>        ✅ Networking ipsec: basic netns - tunnel
>>        ✅ Libkcapi AF_ALG test
>>        ✅ pciutils: sanity smoke test
>>        ✅ pciutils: update pci ids test
>>        ✅ ALSA PCM loopback test
>>        ✅ ALSA Control (mixer) Userspace Element test
>>        ✅ storage: SCSI VPD
>>         ✅ CIFS Connectathon
>>         ✅ POSIX pjd-fstest suites
>>         ✅ Firmware test suite
>>         ✅ jvm - jcstress tests
>>         ✅ Memory function: kaslr
>>         ✅ Ethernet drivers sanity
>>         ✅ Networking firewall: basic netfilter test
>>         ✅ audit: audit testsuite test
>>         ✅ trace: ftrace/tracer
>>         ✅ kdump - kexec_boot
>>
>>     Host 2: https://beaker.engineering.redhat.com/recipes/9156937
>>        ✅ Boot test
>>         ✅ kdump - sysrq-c
>>         ✅ kdump - file-load
>>
>>     Host 3: https://beaker.engineering.redhat.com/recipes/9156938
>>
>>        ⚡ Internal infrastructure issues prevented one or more tests (marked
>>        with ⚡⚡⚡) from running on this architecture.
>>        This is not the fault of the kernel that was tested.
>>
>>        ✅ Boot test
>>        ✅ selinux-policy: serge-testsuite
>>        ❌ storage: software RAID testing
>>        ✅ stress: stress-ng
>>         ❌ CPU: Frequency Driver Test
>>         ✅ CPU: Idle Test
>>         ❌ xfstests - ext4
>>         ✅ xfstests - xfs
>>         ✅ xfstests - btrfs
>>         ⚡⚡⚡ IPMI driver test
>>         ⚡⚡⚡ IPMItool loop stress test
>>         ⚡⚡⚡ power-management: cpupower/sanity test
>>         ⚡⚡⚡ Storage blktests
>>         ⚡⚡⚡ Storage block - filesystem fio test
>>         ⚡⚡⚡ Storage block - queue scheduler test
>>         ⚡⚡⚡ Storage nvme - tcp
>>         ⚡⚡⚡ Storage: swraid mdadm raid_module test
>>
>>   Test sources: https://gitlab.com/cki-project/kernel-tests
>>     Pull requests are welcome for new tests or improvements to existing tests!
>>
>> Aborted tests
>> -------------
>> Tests that didn't complete running successfully are marked with ⚡⚡⚡.
>> If this was caused by an infrastructure issue, we try to mark that
>> explicitly in the report.
>>
>> Waived tests
>> ------------
>> If the test run included waived tests, they are marked with . Such tests are
>> executed but their results are not taken into account. Tests are waived when
>> their results are not reliable enough, e.g. when they're just introduced or are
>> being fixed.
>>
>> Testing timeout
>> ---------------
>> We aim to provide a report within reasonable timeframe. Tests that haven't
>> finished running yet are marked with ⏱.
>>
>> Reproducing results
>> -------------------
>> Click on a link below to access a web page that allows you to adjust the
>> Beaker job and re-run any failed tests. These links are generated for
>> failed or aborted tests that are not waived. Please adjust the Beaker
>> job whiteboard string in the web page so that it is easy for you to find
>> and so that it is not confused with a regular CKI job.
>>
>> After clicking the "Submit the job!" button, a dialog will open that should
>> contain a link to the newly submitted Beaker job.
>>
>>   https://beaker-respin.internal.cki-project.org/respin?whiteboard=respin_job_4794082&recipe_id=9156933&recipe_id=9156932&job_id=4794082
>>   https://beaker-respin.internal.cki-project.org/respin?whiteboard=respin_job_4794083&recipe_id=9156935&recipe_id=9156934&job_id=4794083
>>   https://beaker-respin.internal.cki-project.org/respin?whiteboard=respin_job_4794085&recipe_id=9156939&job_id=4794085
>>   https://beaker-respin.internal.cki-project.org/respin?whiteboard=respin_job_4794084&recipe_id=9156936&recipe_id=9156938&job_id=4794084
>>