Re: ❌ FAIL: Stable queue: queue-5.3

[Laura Abbott <labbott@redhat.com>]

On 11/18/19 7:07 PM, Rachel Sibley wrote:
> 
> On 11/18/19 10:00 AM, CKI Project wrote:
>> Hello,
>>
>> We ran automated tests on a patchset that was proposed for merging into this
>> kernel tree. The patches were applied to:
>>
>>         Kernel repo:https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
>>              Commit: 116a395b7061 - Linux 5.3.11
>>
>> The results of these automated tests are provided below.
>>
>>      Overall result: FAILED (see details below)
>>               Merge: OK
>>             Compile: OK
>>               Tests: FAILED
>>
>> All kernel binaries, config files, and logs are available for download here:
>>
>>    https://artifacts.cki-project.org/pipelines/293063
>>
>> One or more kernel tests failed:
>>
>>      aarch64:
>>       ❌ LTP lite
> 
> I see a panic when installing the LTP dependencies
> 
> [  690.625060] Call trace:
> [  690.627495]  bfq_find_set_group+0x8c/0xf0
> [  690.631491]  bfq_bic_update_cgroup+0xbc/0x218
> [  690.635834]  bfq_init_rq+0xac/0x808
> [  690.639309]  bfq_insert_request.isra.0+0xe0/0x200
> [  690.643999]  bfq_insert_requests+0x68/0x88
> [  690.648085]  blk_mq_sched_insert_requests+0x84/0x140
> [  690.653036]  blk_mq_flush_plug_list+0x170/0x2b0
> [  690.657555]  blk_flush_plug_list+0xec/0x100
> [  690.661725]  blk_mq_make_request+0x200/0x5e8
> [  690.665982]  generic_make_request+0x94/0x270
> [  690.670239]  submit_bio+0x34/0x168
> [  690.673712]  xfs_submit_ioend.isra.0+0x9c/0x180 [xfs]
> [  690.678798]  xfs_do_writepage+0x234/0x458 [xfs]
> [  690.683318]  write_cache_pages+0x1a4/0x3f8
> [  690.687442]  xfs_vm_writepages+0x84/0xb8 [xfs]
> [  690.691874]  do_writepages+0x3c/0xe0
> [  690.695438]  __writeback_single_inode+0x48/0x440
> [  690.700042]  writeback_sb_inodes+0x1ec/0x4b0
> [  690.704298]  __writeback_inodes_wb+0x50/0xe8
> [  690.708555]  wb_writeback+0x264/0x388
> [  690.712204]  wb_do_writeback+0x300/0x358
> [  690.716113]  wb_workfn+0x80/0x1e0
> [  690.719418]  process_one_work+0x1bc/0x3e8
> [  690.723414]  worker_thread+0x54/0x440
> [  690.727064]  kthread+0x104/0x130
> [  690.730281]  ret_from_fork+0x10/0x18
> [  690.733847] Code: eb00007f 54000220 b4000040 f8568022 (f9401c42)
> [  690.739928] ---[ end trace d3fd392f569e86d3 ]---
> 
> https://artifacts.cki-project.org/pipelines/293063/logs/aarch64_host_2_console.log
> 

This looks like that same issue
https://bugzilla.redhat.com/show_bug.cgi?id=1767539

I don't think the BFQ fix has been sent to stable yet, or at least
it was not in 5.3.11

>> We hope that these logs can help you find the problem quickly. For the full
>> detail on our testing procedures, please scroll to the bottom of this message.
>>
>> Please reply to this email if you have any questions about the tests that we
>> ran or if you have any suggestions on how to make future tests more effective.
>>
>>          ,-.   ,-.
>>         ( C ) ( K )  Continuous
>>          `-',-.`-'   Kernel
>>            ( I )     Integration
>>             `-'
>> ______________________________________________________________________________
>>
>> Merge testing
>> -------------
>>
>> We cloned this repository and checked out the following commit:
>>
>>    Repo:https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
>>    Commit: 116a395b7061 - Linux 5.3.11
>>
>>
>> We grabbed the c4a7b00e0626 commit of the stable queue repository.
>>
>> We then merged the patchset with `git am`:
>>
>>    scsi-core-handle-drivers-which-set-sg_tablesize-to-zero.patch
>>    ax88172a-fix-information-leak-on-short-answers.patch
>>    devlink-disallow-reload-operation-during-device-cleanup.patch
>>    ipmr-fix-skb-headroom-in-ipmr_get_route.patch
>>    mlxsw-core-enable-devlink-reload-only-on-probe.patch
>>    net-gemini-add-missed-free_netdev.patch
>>    net-smc-fix-fastopen-for-non-blocking-connect.patch
>>    net-usb-qmi_wwan-add-support-for-foxconn-t77w968-lte-modules.patch
>>    slip-fix-memory-leak-in-slip_open-error-path.patch
>>    tcp-remove-redundant-new-line-from-tcp_event_sk_skb.patch
>>    dpaa2-eth-free-already-allocated-channels-on-probe-defer.patch
>>    devlink-add-method-for-time-stamp-on-reporter-s-dump.patch
>>    net-smc-fix-refcount-non-blocking-connect-part-2.patch
>>
>> Compile testing
>> ---------------
>>
>> We compiled the kernel for 3 architectures:
>>
>>      aarch64:
>>        make options: -j30 INSTALL_MOD_STRIP=1 targz-pkg
>>
>>      ppc64le:
>>        make options: -j30 INSTALL_MOD_STRIP=1 targz-pkg
>>
>>      x86_64:
>>        make options: -j30 INSTALL_MOD_STRIP=1 targz-pkg
>>
>>
>> Hardware testing
>> ----------------
>> We booted each kernel and ran the following tests:
>>
>>    aarch64:
>>      Host 1:
>>         ✅ Boot test
>>         🚧 ✅ selinux-policy: serge-testsuite
>>
>>      Host 2:
>>         ✅ Boot test
>>         ✅ Podman system integration test (as root)
>>         ✅ Podman system integration test (as user)
>>         ❌ LTP lite
>>         ⚡⚡⚡ jvm test suite
>>         ⚡⚡⚡ AMTU (Abstract Machine Test Utility)
>>         ⚡⚡⚡ LTP: openposix test suite
>>         ⚡⚡⚡ Ethernet drivers sanity
>>         ⚡⚡⚡ Networking socket: fuzz
>>         ⚡⚡⚡ audit: audit testsuite test
>>         ⚡⚡⚡ httpd: mod_ssl smoke sanity
>>         ⚡⚡⚡ iotop: sanity
>>         ⚡⚡⚡ tuned: tune-processes-through-perf
>>         ⚡⚡⚡ storage: SCSI VPD
>>         ⚡⚡⚡ stress: stress-ng
>>
>>    ppc64le:
>>
>>      ⚡ Internal infrastructure issues prevented one or more tests (marked
>>      with ⚡⚡⚡) from running on this architecture.
>>      This is not the fault of the kernel that was tested.
>>
>>    x86_64:
>>      Host 1:
>>         ✅ Boot test
>>         🚧 ✅ selinux-policy: serge-testsuite
>>
>>      Host 2:
>>         ✅ Boot test
>>         ✅ Podman system integration test (as root)
>>         ✅ Podman system integration test (as user)
>>         ✅ LTP lite
>>         ✅ jvm test suite
>>         ✅ AMTU (Abstract Machine Test Utility)
>>         ✅ LTP: openposix test suite
>>         ✅ Ethernet drivers sanity
>>         ✅ Networking socket: fuzz
>>         ✅ audit: audit testsuite test
>>         ✅ httpd: mod_ssl smoke sanity
>>         ✅ iotop: sanity
>>         ✅ tuned: tune-processes-through-perf
>>         ✅ pciutils: sanity smoke test
>>         ✅ storage: SCSI VPD
>>         ✅ stress: stress-ng
>>
>>    Test sources:https://github.com/CKI-project/tests-beaker
>>      💚 Pull requests are welcome for new tests or improvements to existing tests!
>>
>> Waived tests
>> ------------
>> If the test run included waived tests, they are marked with 🚧. Such tests are
>> executed but their results are not taken into account. Tests are waived when
>> their results are not reliable enough, e.g. when they're just introduced or are
>> being fixed.
>>
>> Testing timeout
>> ---------------
>> We aim to provide a report within reasonable timeframe. Tests that haven't
>> finished running are marked with ⏱. Reports for non-upstream kernels have
>> a Beaker recipe linked to next to each host.
>>