Thanks for the reply. I was able to reproduce it a couple of weeks ago and I have already submitted a fix. - Russ On 8/31/22 22:43, Yujie Liu wrote: > Hi Russ, > > On 8/2/2022 04:42, Russ Weight wrote: >> Oliver, >> >> On 7/29/22 00:08, kernel test robot wrote: >>> >>> Greeting, >>> >>> FYI, we noticed the following commit (built with gcc-11): >>> >>> commit: a37ddddd86037c896c702b4df416bc4e51b2a5a0 ("selftests: firmware: Add firmware upload selftests") >>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master >>> >>> in testcase: kernel-selftests >>> version: kernel-selftests-x86_64-4cb0bec3-1_20220724 >>> with following parameters: >>> >>>     group: firmware >>>     ucode: 0xec >>> >>> test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel. >>> test-url: https://www.kernel.org/doc/Documentation/kselftest.txt >>> >>> >>> on test machine: 8 threads Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz with 28G memory >>> >>> caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): >>> >>> >>> >>> If you fix the issue, kindly add following tag >>> Reported-by: kernel test robot >>> >>> >>> [ 103.520572][ T2443] BUG: KASAN: use-after-free in firmware_upload_unregister (drivers/base/firmware_loader/sysfs_upload.c:395) >>> [  103.520579][ T2443] Read of size 8 at addr ffff8881e186c808 by task fw_upload.sh/2443 >>> [  103.528481][  T395] >>> [  103.534696][ T2443] >>> [  103.534698][ T2443] CPU: 7 PID: 2443 Comm: fw_upload.sh Not tainted 5.18.0-rc2-00036-ga37ddddd8603 #1 >>> [  103.534701][ T2443] Hardware name: Dell Inc. OptiPlex 7040/0Y7WYT, BIOS 1.2.8 01/26/2016 >>> [  103.534703][ T2443] Call Trace: >>> [  103.534705][ T2443]  >>> [ 103.534707][ T2443] ? firmware_upload_unregister (drivers/base/firmware_loader/sysfs_upload.c:395) >> I believe I understand the problem, but I have been unable to reproduce the error to verify the fix: >> >> 394         device_unregister(&fw_sysfs->dev); >> 395         module_put(fw_upload_priv->module); >> >> The device_unregister() call could result in the dev_release >> function freeing the fw_upload_priv structure before it is >> dereferenced on line 395. Copying fw_upload_priv->module to a >> local variable for use when calling device_unregister() >> should fix the problem. >> >>> [ 103.534713][ T2443] dump_stack_lvl (lib/dump_stack.c:107 (discriminator 4)) >>> [ 103.588011][ T2443] print_address_description+0x1f/0x200 >>> [ 103.594406][ T2443] ? firmware_upload_unregister (drivers/base/firmware_loader/sysfs_upload.c:395) >>> [ 103.600112][ T2443] print_report.cold (mm/kasan/report.c:430) >>> [ 103.604782][ T2443] ? do_raw_spin_lock (arch/x86/include/asm/atomic.h:202 include/linux/atomic/atomic-instrumented.h:543 include/asm-generic/qspinlock.h:82 kernel/locking/spinlock_debug.c:115) >>> [ 103.609624][ T2443] kasan_report (mm/kasan/report.c:162 mm/kasan/report.c:493) >>> [ 103.613861][ T2443] ? firmware_upload_unregister (drivers/base/firmware_loader/sysfs_upload.c:395) >>> [ 103.619561][ T2443] firmware_upload_unregister (drivers/base/firmware_loader/sysfs_upload.c:395) >>> [ 103.625091][ T2443] upload_unregister_store (lib/test_firmware.c:1060 lib/test_firmware.c:1321) >>> [ 103.630377][ T2443] ? sysfs_file_ops (fs/sysfs/file.c:129) >>> [ 103.635046][ T2443] kernfs_fop_write_iter (fs/kernfs/file.c:294) >>> [ 103.640145][ T2443] new_sync_write (fs/read_write.c:505 (discriminator 1)) >>> [ 103.644642][ T2443] ? new_sync_read (fs/read_write.c:494) >>> [ 103.649225][ T2443] ? ksys_write (fs/read_write.c:644) >>> [ 103.653463][ T2443] ? rcu_read_unlock (include/linux/rcupdate.h:723 (discriminator 5)) >>> [ 103.658057][ T2443] ? lock_is_held_type (kernel/locking/lockdep.c:5382 kernel/locking/lockdep.c:5684) >>> [ 103.662909][ T2443] vfs_write (fs/read_write.c:591) >>> [ 103.666984][ T2443] ksys_write (fs/read_write.c:644) >>> [ 103.671057][ T2443] ? __ia32_sys_read (fs/read_write.c:634) >>> [ 103.675645][ T2443] ? lockdep_hardirqs_on_prepare (kernel/locking/lockdep.c:4501) >>> [ 103.682051][ T2443] ? syscall_enter_from_user_mode (arch/x86/include/asm/irqflags.h:45 arch/x86/include/asm/irqflags.h:80 kernel/entry/common.c:109) >>> [ 103.687756][ T2443] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) >>> [ 103.692010][ T2443] ? pick_file (fs/file.c:660) >>> [ 103.696165][ T2443] ? do_raw_spin_unlock (arch/x86/include/asm/atomic.h:29 include/linux/atomic/atomic-instrumented.h:28 include/asm-generic/qspinlock.h:28 kernel/locking/spinlock_debug.c:100 kernel/locking/spinlock_debug.c:140) >>> [ 103.701094][ T2443] ? syscall_exit_to_user_mode (kernel/entry/common.c:129 kernel/entry/common.c:296) >>> [ 103.706539][ T2443] ? lockdep_hardirqs_on_prepare (kernel/locking/lockdep.c:4501) >>> [ 103.712929][ T2443] ? do_syscall_64 (arch/x86/entry/common.c:87) >>> [ 103.717343][ T2443] ? lockdep_hardirqs_on_prepare (kernel/locking/lockdep.c:4501) >>> [ 103.723747][ T2443] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:115) >>> [  103.729451][ T2443] RIP: 0033:0x7f1020308f33 >>> [ 103.733709][ T2443] Code: 8b 15 61 ef 0c 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 64 8b 04 25 18 00 00 00 85 c0 75 14 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 55 c3 0f 1f 40 00 48 83 ec 28 48 89 54 24 18 >>> All code >>> ======== >>>     0:    8b 15 61 ef 0c 00        mov    0xcef61(%rip),%edx        # 0xcef67 >>>     6:    f7 d8                    neg    %eax >>>     8:    64 89 02                 mov    %eax,%fs:(%rdx) >>>     b:    48 c7 c0 ff ff ff ff     mov    $0xffffffffffffffff,%rax >>>    12:    eb b7                    jmp    0xffffffffffffffcb >>>    14:    0f 1f 00                 nopl   (%rax) >>>    17:    64 8b 04 25 18 00 00     mov    %fs:0x18,%eax >>>    1e:    00 >>>    1f:    85 c0                    test   %eax,%eax >>>    21:    75 14                    jne    0x37 >>>    23:    b8 01 00 00 00           mov    $0x1,%eax >>>    28:    0f 05                    syscall >>>    2a:*    48 3d 00 f0 ff ff        cmp    $0xfffffffffffff000,%rax        <-- trapping instruction >>>    30:    77 55                    ja     0x87 >>>    32:    c3                       retq >>>    33:    0f 1f 40 00              nopl   0x0(%rax) >>>    37:    48 83 ec 28              sub    $0x28,%rsp >>>    3b:    48 89 54 24 18           mov    %rdx,0x18(%rsp) >>> >>> Code starting with the faulting instruction >>> =========================================== >>>     0:    48 3d 00 f0 ff ff        cmp    $0xfffffffffffff000,%rax >>>     6:    77 55                    ja     0x5d >>>     8:    c3                       retq >>>     9:    0f 1f 40 00              nopl   0x0(%rax) >>>     d:    48 83 ec 28              sub    $0x28,%rsp >>>    11:    48 89 54 24 18           mov    %rdx,0x18(%rsp) >>> [  103.753040][ T2443] RSP: 002b:00007fffe4075988 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 >>> [  103.761244][ T2443] RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007f1020308f33 >>> [  103.769013][ T2443] RDX: 0000000000000003 RSI: 00005582df845b80 RDI: 0000000000000001 >>> [  103.776791][ T2443] RBP: 00005582df845b80 R08: 00000000ffffffff R09: 0000000000000003 >>> [  103.784561][ T2443] R10: 00005582df833c80 R11: 0000000000000246 R12: 0000000000000003 >>> [  103.792328][ T2443] R13: 00007f10203d96a0 R14: 0000000000000003 R15: 00007f10203d98a0 >>> [  103.800100][ T2443]  >>> [  103.802957][ T2443] >>> [  103.805125][ T2443] Allocated by task 2443: >>> [ 103.809276][ T2443] kasan_save_stack (mm/kasan/common.c:39) >>> [ 103.813781][ T2443] __kasan_kmalloc (mm/kasan/common.c:45 mm/kasan/common.c:436 mm/kasan/common.c:515 mm/kasan/common.c:524) >>> [ 103.818190][ T2443] firmware_upload_register (drivers/base/firmware_loader/sysfs_upload.c:160) >>> [ 103.824150][ T2443] upload_register_store (lib/test_firmware.c:1279) >>> [ 103.829250][ T2443] kernfs_fop_write_iter (fs/kernfs/file.c:294) >>> [ 103.834350][ T2443] new_sync_write (fs/read_write.c:505 (discriminator 1)) >>> [ 103.838846][ T2443] vfs_write (fs/read_write.c:591) >>> [ 103.842910][ T2443] ksys_write (fs/read_write.c:644) >>> [ 103.846975][ T2443] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) >>> [ 103.851217][ T2443] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:115) >>> [  103.856932][ T2443] >>> [  103.859100][ T2443] Freed by task 2443: >>> [ 103.862907][ T2443] kasan_save_stack (mm/kasan/common.c:39) >>> [ 103.867415][ T2443] kasan_set_track (mm/kasan/common.c:45) >>> [ 103.871822][ T2443] kasan_set_free_info (mm/kasan/generic.c:372) >>> [ 103.876579][ T2443] __kasan_slab_free (mm/kasan/common.c:368 mm/kasan/common.c:328 mm/kasan/common.c:374) >>> [ 103.881331][ T2443] slab_free_freelist_hook (mm/slub.c:1754) >>> [ 103.886517][ T2443] kfree (mm/slub.c:3510 mm/slub.c:4552) >>> [ 103.890156][ T2443] fw_dev_release (drivers/base/firmware_loader/sysfs.c:102) >>> [ 103.894483][ T2443] device_release (drivers/base/core.c:2235) >>> [ 103.898902][ T2443] kobject_cleanup (lib/kobject.c:677) >>> [ 103.903492][ T2443] firmware_upload_unregister (drivers/base/firmware_loader/sysfs_upload.c:395) >>> [ 103.909034][ T2443] upload_unregister_store (lib/test_firmware.c:1060 lib/test_firmware.c:1321) >>> [ 103.914311][ T2443] kernfs_fop_write_iter (fs/kernfs/file.c:294) >>> [ 103.919429][ T2443] new_sync_write (fs/read_write.c:505 (discriminator 1)) >>> [ 103.923927][ T2443] vfs_write (fs/read_write.c:591) >>> [ 103.927990][ T2443] ksys_write (fs/read_write.c:644) >>> [ 103.932054][ T2443] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) >>> [ 103.936290][ T2443] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:115) >>> [  103.941992][ T2443] >>> [  103.944159][ T2443] Last potentially related work creation: >>> [ 103.949704][ T2443] kasan_save_stack (mm/kasan/common.c:39) >>> [ 103.954216][ T2443] __kasan_record_aux_stack (mm/kasan/generic.c:348) >>> [ 103.959400][ T2443] insert_work (include/linux/instrumented.h:71 include/asm-generic/bitops/instrumented-non-atomic.h:134 kernel/workqueue.c:635 kernel/workqueue.c:642 kernel/workqueue.c:1361) >>> [ 103.963552][ T2443] __queue_work (kernel/workqueue.c:1520) >>> [ 103.967888][ T2443] queue_work_on (kernel/workqueue.c:1546) >>> [ 103.972141][ T2443] fw_upload_start (drivers/base/firmware_loader/sysfs_upload.c:263) >>> [ 103.976723][ T2443] firmware_loading_store (drivers/base/firmware_loader/sysfs.c:213) >>> [ 103.981910][ T2443] kernfs_fop_write_iter (fs/kernfs/file.c:294) >>> [ 103.987022][ T2443] new_sync_write (fs/read_write.c:505 (discriminator 1)) >>> [ 103.991537][ T2443] vfs_write (fs/read_write.c:591) >>> [ 103.995604][ T2443] ksys_write (fs/read_write.c:644) >>> [ 103.999673][ T2443] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) >>> [ 104.003930][ T2443] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:115) >>> [  104.009631][ T2443] >>> [  104.011800][ T2443] Second to last potentially related work creation: >>> [ 104.018219][ T2443] kasan_save_stack (mm/kasan/common.c:39) >>> [ 104.022727][ T2443] __kasan_record_aux_stack (mm/kasan/generic.c:348) >>> [ 104.027938][ T2443] insert_work (include/linux/instrumented.h:71 include/asm-generic/bitops/instrumented-non-atomic.h:134 kernel/workqueue.c:635 kernel/workqueue.c:642 kernel/workqueue.c:1361) >>> [ 104.032101][ T2443] __queue_work (kernel/workqueue.c:1520) >>> [ 104.036423][ T2443] queue_work_on (kernel/workqueue.c:1546) >>> [ 104.040658][ T2443] fw_upload_start (drivers/base/firmware_loader/sysfs_upload.c:263) >>> [ 104.045240][ T2443] firmware_loading_store (drivers/base/firmware_loader/sysfs.c:213) >>> [ 104.050423][ T2443] kernfs_fop_write_iter (fs/kernfs/file.c:294) >>> [ 104.055522][ T2443] new_sync_write (fs/read_write.c:505 (discriminator 1)) >>> [ 104.060016][ T2443] vfs_write (fs/read_write.c:591) >>> [ 104.064081][ T2443] ksys_write (fs/read_write.c:644) >>> [ 104.068144][ T2443] do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) >>> [ 104.072381][ T2443] entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:115) >>> [  104.078083][ T2443] >>> [  104.080249][ T2443] The buggy address belongs to the object at ffff8881e186c800 >>> [  104.080249][ T2443]  which belongs to the cache kmalloc-512 of size 512 >>> [  104.094049][ T2443] The buggy address is located 8 bytes inside of >>> [  104.094049][ T2443]  512-byte region [ffff8881e186c800, ffff8881e186ca00) >>> [  104.106914][ T2443] >>> [  104.109084][ T2443] The buggy address belongs to the physical page: >>> [  104.115315][ T2443] page:0000000037a5888d refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x1e1868 >>> [  104.125336][ T2443] head:0000000037a5888d order:3 compound_mapcount:0 compound_pincount:0 >>> [  104.133454][ T2443] flags: 0x17ffffc0010200(slab|head|node=0|zone=2|lastcpupid=0x1fffff) >>> [  104.141498][ T2443] raw: 0017ffffc0010200 ffffea0005491e00 dead000000000002 ffff888100042c80 >>> [  104.149885][ T2443] raw: 0000000000000000 0000000000200020 00000001ffffffff 0000000000000000 >>> [  104.158274][ T2443] page dumped because: kasan: bad access detected >>> [  104.164492][ T2443] >>> >>> >>> To reproduce: >>> >>>          git clone https://github.com/intel/lkp-tests.git >>>          cd lkp-tests >>>          sudo bin/lkp install job.yaml           # job file is attached in this email >> I have tried these steps on Fedora35 and on CentOS Stream. In both >> cases I have missing packages that I have not yet resolved: >> >> Error: Unable to find a match: arping lib32gcc-dev libc6-dev-i386 libc6-i386 libc6-x32 libhugetlbfs-dev libmnl-dev libmount-dev libpci3 libreadline-dev libx32asan5 libx32atomic1 libx32gcc1 libx32gcc-dev libx32gomp1 libx32itm1 libx32quadmath0 libx32ubsan1 linux-libc-dev-amd64-cross netcat-openbsd openvswitch-common openvswitch-switch sendip libpci-dev >> >> Simply running the fw_upload selftests in a loop is not sufficient to >> trigger the error. Can you provide additional instructions for >> running the lkp tests manually? Do I need a specific OS? How can I >> access the missing packages? > > We can reproduce this error on bare metal by simply running fw_upload.sh. > > ~/linux/tools/testing/selftests/firmware# ./fw_upload.sh > ./fw_upload.sh: firmware upload cancellation works > ./fw_upload.sh: firmware upload error handling works > ./fw_upload.sh: oversized firmware error handling works > ./fw_upload.sh: firmware upload for fw1 works > ./fw_upload.sh: firmware upload for fw2 works > ./fw_upload.sh: firmware upload for fw3 works > > Message from syslogd(a)debian-x8664 at Sep  1 05:06:54 ... >  kernel:[ 1090.872590][ T1293] Kernel panic - not syncing: Fatal exception > > dmesg read from serial: > > [ 1089.654274][ T1293] ================================================================== > [ 1089.662192][ T1293] BUG: KASAN: use-after-free in firmware_upload_unregister+0x16e/0x1c0 > [ 1089.670282][ T1293] Read of size 8 at addr ffff88873a872008 by task fw_upload.sh/1293 > [ 1089.678107][ T1293] > [ 1089.680291][ T1293] CPU: 4 PID: 1293 Comm: fw_upload.sh Not tainted 5.18.0-rc2-00036-ga37ddddd8603 #1 > [ 1089.689527][ T1293] Hardware name: Dell Inc. OptiPlex 7040/0Y7WYT, BIOS 1.2.8 01/26/2016 > [ 1089.697612][ T1293] Call Trace: > [ 1089.700751][ T1293]  > [ 1089.703557][ T1293]  ? firmware_upload_unregister+0x16e/0x1c0 > [ 1089.709313][ T1293]  dump_stack_lvl+0x45/0x59 > [ 1089.713669][ T1293]  print_address_description.constprop.0+0x1f/0x200 > [ 1089.720107][ T1293]  ? firmware_upload_unregister+0x16e/0x1c0 > [ 1089.725851][ T1293]  print_report.cold+0x55/0x22c > [ 1089.730556][ T1293]  ? do_raw_spin_lock+0x12e/0x280 > [ 1089.735432][ T1293]  kasan_report+0xbe/0x1c0 > > > Could you please help check if the .config file used to compile the kernel > matches the one we attached in the original report? Here we attach it again > for your reference. > > > About the issue of missing packages during setting up lkp test environment, > we wish to support various OS and distributions, but sometimes our package > dependencies are not updated in time, sorry for the inconvenience. We use > debian OS in our test environment, and it can install the required packages > successfully. We will fix the package issue on other OS soon. For this case, > we still recommend to run fw_upload.sh to trigger the error, because it's > much easier than setting up lkp tests environment. > > -- > Thanks, > Yujie > >> >> Thanks, >> - Russ >>>          bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run >>>          sudo bin/lkp run generated-yaml-file >>> >>>          # if come across any failure that blocks the test, >>>          # please remove ~/.lkp and /lkp dir to run from a clean state. >>> >>> >>> >>