warning on PREEMPT_RT kernel

warning on PREEMPT_RT kernel

[Maurizio Lombardi]

Hello,

FYI, I hit this warning when target is used on a real-time kernel:

[ 7757.735755] WARNING: CPU: 1 PID: 829784 at
drivers/target/target_core_transport.c:3013
__transport_wait_for_tasks+0x122/0x150 [target_core_mod]
[ 7757.735757] Modules linked in: iscsi_tcp libiscsi_tcp libiscsi
target_core_user uio target_core_pscsi target_core_file
target_core_iblock iscsi_target_mod target_core_mod xt_multiport
ip_gre ip_tunnel gre loop tun af_key crypto_user scsi_transport_iscsi
bluetooth ecdh_generic rfkill overlay xt_CONNSECMARK xt_SECMARK
nft_counter xt_state xt_conntrack nft_compat ah6 ah4 nft_objref nft_ct
nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables_set nf_tables
nfnetlink vfat fat sctp nfsd nfs_acl dm_log_writes dm_flakey dm_mod
rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache
sunrpc intel_rapl_msr intel_rapl_common sb_edac x86_pkg_temp_thermal
intel_powerclamp coretemp crct10dif_pclmul crc32_pclmul ipmi_ssif
ghash_clmulni_intel intel_cstate iTCO_wdt intel_uncore
iTCO_vendor_support ext4 intel_rapl_perf mbcache pcspkr jbd2 hpilo
hpwdt ipmi_si ipmi_devintf ipmi_msghandler acpi_tad ioatdma i2c_i801
lpc_ich wmi acpi_power_meter ip_tables xfs libcrc32c sd_mod sg mgag200
[ 7757.735797]  drm_kms_helper syscopyarea sysfillrect sysimgblt
fb_sys_fops drm_vram_helper drm_ttm_helper ttm igb crc32c_intel
serio_raw drm dca hpsa i2c_algo_bit scsi_transport_sas [last unloaded:
raid10]
[ 7757.735807] CPU: 1 PID: 829784 Comm: iscsi_ttx Kdump: loaded
Tainted: G           OE    --------- -  -
4.18.0-240.22.1.rt7.77.el8_3.mreq409.x86_64 #1
[ 7757.735809] Hardware name: HP ProLiant XL420 Gen9/ProLiant XL420
Gen9, BIOS U19 12/27/2015
[ 7757.735818] RIP: 0010:__transport_wait_for_tasks+0x122/0x150
[target_core_mod]
[ 7757.735820] Code: 04 01 00 00 dd b8 01 00 00 00 f6 05 81 57 03 00
04 75 1e 48 83 c4 08 5b 5d 41 5c 41 5d 41 5e c3 c6 01 01 e9 39 ff ff
ff 0f 0b <0f> 0b e9 06 ff ff ff 48 8b 53 08 48 c7 c6 b0 db c4 c0 48 c7
c7 f8
[ 7757.735821] RSP: 0018:ffffa89f48e53d68 EFLAGS: 00010202
[ 7757.735823] RAX: 0000000000000286 RBX: ffff932daf8a58e8 RCX: ffffa89f48e53da7
[ 7757.735824] RDX: ffffa89f48e53da6 RSI: 0000000000000001 RDI: ffff932daf8a58e8
[ 7757.735825] RBP: ffff932daf8a59f0 R08: ffffa89f48e53da8 R09: ffffffffc0c9201f
[ 7757.735825] R10: 0000000000000200 R11: 0000000000000000 R12: 0000000000000001
[ 7757.735826] R13: 0000000000000001 R14: ffffa89f48e53e48 R15: ffffa89f48e53b70
[ 7757.735828] FS:  0000000000000000(0000) GS:ffff932a6f640000(0000)
knlGS:0000000000000000
[ 7757.735828] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7757.735829] CR2: 0000555556555e88 CR3: 000000079240e003 CR4: 00000000001606e0
[ 7757.735830] Call Trace:
[ 7757.735839]  ? migrate_disable+0x38/0xc0
[ 7757.735850]  transport_generic_free_cmd+0xec/0x160 [target_core_mod]
[ 7757.735869]  iscsit_free_cmd+0x56/0xc0 [iscsi_target_mod]
[ 7757.735877]  iscsit_close_connection+0x70c/0xad0 [iscsi_target_mod]
[ 7757.735885]  ? iscsit_thread_get_cpumask+0x70/0x70 [iscsi_target_mod]
[ 7757.735891]  iscsit_take_action_for_connection_exit+0xb4/0x180
[iscsi_target_mod]
[ 7757.735897]  iscsi_target_tx_thread+0x16f/0x1f0 [iscsi_target_mod]
[ 7757.735903]  ? finish_wait+0x70/0x70
[ 7757.735910]  kthread+0x112/0x130
[ 7757.735912]  ? kthread_flush_work_fn+0x10/0x10
[ 7757.735917]  ret_from_fork+0x35/0x40
[ 7757.735920] ---[ end trace 0000000000000002 ]---

static bool
__transport_wait_for_tasks(struct se_cmd *cmd, bool fabric_stop,
                           bool *aborted, bool *tas, unsigned long *flags)
        __releases(&cmd->t_state_lock)
        __acquires(&cmd->t_state_lock)
{

        assert_spin_locked(&cmd->t_state_lock);
        WARN_ON_ONCE(!irqs_disabled());   <----------

I guess this is due to the fact that on realtime kernels spin_lock_irq*() do not
disable the interrupts, so the call to irqs_disabled() returns false.

Maurizio

Re: warning on PREEMPT_RT kernel

[Bart Van Assche]

On 5/17/21 5:39 AM, Maurizio Lombardi wrote:
> static bool
> __transport_wait_for_tasks(struct se_cmd *cmd, bool fabric_stop,
>                            bool *aborted, bool *tas, unsigned long *flags)
>         __releases(&cmd->t_state_lock)
>         __acquires(&cmd->t_state_lock)
> {
> 
>         assert_spin_locked(&cmd->t_state_lock);
>         WARN_ON_ONCE(!irqs_disabled());   <----------
> 
> I guess this is due to the fact that on realtime kernels spin_lock_irq*() do not
> disable the interrupts, so the call to irqs_disabled() returns false.

It's probably fine to leave out that WARN_ON_ONCE() call. And the
assert_spin_locked() probably should be changed into a
lockdep_assert_held() call.

Thanks,

Bart.