flexcan irq error on kernel >= 5.10

flexcan irq error on kernel >= 5.10

[Julien Blanc]

Hi all,

I'm currently in the process of migrating our imx6ull based board from
linux 5.4 + ipipe + xenomai 3.2.2 to linux 5.15.98-dovetail1 + xenomai
3.2 trunk.

I can't get the flexcan driver to work. Trying to bring rtcan0 (resp
rtcan1) up fails with the following error:

[   85.788631] ------------[ cut here ]------------
[   85.788657] WARNING: CPU: 0 PID: 169 at kernel/irq/manage.c:1957
free_irq+0x230/0x408
[   85.788724] Trying to free already-free IRQ 24
[   85.788733] Modules linked in: imx_sdma
[   85.788766] CPU: 0 PID: 169 Comm: rtcanconfig Not tainted 5.15.98
#12
[   85.788791] Hardware name: Freescale i.MX6 Ultralite (Device Tree)
[   85.788801] IRQ stage: Xenomai
[   85.788824] [<c010e08c>] (unwind_backtrace) from [<c010b620>]
(show_stack+0x10/0x14)
[   85.788875] [<c010b620>] (show_stack) from [<c011f1f8>]
(__warn+0x88/0x13c)
[   85.788919] [<c011f1f8>] (__warn) from [<c08c6a30>]
(warn_slowpath_fmt+0x78/0xac)
[   85.788967] [<c08c6a30>] (warn_slowpath_fmt) from [<c0173de8>]
(free_irq+0x230/0x408)
[   85.789014] [<c0173de8>] (free_irq) from [<c021a004>]
(rtdm_irq_request+0x84/0x98)
[   85.789060] [<c021a004>] (rtdm_irq_request) from [<c075b8e8>]
(flexcan_set_mode+0x24c/0x31c)
[   85.789113] [<c075b8e8>] (flexcan_set_mode) from [<c0759d18>]
(rtcan_raw_ioctl_dev+0x85c/0x97c)
[   85.789169] [<c0759d18>] (rtcan_raw_ioctl_dev) from [<c021afd4>]
(rtdm_fd_ioctl+0xbc/0x2b4)
[   85.789217] [<c021afd4>] (rtdm_fd_ioctl) from [<c022ee1c>]
(handle_root_syscall+0x128/0x3c0)
[   85.789266] [<c022ee1c>] (handle_root_syscall) from [<c01de688>]
(__pipeline_syscall+0xb8/0x1f0)
[   85.789318] [<c01de688>] (__pipeline_syscall) from [<c010028c>]
(pipeline_syscall+0x8/0x1c)
[   85.789363] Exception stack(0xc32c5fb0 to 0xc32c5ff8)
[   85.789388] 5fa0:                                     10000050
00000003 401c0305 becacc10
[   85.789415] 5fc0: 00000001 00000003 401c0305 000f0042 becacc10
004d1f64 00000000 00000003
[   85.789438] 5fe0: 000f0042 becacb94 b6f900a0 b6f31a20 80030010
10000050
[   85.789456] irq event stamp: 33168
[   85.789464] hardirqs last  enabled at (33167): [<c08d9528>]
_raw_spin_unlock_irqrestore+0x50/0x54
[   85.789505] hardirqs last disabled at (33168): [<c08d1e80>]
__schedule+0xa8/0x814
[   85.789535] softirqs last  enabled at (28352): [<c010165c>]
__do_softirq+0x33c/0x4ec
[   85.789567] softirqs last disabled at (28343): [<c0124cc8>]
__irq_exit_rcu+0x1b0/0x1fc
[   85.789600] ---[ end trace 2e0cfdf43bf6263c ]---


which seems to indicate that requesting the irq failed.

I checked that using the same kernel, the linux flexcan driver works
correctly, using the same interrupts (24 and 25, whereas with 5.4
kernel it was using 23 and 24 - not sure if that's relevant).

It was working correctly with kernel 5.4 + xenomai, using exactly the
same dts file and same xenomai config.

Here are some relevant lines in the boot log:

[    7.402048] RT-Socket-CAN 0.90.2 - (C) 2006 RT-Socket-CAN
Development Team
[    7.424047] flexcan 2090000.can: supply xceiver not found, using
dummy regulator
[    7.442419] rtcan: registered rtcan0
[    7.446398] flexcan 2090000.can: device registered
(reg_base=290b71d7, irq=24)
[    7.456370] flexcan 2094000.can: supply xceiver not found, using
dummy regulator
[    7.468282] rtcan: registered rtcan1
[    7.472201] flexcan 2094000.can: device registered
(reg_base=e79a29fe, irq=25)
[    7.486262] NET: Registered PF_PACKET protocol family
[    7.491887] can: controller area network core
[    7.498246] NET: Registered PF_CAN protocol family
[    7.503321] can: raw protocol
[    7.507028] can: broadcast manager protocol
[    7.512216] can: netlink gateway - max_hops=1

I find the reg_base value rather strange (with 5.4 it was just iirc
"(ptr)", so i don't have something to compare with).

I must say i'm a bit out of ideas here. Any advice on how to further
debug this would be greatly appreciated. Here's the output of

# cat /sys/kernel/debug/irq/irqs/24
handler:  handle_fasteoi_irq
device:   (null)
status:   0x00000000
istate:   0x00004000
ddepth:   1
wdepth:   0
dstate:   0x13030004
            IRQ_TYPE_LEVEL_HIGH
            IRQD_IRQ_DISABLED
            IRQD_IRQ_MASKED
            IRQD_SINGLE_TARGET
            IRQD_DEFAULT_TRIGGER_SET
            IRQD_HANDLE_ENFORCE_IRQCTX
node:     0
domain:  :soc:bus@2000000:gpc@20dc000
 hwirq:   0x6e
 chip:    GPC
  flags:   0x800
 parent:
    domain:  :soc:interrupt-controller@a01000
     hwirq:   0x8e
     chip:    GIC-0
      flags:   0x815
                 IRQCHIP_SET_TYPE_MASKED
                 IRQCHIP_MASK_ON_SUSPEND
                 IRQCHIP_SKIP_SET_WAKE


It looks like the interrupt is not enabled at the hardware level, and
for some reason the driver fails to enable it.

Regards,

Julien

Re: flexcan irq error on kernel >= 5.10

[Julien Blanc]

Le vendredi 31 mars 2023 à 09:29 +0000, Julien Blanc a écrit :
> Hi all,
> 
> 
> It looks like the interrupt is not enabled at the hardware level, and
> for some reason the driver fails to enable it.

Replying to myself, i could get it to work with the following change:

--- a/kernel/cobalt/dovetail/intr.c	2023-03-31 14:40:17.640127580
+0200
+++ b/kernel/cobalt/dovetail/intr.c	2023-03-31 14:39:59.048341842
+0200
@@ -100,9 +100,11 @@
 		if (cpumask_empty(effective_mask))
 			return -EINVAL;
 	}
+#ifdef CONFIG_SMP
 	ret = irq_set_affinity_hint(intr->irq, effective_mask);
 	if (ret)
 		return ret;
+#endif

I believe it's safe to skip irq_set_affinity_hint on non-SMP systems,
as the affinity is nonsense on such systems. Or am I missing something?

I could also get it to work by enabling SMP (although I ran into other
issues, and don't want to do that as my target is single core).

Regards,

Julien

Re: flexcan irq error on kernel >= 5.10

[Jan Kiszka]

On 31.03.23 14:55, Julien Blanc wrote:
> Le vendredi 31 mars 2023 à 09:29 +0000, Julien Blanc a écrit :
>> Hi all,
>>
>>
>> It looks like the interrupt is not enabled at the hardware level, and
>> for some reason the driver fails to enable it.
> 
> Replying to myself, i could get it to work with the following change:
> 
> --- a/kernel/cobalt/dovetail/intr.c	2023-03-31 14:40:17.640127580
> +0200
> +++ b/kernel/cobalt/dovetail/intr.c	2023-03-31 14:39:59.048341842
> +0200
> @@ -100,9 +100,11 @@
>  		if (cpumask_empty(effective_mask))
>  			return -EINVAL;
>  	}
> +#ifdef CONFIG_SMP
>  	ret = irq_set_affinity_hint(intr->irq, effective_mask);
>  	if (ret)
>  		return ret;
> +#endif
> 
> I believe it's safe to skip irq_set_affinity_hint on non-SMP systems,
> as the affinity is nonsense on such systems. Or am I missing something?
> 

Nope, this is correct: irq_set_affinity_hint returns -EINVAL in case of
!SMP. Please send a patch, maybe also including the invocation in
xnintr_detach, for consistency reasons.

> I could also get it to work by enabling SMP (although I ran into other
> issues, and don't want to do that as my target is single core).
> 

Can you give at least some hints about that other issue?

Thanks,
Jan

-- 
Siemens AG, Technology
Competence Center Embedded Linux

Re: flexcan irq error on kernel >= 5.10

[Julien Blanc]

Le lundi 03 avril 2023 à 07:29 +0200, Jan Kiszka a écrit :
> On 31.03.23 14:55, Julien Blanc wrote:
> > 
> 
> Nope, this is correct: irq_set_affinity_hint returns -EINVAL in case of
> !SMP. Please send a patch, maybe also including the invocation in
> xnintr_detach, for consistency reasons.

Should be okay, unless my mail client messed up the format for some reason.

> 
> > I could also get it to work by enabling SMP (although I ran into other
> > issues, and don't want to do that as my target is single core).
> > 
> 
> Can you give at least some hints about that other issue?

I can provide a backtrace. I thought i saved it, but apparently not. I put that in my todo, i should have the time later this
week.

Regards,

Julien