CAN FD controllers (M-CAN tcan4x5x as well as Microchip mcp251xfd) fails on iMX6 eCSPI interface

CAN FD controllers (M-CAN tcan4x5x as well as Microchip mcp251xfd) fails on iMX6 eCSPI interface

[Pavel Pisa]

Hello everybody,

my colleague at Elektroline.cz works on design of iMX6
based system with CAN FD support realized by tcan4x5x
chip connected to eCSPI. It seems that there are problems
with eCSPI DMA stucks and other troubles. When the same
chip (or even Microchip's mcp251xfd) is connected to
other (less industry sound platforms) as Allwinner etc...
drivers seems to work reliably, but tests on iMX6 results
in failures. They consider fast redesign to slCAN connected
second Microchip MCU to resolve critical problem for the
project now....

The setup on 5.7 kernel partially wrks

Linux RAPTOR 5.7.8 #2 SMP Fri Oct 30 18:55:02 CET 2020 armv7l GNU/Linux

&ecspi1 {
	pinctrl-names = "default";
	pinctrl-0 = <&pinctrl_raptor_ecspi1>;
	cs-gpios = <&gpio2 30 0> ;
	status = "okay";

	tcan4550: tcan4x5x@0 {
		compatible = "ti,tcan4x5x";
		reg = <0>;
		#address-cells = <1>;
		#size-cells = <1>;
		spi-max-frequency = <16000000>;
		bosch,mram-cfg = <0x0 0 0 32 0 0 1 1>;
		interrupt-parent = <&gpio5>;
		interrupts = <4 GPIO_ACTIVE_LOW>;
		device-wake-gpios = <&gpio2 16 GPIO_ACTIVE_LOW>;
	};
};

Transmit test

cangen can1 -g 5 -I 123 -n100
OK
cangen can1 -g 3 -I 123 -n100
write: No buffer space available
[  857.829179] tcan4x5x spi0.0 can1: hard_xmit called while tx busy
[  857.836124] tcan4x5x spi0.0 can1: hard_xmit called while tx busy
[  857.842537] tcan4x5x spi0.0 can1: hard_xmit called while tx busy
[  857.852634] NOHZ: local_softirq_pending 08
[  857.862952] NOHZ: local_softirq_pending 08

Receive test
cangen can0 -g 5 -I 123 -n100
OK
cangen can0 -g 3 -I 123 -n100
[ 1161.348615] tcan4x5x spi0.0 can1: msg lost in rxf0
[ 1161.355672] tcan4x5x spi0.0 can1: msg lost in rxf0
[ 1161.362845] tcan4x5x spi0.0 can1: msg lost in rxf0
[ 1161.369976] tcan4x5x spi0.0 can1: msg lost in rxf0
[ 1161.377074] tcan4x5x spi0.0 can1: msg lost in rxf0
[ 1161.384021] tcan4x5x spi0.0 can1: msg lost in rxf0
[ 1161.391211] tcan4x5x spi0.0 can1: msg lost in rxf0
[ 1161.398338] tcan4x5x spi0.0 can1: msg lost in rxf0
[ 1161.405299] tcan4x5x spi0.0 can1: msg lost in rxf0
[ 1161.412445] tcan4x5x spi0.0 can1: msg lost in rxf0
[ 1161.419680] tcan4x5x spi0.0 can1: msg lost in rxf0
...

the 5.10 RT (preffered) and non-RT seems to be completely broken 

Linux RAPTOR 5.10.19 #6 SMP Tue Mar 9 13:29:28 CET 2021 armv7l GNU/Linux

&ecspi1 {
	pinctrl-names = "default";
	pinctrl-0 = <&pinctrl_raptor_ecspi1>;
	cs-gpios = <&gpio2 30 0> ;
	status = "okay";

	tcan4550: tcan4x5x@0 {
		compatible = "ti,tcan4x5x";
		reg = <0>;
		#address-cells = <1>;
		#size-cells = <1>;
		spi-max-frequency = <16000000>;
		bosch,mram-cfg = <0x0 0 0 32 0 0 1 1>;
		interrupt-parent = <&gpio5>;
		interrupts = <4 GPIO_ACTIVE_LOW>;
		device-wake-gpios = <&gpio2 16 GPIO_ACTIVE_LOW>;
	};
};

Transmit test, only one message transmited.

cangen can1 -g 100 -I 123 -n100
write: No buffer space available

candump can0
  can0  123   [8]  06 56 C6 01 AF E3 B6 4A

Receive test no messages received

[  332.449605] tcan4x5x spi0.0 can1: msg lost in rxf0

We can rebuild kernel, test patches, try to investgate what happens.

All suggestions what to test and what can be cause welcomed.

Thanks in advance,

                Pavel
-- 
                Pavel Pisa
    phone:      +420 603531357
    e-mail:     pisa@cmp.felk.cvut.cz
    Department of Control Engineering FEE CVUT
    Karlovo namesti 13, 121 35, Prague 2
    university: http://dce.fel.cvut.cz/
    company:    https://www.pikron.com/
    personal:   http://cmp.felk.cvut.cz/~pisa
    projects:   https://www.openhub.net/accounts/ppisa
    CAN related:http://canbus.pages.fel.cvut.cz/

Re: CAN FD controllers (M-CAN tcan4x5x as well as Microchip mcp251xfd) fails on iMX6 eCSPI interface

[Marc Kleine-Budde]

[-- Attachment #1: Type: text/plain, Size: 1769 bytes --]

On 22.03.2021 09:06:52, Pavel Pisa wrote:
> my colleague at Elektroline.cz works on design of iMX6
> based system with CAN FD support realized by tcan4x5x
> chip connected to eCSPI. It seems that there are problems
> with eCSPI DMA stucks and other troubles. When the same
> chip (or even Microchip's mcp251xfd) is connected to
> other (less industry sound platforms) as Allwinner etc...
> drivers seems to work reliably, but tests on iMX6 results
> in failures. They consider fast redesign to slCAN connected
> second Microchip MCU to resolve critical problem for the
> project now....

Don't use slcan, just don't.

- If you want to stick to the SPI, use a mcp2518fd.
- If you don't need CAN-FD, attach a stm32f042 or f072 via USB. There is
  a open source firmware and Linux drivers.
- If you need CAN-FD, use a more modern stm32. I think some of the "G"
  series have CAN-FD. But the firmware and Linux drivers are not
  adopted, yet.

Expect quite some CPU load for the SPI based CAN controllers, due to the
high Linux SPI overhead and the not that optimized imx SPI host driver.

> The setup on 5.7 kernel partially works

For the tcan4x5x better use latest v5.12 plus this series:
https://lore.kernel.org/linux-can/20210308102427.63916-1-torin@maxiluxsystems.com/

If the SPI DMA makes troubles, deactivate it. I think the tcan4x5x driver
uses single tcan4x5x register reads, which results in small SPI
transfers, so DMA brings no benefits.

regards,
Marc

-- 
Pengutronix e.K.                 | Marc Kleine-Budde           |
Embedded Linux                   | https://www.pengutronix.de  |
Vertretung West/Dortmund         | Phone: +49-231-2826-924     |
Amtsgericht Hildesheim, HRA 2686 | Fax:   +49-5121-206917-5555 |

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: CAN FD controllers (M-CAN tcan4x5x as well as Microchip mcp251xfd) fails on iMX6 eCSPI interface

[Pavel Pisa]

Hello Marc,

thanks much for the fast reply.

On Monday 22 of March 2021 09:31:28 Marc Kleine-Budde wrote:
> On 22.03.2021 09:06:52, Pavel Pisa wrote:
> > my colleague at Elektroline.cz works on design of iMX6
> > based system with CAN FD support realized by tcan4x5x
> > chip connected to eCSPI. It seems that there are problems
> > with eCSPI DMA stucks and other troubles. When the same
> > chip (or even Microchip's mcp251xfd) is connected to
> > other (less industry sound platforms) as Allwinner etc...
> > drivers seems to work reliably, but tests on iMX6 results
> > in failures. They consider fast redesign to slCAN connected
> > second Microchip MCU to resolve critical problem for the
> > project now....
>
> Don't use slcan, just don't.

Yes, I agree with it and argued it to my colleagues but they are
so frustrated by more problems in iMX6 and imxRT erratas that they
believe that serial port has highest probability to not been broken.

> - If you want to stick to the SPI, use a mcp2518fd.
> - If you don't need CAN-FD, attach a stm32f042 or f072 via USB. There is
>   a open source firmware and Linux drivers.

They have Microchip PIC32 for power management in the design and quite
good experience with it, so they can use little more advanced one
with CAN FD and use it at CAN interface. The idea to use SPI connected
MCU (in my case NXP LPC) come to my mind at LinCAN era when everybody
used MCP2515 with horrible single register operation overhead.
Can you suggest SPI protocol for CAN, CAN FD MCU connection as the
SocketCAN interface? Is there plan for CAN FD version?
Anyway if the problems are caused by NXP SPI, then they can creep
in still.

> - If you need CAN-FD, use a more modern stm32. I think some of the "G"
>   series have CAN-FD. But the firmware and Linux drivers are not
>   adopted, yet.

We have solved and mainlined CAN FD on imxRT on NuttX and Microchip
SAME70 (mainlining to NuttX expected soon) so we can reuse these.

> Expect quite some CPU load for the SPI based CAN controllers, due to the
> high Linux SPI overhead and the not that optimized imx SPI host driver.

Yes, I am not fan of these solution (you know our CTU CAN FD effort,
hopefully headers generator rewrite comes to the table next month),
but Elektroline company needs industrial range system and could not
wait for iMX8X with CAN FD controllers at the project start time.

> > The setup on 5.7 kernel partially works
>
> For the tcan4x5x better use latest v5.12 plus this series:
> https://lore.kernel.org/linux-can/20210308102427.63916-1-torin@maxiluxsyste
>ms.com/

Thanks, we will test that for sure but for production we probably
need to backport to 5.10 because it has chance for serious LTS
support from Civil Infrastructure Platform (adding Pavel to CC)
for standard and even better preempt-RT kernels.

> If the SPI DMA makes troubles, deactivate it. I think the tcan4x5x driver
> uses single tcan4x5x register reads, which results in small SPI
> transfers, so DMA brings no benefits.

Yes, we try that. I have some reminiscence form old time that we have
done some similar tricks on imx53 to make it work in infussion system demo.

Again thanks much for fast response,

                Pavel
--
                Pavel Pisa
    phone:      +420 603531357
    e-mail:     pisa@cmp.felk.cvut.cz
    Department of Control Engineering FEE CVUT
    Karlovo namesti 13, 121 35, Prague 2
    university: http://dce.fel.cvut.cz/
    personal:   http://cmp.felk.cvut.cz/~pisa
    projects:   https://www.openhub.net/accounts/ppisa
    CAN related:http://canbus.pages.fel.cvut.cz/

Re: CAN FD controllers (M-CAN tcan4x5x as well as Microchip mcp251xfd) fails on iMX6 eCSPI interface

[Marc Kleine-Budde]

[-- Attachment #1: Type: text/plain, Size: 4887 bytes --]

On 22.03.2021 10:45:40, Pavel Pisa wrote:
> Hello Marc,
> 
> thanks much for the fast reply.
> 
> On Monday 22 of March 2021 09:31:28 Marc Kleine-Budde wrote:
> > On 22.03.2021 09:06:52, Pavel Pisa wrote:
> > > my colleague at Elektroline.cz works on design of iMX6
> > > based system with CAN FD support realized by tcan4x5x
> > > chip connected to eCSPI. It seems that there are problems
> > > with eCSPI DMA stucks and other troubles. When the same
> > > chip (or even Microchip's mcp251xfd) is connected to
> > > other (less industry sound platforms) as Allwinner etc...
> > > drivers seems to work reliably, but tests on iMX6 results
> > > in failures. They consider fast redesign to slCAN connected
> > > second Microchip MCU to resolve critical problem for the
> > > project now....
> >
> > Don't use slcan, just don't.
> 
> Yes, I agree with it and argued it to my colleagues but they are
> so frustrated by more problems in iMX6 and imxRT erratas that they

imxrt, as in the mmu-less µC?

> believe that serial port has highest probability to not been broken.

I can ask my coworker Uwe to tell horror stories about the imx serial
port driver if you want :D
I haven't thought that far, I even don't recommend slcan from the CAN
point of view.

> > - If you want to stick to the SPI, use a mcp2518fd.
> > - If you don't need CAN-FD, attach a stm32f042 or f072 via USB. There is
> >   a open source firmware and Linux drivers.
> 
> They have Microchip PIC32 for power management in the design and quite
> good experience with it, so they can use little more advanced one
> with CAN FD and use it at CAN interface.

I suggest to use a PIC32 with USB and CAN, to get rid of the SPI. The
existing STM32F0x2 firmware can probably be ported to the PIC32 so that
you can re-use the gs-usb driver unter Linux.

> The idea to use SPI connected MCU (in my case NXP LPC) come to my mind
> at LinCAN era when everybody used MCP2515 with horrible single
> register operation overhead. Can you suggest SPI protocol for CAN, CAN
> FD MCU connection as the SocketCAN interface?

Don't do SPI in the first place, use USB.

Back to the SPI:
If you have a full blown ̧̧µC you don't want to access the µC's CAN
controller on the register level, but send/receive complete CAN
messages. We don't have a SPI driver for that, but you can model the SPI
messages like the gs-usb USB messages.

> Is there plan for CAN FD version? Anyway if the problems are caused by
> NXP SPI, then they can creep in still.

The only advantage on sending/receiving full CAN messages over SPI is
that you have less overhead compared to register level access. But the
imx SPI host driver will probably still use a lot of CPU.

> > - If you need CAN-FD, use a more modern stm32. I think some of the "G"
> >   series have CAN-FD. But the firmware and Linux drivers are not
> >   adopted, yet.
> 
> We have solved and mainlined CAN FD on imxRT on NuttX and Microchip
> SAME70 (mainlining to NuttX expected soon) so we can reuse these.

Sounds good. Next step would be to add a gs-usb compatible USB device
driver. There already is a not mainlined gs-usb-fd
(https://github.com/linklayer/gs_usb_fd), but linklayer lost interest in
mainlining it.

> > Expect quite some CPU load for the SPI based CAN controllers, due to the
> > high Linux SPI overhead and the not that optimized imx SPI host driver.
> 
> Yes, I am not fan of these solution (you know our CTU CAN FD effort,
> hopefully headers generator rewrite comes to the table next month),
> but Elektroline company needs industrial range system and could not
> wait for iMX8X with CAN FD controllers at the project start time.

don't use imx8x, use the imx8mp instead.

> > > The setup on 5.7 kernel partially works
> >
> > For the tcan4x5x better use latest v5.12 plus this series:
> > https://lore.kernel.org/linux-can/20210308102427.63916-1-torin@maxiluxsyste
> >ms.com/
> 
> Thanks, we will test that for sure but for production we probably
> need to backport to 5.10 because it has chance for serious LTS
> support from Civil Infrastructure Platform (adding Pavel to CC)
> for standard and even better preempt-RT kernels.

Ok

> > If the SPI DMA makes troubles, deactivate it. I think the tcan4x5x driver
> > uses single tcan4x5x register reads, which results in small SPI
> > transfers, so DMA brings no benefits.
> 
> Yes, we try that. I have some reminiscence form old time that we have
> done some similar tricks on imx53 to make it work in infussion system demo.

regards,
Marc

-- 
Pengutronix e.K.                 | Marc Kleine-Budde           |
Embedded Linux                   | https://www.pengutronix.de  |
Vertretung West/Dortmund         | Phone: +49-231-2826-924     |
Amtsgericht Hildesheim, HRA 2686 | Fax:   +49-5121-206917-5555 |

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]