Methods for testing CAN drivers tx/rx paths

Methods for testing CAN drivers tx/rx paths

[Ahmed S. Darwish]

Hi Marc, everyone,

Do you have any comments on how to heavily test a CAN driver tx/rx
code paths?

Below is the approach I use, which was succesful in triggering many
issues in the kvaser_usb driver. I wonder though if you guys have
any different tricks you use for your own daily CAN driver
development tx/rx testing?


Testing method:
---------------

On tx:

	$ cangen -g1 -I111 -Di -L4 can0 &
	$ cangen -g1 -I222 -Di -L4 can0 &
	$ cangen -g1 -I333 -Di -L4 can0 &
	$ cangen -g1 -I444 -Di -L4 can0 &

On the other end of the CAN bus, to test rx:

	$ cangen -g0 -I555 -Di -L4 can0 &
	$ cangen -g1 -I666 -Di -L4 can0 &
	$ cangen -g1 -I777 -Di -L4 can0 &

Echo is enabled in the driver. Now doing a basic

	$ candump -d -e can0,0:0,#FFFFFFFF > output.txt

produces too many dropped packets after around 10 minutes of use. [1]

Using in-kernel CAN ID filtering solves the issue nicely:

	for i in {1..7}; do
	    candump -e -d can0,$i$i$i:7FF > output-$i$i$i.txt &
	done

Then the machine is left for a day or two. Afterwards, using simple
Python scripts, I analyze each output-XXX.txt file to ensure that
no packets got dropped for each `cangen' path above, and that the
sequential order of data generated by `cangen -Di' is still
maintained.


Thanks a lot,
Darwish


[1] As reported by SO_RCVQ_OFL, skb-attached, ancilliary messages.
    Thanks Oliver, candump `-d' was a life-saving feature!
    http://article.gmane.org/gmane.network.socketcan.user/179

Re: Methods for testing CAN drivers tx/rx paths

[Oliver Hartkopp]

Hi Ahmed,

I'm taking care of the network layer packet flow with this test script:

https://github.com/linux-can/can-tests/blob/master/test_netlayer.sh

The script checks whether CAN specific information inside the skbuff survives
the networking stack which is always in move.

For driver testing I use this CAN hardware
http://www.peak-system.com/PCAN-MicroMod-Evaluation.221.0.html
with a special firmware that's able to generate 100% busload.

Putting this load to a CAN bus I additionally try to send frames on the bus
with that interface or do unfriendly unplugging (e.g. for USB / PCMCIA).

http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit?id=a7762b10c12a70c5dbf2253142764b728ac88c3a

So when doing some test I'm mainly checking for stability under load.

There's a CAN full duplex test application at the can-utils:
https://github.com/linux-can/can-utils/blob/master/canfdtest.c
which goes into your proposed direction (and which should probably go into
can-tests repository).

Your described test is pretty interesting too! When I find some time I will
try your test too. Can you also post your Phython scripts here?

Best regards,
Oliver

On 03/14/2015 07:15 PM, Ahmed S. Darwish wrote:
> Hi Marc, everyone,
> 
> Do you have any comments on how to heavily test a CAN driver tx/rx
> code paths?
> 
> Below is the approach I use, which was succesful in triggering many
> issues in the kvaser_usb driver. I wonder though if you guys have
> any different tricks you use for your own daily CAN driver
> development tx/rx testing?
> 
> 
> Testing method:
> ---------------
> 
> On tx:
> 
> 	$ cangen -g1 -I111 -Di -L4 can0 &
> 	$ cangen -g1 -I222 -Di -L4 can0 &
> 	$ cangen -g1 -I333 -Di -L4 can0 &
> 	$ cangen -g1 -I444 -Di -L4 can0 &
> 
> On the other end of the CAN bus, to test rx:
> 
> 	$ cangen -g0 -I555 -Di -L4 can0 &
> 	$ cangen -g1 -I666 -Di -L4 can0 &
> 	$ cangen -g1 -I777 -Di -L4 can0 &
> 
> Echo is enabled in the driver. Now doing a basic
> 
> 	$ candump -d -e can0,0:0,#FFFFFFFF > output.txt
> 
> produces too many dropped packets after around 10 minutes of use. [1]
> 
> Using in-kernel CAN ID filtering solves the issue nicely:
> 
> 	for i in {1..7}; do
> 	    candump -e -d can0,$i$i$i:7FF > output-$i$i$i.txt &
> 	done
> 
> Then the machine is left for a day or two. Afterwards, using simple
> Python scripts, I analyze each output-XXX.txt file to ensure that
> no packets got dropped for each `cangen' path above, and that the
> sequential order of data generated by `cangen -Di' is still
> maintained.
> 
> 
> Thanks a lot,
> Darwish
> 
> 
> [1] As reported by SO_RCVQ_OFL, skb-attached, ancilliary messages.
>     Thanks Oliver, candump `-d' was a life-saving feature!
>     http://article.gmane.org/gmane.network.socketcan.user/179
> --
> To unsubscribe from this list: send the line "unsubscribe linux-can" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: Methods for testing CAN drivers tx/rx paths

[Ahmed S. Darwish]

Hi,

On Mon, Mar 16, 2015 at 09:37:08AM +0100, Oliver Hartkopp wrote:
> Hi Ahmed,
> 
> I'm taking care of the network layer packet flow with this test script:
> 
> https://github.com/linux-can/can-tests/blob/master/test_netlayer.sh
> 
> The script checks whether CAN specific information inside the skbuff survives
> the networking stack which is always in move.
> 
> For driver testing I use this CAN hardware
> http://www.peak-system.com/PCAN-MicroMod-Evaluation.221.0.html
> with a special firmware that's able to generate 100% busload.
> 
> Putting this load to a CAN bus I additionally try to send frames on the bus
> with that interface or do unfriendly unplugging (e.g. for USB / PCMCIA).
> 
> http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit?id=a7762b10c12a70c5dbf2253142764b728ac88c3a
> 
> So when doing some test I'm mainly checking for stability under load.
> 
> There's a CAN full duplex test application at the can-utils:
> https://github.com/linux-can/can-utils/blob/master/canfdtest.c
> which goes into your proposed direction (and which should probably go into
> can-tests repository).
> 

Aha, thanks a lot for sharing. I'll definitely begin incorporating
those, especially `canfdtest', which I did not know of its existence
before.

> Your described test is pretty interesting too! When I find some time I will
> try your test too. Can you also post your Phython scripts here?
> 

They're a quick-n-dirty collection of python+bash scripts. I'll see
if I can port them to a basic C utility and post the results back to
can-utils :-)

Thanks,
Darwish

> Best regards,
> Oliver
> 
> On 03/14/2015 07:15 PM, Ahmed S. Darwish wrote:
> > Hi Marc, everyone,
> > 
> > Do you have any comments on how to heavily test a CAN driver tx/rx
> > code paths?
> > 
> > Below is the approach I use, which was succesful in triggering many
> > issues in the kvaser_usb driver. I wonder though if you guys have
> > any different tricks you use for your own daily CAN driver
> > development tx/rx testing?
> > 
> > 
> > Testing method:
> > ---------------
> > 
> > On tx:
> > 
> > 	$ cangen -g1 -I111 -Di -L4 can0 &
> > 	$ cangen -g1 -I222 -Di -L4 can0 &
> > 	$ cangen -g1 -I333 -Di -L4 can0 &
> > 	$ cangen -g1 -I444 -Di -L4 can0 &
> > 
> > On the other end of the CAN bus, to test rx:
> > 
> > 	$ cangen -g0 -I555 -Di -L4 can0 &
> > 	$ cangen -g1 -I666 -Di -L4 can0 &
> > 	$ cangen -g1 -I777 -Di -L4 can0 &
> > 
> > Echo is enabled in the driver. Now doing a basic
> > 
> > 	$ candump -d -e can0,0:0,#FFFFFFFF > output.txt
> > 
> > produces too many dropped packets after around 10 minutes of use. [1]
> > 
> > Using in-kernel CAN ID filtering solves the issue nicely:
> > 
> > 	for i in {1..7}; do
> > 	    candump -e -d can0,$i$i$i:7FF > output-$i$i$i.txt &
> > 	done
> > 
> > Then the machine is left for a day or two. Afterwards, using simple
> > Python scripts, I analyze each output-XXX.txt file to ensure that
> > no packets got dropped for each `cangen' path above, and that the
> > sequential order of data generated by `cangen -Di' is still
> > maintained.
> > 
> > 
> > Thanks a lot,
> > Darwish
> > 
> > 
> > [1] As reported by SO_RCVQ_OFL, skb-attached, ancilliary messages.
> >     Thanks Oliver, candump `-d' was a life-saving feature!
> >     http://article.gmane.org/gmane.network.socketcan.user/179
> >