netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Ramón Nordin Rodriguez" <ramon.nordin.rodriguez@ferroamp.se>
To: Piergiorgio Beruto <Pier.Beruto@onsemi.com>
Cc: Andrew Lunn <andrew@lunn.ch>,
	"Parthiban.Veerasooran@microchip.com"
	<Parthiban.Veerasooran@microchip.com>,
	"davem@davemloft.net" <davem@davemloft.net>,
	"edumazet@google.com" <edumazet@google.com>,
	"kuba@kernel.org" <kuba@kernel.org>,
	"pabeni@redhat.com" <pabeni@redhat.com>,
	"horms@kernel.org" <horms@kernel.org>,
	"saeedm@nvidia.com" <saeedm@nvidia.com>,
	"anthony.l.nguyen@intel.com" <anthony.l.nguyen@intel.com>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"corbet@lwn.net" <corbet@lwn.net>,
	"linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
	"robh+dt@kernel.org" <robh+dt@kernel.org>,
	"krzysztof.kozlowski+dt@linaro.org"
	<krzysztof.kozlowski+dt@linaro.org>,
	"conor+dt@kernel.org" <conor+dt@kernel.org>,
	"devicetree@vger.kernel.org" <devicetree@vger.kernel.org>,
	"Horatiu.Vultur@microchip.com" <Horatiu.Vultur@microchip.com>,
	"ruanjinjie@huawei.com" <ruanjinjie@huawei.com>,
	"Steen.Hegelund@microchip.com" <Steen.Hegelund@microchip.com>,
	"vladimir.oltean@nxp.com" <vladimir.oltean@nxp.com>,
	"UNGLinuxDriver@microchip.com" <UNGLinuxDriver@microchip.com>,
	"Thorsten.Kummermehr@microchip.com"
	<Thorsten.Kummermehr@microchip.com>,
	Selvamani Rajagopal <Selvamani.Rajagopal@onsemi.com>,
	"Nicolas.Ferre@microchip.com" <Nicolas.Ferre@microchip.com>,
	"benjamin.bigler@bernformulastudent.ch"
	<benjamin.bigler@bernformulastudent.ch>
Subject: Re: [PATCH net-next v4 05/12] net: ethernet: oa_tc6: implement error interrupts unmasking
Date: Mon, 13 May 2024 08:41:45 +0200	[thread overview]
Message-ID: <ZkG2Kb_1YsD8T1BF@minibuilder> (raw)
In-Reply-To: <BY5PR02MB678683EADBC47A29A4F545A59D1C2@BY5PR02MB6786.namprd02.prod.outlook.com>

> The RXBOE is basically a warning, indicating that for some reason the SPI host is not fast enough in retrieving frames from the device.
> In my experience, this is typically caused by excessive latency of the SPI driver, or if you have an unoptimized network driver for the MACPHY.
> 
> Thanks,
> Piergiorgio
> 
> 
> > [  285.482275] LAN865X Rev.B0 Internal Phy spi0.0:00: attached PHY 
> > driver (mii_bus:phy_addr=spi0.0:00, irq=POLL) [  285.534760] lan865x 
> > spi0.0 eth1: Link is Up - 10Mbps/Half - flow control off [  
> > 341.466221] eth1: Receive buffer overflow error [  345.730222] eth1: 
> > Receive buffer overflow error [  345.891126] eth1: Receive buffer 
> > overflow error [  346.074220] eth1: Receive buffer overflow error
> 
> Generally we only log real errors. Is a receive buffer overflow a real error? I would say not. But it would be good to count it.
> 	Andrew

Hi,

I've been busy throwing stuff at the wall until something sticks. I've
managed to narrow a few things down.
First and foremost, when running a periodic udp6 multicast in the
background I don't get a hang in the driver, or it becomes considerably
harder to provoke.

When I make sure that the bespoke Ferroamp upd server is not started
(which just joins a mcast group and sends a less than MTU packet every
~500ms and listens for incoming multicast messages in the same group),
it becomes very simple to get to a live-lock.

My steasp of reproducing is setting a ipv4 address on both ends of the
link, then running the follwing script on both ends using the other ends
ip as argument.

#!/bin/env python3
import socket
import sys

if __name__ == '__main__':
    sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

    while True:
        sock.sendto(b'0'*2048, (sys.argv[1], 4040))

Neither ends opens a listening socket. I get to the live lock in 10s or
less.
I've enabled some debugging options but so far nothing seems to hit.
What I've been able to conclude is that there still is SPI
communication, the macphy interrupt is still pulled low, and the cpu
does the ack so that it's reset to inactive.
But from there it seems no data is passed up the network stack. Some
symptoms are

* net_device stats are no longer incremented
* can't ping
* can't connect to sockets on the board etc.
* cpu usage jumps to and stays at 100% for the worker thread

The worker thread is released by the irq handler and it does some of the
expected work, but not all.
I'm adding some instrumentation to the code in an effort to figure out
where things break apart.
It might be possible to catch it in gdb as well, but I think you only
get one try as the timing will be pretty borked after the first break.

R

  parent reply	other threads:[~2024-05-13  6:41 UTC|newest]

Thread overview: 124+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-04-18 12:56 [PATCH net-next v4 00/12] Add support for OPEN Alliance 10BASE-T1x MACPHY Serial Interface Parthiban Veerasooran
2024-04-18 12:56 ` [PATCH net-next v4 01/12] Documentation: networking: add OPEN Alliance 10BASE-T1x MAC-PHY serial interface Parthiban Veerasooran
2024-04-28  9:33   ` Bagas Sanjaya
2024-04-29 11:31     ` Parthiban.Veerasooran
2024-04-29 16:17   ` Simon Horman
2024-04-30 11:30     ` Parthiban.Veerasooran
2024-04-18 12:56 ` [PATCH net-next v4 02/12] net: ethernet: oa_tc6: implement register write operation Parthiban Veerasooran
2024-04-22 23:48   ` Andrew Lunn
2024-04-23  4:38     ` Parthiban.Veerasooran
2024-04-23 23:14   ` Andrew Lunn
2024-04-26  5:55     ` Parthiban.Veerasooran
2024-04-29 16:36   ` Simon Horman
2024-04-30  8:14     ` Parthiban.Veerasooran
2024-04-18 12:56 ` [PATCH net-next v4 03/12] net: ethernet: oa_tc6: implement register read operation Parthiban Veerasooran
2024-04-23 23:17   ` Andrew Lunn
2024-04-26  5:56     ` Parthiban.Veerasooran
2024-04-18 12:56 ` [PATCH net-next v4 04/12] net: ethernet: oa_tc6: implement software reset Parthiban Veerasooran
2024-04-23 23:26   ` Andrew Lunn
2024-04-26  6:38     ` Parthiban.Veerasooran
2024-04-18 12:56 ` [PATCH net-next v4 05/12] net: ethernet: oa_tc6: implement error interrupts unmasking Parthiban Veerasooran
2024-04-23 23:27   ` Andrew Lunn
2024-04-27 19:52   ` Ramón Nordin Rodriguez
2024-04-27 21:17     ` Andrew Lunn
2024-04-27 21:55       ` Ramón Nordin Rodriguez
2024-04-28 14:48         ` Andrew Lunn
2024-04-28  9:54       ` Ramón Nordin Rodriguez
2024-04-28 14:59         ` Andrew Lunn
2024-04-28 22:04           ` Ramón Nordin Rodriguez
2024-05-01 18:29           ` Ramón Nordin Rodriguez
2024-05-01 19:58             ` Andrew Lunn
2024-05-01 20:07             ` Andrew Lunn
2024-05-02 10:10             ` Parthiban.Veerasooran
2024-05-02 10:19               ` Ramón Nordin Rodriguez
2024-05-03  7:10                 ` Parthiban.Veerasooran
2024-05-06  1:21                   ` Andrew Lunn
2024-05-06  6:47                     ` Piergiorgio Beruto
2024-05-07 12:44                       ` Ramón Nordin Rodriguez
2024-05-13  6:41                       ` Ramón Nordin Rodriguez [this message]
2024-05-13 13:00                         ` Andrew Lunn
2024-05-13 13:50                           ` Ramón Nordin Rodriguez
2024-05-13 14:04                             ` Andrew Lunn
2024-05-15 21:45                               ` Ramón Nordin Rodriguez
2024-05-14  4:46                             ` Parthiban.Veerasooran
2024-05-15 21:48                               ` Ramón Nordin Rodriguez
2024-05-17  9:38                                 ` Parthiban.Veerasooran
2024-05-17 12:43                                   ` Ramón Nordin Rodriguez
2024-05-24 18:12                                   ` Ramón Nordin Rodriguez
2024-05-24 18:31                                     ` Andrew Lunn
2024-05-24 18:49                                       ` Piergiorgio Beruto
2024-05-27  9:30                                       ` Parthiban.Veerasooran
2024-05-27  8:38                                     ` Parthiban.Veerasooran
2024-04-18 12:56 ` [PATCH net-next v4 06/12] net: ethernet: oa_tc6: implement internal PHY initialization Parthiban Veerasooran
2024-04-23 23:48   ` Andrew Lunn
2024-04-26 13:17     ` Parthiban.Veerasooran
2024-04-18 12:56 ` [PATCH net-next v4 07/12] net: ethernet: oa_tc6: enable open alliance tc6 data communication Parthiban Veerasooran
2024-04-23 23:49   ` Andrew Lunn
2024-04-18 12:56 ` [PATCH net-next v4 08/12] net: ethernet: oa_tc6: implement transmit path to transfer tx ethernet frames Parthiban Veerasooran
2024-04-24  0:02   ` Andrew Lunn
2024-04-26 13:19     ` Parthiban.Veerasooran
2024-04-18 12:56 ` [PATCH net-next v4 09/12] net: ethernet: oa_tc6: implement receive path to receive rx " Parthiban Veerasooran
2024-04-24  0:08   ` Andrew Lunn
2024-04-26 13:45     ` Parthiban.Veerasooran
2024-04-26 18:13       ` Andrew Lunn
2024-04-29  6:13         ` Parthiban.Veerasooran
2024-04-27 20:02   ` Ramón Nordin Rodriguez
2024-04-29  8:32     ` Parthiban.Veerasooran
2024-04-29 10:45       ` Ramón Nordin Rodriguez
2024-04-18 12:56 ` [PATCH net-next v4 10/12] net: ethernet: oa_tc6: implement mac-phy interrupt Parthiban Veerasooran
2024-04-24  0:10   ` Andrew Lunn
2024-04-18 12:56 ` [PATCH net-next v4 11/12] microchip: lan865x: add driver support for Microchip's LAN865X MAC-PHY Parthiban Veerasooran
2024-04-24  0:27   ` Andrew Lunn
2024-04-26 13:32     ` Parthiban.Veerasooran
2024-04-26 18:14       ` Andrew Lunn
2024-04-29  6:13         ` Parthiban.Veerasooran
2024-04-27 19:19   ` Ramón Nordin Rodriguez
2024-04-27 19:57     ` Conor Dooley
2024-04-27 20:13       ` Ramón Nordin Rodriguez
2024-04-27 20:22         ` Conor Dooley
2024-04-27 21:09           ` Ramón Nordin Rodriguez
2024-04-29  9:47       ` Parthiban.Veerasooran
2024-04-29 12:09         ` Andrew Lunn
2024-04-30 13:30           ` Parthiban.Veerasooran
2024-04-30 16:55             ` Conor Dooley
2024-05-02  5:56               ` Parthiban.Veerasooran
2024-04-27 20:40     ` Andrew Lunn
2024-04-27 21:06       ` Ramón Nordin Rodriguez
2024-04-28 14:18         ` Andrew Lunn
2024-04-27 19:35   ` Ramón Nordin Rodriguez
2024-04-27 20:58     ` Andrew Lunn
2024-04-27 21:29       ` Ramón Nordin Rodriguez
2024-04-28 14:25         ` Andrew Lunn
2024-04-28 22:00           ` Ramón Nordin Rodriguez
2024-04-18 12:56 ` [PATCH net-next v4 12/12] dt-bindings: net: add Microchip's LAN865X 10BASE-T1S MACPHY Parthiban Veerasooran
2024-04-18 15:39   ` Conor Dooley
2024-04-22  3:59     ` Parthiban.Veerasooran
2024-04-22 15:50       ` Conor Dooley
2024-04-23  3:27         ` Parthiban.Veerasooran
2024-04-22 20:07 ` [PATCH net-next v4 00/12] Add support for OPEN Alliance 10BASE-T1x MACPHY Serial Interface Andrew Lunn
2024-04-22 20:08 ` Andrew Lunn
2024-04-22 23:23   ` Andrew Lunn
2024-05-08 13:05     ` Parthiban.Veerasooran
2024-05-08 17:04       ` Andrew Lunn
2024-05-09 13:04         ` Parthiban.Veerasooran
2024-05-09 20:39           ` Andrew Lunn
2024-05-10 11:22             ` Parthiban.Veerasooran
2024-05-24 20:52               ` Selvamani Rajagopal
2024-05-24 21:27                 ` Andrew Lunn
2024-05-24 21:45                   ` Piergiorgio Beruto
2024-05-24 21:54                     ` Andrew Lunn
2024-05-24 22:08                       ` Piergiorgio Beruto
2024-05-25 14:46                         ` Andrew Lunn
2024-04-22 23:43 ` Andrew Lunn
2024-04-28 21:16 ` [PATCH net-next v4 13/12] net: lan865x: optional hardware reset Ramón Nordin Rodriguez
2024-04-28 23:17   ` Andrew Lunn
2024-04-29 10:42     ` Ramón Nordin Rodriguez
2024-04-29  6:09   ` Parthiban.Veerasooran
2024-04-29 10:38     ` Ramón Nordin Rodriguez
2024-04-29 12:19       ` Andrew Lunn
2024-04-30  8:04         ` Ramón Nordin Rodriguez
2024-04-30 13:30         ` Parthiban.Veerasooran
2024-04-30 14:14           ` Andrew Lunn
2024-05-02 10:10             ` Parthiban.Veerasooran
2024-04-29  6:09   ` Krzysztof Kozlowski
2024-04-29 10:44     ` Ramón Nordin Rodriguez

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ZkG2Kb_1YsD8T1BF@minibuilder \
    --to=ramon.nordin.rodriguez@ferroamp.se \
    --cc=Horatiu.Vultur@microchip.com \
    --cc=Nicolas.Ferre@microchip.com \
    --cc=Parthiban.Veerasooran@microchip.com \
    --cc=Pier.Beruto@onsemi.com \
    --cc=Selvamani.Rajagopal@onsemi.com \
    --cc=Steen.Hegelund@microchip.com \
    --cc=Thorsten.Kummermehr@microchip.com \
    --cc=UNGLinuxDriver@microchip.com \
    --cc=andrew@lunn.ch \
    --cc=anthony.l.nguyen@intel.com \
    --cc=benjamin.bigler@bernformulastudent.ch \
    --cc=conor+dt@kernel.org \
    --cc=corbet@lwn.net \
    --cc=davem@davemloft.net \
    --cc=devicetree@vger.kernel.org \
    --cc=edumazet@google.com \
    --cc=horms@kernel.org \
    --cc=krzysztof.kozlowski+dt@linaro.org \
    --cc=kuba@kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=robh+dt@kernel.org \
    --cc=ruanjinjie@huawei.com \
    --cc=saeedm@nvidia.com \
    --cc=vladimir.oltean@nxp.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).