Re: [PATCH net] net/sched: Don't print dump stack in event of transmission timeout

From: Cong Wang <xiyou.wangcong@gmail.com>
To: Jakub Kicinski <kuba@kernel.org>
Cc: Leon Romanovsky <leon@kernel.org>,
	"David S. Miller" <davem@davemloft.net>,
	Leon Romanovsky <leonro@mellanox.com>,
	Arjan van de Ven <arjan@linux.intel.com>,
	Jamal Hadi Salim <jhs@mojatatu.com>,
	Jiri Pirko <jiri@resnulli.us>,
	Linux Kernel Network Developers <netdev@vger.kernel.org>,
	Itay Aveksis <itayav@mellanox.com>
Subject: Re: [PATCH net] net/sched: Don't print dump stack in event of transmission timeout
Date: Thu, 2 Apr 2020 21:33:05 -0700	[thread overview]
Message-ID: <CAM_iQpVRyCpyBWabEyYLGx69ZY1+exrV+cU034vrmodeubkqaA@mail.gmail.com> (raw)
In-Reply-To: <20200402155723.534147ac@kicinski-fedora-PC1C0HJN>

On Thu, Apr 2, 2020 at 3:57 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Thu,  2 Apr 2020 18:23:36 +0300 Leon Romanovsky wrote:
> > From: Leon Romanovsky <leonro@mellanox.com>
> >
> > In event of transmission timeout, the drivers are given an opportunity
> > to recover and continue to work after some in-house cleanups.
> >
> > Such event can be caused by HW bugs, wrong congestion configurations
> > and many more other scenarios. In such case, users are interested to
> > get a simple  "NETDEV WATCHDOG ... " print, which points to the relevant
> > netdevice in trouble.
> >
> > The dump stack printed later was added in the commit b4192bbd85d2
> > ("net: Add a WARN_ON_ONCE() to the transmit timeout function") to give
> > extra information, like list of the modules and which driver is involved.
> >
> > While the latter is already printed in "NETDEV WATCHDOG ... ", the list
> > of modules rarely needed and can be collected later.
> >
> > So let's remove the WARN_ONCE() and make dmesg look more user-friendly in
> > large cluster setups.
>
> I'm of two minds about this. As much as printing a stack dump here is
> not that useful indeed, it's certainly a good way of getting user's
> attention. TX queue time outs should never happen, and there's a bunch
> of log crawlers out there looking for kernel warnings.

Rasdaemon is also able to capture this via trace_net_dev_xmit_timeout()
and send it to ABRT too. So, I don't think this is a big problem.

[...]
> > diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c
> > index 6c9595f1048a..c12530fe8b21 100644
> > --- a/net/sched/sch_generic.c
> > +++ b/net/sched/sch_generic.c
> > @@ -439,8 +439,9 @@ static void dev_watchdog(struct timer_list *t)
> >
> >                       if (some_queue_timedout) {
> >                               trace_net_dev_xmit_timeout(dev, i);
> > -                             WARN_ONCE(1, KERN_INFO "NETDEV WATCHDOG: %s (%s): transmit queue %u timed out\n",
> > -                                    dev->name, netdev_drivername(dev), i);
> > +                             pr_info_once("NETDEV WATCHDOG: %s (%s): transmit queue %u timed out\n",
>
> I'd say pr_err_once(). Or dev_err_once().

Or pr_warn().

Thanks.