Re: [PATCH rdma-next v1 10/15] RDMA/cm: Use an attribute_group on the ib_port_attribute intead of kobj's

From: Jason Gunthorpe <jgg@nvidia.com>
To: Greg KH <gregkh@linuxfoundation.org>
Cc: Leon Romanovsky <leon@kernel.org>,
	Doug Ledford <dledford@redhat.com>,
	Kees Cook <keescook@chromium.org>,
	Nathan Chancellor <nathan@kernel.org>,
	Adit Ranadive <aditr@vmware.com>,
	Ariel Elior <aelior@marvell.com>,
	Christian Benvenuti <benve@cisco.com>,
	clang-built-linux@googlegroups.com,
	Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>,
	Devesh Sharma <devesh.sharma@broadcom.com>,
	Gal Pressman <galpress@amazon.com>,
	linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org,
	Michal Kalderon <mkalderon@marvell.com>,
	Mike Marciniszyn <mike.marciniszyn@cornelisnetworks.com>,
	Mustafa Ismail <mustafa.ismail@intel.com>,
	Naresh Kumar PBS <nareshkumar.pbs@broadcom.com>,
	Nelson Escobar <neescoba@cisco.com>,
	Nick Desaulniers <ndesaulniers@google.com>,
	Potnuri Bharat Teja <bharat@chelsio.com>,
	Selvin Xavier <selvin.xavier@broadcom.com>,
	Shiraz Saleem <shiraz.saleem@intel.com>,
	VMware PV-Drivers <pv-drivers@vmware.com>,
	Yishai Hadas <yishaih@nvidia.com>,
	Zhu Yanjun <zyjzyj2000@gmail.com>
Subject: Re: [PATCH rdma-next v1 10/15] RDMA/cm: Use an attribute_group on the ib_port_attribute intead of kobj's
Date: Mon, 7 Jun 2021 09:50:12 -0300	[thread overview]
Message-ID: <20210607125012.GE1002214@nvidia.com> (raw)
In-Reply-To: <YL4TkfVlTellmnc+@kroah.com>

On Mon, Jun 07, 2021 at 02:39:45PM +0200, Greg KH wrote:
> On Mon, Jun 07, 2021 at 09:14:11AM -0300, Jason Gunthorpe wrote:
> > On Mon, Jun 07, 2021 at 12:25:03PM +0200, Greg KH wrote:
> > > On Mon, Jun 07, 2021 at 11:17:35AM +0300, Leon Romanovsky wrote:
> > > > From: Jason Gunthorpe <jgg@nvidia.com>
> > > > 
> > > > This code is trying to attach a list of counters grouped into 4 groups to
> > > > the ib_port sysfs. Instead of creating a bunch of kobjects simply express
> > > > everything naturally as an ib_port_attribute and add a single
> > > > attribute_groups list.
> > > > 
> > > > Remove all the naked kobject manipulations.
> > > 
> > > Much nicer.
> > > 
> > > But why do you need your counters to be atomic in the first place?  What
> > > are they counting that requires this?  
> > 
> > The write side of the counter is being updated from concurrent kernel
> > threads without locking, so this is an atomic because the write side
> > needs atomic_add().
> 
> So the atomic write forces a lock :(

Of course, but a single atomic is cheaper than the double atomic in a
full spinlock.

> > Making them a naked u64 will cause significant corruption on the write
> > side, and packet counters that are not accurate after quiescence are
> > not very useful things.
> 
> How "accurate" do these have to be?

They have to be accurate. They are networking packet counters. What is
the point of burning CPU cycles keeping track of inaccurate data?

> And have you all tried them?

I've used them over the years. This stuff is something like 15 years
old now.

> I'm pushing back here as I see a lot of atomics used for debugging
> statistics for no good reason all over the place.  Especially when
> userspace just does not care.

If userspace doesn't care then just delete the counter entirely.

Reporting a wrong/misleading debugging counter data sounds just
horrible to me.

What good is any debug result you get from the counter if it has to be
questioned because the counter is allowed to be wrong?

""The sender says it sent 7 packets, but the receivers debug counter
reports only 6! I guess my bug is a lost packet in the network.""

Jason