Re: [RFC PATCH v7 00/16] Add support for qca8k mdio rw in Ethernet packet

From: Ansuel Smith <ansuelsmth@gmail.com>
To: Jakub Kicinski <kuba@kernel.org>
Cc: Vladimir Oltean <olteanv@gmail.com>,
	Florian Fainelli <f.fainelli@gmail.com>,
	Andrew Lunn <andrew@lunn.ch>,
	Vivien Didelot <vivien.didelot@gmail.com>,
	"David S. Miller" <davem@davemloft.net>,
	linux-kernel@vger.kernel.org, netdev@vger.kernel.org
Subject: Re: [RFC PATCH v7 00/16] Add support for qca8k mdio rw in Ethernet packet
Date: Thu, 3 Feb 2022 21:31:34 +0100	[thread overview]
Message-ID: <Yfw7phxyufXvowJl@Ansuel-xps.localdomain> (raw)
In-Reply-To: <20220203121027.7a6ea0f8@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>

On Thu, Feb 03, 2022 at 12:10:27PM -0800, Jakub Kicinski wrote:
> On Thu, 3 Feb 2022 20:21:28 +0200 Vladimir Oltean wrote:
> > To my knowledge, when you call dev_queue_xmit(), the skb is no longer
> > yours, end of story, it doesn't matter whether you increase the refcount
> > on it or not. The DSA master may choose to do whatever it wishes with
> > that buffer after its TX completion interrupt fires: it may not call
> > napi_consume_skb() but directly recycle that buffer in its pool of RX
> > buffers, as part of some weird buffer recycling scheme. So you'll think
> > that the buffer is yours, but it isn't, because the driver hasn't
> > returned it to the allocator, and your writes for the next packet may be
> > concurrent with some RX DMA transactions. I don't have a mainline
> > example to give you, but I've seen the pattern, and I don't think it's
> > illegal (although of course, I stand to be corrected if necessary).
> 
> Are we talking about holding onto the Tx skb here or also recycling 
> the Rx one? Sorry for another out of context comment in advance..
>

Here we are talking about tx skb. (We can't really operate on the
received skb handled by the tagger)
What I want to improve is the fact that we alloc 3 very small skb that
will have only some part of them modified and the rest equal on all 3
skb. So my idea is find a way to preallocate a space and build a skb
around it. In short as we force a 1:1 comunication where we send an skb
and we wait for the response to be processed, we can reuse the same skb
everytime and just modify the data in it. I'm asking if anyone have some
hint/direction without proposing a patch that is a big massive hack that
will be NACK in 0.1 ms LOL

> AFAIK in theory shared skbs are supposed to be untouched or unshared
> explicitly by the driver on Tx. pktgen takes advantage of it.
> We have IFF_TX_SKB_SHARING. 
> 

Will check how this flags is used. If you have any hint about this feel
free to suggest it.

> In practice everyone gets opted into SKB_SHARING because ether_setup()
> sets the flag. A lot of drivers are not aware of the requirement and
> will assume full ownership (and for example use skb->cb[]) :/
> 

Consider that currently this will be used by stmmac driver, brcm driver
and someother atheros driver.

> I don't think there is any Tx completion -> Rx pool recycling scheme
> inside the drivers (if that's what you described).

-- 
	Ansuel