Re: [RFC PATCH 1/6] docs: networking: add the document for DFL Ether Group driver

[Xu Yilun <yilun.xu@intel.com>]

On Mon, Oct 26, 2020 at 11:35:52AM -0700, Jakub Kicinski wrote:
> On Tue, 27 Oct 2020 01:38:04 +0800 Xu Yilun wrote:
> > > > The line/host side Ether Group is not the terminal of the network data stream.
> > > > Eth1 will not paticipate in the network data exchange to host.
> > > > 
> > > > The main purposes for eth1 are:
> > > > 1. For users to monitor the network statistics on Line Side, and by comparing the
> > > > statistics between eth0 & eth1, users could get some knowledge of how the User
> > > > logic is taking function.
> > > > 
> > > > 2. Get the link state of the front panel. The XL710 is now connected to
> > > > Host Side of the FPGA and the its link state would be always on. So to
> > > > check the link state of the front panel, we need to query eth1.  
> > > 
> > > This is very non-intuitive. We try to avoid this in the kernel and the
> > > API to userspace. Ethernet switches are always modelled as
> > > accelerators for what the Linux network stack can already do. You
> > > configure an Ethernet switch port in just the same way configure any
> > > other netdev. You add an IP address to the switch port, you get the
> > > Ethernet statistics from the switch port, routing protocols use the
> > > switch port.
> > > 
> > > You design needs to be the same. All configuration needs to happen via
> > > eth1.
> > > 
> > > Please look at the DSA architecture. What you have here is very
> > > similar to a two port DSA switch. In DSA terminology, we would call
> > > eth0 the master interface.  It needs to be up, but otherwise the user
> > > does not configure it. eth1 is the slave interface. It is the user
> > > facing interface of the switch. All configuration happens on this
> > > interface. Linux can also send/receive packets on this netdev. The
> > > slave TX function forwards the frame to the master interface netdev,
> > > via a DSA tagger. Frames which eth0 receive are passed through the
> > > tagger and then passed to the slave interface.
> > > 
> > > All the infrastructure you need is already in place. Please use
> > > it. I'm not saying you need to write a DSA driver, but you should make
> > > use of the same ideas and low level hooks in the network stack which
> > > DSA uses.  
> > 
> > I did some investigation about the DSA, and actually I wrote a
> > experimental DSA driver. It works and almost meets my need, I can make
> > configuration, run pktgen on slave inf.
> > 
> > A main concern for dsa is the wiring from slave inf to master inf depends
> > on the user logic. If FPGA users want to make their own user logic, they
> > may need a new driver. But our original design for the FPGA is, kernel
> > drivers support the fundamental parts - FPGA FIU (where Ether Group is in)
> > & other peripherals on board, and userspace direct I/O access for User
> > logic. Then FPGA user don't have to write & compile a driver for their
> > user logic change.
> > It seems not that case for netdev. The user logic is a part of the whole
> > functionality of the netdev, we cannot split part of the hardware
> > component to userspace and the rest in kernel. I really need to
> > reconsider this.
> 
> This is obviously on purpose. Your design as it stands will not fly
> upstream, sorry.
> 
> >From netdev perspective the user should not care how many hardware
> blocks are in the pipeline, and on which piece of silicon. You have 
> a 2 port (modulo port splitting) card, there should be 2 netdevs, and
> the link config and forwarding should be configured through those.
> 
> Please let folks at Intel know that we don't like the "SDK in user
> space with reuse [/abuse] of parts of netdev infra" architecture.
> This is a second of those we see in a short time. Kernel is not a
> library for your SDK to use. 

I get your point. I'll share the information internally and reconsider
the design.

Thanks,
Yilun