Re: Port mirroring, v2 (RFC)

[Alex Elder <elder@linaro.org>]

Yikes!  I don't know why that turned out double-spaced.  I hope
this one turns out better.

					-Alex

This is a second RFC for a design to implement new functionality
in the Qualcomm IPA driver.  Since last time I've looked into some
options based on feedback.  This time I'll provide some more detail
about the hardware, and what the feature is doing.  And I'll end
with two possible implementations, and some questions.

My objective is to get a general sense that what I plan to do
is reasonable, so the patches that implement it will be acceptable.


The feature provides the AP access to information about the packets
that the IPA hardware processes as it carries them between its
"ports".  It is intended as a debug/informational interface only.
Before going further I'll briefly explain what the IPA hardware
does.

The upstream driver currently uses the hardware only as the path
that provides access to a 5G/LTE cellular network via a modem
embedded in a Qualcomm SoC.

        \|/
         |
   ------+-----   ------
   | 5G Modem |   | AP |
   ------------   ------
              \\    || <-- IPA channels, or "ports"
             -----------
             |   IPA   |
             -----------

But the hardware also provides a path to carry network traffic to
and from other entities as well, such as a PCIe root complex (or
endpoint).  For example an M.2 WiFi card can use a PCIe slot that
is IPA connected, and the IPA hardware can carry packets between
the AP and that WiFi card.  (A separate MHI host driver manages the
interaction between PCIe and IPA in this case.)

        \|/                PCIe bus --.     \|/
         |                            |      |
   ------+-----  ------   ----------- v ------+-----
   | 5G Modem |  | AP |...| PCIe RC |===| M.2 WiFi |
   ------------  ------   -----------   ------------
              \\   ||    // <-- IPA channels
               -----------
               |   IPA   |
               -----------

In the above scenario, the IPA hardware is actually able to directly
route packets between the embedded modem and the WiFi card without
AP involvement.  But supporting that is a future problem, and I
don't want to get ahead of myself.

The point is that the IPA hardware can carry network packets between
any pair of its "ports".  And the AP can get information about all
of the traffic the IPA handles, using a special interface.

The "information" consists of replicas of each packet transferred
(possibly truncated), each preceded by a fixed-size "status" header.
It amounts to a stream of packets delivered by the IPA hardware to
the AP.  This stream is distinct from "normal" traffic (such as
packets exchanged between the AP and modem); but note that even
those packets would be replicated.


I originally described this feature as "port mirroring" because it
seemed to be similar to that feature of smart network switches.  But
the "mirroring" term was interpreted as a something Linux would do,
so at a minimum, that's the wrong term.  Andrew Lunn (and others)
suggested that WiFi monitor mode might be a good model.  I looked
into that, and I don't think that quite fits either.  I really think
this should be represented separate from the "normal" network
devices associated with IPA.


Below I will describe two possible implementations I'm considering.
I would like to know which approach makes the most sense (or if
neither does, what alternative would be better).  On top of that I
guess I'd like suggestions for the *name* for this (i.e., what
should I call the interface that implements this?).

The two alternative implementations I'm considering are a network
device, and a "misc" (character) device.  In both cases, a user
space program would open the interface and read from it.  The data
read would just be the raw data received--a stream of the (possibly
truncated) packets together with their "status" headers.  I envision
either one could be a source of packets processed by libpcap/tcpdump.

My preference is to use a network device.  I think it fits the
"stream of packets" best, and existing networking code would take
care of all the details of queueing and packet management.  One down
side is that this is not a "normal" network interface--there is no
reason to associate it with an IP stack, for example.

A misc device would avoid the interface being treated as a "normal"
network device.  It could present all packet data to user space, but
the IPA driver would have to manage buffering, including limiting
the amount of received buffers.  Implementing this would be fine,
but I think it would just be nicer to use the network model.


So bottom line, given what I've described above:
- Is a distinct network device a reasonable and acceptable way of
   implementing this feature?  If not, why not?
- Would implementing this as a misc device be preferable?  Why?
- Is there a better alternative than either of the above?
- Can anyone suggest a name for this functionality, something that
   is meaningful but would not be confused with other existing terms?

Thanks.

					-Alex