Re: [dpdk-dev] [PATCH v1 02/12] ethdev: add eswitch port item to flow API

* Re: [dpdk-dev] [PATCH v1 02/12] ethdev: add eswitch port item to flow API
@ 2021-10-06 15:30 Ivan Malov
  2021-10-06 18:00 ` Ivan Malov
  0 siblings, 1 reply; 16+ messages in thread
From: Ivan Malov @ 2021-10-06 15:30 UTC (permalink / raw)
  To: Ori Kam, Andrew Rybchenko, dev; +Cc: Thomas Monjalon

Hi Ori,

By the looks of it, we are starting to run into slight misunderstanding.

As I see it, the main consequence of our Sep 14 gathering in Jitsi was
understanding of the fact that the concept of item / action PORT_ID is
vague and needs a replacement. As a bare minimum, separate items
should be used: one for an ethdev port and another one for
the "represented entity".

This "represented entity" can be network (via network port), or a guest
machine (via a PF / VF plugged to it), or another ethdev (in the case
when the ethdev we are looking at is a PF/VF representor and this
PF/VF is also plugged to the DPDK application).

So, if I get this right, you don't object this summary. Very well.

But, in the current approach, we stick with term "ESWITCH_PORT" for
that "represented entity", and, as I see it, this term is not quite
descriptive when someone tries to understand which exact port of
the embedded switch is implied. Attempts to clarify it by virtue
of terms "external" or "the most remote" are not-so-successful.

I fully understand that.

But the good news is that the original diagram of the new concept can
be improved in a way that can allow to dispose of misleading words.

      [ A ]       <-- ethdev
        |
      [ B ]       <-- embedded switch (logical) port
        |
        |
        |
===============  <-- plane of symmetry
        |
        |
        |
      [ C ]       <-- embedded switch (logical) port
        |
      [ D ]       <-- represented entity

1. The application sees the ethdev (A) and can assume that this
    ethdev is served by some logical port (B) in the embedded
    switch. Precise nature of this port is vendor-specific.

    For example, for a regular (non-representor) ethdev,
    this port can be a PF, or a VF. This is obvious to
    DPDK developers, but the application doesn't need
    to have this knowledge. It only sees the ethdev.

    If this ethdev is a representor, port (B) can be a truly
    separate logical port or, alternatively, some vendors
    may use "PF + metadata" approach. This port (B) is
    just assumed to exist. The rest is vendor-specific.

2. The application fully understands that the "wire" plugged to
    the ethdev it sees has an opposite end. Over there, there's
    some "represented entity" (D). Once again, the application
    does not know the nature of that "represented entity".
    To it, this entity is just some traffic endpoint.

    And the application understands that this "represented entity"
    is connected to the NIC by means of another logical port (C).
    The nature of this port is unknown. The application does not
    need to have this knowledge. To it, this port just exists.

    Examples of precise nature of (C) / (D):
    -- (D) = network;        (C) = network port
    -- (D) = guest machine;  (C) = VF / PF passed to the VM
    -- (D) = another ethdev; (C) = the ethdev's logical port

    For "the ethdev's logical port" - see the explanation in (1).

3. The pair of points (A), (B) is SYMMETRICAL to the pair
    of points (C), (D). With respect to any given ethdev,
    the embedded switch is split by a plane of symmetry.

4. The property of symmetry in this drawing allows to perceive
    point (D) as a reflection of point (A). A mirrored image.

So, the short of it, looking at the problem the way this picture
looks at it, one can easily understand that the "represented
entity" is just a reflection of the ethdev (and vice versa).

No more need for wordy descriptions like "the most remote port".
The symmetry says it all. But the problem is the name for the
new item and action. Instead of ESWITCH_PORT, something else
should be suggested to make readers think of this symmetry.

To me, the most clear term here would be MIRROR_PORT. As per the rules
of English language, "MIRROR" here is like an adjective. It describes
the property of the "PORT" being a reflection of the corresponding
ethdev. And I'd call the latter an ETHDEV_PORT.

So, ETHDEV_PORT and its MIRROR_PORT.

Yes-yes, I do understand that "mirror" may at some point turn
misleading to some readers who may perceive this word as a
verb and confuse the item / action meaning with the concept
of port mirroring. Alright: I'm aware of that. So let me
then suggest another option: REFLEX_PORT.

REFLEX_PORT doesn't seem to sound as good as MIRROR_PORT,
but it also indicates the property of this entity being
a reflection of the ethdev port (and vice versa).

What do you think?  Does it make sense to make another version of
our patch series with the new term in use and with the new
explanation? Should you wish to suggest your own naming
variants, you're welcome to do so.

On 05/10/2021 15:12, Ivan Malov wrote:
 > Hi Ori, Andrew,
 >
 > On 05/10/2021 12:19, Andrew Rybchenko wrote:
 > > Hi Ori,
 > >
 > > On 10/5/21 9:20 AM, Ori Kam wrote:
 > >> Hi Ivan,
 > >>
 > >>> -----Original Message-----
 > >>> From: Ivan Malov <Ivan.Malov at oktetlabs.ru>
 > >>> Cc: dev at dpdk.org
 > >>> Subject: Re: [PATCH v1 02/12] ethdev: add eswitch port item to 
flow API
 > >>>
 > >>> Hi Ori,
 > >>>
 > >>> On 04/10/2021 14:37, Ori Kam wrote:
 > >>>> Hi Ivan,
 > >>>>
 > >>>>> -----Original Message-----
 > >>>>> From: Ivan Malov <Ivan.Malov at oktetlabs.ru>
 > >>>>> Sent: Monday, October 4, 2021 2:06 PM
 > >>>>> Cc: dev at dpdk.org
 > >>>>> Subject: Re: [PATCH v1 02/12] ethdev: add eswitch port item to flow
 > >>>>> API
 > >>>>>
 > >>>>> Hi Ori,
 > >>>>>
 > >>>>> On 04/10/2021 08:45, Ori Kam wrote:
 > >>>>>> Hi Ivan,
 > >>>>>>
 > >>>>>>> -----Original Message-----
 > >>>>>>> From: Ivan Malov <Ivan.Malov at oktetlabs.ru>
 > >>>>>>> Sent: Sunday, October 3, 2021 9:11 PM
 > >>>>>>> Subject: Re: [PATCH v1 02/12] ethdev: add eswitch port item 
to flow
 > >>>>>>> API
 > >>>>>>>
 > >>>>>>>
 > >>>>>>>
 > >>>>>>> On 03/10/2021 15:40, Ori Kam wrote:
 > >>>>>>>> Hi Andrew and Ivan,
 > >>>>>>>>
 > >>>>>>>>> -----Original Message-----
 > >>>>>>>>> From: Andrew Rybchenko <andrew.rybchenko at oktetlabs.ru>
 > >>>>>>>>> Sent: Friday, October 1, 2021 4:47 PM
 > >>>>>>>>> Subject: [PATCH v1 02/12] ethdev: add eswitch port item to flow
 > >>>>>>>>> API
 > >>>>>>>>>
 > >>>>>>>>> From: Ivan Malov <ivan.malov at oktetlabs.ru>
 > >>>>>>>>>
 > >>>>>>>>> For use with "transfer" flows. Supposed to match traffic 
entering
 > >>>>>>>>> the e-switch from the external world (network, guests) via the
 > >>>>>>>>> port which is logically connected with the given ethdev.
 > >>>>>>>>>
 > >>>>>>>>> Must not be combined with attributes "ingress" / "egress".
 > >>>>>>>>>
 > >>>>>>>>> This item is meant to use the same structure as ethdev item.
 > >>>>>>>>>
 > >>>>>>>>
 > >>>>>>>> In case the app is not working with representors, meaning each
 > >>>>>>>> switch port is mapped to ethdev.
 > >>>>>>>> both items (ethdev and eswitch port ) have the same meaning?
 > >>>>>>>
 > >>>>>>> No. Ethdev means ethdev, and e-switch port is the point where 
this
 > >>>>>>> ethdev is plugged to. For example, "transfer + ESWITCH_PORT" 
for a
 > >>>>>>> regular PF ethdev typically means the network port (maybe you can
 > >>>>>>> recall the idea that a PF ethdev "represents" the network 
port it's
 > >>>>> associated with).
 > >>>>>>>
 > >>>>>>> I believe, that diagrams which these patches add to
 > >>>>>>> "doc/guides/prog_guide/rte_flow.rst" may come in handy to
 > >>>>>>> understand the meaning. Also, you can take a look at our larger
 > >>>>>>> diagram from the Sep 14 gathering.
 > >>>>>>>
 > >>>>>>
 > >>>>>> Lets look at the following system:
 > >>>>>> E-Switch has 3 ports - PF, VF1, VF2
 > >>>>>> The ports are distributed as follows:
 > >>>>>> DPDK application:
 > >>>>>> ethdev(0) pf,
 > >>>>>> ethdev(1) representor to VF1
 > >>>>>> ethdev(2) representor to VF2
 > >>>>>> ethdev(3) VF1
 > >>>>>>
 > >>>>>> VM:
 > >>>>>> VF2
 > >>>>>>
 > >>>>>> As we know all representors are realy connected to the PF(at least
 > >>>>>> in this example)
 > >>>>>
 > >>>>> This example tries to say that the e-switch has 3 ports in total,
 > >>>>> and, given your explanation, one may indeed agree that *in this
 > >>>>> example* representors re-use e-switch port of ethdev=0 (with some
 > >>>>> metadata to distinguish packets, etc.). But one can hardly assume
 > >>>>> that *all* representors with any vendor's NIC are connected to the
 > >>>>> e-switch the same way. It's vendor specific. Well, at least,
 > >>>>> applications don't have this knowledge and don't need to.
 > >>>>>
 > >>>>>>
 > >>>>>> So matching on ethdev(3)  means matching on traffic sent from DPDK
 > >>>>>> port
 > >>>>> 3 right?
 > >>>>>
 > >>>>> Item ETHDEV (ethdev_id=3) matches traffic sent by DPDK port 3. 
Looks
 > >>>>> like we're on the same page here.
 > >>>>>
 > >>>>
 > >>>> Good.
 > >>>>
 > >>>>>> And matching on eswitch_port(3) means matching in traffic that 
goes
 > >>>>>> into VF1 which is the same traffic as ethdev(3) right?
 > >>>>>
 > >>>>> I didn't catch the thought about "the same traffic". Direction 
is not the
 > >>> same.
 > >>>>> Item ESWITCH_PORT (ethdev_id=3) matches traffic sent by DPDK 
port 1.
 > >>>>>
 > >>>> This is the critical part for my understanding.
 > >>>> Matching on ethdev_id(3) means matching on traffic that is 
coming from
 > >>> DPDK port3.
 > >>>
 > >>> Right.
 > >>>
 > >>>> So from E-Switch view point it is traffic that goes into VF1?
 > >>>
 > >>> No. Above you clearly say "coming from DPDK port3". That is, from 
the VF1.
 > >>> *Not* going into it. Port 3 (ethdev_id=3) *is* VF1.
 > >>>
 > >> But taffic that goes from DPDK port 3 goes into VF1,
 > >> what am I missing?
 >
 > Terms like "PF", "VF", "PHY_PORT" describe the nature of the e-switch
 > ports. In your example, DPDK port 3 is just based on VF1. The
 > application doesn't have such knowledge. To it, this is a regular ethdev.
 >
 > In order to gain correct understanding, you should imagine an observer
 > standing inside the e-switch. When that observer sees packets entering
 > the e-switch FROM the VF1, we effectively talk about packets sent by the
 > ethdev sitting on top of that VF1, that is, packets coming FROM DPDK
 > port 3. And vice versa: if you say "traffic that goes into VF1", you
 > mean traffic that LEAVES the e-switch via VF1, that is, traffic, going
 > TO DPDK port 3.
 >
 > "VF1" is just a name of a logical "window" between the e-switch and its
 > surroundings. If this VF is passed to a guest machine, then this VF is a
 > "window" between the e-switch and the guest machine. If you plug this VF
 > to the DPDK application (create an ethdev on top of this VF), then this
 > VF is a "window" between the e-switch and the PMD. That's it.
 >
 > >
 > > DPDK port 3 is a VF1 itself. So, is it loopback? I think no.
 > >
 > > Let me repeat your example:
 > >
 > > DPDK application:
 > > ethdev(0) pf,
 > > ethdev(1) representor to VF1
 > > ethdev(2) representor to VF2
 > > ethdev(3) VF1
 > >
 > > VM:
 > > VF2
 > >
 > > Traffic that goes from DPDK port 3 (which is VF1 bound to DPDK)
 > > goes to its representor by default, i.e. ethdev(1).
 >
 > +1
 >
 > >
 > >>
 > >>>> While matching on E-Switch_port(3) means matching on traffic coming
 > >>> from VF1?
 > >>>
 > >>> No. It means matching on traffic coming from ethdev 1. From the VF1's
 > >>> representor.
 > >>>
 > >>>>
 > >>>> And by the same logic matching on ethdev_id(1) means matching on
 > >>>> taffic that was sent from DPDK port 1 and matching on 
E-Switch_port(1)
 > >>>> means matching on traffic coming from
 > >>>> VF1
 > >>>
 > >>> In this case, you've got this right. But please see my above 
notes. By the
 > >>> looks of it, you might have run into confusion over there.
 > >>>
 > >> That is the issue I'm not sure I understand, and I think that
 > >> if I don't underdand this, this miss underdanding will be shared 
by others.
 >
 > Please see my diagram below.
 >
 > >
 > > In order to address the confusion and misunderstanding we
 > > should find out the source of it. We always match on source
 > > (inbound port) and the result is a destination (outgoing port).
 > > So, if item is ETHDEV, the source is DPDK application via
 > > corresponding ethdev port.  If ETHDEV is an action, the
 > > destination is the DPDK application via specified ethdev port.
 > > Above is regardless of the ethdev port type (representor or
 > > not). Always, no exceptions.
 > >
 > >>>>
 > >>>> So in this case eswitch_port(3) is equal ot eswitch_port(1) right?
 > >>>> While ethdev(1) is not equal to ethdev(3)
 > >>>
 > >>> No.
 > >>>
 > >>> Item ETHDEV (ethdev_id=1) equals item ESWITCH_PORT (ethdev_id=3).
 > >>> Item ETHDEV (ethdev_id=3) equals item ESWITCH_PORT (ethdev_id=1).
 > >>>
 > >> I think this was my first undestaning, lets see if I can explain 
it using my words.
 > >> ETHDEV - means that the source port that we are matching on is the 
closest one
 > >> the dpdk application.
 > >
 > > Yes, it sounds right.
 > >
 > >> Example like above ETHDEV(ethdev_id=1) means matching
 > >> on traffic coming from the PF with some metadata that marks this 
DPDK port.
 >
 > Some vendors support allocation of separate logical e-switch ports for
 > representors. Other vendors can't support that, and, in this case, they
 > indeed have to devise "PF + metadata" solution. But DPDK framework (I
 > mean, vendor-agnostic code) can't assume any of these options as a
 > "standard". Neither can applications do that. It's truly vendor-specific.
 >
 > >
 > > No, no, no. It is the problem of too much knowledge and
 > > diving too deep. Neither you nor API user should not
 > > think about it. Representor ethdev(1) is a DPDK ethdev
 > > port that's it. An application can send traffic via the
 > > ethdev port and receive traffic from it. How the traffic
 > > goes inside is vendor-specific implementation detail.
 > > Application just know that if it sends traffic to VF1
 > > representor ethdev(1), the traffic will be received
 > > from VF1 by default. If it sends traffic from VF1,
 > > the traffic will be received from VF1 representor
 > > by default. It is the definition of the representor.
 >
 > +1
 >
 > >
 > >> ETHDEV(ethdev_id=3) means matching on traffic coming from VF1.
 > >
 > > Yes.
 > >
 > >>
 > >> ESWITCH_PORT meaning matching on the port that is connected to the 
DPDK port
 > >> (the other side of the wire). Example ESWITCH_PORT(ethdev_id=1) 
means matching
 > >> on traffic coming from VF1
 > >
 > > Yes, since ethdev(1) is VF1 representor.
 > >
 > >> While matching on ESWITCH_PORT(ethdev_id=3) means matching on PF 
with some
 > >> metadata.
 > >
 > > Again, yes and no. No, since you should not think about it this way. It
 > > is just traffic sent via VF1 representor (since ethdev(3) is VF1 
and its
 > > other side of the logical wire is VF1
 > > representor). No vendor-specific implementation details.
 > >
 > >>
 > >> Everything assume that representors are on the PF and use some 
metadata.
 > >
 > > No, it is a vendor-specific implementation details.
 > > Flow API definitions should not mention it.
 >
 > +1
 >
 > >
 > >>
 > >> Did I get it right?
 >
 > Let's try to look at the overall idea one more time. Just to get us on
 > the same page.
 >
 > In general, for any given ethdev you have:
 >
 >
 > ETHDEV (A)
 > |
 > |    (PMD area)
 > |
 > CLOSEST ESWITCH_PORT (B)
 > |
 > |    (e-switch area)
 > |
 > REMOTE ESWITCH_PORT (C)
 > |
 > X
 >
 >
 > The point "B" is located closest to the ethdev (A). Not to the
 > application in general, but to the given ethdev.
 >
 > The point "C" is the most remote point from the ethdev's perspective.
 > Again, it's not the most remote from the application. It's the most
 > remote from this specific ethdev.
 >
 >  From the application perspective, the nature of (B) and (C) is unknown.
 > But the application knows for sure that these points just exist. To it,
 > these points are some *logical* e-switch ports.
 >
 > Now.
 >
 > When you say "item ETHDEV", you tell the PMD that you want to match
 > packets entering the *PMD area* FROM point (A). But you also add
 > "transfer" attribute. This attribute makes the PMD translate your
 > request to the e-switch viewpoint by "transferring" point "A" one level
 > below. What was "A" becomes "B". This way, an imaginary observer
 > standing inside the *e-switch area* intercepts packets which enter this
 > area FROM (B).
 > Summary: this way, you match packets going *DOWN* from (A).
 >
 > When you say "item ESWITCH_PORT", you tell the PMD that you want to
 > match packets entering the *PMD area* FROM point (B). Once again,
 > "transfer" translates this to the e-switch viewpoint, that is, (B)
 > becomes (C). So, the e-switch knows that it should intercept packets
 > entering its area FROM point (C).
 > Summary: this way, you match packets going *UP* from (X).
 >
 >
 > Some use case examples:
 >
 > - The ethdev (A) is a DPDK port based on a PF associated with a network
 > port. In this case, (B) == PF and (C) == (X) == network port.
 >
 > - The ethdev (A) is a representor of VF1 which is passed to a VM. In
 > this case, (B) is just an e-switch logical port. It's nature is vendor
 > specific. It can be a PF, it can be not. It depends. (C) == VF, (X) 
== VM.
 >
 > - The ethdev (A) is a representor of VF1 which is plugged to the same
 > DPDK application. In this case, (B) is, once again, some logical
 > e-switch port. (C) == VF. But (X) here is another ethdev. So, this way,
 > ethdev (A) is a representor's ethdev and ethdev (X) is a VF's ethdev.
 >
 > In all cases the application doesn't care about the nature of the
 > e-switch ports (B) and (C). It just knows that they exist.
 >
 > >
 > > I think you get overall idea right, but explanations
 > > are too complicated. It is simpler in fact.
 > >
 > >>>>
 > >>>> And just to complete the picture, matching on ethdev(2) will 
result in
 > >>>> traffic coming from the dpdk port and matching on 
eswitch_port(2) will
 > >>>> match on traffic coming from VF2
 > >>>
 > >>> Exactly.
 > >>>
 > >>>
 > >>> But, Ori, let me draw your attention to the following issue. In 
order to
 > >>> simplify understanding, I suggest that we refrain from saying 
"traffic that
 > >>> GOES TO". Where it goes depends on default rules that are 
supposed to be
 > >>> maintained by the PMD when ports get plugged / unplugged.
 > >>>
 > >>> The flow items ETHDEV and ESWITH_PORT define the SOURCE of traffic.
 > >>> That's it. They define where the traffic "goes FROM".
 > >>>
 > >>> Say, the DPDK application sends a packet from ethdev 0. This 
packet enters
 > >>> the e-switch. Match engine sits in the e-switch and intercepts 
the packet. It
 > >>> doesn't care where the packet *would go* if it wasn't 
intercepted. It cares
 > >>> about where the packet comes from. And it comes from ethdev 0. 
So, in the
 > >>> focus, we have the SOURCE of the packet.
 > >>>
 > >>
 > >> Agree with you we should only look at coming from,
 > >> but something in the patch made me thing otherwise (not sure what 
part)
 > >
 > > I've reread the documentation and failed to find,
 > > but it will be hard for me to do it, since I read
 > > it too many times already.
 > >
 > > Thanks,
 > > Andrew.
 > >

-- 
Ivan M

^ permalink raw reply	[flat|nested] 16+ messages in thread