[Bridge] Bridge and PACKET-socket

[Bridge] Bridge and PACKET-socket

[Luke Gorrie]

Ahoy,

I've encountered some confusing semantics with using PACKET(7) sockets
on bridge-enslaved interfaces. Specifically, if my socket accepts all
types of frame (bind() to ETH_P_ALL) then it gets all packets; but if
it accepts any specific type (e.g. ETH_P_IP), then it receives no
packets at all.

That is how it's coded in net/core/dev.c's netif_receive_skb(). First
ETH_P_ALL handlers are called, then the bridge, and then -- only if
the bridge declines the packet -- the protocol-specific handlers are
called.

My interpretation is that ETH_P_ALL handlers are expected to be
programs like tcpdump which can safely be fed all packets, whereas
more specific handlers are expected to be actual protocol
implementations that shouldn't run directly on enslaved ports.

Is that correct?

In my case I am implementing a specific protocol in userspace (via
PACKET-socket) and this protocol must run directly on enslaved
ports. It's an old layer-2 neighbour-discovery protocol that has to
operate on physical ports, below other abstractions like
bridges. ("don't ask.") It looks like I can't do this today, unless I
use ETH_P_ALL, perhaps with a BPF filter -- but I'm fearful of
introducing overhead on all packets.

I was thinking of adding an extra hashtable in dev.c, like ptype_base
but processed before offering packets to the bridge. Then an ioctl to
have a PACKET-socket moved into this table if it's supposed to receive
packets on enslaved interfaces.

Does that sound like a good solution?

Cheers,
Luke

Re: [Bridge] Bridge and PACKET-socket

[Bart De Schuymer]

On Monday 05 January 2004 23:33, Luke Gorrie wrote:
> My interpretation is that ETH_P_ALL handlers are expected to be
> programs like tcpdump which can safely be fed all packets, whereas
> more specific handlers are expected to be actual protocol
> implementations that shouldn't run directly on enslaved ports.
>
> Is that correct?

Well, with the ebtables BROUTING chain it is possible to have an enslaved port 
as input device for the protocol handlers. In the case of IP, to make any 
sense, this enslaved port should then have its own IP address.

> In my case I am implementing a specific protocol in userspace (via
> PACKET-socket) and this protocol must run directly on enslaved
> ports. It's an old layer-2 neighbour-discovery protocol that has to
> operate on physical ports, below other abstractions like
> bridges. ("don't ask.") It looks like I can't do this today, unless I
> use ETH_P_ALL, perhaps with a BPF filter -- but I'm fearful of
> introducing overhead on all packets.

Well, if you're sending all IP packets to userspace I doubt that's faster than 
an ebtables kernel module...

> I was thinking of adding an extra hashtable in dev.c, like ptype_base
> but processed before offering packets to the bridge. Then an ioctl to
> have a PACKET-socket moved into this table if it's supposed to receive
> packets on enslaved interfaces.
>
> Does that sound like a good solution?

I don't know the details, but can't you hook some kernel function onto 
ETH_P_ALL that only sends IP packets to userspace?

cheers,
Bart

Re: [Bridge] Bridge and PACKET-socket

[Luke Gorrie]

Bart De Schuymer <bdschuym@pandora.be> writes:

> Well, if you're sending all IP packets to userspace I doubt that's faster than 
> an ebtables kernel module...

Right, I forgot to mention that in this case it's only 802.3 frames
that I'm interested in. I expect to get very few packets, and don't
want to impact the performance of the other IP-based traffic.

Also, I've already implemented the protocol in userspace and don't
wanna move it into the kernel ;-)

> I don't know the details, but can't you hook some kernel function onto 
> ETH_P_ALL that only sends IP packets to userspace?

This still sounds like packet-socket territory to me. Wouldn't it be
better to clarify/extend the packet-socket interface to cover bridged
ports?

-Luke

Re: [Bridge] Bridge and PACKET-socket

[Jason Lunz]

luke@member.fsf.org said:
> This still sounds like packet-socket territory to me. Wouldn't it be
> better to clarify/extend the packet-socket interface to cover bridged
> ports?

I think you can just use a packet socket with BPF; that's what dhcpd
does and it doesn't seem to incur noticable overhead on non-dhcp
traffic.

Jason

Re: [Bridge] Bridge and PACKET-socket

[Luke Gorrie]

Jason Lunz <lunz@falooley.org> writes:

> luke@member.fsf.org said:
> > This still sounds like packet-socket territory to me. Wouldn't it be
> > better to clarify/extend the packet-socket interface to cover bridged
> > ports?
> 
> I think you can just use a packet socket with BPF; that's what dhcpd
> does and it doesn't seem to incur noticable overhead on non-dhcp
> traffic.

Good point. I had assumed that any per-packet overhead would be
unacceptable since the box has to get "maximum" throughput. But we're
already running a PACKET-socket based DHCP relay, and I don't think
that impacted performance measurably.

Thanks for the tip!

Still, I find the current semantics of packet sockets on bridged ports
pretty confusing. It took me a full day to figure out why I wasn't
receiving the packets I wanted, even though tcpdump did.

I applied my proposed fix in my local tree: I have a separate clone of
the ptype_base hashtable dev.c for specific-protocol handlers that
want to run before the bridge, and an ioctl to move a packet-socket
into that table. That way the user can choose whether he wants to get
packets from enslaved interfaces or not.

If Someone Important thinks that change sounds reasonable and should
be made in Linux then I can port it to 2.6 and send a patch.

Cheers,
Luke

Re: [Bridge] Bridge and PACKET-socket

[Bart De Schuymer]

On Monday 12 January 2004 22:02, Luke Gorrie wrote:
> Still, I find the current semantics of packet sockets on bridged ports
> pretty confusing. It took me a full day to figure out why I wasn't
> receiving the packets I wanted, even though tcpdump did.

Well, that list for protocol-specific functions contains functions to deal 
with the packet, e.g. route it. Obviously we don't want these functions to be 
executed before the bridge code can have a look at it, so that's not very 
confusing.
If you really want to use the protocol-specific list for packets entering on a 
bridge port, you can. Just use the ebtables brouting chain to "broute" the 
packet, see the website. I guess you lose a little performance there, but you 
shouldn't really feel it.

cheers,
Bart

Re: [Bridge] Bridge and PACKET-socket

[Luke Gorrie]

Bart De Schuymer <bdschuym@pandora.be> writes:

> On Monday 12 January 2004 22:02, Luke Gorrie wrote:
> > Still, I find the current semantics of packet sockets on bridged ports
> > pretty confusing. It took me a full day to figure out why I wasn't
> > receiving the packets I wanted, even though tcpdump did.
> 
> Well, that list for protocol-specific functions contains functions to deal 
> with the packet, e.g. route it. Obviously we don't want these functions to be 
> executed before the bridge code can have a look at it, so that's not very 
> confusing.

I agree that dev.c is not confusing, but the PACKET(7) interface
is. If you bind to ETH_P_ALL you will receive every packet, but if you
bind to any other protocol you will only receive packets the bridge
doesn't take. The manpage doesn't say anything about this, and one
might actually want different semantics (as in my case). IMO it would
be nicer to have an interface that treats "what protocol do I want to
see" and "do I want bridged packets" as orthogonal.

> If you really want to use the protocol-specific list for packets entering on a 
> bridge port, you can. Just use the ebtables brouting chain to "broute" the 
> packet, see the website. I guess you lose a little performance there, but you 
> shouldn't really feel it.

I love ebtables and I'm using it extensively. But there is a limit to
how much more logic I can push into it (not to mention iptables)
before the whole thing becomes incomprehensible :-)

In this case I have a simple program that wants to receive and send
802.3 packets on every physical port on the machine, irrespective of
bridging, etc. The modified packet socket interface is a great match.

.. but come to think of it, I'd better see what happens with packet
sockets and the bonding driver too. Maybe it will spoil my fun.

Cheers,
Luke

[Bridge] Re: Bridge and PACKET-socket

[Jason Lunz]

luke@bluetail.com said:
> Still, I find the current semantics of packet sockets on bridged ports
> pretty confusing. It took me a full day to figure out why I wasn't
> receiving the packets I wanted, even though tcpdump did.
>
> I applied my proposed fix in my local tree: I have a separate clone of
> the ptype_base hashtable dev.c for specific-protocol handlers that
> want to run before the bridge, and an ioctl to move a packet-socket
> into that table. That way the user can choose whether he wants to get
> packets from enslaved interfaces or not.

I'm not sure you need to do that. On frame recieve, the loop over
ptype_base happens twice on a bridge, once before the bridging and once
after. For locally-destined packets, that is. (br_pass_frame_up() ends
up using netif_rx, sending the frame through dev.c again, right?)

If you want to see frames before the bridge has a chance to mess
w/ them, then bind your packet socket to the slave interface; otherwise,
bind to the virtual-bridge interface. Will this work? forwarded packets
will be hooked into open packet sockets somewhere in the TX path, iirc.

I may be misunderstanding you, but have you explored this?

Jason

Re: [Bridge] Bridge and PACKET-socket

[Bart De Schuymer]

On Monday 12 January 2004 23:14, Luke Gorrie wrote:
> I agree that dev.c is not confusing, but the PACKET(7) interface
> is. If you bind to ETH_P_ALL you will receive every packet, but if you
> bind to any other protocol you will only receive packets the bridge
> doesn't take. The manpage doesn't say anything about this, and one
> might actually want different semantics (as in my case). IMO it would
> be nicer to have an interface that treats "what protocol do I want to
> see" and "do I want bridged packets" as orthogonal.

Reading the man page is indeed confusing for a user, w.r.t. bridge ports.
I think it would be more logical if all PF_PACKETsockets see the frames before 
the bridge code.
How about placing the call to __handle_bridge() right after the second 
list_for_each? If I'm not mistaken the relevant pt_pre->func that would deal 
with the packet will not have been executed yet, while those PF_PACKET 
functions will already have been called...

If you want the opinion of someone more knowledgeable than my humble self, the 
network guru's are located at netdev@oss.sgi.com.

cheers,
Bart

Re: [Bridge] Bridge and PACKET-socket

[Eble, Dan]

On Tuesday 13 January 2004 12:41:55 -0800, Bart De Schuymer wrote:
> Reading the man page is indeed confusing for a user, w.r.t. bridge ports.
> I think it would be more logical if all PF_PACKET
> sockets see the frames before the bridge code.

I agree.  I just ran into this problem trying to read bridge PDUs from a
daemon listening on a packet socket.  I'm glad I found this thread; it saved
me a lot of debugging time.

> How about placing the call to __handle_bridge() right after the second 
> list_for_each? If I'm not mistaken the relevant pt_pre->func that would
deal 
> with the packet will not have been executed yet, while those PF_PACKET 
> functions will already have been called...

Has anyone tested this in the two weeks since it was suggested?

-- 
Dan Eble <dane@aiinet.com>  _____  .
                           |  _  |/|
Applied Innovation Inc.    | |_| | |    "All Protocols MUST be odd"
http://www.aiinet.com/     |__/|_|_|      -- RFC 1661 (PPP)