BUG?: receiving on a packet socket with .sll_protocoll and bridging

BUG?: receiving on a packet socket with .sll_protocoll and bridging

[Uwe Kleine-König]

Hello,

my eventual goal is to implement MRP and for that I started to program a
bit and stumbled over a problem I don't understand.

For testing purposes I created a veth device pair (veth0 veth1), open a
socket for each of the devices and send packets around between them. In
tcpdump a typical package looks as follows:

10:36:34.755208 ae:a9:da:50:48:db (oui Unknown) > 01:15:e4:00:00:01 (oui Unknown), ethertype Unknown (0x88e3), length 58: 
	0x0000:  0001 0212 8000 aea9 da50 48db 0000 0000  .........PH.....
	0x0010:  0000 0589 40f2 6574 6800 0000 0000 0000  ....@.eth.......
	0x0020:  0000 0100 0a80 3d38 4c5e 0000            ......=8L^..

The socket to receive these packages is opened using:

	#define ETH_P_MRP 0x88e3

	struct sockaddr_ll sa_ll = {
		.sll_family = AF_PACKET,
		.sll_protocol = htons(ETH_P_MRP),
		.sll_ifindex = if_nametoindex("veth0")
	};

	fd = socket(AF_PACKET, SOCK_RAW, htons(ETH_P_MRP));
	bind(fd, (struct sockaddr *)&sa_ll, sizeof(sa_ll));

So far everything works fine and I can receive the packets I send.

If now I add veth0 to a bridge (e.g.

	ip link add br0 type bridge
	ip link set dev veth0 master br0

) and continue to send on veth1 and receive on veth0 I don't receive
the packets any more. The other direction (veth0 sending, veth1
receiving) still works fine. Each of the following changes allow to
receive again:

 a) take veth0 out of the bridge
 b) bind(2) the receiving socket to br0 instead of veth0
 c) use .sll_protocol = htons(ETH_P_ALL) for bind(2)

In the end only c) could be sensible (because I need to know the port
the packet entered the stack and that might well be bridged), but I
wonder why .sll_protocol = htons(ETH_P_MRP) seems to do the right thing
for an unbridged device but not for a bridged one.

Is this a bug or a feature I don't understand?

If someone wants to reproduce this locally, I can simplify my program
and provide it here. I tested this on a Debian 4.15 kernel (x86), but
also with the same symptoms on an arm64 with 4.16 and a dsa switch.

Best regards
Uwe

-- 
Pengutronix e.K.                           | Uwe Kleine-König            |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |

Re: BUG?: receiving on a packet socket with .sll_protocoll and bridging

[Willem de Bruijn]

On Sat, May 5, 2018 at 10:57 AM, Uwe Kleine-König
<u.kleine-koenig@pengutronix.de> wrote:
> Hello,
>
> my eventual goal is to implement MRP and for that I started to program a
> bit and stumbled over a problem I don't understand.
>
> For testing purposes I created a veth device pair (veth0 veth1), open a
> socket for each of the devices and send packets around between them. In
> tcpdump a typical package looks as follows:
>
> 10:36:34.755208 ae:a9:da:50:48:db (oui Unknown) > 01:15:e4:00:00:01 (oui Unknown), ethertype Unknown (0x88e3), length 58:
>         0x0000:  0001 0212 8000 aea9 da50 48db 0000 0000  .........PH.....
>         0x0010:  0000 0589 40f2 6574 6800 0000 0000 0000  ....@.eth.......
>         0x0020:  0000 0100 0a80 3d38 4c5e 0000            ......=8L^..
>
> The socket to receive these packages is opened using:
>
>         #define ETH_P_MRP 0x88e3
>
>         struct sockaddr_ll sa_ll = {
>                 .sll_family = AF_PACKET,
>                 .sll_protocol = htons(ETH_P_MRP),
>                 .sll_ifindex = if_nametoindex("veth0")
>         };
>
>         fd = socket(AF_PACKET, SOCK_RAW, htons(ETH_P_MRP));
>         bind(fd, (struct sockaddr *)&sa_ll, sizeof(sa_ll));
>
> So far everything works fine and I can receive the packets I send.
>
> If now I add veth0 to a bridge (e.g.
>
>         ip link add br0 type bridge
>         ip link set dev veth0 master br0
>
> ) and continue to send on veth1 and receive on veth0 I don't receive
> the packets any more. The other direction (veth0 sending, veth1
> receiving) still works fine.
>
> Each of the following changes allow to
> receive again:
>
>  a) take veth0 out of the bridge
>  b) bind(2) the receiving socket to br0 instead of veth0
>  c) use .sll_protocol = htons(ETH_P_ALL) for bind(2)
>
> In the end only c) could be sensible (because I need to know the port
> the packet entered the stack and that might well be bridged), but I
> wonder why .sll_protocol = htons(ETH_P_MRP) seems to do the right thing
> for an unbridged device but not for a bridged one.
>
> Is this a bug or a feature I don't understand?

Packets are redirected to the bridge device in __netif_receive_skb_core
at the rx_handler hook.

This happens after packets are passed to packet types attached to
list ptype_all, which includes packet sockets with protocol ETH_P_ALL.
But before packets are passed to protocol specific packet types (and
sockets) attached to ptype_base[].

Re: BUG?: receiving on a packet socket with .sll_protocoll and bridging

[Uwe Kleine-König]

Hello Willem,

On Sun, May 06, 2018 at 06:58:34PM +0200, Willem de Bruijn wrote:
> On Sat, May 5, 2018 at 10:57 AM, Uwe Kleine-König
> <u.kleine-koenig@pengutronix.de> wrote:
> > For testing purposes I created a veth device pair (veth0 veth1), open a
> > socket for each of the devices and send packets around between them. In
> > tcpdump a typical package looks as follows:
> >
> > 10:36:34.755208 ae:a9:da:50:48:db (oui Unknown) > 01:15:e4:00:00:01 (oui Unknown), ethertype Unknown (0x88e3), length 58:
> >         0x0000:  0001 0212 8000 aea9 da50 48db 0000 0000  .........PH.....
> >         0x0010:  0000 0589 40f2 6574 6800 0000 0000 0000  ....@.eth.......
> >         0x0020:  0000 0100 0a80 3d38 4c5e 0000            ......=8L^..
> >
> > The socket to receive these packages is opened using:
> >
> >         #define ETH_P_MRP 0x88e3
> >
> >         struct sockaddr_ll sa_ll = {
> >                 .sll_family = AF_PACKET,
> >                 .sll_protocol = htons(ETH_P_MRP),
> >                 .sll_ifindex = if_nametoindex("veth0")
> >         };
> >
> >         fd = socket(AF_PACKET, SOCK_RAW, htons(ETH_P_MRP));
> >         bind(fd, (struct sockaddr *)&sa_ll, sizeof(sa_ll));
> >
> > So far everything works fine and I can receive the packets I send.
> >
> > If now I add veth0 to a bridge (e.g.
> >
> >         ip link add br0 type bridge
> >         ip link set dev veth0 master br0
> >
> > ) and continue to send on veth1 and receive on veth0 I don't receive
> > the packets any more. The other direction (veth0 sending, veth1
> > receiving) still works fine.
> >
> > Each of the following changes allow to
> > receive again:
> >
> >  a) take veth0 out of the bridge
> >  b) bind(2) the receiving socket to br0 instead of veth0
> >  c) use .sll_protocol = htons(ETH_P_ALL) for bind(2)
> >
> > In the end only c) could be sensible (because I need to know the port
> > the packet entered the stack and that might well be bridged), but I
> > wonder why .sll_protocol = htons(ETH_P_MRP) seems to do the right thing
> > for an unbridged device but not for a bridged one.
> >
> > Is this a bug or a feature I don't understand?
> 
> Packets are redirected to the bridge device in __netif_receive_skb_core
> at the rx_handler hook.

OK, thanks for finding that place. It would have taken quite some of my
time to find it.

> This happens after packets are passed to packet types attached to
> list ptype_all, which includes packet sockets with protocol ETH_P_ALL.
> But before packets are passed to protocol specific packet types (and
> sockets) attached to ptype_base[].

Still I wonder if there is something to fix in the kernel or if this
inconsistency is intended (or at least accepted).

Best regards
Uwe

-- 
Pengutronix e.K.                           | Uwe Kleine-König            |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |

Re: BUG?: receiving on a packet socket with .sll_protocoll and bridging

[Willem de Bruijn]

>> > If now I add veth0 to a bridge (e.g.
>> >
>> >         ip link add br0 type bridge
>> >         ip link set dev veth0 master br0
>> >
>> > ) and continue to send on veth1 and receive on veth0 I don't receive
>> > the packets any more. The other direction (veth0 sending, veth1
>> > receiving) still works fine.
>> >
>> > Each of the following changes allow to
>> > receive again:
>> >
>> >  a) take veth0 out of the bridge
>> >  b) bind(2) the receiving socket to br0 instead of veth0
>> >  c) use .sll_protocol = htons(ETH_P_ALL) for bind(2)
>> >
>> > In the end only c) could be sensible (because I need to know the port
>> > the packet entered the stack and that might well be bridged), but I
>> > wonder why .sll_protocol = htons(ETH_P_MRP) seems to do the right thing
>> > for an unbridged device but not for a bridged one.
>> >
>> > Is this a bug or a feature I don't understand?
>>
>> Packets are redirected to the bridge device in __netif_receive_skb_core
>> at the rx_handler hook.
>
> OK, thanks for finding that place. It would have taken quite some of my
> time to find it.
>
>> This happens after packets are passed to packet types attached to
>> list ptype_all, which includes packet sockets with protocol ETH_P_ALL.
>> But before packets are passed to protocol specific packet types (and
>> sockets) attached to ptype_base[].
>
> Still I wonder if there is something to fix in the kernel or if this
> inconsistency is intended (or at least accepted).

It is established behavior.