Bridges

Bridges

[Jonathan Tripathy]

Hi Everyone,

When using a single Linux host with lots of bridges, would there ever be 
a time, even for a few seconds, where traffic would "jump" bridges?

I know a previous poster mentioned that when adding a host to a bridge, 
for a few seconds all packets get sent everywhere, however does this 
only apply to the bridge that the new host was added to, or all bridges 
in the system?

Reason I ask is that I am considering have one bridge for public traffic 
and one bridge for private, and don't want private traffic to be seen by 
hosts connected to the public bridge.

Thanks

Re: Bridges

[Jan Engelhardt]

On Wednesday 2010-08-18 00:44, Jonathan Tripathy wrote:
>
> When using a single Linux host with lots of bridges, would there ever be a
> time, even for a few seconds, where traffic would "jump" bridges?
>
> I know a previous poster mentioned that when adding a host to a bridge, for a
> few seconds all packets get sent everywhere

That seems unlikely. The bridge keeps a table of (mac, bridgeport) 
tuples (cf. `brctl showmacs br0`), and a "broadcast" should only occur 
when there is no matching entry or when the destination L2 address is 
broadcast. (Ignoring multicast for now.)

> Reason I ask is that I am considering have one bridge for public traffic and
> one bridge for private, and don't want private traffic to be seen by hosts
> connected to the public bridge.

Bridges link Ethernet (sub)segments together to form a (large) Ethernet 
segment. In other words, if you keep the trees separate, they will be 
separate.

Re: Bridges

[Stephen Hemminger]

On Wed, 18 Aug 2010 00:56:49 +0200 (CEST)
Jan Engelhardt <jengelh@medozas.de> wrote:

> On Wednesday 2010-08-18 00:44, Jonathan Tripathy wrote:
> >
> > When using a single Linux host with lots of bridges, would there ever be a
> > time, even for a few seconds, where traffic would "jump" bridges?
> >
> > I know a previous poster mentioned that when adding a host to a bridge, for a
> > few seconds all packets get sent everywhere
> 
> That seems unlikely. The bridge keeps a table of (mac, bridgeport) 
> tuples (cf. `brctl showmacs br0`), and a "broadcast" should only occur 
> when there is no matching entry or when the destination L2 address is 
> broadcast. (Ignoring multicast for now.)
> 
> > Reason I ask is that I am considering have one bridge for public traffic and
> > one bridge for private, and don't want private traffic to be seen by hosts
> > connected to the public bridge.
> 
> Bridges link Ethernet (sub)segments together to form a (large) Ethernet 
> segment. In other words, if you keep the trees separate, they will be 
> separate.

The forwarding database has aging so if a host does not send or receive
traffic for that interval (default is 5 minutes), then the entry is forgotten.
This is part of the 802.1d standard and is used to allow host to be moved
from one segment to another.

Re: Bridges

[Thomas Jacob]

On Tue, 2010-08-17 at 23:44 +0100, Jonathan Tripathy wrote:
> Hi Everyone,

> I know a previous poster mentioned that when adding a host to a bridge, 
> for a few seconds all packets get sent everywhere, however does this 
> only apply to the bridge that the new host was added to, or all bridges 
> in the system?

As Stephen already said, this is the standard behaviour of Ethernet
bridges (=switches), that bridges are somehow a security device
for separating Ethernet nodes is an urban myth (unless you employ
other techniques like 801.2q VLANs)

> Reason I ask is that I am considering have one bridge for public traffic 
> and one bridge for private, and don't want private traffic to be seen by 
> hosts connected to the public bridge.

But if you configure two bridge devices (br0 & br1) on your Linux box,
what you get is two separate Ethernets (broadcast domains really),
that cannot directly talk to each other over the Ethernet layer, unless
resorting to a router somewhere (assuming your Ethernets are not
connected to each other elsewhere).

In essence, such a setup should behave as if you had two different
Ethernet switches somewhere and the private machines are only connected
to the private switch and the public machines are only connected
to the public switch.

Whether this setup makes any sense in your case is another matter
though...

Re: Bridges

[Jonathan Tripathy]

> On Tue, 2010-08-17 at 23:44 +0100, Jonathan Tripathy wrote:
>    
>> Hi Everyone,
>>      
>    
>> I know a previous poster mentioned that when adding a host to a bridge,
>> for a few seconds all packets get sent everywhere, however does this
>> only apply to the bridge that the new host was added to, or all bridges
>> in the system?
>>      
> As Stephen already said, this is the standard behaviour of Ethernet
> bridges (=switches), that bridges are somehow a security device
> for separating Ethernet nodes is an urban myth (unless you employ
> other techniques like 801.2q VLANs)
>    
Yes, I understand this and this is acceptable, as all switches do this. 
My main concern is that do the frames stay within the respective bridge 
or not.
>    
>> Reason I ask is that I am considering have one bridge for public traffic
>> and one bridge for private, and don't want private traffic to be seen by
>> hosts connected to the public bridge.
>>      
> But if you configure two bridge devices (br0&  br1) on your Linux box,
> what you get is two separate Ethernets (broadcast domains really),
> that cannot directly talk to each other over the Ethernet layer, unless
> resorting to a router somewhere (assuming your Ethernets are not
> connected to each other elsewhere).
>
> In essence, such a setup should behave as if you had two different
> Ethernet switches somewhere and the private machines are only connected
> to the private switch and the public machines are only connected
> to the public switch.
>    
This is exactly what I want :)
> Whether this setup makes any sense in your case is another matter
> though...
>
>
>    
Makes perfect sense

Re: Bridges

[Grant Taylor]

On 08/17/10 17:44, Jonathan Tripathy wrote:
> When using a single Linux host with lots of bridges, would there ever be 
> a time, even for a few seconds, where traffic would "jump" bridges?

No.  Such should not be possible.

> I know a previous poster mentioned that when adding a host to a bridge, 
> for a few seconds all packets get sent everywhere, however does this 
> only apply to the bridge that the new host was added to, or all bridges 
> in the system?

I believe what the previous poster was alluding to was how a switch / 
bridge goes in to dumb hub mode and forwards frames to unknown 
destinations out all ports until it learns where the destination is. 
This is standard operating procedure for switches / bridges, and is to 
be expected.

I am not aware of any thing specific to bridges that would allow this to 
happen (short of an as of yet unknown bug in the kernel).  The closest 
thing that I can think of that might make it seem like this is happening 
is if someone is sending you some sort of VLAN hopping attack.  And as I 
(mis)understand that, that traffic would have to be with in a layer 2 
network, so they attacker is likely to be close, not across the internet.

> Reason I ask is that I am considering have one bridge for public traffic 
> and one bridge for private, and don't want private traffic to be seen by 
> hosts connected to the public bridge.

I think you should be safe (enough) with this.  In fact, you are 
starting to get in to some more theoretical discussions about what is 
and is not safe to do as far as having both public and private VLAN (or 
bridge) traffic on the same wire (system).  There are a number of people 
(my self included) that think you are safe enough for most 
non-uber-secure situations to go ahead and do what you are wanting to do.



Grant. . . .

Re: Bridges

[Jonathan Tripathy]

On 18/08/10 22:51, Grant Taylor wrote:
> On 08/17/10 17:44, Jonathan Tripathy wrote:
>> When using a single Linux host with lots of bridges, would there ever 
>> be a time, even for a few seconds, where traffic would "jump" bridges?
>
> No.  Such should not be possible.
>
>> I know a previous poster mentioned that when adding a host to a 
>> bridge, for a few seconds all packets get sent everywhere, however 
>> does this only apply to the bridge that the new host was added to, or 
>> all bridges in the system?
>
> I believe what the previous poster was alluding to was how a switch / 
> bridge goes in to dumb hub mode and forwards frames to unknown 
> destinations out all ports until it learns where the destination is. 
> This is standard operating procedure for switches / bridges, and is to 
> be expected.
>
> I am not aware of any thing specific to bridges that would allow this 
> to happen (short of an as of yet unknown bug in the kernel).  The 
> closest thing that I can think of that might make it seem like this is 
> happening is if someone is sending you some sort of VLAN hopping 
> attack.  And as I (mis)understand that, that traffic would have to be 
> with in a layer 2 network, so they attacker is likely to be close, not 
> across the internet.
>
>> Reason I ask is that I am considering have one bridge for public 
>> traffic and one bridge for private, and don't want private traffic to 
>> be seen by hosts connected to the public bridge.
>
> I think you should be safe (enough) with this.  In fact, you are 
> starting to get in to some more theoretical discussions about what is 
> and is not safe to do as far as having both public and private VLAN 
> (or bridge) traffic on the same wire (system).  There are a number of 
> people (my self included) that think you are safe enough for most 
> non-uber-secure situations to go ahead and do what you are wanting to do.
>
>
>
> Grant. . . .
> -- 
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Thanks Grant. Just to put it into a little context, it's a Xen host. One 
bridge for the "public" VMs for some customers. The other bridge is more 
my private stuff (including the Dom0 xen host itself).

Incidentally, would using ebtables rules prevent the bridge from going 
into "dumb hub" mode? Like let's say I said that "all traffic leaving 
this interface must have this destination MAC address".

Cheers

Re: Bridges

[Grant Taylor]

On 08/18/10 16:57, Jonathan Tripathy wrote:
> Thanks Grant. Just to put it into a little context, it's a Xen host. One 
> bridge for the "public" VMs for some customers. The other bridge is more 
> my private stuff (including the Dom0 xen host itself).

You are welcome.

I wondered if something like that might be going on.

> Incidentally, would using ebtables rules prevent the bridge from going 
> into "dumb hub" mode? Like let's say I said that "all traffic leaving 
> this interface must have this destination MAC address".

No, EBTables will not prevent a bridge from having to go through the 
learning process.  EBTables might filter out the frame so that it 
doesn't (fully) traverse the bridge, but it will not educate the MAC table.

Remember that the bridge will behave just like any switch would that 
multiple servers are plugged in to.

> Cheers

Likewise.



Grant. . . .

Re: Bridges

[Jonathan Tripathy]

>> Incidentally, would using ebtables rules prevent the bridge from 
>> going into "dumb hub" mode? Like let's say I said that "all traffic 
>> leaving this interface must have this destination MAC address".
>
> No, EBTables will not prevent a bridge from having to go through the 
> learning process.  EBTables might filter out the frame so that it 
> doesn't (fully) traverse the bridge, but it will not educate the MAC 
> table.
>
> Remember that the bridge will behave just like any switch would that 
> multiple servers are plugged in to.
>
Sorry, I used a bad choice of words - Would ebtables stop the frame 
reaching the remote host (VM in my case) is what I meant to say :)

Re: Bridges

[Jan Engelhardt]

On Thursday 2010-08-19 00:15, Jonathan Tripathy wrote:
>
>>> Incidentally, would using ebtables rules prevent the bridge from going into
>>> "dumb hub" mode? Like let's say I said that "all traffic leaving this
>>> interface must have this destination MAC address".
>>
>> No, EBTables will not prevent a bridge from having to go through the learning
>> process.  EBTables might filter out the frame so that it doesn't (fully)
>> traverse the bridge, but it will not educate the MAC table.
>>
>> Remember that the bridge will behave just like any switch would that multiple
>> servers are plugged in to.
>>
> Sorry, I used a bad choice of words - Would ebtables stop the frame reaching
> the remote host (VM in my case) is what I meant to say :)

No. The two bridges are not connected to another in the first place,
so the only way for a packet to come in on br0 and go out on br1 is
routing, for which iptables is needed to filter.

Re: Bridges

[Jonathan Tripathy]

> >>Sorry, I used a bad choice of words - Would ebtables stop the frame reaching
> >>  the remote host (VM in my case) is what I meant to say:)
>
> >No. The two bridges are not connected to another in the first place,
> >so the only way for a packet to come in on br0 and go out on br1 is
> >routing, for which iptables is needed to filter.
>    

But even without iptables, traffic coudn't cross without a router in the 
middle, right?

BTW, my post above wasn't really related to having 2 bridges, but more 
of the "dumb hub" situation.

Cheers

Re: Bridges

[Pascal Hambourg]

Jonathan Tripathy a écrit :
>>>> Sorry, I used a bad choice of words - Would ebtables stop the frame reaching
>>>>  the remote host (VM in my case) is what I meant to say:)
>>> No. The two bridges are not connected to another in the first place,
>>> so the only way for a packet to come in on br0 and go out on br1 is
>>> routing, for which iptables is needed to filter.
> 
> But even without iptables, traffic coudn't cross without a router in the 
> middle, right?

Remember that Linux itself can act as a router.

> BTW, my post above wasn't really related to having 2 bridges, but more 
> of the "dumb hub" situation.

I think Jan misunderstood your question which was

> Incidentally, would using ebtables rules prevent the bridge from 
> going into "dumb hub" mode? Like let's say I said that "all traffic 
> leaving this interface must have this destination MAC address".

IIUC your question, yes, ebtables could do that. But beware when doing
this, you could easily break very useful things such as ARP resolution
(which uses broadcast) or IPv6 neighbour discovery (which uses multicast).

Re: Bridges

[Jonathan Tripathy]

> you could easily break very useful things such as ARP resolution
> (which uses broadcast) or IPv6 neighbour discovery (which uses multicast).
>    
And I did at first :)

Made an exception for ARP though

What is the best way to stop a linux host from becoming a router? I 
don't think disabling ip forwarding is possible in my case (As I'm doing 
a lot of forwarding in my bridges)

Thanks

Re: Bridges

[Pascal Hambourg]

Jonathan Tripathy a écrit :
> 
> What is the best way to stop a linux host from becoming a router?

Disable global IP forwarding. If not possible because you need to enable
forwarding on some interfaces, disable interface-specific forwarding.
Otherwise, use iptables rules in the FORWARD chain (beware undesirable
interactions with bridged IP traffic through bridge-nf if enabled vith
sysctl net.bridge.bridge-nf-call-iptables=1).

> I don't think disabling ip forwarding is possible in my case (As I'm 
> doing a lot of forwarding in my bridges)

Forwarding of ethernet frames within a bridge is completely independent
of IP forwarding.

Re: Bridges

[Jonathan Tripathy]

> use iptables rules in the FORWARD chain (beware undesirable 
> interactions with bridged IP traffic through bridge-nf if enabled vith 
> sysctl net.bridge.bridge-nf-call-iptables=1).
I think I'm doing that anyway. For every VM that is started, a strict 
set of iptables rules are set up which only allow traffic in/out of 
specific interfaces (using physdev). The rules also prevent ip spoofing.

Re: Bridges

[Jonathan Tripathy]

> Disable global IP forwarding. 
I just ran "cat /proc/sys/net/ipv4/ip_forward" and it returned 0. Does 
this mean that I have "global IP forwarding" off? Thanks

Re: Bridges

[Pascal Hambourg]

Jonathan Tripathy a écrit :
>> Disable global IP forwarding. 
> I just ran "cat /proc/sys/net/ipv4/ip_forward" and it returned 0. Does 
> this mean that I have "global IP forwarding" off?

Yes. So unless you enable /proc/sys/net/ipv4/conf/<interface>/forwarding
for a bridge, packets won't jump from that bridge to another interface.

Re: Bridges

[Jonathan Tripathy]

>
>>> Disable global IP forwarding.
>>>        
>> I just ran "cat /proc/sys/net/ipv4/ip_forward" and it returned 0. Does
>> this mean that I have "global IP forwarding" off?
>>      
> Yes. So unless you enable /proc/sys/net/ipv4/conf/<interface>/forwarding
> for a bridge, packets won't jump from that bridge to another interface.
>    
Ok so it's all 0. However, in the /proc/sys/net/ipv4/conf folder, I 
don't see my "public" bridge, only the private one. Incidently, the 
private one is the only bridge in my system that has an IP address 
asigned to it...

Re: Bridges

[Pascal Hambourg]

Jonathan Tripathy a écrit :
>>    
> However, in the /proc/sys/net/ipv4/conf folder, I 
> don't see my "public" bridge, only the private one. Incidently, the 
> private one is the only bridge in my system that has an IP address 
> asigned to it...

Ok. May I ask what is your kernel version ? Older than 2.6.21 ?

Re: Bridges

[Jonathan Tripathy]

>
>>>
>>>        
>> However, in the /proc/sys/net/ipv4/conf folder, I
>> don't see my "public" bridge, only the private one. Incidently, the
>> private one is the only bridge in my system that has an IP address
>> asigned to it...
>>      
> Ok. May I ask what is your kernel version ? Older than 2.6.21 ?
>    
Linux xx.xx.xx.xx 2.6.18-194.8.1.el5xen #1 SMP Thu Jul 1 19:41:05 EDT 
2010 x86_64 x86_64 x86_64 GNU/Linux

It's CentOS 5.5

Thanks

Re: Bridges

[Jan Engelhardt]

On Thursday 2010-08-19 21:37, Jonathan Tripathy wrote:
>>>>       
>>> However, in the /proc/sys/net/ipv4/conf folder, I
>>> don't see my "public" bridge, only the private one. Incidently, the
>>> private one is the only bridge in my system that has an IP address
>>> asigned to it...
>>>     
>> Ok. May I ask what is your kernel version ? Older than 2.6.21 ?
>>   
> Linux xx.xx.xx.xx 2.6.18-194.8.1.el5xen #1 SMP Thu Jul 1 19:41:05 EDT 2010
> x86_64 x86_64 x86_64 GNU/Linux
>
> It's CentOS 5.5

That's eligible for submission to failblog.org ;-)

Re: Bridges

[Jonathan Tripathy]

>
> On Thursday 2010-08-19 21:37, Jonathan Tripathy wrote:
>    
>>>>>
>>>>>            
>>>> However, in the /proc/sys/net/ipv4/conf folder, I
>>>> don't see my "public" bridge, only the private one. Incidently, the
>>>> private one is the only bridge in my system that has an IP address
>>>> asigned to it...
>>>>
>>>>          
>>> Ok. May I ask what is your kernel version ? Older than 2.6.21 ?
>>>
>>>        
>> Linux xx.xx.xx.xx 2.6.18-194.8.1.el5xen #1 SMP Thu Jul 1 19:41:05 EDT 2010
>> x86_64 x86_64 x86_64 GNU/Linux
>>
>> It's CentOS 5.5
>>      
> That's eligible for submission to failblog.org ;-)
>    

Why? What did I do wrong? :)

Re: Bridges

[Pascal Hambourg]

Jonathan Tripathy a écrit :
>>>        
>>> However, in the /proc/sys/net/ipv4/conf folder, I
>>> don't see my "public" bridge, only the private one. Incidently, the
>>> private one is the only bridge in my system that has an IP address
>>> asigned to it...
>>>      
>> Ok. May I ask what is your kernel version ? Older than 2.6.21 ?
>>    
> Linux xx.xx.xx.xx 2.6.18-194.8.1.el5xen #1 SMP Thu Jul 1 19:41:05 EDT 
> 2010 x86_64 x86_64 x86_64 GNU/Linux

As I expected. A change occured in 2.6.21, recent kernels automatically
link interfaces to the IP stack.

Re: Bridges

[Jonathan Tripathy]

>
>>>>
>>>> However, in the /proc/sys/net/ipv4/conf folder, I
>>>> don't see my "public" bridge, only the private one. Incidently, the
>>>> private one is the only bridge in my system that has an IP address
>>>> asigned to it...
>>>>
>>>>          
>>> Ok. May I ask what is your kernel version ? Older than 2.6.21 ?
>>>
>>>        
>> Linux xx.xx.xx.xx 2.6.18-194.8.1.el5xen #1 SMP Thu Jul 1 19:41:05 EDT
>> 2010 x86_64 x86_64 x86_64 GNU/Linux
>>      
> As I expected. A change occured in 2.6.21, recent kernels automatically
> link interfaces to the IP stack.
>    
So in my case, am I at a disadvantage, or maybe it doesn't make a 
difference?

Re: Bridges

[Pascal Hambourg]

Jonathan Tripathy a écrit :
>>>>> However, in the /proc/sys/net/ipv4/conf folder, I
>>>>> don't see my "public" bridge, only the private one. Incidently, the
>>>>> private one is the only bridge in my system that has an IP address
>>>>> asigned to it...
>>>>>          
>>>> Ok. May I ask what is your kernel version ? Older than 2.6.21 ?
>>>>        
>>> Linux xx.xx.xx.xx 2.6.18-194.8.1.el5xen #1 SMP Thu Jul 1 19:41:05 EDT
>>> 2010 x86_64 x86_64 x86_64 GNU/Linux
>>>      
>> As I expected. A change occured in 2.6.21, recent kernels automatically
>> link interfaces to the IP stack.
>>    
> So in my case, am I at a disadvantage

On the contrary, the old behaviour is an advantage if you don't want an
interface to participate in IP operation.

Re: Bridges

[Jonathan Tripathy]

>
>>>>>> However, in the /proc/sys/net/ipv4/conf folder, I
>>>>>> don't see my "public" bridge, only the private one. Incidently, the
>>>>>> private one is the only bridge in my system that has an IP address
>>>>>> asigned to it...
>>>>>>
>>>>>>              
>>>>> Ok. May I ask what is your kernel version ? Older than 2.6.21 ?
>>>>>
>>>>>            
>>>> Linux xx.xx.xx.xx 2.6.18-194.8.1.el5xen #1 SMP Thu Jul 1 19:41:05 EDT
>>>> 2010 x86_64 x86_64 x86_64 GNU/Linux
>>>>
>>>>          
>>> As I expected. A change occured in 2.6.21, recent kernels automatically
>>> link interfaces to the IP stack.
>>>
>>>        
>> So in my case, am I at a disadvantage
>>      
> On the contrary, the old behaviour is an advantage if you don't want an
> interface to participate in IP operation.
>    

Excellent! If I were to upgrade at a later date, is it just a matter of 
making sure that everything returns 0 in the above files?

Re: Bridges

[Pascal Hambourg]

Jonathan Tripathy a écrit :
>>>>>>> However, in the /proc/sys/net/ipv4/conf folder, I
>>>>>>> don't see my "public" bridge, only the private one. Incidently, the
>>>>>>> private one is the only bridge in my system that has an IP address
>>>>>>> asigned to it...
>>>>>>>              
>>>>>> Ok. May I ask what is your kernel version ? Older than 2.6.21 ?
>>>>>>            
>>>>> Linux xx.xx.xx.xx 2.6.18-194.8.1.el5xen #1 SMP Thu Jul 1 19:41:05 EDT
>>>>> 2010 x86_64 x86_64 x86_64 GNU/Linux
>>>>>          
>>>> As I expected. A change occured in 2.6.21, recent kernels automatically
>>>> link interfaces to the IP stack.
>>>>        
>>> So in my case, am I at a disadvantage
>>>      
>> On the contrary, the old behaviour is an advantage if you don't want an
>> interface to participate in IP operation.
> 
> Excellent! If I were to upgrade at a later date, is it just a matter of 
> making sure that everything returns 0 in the above files?

Setting net.ipv4.conf.<interface>.forwarding=0 will disable forwarding
of IP packets received on <interface>. But I'm afraid it won't disable
all IP operation on it (ARP resolution, accept IP packets for local
delivery...)

Re: Bridges

[Jonathan Tripathy]

On 20/08/10 09:53, Pascal Hambourg wrote:
> Jonathan Tripathy a écrit :
>    
>>>>>>>> However, in the /proc/sys/net/ipv4/conf folder, I
>>>>>>>> don't see my "public" bridge, only the private one. Incidently, the
>>>>>>>> private one is the only bridge in my system that has an IP address
>>>>>>>> asigned to it...
>>>>>>>>
>>>>>>>>                  
>>>>>>> Ok. May I ask what is your kernel version ? Older than 2.6.21 ?
>>>>>>>
>>>>>>>                
>>>>>> Linux xx.xx.xx.xx 2.6.18-194.8.1.el5xen #1 SMP Thu Jul 1 19:41:05 EDT
>>>>>> 2010 x86_64 x86_64 x86_64 GNU/Linux
>>>>>>
>>>>>>              
>>>>> As I expected. A change occured in 2.6.21, recent kernels automatically
>>>>> link interfaces to the IP stack.
>>>>>
>>>>>            
>>>> So in my case, am I at a disadvantage
>>>>
>>>>          
>>> On the contrary, the old behaviour is an advantage if you don't want an
>>> interface to participate in IP operation.
>>>        
>> Excellent! If I were to upgrade at a later date, is it just a matter of
>> making sure that everything returns 0 in the above files?
>>      
> Setting net.ipv4.conf.<interface>.forwarding=0 will disable forwarding
> of IP packets received on<interface>. But I'm afraid it won't disable
> all IP operation on it (ARP resolution, accept IP packets for local
> delivery...)
>    

Just out of interest, if routing was enabled on the Dom0, how would the 
guests be able to use it, if the bridge that they were connected to 
didn't have an IP address? Assuming that all IPs for the other 
interfaces on the Dom0 were on a different subnet..

Thanks

Re: Bridges

[Jan Engelhardt]

On Saturday 2010-08-21 23:46, Jonathan Tripathy wrote:
>>>     
>> Setting net.ipv4.conf.<interface>.forwarding=0 will disable forwarding
>> of IP packets received on<interface>. But I'm afraid it won't disable
>> all IP operation on it (ARP resolution, accept IP packets for local
>> delivery...)
>
>Just out of interest, if routing was enabled on the Dom0, how would
>the guests be able to use it,

Well, by means of `ebtables -t broute -j DROP` which causes packets
to be routed instead of bridged.

>if the bridge that they were connected
>to didn't have an IP address?

Because routing won't happen with brX, but with vifX.

>Assuming that all IPs for the other
>interfaces on the Dom0 were on a different subnet..

Re: Bridges

[Jonathan Tripathy]

On 22/08/10 00:25, Jan Engelhardt wrote:
> On Saturday 2010-08-21 23:46, Jonathan Tripathy wrote:
>    
>>>>
>>>>          
>>> Setting net.ipv4.conf.<interface>.forwarding=0 will disable forwarding
>>> of IP packets received on<interface>. But I'm afraid it won't disable
>>> all IP operation on it (ARP resolution, accept IP packets for local
>>> delivery...)
>>>        
>> Just out of interest, if routing was enabled on the Dom0, how would
>> the guests be able to use it,
>>      
> Well, by means of `ebtables -t broute -j DROP` which causes packets
> to be routed instead of bridged.
>
>    
>> if the bridge that they were connected
>> to didn't have an IP address?
>>      
> Because routing won't happen with brX, but with vifX.
>
>    
>> Assuming that all IPs for the other
>> interfaces on the Dom0 were on a different subnet..
>>      

Ah right, but something would have to "make it" do that? (i.e. your 
ebtables broute example above)

Thanks

Re: Bridges

[Jan Engelhardt]

On Sunday 2010-08-22 11:05, Jonathan Tripathy wrote:
>>>>>         
>>>> Setting net.ipv4.conf.<interface>.forwarding=0 will disable forwarding
>>>> of IP packets received on<interface>. But I'm afraid it won't disable
>>>> all IP operation on it (ARP resolution, accept IP packets for local
>>>> delivery...)
>>>>       
>>> Just out of interest, if routing was enabled on the Dom0, how would
>>> the guests be able to use it,
>>>     
>> Well, by means of `ebtables -t broute -j DROP` which causes packets
>> to be routed instead of bridged.
>>
>>> Assuming that all IPs for the other
>>> interfaces on the Dom0 were on a different subnet..
>
> Ah right, but something would have to "make it" do that? (i.e. your ebtables
> broute example above)

As I said.

And as previously mentioned, routing does not usually care about addresses,
but routes (`ip r`).

Re: Bridges

[Jonathan Tripathy]

On 22/08/10 10:09, Jan Engelhardt wrote:
> On Sunday 2010-08-22 11:05, Jonathan Tripathy wrote:
>    
>>>>>>
>>>>>>              
>>>>> Setting net.ipv4.conf.<interface>.forwarding=0 will disable forwarding
>>>>> of IP packets received on<interface>. But I'm afraid it won't disable
>>>>> all IP operation on it (ARP resolution, accept IP packets for local
>>>>> delivery...)
>>>>>
>>>>>            
>>>> Just out of interest, if routing was enabled on the Dom0, how would
>>>> the guests be able to use it,
>>>>
>>>>          
>>> Well, by means of `ebtables -t broute -j DROP` which causes packets
>>> to be routed instead of bridged.
>>>
>>>        
>>>> Assuming that all IPs for the other
>>>> interfaces on the Dom0 were on a different subnet..
>>>>          
>> Ah right, but something would have to "make it" do that? (i.e. your ebtables
>> broute example above)
>>      
> As I said.
>
> And as previously mentioned, routing does not usually care about addresses,
> but routes (`ip r`).
>    

I'm sorry I don't really follow. Are you saying that there is no way to 
disable IP forwarding in a Linux box?

Re: Bridges

[Pascal Hambourg]

Jonathan Tripathy a écrit :
> On 22/08/10 10:09, Jan Engelhardt wrote:
>>
>> And as previously mentioned, routing does not usually care about addresses,
>> but routes (`ip r`).
> 
> I'm sorry I don't really follow. Are you saying that there is no way to 
> disable IP forwarding in a Linux box?

No one said that. As we already explained, IP forwarding can be disabled
globally or per interface.

"IP routing" may have different meanings. It can be a synonym for "IP
forwarding", like in "IP routers do IP routing". But in the context of
the Linux IP stack, it rather means the handling of an IP packet
according to the routing table. In short :
- for an incoming packet, determine whether it is for local delivery or
to be forwarded ;
- for an outgoing or forwarded packet, determine the output interface
and gateway.

Re: Bridges

[Jonathan Tripathy]

> "IP routing" may have different meanings. It can be a synonym for "IP
> forwarding", like in "IP routers do IP routing". But in the context of
> the Linux IP stack, it rather means the handling of an IP packet
> according to the routing table. In short :
> - for an incoming packet, determine whether it is for local delivery or
> to be forwarded ;
> - for an outgoing or forwarded packet, determine the output interface
> and gateway.
>    

I definitely still want link to do that though, with my bridge traffic...

Re: Bridges

[Pascal Hambourg]

Jonathan Tripathy a écrit :
>> "IP routing" may have different meanings. It can be a synonym for "IP
>> forwarding", like in "IP routers do IP routing". But in the context of
>> the Linux IP stack, it rather means the handling of an IP packet
>> according to the routing table. In short :
>> - for an incoming packet, determine whether it is for local delivery or
>> to be forwarded ;
>> - for an outgoing or forwarded packet, determine the output interface
>> and gateway.
> 
> I definitely still want link to do that though, with my bridge traffic...

Do what ? I don't understand, sorry.

Re: Bridges

[Jonathan Tripathy]

On 23/08/10 09:22, Pascal Hambourg wrote:
> Jonathan Tripathy a écrit :
>    
>>> "IP routing" may have different meanings. It can be a synonym for "IP
>>> forwarding", like in "IP routers do IP routing". But in the context of
>>> the Linux IP stack, it rather means the handling of an IP packet
>>> according to the routing table. In short :
>>> - for an incoming packet, determine whether it is for local delivery or
>>> to be forwarded ;
>>> - for an outgoing or forwarded packet, determine the output interface
>>> and gateway.
>>>        
>> I definitely still want link to do that though, with my bridge traffic...
>>      
> Do what ? I don't understand, sorry.
>    

Sorry, I'm not wording my questions very well. Let me start from the 
top. Thanks everyone for the fantastic help by the way :)

Ok, so as I understand Linux, and please correct me if I'm wrong, when a 
packet comes into a physical interface on a Linux machine, regardless of 
the status of the net.ipv4.conf.<interface>.forwarding, Linux will 
always put the packet onto a "chain". This will either be the INPUT 
chain or the FORWARD chain. Is that correct?

Next, if the packet it destined for an IP of one of the local 
interfaces, it puts it onto the INPUT chain, correct?

However if the packet it destined for a non-local host, it puts it onto 
the FORWARD chain, correct?

Then, when the packet is on the forward chain, it depends on if the 
incoming interface is connected to a bridge or not. If it is connected 
to a bridge, the packet will transverse the FORWARD chain and go out the 
interface which must be part of the same bridge, correct?

If, however, the incoming interface is not connected to a bridge, Linux 
will fordward the packet out of another interface which is thinks is 
correct (as defined by the routing table), but it will only do this if 
net.ipv4.conf.<interface>.forwarding is set to 1, correct?

Before I go on with my next set of question, I'd just like someone to 
correct the above if it is wrong

Many Thanks

Re: Bridges

[Karel Rericha]

See this graph (created by Jan Engelhardt), it should explain what you
are asking for:

http://jengelh.medozas.de/images/nf-packet-flow.png

Karel

> Sorry, I'm not wording my questions very well. Let me start from the top.
> Thanks everyone for the fantastic help by the way :)
>
> Ok, so as I understand Linux, and please correct me if I'm wrong, when a
> packet comes into a physical interface on a Linux machine, regardless of the
> status of the net.ipv4.conf.<interface>.forwarding, Linux will always put
> the packet onto a "chain". This will either be the INPUT chain or the
> FORWARD chain. Is that correct?
>
> Next, if the packet it destined for an IP of one of the local interfaces, it
> puts it onto the INPUT chain, correct?
>
> However if the packet it destined for a non-local host, it puts it onto the
> FORWARD chain, correct?
>
> Then, when the packet is on the forward chain, it depends on if the incoming
> interface is connected to a bridge or not. If it is connected to a bridge,
> the packet will transverse the FORWARD chain and go out the interface which
> must be part of the same bridge, correct?
>
> If, however, the incoming interface is not connected to a bridge, Linux will
> fordward the packet out of another interface which is thinks is correct (as
> defined by the routing table), but it will only do this if
> net.ipv4.conf.<interface>.forwarding is set to 1, correct?
>
> Before I go on with my next set of question, I'd just like someone to
> correct the above if it is wrong
>
> Many Thanks

Re: Bridges

[Pascal Hambourg]

Jonathan Tripathy a écrit :
> 
> Ok, so as I understand Linux, and please correct me if I'm wrong, when a 
> packet comes into a physical interface on a Linux machine, regardless of 
> the status of the net.ipv4.conf.<interface>.forwarding, Linux will 
> always put the packet onto a "chain". This will either be the INPUT 
> chain or the FORWARD chain. Is that correct?

This is correct if you consider only the IP layer (IP stack, IP
packets). If you consider the link layer (bridging, VLAN, bonding,
tunnels...) things get more complicated.

> Next, if the packet it destined for an IP of one of the local 
> interfaces, it puts it onto the INPUT chain, correct?

Correct.

> However if the packet it destined for a non-local host, it puts it onto 
> the FORWARD chain, correct?

Only if net.ipv4.ip_forward=1 or net.ipv4.conf.<interface>.forwarding=1,
and the TTL is > 0 after being decremented. Otherwise the packet is
discarded before entering the FORWARD chain.

> Then, when the packet is on the forward chain, it depends on if the 
> incoming interface is connected to a bridge or not.

No, it doesn't. In the IP stack, the interface type does not not matter.
Only the routing table matters.

Jan's diagram pointed to by Karel Rericha explains paths in the IP layer
and the link layer. However bridge-netfilter (the capability to send
bridged packets through {ip,arp,ipv6}tables chains whereas they are not
processed by the IP stack) makes things a bit more complicated.

Re: Bridges

[Jonathan Tripathy]

On 24/08/10 15:44, Pascal Hambourg wrote:
> Jonathan Tripathy a écrit :
>    
>> Ok, so as I understand Linux, and please correct me if I'm wrong, when a
>> packet comes into a physical interface on a Linux machine, regardless of
>> the status of the net.ipv4.conf.<interface>.forwarding, Linux will
>> always put the packet onto a "chain". This will either be the INPUT
>> chain or the FORWARD chain. Is that correct?
>>      
> This is correct if you consider only the IP layer (IP stack, IP
> packets). If you consider the link layer (bridging, VLAN, bonding,
> tunnels...) things get more complicated.
>
>    
>> Next, if the packet it destined for an IP of one of the local
>> interfaces, it puts it onto the INPUT chain, correct?
>>      
> Correct.
>
>    
>> However if the packet it destined for a non-local host, it puts it onto
>> the FORWARD chain, correct?
>>      
> Only if net.ipv4.ip_forward=1 or net.ipv4.conf.<interface>.forwarding=1,
> and the TTL is>  0 after being decremented. Otherwise the packet is
> discarded before entering the FORWARD chain.
>
>    
>> Then, when the packet is on the forward chain, it depends on if the
>> incoming interface is connected to a bridge or not.
>>      
> No, it doesn't. In the IP stack, the interface type does not not matter.
> Only the routing table matters.
>
> Jan's diagram pointed to by Karel Rericha explains paths in the IP layer
> and the link layer. However bridge-netfilter (the capability to send
> bridged packets through {ip,arp,ipv6}tables chains whereas they are not
> processed by the IP stack) makes things a bit more complicated.
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>    

I think what I am getting confused with is how am I able to use the 
FORWARD chain in iptables with my bridge setup, even though forwarding 
is disabled?

Thanks

Re: Bridges

[Pascal Hambourg]

Jonathan Tripathy a écrit :
> On 24/08/10 15:44, Pascal Hambourg wrote:
>>
>> Jan's diagram pointed to by Karel Rericha explains paths in the IP layer
>> and the link layer. However bridge-netfilter (the capability to send
>> bridged packets through {ip,arp,ipv6}tables chains whereas they are not
>> processed by the IP stack) makes things a bit more complicated.
> 
> I think what I am getting confused with is how am I able to use the 
> FORWARD chain in iptables with my bridge setup, even though forwarding 
> is disabled?

Because of netfilter-bridge which allows to pass bridged packets through
iptables chains. See the "link layer" part of the diagram, where
bridging takes place. This behaviour can be controlled by sysctls in
/proc/sys/net/bridge/ (see Documentation/networking/ip-sysctl.txt).

Re: Bridges

[Jonathan Tripathy]

On 24/08/10 19:07, Pascal Hambourg wrote:
> Jonathan Tripathy a écrit :
>    
>> On 24/08/10 15:44, Pascal Hambourg wrote:
>>      
>>> Jan's diagram pointed to by Karel Rericha explains paths in the IP layer
>>> and the link layer. However bridge-netfilter (the capability to send
>>> bridged packets through {ip,arp,ipv6}tables chains whereas they are not
>>> processed by the IP stack) makes things a bit more complicated.
>>>        
>> I think what I am getting confused with is how am I able to use the
>> FORWARD chain in iptables with my bridge setup, even though forwarding
>> is disabled?
>>      
> Because of netfilter-bridge which allows to pass bridged packets through
> iptables chains. See the "link layer" part of the diagram, where
> bridging takes place. This behaviour can be controlled by sysctls in
> /proc/sys/net/bridge/ (see Documentation/networking/ip-sysctl.txt).
>    
Ah I see. Just some special magic in netfilter-bridge.

So bottom line: no non-bridged IP traffic can get into the FORWARD chain 
without IP forwarding being enabled? And also, bridged traffic can not 
cross bridges unless forwarding is enabled (hence Linux is being used as 
a router itself), or there is a router of some sort in the middle?

Cheers

Re: Bridges

[Pascal Hambourg]

Jonathan Tripathy a écrit :
> 
> So bottom line: no non-bridged IP traffic can get into the FORWARD chain 
> without IP forwarding being enabled?

Correct.

> And also, bridged traffic can not 
> cross bridges unless forwarding is enabled (hence Linux is being used as 
> a router itself), or there is a router of some sort in the middle?

What do you mean by "cross bridges" ?

Re: Bridges

[Jan Engelhardt]

On Friday 2010-08-20 00:04, Pascal Hambourg wrote:
>>>>>        
>>>> Linux xx.xx.xx.xx 2.6.18-194.8.1.el5xen #1 SMP Thu Jul 1 19:41:05 EDT
>>>> 2010 x86_64 x86_64 x86_64 GNU/Linux
>>>>      
>>> As I expected. A change occured in 2.6.21, recent kernels automatically
>>> link interfaces to the IP stack.
>>>    
>> So in my case, am I at a disadvantage
>
>On the contrary, the old behaviour is an advantage if you don't want an
>interface to participate in IP operation.

Well not really, there is always iptables and ebtables.

Re: Bridges

[Pascal Hambourg]

Jan Engelhardt a écrit :
> On Friday 2010-08-20 00:04, Pascal Hambourg wrote:
>>>>>>        
>>>>> Linux xx.xx.xx.xx 2.6.18-194.8.1.el5xen #1 SMP Thu Jul 1 19:41:05 EDT
>>>>> 2010 x86_64 x86_64 x86_64 GNU/Linux
>>>>>      
>>>> As I expected. A change occured in 2.6.21, recent kernels automatically
>>>> link interfaces to the IP stack.
>>>>    
>>> So in my case, am I at a disadvantage
>> On the contrary, the old behaviour is an advantage if you don't want an
>> interface to participate in IP operation.
> 
> Well not really, there is always iptables and ebtables.

Ebtables is available only on bridge interfaces. Here the interface is a
bridge, but what if you don't want IP on a normal interface ?
Also, iptables won't disable ARP so arptables is required too.

Isn't there a cleaner way to "detach" an interface from the IP stack ?

Re: Bridges

[Jan Engelhardt]

On Friday 2010-08-20 11:05, Pascal Hambourg wrote:
>>>>>    
>>>> So in my case, am I at a disadvantage
>>> On the contrary, the old behaviour is an advantage if you don't want an
>>> interface to participate in IP operation.
>> 
>> Well not really, there is always iptables and ebtables.
>
>Ebtables is available only on bridge interfaces.

Which makes sense.

>Here the interface is a
>bridge, but what if you don't want IP on a normal interface ?

You still use iptables to filter potential traffic that goes from br0 to 
br1 or vice versa.

Re: Bridges

[Pascal Hambourg]

Jan Engelhardt a écrit :
> On Friday 2010-08-20 11:05, Pascal Hambourg wrote:
>    
>> Ebtables is available only on bridge interfaces.
> 
> Which makes sense.

Well, in a way, but sometimes it could be useful to do layer-2 filtering
on normal ethernet interfaces too. Ebtables has INPUT and OUTPUT chains,
so it is not only for bridged traffic. Hasn't anybody ever created a
bridge with only one interface just in order to be able to use ebtables
on this interface ?

Re: Bridges

[Grant Taylor]

On 08/20/10 05:26, Pascal Hambourg wrote:
> Hasn't anybody ever created a bridge with only one interface just in 
> order to be able to use ebtables on this interface ?

I'd be very surprised if someone somewhere has not done that.

I would think that it would be a very special use case scenario where 
you needed to do something in EBTables that couldn't be done in IPTables 
/ ARPTables.  But that's not to say that the need doesn't exist.  ;)



Grant. . . .

Re: Bridges

[Jan Engelhardt]

On Friday 2010-08-20 18:02, Grant Taylor wrote:

> On 08/20/10 05:26, Pascal Hambourg wrote:
>> Hasn't anybody ever created a bridge with only one interface just in order to
>> be able to use ebtables on this interface ?
>
> I'd be very surprised if someone somewhere has not done that.
>
> I would think that it would be a very special use case scenario where you
> needed to do something in EBTables that couldn't be done in IPTables /
> ARPTables.  But that's not to say that the need doesn't exist.  ;)

ARP proxying with ebtables, basically. (Though there is also a 
routing-level arp proxy, but somehow it does not feel as steady.)

Re: Bridges

[Grant Taylor]

On 08/20/10 11:18, Jan Engelhardt wrote:
> ARP proxying with ebtables, basically. (Though there is also a 
> routing-level arp proxy, but somehow it does not feel as steady.)

ARP proxying with EBTables?  Would you mind providing an example use?

The only thing that I can think of that would come close to this would 
be to use EBTables to modify the source and destination MAC addresses of 
ethernet frames so that two end stations on the same segment are talking 
to each other via the bridge performing a man-in-the-middle type operation.

I agree that is a use, I just don't understand it or the motivation 
behind it as well as I'd like to.  -  Said another way, just my morbid 
curiosity.



Grant. . . .

Re: Bridges

[Jan Engelhardt]

On Friday 2010-08-20 18:25, Grant Taylor wrote:

> On 08/20/10 11:18, Jan Engelhardt wrote:
>> ARP proxying with ebtables, basically. (Though there is also a routing-level
>> arp proxy, but somehow it does not feel as steady.)
>
> ARP proxying with EBTables?  Would you mind providing an example use?

ebtables -t broute ! -p arp -j DROP
ebtables -t nat -A PREROUTING -i br0 \
	-p arp --arp-opcode request --arp-ip-dst mysubnet/CIDR \
	-j arpreply --arpreply-mac \
	$(cat /sys/class/net/br0/address)

Something like that it was.

ARP proxying can be helpful when your entire mysubnet/CIDR is 
direct-attached at the ISP's end but you don't want to bridge your 
network (for that would mean more ebtables and iptables rules) but 
instead force routing.

Re: Bridges

[Pascal Hambourg]

Grant Taylor a écrit :
> On 08/20/10 05:26, Pascal Hambourg wrote:
>> Hasn't anybody ever created a bridge with only one interface just in 
>> order to be able to use ebtables on this interface ?
> 
> I'd be very surprised if someone somewhere has not done that.
> 
> I would think that it would be a very special use case scenario where 
> you needed to do something in EBTables that couldn't be done in IPTables 
> / ARPTables.

Ethernet does not carry only IP and ARP. There are other protocols in
use out there. One may want to do MAC-based filtering and mangling on
normal ethernet interfaces.
IMO restricting ebtables to bridges would be like restricting iptables
to routers.

Re: Bridges

[Grant Taylor]

Pascal Hambourg wrote:
> Ethernet does not carry only IP and ARP. There are other protocols in
> use out there. One may want to do MAC-based filtering and mangling on
> normal ethernet interfaces.

Agreed.

However I do think (my opinion is) that IP is by far the simple majority 
of traffic.  Other protocols are less common and closer to the special 
use scenario than they are away from it.

> IMO restricting ebtables to bridges would be like restricting iptables
> to routers.

I in no way was implying that EBTables should be restricted.  Rather I 
was commenting on what I believe the use scenario is, /now/ any form of 
restriction.



Grant. . . .

Re: Bridges

[Jan Engelhardt]

On Thursday 2010-08-19 20:47, Jonathan Tripathy wrote:
>>
>>>> Disable global IP forwarding.
>>>>       
>>> I just ran "cat /proc/sys/net/ipv4/ip_forward" and it returned 0. Does
>>> this mean that I have "global IP forwarding" off?
>>>     
>> Yes. So unless you enable /proc/sys/net/ipv4/conf/<interface>/forwarding
>> for a bridge, packets won't jump from that bridge to another interface.
>>   
> Ok so it's all 0. However, in the /proc/sys/net/ipv4/conf folder, I don't see
> my "public" bridge, only the private one. Incidently, the private one is the
> only bridge in my system that has an IP address asigned to it...

How old a kernel is that?

Re: Bridges

[Pascal Hambourg]

Hello,

Grant Taylor a écrit :
> On 08/17/10 17:44, Jonathan Tripathy wrote:
>> When using a single Linux host with lots of bridges, would there ever be 
>> a time, even for a few seconds, where traffic would "jump" bridges?
> 
> No.  Such should not be possible.

It depends what "traffic" means. Ethernet frames, no. IP packets, yes if
the box also acts as an IP router between the bridges.

Re: Bridges

[Jonathan Tripathy]

On 18/08/10 23:59, Pascal Hambourg wrote:
> Hello,
>
> Grant Taylor a écrit :
>    
>> On 08/17/10 17:44, Jonathan Tripathy wrote:
>>      
>>> When using a single Linux host with lots of bridges, would there ever be
>>> a time, even for a few seconds, where traffic would "jump" bridges?
>>>        
>> No.  Such should not be possible.
>>      
> It depends what "traffic" means. Ethernet frames, no. IP packets, yes if
> the box also acts as an IP router between the bridges.
>    
Hmm for such to happen, the bridge where the traffic came in on would 
have to have an IP address, wouldn't it? Currently, my "public" bridge 
doesn't have an IP assigned to it...

Re: Bridges

[Pascal Hambourg]

Jonathan Tripathy a écrit :
> On 18/08/10 23:59, Pascal Hambourg wrote:
>>
>> Grant Taylor a écrit :
>>    
>>> On 08/17/10 17:44, Jonathan Tripathy wrote:
>>>      
>>>> When using a single Linux host with lots of bridges, would there ever be
>>>> a time, even for a few seconds, where traffic would "jump" bridges?
>>>>        
>>> No.  Such should not be possible.
>>>      
>> It depends what "traffic" means. Ethernet frames, no. IP packets, yes if
>> the box also acts as an IP router between the bridges.
>>    
> Hmm for such to happen, the bridge where the traffic came in on would 
> have to have an IP address, wouldn't it?

I don't think so, the bridge interface just needs to be linked to the IP
stack (which is now done automatically by recent kernels AFAIK). Routers
can have unnumbered interfaces.

Re: Bridges

[Jan Engelhardt]

On Thursday 2010-08-19 01:00, Jonathan Tripathy wrote:
>> It depends what "traffic" means. Ethernet frames, no. IP packets, yes if
>> the box also acts as an IP router between the bridges.
>>   
>Hmm for such to happen, the bridge where the traffic came in on
>would have to have an IP address, wouldn't it?

Routes are independent of assigned addresses. Though I guess it may 
obstruct ICMP "TTL outlived" replies if the router itself has no address 
to send with.

Re: Bridges

[Pascal Hambourg]

Jan Engelhardt a écrit :
> On Thursday 2010-08-19 01:00, Jonathan Tripathy wrote:
>>> It depends what "traffic" means. Ethernet frames, no. IP packets, yes if
>>> the box also acts as an IP router between the bridges.
>>>   
>> Hmm for such to happen, the bridge where the traffic came in on
>> would have to have an IP address, wouldn't it?
> 
> Routes are independent of assigned addresses.

Routes and addresses are not the only problem. IIRC, older kernels did
not automatically link an interface to the IP stack (it was not present
in /proc/sys/net/ipv4/conf/) until you do something such as assign an IP
address to it.

> Though I guess it may 
> obstruct ICMP "TTL outlived" replies if the router itself has no address 
> to send with.

Obviously, a router needs at least one non-loopback address in order to
operate properly. If such an address is assigned to another interface,
Linux can use it as a source address on the unnumbered interface.

Re: Bridges

[Grant Taylor]

Pascal Hambourg wrote:
> It depends what "traffic" means. Ethernet frames, no. IP packets, yes 
> if the box also acts as an IP router between the bridges.

I took Jonathan's comment to be referring to ethernet frames.

Even if the box is acting as an IP router between the bridges, I view 
that as an operation that is happening out side of the bridge(s).  Thus 
there is an external process that is taking IP packets and passing them 
between the bridges.  As such, the ""traffic is still not ""leaking / 
""flooding between the bridges as a failure or shortcoming of the 
bridge, but rather an external influence.

The same thing can be said about two switches that have a router between 
them.  When the ""traffic goes between the switches it is not because of 
something the switches did, rather the external router's doing.  ;-)



Grant. . . .

Re: Bridges

[Pascal Hambourg]

Grant Taylor a écrit :
> Pascal Hambourg wrote:
>> It depends what "traffic" means. Ethernet frames, no. IP packets, yes 
>> if the box also acts as an IP router between the bridges.
> 
> I took Jonathan's comment to be referring to ethernet frames.

Me too, but I wanted to be more general.

> Even if the box is acting as an IP router between the bridges, I view 
> that as an operation that is happening out side of the bridge(s).

Of course.

> The same thing can be said about two switches that have a router between 
> them.

Or one L3 switch with two port-based VLANs. Packets should not leak
between VLANs unless they're routed.

Re: Bridges

[Grant Taylor]

On 08/19/10 02:33, Pascal Hambourg wrote:
> Me too, but I wanted to be more general.

Fair enough.  ;-)

> Or one L3 switch with two port-based VLANs. Packets should not leak 
> between VLANs unless they're routed.

Heh.  I'll argue that even if the device that ""leaks the packet from 
one interface to another is called a switch, that it is still doing a 
layer 3 function, namely routing.  Layer 3 switches are just really 
efficient routers when it comes to routing from one vlan to another.

(That or I vastly mis-understand how layer 3 switching is different from 
routing.  At least when it comes to what is done with the IP packet, 
regardless of how it's technically done.  If I am mistaken I'd love to 
have someone correct me.)



Grant. . . .

Re: Bridges

[Jan Engelhardt]

On Thursday 2010-08-19 16:51, Grant Taylor wrote:

> (That or I vastly mis-understand how layer 3 switching is different from
> routing.


"Layer 3 switching" is usually a bullshit bingo word for routing.

Re: Bridges

[Grant Taylor]

On 08/19/10 09:56, Jan Engelhardt wrote:
> "Layer 3 switching" is usually a bullshit bingo word for routing.

*chuckle*

I like how you flop your opinion on the table Jan'.  :-)

I'm actually having a discussion about routing vs l3 switching with a 
friend and colleague, prompted by this thread.

It is my belief (and my colleague / friend can't say that I'm wrong) 
that both routing and layer 3 switching will take an ethernet frame that 
is coming in one interface with one set of source / dest MAC addresses 
and forward it out another interface with a different set of source / 
dest MAC addresses in the process of forwarding the IP packet on to its 
ultimate destination.  (They both will also do some other things like 
validating checksum, decrement the TTL, etc.)

IMHO the real (and only) difference (as I (mis)understand it) between 
routing and layer 3 switching is /how/ each accomplishes its task. 
Routing is traditionally a software in CPU operation where as layer 3 
switching is hardware ASICs.  Software in CPU is quit a bit slower than 
optimized hardware ASICs.  Thus l3 switches can usually much more 
closely approach wire speed than routers can.

Skim the following article on Cisco's web site.

Link - Layer 2 and Layer 3 Switch Evolution - The Internet Protocol 
Journal - Volume 1, No. 2 - Cisco Systems
  - 
http://www.cisco.com/web/about/ac123/ac147/archived_issues/ipj_1-2/switch_evolution.html

As the article points out, layer 3 switching is done for the simple 
routing tasks.  Any time there is complexity involved, the packets are 
passed up to the CPU / software to be routed traditionally.

Along the same lines, it is my belief that (currently) layer 3 switches 
can't switch between desperate network technologies.  I.e. Ethernet / 
FDDI or Ethernet / Token Ring or Ethernet / ATM can not be layer 3 
switches, they require traditional routing because of the differences in 
the physical topologies.



Grant. . . .

Re: Bridges

[Jan Engelhardt]

On Thursday 2010-08-19 17:49, Grant Taylor wrote:
>On 08/19/10 09:56, Jan Engelhardt wrote:
>>"Layer 3 switching" is usually a bullshit bingo word for routing.
>
>I like how you flop your opinion on the table Jan'.  :-)
>[...]
>IMHO the real (and only) difference (as I (mis)understand it) between routing
>and layer 3 switching is /how/ each accomplishes its task. Routing is
>traditionally a software in CPU operation where as layer 3 switching is
>hardware ASICs.  Software in CPU is quit a bit slower than optimized hardware
>ASICs.


Routing as a concept remains hardware-agnostic. Otherwise you would
have to tell me whether a bulk of RFC1149-conforming pigeons do
"routing" or "L3-switching". Absurd comparison? Optimized ICs surely
existed before people started calling routing l3-switching.

> Skim the following article on Cisco's web site.

(tl;dr)
Right... directly from the propaganda ministry. I can see the flaw in that.

Re: Bridges

[Grant Taylor]

On 08/19/10 11:21, Jan Engelhardt wrote:
> Routing as a concept remains hardware-agnostic.

Agreed.

> Otherwise you would have to tell me whether a bulk of 
> RFC1149-conforming pigeons do "routing" or "L3-switching". Absurd 
> comparison?

I believe layer 3 switching is a subset of the generic concept of routing.

I forgot how funny RFC 1149 was.  Thank you for the chuckle.  :-)

Seeing as how the RFC 1149 specifies Avian Carriers and the printing / 
scanning of data, I think it defines both layers 1 and 2.  Further, the 
RFC states that A.C. are a point to point technology.  (Where said 
points are is subject to change and storms.)

As such, I don't believe that A.C. do routing in any form at all.  I 
believe the routing is better left to the layer 2.5 / 3 human that is 
printing, duct taping, tossing, removing, and scanning of datagrams.

Directly, it is my belief that RFC1149-conforming pigeons don't do 
"routing", much less "L3-switching".

Nope, your comparison is not absurd.  Though I do think it does open an 
can of worms.  ;-)

> Optimized ICs surely existed before people started calling routing 
> l3-switching.

Quite possibly.

I can't recall when I first heard about layer 3 switching.  Though if we 
look at the technology of some older (mid to late 90's) higher end 
routers, they had switching planes in them.  So I'd say that l3 
switching (in a switch) is an evolution of technology and More's Law 
allowing said technology to come to the l3 wiring closet switch.

> Right... directly from the propaganda ministry. I can see the flaw in 
> that.

Hehe.  ;-)

Cisco may be the propaganda ministry, but most of their technical 
documentation is accurate.  Or at least what it's describing is often 
imitated by other vendors.  Common law standard any one?



Grant. . . .

Re: Bridges

[Jan Engelhardt]

On Thursday 2010-08-19 18:41, Grant Taylor wrote:
> On 08/19/10 11:21, Jan Engelhardt wrote:
>
>>Otherwise you would have to tell me whether a bulk of RFC1149-conforming
>>pigeons do "routing" or "L3-switching".
>
>Seeing as how the RFC 1149 specifies Avian Carriers and the printing
>/ scanning of data, I think it defines both layers 1 and 2. Further,
>the RFC states that A.C. are a point to point technology. (Where
>said points are is subject to change and storms.)

It's multihop.

>As such, I don't believe that A.C. do routing in any form at all.

They do have a routing engine however. Otherwise they'd go
straight up the next wall.

>> Right... directly from the propaganda ministry. I can see the flaw in that.
>
>Hehe.  ;-)
>
>Cisco may be the propaganda ministry, but most of their technical documentation
>is accurate.

Did they ever tell you about that red button in their products?

>  Or at least what it's describing is often imitated by other
>vendors.  Common law standard any one?

Bad plagiarism? Doing a thing wrong N times doesn't make it
(100-100/N)% more right. :-)

Re: Bridges

[Grant Taylor]

On 08/19/10 12:10, Jan Engelhardt wrote:
> It's multihop.
> 
> They do have a routing engine however. Otherwise they'd go
> straight up the next wall.

Ok.  Fair enough.  :-)

> Did they ever tell you about that red button in their products?

Nope.

I assume that you aren't talking about Random Early Drop.

I didn't say it was complete, just that the parts that are there are 
(usually) technically accurate.

> Bad plagiarism?

Chuckle.

> Doing a thing wrong N times doesn't make it (100-100/N)% more right. :-)

True.

Though a "standard" does not imply right, just the same.

I'm not overly thrilled with SMB / CIFS networking, but it is (mostly) 
the de facto standard (for now).  (At least the simple majority.)



Grant. . . .

Re: Bridges

[Rick Jones]

It starts to stretch my DIMM wetware memory, but "switching" started being 
attached to devices doing routing around the time that switch vendors (layer2) 
were saying people should build fewer, flatter subnets and have the switch 
vendors' switches pass traffic around because "switching is faster than routing" 
  Also asserted at the time was that "switching is simpler than routing."

Toss-in a bit of keeping-up, as well as some vendors adding "routing" ability to 
their "switches" and poof, we have the bingo term L3 switching.  Have it stick 
around long enough and one will end-up with the telephone dressers and hair 
sanitizers finding some hairs to split to differentiate between an "L3 switch" 
and an "L3 router" ...

rick jones

bridges

[Ross Boylan]

I'm trying to understand bridging with KVM, but am still puzzled.
I think that the recommended bridging with TAP means that packets from
the VM will end up going out the host card attached to the default
gateway.  But it looks to me as if their IP address is unchanged, which
means replies will never reach me.  Is that correct?  Do I need to NAT
the packets, or is something already doing that?

Some documents indicate that I need to bring the interfaces (e.g., eth0)
down before I bring the bridge up, and that afterwards only the bridge
will have an IP address.  Is that right?

Some documents, e.g.,
http://ebtables.sourceforge.net/br_fw_ia/br_fw_ia.html, indicate
iptables should "just work" with bridging.  However, I've seen someone
with a 2.6.15 kernel ask about firewalling and be told they needed to
patch the kernel to get it work (don't have the reference handy).
Should it just work?

I'm running a 2.6.29 kernel on Debian Lenny with kvm 72+dfsg-5~lenny1.
Version 84+dfsg-2 is available in experimental.  Is there much to be
gained by going with the more recent version?

Please cc me; I'm not on the list.

Thanks.
Ross Boylan

Re: bridges

[Ross Boylan]

On Thu, 2009-05-07 at 11:13 -0600, Cam Macdonell wrote:
> 
> Ross Boylan wrote:
> > I'm trying to understand bridging with KVM, but am still puzzled.
> > I think that the recommended bridging with TAP means that packets
> from
> > the VM will end up going out the host card attached to the default
> > gateway.  But it looks to me as if their IP address is unchanged,
> which
> > means replies will never reach me.  Is that correct?  Do I need to
> NAT
> > the packets, or is something already doing that?
> 
> Hi Ross,
> 
> This is the place to start
> 
> http://www.linux-kvm.org/page/Networking.  
I saw that; it gives some recipes but I wasn't sure what their effect
was.

> You want a public bridge.
> 
> I'm not sure what "their" and "me" mean in your email.  In short,
> with 
> bridging each VM has its own IP and that VM can be accessed directly 
> from the network.
"their" = the VM.
"me" = my host machine.

So if the VM's are running on their own subnet, e.g., 10.0.2.* (I've
been assuming the subnet with TAP is like the one with the User Mode
Network stack in 3.7.3 of http://www.nongnu.org/qemu/qemu-doc.html) and
my host machine is on another net, e.g., 10.0.8.* then I think the
packet will go out with an IP of 10.0.2.2 (say).  When some other
machine tries to reply to 10.0.2.2, the packet gets lost because the
outside network thinks 10.0.2.* is not for it.  At least that's my
concern.  If the return IP address on the packet were 10.0.8.44
(supposing that's the IP of my host machine) then the packets could find
their way back.

My host machine is on an internal network with a 10.* IP.  The example
might be clearer if one supposed that the VM's were on a 192.168.*
network.

I am perhaps being influenced by the fact that I don't want to ask for
more IP's, so I don't want to configure the VM's to use an IP on our
10.0.8 network.

Does the TAP networking setup a whole subnet like the user mode network
stack (e.g., running a DHCP server), or is the idea that I would just
give the VM an IP on my subnet (10.0.8.*) in this example?

If the latter is the case (I'm now suspecting it is) then I think the
solution is clear.  I just stick the VM's on a private (to my machine)
subnet, like 192.168.*, and I do NAT on the packets as they go out.

Re: bridges

[Cam Macdonell]

Ross Boylan wrote:
> On Thu, 2009-05-07 at 11:13 -0600, Cam Macdonell wrote:
>> Ross Boylan wrote:
>>> I'm trying to understand bridging with KVM, but am still puzzled.
>>> I think that the recommended bridging with TAP means that packets
>> from
>>> the VM will end up going out the host card attached to the default
>>> gateway.  But it looks to me as if their IP address is unchanged,
>> which
>>> means replies will never reach me.  Is that correct?  Do I need to
>> NAT
>>> the packets, or is something already doing that?
>> Hi Ross,
>>
>> This is the place to start
>>
>> http://www.linux-kvm.org/page/Networking.  
> I saw that; it gives some recipes but I wasn't sure what their effect
> was.
> 
>> You want a public bridge.
>>
>> I'm not sure what "their" and "me" mean in your email.  In short,
>> with 
>> bridging each VM has its own IP and that VM can be accessed directly 
>> from the network.
> "their" = the VM.
> "me" = my host machine.
> 
> So if the VM's are running on their own subnet, 

VMs do not run on their own subnet with bridged networking.

>e.g., 10.0.2.* (I've
> been assuming the subnet with TAP is like the one with the User Mode
> Network stack in 3.7.3 of http://www.nongnu.org/qemu/qemu-doc.html) and
> my host machine is on another net, e.g., 10.0.8.* then I think the
> packet will go out with an IP of 10.0.2.2 (say).  When some other
> machine tries to reply to 10.0.2.2, the packet gets lost because the
> outside network thinks 10.0.2.* is not for it.  At least that's my
> concern.  If the return IP address on the packet were 10.0.8.44
> (supposing that's the IP of my host machine) then the packets could find
> their way back.

Using bridged networking is very different from the user stack.  The 
user stack is extremely limited and slow.

> 
> My host machine is on an internal network with a 10.* IP.  The example
> might be clearer if one supposed that the VM's were on a 192.168.*
> network.
> 
> I am perhaps being influenced by the fact that I don't want to ask for
> more IP's, so I don't want to configure the VM's to use an IP on our
> 10.0.8 network.

Then you probably want to use a NAT network.  A NAT setup puts all the 
VMs on their own network within the host machine.  iptables is necessary 
to forward the subnet packets out to the world and back.

Here is some older documentation, but not much has changed.  Look at the 
first entry under "Advanced Networking".

https://help.ubuntu.com/community/KVMFeisty

> Does the TAP networking setup a whole subnet like the user mode network
> stack (e.g., running a DHCP server), or is the idea that I would just
> give the VM an IP on my subnet (10.0.8.*) in this example?

No, bridge networking using taps (one tap per VM) and effectively sits 
all the VMs on the same network your host is on.  You would need to get 
IPs from sysadmin for each VM.

> If the latter is the case (I'm now suspecting it is) then I think the
> solution is clear.  I just stick the VM's on a private (to my machine)
> subnet, like 192.168.*, and I do NAT on the packets as they go out.

NAT is a very common solution.  Use VDE (vde.sourceforget.net) to create 
a virtual switch on your host for the VMs.  dnsmasq can serve dynamic 
IPs to the VMs on their own subnet that doesn't bother your sysadmin at 
all.  Use iptables to forward and receive packets through your host's 
NIC.

Cam

Re: bridges

[Matthew Palmer]

On Thu, May 07, 2009 at 08:57:03AM -0700, Ross Boylan wrote:
> I'm trying to understand bridging with KVM, but am still puzzled.
> I think that the recommended bridging with TAP means that packets from
> the VM will end up going out the host card attached to the default
> gateway.  But it looks to me as if their IP address is unchanged, which
> means replies will never reach me.  Is that correct?  Do I need to NAT
> the packets, or is something already doing that?
> 
> Some documents indicate that I need to bring the interfaces (e.g., eth0)
> down before I bring the bridge up, and that afterwards only the bridge
> will have an IP address.  Is that right?

Here's how I think of a Linux "soft" bridge: the bridge consists of an
Ethernet switch, and a regular interface (named after the bridge) that is
connected to that switch.  This is why you "give an IP address to the
bridge", because "the bridge" is also a NIC of it's own.

If you attach any physical interfaces (eg ethN) to the bridge, they aren't
NICs any more, they're just network cables you plug into the switch to pass
traffic to other switches.  Attaching VMs to the switch is just hooking up
more cables between the switch and the VMs.

If you want your host to do NAT for your VMs, then you do as you would for
any other firewall -- you have one switch (the bridge, in this case) with
all of your VMs and the "internal" interface of the host (in this case, the
bridge as well) all plugged in, and then a second interface to the outside
world (the physical NIC).

> Some documents, e.g.,
> http://ebtables.sourceforge.net/br_fw_ia/br_fw_ia.html, indicate
> iptables should "just work" with bridging.

Yes, iptables *does* "just work" with bridging, in the sense that iptables
can still filter IP packets passing through it's interfaces.  What you
*can't* do, though, is have some sort of magic iptables filter deep in the
bridge that plays with all traffic as it traverses.  For that, there's
ebtables, which is iptables but for Ethernet (rather than IP) traffic. 
Personally, I've never used ebtables in my life.

> However, I've seen someone
> with a 2.6.15 kernel ask about firewalling and be told they needed to
> patch the kernel to get it work (don't have the reference handy).
> Should it just work?

It should Just Work, and if you've got to patch any 2.6 (or even probably
2.4) kernel then you're doing something *very* esoteric.

- Matt