BUG: 2 NICs on same network

BUG: 2 NICs on same network

[Frank Louwers]

Hi,

We recently stummed across a rather annoying bug when 2 nics are on
the same network.

Our situation is this: we have a server with 2 nics, each with a
different IP on the same network, connected to the same switch. Let's
assume eth0 has ip 1.2.3.1 and eth1 has 1.2.3.2, with a both with a
netmask of 255.255.255.0.

Now the strange thing is that traffic for 1.2.3.2 arrives at eth0 no
matter what!

Even if we disconnect the cable for eth1, 1.2.3.2 still replies to
pings, ssh, web, ...

We tested this on IA32 architecture, different 2.4.x kernels and
different nics ...

Is this a bug or a known issue? If it is not a bug, how can it be
solved?

Kind Regards,
Frank Louwers

-- 
Openminds bvba                www.openminds.be
Tweebruggenstraat 16  -  9000 Gent  -  Belgium

Re: BUG: 2 NICs on same network

[Frank Louwers]

On Tue, Apr 23, 2002 at 04:23:25PM +0530, rpm wrote:
> check the MAC address  of the two cards cards !  

Mac addresses are different! (In my first setup the addresses only
differed 1 bit (server with 2 nics onboard)), but later I tried with 2
different brand of nics, so they should have been different enough.

> i am also using similar configuration but am not getting any thing like that !

When i disconnect the cable of eth1, and ping it's ip from another
machine and check the arp table for that machine, I get the mac
address for eth0 ...


Frank

Re: BUG: 2 NICs on same network

[rpm]

check the MAC address  of the two cards cards !  
i am also using similar configuration but am not getting any thing like that !

rpm
On Tuesday 23 April 2002 03:09 pm, Frank Louwers wrote:
> Hi,
>
> We recently stummed across a rather annoying bug when 2 nics are on
> the same network.
>
> Our situation is this: we have a server with 2 nics, each with a
> different IP on the same network, connected to the same switch. Let's
> assume eth0 has ip 1.2.3.1 and eth1 has 1.2.3.2, with a both with a
> netmask of 255.255.255.0.
>
> Now the strange thing is that traffic for 1.2.3.2 arrives at eth0 no
> matter what!
>
> Even if we disconnect the cable for eth1, 1.2.3.2 still replies to
> pings, ssh, web, ...
>
> We tested this on IA32 architecture, different 2.4.x kernels and
> different nics ...
>
> Is this a bug or a known issue? If it is not a bug, how can it be
> solved?
>
> Kind Regards,
> Frank Louwers

Re: BUG: 2 NICs on same network

[Frank Louwers]

On Tue, Apr 23, 2002 at 01:41:35PM +0200, Vincent Guffens wrote:
> I see it here too, 

[CCing back to list as Vincent saw the same behaviour]

> 
> usermode:~# ping 192.168.0.5
> PING 192.168.0.5 (192.168.0.5): 56 data bytes
> 64 bytes from 192.168.0.5: icmp_seq=0 ttl=255 time=20.3 ms
> 
> usermode:~# ping 192.168.0.6
> PING 192.168.0.6 (192.168.0.6): 56 data bytes
> 64 bytes from 192.168.0.6: icmp_seq=0 ttl=255 time=3.5 ms
> 
> usermode:~# arp -a
> ? (192.168.0.5) at FE:FD:C0:A8:00:05 [ether] on eth0
> ? (192.168.0.6) at FE:FD:C0:A8:00:05 [ether] on eth0
> usermode:~# 
> 
> 
> 
> bash-2.05# tcpdump -e -i eth0                                                                                                       
> tcpdump: listening on eth0
> 07:21:00.284894 fe:fd:c0:a8:0:1 Broadcast arp 42: arp who-has 192.168.0.5 tell 192.168.0.1
> 07:21:00.285020 fe:fd:c0:a8:0:5 fe:fd:c0:a8:0:1 arp 42: arp reply 192.168.0.5 is-at fe:fd:c0:a8:0:5
> 07:21:00.301634 fe:fd:c0:a8:0:5 Broadcast arp 42: arp who-has router tell 192.168.0.5
> 07:21:00.303418 fe:fd:c0:a8:0:1 fe:fd:c0:a8:0:5 ip 98: 192.168.0.1 > 192.168.0.5: icmp: echo request (DF)
> 07:21:00.303589 fe:fd:c0:a8:0:5 fe:fd:c0:a8:0:1 ip 98: 192.168.0.5 > 192.168.0.1: icmp: echo reply
> 07:21:01.324561 fe:fd:c0:a8:0:5 Broadcast arp 42: arp who-has router tell 192.168.0.5
> 07:21:02.364564 fe:fd:c0:a8:0:5 Broadcast arp 42: arp who-has router tell 192.168.0.5
> 07:21:03.544592 fe:fd:c0:a8:0:1 Broadcast arp 42: arp who-has 192.168.0.6 tell 192.168.0.1
> 07:21:03.544714 fe:fd:c0:a8:0:5 fe:fd:c0:a8:0:1 arp 42: arp reply 192.168.0.6 is-at fe:fd:c0:a8:0:5      <------
> 
> 
> the interface having .5 is replying on behalf of .6. It probably comes from the fact that it is not legal to put two different
> interfaces in the same subnet. You should probably be using a load balancer interface or a bridge interface or subnet your /24
> . But as I imagine, you can't subnet, if you use the bridge you will have to enable stp which will disable one of the link and
> if you use eql, you switch has to understand it (would probably be the best though)
> 
> You can still add some static arp entries but it not very scalable and not very beautifull either ...
> 
> I'm curious about what they will say on the list,

Well, load balancing or bridging is an option, but not the one I
want... I will use the first nic as "normal" (firewalled, traffic
shaped, ...) interface for my customer's websites, and eth1 as the
"backup" and maintenance nic in case something goes wrong ...

I don't understand why it should be illegal to have to nics on the
same server on the same subnet ...

Regarding static arps: am I correct these should be added to all
machines on the subnet and to the router? (That last is not possible,
I don't have permission to change stuff on the router)

Vriendelijke groeten,
Frank Louwers

-- 
Openminds bvba                www.openminds.be
Tweebruggenstraat 16  -  9000 Gent  -  Belgium

Re: BUG: 2 NICs on same network

[Frank Louwers]

On Tue, Apr 23, 2002 at 02:05:02PM -0200, Denis Vlasenko wrote:
> On 23 April 2002 07:39, Frank Louwers wrote:
> > Is this a bug or a known issue? If it is not a bug, how can it be
> > solved?
> 
> Do you have ip forwarding enabled? If yes, kernel just forwards packets:
> network -> eth0 -> kernel
> 
> Try traceroute to eth1' ip. Examine arp tables. What MAC is listed there for 
> eth1 IP?

ipforwarding is disabled, arp proxy is disabled.

The mac address is that of eth0 ...

Vriendelijke groeten,
Frank Louwers

-- 
Openminds bvba                www.openminds.be
Tweebruggenstraat 16  -  9000 Gent  -  Belgium

Re: BUG: 2 NICs on same network

[rpm]

try  disconnecting eth0 interface ( with eth1 connected )
and then pinging it  eth0 ! 

On Tuesday 23 April 2002 04:22 pm, Frank Louwers wrote:
> On Tue, Apr 23, 2002 at 04:23:25PM +0530, rpm wrote:
> > check the MAC address  of the two cards cards !
>
> Mac addresses are different! (In my first setup the addresses only
> differed 1 bit (server with 2 nics onboard)), but later I tried with 2
> different brand of nics, so they should have been different enough.
>
> > i am also using similar configuration but am not getting any thing like
> > that !
>
> When i disconnect the cable of eth1, and ping it's ip from another
> machine and check the arp table for that machine, I get the mac
> address for eth0 ...
>
>
> Frank

Re: BUG: 2 NICs on same network

[Chris Friesen]

Frank Louwers wrote:
> 
> Hi,
> 
> We recently stummed across a rather annoying bug when 2 nics are on
> the same network.
> 
> Our situation is this: we have a server with 2 nics, each with a
> different IP on the same network, connected to the same switch. Let's
> assume eth0 has ip 1.2.3.1 and eth1 has 1.2.3.2, with a both with a
> netmask of 255.255.255.0.
> 
> Now the strange thing is that traffic for 1.2.3.2 arrives at eth0 no
> matter what!


This is actually standards compliant behaviour, as silly as it sounds.  However,
if you want stricter arp behaviour I *think* that the following will fix it.  At
least it used to...


echo 1 > /proc/sys/net/ipv4/conf/all/arp_filter


If this is no longer correct I'd love to hear it.

Chris


Chris Friesen                    | MailStop: 043/33/F10  
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: BUG: 2 NICs on same network

[Jesse Pollard]

---------  Received message begins Here  ---------

> 
> On 23 April 2002 09:11, Frank Louwers wrote:
> > > > Is this a bug or a known issue? If it is not a bug, how can it be
> > > > solved?
> > >
> > > Do you have ip forwarding enabled? If yes, kernel just forwards packets:
> > > network -> eth0 -> kernel
> > >
> > > Try traceroute to eth1' ip. Examine arp tables. What MAC is listed there
> > > for eth1 IP?
> >
> > ipforwarding is disabled, arp proxy is disabled.
> >
> > The mac address is that of eth0 ...
> 
> Kernel bug I think. Sorry can't help you here.
> I'm afraid you have to dig deeper...

I would suspect the default route is directing the reply (netstat -r should
show if this is the case).
This happens whenever there are two NICs to the same network. An incoming
connection will go out the default route IF the default route is the same
as the network address. This gives preferential activity to the designated
route. We've done this when phasing out one interface in favor of another
(fddi->GEthernet)..

The usual reccomendation is not to have both interfaces active simultaneiously
as this causes (relatively minor) confusion. The connections should
work, but one will be relayed to the other interface since there is no
distinguishing characteristic about which interface to use - they are
equivalent. Once the default route is switched, the inverse situation
occurs.

If you want only one port, just down the undesired interface. Use it only
when the primary fails. If you need both IPs active, then use ifconfig eth0:1
and have both on the same interface (I think that still works).

Load balancing really calls for two different subnets. Otherwise you get
collisions between the two interfaces...

-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@navo.hpc.mil

Any opinions expressed are solely my own.

Re: BUG: 2 NICs on same network

[Jeff Garzik]

On Tue, Apr 23, 2002 at 11:39:35AM +0200, Frank Louwers wrote:
> Our situation is this: we have a server with 2 nics, each with a
> different IP on the same network, connected to the same switch. Let's
> assume eth0 has ip 1.2.3.1 and eth1 has 1.2.3.2, with a both with a
> netmask of 255.255.255.0.
> 
> Now the strange thing is that traffic for 1.2.3.2 arrives at eth0 no
> matter what!
> 
> Even if we disconnect the cable for eth1, 1.2.3.2 still replies to
> pings, ssh, web, ...

If the two NICs are on the same network, the kernel will send the
packets to the first available interface.

The kernel has acted like this for years...

	Jeff

Re: BUG: 2 NICs on same network

[J.A. Magallon]

On 2002.04.23 Frank Louwers wrote:
>Hi,
>
>We recently stummed across a rather annoying bug when 2 nics are on
>the same network.
>
>Our situation is this: we have a server with 2 nics, each with a
>different IP on the same network, connected to the same switch. Let's
>assume eth0 has ip 1.2.3.1 and eth1 has 1.2.3.2, with a both with a
>netmask of 255.255.255.0.
>
>Now the strange thing is that traffic for 1.2.3.2 arrives at eth0 no
>matter what!
>

>From wht I understand for net, interface selection for a connection is
done based on network address. So if you first configure eth0 with
mask 255.255.255.0, and a subnet of 1.2.3.0,
and eth1 is configured just the same (ip adresses only
differ in last number), the kernel just uses the first interface it
founds for a subnet. To prove this, you could try to load the interfaces
in reverse order, and all traffic should go to eth1.

Do you really need the two interfaces to be in the same subnet ? I use
tw parallel nets for a cluster, but configured both as independent
subnets, 10.0.0.0 and 10.0.1.0. So you can drive all nfs through one
interface mounting the server as 10.0.0.1, and all the bproc traffic
through the other (or all the ssh through the other connecting
always to 10.0.1.1).

Hope this helps.

-- 
J.A. Magallon                           #  Let the source be with you...        
mailto:jamagallon@able.es
Mandrake Linux release 8.3 (Cooker) for i586
Linux werewolf 2.4.19-pre7-jam5 #1 SMP mar abr 23 01:29:38 CEST 2002 i686

Re: BUG: 2 NICs on same network

[Frank Louwers]

 
> Do you really need the two interfaces to be in the same subnet ? I use
> tw parallel nets for a cluster, but configured both as independent
> subnets, 10.0.0.0 and 10.0.1.0. So you can drive all nfs through one
> interface mounting the server as 10.0.0.1, and all the bproc traffic
> through the other (or all the ssh through the other connecting
> always to 10.0.1.1).
> 
> Hope this helps.

Well, I was planning on using it as a backup link: use eth0 as my
"normal" interface card, with a traffic shaper on it, firewalled etc.

If however, something goes wrong with either the firewall, or traffic
shaper, or so, I have to drive 60 miles to my server to correct things
on the console.

I was hoping on configuring that second nic as a "backup" nic ...

Frank

Re: BUG: 2 NICs on same network

[Harley Stenzel]

Frank Louwers wrote:
> Hi,
> 
> We recently stummed across a rather annoying bug when 2 nics are on
> the same network.
> 
> Our situation is this: we have a server with 2 nics, each with a
> different IP on the same network, connected to the same switch. Let's
> assume eth0 has ip 1.2.3.1 and eth1 has 1.2.3.2, with a both with a
> netmask of 255.255.255.0.
. . .
> 
> Is this a bug or a known issue? If it is not a bug, how can it be
> solved?

It is a known issue with 2.4 Linux kernels and ARP.  Linux will respond 
to arp requests for any address configured on the box on any interface 
that receives the arp request.  Patches have been proposed in the past, 
but the maintainers have elected to not accept the patches on the basis 
that the current behavior is RFC-compliant.  The behavior you describe 
is also RFC-compliant, and is in fact what the other OSes that I'm 
familiar with do.

In your situation, where you have a single nic that you want to use as 
backup only, you could set noarp on the backup nic.  Then, when you want 
to talk to the machine on the backup nic, you could use a static arp 
entry specifying the MAC address of the backup and any IP address on the 
box.

Alternatively, you could put the NICs on different physical segments or 
you could dig up the proposed patches (2.4.0,2,and 12 if I remember 
correctly) and port them forward and apply them.

> 
> Kind Regards,
> Frank Louwers
> 

--Harley Stenzel
   hstenzel@nc.rr.com

Re: BUG: 2 NICs on same network

[Vincent Guffens]

On Tue, Apr 23, 2002 at 09:10:58AM -0400, Chris Friesen wrote:
> Frank Louwers wrote:
> > 
> > Hi,
> > 
> > We recently stummed across a rather annoying bug when 2 nics are on
> > the same network.
> > 
> > Our situation is this: we have a server with 2 nics, each with a
> > different IP on the same network, connected to the same switch. Let's
> > assume eth0 has ip 1.2.3.1 and eth1 has 1.2.3.2, with a both with a
> > netmask of 255.255.255.0.
> > 
> > Now the strange thing is that traffic for 1.2.3.2 arrives at eth0 no
> > matter what!
> 
> 
> This is actually standards compliant behaviour, as silly as it sounds.  However,
> if you want stricter arp behaviour I *think* that the following will fix it.  At
> least it used to...
> 
> 
> echo 1 > /proc/sys/net/ipv4/conf/all/arp_filter
> 
> 
> If this is no longer correct I'd love to hear it.


Hi,

I'm looking at that as well and it doesn't look as it solves the problem, in fact I think that the arp_filter is meant to
insure that the reply will be sent out to the interface where linux would route it, that is to say, the other way around. It
should be something like that instead :

ip_route_output(&rt,tip, sip, 0, 0) 

But this function will route tip to eth0 ( the wrong one in that context). But in fact, it comes back to the fact that the
two interfaces are put on the same subnet. I know that on some routers, this not even possible to configure  two interfaces on
the same subnet, it is supposed to be possible on linux ? ok, from what I see in the reply, this is possible but linux will
take the first one into account and install a local route to that subnet pointing to the first interface , making the other one
useless as we see in this example.  

   

static int arp_filter(__u32 sip, __u32 tip, struct net_device *dev)
{
        struct rtable *rt;
        int flag = 0;
        /*unsigned long now; */

        
        if (ip_route_output(&rt, sip, tip, 0, 0) < 0)
                return 1;             
        if (rt->u.dst.dev != dev) {
                NET_INC_STATS_BH(ArpFilter);
                flag = 1;
        } 

        ip_rt_put(rt);



> 
> Chris
> 
> 
> Chris Friesen                    | MailStop: 043/33/F10  
> Nortel Networks                  | work: (613) 765-0557
> 3500 Carling Avenue              | fax:  (613) 765-2986
> Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

-- 
		Vincent Guffens
		Unite d'automatique, de dynamique et d'analyse des systemes, UCL

Re: BUG: 2 NICs on same network

[Denis Vlasenko]

On 23 April 2002 07:39, Frank Louwers wrote:
> Hi,
>
> We recently stummed across a rather annoying bug when 2 nics are on
> the same network.
>
> Our situation is this: we have a server with 2 nics, each with a
> different IP on the same network, connected to the same switch. Let's
> assume eth0 has ip 1.2.3.1 and eth1 has 1.2.3.2, with a both with a
> netmask of 255.255.255.0.
>
> Now the strange thing is that traffic for 1.2.3.2 arrives at eth0 no
> matter what!
>
> Even if we disconnect the cable for eth1, 1.2.3.2 still replies to
> pings, ssh, web, ...
>
> We tested this on IA32 architecture, different 2.4.x kernels and
> different nics ...
>
> Is this a bug or a known issue? If it is not a bug, how can it be
> solved?

Do you have ip forwarding enabled? If yes, kernel just forwards packets:
network -> eth0 -> kernel

Try traceroute to eth1' ip. Examine arp tables. What MAC is listed there for 
eth1 IP?
--
vda

Re: BUG: 2 NICs on same network

[Denis Vlasenko]

On 23 April 2002 09:11, Frank Louwers wrote:
> > > Is this a bug or a known issue? If it is not a bug, how can it be
> > > solved?
> >
> > Do you have ip forwarding enabled? If yes, kernel just forwards packets:
> > network -> eth0 -> kernel
> >
> > Try traceroute to eth1' ip. Examine arp tables. What MAC is listed there
> > for eth1 IP?
>
> ipforwarding is disabled, arp proxy is disabled.
>
> The mac address is that of eth0 ...

Kernel bug I think. Sorry can't help you here.
I'm afraid you have to dig deeper...
--
vda

Re: BUG: 2 NICs on same network

[Chris Friesen]

Vincent Guffens wrote:
> 
> On Tue, Apr 23, 2002 at 09:10:58AM -0400, Chris Friesen wrote:

> > This is actually standards compliant behaviour, as silly as it sounds.  However,
> > if you want stricter arp behaviour I *think* that the following will fix it.  At
> > least it used to...
> >
> > echo 1 > /proc/sys/net/ipv4/conf/all/arp_filter

> I'm looking at that as well and it doesn't look as it solves the problem, in fact I think that the arp_filter is meant to
> insure that the reply will be sent out to the interface where linux would route it, that is to say, the other way around.

Actually, as long as you have your system set up to route based on source
address (which you probably should if you care about strict arp replies)
arp_filter seems to work properly.  I just tested with a 2.4.18 box, with the
following address configuration:


[root@pcary1k5 /]# ip add
1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100
    link/ether 00:30:65:bf:46:ba brd ff:ff:ff:ff:ff:ff
    inet 47.129.82.58/24 brd 47.129.82.255 scope global eth0
3: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100
    link/ether 00:50:ff:90:04:44 brd ff:ff:ff:ff:ff:ff
    inet 47.129.82.107/24 brd 47.129.82.255 scope global eth1
[root@pcary1k5 /]# ip ru
0:      from all lookup local
32764:  from 47.129.82.107 lookup 101
32765:  from 47.129.82.58 lookup 100
32766:  from all lookup main
32767:  from all lookup 253
[root@pcary1k5 /]# ip ro
47.129.82.0/24 dev eth0  proto kernel  scope link  src 47.129.82.58
47.129.82.0/24 dev eth1  proto kernel  scope link  src 47.129.82.107
127.0.0.0/8 dev lo  scope link
default via 47.129.82.1 dev eth0
[root@pcary1k5 /]# ip ro li table 100
47.129.82.0/24 dev eth0  scope link
default via 47.129.82.1 dev eth0
[root@pcary1k5 /]# ip ro li table 101
47.129.82.0/24 dev eth1  scope link
default via 47.129.82.1 dev eth1


With this setup, arp requests for 47.129.82.58 will only be answered by eth0,
and similarly for 47.129.82.107 and eth1.

Chris

                                                                             
-- 
Chris Friesen                    | MailStop: 043/33/F10  
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: BUG: 2 NICs on same network

[dean gaudet]

On Tue, 23 Apr 2002, Frank Louwers wrote:

> We recently stummed across a rather annoying bug when 2 nics are on
> the same network.

the powers that be consider this a feature, for reasons i've never quite
understood.  your question comes up regularly, and typically the
suggestion is some convoluted filtering thing, and i've never been
sufficiently convinced the filter is worth the overhead.  (especially in
my case when i wanted two interfaces for performance reasons.)

last i investigated this problem the kernel actually sent arp responses
out *both* interfaces, and it was basically a race condition as to which
the recipient would use.

anyhow, below is a patch i created for 2.4.2... hopefully it still works.
apply this and also disable proxy arp in /proc/somewhere.

-dean

--- linux/net/ipv4/arp.c.badproxy	Mon Feb 12 17:28:48 2001
+++ linux/net/ipv4/arp.c	Tue Feb 13 20:06:37 2001
@@ -737,10 +737,12 @@
 		addr_type = rt->rt_type;

 		if (addr_type == RTN_LOCAL) {
+			if ((rt->rt_flags&RTCF_DIRECTSRC) || IN_DEV_PROXY_ARP(in_dev)) {
 			n = neigh_event_ns(&arp_tbl, sha, &sip, dev);
 			if (n) {
 				arp_send(ARPOP_REPLY,ETH_P_ARP,sip,dev,tip,sha,dev->dev_addr,sha);
 				neigh_release(n);
+			}
 			}
 			goto out;
 		} else if (IN_DEV_FORWARD(in_dev)) {

Re: BUG: 2 NICs on same network

[Chris Friesen]

Vincent Guffens wrote:

> ok, I see, so both interfaces are replying to the arp request but this time, the first interface to reply, eth0, will have its
> arp reply filtered by the arp_filter as ip_route_output will indicate eth1 as the outgoing interface for a packet that has its
> source set to the ip to resolve, which has a source route policy entry to point to eth1, is it ?

Correct.  Both NICs receive the arp request, only one of them actually sends a
reply.

> But then, by curiosity, is there some reasons why two different interfaces shouldn't be in the same subnet ? I was in this
> impression having configured some cisco routers which don't even accept this configuration. Is it possible on other network
> devices ? Is there any theoretical justification ?

I don't think there are any theoretical objections.  I'm pretty sure windows
will let you do it, but they don't give you nearly as much control over the
routing.

Without source routing or arp filter, traffic to any of the interfaces on the
same subnet can go through any of the interfaces on that subnet--whichever arp
reply gets there last is the winner.  This can be useful for quasi-redundant
type communications, but causes big problems if you're trying to monitor the
state of your network communications paths.  This is where the more strict
behaviour becomes useful.

Chris


-- 
Chris Friesen                    | MailStop: 043/33/F10  
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: BUG: 2 NICs on same network

[Vincent Guffens]

> 
> Actually, as long as you have your system set up to route based on source
> address (which you probably should if you care about strict arp replies)
> arp_filter seems to work properly.  I just tested with a 2.4.18 box, with the
> following address configuration:
> 

ok, I see, so both interfaces are replying to the arp request but this time, the first interface to reply, eth0, will have its
arp reply filtered by the arp_filter as ip_route_output will indicate eth1 as the outgoing interface for a packet that has its
source set to the ip to resolve, which has a source route policy entry to point to eth1, is it ?

But then, by curiosity, is there some reasons why two different interfaces shouldn't be in the same subnet ? I was in this
impression having configured some cisco routers which don't even accept this configuration. Is it possible on other network
devices ? Is there any theoretical justification ?


thanks for your explanations,

--
			Vincent Guffens

Re: BUG: 2 NICs on same network

[Bill Davidsen]

On Tue, 23 Apr 2002, Frank Louwers wrote:

> We recently stummed across a rather annoying bug when 2 nics are on
> the same network.
> 
> Our situation is this: we have a server with 2 nics, each with a
> different IP on the same network, connected to the same switch. Let's
> assume eth0 has ip 1.2.3.1 and eth1 has 1.2.3.2, with a both with a
> netmask of 255.255.255.0.
> 
> Now the strange thing is that traffic for 1.2.3.2 arrives at eth0 no
> matter what!
> 
> Even if we disconnect the cable for eth1, 1.2.3.2 still replies to
> pings, ssh, web, ...
> 
> We tested this on IA32 architecture, different 2.4.x kernels and
> different nics ...
> 
> Is this a bug or a known issue? If it is not a bug, how can it be
> solved?

Let me say that this is not a kernel question, but a networking
question, there's no bug, you just have several configuration issues.
However, having been beaten to death and the corpe dragged through the
street, hopefully this might help.

The primary issue is that you are responding to arp and sending out the
wrong MAC address. Don't wait for the router to ask, tell it where the
MAC is for that IP.

  ifconfig eth0 10.8.0.5
  ifconfig eth1 10.8.0.6
  route add www.favorite.com dev eth1
  ping -c2 www.favorite.com

The last two lines are the charm... force the route to any host which
goes via the router to use the "other" IP address. Good, now the router
knows that the IP amd MAC belong together. BTW: if you do an IP takeover
(add an alias IP for a server which fails) you also want to do this, or
the router will send packets to the old MAC until the ARP tables
timeout.

This takes care of incoming connections, the traffic should go to the
right NIC and be sent back through the right NIC. To balance outgoing
connections you can try this instead of defining a default route:

  route add -net 0.0.0.0 netmask 0.0.0.1 dev eth0
  route add -net 0.0.0.1 netmask 0.0.0.1 dev eth1

which used to work to send odd IP destinations out one NIC and even out
the other. If that fails and you care, just get iproute2 and claw your
way up the learning curve.

If you just want to use this for backup, when the machine detects that
the NIC has failed (blessedly unusual, I have 15 dual NIC machines in
four timezones) it can just ifconfig the first down, the second up, and
send out a ping, or better yet use one which allows the MAC address to
be set.

Some of this should get you going, or at least looking in the right
place.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: BUG: 2 NICs on same network

[Roland Kuhn]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1178 bytes --]

Hi!

On Tue, 23 Apr 2002, Frank Louwers wrote:

> Hi,
> 
> We recently stummed across a rather annoying bug when 2 nics are on
> the same network.
> 
> Our situation is this: we have a server with 2 nics, each with a
> different IP on the same network, connected to the same switch. Let's
> assume eth0 has ip 1.2.3.1 and eth1 has 1.2.3.2, with a both with a
> netmask of 255.255.255.0.
> 
> Now the strange thing is that traffic for 1.2.3.2 arrives at eth0 no
> matter what!
> 
This is in perfect agreement with rfc1122, section 3.3.4.2. Linux uses the 
weak ES model, so the interface doesn't matter. There was once a patch 
floating around for 2.4.6 that made the kernel accept only arp coming from 
the 'right' interface. arp_self_only, or something. I have attached it, 
but I have no idea if it still applies/works.

Ciao,
					Roland

+---------------------------+-------------------------+
|    TU Muenchen            |                         |
|    Physik-Department E18  |  Raum    3558           |
|    James-Franck-Str.      |  Telefon 089/289-12592  |
|    85747 Garching         |                         |
+---------------------------+-------------------------+

[-- Attachment #2: Type: TEXT/PLAIN, Size: 1718 bytes --]

diff -Naur --exclude=.* --exclude=*.o linux-2.4.6.1/net/ipv4/arp.c linux-2.4.6/net/ipv4/arp.c
--- linux-2.4.6.1/net/ipv4/arp.c  Wed May 16 19:21:45 2001
+++ linux-2.4.6/net/ipv4/arp.c  Thu Aug  9 16:20:57 2001
@@ -760,6 +760,9 @@
                                int dont_send = 0;
                                if (IN_DEV_ARPFILTER(in_dev))
                                  dont_send |= arp_filter(sip,tip,dev);
+                               if (IN_DEV_ARP_SELF_ONLY(in_dev))
+                                 if (dev != ip_dev_find(tip))
+                                       dont_send = 1;
                                if (!dont_send)
                                  arp_send(ARPOP_REPLY,ETH_P_ARP,sip,dev,tip,sha,dev->dev_addr,sha);

diff -Naur --exclude=.* --exclude=*.o linux-2.4.6.1/net/ipv4/devinet.c linux-2.4.6/net/ipv4/devinet.c
--- linux-2.4.6.1/net/ipv4/devinet.c    Wed May 16 19:21:45 2001
+++ linux-2.4.6/net/ipv4/devinet.c      Thu Aug  9 16:20:57 2001
@@ -1016,7 +1016,7 @@
 static struct devinet_sysctl_table
 {
        struct ctl_table_header *sysctl_header;
-       ctl_table devinet_vars[14];
+       ctl_table devinet_vars[15];
        ctl_table devinet_dev[2];
        ctl_table devinet_conf_dir[2];
        ctl_table devinet_proto_dir[2];
@@ -1062,6 +1062,9 @@
        {NET_IPV4_CONF_ARPFILTER, "arp_filter",
         &ipv4_devconf.arp_filter, sizeof(int), 0644, NULL,
         &proc_dointvec},
+        {NET_IPV4_CONF_ARP_SELF_ONLY, "arp_self_only",
+         &ipv4_devconf.arp_self_only, sizeof(int), 0644, NULL,
+         &proc_dointvec},
         {0}},

        {{NET_PROTO_CONF_ALL, "all", NULL, 0, 0555, devinet_sysctl.devinet_vars},{0}},

Re: BUG: 2 NICs on same network

[Dag Bakke]

On Tue, Apr 23, 2002 at 11:39:35AM +0200, Frank Louwers wrote:
> Our situation is this: we have a server with 2 nics, each with a
> different IP on the same network, connected to the same switch. Let's
> assume eth0 has ip 1.2.3.1 and eth1 has 1.2.3.2, with a both with a
> netmask of 255.255.255.0.
>
> Now the strange thing is that traffic for 1.2.3.2 arrives at eth0 no
> matter what!
>
> Even if we disconnect the cable for eth1, 1.2.3.2 still replies to
> pings, ssh, web, ...

Do:
netstat -rn

and you will (should?) understand why.

See:
/usr/src/linux/Documentation/networking/bonding.txt

to (possibly) achieve what you want..

Dag B