RE: [2.4 PATCH] bugfix: ARP respond on all devices

RE: [2.4 PATCH] bugfix: ARP respond on all devices

[Richard Underwood]

David S. Miller wrote:
>
> > _And_ you did not explain so far why these implementations should
> > not be RFC-conform or else illegal.
> 
> Both responding and not responding on all interfaces for ARPs
> is RFC conformant.  This means both Linux and other systems
> are within the rules.
> 
	Firstly, can I point out that you have consistently talked about
REPLIES when everyone else has been talking about REQUESTS. I suspect that
this may be confusing more people than you realise.

	The RFC I quoted (985) says the ARP packets generated by Linux
should be dropped. Sure, the RFC isn't a standard, but there ARE plenty of
implementations that obey it for perfectly valid security reasons.

> Under Linux, by default, IP addresses are owned by the system
> not by interfaces.  This increases the likelyhood of successful
> communication on a subnet.
> 
	This is crap.

	ARP is local to a broadcast net. The ARP standard explicitly
prohibits responding to an ARP request on a different interface.

	If you broadcast a request asking for a reply on an entirely
different subnet, you're asking for trouble. You REDUCE the likelyhood of a
successful ARP reply, not increase it.

	All you can possibly achieve by sending REQUESTS from the wrong IP
number is assist screwed up networks where you've got multiple subnets on
the same copper and cause a shed-load of security issues.

> For scenerios where this doesn't work, we have ways to make the
> kernel behave the way you want it to.
> 
	There are many ways of "fixing" it. I've chosen a static ARP entry
for my next-hop. I really don't care. The issue is that the Linux ARP code
is, apparently by design, flawed.

> There is no discussion about changing the default, because that
> might break things for some people.  So this discussion is pretty
> useless.

	Can you give one good example where this is the case?

	What makes all this worse is that once an ARP request has been
queued using the wrong IP number, further connections that would otherwise
have generated a valid ARP request will be blocked as Linux won't queue a
second request - despite it coming from a different IP number.

	This means that connectivity is non-deterministic, and while
everything may work for 99.9% of the time, when an ARP entry gets deleted
and the next ARP request comes from the wrong IP number you lose
connectivity.

	I wonder how many unsolved random network problems there have been
due to this. "Just reboot it, it'll work again." Great!

	If you insist on leaving the code as it is, at the very least allow
multiple incomplete ARP requests, one per source IP.

	Thanks,

		Richard

RE: [2.4 PATCH] bugfix: ARP respond on all devices

[Alan Cox]

On Maw, 2003-08-19 at 13:02, Richard Underwood wrote:
> 	ARP is local to a broadcast net. The ARP standard explicitly
> prohibits responding to an ARP request on a different interface.

Correct, but we don't do that

> 	If you broadcast a request asking for a reply on an entirely
> different subnet, you're asking for trouble. You REDUCE the likelyhood of a
> successful ARP reply, not increase it.

You increase it and you shortcut on shared lans. Thats really a seperate
issue to the question of which source is used. If you loopback someone
elses address on your own lo device I'm not suprised weird shit happens,
put the alias on eth0 where it belongs.

> 	All you can possibly achieve by sending REQUESTS from the wrong IP
> number is assist screwed up networks where you've got multiple subnets on
> the same copper and cause a shed-load of security issues.

Not in general. If you are using ARP your lan is hardly "secure". For
most situations the trust across multiple aggregated lans is the same,
if it isnt people use vlan (which rarely helps 8))

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[Bas Bloemsaat]

> The RFC I quoted (985) says the ARP packets generated by Linux
> should be dropped. Sure, the RFC isn't a standard, but there ARE plenty of
> implementations that obey it for perfectly valid security reasons.
Same goes for 1180. It it doesn't define a standard either, but makes
perfectly clear that any interface has it's own ARP, not one ARP for the
entire system.

Regards,
Bas

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[David S. Miller]

On Tue, 19 Aug 2003 15:11:59 +0200
"Bas Bloemsaat" <bloemsaa@xs4all.nl> wrote:

> It it doesn't define a standard either, but makes
> perfectly clear that any interface has it's own ARP,
> not one ARP for the entire system.

This is absolutely not true.

There are two valid ways the RFCs allow systems to handle
IP addresses.

1) IP addresses are owned by "the host"
2) IP addresses are owned by "the interface"

Linux does #1, many systems do #2, both are correct.

We provide ways for you to obtain the behavior or #2
so there is nothing to complain about.

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[Stephan von Krawczynski]

On Tue, 19 Aug 2003 15:11:59 +0200
"Bas Bloemsaat" <bloemsaa@xs4all.nl> wrote:

> > The RFC I quoted (985) says the ARP packets generated by Linux
> > should be dropped. Sure, the RFC isn't a standard, but there ARE plenty of
> > implementations that obey it for perfectly valid security reasons.
> Same goes for 1180. It it doesn't define a standard either, but makes
> perfectly clear that any interface has it's own ARP, not one ARP for the
> entire system.

Does "has its own ARP" mean "has its own ARP-table"?
I just want to understand you correctly.

Regards,
Stephan

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[David S. Miller]

On Tue, 19 Aug 2003 15:11:59 +0200
"Bas Bloemsaat" <bloemsaa@xs4all.nl> wrote:

> > The RFC I quoted (985) says the ARP packets generated by Linux
> > should be dropped. Sure, the RFC isn't a standard, but there ARE plenty of
> > implementations that obey it for perfectly valid security reasons.
>
> Same goes for 1180. It it doesn't define a standard either, but makes
> perfectly clear that any interface has it's own ARP, not one ARP for the
> entire system.

Which is all irrelevant because the IPv4 RFCs say that host
based and interface based address ownership are both valid
system models.

Any document saying that they must be per-interface is therefore
invalid.

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[David S. Miller]

On Tue, 19 Aug 2003 13:02:20 +0100
Richard Underwood <richard@aspectgroup.co.uk> wrote:

> David S. Miller wrote:
> > Under Linux, by default, IP addresses are owned by the system
> > not by interfaces.  This increases the likelyhood of successful
> > communication on a subnet.
> > 
> 	This is crap.

Nope, the RFCs allow this.

So this is where we must agree to disagree.  Because host ownership of
IP addresses is the basis for all of the arguments and it completely
justifies Linux's ARP behavior on both sides.

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[Stephan von Krawczynski]

On Tue, 19 Aug 2003 09:54:54 -0700
"David S. Miller" <davem@redhat.com> wrote:

> Which is all irrelevant because the IPv4 RFCs say that host
> based and interface based address ownership are both valid
> system models.

Can you (for completeness) name (the number) or quote the RFC you are talking
about.

Regards,
Stephan

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[David S. Miller]

On Tue, 19 Aug 2003 19:39:20 +0200
Lars Marowsky-Bree <lmb@suse.de> wrote:

> On 2003-08-19T08:34:38,
>    "David S. Miller" <davem@redhat.com> said:
> 
> > There are two valid ways the RFCs allow systems to handle
> > IP addresses.
> > 
> > 1) IP addresses are owned by "the host"
> > 2) IP addresses are owned by "the interface"
> > 
> > Linux does #1, many systems do #2, both are correct.
> 
> Yes, both are "correct" in the sense that the RFC allows this
> interpretation. The _sensible_ interpretation for practical networking
> however is #2, and the only persons who seem to believe differently are
> those in charge of the Linux network code...

And, as Alan said, we provide a way for one to obtain your networking
religion of week.

Changing the default is not an option, that would undoubtedly
break things.

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[Lars Marowsky-Bree]

On 2003-08-19T08:34:38,
   "David S. Miller" <davem@redhat.com> said:

> There are two valid ways the RFCs allow systems to handle
> IP addresses.
> 
> 1) IP addresses are owned by "the host"
> 2) IP addresses are owned by "the interface"
> 
> Linux does #1, many systems do #2, both are correct.

Yes, both are "correct" in the sense that the RFC allows this
interpretation. The _sensible_ interpretation for practical networking
however is #2, and the only persons who seem to believe differently are
those in charge of the Linux network code...


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
High Availability & Clustering		ever tried. ever failed. no matter.
SuSE Labs				try again. fail again. fail better.
Research & Development, SuSE Linux AG		-- Samuel Beckett

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[David S. Miller]

On 19 Aug 2003 14:30:26 -0400
Daniel Gryniewicz <dang@fprintf.net> wrote:

> If you are not on a shared lan, then it will *ONLY* work if linux is
> on the other end.  No other system will work.

And these other systems are broken.  (actually, older Cisco equipment
correctly responds to the ARP regardless of source IP)

Just because some Cisco engineer says that it is correct doesn't
make it is.

Consider the situation logically.  When you're replying to an
ARP, _HOW_ do you know what IP addresses are assigned to _MY_
outgoing interfaces and _HOW_ do you know what subnets _EXIST_
on the LAN?

The answer to both is, you'd don't know these things _EVEN_ if
you are a router/gateway.

Therefore there is no valid reason not to respond to an ARP using one
source address as opposed to another.

RE: [2.4 PATCH] bugfix: ARP respond on all devices

[Daniel Gryniewicz]

On Tue, 2003-08-19 at 08:35, Alan Cox wrote:
> You increase it and you shortcut on shared lans. Thats really a seperate
> issue to the question of which source is used. If you loopback someone
> elses address on your own lo device I'm not suprised weird shit happens,
> put the alias on eth0 where it belongs.

Only if you are on a shared lan.  If you are not on a shared lan, then
it will *ONLY* work if linux is on the other end.  No other system will
work.  And, you don't need an alias on loopback.  Merely changing the
default route will result in this.  Change default route from gw 1.1.1.1
on eth0 to gw 2.2.2.2 on eth1 (making sure that 2.2.2.2 doesn't have an
arp entry), and linux will say, on eth1:
whohas 2.2.2.2 tell 1.1.1.2
where 1.1.1.2 is the address on eth0.  No one will respond to this, so
all communication from beyond a directly connected network will now
fail.

-- 
Daniel Gryniewicz <dang@fprintf.net>

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[David S. Miller]

On 19 Aug 2003 15:12:43 -0400
Daniel Gryniewicz <dang@fprintf.net> wrote:

> (And it's not just cisco, all the *BSDs (and therefore anything
> that took the BSD stack such as MS) work this way too.)

Not true.  Microsoft systems do respond properly to these ARPs.

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[Daniel Gryniewicz]

I realize that, but is that a reason to keep linux from working with
these?  (And it's not just cisco, all the *BSDs (and therefore anything
that took the BSD stack such as MS) work this way too.)  As nearly as I
can tell, there's no way to make linux work with these, and the
situation I gave is one where linux could easily get it right, and
isn't.  Saying "They're broken.  Tough." is not really helpful.  At
least can we get a "work with broken other systems in certain
circumstances" switch somewhere?

(Funny you should mention Cisco, as we write routing software and must
interoperate with Cisco.  What Cisco says *does* go in the routing
community, if you wish your product to be used.  Currently, when our
customers come to us, we have to say "Either don't use Linux or run 2.2
with the arp fix patch.")

Daniel

On Tue, 2003-08-19 at 14:29, David S. Miller wrote:
> On 19 Aug 2003 14:30:26 -0400
> Daniel Gryniewicz <dang@fprintf.net> wrote:
> 
> > If you are not on a shared lan, then it will *ONLY* work if linux is
> > on the other end.  No other system will work.
> 
> And these other systems are broken.  (actually, older Cisco equipment
> correctly responds to the ARP regardless of source IP)
> 
> Just because some Cisco engineer says that it is correct doesn't
> make it is.
> 
> Consider the situation logically.  When you're replying to an
> ARP, _HOW_ do you know what IP addresses are assigned to _MY_
> outgoing interfaces and _HOW_ do you know what subnets _EXIST_
> on the LAN?
> 
> The answer to both is, you'd don't know these things _EVEN_ if
> you are a router/gateway.
> 
> Therefore there is no valid reason not to respond to an ARP using one
> source address as opposed to another.
-- 
Daniel Gryniewicz <dang@fprintf.net>

Discussion fucking closed WAS(Re: [2.4 PATCH] bugfix: ARP respond on all devices

[jamal]

Ok, go ahead and call my mama - tell her i used a bad word on a mailing
list.
Folks, this is a actually a republic model: in other words its a
democracy upto a certain level then there maybe a veto. 
You have been provided a facility to go and do what funky thing that
pleases you. Use ARPTABLEs; if you dont like it maintain your own
patches - you have that freedom; dont enforce your freedom on someone
actually doing the maintanace work - they have more important things to
worry about.

You can quote all the RFCs you want - it wont change anything soon. I
got tired of following same cyclic arguements. What Linux is doing is
conformant. What other people following CISCO are doing is also
conformant. RFCs are written in an ambigous language called english.
People actually (lately anyways) sneak in ambiguity to make their
implementation look correct. So please stop quoting stoopid RFCs. 

cheers,
jamal

On Tue, 2003-08-19 at 14:29, David S. Miller wrote:
> On 19 Aug 2003 14:30:26 -0400
> Daniel Gryniewicz <dang@fprintf.net> wrote:
> 
> > If you are not on a shared lan, then it will *ONLY* work if linux is
> > on the other end.  No other system will work.
> 
> And these other systems are broken.  (actually, older Cisco equipment
> correctly responds to the ARP regardless of source IP)
> 
> Just because some Cisco engineer says that it is correct doesn't
> make it is.
> 
> Consider the situation logically.  When you're replying to an
> ARP, _HOW_ do you know what IP addresses are assigned to _MY_
> outgoing interfaces and _HOW_ do you know what subnets _EXIST_
> on the LAN?
> 
> The answer to both is, you'd don't know these things _EVEN_ if
> you are a router/gateway.
> 
> Therefore there is no valid reason not to respond to an ARP using one
> source address as opposed to another.
> 
> 

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[bill davidsen]

In article <20030819112912.359eaea6.davem@redhat.com>,
David S. Miller <davem@redhat.com> wrote:
| On 19 Aug 2003 14:30:26 -0400
| Daniel Gryniewicz <dang@fprintf.net> wrote:
| 
| > If you are not on a shared lan, then it will *ONLY* work if linux is
| > on the other end.  No other system will work.
| 
| And these other systems are broken.  (actually, older Cisco equipment
| correctly responds to the ARP regardless of source IP)
| 
| Just because some Cisco engineer says that it is correct doesn't
| make it is.

What you say is true, but in the real world being able to work with the
most commonly used network hardware is a hard requirement. When
"conforms to RFC" colides with "works" there's an issue, particularly
when the RFC allows several behaviours (bad RFC!).

I would hope that it is possible to get a single flag to force all
packets out of a NIC configured with the source IP in the packet, I
don't feel the need to make that the default behaviour, just to find
some alternative to patches or source routing.
-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[Harley Stenzel]

David S. Miller wrote:
> 
> And, as Alan said, we provide a way for one to obtain your networking
> religion of week.

To the best of my knowledge, there is presently no way to change the arp 
behavior of Linux such that it uses the interface-based arp mechanizm in 
  a manner compatable with load-balancing and hot-standby techniques 
involving aliasing the loopback interface.  In all the proposed 
solutions the cache-update by an arp request problem still exists (arp 
source ip problem).

I would love to be proven wrong.  Presently I have to either patch the 
kernel or suffer the throughput penalty of doing dnat to myself, all to 
do something that Solaris, AIX, HP-UX, *BSD, and even Windows can do 
natively.

--Harley

host vs interface address ownership [Re: [2.4 PATCH] bugfix: ARP respond on all devices]

[Pekka Savola]

On Tue, 19 Aug 2003, David S. Miller wrote:

> On Tue, 19 Aug 2003 13:02:20 +0100
> Richard Underwood <richard@aspectgroup.co.uk> wrote:
> 
> > David S. Miller wrote:
> > > Under Linux, by default, IP addresses are owned by the system
> > > not by interfaces.  This increases the likelyhood of successful
> > > communication on a subnet.
> > > 
> > 	This is crap.
> 
> Nope, the RFCs allow this.
> 
> So this is where we must agree to disagree.  Because host ownership of
> IP addresses is the basis for all of the arguments and it completely
> justifies Linux's ARP behavior on both sides.

Maybe I'm missing something -- I'm not sure what exactly you're including
in the models -- but wouldn't it be possible to implement the "host 
ownership" model so that it would STILL honor any RFC out there (and 
similarly for "interface ownership")?

For example, many IETF documents may state things like:

                                                                     The
   Home Agents List MAY be implemented in any manner consistent with the
   external behavior described in this document.

.. which *seems* (without knowing which RFCs and sections of them you 
refer to for justifying host/interface ownership) to be a probable intent 
of allowing either model.  Just as long as the external behaviour is 
consistent, you can implement it with any internal structure you wish.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

ARP and knowledge of IP addresses [Re: [2.4 PATCH] bugfix: ARP respond on all devices]

[Pekka Savola]

On Tue, 19 Aug 2003, David S. Miller wrote:
[...]
> Consider the situation logically.  When you're replying to an
> ARP, _HOW_ do you know what IP addresses are assigned to _MY_
> outgoing interfaces and _HOW_ do you know what subnets _EXIST_
> on the LAN?
> 
> The answer to both is, you'd don't know these things _EVEN_ if
> you are a router/gateway.

Maybe I'm missing something but -- isn't it perfectly valid to assume the
node *ITSELF* knows about its interfaces, IP addresses, and its routes?
(Certainly, it can't know of what subnets exist on the LAN if those
haven't been configured on the node, e.g. in the form of an "interface
routes".)

ARP could look it up.

Sure, it would seem a bit like an OSI layering violation, but that's no 
news as OSI layering isn't strict anyway and has been shredded to pieces 
already in many other places.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

Re: host vs interface address ownership [Re: [2.4 PATCH] bugfix: ARP respond on all devices]

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 788 bytes --]

On Wed, 20 Aug 2003 08:18:09 +0300, Pekka Savola said:

> .. which *seems* (without knowing which RFCs and sections of them you 
> refer to for justifying host/interface ownership) to be a probable intent 
> of allowing either model.  Just as long as the external behaviour is 
> consistent, you can implement it with any internal structure you wish.

Hmm.. depends what you mean by "consistent"... "you can implement the
insides either way as long as the outsides look the same", or "you can do it
either way, as long as you do it the same way throughout the network".

The problem is that the *externally visible* behavior is different depending on
whether you choose host or interface ownership, and either model can probably
be made to work if *the rest of the network plays along*.


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[Bill Davidsen]

On 19 Aug 2003, Daniel Gryniewicz wrote:

> I realize that, but is that a reason to keep linux from working with
> these?  (And it's not just cisco, all the *BSDs (and therefore anything
> that took the BSD stack such as MS) work this way too.)  As nearly as I
> can tell, there's no way to make linux work with these, and the
> situation I gave is one where linux could easily get it right, and
> isn't.  Saying "They're broken.  Tough." is not really helpful.  At
> least can we get a "work with broken other systems in certain
> circumstances" switch somewhere?

I have been asking for a similar thing as well, David mentioned some
things that would break, but I believe they break if you use source
routing, so that seems not to be a real objection.

> 
> (Funny you should mention Cisco, as we write routing software and must
> interoperate with Cisco.  What Cisco says *does* go in the routing
> community, if you wish your product to be used.  Currently, when our
> customers come to us, we have to say "Either don't use Linux or run 2.2
> with the arp fix patch.")

Unless all your customers do odd things with networking, or you are not
using source routing for some reason, you don't do customers a favor by
giving that advice. Most users have one NIC with one IP and Linux works.

The Linux implementation is not broken by standard, it's just that there
were two ways to do it, and the one chosen is allowed by frequently
non-functional.

I find it interesting that we can't change networking because a few
complex systems would have to be reconfigured, but we *can* change modules
which requires config changes on probably 90% of all systems (commercial
distributions). Linus has rethought some of his design decisions, but
David seems intent on not only keeping this one, but preventing the
addition of a flag which would solve the problem for most people.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[David S. Miller]

On Wed, 20 Aug 2003 12:49:14 -0400 (EDT)
Bill Davidsen <davidsen@tmr.com> wrote:

> On 19 Aug 2003, Daniel Gryniewicz wrote:
> 
> I have been asking for a similar thing as well, David mentioned some
> things that would break, but I believe they break if you use source
> routing, so that seems not to be a real objection.

It's not about source routing.  It's about failover and being
able to use ARP on interfaces which don't have addresses assigned
to them yet.

> I find it interesting that we can't change networking because a few
> complex systems would have to be reconfigured, but we *can* change modules
> which requires config changes on probably 90% of all systems (commercial
> distributions).

Decisions about Networking will always be in a different domain
because the way one behaves has effects upon other systems not
just the local one.

BTW, another thing which makes the source address selection for
outgoing ARPs a real touchy area is the following.  Some weird
configurations actually respond with different ARP answers based upon
the source address in the ARP request.  You can ask Julian Anastasov
about such (arguably pathological) setups.

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[Ben Greear]

David S. Miller wrote:
> On Wed, 20 Aug 2003 12:49:14 -0400 (EDT)
> Bill Davidsen <davidsen@tmr.com> wrote:
> 
> 
>>On 19 Aug 2003, Daniel Gryniewicz wrote:
>>
>>I have been asking for a similar thing as well, David mentioned some
>>things that would break, but I believe they break if you use source
>>routing, so that seems not to be a real objection.
> 
> 
> It's not about source routing.  It's about failover and being
> able to use ARP on interfaces which don't have addresses assigned
> to them yet.

[snip]

> BTW, another thing which makes the source address selection for
> outgoing ARPs a real touchy area is the following.  Some weird
> configurations actually respond with different ARP answers based upon
> the source address in the ARP request.  You can ask Julian Anastasov
> about such (arguably pathological) setups.

It seems that these reasons would not preclude the addition of a flag
that would default to the current behaviour but allow the behaviour that
other setups desire easily?  That seems to be all that folks are really
arguing for.  If/when the user enabled this new flag, then they should
be fully responsible for the change in behaviour, and they can deal with
it as needed.

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[David S. Miller]

On Wed, 20 Aug 2003 10:44:41 -0700
Ben Greear <greearb@candelatech.com> wrote:

> It seems that these reasons would not preclude the addition of a flag
> that would default to the current behaviour but allow the behaviour that
> other setups desire easily?

I would accept a patch that did something like
the following in arp_solicit().

	if (skb && inet_addr_type(skb->nh.iph->saddr) == RTN_LOCAL &&
	    (in_dev->conf.shared_media ||
	     inet_addr_onlink(dev, skb->nh.iph->saddr, 0)))
		saddr = skb->nh.iph->saddr;
	else
		saddr = inet_select_addr(dev, target, RT_SCOPE_LINE);

Then people can frob the shared_media sysctl for devices
where they want the behavior to be that we will only use
addresses assigned to the device as the solicitor address.

The shared_media setting defaults to one and thus would preserve
current behavior by default.

The idea is not mine, Alexey suggested it to me the other day.

I hope this pleases people wrt. ARP request solicitor address
handling.

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[Bill Davidsen]

On Wed, 20 Aug 2003, David S. Miller wrote:

> On Wed, 20 Aug 2003 12:49:14 -0400 (EDT)
> Bill Davidsen <davidsen@tmr.com> wrote:
> 
> > On 19 Aug 2003, Daniel Gryniewicz wrote:
> > 
> > I have been asking for a similar thing as well, David mentioned some
> > things that would break, but I believe they break if you use source
> > routing, so that seems not to be a real objection.
> 
> It's not about source routing.  It's about failover and being
> able to use ARP on interfaces which don't have addresses assigned
> to them yet.

David mentioned that you could solve the problem by using *rp_filter and
source routing. I don't think that's entirely true, but doing so has the
same drawbacks and breaks the same things as a flag to make Linux behave
like Sun/BSD/Windows (and work with Cisco is the cases previously
mentioned).

> 
> > I find it interesting that we can't change networking because a few
> > complex systems would have to be reconfigured, but we *can* change modules
> > which requires config changes on probably 90% of all systems (commercial
> > distributions).
> 
> Decisions about Networking will always be in a different domain
> because the way one behaves has effects upon other systems not
> just the local one.

Yes, that's exactly the point, the way Linux works has bad effects on
certain other machines, as in leaves them disconnected to the Linux
system.

> 
> BTW, another thing which makes the source address selection for
> outgoing ARPs a real touchy area is the following.  Some weird
> configurations actually respond with different ARP answers based upon
> the source address in the ARP request.  You can ask Julian Anastasov
> about such (arguably pathological) setups.
> 

I don't think anyone is asking for a change in the default behaviour
(although my point about breaking modules does apply), people would be
satisfied, even ecstatic, if we had a simple way (flag) to set to make
Linux work without setting /proc filters, using arpfilter, applying source
routes (David's suggestion) and generally jumping through hoops.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[Bas Bloemsaat]

> I don't think anyone is asking for a change in the default behaviour
> (although my point about breaking modules does apply), people would be
> satisfied, even ecstatic, if we had a simple way (flag) to set to make
> Linux work without setting /proc filters, using arpfilter, applying source
> routes (David's suggestion) and generally jumping through hoops.
Agree! Just a flag (ARP_ISOLATED, default to 0) in
/proc/sys/net/ipv4/conf/*? The default behaviour of the current (and future
kernels) stays as it is now, so it doesn't break for anyone, and a lot of
people (including me :) benefit from a much easier setup.

No need to implement a whole hidden scenario either.

Regards,
Bas

[RFC][2.4 PATCH] source address selection for ARP requests

[Willy Tarreau]

Hi David,

On Wed, Aug 20, 2003 at 10:48:31AM -0700, David S. Miller wrote:
> On Wed, 20 Aug 2003 10:44:41 -0700
> Ben Greear <greearb@candelatech.com> wrote:
> 
> > It seems that these reasons would not preclude the addition of a flag
> > that would default to the current behaviour but allow the behaviour that
> > other setups desire easily?
> 
> I would accept a patch that did something like
> the following in arp_solicit().
> 
> 	if (skb && inet_addr_type(skb->nh.iph->saddr) == RTN_LOCAL &&
> 	    (in_dev->conf.shared_media ||
> 	     inet_addr_onlink(dev, skb->nh.iph->saddr, 0)))
> 		saddr = skb->nh.iph->saddr;
> 	else
> 		saddr = inet_select_addr(dev, target, RT_SCOPE_LINE);
> 

I finally found some time to code and test both Alexey's idea and the one I
derived from it the other day.

1) Alexey's solution (above, patch below)

It solves most issues discussed previously, which showed up on somewhat common
setups like this one :

                Server                        Gateway
    eth0=10.0.0.1   eth1=11.0.0.1 --------- IP=11.0.0.2

When server receives a ping to 10.0.0.1 from Gateway or some host behind it, it
will now properly select 11.0.0.1 for the ARP request prior to sending its
echo reply. This behaviour requires the user to explicitly set
eth1/shared_media and all/shared_media to 0 (not too hard).

=> So the patch below fixes most problems.

-8<--------------------------

--- linux-2.4.22-rc2/net/ipv4/arp.c	Wed Jul 30 09:19:06 2003
+++ linux-2.4.22-rc2-arp/net/ipv4/arp.c	Wed Aug 20 21:19:42 2003
@@ -320,13 +320,22 @@
 	u32 saddr;
 	u8  *dst_ha = NULL;
 	struct net_device *dev = neigh->dev;
+	struct in_device *in_dev = in_dev_get(dev);
 	u32 target = *(u32*)neigh->primary_key;
 	int probes = atomic_read(&neigh->probes);
 
-	if (skb && inet_addr_type(skb->nh.iph->saddr) == RTN_LOCAL)
+	if (in_dev == NULL)
+		return;
+
+	if (in_dev->ifa_list == NULL ||
+	    (skb && inet_addr_type(skb->nh.iph->saddr) == RTN_LOCAL &&
+	    (IN_DEV_SHARED_MEDIA(in_dev) ||
+	    inet_addr_onlink(in_dev, skb->nh.iph->saddr, 0))))
 		saddr = skb->nh.iph->saddr;
 	else
 		saddr = inet_select_addr(dev, target, RT_SCOPE_LINK);
+
+	in_dev_put(in_dev);
 
 	if ((probes -= neigh->parms->ucast_probes) < 0) {
 		if (!(neigh->nud_state&NUD_VALID))

-8<--------------------------

However, there still is a case which is not covered : when the source address
is itself on the same interface. Let's take the previous example and add an
alias to eth1 :

                Server                        Gateway
    eth0=10.0.0.1   eth1=11.0.0.1 --------- IP=11.0.0.2
                         12.0.0.1

If gateway pings 12.0.0.1, Server will use this address to send its ARP
requests (because of the 'inet_addr_onlink' above). The workaround would
simply be to move the alias somewhere else...

Second, you were concerned about breaking setups with no IP address on eth1
because inet_addr_onlink() will return 0,  and inet_select_addr() will fail,
in the event they would run with shared_media=0 :

                Server                        Gateway
    eth0=10.0.0.1   eth1=*no IP* ---------- IP=11.0.0.2

In fact, inet_select_addr() is smarter than inet_addr_onlink() in that it can
also return non-loopback addresses set to the loopback interface. Moreover, if
it fails, it returns 0, which is a good test to drop back to the current
behaviour (use skb->nh.iph->saddr). I didn't manage to get my interface to
send ARP requests when I have no IP address on it, because I don't know how
to do, since I cannot add a route to it. I presume I could make it work with
SO_BINDTODEVICE + MSG_DONTROUTE, but I don't have time to try all this.

So please look at this code now :

-8<-------------------------------

diff -urN linux-2.4.22-rc2/net/ipv4/arp.c linux-2.4.22-rc2-arp3/net/ipv4/arp.c
--- linux-2.4.22-rc2/net/ipv4/arp.c	Wed Jul 30 09:19:06 2003
+++ linux-2.4.22-rc2-arp3/net/ipv4/arp.c	Wed Aug 20 23:11:53 2003
@@ -320,13 +320,21 @@
 	u32 saddr;
 	u8  *dst_ha = NULL;
 	struct net_device *dev = neigh->dev;
+	struct in_device *in_dev = in_dev_get(dev);
 	u32 target = *(u32*)neigh->primary_key;
 	int probes = atomic_read(&neigh->probes);
 
-	if (skb && inet_addr_type(skb->nh.iph->saddr) == RTN_LOCAL)
-		saddr = skb->nh.iph->saddr;
+	if (in_dev == NULL)
+		return;
+
+	if (skb && inet_addr_type(skb->nh.iph->saddr) == RTN_LOCAL &&
+	    (IN_DEV_SHARED_MEDIA(in_dev) ||
+	     (saddr = inet_select_addr(dev, target, RT_SCOPE_LINK)) == 0))
+			saddr = skb->nh.iph->saddr;
 	else
 		saddr = inet_select_addr(dev, target, RT_SCOPE_LINK);
+
+	in_dev_put(in_dev);
 
 	if ((probes -= neigh->parms->ucast_probes) < 0) {
 		if (!(neigh->nud_state&NUD_VALID))


-8<-------------------------------

It will correctly pick the most appropriate address when shared_media=0, and
will also drop back to the current behaviour when there's no IP yet on the
device.

It also enhances an interesting point compared to the previous one : it allows
broken setups such as the one below to select the valid source IP depending on
source route :


                Server                        Gateway
    eth0=11.0.0.1   eth1=10.0.0.1 --------- IP=11.0.0.2

    ip addr  add 11.0.0.1/N dev eth0
    ip addr  add 10.0.0.1/M dev eth1 scope host
    ip route add 11.0.0.2 dev eth1 src 11.0.0.1

=> ARP requests sent to 11.0.0.2 "correctly" use 11.0.0.1 as the source IP.

I'm not sure this setup will concern anyone, but I came across it during my
tests of the patch, and thought that for evry setup which people will be able
to configure themselves without patching, there will be less whiners ;-)

I'd sincerely prefer the later patch to be tested and integrated, but Alexey's
first idea was the former, so I don't know which one you'll pick.

Of course, if you're willing to apply one of them, I'll try to port them to
2.6.

Cheers,
Willy

Re: [RFC][2.4 PATCH] source address selection for ARP requests

[David S. Miller]

On Wed, 20 Aug 2003 23:34:43 +0200
Willy Tarreau <willy@w.ods.org> wrote:

> +	if (in_dev == NULL)
> +		return;

A NULL in_dev is what you'll see if no addresses are
assigned to the interface, therefore you must treat
this case similarly.

Re: [RFC][2.4 PATCH] source address selection for ARP requests

[Willy Tarreau]

On Wed, Aug 20, 2003 at 02:47:11PM -0700, David S. Miller wrote:
> On Wed, 20 Aug 2003 23:34:43 +0200
> Willy Tarreau <willy@w.ods.org> wrote:
> 
> > +	if (in_dev == NULL)
> > +		return;
> 
> A NULL in_dev is what you'll see if no addresses are
> assigned to the interface, therefore you must treat
> this case similarly.

OK, that's fine. It is what I wanted to test with in_dev->ifa_list==NULL. I
don't know if I must keep the test or not (anyway, it doesn't hurt, this isn't
a fast path).

Better this way ?

Cheers,
Willy

--------

patch 1 :

diff -urN linux-2.4.22-rc2/net/ipv4/arp.c linux-2.4.22-rc2-arp/net/ipv4/arp.c
--- linux-2.4.22-rc2/net/ipv4/arp.c	Fri Aug  1 23:06:29 2003
+++ linux-2.4.22-rc2-arp/net/ipv4/arp.c	Thu Aug 21 00:20:19 2003
@@ -320,13 +320,20 @@
 	u32 saddr;
 	u8  *dst_ha = NULL;
 	struct net_device *dev = neigh->dev;
+	struct in_device *in_dev = in_dev_get(dev);
 	u32 target = *(u32*)neigh->primary_key;
 	int probes = atomic_read(&neigh->probes);
 
-	if (skb && inet_addr_type(skb->nh.iph->saddr) == RTN_LOCAL)
+	if (in_dev == NULL || in_dev->ifa_list == NULL ||
+	    (skb && inet_addr_type(skb->nh.iph->saddr) == RTN_LOCAL &&
+	    (IN_DEV_SHARED_MEDIA(in_dev) ||
+	    inet_addr_onlink(in_dev, skb->nh.iph->saddr, 0))))
 		saddr = skb->nh.iph->saddr;
 	else
 		saddr = inet_select_addr(dev, target, RT_SCOPE_LINK);
+
+	if (in_dev == NULL)
+		in_dev_put(in_dev);
 
 	if ((probes -= neigh->parms->ucast_probes) < 0) {
 		if (!(neigh->nud_state&NUD_VALID))

---------

patch 2 :


diff -urN linux-2.4.22-rc2/net/ipv4/arp.c linux-2.4.22-rc2-arp2/net/ipv4/arp.c
--- linux-2.4.22-rc2/net/ipv4/arp.c	Fri Aug  1 23:06:29 2003
+++ linux-2.4.22-rc2-arp2/net/ipv4/arp.c	Thu Aug 21 00:24:25 2003
@@ -320,13 +320,19 @@
 	u32 saddr;
 	u8  *dst_ha = NULL;
 	struct net_device *dev = neigh->dev;
+	struct in_device *in_dev = in_dev_get(dev);
 	u32 target = *(u32*)neigh->primary_key;
 	int probes = atomic_read(&neigh->probes);
 
-	if (skb && inet_addr_type(skb->nh.iph->saddr) == RTN_LOCAL)
-		saddr = skb->nh.iph->saddr;
+	if (skb && inet_addr_type(skb->nh.iph->saddr) == RTN_LOCAL &&
+	    (in_dev == NULL || IN_DEV_SHARED_MEDIA(in_dev) ||
+	     (saddr = inet_select_addr(dev, target, RT_SCOPE_LINK)) == 0))
+			saddr = skb->nh.iph->saddr;
 	else
 		saddr = inet_select_addr(dev, target, RT_SCOPE_LINK);
+
+	if (in_dev == NULL)
+		in_dev_put(in_dev);
 
 	if ((probes -= neigh->parms->ucast_probes) < 0) {
 		if (!(neigh->nud_state&NUD_VALID))

Re: [RFC][2.4 PATCH] source address selection for ARP requests

[David S. Miller]

On Thu, 21 Aug 2003 00:27:10 +0200
Willy Tarreau <willy@w.ods.org> wrote:

> +	if (in_dev == NULL)
> +		in_dev_put(in_dev);

What the heck is this? :-)

Re: [RFC][2.4 PATCH] source address selection for ARP requests

[Willy Tarreau]

On Wed, Aug 20, 2003 at 03:35:48PM -0700, David S. Miller wrote:
> On Thu, 21 Aug 2003 00:27:10 +0200
> Willy Tarreau <willy@w.ods.org> wrote:
> 
> > +	if (in_dev == NULL)
> > +		in_dev_put(in_dev);
> 
> What the heck is this? :-)

Hmmm... don't you see ? A copy-paste bug, of course ! Sorry for this, I hope
noone's fool enough to use it as-is. I guess it's becoming late, and I'm a bit
weak in front of my temptations to use 'dd' .... 'P' in vi !

Here's the fixed one (hand-edited, not tested). At least, I'm happy to see that
I'm not the only one reading other's patches :-)

BTW, you didn't tell me if it's necessary to test in_dev->ifa_list. I see that
I kept the test in patch 1 but removed it in patch 2.

Willy

patch 1 :

diff -urN linux-2.4.22-rc2/net/ipv4/arp.c linux-2.4.22-rc2-arp/net/ipv4/arp.c
--- linux-2.4.22-rc2/net/ipv4/arp.c	Fri Aug  1 23:06:29 2003
+++ linux-2.4.22-rc2-arp/net/ipv4/arp.c	Thu Aug 21 00:20:19 2003
@@ -320,13 +320,20 @@
 	u32 saddr;
 	u8  *dst_ha = NULL;
 	struct net_device *dev = neigh->dev;
+	struct in_device *in_dev = in_dev_get(dev);
 	u32 target = *(u32*)neigh->primary_key;
 	int probes = atomic_read(&neigh->probes);
 
-	if (skb && inet_addr_type(skb->nh.iph->saddr) == RTN_LOCAL)
+	if (in_dev == NULL || in_dev->ifa_list == NULL ||
+	    (skb && inet_addr_type(skb->nh.iph->saddr) == RTN_LOCAL &&
+	    (IN_DEV_SHARED_MEDIA(in_dev) ||
+	    inet_addr_onlink(in_dev, skb->nh.iph->saddr, 0))))
 		saddr = skb->nh.iph->saddr;
 	else
 		saddr = inet_select_addr(dev, target, RT_SCOPE_LINK);
+
+	if (in_dev != NULL)
+		in_dev_put(in_dev);
 
 	if ((probes -= neigh->parms->ucast_probes) < 0) {
 		if (!(neigh->nud_state&NUD_VALID))


patch 2 :

diff -urN linux-2.4.22-rc2/net/ipv4/arp.c linux-2.4.22-rc2-arp2/net/ipv4/arp.c
--- linux-2.4.22-rc2/net/ipv4/arp.c	Fri Aug  1 23:06:29 2003
+++ linux-2.4.22-rc2-arp2/net/ipv4/arp.c	Thu Aug 21 00:24:25 2003
@@ -320,13 +320,19 @@
 	u32 saddr;
 	u8  *dst_ha = NULL;
 	struct net_device *dev = neigh->dev;
+	struct in_device *in_dev = in_dev_get(dev);
 	u32 target = *(u32*)neigh->primary_key;
 	int probes = atomic_read(&neigh->probes);
 
-	if (skb && inet_addr_type(skb->nh.iph->saddr) == RTN_LOCAL)
-		saddr = skb->nh.iph->saddr;
+	if (skb && inet_addr_type(skb->nh.iph->saddr) == RTN_LOCAL &&
+	    (in_dev == NULL || IN_DEV_SHARED_MEDIA(in_dev) ||
+	     (saddr = inet_select_addr(dev, target, RT_SCOPE_LINK)) == 0))
+			saddr = skb->nh.iph->saddr;
 	else
 		saddr = inet_select_addr(dev, target, RT_SCOPE_LINK);
+
+	if (in_dev != NULL)
+		in_dev_put(in_dev);
 
 	if ((probes -= neigh->parms->ucast_probes) < 0) {
 		if (!(neigh->nud_state&NUD_VALID))

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[Julian Anastasov]

	Hello,

On Wed, 20 Aug 2003, David S. Miller wrote:

> Then people can frob the shared_media sysctl for devices
> where they want the behavior to be that we will only use
> addresses assigned to the device as the solicitor address.
>
> The shared_media setting defaults to one and thus would preserve
> current behavior by default.

	Hm, you are trying to save one additional flag :)

> The idea is not mine, Alexey suggested it to me the other day.
>
> I hope this pleases people wrt. ARP request solicitor address
> handling.

	More ideas/issues:

- can we add medium_id checks near the inet_addr_onlink check,
i.e. to extend the check for same medium, not only for same outdev,
eg. saddr and outdev can be from same medium. That means we
have to use ip_dev_find as replacement for inet_addr_type and
inet_addr_onlink

- fib_validate_source already drops packets from device with
in_dev==NULL, there is no good reason to send ARP requests.
Even ip_route_output_slow disallows such devices. That is, IP
is disabled, so and ARP.

- in the last days/weeks I have a doubt how asymmetric routing
can safely work with the default behaviour. The arp_queue can
contain packets with saddrs from different interfaces and subnets,
all to the same resolved target IP. I'm not sure the remote system
can properly answer to our requests in this case, at least, can
not do it properly without rp_filter_mask [1] if there are 2 or
more interfaces. Of course, the problem is when rp_filter is used
there.

- Broadcasting announcements for one saddr through many devices
can create problems for us if we later receive traffic for this
saddr when rp_filter=1 and there is no rp_filter_mask set. Can
someone provide example setup for asymmetric routing that relies
on the current behavior, I'll be glad to think on it.

- can we swicth to safe behavior according to the probe number?
For example, after the 1st ucast or bcast probe we can switch
to source auto selection?

[1] http://www.ssi.bg/~ja/#rp_filter_mask

Regards

--
Julian Anastasov <ja@ssi.bg>

Re: [2.4 PATCH] bugfix: ARP respond on all devices

[Bill Davidsen]

On Wed, 20 Aug 2003, David S. Miller wrote:

> On Wed, 20 Aug 2003 10:44:41 -0700
> Ben Greear <greearb@candelatech.com> wrote:
> 
> > It seems that these reasons would not preclude the addition of a flag
> > that would default to the current behaviour but allow the behaviour that
> > other setups desire easily?
> 
> I would accept a patch that did something like
> the following in arp_solicit().
> 
> 	if (skb && inet_addr_type(skb->nh.iph->saddr) == RTN_LOCAL &&
> 	    (in_dev->conf.shared_media ||
> 	     inet_addr_onlink(dev, skb->nh.iph->saddr, 0)))
> 		saddr = skb->nh.iph->saddr;
> 	else
> 		saddr = inet_select_addr(dev, target, RT_SCOPE_LINE);
> 
> Then people can frob the shared_media sysctl for devices
> where they want the behavior to be that we will only use
> addresses assigned to the device as the solicitor address.
> 
> The shared_media setting defaults to one and thus would preserve
> current behavior by default.
> 
> The idea is not mine, Alexey suggested it to me the other day.
> 
> I hope this pleases people wrt. ARP request solicitor address
> handling.

I'm not sure if you changed your mind or someone finally made a proposal
you like on the ARP issue, but is there an implementation your would find
acceptable (other than source routing) to send packets out from the NIC
with the SIP configured when there are multiple NICs and IPs in the same
subnet? Using a random NIC for a given SIP confuses Cisco routers (and
other things).

Source routing becomes very complicated when there are a lot of IPs and
they are changing, and there are several patches which force binding a SIP
to a NIC, but you don't seem to like any of them. Please suggest a better
way. 

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.