* Re: Route cache problem.
[not found] <20111103143734.GA17218@bitwizard.nl>
@ 2011-11-03 15:16 ` Eric Dumazet
2011-11-18 8:17 ` Rogier Wolff
0 siblings, 1 reply; 2+ messages in thread
From: Eric Dumazet @ 2011-11-03 15:16 UTC (permalink / raw)
To: Rogier Wolff; +Cc: linux-kernel, netdev
Le jeudi 03 novembre 2011 à 15:37 +0100, Rogier Wolff a écrit :
> Hi,
>
> My workstation has an incorrect route cache entry:
>
What kernel version ?
> assurancetourix:~> route -nC | head -2 ; route -nC | grep 234.34
> Kernel IP routing cache
> Source Destination Gateway Flags Metric Ref Use Iface
> 192.168.235.8 192.168.234.34 192.168.235.251 0 0 3 eth0
> 192.168.235.8 192.168.234.34 192.168.235.251 0 0 4 eth0
> 192.168.235.8 192.168.234.34 192.168.235.251 0 0 2 eth0
>
> (I don't know why there are three).
>
Different routing keys, like TOS.
for tos in `seq 2 2 34`; do ping -c 1 -Q $tos 10.37.168.112; done
# route -nC|grep 10.37.168.112
10.37.168.112 192.168.20.108 192.168.20.108 l 0 0 479 lo
192.168.20.108 10.37.168.112 192.168.20.254 0 0 3 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 3 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 3 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 3 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 3 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 5 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 5 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 5 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 0 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 0 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 0 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 0 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 0 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 0 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 0 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 0 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 0 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 0 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 0 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 0 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 1 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 1 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 0 0 eth3
192.168.20.108 10.37.168.112 192.168.20.254 0 1 2 eth3
> the correct routing cache entries would look something like this:
> (this one works):
> assurancetourix:~> route -nC | head -2 ; route -nC | grep 234.20
> Kernel IP routing cache
> Source Destination Gateway Flags Metric Ref Use Iface
> 192.168.235.8 192.168.234.20 192.168.235.4 0 0 1 eth0
> 192.168.234.20 192.168.235.8 192.168.235.8 l 0 0 0 lo
> 192.168.235.8 192.168.234.20 192.168.235.4 0 0 0 eth0
>
> The routing table is:
>
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 192.168.235.251 0.0.0.0 UG 0 0 0 eth0
> 169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 eth0
> 192.168.234.0 192.168.235.4 255.255.255.0 UG 0 0 0 eth0
> 192.168.235.0 0.0.0.0 255.255.255.0 U 1 0 0 eth0
> 192.168.235.2 192.168.235.4 255.255.255.255 UGH 0 0 0 eth0
>
> It's the third line that is supposed to steer packets for '234.34 to
> the proper router that knows how to reach the 234.0 network.
>
> As a temporary workaround I've added the route to 192.168.235.2 which
> is that same host, but not in the nameserver, so it's annoying.
> (the other host that I can't reach due to this problem doesn't have
> a second IP address (yet)).
>
> Oh... routing to 192.168.234.34 works on the router 192.168.235.4:
> PING 192.168.234.34 (192.168.234.34) 56(84) bytes of data.
> 64 bytes from 192.168.234.34: icmp_req=1 ttl=64 time=41.5 ms
>
> Anyway, what would you suggest for me to try to get that invalid
> route cache entry dropped?
>
> Drop the default route? Ok. Done:
> # route del default
> # ping 192.168.234.34
> 2 packets transmitted, 0 received, 100% packet loss, time 1007ms
>
> I'm used to the default route being at the bottom, but deleting it should
> be enough to prevent it from being found first, right? :-)
>
> Add a host route to this host explicitly naming the router?
>
> assurancetourix:~# route add 192.168.234.34 gw driepoot
> assurancetourix:~# route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 eth0
> 192.168.234.0 192.168.235.4 255.255.255.0 UG 0 0 0 eth0
> 192.168.234.34 192.168.235.4 255.255.255.255 UGH 0 0 0 eth0
> 192.168.235.0 0.0.0.0 255.255.255.0 U 1 0 0 eth0
> 192.168.235.2 192.168.235.4 255.255.255.255 UGH 0 0 0 eth0
> assurancetourix:~# ping 192.168.234.34 -c 2
> ....
> 2 packets transmitted, 0 received, 100% packet loss, time 1008ms
>
> Still the packets end up on the ethernet with the 192.168.235.251 router's
> Ethernet address.....
>
> assurancetourix:~# route -nC | head -2 ; route -nC | grep 234.34
> Kernel IP routing cache
> Source Destination Gateway Flags Metric Ref Use Iface
> 192.168.235.8 192.168.234.34 192.168.235.251 0 0 0 eth0
> 192.168.235.8 192.168.234.34 192.168.235.251 0 0 0 eth0
> 192.168.235.8 192.168.234.34 192.168.235.251 0 0 5 eth0
>
Better use "ip -s route list cache" to diagnose problems (more
information)
>
> # ifconfig eth0 down
> # route -n
> <empty table>
> # ifconfig eth0 up
> <old routing table is restored automatically??? apparently with the routing
> cache entries as well....>
>
>
> I initially thought that this was a problem with the routing
> cache entry being too persistent in the kernel. While documenting
> this while writing this email, I've found that I can flush the whole routing
> cache with "ip route flush cache" .
>
> However the routing cache entry springs back to life when I first
> ping the 234.34 host. Even when the problem machine doesn't have a
> default route, so it shouldn't know about the 235.251 default router.
>
> This is getting weirder and weirder.
>
> During all this I have
> # tcpdump -nei eth0 net 192.168.234.0/24
> running. If my machine were to get an ICMP redirect from somewhere
> I'd see it, right?
>
> It could be that the 192.168.235.251 router is proxy-arping (incorreclty)
> for the problem hosts. But then my workstation would have to be
> ARPing in the first place.
>
> # route add 192.168.234.200 eth0
> # ping 192.168.234.200
> gives:
> 15:31:33.857343 00:23:54:15:1f:a9 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.234.200 tell 192.168.235.8, length 28
> in the TCPDUMP, so my machine is not arping for 192.168.234.34.
>
> Any suggestions? Any at all?
>
> Roger.
>
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: Route cache problem.
2011-11-03 15:16 ` Route cache problem Eric Dumazet
@ 2011-11-18 8:17 ` Rogier Wolff
0 siblings, 0 replies; 2+ messages in thread
From: Rogier Wolff @ 2011-11-18 8:17 UTC (permalink / raw)
To: Eric Dumazet; +Cc: linux-kernel, netdev
On Thu, Nov 03, 2011 at 04:16:50PM +0100, Eric Dumazet wrote:
> Le jeudi 03 novembre 2011 à 15:37 +0100, Rogier Wolff a écrit :
> > Hi,
> >
> > My workstation has an incorrect route cache entry:
> >
>
> What kernel version ?
Linux version 3.0.0-12-generic (from Ubunutu oneiric.)
> > assurancetourix:~> route -nC | head -2 ; route -nC | grep 234.34
> > Kernel IP routing cache
> > Source Destination Gateway Flags Metric Ref Use Iface
> > 192.168.235.8 192.168.234.34 192.168.235.251 0 0 3 eth0
> > 192.168.235.8 192.168.234.34 192.168.235.251 0 0 4 eth0
> > 192.168.235.8 192.168.234.34 192.168.235.251 0 0 2 eth0
> >
> > (I don't know why there are three).
Today there are four.
> 192.168.20.108 10.37.168.112 192.168.20.254 0 1 2 eth3
That indeed got me a full complement of route cache entries.
> Better use "ip -s route list cache" to diagnose problems (more
> information)
After doing the tos ping you suggested All TOS levels have a route
cache entry.
192.168.234.34 from 192.168.235.8 tos 0x1c via 192.168.235.251 dev eth0
cache <redirected> age 77sec ipid 0xaa09 rtt 47ms rttvar 15ms ssthresh 7 cwnd 9
192.168.234.34 tos 0x1c via 192.168.235.251 dev eth0 src 192.168.235.8
cache <redirected> used 3 age 72sec ipid 0xaa09 rtt 47ms rttvar 15ms ssthresh 7 cwnd 9
192.168.234.34 from 192.168.235.8 tos 0x1c via 192.168.235.251 dev eth0
cache <redirected> age 72sec ipid 0xaa09 rtt 47ms rttvar 15ms ssthresh 7 cwnd 9
> > Any suggestions? Any at all?
Last time, as well as this time, it is triggered by a network error
that leads to the 192.168.235.4 router not being able to reach
192.168.234.34 or any other host on the 192.168.234.0/24 network.
During that time the VPN to 192.168.234.0/24 is down, so 192.168.235.4
doesn't have a route to 192.168.234.0/24 and it is logical that
with that route gone, it sends packets for 192.168.234.0/24 to the default
router 192.168.235.251. As it sees itself forwarding packets that come
in on eth0 back to eth0, it will send a redirect. However that redirect
should somehow expire, and not survive things like dropping the route
to 192.168.234.0/24, dropping the default route, shutting down the
interface or some time passing (that network problem was solved 20
hours ago)......
Roger.
--
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2600998 **
** Delftechpark 26 2628 XH Delft, The Netherlands. KVK: 27239233 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
The plan was simple, like my brother-in-law Phil. But unlike
Phil, this plan just might work.
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2011-11-18 8:17 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20111103143734.GA17218@bitwizard.nl>
2011-11-03 15:16 ` Route cache problem Eric Dumazet
2011-11-18 8:17 ` Rogier Wolff
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).