Re: vxlan mac address generation

Re: vxlan mac address generation

[Russell Strong]

On Tue, 14 Apr 2020 17:23:48 -0700
Stephen Hemminger <stephen@networkplumber.org> wrote:

> On Wed, 15 Apr 2020 10:05:24 +1000
> Russell Strong <russell@strong.id.au> wrote:
> 
> > Hi Stephen,
> > 
> > I've hit a problem with vxlan not communicating because mac
> > addresses being duplicated when I use the same IFNAME across
> > multiple virtual machines. The mac address appears to be some sort
> > of hash related to the IFNAME. Changing the name changes the mac
> > address.
> > 
> > Looking at vxlan_setup this should be random (eth_hw_addr_random)
> > but it is not.
> > 
> > Is there a bug here?
> > 
> > Regards,
> > Russell Strong  
> 
> Please forward questions to netdev@vger.kernel.org.
> Do you have weak seeding on your platform?  This happens early in boot
> process and maybe PRNG is not seeded yet.

I have checked the diff of /var/lib/systemd/random-seed.  Each machine
has a different seed.  Plus each machine has been running for a long
period of time.  They are however, cloned from the same source machine
wayback.

As an example, if I execute the following commands

ip link add v0 type vxlan id 1

I get the mac address 6e:7a:da:1c:12:0c

then delete and recreate,

ip link del v0
ip link add v0 type vxlan id 1

I get back the same address 6e:7a:da:1c:12:0c

I would have expected a different mac as the PRNG should have
advanced.  Also, if I change v0 to something else, I get a different mac
address but the same repeatability.

Re: vxlan mac address generation

[Russell Strong]

On Tue, 14 Apr 2020 21:12:06 -0700
Stephen Hemminger <stephen@networkplumber.org> wrote:

> On Wed, 15 Apr 2020 10:05:24 +1000
> Russell Strong <russell@strong.id.au> wrote:
> 
> > Hi Stephen,
> > 
> > I've hit a problem with vxlan not communicating because mac
> > addresses being duplicated when I use the same IFNAME across
> > multiple virtual machines. The mac address appears to be some sort
> > of hash related to the IFNAME. Changing the name changes the mac
> > address.
> > 
> > Looking at vxlan_setup this should be random (eth_hw_addr_random)
> > but it is not.
> > 
> > Is there a bug here?
> > 
> > Regards,
> > Russell Strong  
> 
> I don't know what platform you are using but on x86-64 Debian 10.
> 
> # ip li add vxlan0 type vxlan id 1
> vxlan: destination port not specified
> Will use Linux kernel default (non-standard value)
> Use 'dstport 4789' to get the IANA assigned value
> Use 'dstport 0' to get default and quiet this message
> # ip li show dev vxlan0
> 8: vxlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT group default qlen 1000 link/ether 5a:09:e2:81:08:61 brd
> ff:ff:ff:ff:ff:ff # ip li del dev vxlan0
> # ip li add vxlan0 type vxlan id 1
> vxlan: destination port not specified
> Will use Linux kernel default (non-standard value)
> Use 'dstport 4789' to get the IANA assigned value
> Use 'dstport 0' to get default and quiet this message
> # ip li show dev vxlan0
> 9: vxlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT group default qlen 1000 link/ether 06:8d:c0:4a:73:90 brd
> ff:ff:ff:ff:ff:ff # ip li del dev vxlan0
> 
> 
> # ip li add vxlan0 type vxlan id 1
> vxlan: destination port not specified
> Will use Linux kernel default (non-standard value)
> Use 'dstport 4789' to get the IANA assigned value
> Use 'dstport 0' to get default and quiet this message
> # ip li show dev vxlan0
> 11: vxlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> DEFAULT group default qlen 1000 link/ether a2:2e:04:f6:85:73 brd
> ff:ff:ff:ff:ff:ff
> 

I tried debian ( 4.19.0-8-amd64 ) and got the same result as you.  I am
using Fedora 31 ( 5.5.15-200.fc31.x86_64 ).  I have discovered a
difference:

On fedora /sys/class/net/v0/addr_assign_type = 3
On debian /sys/class/net/v0/addr_assign_type = 1

The debian value is what I would expect (NET_ADDR_RANDOM).  I thought
addr_assign_type was controlled by the driver.  Do you think this could
be a Fedora bug, or perhaps something has changed between 4.19 and 5.5?

Re: vxlan mac address generation

[Ido Schimmel]

On Wed, Apr 15, 2020 at 05:28:00PM +1000, Russell Strong wrote:
> I tried debian ( 4.19.0-8-amd64 ) and got the same result as you.  I am
> using Fedora 31 ( 5.5.15-200.fc31.x86_64 ).  I have discovered a
> difference:
> 
> On fedora /sys/class/net/v0/addr_assign_type = 3
> On debian /sys/class/net/v0/addr_assign_type = 1
> 
> The debian value is what I would expect (NET_ADDR_RANDOM).  I thought
> addr_assign_type was controlled by the driver.  Do you think this could
> be a Fedora bug, or perhaps something has changed between 4.19 and 5.5?

I assume you're using systemd 242 or later. I also hit this issue.
Documented the solution here:

https://github.com/Mellanox/mlxsw/wiki/Persistent-Configuration#required-changes-to-macaddresspolicy

Re: vxlan mac address generation

[Russell Strong]

On Wed, 15 Apr 2020 13:21:31 +0300
Ido Schimmel <idosch@idosch.org> wrote:

> On Wed, Apr 15, 2020 at 05:28:00PM +1000, Russell Strong wrote:
> > I tried debian ( 4.19.0-8-amd64 ) and got the same result as you.
> > I am using Fedora 31 ( 5.5.15-200.fc31.x86_64 ).  I have discovered
> > a difference:
> > 
> > On fedora /sys/class/net/v0/addr_assign_type = 3
> > On debian /sys/class/net/v0/addr_assign_type = 1
> > 
> > The debian value is what I would expect (NET_ADDR_RANDOM).  I
> > thought addr_assign_type was controlled by the driver.  Do you
> > think this could be a Fedora bug, or perhaps something has changed
> > between 4.19 and 5.5?  
> 
> I assume you're using systemd 242 or later. I also hit this issue.
> Documented the solution here:
> 
> https://github.com/Mellanox/mlxsw/wiki/Persistent-Configuration#required-changes-to-macaddresspolicy

Thank you.  You have hit on the issue.

So for future googlers:

I was not using systemds' networking, I'm building my own embedded
router for radio and satellite networks, but because they have spilled
their functionality into udev as well, it can not be turned off without
modifying /usr/lib/systemd/network/99-default.link. This is not a
directory listed in the udev man page, and udev seems to be ignoring any
administrative overrides in /etc/udev/rules.d

I changed the line
MACAddressPolicy=persistent
to
MACAddressPolicy=none

The systemd requirement that implemented this says it all:

Keep MACPolicy=persistent. If people don't want it, they can always
apply local configuration, but in general stable MACs are a good thing.
I have never seen anyone complain about that.

Not the first time and won't be the last time systemd policy decisions
have caught me out.

Thanks again everyone,
Russell