phosphor-networkd clobbering usb0 network config

phosphor-networkd clobbering usb0 network config

[Oskar Senft]

[-- Attachment #1: Type: text/plain, Size: 1966 bytes --]

Hi everyone

I couldn't find any other mention of this and hope this hasn't been asked /
answered / solved before.

We're using both an NC-SI based NIC and the USB virtual NIC on a AST2500
BMC (on the TYAN S7106 mainboard). I found that phosphor-networkd clobbers
the networking configuration (IP address) for the USB virtual NIC (usb0) in
the following scenario:

   1. The USB virtual NIC (usb0) has it default IP address hard coded in
   /etc/systemd/network/00-bmc-usb0.network.
   2. The host has not yet loaded the USB NIC driver (cdc_ether). In this
   case the USB NIC on the BMC does not have an IP address assigned (I haven't
   investigated why that is, but it seems ok).
   3. A process actively assigns / changes the IP address for the BMC's
   other NIC (i.e. eth0) via phosphor-networkd, e.g. via IPMI from the host.

At step #3 phosphor-networkd overwrites all files in /etc/systemd/network
(EthernetInterface::writeConfigurationFile() called from
Manager::writeToConfigurationFile()). Specifically, it rewrites all files
with information captured from the running system. Since the USB NIC (usb0)
doesn't have an IP address at that time, the rewritten file is missing the
IP address, too.

I can think of various ways to fix this:

   - Make the host explicitly configure usb0 via IPMI before trying to talk
   with the BMC via the USB NIC. This won't work since we'd like to stop using
   IPMI from the host completely.
   - Enhance phosphor-networkd to always explicitly exclude "usb0" as a
   managed device. I wonder if this could be done by adding a new key/value
   pair to /etc/systemd/network/00-bmc-usb0.network, e.g. "[PhosphorNetworkD]
   managed=false". This seems pretty straightforward.
   - Come up with some "automatic" way to not clobber the configuration
   file if the running configuration does not match. It feels that this goes
   against the fundamental design of phosphor-networkd.

Thoughts? Ideas? Opinions?

Thanks
Oskar.

[-- Attachment #2: Type: text/html, Size: 2220 bytes --]

Re: phosphor-networkd clobbering usb0 network config

[Oskar Senft]

[-- Attachment #1: Type: text/plain, Size: 3615 bytes --]

Ok, I think I found a way forward that's not too hacky and is only a few
lines of code change.

Part 1 is to tell systemd-networkd to always configure the IP address on
the interface even if it doesn't have link. Is there a reason we're not
already doing this? Right now, from what I can tell we lose an IP address
on any interface if its link is not up and the IP address for another
interface gets set or changed and then persisted by phosphor-networkd.

systemd-networkd can be reconfigured by adding the following to the
"[Network]" section of the USB virtual NIC on the BMC:
ConfigureWithoutCarrier=true
IgnoreCarrierLoss=true

This looks as following then:
root@bmc:~# cat /etc/systemd/network/00-bmc-usb0.network
[Match]
Name=usb0
[Link]
MACAddress=bc:46:a:4a:8a:d8
[Network]
LinkLocalAddressing=no
IPv6AcceptRA=false
DHCP=false
Address=169.254.254.254/24


*ConfigureWithoutCarrier=trueIgnoreCarrierLoss=true*[DHCP]
ClientIdentifier=mac
UseDNS=true
UseNTP=true
UseHostname=true
SendHostname=true


However, this alone is not sufficient, since phosphor-networkd still
clobbers the configuration file but without these parameters.

Part 2 is to tell phosphor-networkd to include
"ConfigureWithoutCarrier=true" and "IgnoreCarrierLoss=true" in the
generated configuration files. I'm proposing to make this behavior change
via a configure parameter, similar to what we've done with
--disable-link-local-autoconfiguration.

I'll make the necessary changes and will send a commit for review.

Oskar.

On Tue, Dec 10, 2019 at 1:00 PM Oskar Senft <osk@google.com> wrote:

> Hi everyone
>
> I couldn't find any other mention of this and hope this hasn't been asked
> / answered / solved before.
>
> We're using both an NC-SI based NIC and the USB virtual NIC on a AST2500
> BMC (on the TYAN S7106 mainboard). I found that phosphor-networkd clobbers
> the networking configuration (IP address) for the USB virtual NIC (usb0) in
> the following scenario:
>
>    1. The USB virtual NIC (usb0) has it default IP address hard coded in
>    /etc/systemd/network/00-bmc-usb0.network.
>    2. The host has not yet loaded the USB NIC driver (cdc_ether). In this
>    case the USB NIC on the BMC does not have an IP address assigned (I haven't
>    investigated why that is, but it seems ok).
>    3. A process actively assigns / changes the IP address for the BMC's
>    other NIC (i.e. eth0) via phosphor-networkd, e.g. via IPMI from the host.
>
> At step #3 phosphor-networkd overwrites all files in /etc/systemd/network
> (EthernetInterface::writeConfigurationFile() called from
> Manager::writeToConfigurationFile()). Specifically, it rewrites all files
> with information captured from the running system. Since the USB NIC (usb0)
> doesn't have an IP address at that time, the rewritten file is missing the
> IP address, too.
>
> I can think of various ways to fix this:
>
>    - Make the host explicitly configure usb0 via IPMI before trying to
>    talk with the BMC via the USB NIC. This won't work since we'd like to stop
>    using IPMI from the host completely.
>    - Enhance phosphor-networkd to always explicitly exclude "usb0" as a
>    managed device. I wonder if this could be done by adding a new key/value
>    pair to /etc/systemd/network/00-bmc-usb0.network, e.g. "[PhosphorNetworkD]
>    managed=false". This seems pretty straightforward.
>    - Come up with some "automatic" way to not clobber the configuration
>    file if the running configuration does not match. It feels that this goes
>    against the fundamental design of phosphor-networkd.
>
> Thoughts? Ideas? Opinions?
>
> Thanks
> Oskar.
>

[-- Attachment #2: Type: text/html, Size: 4466 bytes --]