Re: Linux kernel commit breaks IPMI on iface downing - Chen-Yu Tsai

From: Chen-Yu Tsai <wens@csie.org>
To: Harish Patil <harish.patil@qlogic.com>
Cc: "Michael Chan" <mchan@broadcom.com>,
	"Sébastien Bocahu" <sbu@reportlinker.com>,
	"David Miller" <davem@davemloft.net>,
	netdev <netdev@vger.kernel.org>,
	"Ameen Rahman" <ameen.rahman@qlogic.com>
Subject: Re: Linux kernel commit breaks IPMI on iface downing
Date: Tue, 2 Aug 2016 15:58:57 +0800	[thread overview]
Message-ID: <CAGb2v64BUxFn0GWZVJN=SzDQXGMW7+8Yh-QmRKUSz8ECNZ8wDQ@mail.gmail.com> (raw)
In-Reply-To: <D214D9E7.BFB25%harish.patil@qlogic.com>

Hi,

On Wed, Sep 9, 2015 at 9:33 AM, Harish Patil <harish.patil@qlogic.com> wrote:
>
>>On Fri, 2015-09-04 at 09:55 +0200, Sébastien Bocahu wrote:
>>> Hi,
>>>
>>> Any chance this behaviour gets fixed, with either a new firmware or a
>>> workaround in the kernel ?
>>>
>>
>>As I said earlier, when we call bnx2_shutdown_chip(), we inform the
>>firmware that the driver is shutting down.  The firmware should know
>>that there is IPMI firmware and the link needs to stay up.
>>
>>In the older driver, we would also call bnx2_set_power_state() which
>>would send some additional messages to the firmware before putting the
>>device in D3hot.  May be these messages are required by the firmware to
>>keep the link up.  Harish, please check with your firmware team.
>>Thanks.
>>
>>
>>
>>
>
> Hi Michael/Sebastien,
> ACK. Sure, I will look into it and get back.

Sorry to resurrect such an old thread. I'm still having problems with this.

We have a whole bunch of Dell PowerEdge R210 II servers at our datacenter.
These have the NetXtreme II BCM 5716 controller. Most still have the
factory firmware (6.2.12) while a few have been updated to the latest
firmware released by Dell (7.12.19). With both versions, IPMI over LAN
becomes unusable as soon as Ubuntu's installer reconfigures the network.
It only returns after installation is complete and the system is reboot,
after the firmware is reloaded.

This means if anything were to fail during an automated installation,
the system would no longer be controllable remotely. For us this is a
huge inconvenience, as we have hundreds of them. If any go down due
to bad installation scripts or entering a wrong command, someone has
to literally go there and reboot the machine.

I also tested with Ubuntu 16.04.1, running Ubuntu's 4.4.0-31-generic
kernel. IPMI over LAN is unusable as soon as "ip link set XXX down" is
run, and recovers in 1 second after "ip link set XXX up" is run, or when
the system is restarted.

So in addition to IPMI not working on iface down, there might be a
problem with how the installer works, resulting in the prolonged
outage.

Regards
ChenYu