From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?iso-8859-1?Q?Ga=EBtan?= Rivet Subject: Re: [PATCH v2 00/13] introduce fail-safe PMD Date: Fri, 10 Mar 2017 10:13:32 +0100 Message-ID: <20170310091332.GI908@bidouze.vm.6wind.com> References: <20170308165402.GA20936@neilslaptop.think-freely.org> <20170309091514.GB302480@bricha3-MOBL3.ger.corp.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: Neil Horman , dev@dpdk.org, Thomas Monjalon , Adrien Mazarguil To: Bruce Richardson Return-path: Received: from mail-wm0-f42.google.com (mail-wm0-f42.google.com [74.125.82.42]) by dpdk.org (Postfix) with ESMTP id 08F83374F for ; Fri, 10 Mar 2017 10:13:41 +0100 (CET) Received: by mail-wm0-f42.google.com with SMTP id v203so2011514wmg.0 for ; Fri, 10 Mar 2017 01:13:41 -0800 (PST) Content-Disposition: inline In-Reply-To: <20170309091514.GB302480@bricha3-MOBL3.ger.corp.intel.com> List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Thu, Mar 09, 2017 at 09:15:14AM +0000, Bruce Richardson wrote: >On Wed, Mar 08, 2017 at 11:54:02AM -0500, Neil Horman wrote: >> On Wed, Mar 08, 2017 at 04:15:33PM +0100, Gaetan Rivet wrote: >> > This PMD intercepts and manages Ethernet device removal events issued by >> > slave PMDs and re-initializes them transparently when brought back so that >> > existing applications do not need to be modified to benefit from true >> > hot-plugging support. >> > >> > The stacked PMD approach shares many similarities with the bonding PMD but >> > with a different purpose. While bonding provides the ability to group >> > several links into a single logical device for enhanced throughput and >> > supports fail-over at link level, this one manages the sudden disappearance >> > of the underlying device; it guarantees applications face a valid device in >> > working order at all times. >> > >> Why not just add this feature to the bonding pmd then? A bond is perfectly >> capable of handling the trivial case of a single underlying device, and adding >> an option to make the underly slave 'persistent' seem both much simpler in terms >> of implementation and code size, than adding an entire new pmd, along with its >> supporting code. >> >> Neil >> @Neil I don't know if you saw my answer to Bruce on the matter [1], it partially adresses your point. >+1 >I don't like the idea of having multiple PMDs in DPDK to handle >combining multiple other devices into one. > >/Bruce I understand the concern. Let's first put aside for the moment the link grouping, which is only part of the fail-safe PMD function. The fail-safe PMD at its core, provides an alternative paradigm, a new proposal for a hot-plug functionality in a lightweight form-factor from a user standpoint. The central question that I would like to tackle is this: why should we require from our users declaring a bonding device to have hot-plug support? I took some time to illustrate a few modes of operation: Fig. 1 .-------------. | application | `------.------' | .----'-----.---------. <------ init, conf, Rx/Tx | | | | .---|--.------|--. <--- conf, link check, Rx/Tx | | | | | | v | v v v v .---------. | .-------. .------. | bonding | | | ixgbe | | mlx4 | `----.----' | `-------' `------' | | `------' Typical link fail-over. Fig. 2 .-------------. | application | `------.------' | <-------- init, conf, Rx/Tx v .-----------. | fail-safe | `-----.-----' | .---'----. <--- init, conf, dev check, Rx/Tx | | v v .-------. .------. | ixgbe | | mlx4 | `-------' `------' Typical automatic hot-plug handling with device fail-over. Fig. 3 .-------------. | application | `------.------' | .----'-----.-------------. <---------- init, conf, Rx/Tx | | | | .---|--.----------|--. <------- conf, link check, Rx/Tx | | | | | | v | v v v v .---------. | .-----------. .-----------. | bonding | | | fail-safe | | fail-safe | `----.----' | `-----.-----' `-----.-----' | | | | <------ init, conf, dev check, Rx/Tx `------' v v .-------. .------. | ixgbe | | mlx4 | `-------' `------' Combination to provide link fail-over with automatic hot-plug handling. Fig. 4 .-------------. | application | `------.------' | .----'-----.-------------. <---------- init, conf, Rx/Tx | | | | .---|--.----------|--. <------- conf, link check, Rx/Tx | | | | | | v | v v v v .---------. | .-----------. .-----------. | bonding | | | fail-safe | | fail-safe | `----.----' | `-----.-----' `-----.-----' | | | | <------- init, conf, dev check, Rx/Tx `------' | | .------'---. .---'------. | | | | v v v v .--------. .--------. .--------. .--------. | mlx4 1 | | mlx4 2 | | mlx4 1 | | mlx4 2 | | port 1 | | port 1 | | port 2 | | port 2 | `--------' `--------' `--------' `--------' Complex use case with link fail-over at port level and automatic hot-plug handling with device fail-over. 1. LSC vs. RMV A link status change is a valid state for a device. It calls for specific responses, e.g. a link switch in a bonding device, without losing the general configuration of the port. The removal of a device calls for more than pausing operations and switching an active device. The party responsible for initializing the device should take care of closing it properly. If this party also wants to be able to restore the device if it was plugged back in, it would need be able to initialize it back and reconfigure its previous state. As we can see that in [Fig. 1], this responsibility lies upon the application. 2. Bonding and link availability The hot-plug functionality is not a core function of the bonding PMD. It is only interested in knowing if the link is active or not. Adding the device persistence to the bonding PMD would mean adding the ability to flexibly parse device definitions to cope with plug-ins in evolving busses (PCI hot-plug could mean changing bus addresses), being able to emulate the EAL and the ether layer and to properly store the device configuration. This means formally describing the life of a device in a DPDK application from start to finish. All of this hot-plug support, in order for the bonding to be aware of the status of some of its links. This seems like scope-creep. 3. Fail-safe and hot-plug support An attach / detach (pseudo-hotplug) API exists in DPDK. The main problem of this API is that it does not offer reacting to a device plug-out, only triggering a device detaching from an application. This is a completely different approach from an application standpoint. The fail-safe PMD offers an out-of-the-box implementation of a newly proposed hot-plug API [2]. It allows a seamless integration to users for device removal support in their applications, without requiring evolutions [Fig. 2]. This flexibility allows it to be used as part of a bond [Fig. 3], [Fig. 4], while current bonding does not allow for detaching devices, even if it were to be considered hot-plug. There is no reason however to require declaring a bonding device to be able to use it, from a user perspective this seems backward. Both bonding and fail-safe PMDs deal with enslaving devices and acting upon their state changing. The bonding function performs at link level, while the hot-plug function deals with the device itself. I do not think this similarity justify merging both functions. Maintenance would be easier with clear, simpler functions implemented in separate PMDs. [1]: http://dpdk.org/ml/archives/dev/2017-March/059446.html [2]: http://dpdk.org/ml/archives/dev/2017-March/059217.html -- Gaëtan Rivet 6WIND