From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5118C352A1 for ; Tue, 6 Dec 2022 15:06:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235252AbiLFPGg (ORCPT ); Tue, 6 Dec 2022 10:06:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60782 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234577AbiLFPGO (ORCPT ); Tue, 6 Dec 2022 10:06:14 -0500 Received: from verein.lst.de (verein.lst.de [213.95.11.211]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD3202F002; Tue, 6 Dec 2022 07:01:50 -0800 (PST) Received: by verein.lst.de (Postfix, from userid 2407) id 3A0CA68CFE; Tue, 6 Dec 2022 16:01:33 +0100 (CET) Date: Tue, 6 Dec 2022 16:01:31 +0100 From: Christoph Hellwig To: Jason Gunthorpe Cc: Christoph Hellwig , Lei Rao , kbusch@kernel.org, axboe@fb.com, kch@nvidia.com, sagi@grimberg.me, alex.williamson@redhat.com, cohuck@redhat.com, yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com, kevin.tian@intel.com, mjrosato@linux.ibm.com, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, kvm@vger.kernel.org, eddie.dong@intel.com, yadong.li@intel.com, yi.l.liu@intel.com, Konrad.wilk@oracle.com, stephen@eideticom.com, hang.yuan@intel.com Subject: Re: [RFC PATCH 5/5] nvme-vfio: Add a document for the NVMe device Message-ID: <20221206150131.GA32365@lst.de> References: <20221206055816.292304-1-lei.rao@intel.com> <20221206055816.292304-6-lei.rao@intel.com> <20221206062604.GB6595@lst.de> <20221206130901.GB24358@lst.de> <20221206140002.GB27689@lst.de> <20221206143126.GB30297@lst.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.17 (2007-11-01) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Dec 06, 2022 at 10:48:22AM -0400, Jason Gunthorpe wrote: > Sadly in Linux we don't have a SRIOV VF lifecycle model that is any > use. Beward: The secondary function might as well be a physical function as well. In fact one of the major customers for "smart" multifunction nvme devices prefers multi-PF devices over SR-IOV VFs. (and all the symmetric dual ported devices are multi-PF as well). So this isn't really about a VF live cycle, but how to manage life migration, especially on the receive / restore side. And restoring the entire controller state is extremely invasive and can't be done on a controller that is in any classic form live. In fact a lot of the state is subsystem-wide, so without some kind of virtualization of the subsystem it is impossible to actually restore the state. To cycle back to the hardware that is posted here, I'm really confused how it actually has any chance to work and no one has even tried to explain how it is supposed to work.