From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DFA69C432BE for ; Wed, 28 Jul 2021 04:58:01 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 778E260EB9 for ; Wed, 28 Jul 2021 04:58:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 778E260EB9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:37812 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m8bdo-0005KC-MM for qemu-devel@archiver.kernel.org; Wed, 28 Jul 2021 00:58:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38790) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m8bca-0002da-3w for qemu-devel@nongnu.org; Wed, 28 Jul 2021 00:56:45 -0400 Received: from szxga08-in.huawei.com ([45.249.212.255]:2262) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m8bcX-0003xQ-Il for qemu-devel@nongnu.org; Wed, 28 Jul 2021 00:56:43 -0400 Received: from dggemv711-chm.china.huawei.com (unknown [172.30.72.54]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4GZLns4xPzz1CPh9; Wed, 28 Jul 2021 12:50:41 +0800 (CST) Received: from dggpeml500005.china.huawei.com (7.185.36.59) by dggemv711-chm.china.huawei.com (10.1.198.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Wed, 28 Jul 2021 12:56:37 +0800 Received: from [10.174.186.51] (10.174.186.51) by dggpeml500005.china.huawei.com (7.185.36.59) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Wed, 28 Jul 2021 12:56:36 +0800 From: Zheng Chuan Subject: Re: [PATCH V5 17/25] vfio-pci: cpr part 2 To: Steven Sistare , Alex Williamson References: <1625678434-240960-1-git-send-email-steven.sistare@oracle.com> <1625678434-240960-18-git-send-email-steven.sistare@oracle.com> <20210716145133.4aa3f341.alex.williamson@redhat.com> <20210719121021.6babb9ff.alex.williamson@redhat.com> <2f85af36-5d64-99b0-5165-ceb73087d247@oracle.com> Message-ID: <93e7660d-6d89-8d35-b415-88ccb7e6d7fc@huawei.com> Date: Wed, 28 Jul 2021 12:56:36 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Thunderbird/68.6.0 MIME-Version: 1.0 In-Reply-To: <2f85af36-5d64-99b0-5165-ceb73087d247@oracle.com> Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [10.174.186.51] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpeml500005.china.huawei.com (7.185.36.59) X-CFilter-Loop: Reflected Received-SPF: pass client-ip=45.249.212.255; envelope-from=zhengchuan@huawei.com; helo=szxga08-in.huawei.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, NICE_REPLY_A=-0.438, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jason Zeng , Juan Quintela , Eric Blake , "Michael S. Tsirkin" , qemu-devel@nongnu.org, "Dr. David Alan Gilbert" , Paolo Bonzini , Stefan Hajnoczi , =?UTF-8?Q?Marc-Andr=c3=a9_Lureau?= , "Daniel P. Berrange" , =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= , =?UTF-8?Q?Alex_Benn=c3=a9e?= , Markus Armbruster Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Hi On 2021/7/20 2:38, Steven Sistare wrote: > On 7/19/2021 2:10 PM, Alex Williamson wrote: >> On Mon, 19 Jul 2021 13:44:08 -0400 >> Steven Sistare wrote: >> >>> On 7/16/2021 4:51 PM, Alex Williamson wrote: >>>> On Wed, 7 Jul 2021 10:20:26 -0700 >>>> Steve Sistare wrote: >>>> >>>>> Finish cpr for vfio-pci by preserving eventfd's and vector state. >>>>> >>>>> Signed-off-by: Steve Sistare >>>>> --- >>>>> hw/vfio/pci.c | 118 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++- >>>>> 1 file changed, 116 insertions(+), 2 deletions(-) >>>>> >>>>> diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c >>>>> index 0f5c542..07bd360 100644 >>>>> --- a/hw/vfio/pci.c >>>>> +++ b/hw/vfio/pci.c >>>> ... >>>>> @@ -3295,14 +3329,91 @@ static void vfio_merge_config(VFIOPCIDevice >>>> *vdev) >>>>> g_free(phys_config); >>>>> } >>>>> >>>>> +static int vfio_pci_pre_save(void *opaque) >>>>> +{ >>>>> + VFIOPCIDevice *vdev = opaque; >>>>> + PCIDevice *pdev = &vdev->pdev; >>>>> + int i; >>>>> + >>>>> + if (vfio_pci_read_config(pdev, PCI_INTERRUPT_PIN, 1)) { >>>>> + error_report("%s: cpr does not support vfio-pci INTX", >>>>> + vdev->vbasedev.name); >>>>> + } >>>> >>>> You're not only not supporting INTx, but devices that support INTx, so >>>> this only works on VFs. Why? Is this just out of scope or is there >>>> something fundamentally difficult about it? >>>> >>>> This makes me suspect there's a gap in INTx routing setup if it's more >>>> than just another eventfd to store and setup. If we hot-add a device >>>> using INTx after cpr restart, are we going to find problems? Thanks, >>> >>> It could be supported, but requires more code (several event fd's plus other state in VFIOINTx >>> to save and restore) for a case that does not seem very useful (a directly assigned device that >>> only supports INTx ?). >> >> It's not testing that the device *only* supports INTx, it's testing >> that the device supports INTx _at_all_. That effectively means this >> excludes anything other than an SR-IOV VF. There are plenty of valid >> and useful cases of assigning PFs, most of which support INTx even if >> we don't expect that's their primary operational mode. Thanks, > > OK, I'll look into it. If this proves problematic, how do you feel about deferring > INTx support to a later patch? > I am curious about that does cpr restart mode work for GPU passthrough? > - Steve > > . > -- Regards. Chuan