From mboxrd@z Thu Jan 1 00:00:00 1970 From: Lan Tianyu Subject: [RFC PATCH V2 00/10] Qemu: Add live migration support for SRIOV NIC Date: Tue, 24 Nov 2015 21:35:17 +0800 Message-ID: <1448372127-28115-1-git-send-email-tianyu.lan@intel.com> Cc: Lan Tianyu To: aik@ozlabs.ru, alex.williamson@redhat.com, amit.shah@redhat.com, anthony@codemonkey.ws, ard.biesheuvel@linaro.org, blauwirbel@gmail.com, cornelia.huck@de.ibm.com, eddie.dong@intel.com, nrupal.jani@intel.com, agraf@suse.de, kvm@vger.kernel.org, pbonzini@redhat.com, qemu-devel@nongnu.org, emil.s.tantilov@intel.com, gerlitz.or@gmail.com, donald.c.skidmore@intel.com, mark.d.rustad@intel.com, mst@redhat.com, kraxel@redhat.com, lcapitulino@redhat.com, quintela@redhat.com Return-path: Received: from mga03.intel.com ([134.134.136.65]:42652 "EHLO mga03.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753355AbbKXNs0 (ORCPT ); Tue, 24 Nov 2015 08:48:26 -0500 Sender: kvm-owner@vger.kernel.org List-ID: This patchset is to propose a solution of adding live migration support for SRIOV NIC. During migration, Qemu needs to let VF driver in the VM to know migration start and end. Qemu adds faked PCI migration capability to help to sync status between two sides during migration. Qemu triggers VF's mailbox irq via sending MSIX msg when migration status is changed. VF driver tells Qemu its mailbox vector index via the new PCI capability. In some cases(NIC is suspended or closed), VF mailbox irq is freed and VF driver can disable irq injecting via new capability. VF driver will put down nic before migration and put up again on the target machine. Lan Tianyu (10): Qemu/VFIO: Create head file pci.h to share data struct. Qemu/VFIO: Add new VFIO_GET_PCI_CAP_INFO ioctl cmd definition Qemu/VFIO: Rework vfio_std_cap_max_size() function Qemu/VFIO: Add vfio_find_free_cfg_reg() to find free PCI config space regs Qemu/VFIO: Expose PCI config space read/write and msix functions Qemu/PCI: Add macros for faked PCI migration capability Qemu: Add post_load_state() to run after restoring CPU state Qemu: Add save_before_stop callback to run just before stopping VCPU during migration Qemu/VFIO: Add SRIOV VF migration support Qemu/VFIO: Misc change for enable migration with VFIO hw/vfio/Makefile.objs | 2 +- hw/vfio/pci.c | 196 +++++++++----------------------------------- hw/vfio/pci.h | 168 +++++++++++++++++++++++++++++++++++++ hw/vfio/sriov.c | 178 ++++++++++++++++++++++++++++++++++++++++ include/hw/pci/pci_regs.h | 19 +++++ include/migration/vmstate.h | 5 ++ include/sysemu/sysemu.h | 1 + linux-headers/linux/vfio.h | 16 ++++ migration/migration.c | 3 +- migration/savevm.c | 28 +++++++ 10 files changed, 459 insertions(+), 157 deletions(-) create mode 100644 hw/vfio/pci.h create mode 100644 hw/vfio/sriov.c -- 1.9.3 From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33489) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a1DxP-00055X-K6 for qemu-devel@nongnu.org; Tue, 24 Nov 2015 08:48:32 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a1DxM-0001D7-Dn for qemu-devel@nongnu.org; Tue, 24 Nov 2015 08:48:31 -0500 Received: from mga02.intel.com ([134.134.136.20]:2994) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a1DxM-0001Cm-7P for qemu-devel@nongnu.org; Tue, 24 Nov 2015 08:48:28 -0500 From: Lan Tianyu Date: Tue, 24 Nov 2015 21:35:17 +0800 Message-Id: <1448372127-28115-1-git-send-email-tianyu.lan@intel.com> Subject: [Qemu-devel] [RFC PATCH V2 00/10] Qemu: Add live migration support for SRIOV NIC List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: aik@ozlabs.ru, alex.williamson@redhat.com, amit.shah@redhat.com, anthony@codemonkey.ws, ard.biesheuvel@linaro.org, blauwirbel@gmail.com, cornelia.huck@de.ibm.com, eddie.dong@intel.com, nrupal.jani@intel.com, agraf@suse.de, kvm@vger.kernel.org, pbonzini@redhat.com, qemu-devel@nongnu.org, emil.s.tantilov@intel.com, gerlitz.or@gmail.com, donald.c.skidmore@intel.com, mark.d.rustad@intel.com, mst@redhat.com, kraxel@redhat.com, lcapitulino@redhat.com, quintela@redhat.com Cc: Lan Tianyu This patchset is to propose a solution of adding live migration support for SRIOV NIC. During migration, Qemu needs to let VF driver in the VM to know migration start and end. Qemu adds faked PCI migration capability to help to sync status between two sides during migration. Qemu triggers VF's mailbox irq via sending MSIX msg when migration status is changed. VF driver tells Qemu its mailbox vector index via the new PCI capability. In some cases(NIC is suspended or closed), VF mailbox irq is freed and VF driver can disable irq injecting via new capability. VF driver will put down nic before migration and put up again on the target machine. Lan Tianyu (10): Qemu/VFIO: Create head file pci.h to share data struct. Qemu/VFIO: Add new VFIO_GET_PCI_CAP_INFO ioctl cmd definition Qemu/VFIO: Rework vfio_std_cap_max_size() function Qemu/VFIO: Add vfio_find_free_cfg_reg() to find free PCI config space regs Qemu/VFIO: Expose PCI config space read/write and msix functions Qemu/PCI: Add macros for faked PCI migration capability Qemu: Add post_load_state() to run after restoring CPU state Qemu: Add save_before_stop callback to run just before stopping VCPU during migration Qemu/VFIO: Add SRIOV VF migration support Qemu/VFIO: Misc change for enable migration with VFIO hw/vfio/Makefile.objs | 2 +- hw/vfio/pci.c | 196 +++++++++----------------------------------- hw/vfio/pci.h | 168 +++++++++++++++++++++++++++++++++++++ hw/vfio/sriov.c | 178 ++++++++++++++++++++++++++++++++++++++++ include/hw/pci/pci_regs.h | 19 +++++ include/migration/vmstate.h | 5 ++ include/sysemu/sysemu.h | 1 + linux-headers/linux/vfio.h | 16 ++++ migration/migration.c | 3 +- migration/savevm.c | 28 +++++++ 10 files changed, 459 insertions(+), 157 deletions(-) create mode 100644 hw/vfio/pci.h create mode 100644 hw/vfio/sriov.c -- 1.9.3