* [Qemu-devel] [PATCH] vfio: assign idstr for VFIO's mmaped regions for migration
@ 2019-01-08 6:03 Zhao Yan
2019-01-08 6:03 ` Zhao Yan
0 siblings, 1 reply; 5+ messages in thread
From: Zhao Yan @ 2019-01-08 6:03 UTC (permalink / raw)
To: qemu-devel; +Cc: pbonzini, alex.williamson, Zhao Yan
if multiple regions in vfio are mmaped, their corresponding ramblocks
are like below, i.e. their idstrs are "".
(qemu) info ramblock
Block Name PSize Offset Used Total
pc.ram 4 KiB 0x0000000000000000 0x0000000020000000 0x0000000020000000
4 KiB 0x0000000021100000 0x0000000020000000 0x0000000020000000
4 KiB 0x0000000020900000 0x0000000000800000 0x0000000000800000
4 KiB 0x0000000020240000 0x0000000000687000 0x0000000000687000
4 KiB 0x00000000200c0000 0x0000000000178000 0x0000000000178000
pc.bios 4 KiB 0x0000000020000000 0x0000000000040000 0x0000000000040000
pc.rom 4 KiB 0x0000000020040000 0x0000000000020000 0x0000000000020000
This is because ramblocks' idstr are assigned by calling
vmstate_register_ram(), but memory region of type ram device ptr does not
call vmstate_register_ram().
vfio_region_mmap
|->memory_region_init_ram_device_ptr
|-> memory_region_init_ram_ptr
Without empty idstrs will cause problem to snapshot copying during
migration, because it uses ramblocks' idstr to identify ramblocks.
ram_save_setup {
…
RAMBLOCK_FOREACH(block) {
qemu_put_byte(f, strlen(block->idstr));
qemu_put_buffer(f, (uint8_t *)block->idstr,strlen(block->idstr));
qemu_put_be64(f, block->used_length);
}
…
}
ram_load() {
block = qemu_ram_block_by_name(id);
if (block) {
if (length != block->used_length) {
qemu_ram_resize(block, length, &local_err);
}
….
}
}
Therefore, in this patch,
vmstate_register_ram() is called for memory region of type ram ptr,
also a unique vfioid is assigned to vfio devices across source
and target vms.
e.g. in source vm, use qemu parameter
-device
vfio-pci,sysfsdev=/sys/bus/pci/devices/0000:00:02.0/
882cc4da-dede-11e7-9180-078a62063ab1,vfioid=igd
and in target vm, use qemu paramter
-device
vfio-pci,sysfsdev=/sys/bus/pci/devices/0000:00:02.0/
5ac1fb20-2bbf-4842-bb7e-36c58c3be9cd,vfioid=igd
Signed-off-by: Zhao Yan <yan.y.zhao@intel.com>
---
hw/vfio/pci.c | 8 +++++++-
include/hw/vfio/vfio-common.h | 1 +
memory.c | 4 ++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index c0cb1ec289..7bc2ed0752 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -2533,7 +2533,12 @@ static void vfio_populate_device(VFIOPCIDevice *vdev, Error **errp)
}
for (i = VFIO_PCI_BAR0_REGION_INDEX; i < VFIO_PCI_ROM_REGION_INDEX; i++) {
- char *name = g_strdup_printf("%s BAR %d", vbasedev->name, i);
+ char *name;
+ if (vbasedev->vfioid) {
+ name = g_strdup_printf("%s BAR %d", vbasedev->vfioid, i);
+ } else {
+ name = g_strdup_printf("%s BAR %d", vbasedev->name, i);
+ }
ret = vfio_region_setup(OBJECT(vdev), vbasedev,
&vdev->bars[i].region, i, name);
@@ -3180,6 +3185,7 @@ static void vfio_instance_init(Object *obj)
static Property vfio_pci_dev_properties[] = {
DEFINE_PROP_PCI_HOST_DEVADDR("host", VFIOPCIDevice, host),
DEFINE_PROP_STRING("sysfsdev", VFIOPCIDevice, vbasedev.sysfsdev),
+ DEFINE_PROP_STRING("vfioid", VFIOPCIDevice, vbasedev.vfioid),
DEFINE_PROP_ON_OFF_AUTO("display", VFIOPCIDevice,
display, ON_OFF_AUTO_OFF),
DEFINE_PROP_UINT32("x-intx-mmap-timeout-ms", VFIOPCIDevice,
diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h
index 1b434d02f6..84bab94f52 100644
--- a/include/hw/vfio/vfio-common.h
+++ b/include/hw/vfio/vfio-common.h
@@ -108,6 +108,7 @@ typedef struct VFIODevice {
struct VFIOGroup *group;
char *sysfsdev;
char *name;
+ char *vfioid;
DeviceState *dev;
int fd;
int type;
diff --git a/memory.c b/memory.c
index d14c6dec1d..dbb29fa989 100644
--- a/memory.c
+++ b/memory.c
@@ -1588,6 +1588,7 @@ void memory_region_init_ram_ptr(MemoryRegion *mr,
uint64_t size,
void *ptr)
{
+ DeviceState *owner_dev;
memory_region_init(mr, owner, name, size);
mr->ram = true;
mr->terminates = true;
@@ -1597,6 +1598,9 @@ void memory_region_init_ram_ptr(MemoryRegion *mr,
/* qemu_ram_alloc_from_ptr cannot fail with ptr != NULL. */
assert(ptr != NULL);
mr->ram_block = qemu_ram_alloc_from_ptr(size, ptr, mr, &error_fatal);
+
+ owner_dev = DEVICE(owner);
+ vmstate_register_ram(mr, owner_dev);
}
void memory_region_init_ram_device_ptr(MemoryRegion *mr,
--
2.17.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [Qemu-devel] [PATCH] vfio: assign idstr for VFIO's mmaped regions for migration
2019-01-08 6:03 [Qemu-devel] [PATCH] vfio: assign idstr for VFIO's mmaped regions for migration Zhao Yan
@ 2019-01-08 6:03 ` Zhao Yan
2019-01-08 17:09 ` Alex Williamson
0 siblings, 1 reply; 5+ messages in thread
From: Zhao Yan @ 2019-01-08 6:03 UTC (permalink / raw)
To: qemu-devel; +Cc: pbonzini, alex.williamson, Zhao Yan
if multiple regions in vfio are mmaped, their corresponding ramblocks
are like below, i.e. their idstrs are "".
(qemu) info ramblock
Block Name PSize Offset Used Total
pc.ram 4 KiB 0x0000000000000000 0x0000000020000000 0x0000000020000000
4 KiB 0x0000000021100000 0x0000000020000000 0x0000000020000000
4 KiB 0x0000000020900000 0x0000000000800000 0x0000000000800000
4 KiB 0x0000000020240000 0x0000000000687000 0x0000000000687000
4 KiB 0x00000000200c0000 0x0000000000178000 0x0000000000178000
pc.bios 4 KiB 0x0000000020000000 0x0000000000040000 0x0000000000040000
pc.rom 4 KiB 0x0000000020040000 0x0000000000020000 0x0000000000020000
This is because ramblocks' idstr are assigned by calling
vmstate_register_ram(), but memory region of type ram device ptr does not
call vmstate_register_ram().
vfio_region_mmap
|->memory_region_init_ram_device_ptr
|-> memory_region_init_ram_ptr
Without empty idstrs will cause problem to snapshot copying during
migration, because it uses ramblocks' idstr to identify ramblocks.
ram_save_setup {
…
RAMBLOCK_FOREACH(block) {
qemu_put_byte(f, strlen(block->idstr));
qemu_put_buffer(f, (uint8_t *)block->idstr,strlen(block->idstr));
qemu_put_be64(f, block->used_length);
}
…
}
ram_load() {
block = qemu_ram_block_by_name(id);
if (block) {
if (length != block->used_length) {
qemu_ram_resize(block, length, &local_err);
}
….
}
}
Therefore, in this patch,
vmstate_register_ram() is called for memory region of type ram ptr,
also a unique vfioid is assigned to vfio devices across source
and target vms.
e.g. in source vm, use qemu parameter
-device
vfio-pci,sysfsdev=/sys/bus/pci/devices/0000:00:02.0/
882cc4da-dede-11e7-9180-078a62063ab1,vfioid=igd
and in target vm, use qemu paramter
-device
vfio-pci,sysfsdev=/sys/bus/pci/devices/0000:00:02.0/
5ac1fb20-2bbf-4842-bb7e-36c58c3be9cd,vfioid=igd
Signed-off-by: Zhao Yan <yan.y.zhao@intel.com>
---
hw/vfio/pci.c | 8 +++++++-
include/hw/vfio/vfio-common.h | 1 +
memory.c | 4 ++++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index c0cb1ec289..7bc2ed0752 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -2533,7 +2533,12 @@ static void vfio_populate_device(VFIOPCIDevice *vdev, Error **errp)
}
for (i = VFIO_PCI_BAR0_REGION_INDEX; i < VFIO_PCI_ROM_REGION_INDEX; i++) {
- char *name = g_strdup_printf("%s BAR %d", vbasedev->name, i);
+ char *name;
+ if (vbasedev->vfioid) {
+ name = g_strdup_printf("%s BAR %d", vbasedev->vfioid, i);
+ } else {
+ name = g_strdup_printf("%s BAR %d", vbasedev->name, i);
+ }
ret = vfio_region_setup(OBJECT(vdev), vbasedev,
&vdev->bars[i].region, i, name);
@@ -3180,6 +3185,7 @@ static void vfio_instance_init(Object *obj)
static Property vfio_pci_dev_properties[] = {
DEFINE_PROP_PCI_HOST_DEVADDR("host", VFIOPCIDevice, host),
DEFINE_PROP_STRING("sysfsdev", VFIOPCIDevice, vbasedev.sysfsdev),
+ DEFINE_PROP_STRING("vfioid", VFIOPCIDevice, vbasedev.vfioid),
DEFINE_PROP_ON_OFF_AUTO("display", VFIOPCIDevice,
display, ON_OFF_AUTO_OFF),
DEFINE_PROP_UINT32("x-intx-mmap-timeout-ms", VFIOPCIDevice,
diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h
index 1b434d02f6..84bab94f52 100644
--- a/include/hw/vfio/vfio-common.h
+++ b/include/hw/vfio/vfio-common.h
@@ -108,6 +108,7 @@ typedef struct VFIODevice {
struct VFIOGroup *group;
char *sysfsdev;
char *name;
+ char *vfioid;
DeviceState *dev;
int fd;
int type;
diff --git a/memory.c b/memory.c
index d14c6dec1d..dbb29fa989 100644
--- a/memory.c
+++ b/memory.c
@@ -1588,6 +1588,7 @@ void memory_region_init_ram_ptr(MemoryRegion *mr,
uint64_t size,
void *ptr)
{
+ DeviceState *owner_dev;
memory_region_init(mr, owner, name, size);
mr->ram = true;
mr->terminates = true;
@@ -1597,6 +1598,9 @@ void memory_region_init_ram_ptr(MemoryRegion *mr,
/* qemu_ram_alloc_from_ptr cannot fail with ptr != NULL. */
assert(ptr != NULL);
mr->ram_block = qemu_ram_alloc_from_ptr(size, ptr, mr, &error_fatal);
+
+ owner_dev = DEVICE(owner);
+ vmstate_register_ram(mr, owner_dev);
}
void memory_region_init_ram_device_ptr(MemoryRegion *mr,
--
2.17.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [Qemu-devel] [PATCH] vfio: assign idstr for VFIO's mmaped regions for migration
2019-01-08 6:03 ` Zhao Yan
@ 2019-01-08 17:09 ` Alex Williamson
2019-01-10 1:19 ` Zhao Yan
0 siblings, 1 reply; 5+ messages in thread
From: Alex Williamson @ 2019-01-08 17:09 UTC (permalink / raw)
To: Zhao Yan; +Cc: qemu-devel, pbonzini
On Tue, 8 Jan 2019 01:03:48 -0500
Zhao Yan <yan.y.zhao@intel.com> wrote:
> if multiple regions in vfio are mmaped, their corresponding ramblocks
> are like below, i.e. their idstrs are "".
>
> (qemu) info ramblock
> Block Name PSize Offset Used Total
> pc.ram 4 KiB 0x0000000000000000 0x0000000020000000 0x0000000020000000
> 4 KiB 0x0000000021100000 0x0000000020000000 0x0000000020000000
> 4 KiB 0x0000000020900000 0x0000000000800000 0x0000000000800000
> 4 KiB 0x0000000020240000 0x0000000000687000 0x0000000000687000
> 4 KiB 0x00000000200c0000 0x0000000000178000 0x0000000000178000
> pc.bios 4 KiB 0x0000000020000000 0x0000000000040000 0x0000000000040000
> pc.rom 4 KiB 0x0000000020040000 0x0000000000020000 0x0000000000020000
>
> This is because ramblocks' idstr are assigned by calling
> vmstate_register_ram(), but memory region of type ram device ptr does not
> call vmstate_register_ram().
> vfio_region_mmap
> |->memory_region_init_ram_device_ptr
> |-> memory_region_init_ram_ptr
>
> Without empty idstrs will cause problem to snapshot copying during
> migration, because it uses ramblocks' idstr to identify ramblocks.
> ram_save_setup {
> …
> RAMBLOCK_FOREACH(block) {
> qemu_put_byte(f, strlen(block->idstr));
> qemu_put_buffer(f, (uint8_t *)block->idstr,strlen(block->idstr));
> qemu_put_be64(f, block->used_length);
> }
> …
> }
> ram_load() {
> block = qemu_ram_block_by_name(id);
> if (block) {
> if (length != block->used_length) {
> qemu_ram_resize(block, length, &local_err);
> }
> ….
> }
> }
>
> Therefore, in this patch,
> vmstate_register_ram() is called for memory region of type ram ptr,
> also a unique vfioid is assigned to vfio devices across source
> and target vms.
> e.g. in source vm, use qemu parameter
> -device
> vfio-pci,sysfsdev=/sys/bus/pci/devices/0000:00:02.0/
> 882cc4da-dede-11e7-9180-078a62063ab1,vfioid=igd
>
> and in target vm, use qemu paramter
> -device
> vfio-pci,sysfsdev=/sys/bus/pci/devices/0000:00:02.0/
> 5ac1fb20-2bbf-4842-bb7e-36c58c3be9cd,vfioid=igd
Why wouldn't we just use the id= (DeviceState.id) value instead of
adding yet another one? I can't imagine anyone, especially libvirt,
wants to deal with a vfio specific id for a device.
> Signed-off-by: Zhao Yan <yan.y.zhao@intel.com>
> ---
> hw/vfio/pci.c | 8 +++++++-
> include/hw/vfio/vfio-common.h | 1 +
> memory.c | 4 ++++
> 3 files changed, 12 insertions(+), 1 deletion(-)
>
> diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
> index c0cb1ec289..7bc2ed0752 100644
> --- a/hw/vfio/pci.c
> +++ b/hw/vfio/pci.c
> @@ -2533,7 +2533,12 @@ static void vfio_populate_device(VFIOPCIDevice *vdev, Error **errp)
> }
>
> for (i = VFIO_PCI_BAR0_REGION_INDEX; i < VFIO_PCI_ROM_REGION_INDEX; i++) {
> - char *name = g_strdup_printf("%s BAR %d", vbasedev->name, i);
> + char *name;
> + if (vbasedev->vfioid) {
> + name = g_strdup_printf("%s BAR %d", vbasedev->vfioid, i);
> + } else {
> + name = g_strdup_printf("%s BAR %d", vbasedev->name, i);
> + }
>
> ret = vfio_region_setup(OBJECT(vdev), vbasedev,
> &vdev->bars[i].region, i, name);
> @@ -3180,6 +3185,7 @@ static void vfio_instance_init(Object *obj)
> static Property vfio_pci_dev_properties[] = {
> DEFINE_PROP_PCI_HOST_DEVADDR("host", VFIOPCIDevice, host),
> DEFINE_PROP_STRING("sysfsdev", VFIOPCIDevice, vbasedev.sysfsdev),
> + DEFINE_PROP_STRING("vfioid", VFIOPCIDevice, vbasedev.vfioid),
> DEFINE_PROP_ON_OFF_AUTO("display", VFIOPCIDevice,
> display, ON_OFF_AUTO_OFF),
> DEFINE_PROP_UINT32("x-intx-mmap-timeout-ms", VFIOPCIDevice,
> diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h
> index 1b434d02f6..84bab94f52 100644
> --- a/include/hw/vfio/vfio-common.h
> +++ b/include/hw/vfio/vfio-common.h
> @@ -108,6 +108,7 @@ typedef struct VFIODevice {
> struct VFIOGroup *group;
> char *sysfsdev;
> char *name;
> + char *vfioid;
> DeviceState *dev;
> int fd;
> int type;
> diff --git a/memory.c b/memory.c
> index d14c6dec1d..dbb29fa989 100644
> --- a/memory.c
> +++ b/memory.c
> @@ -1588,6 +1588,7 @@ void memory_region_init_ram_ptr(MemoryRegion *mr,
> uint64_t size,
> void *ptr)
> {
> + DeviceState *owner_dev;
> memory_region_init(mr, owner, name, size);
> mr->ram = true;
> mr->terminates = true;
> @@ -1597,6 +1598,9 @@ void memory_region_init_ram_ptr(MemoryRegion *mr,
> /* qemu_ram_alloc_from_ptr cannot fail with ptr != NULL. */
> assert(ptr != NULL);
> mr->ram_block = qemu_ram_alloc_from_ptr(size, ptr, mr, &error_fatal);
> +
> + owner_dev = DEVICE(owner);
> + vmstate_register_ram(mr, owner_dev);
Where does the corresponding vmstate_unregister_ram() call occur when
unplugged? Thanks,
Alex
> }
>
> void memory_region_init_ram_device_ptr(MemoryRegion *mr,
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Qemu-devel] [PATCH] vfio: assign idstr for VFIO's mmaped regions for migration
2019-01-08 17:09 ` Alex Williamson
@ 2019-01-10 1:19 ` Zhao Yan
2019-02-20 11:17 ` Gonglei (Arei)
0 siblings, 1 reply; 5+ messages in thread
From: Zhao Yan @ 2019-01-10 1:19 UTC (permalink / raw)
To: Alex Williamson; +Cc: qemu-devel, pbonzini
On Tue, Jan 08, 2019 at 10:09:11AM -0700, Alex Williamson wrote:
> On Tue, 8 Jan 2019 01:03:48 -0500
> Zhao Yan <yan.y.zhao@intel.com> wrote:
>
> > if multiple regions in vfio are mmaped, their corresponding ramblocks
> > are like below, i.e. their idstrs are "".
> >
> > (qemu) info ramblock
> > Block Name PSize Offset Used Total
> > pc.ram 4 KiB 0x0000000000000000 0x0000000020000000 0x0000000020000000
> > 4 KiB 0x0000000021100000 0x0000000020000000 0x0000000020000000
> > 4 KiB 0x0000000020900000 0x0000000000800000 0x0000000000800000
> > 4 KiB 0x0000000020240000 0x0000000000687000 0x0000000000687000
> > 4 KiB 0x00000000200c0000 0x0000000000178000 0x0000000000178000
> > pc.bios 4 KiB 0x0000000020000000 0x0000000000040000 0x0000000000040000
> > pc.rom 4 KiB 0x0000000020040000 0x0000000000020000 0x0000000000020000
> >
> > This is because ramblocks' idstr are assigned by calling
> > vmstate_register_ram(), but memory region of type ram device ptr does not
> > call vmstate_register_ram().
> > vfio_region_mmap
> > |->memory_region_init_ram_device_ptr
> > |-> memory_region_init_ram_ptr
> >
> > Without empty idstrs will cause problem to snapshot copying during
> > migration, because it uses ramblocks' idstr to identify ramblocks.
> > ram_save_setup {
> > …
> > RAMBLOCK_FOREACH(block) {
> > qemu_put_byte(f, strlen(block->idstr));
> > qemu_put_buffer(f, (uint8_t *)block->idstr,strlen(block->idstr));
> > qemu_put_be64(f, block->used_length);
> > }
> > …
> > }
> > ram_load() {
> > block = qemu_ram_block_by_name(id);
> > if (block) {
> > if (length != block->used_length) {
> > qemu_ram_resize(block, length, &local_err);
> > }
> > ….
> > }
> > }
> >
> > Therefore, in this patch,
> > vmstate_register_ram() is called for memory region of type ram ptr,
> > also a unique vfioid is assigned to vfio devices across source
> > and target vms.
> > e.g. in source vm, use qemu parameter
> > -device
> > vfio-pci,sysfsdev=/sys/bus/pci/devices/0000:00:02.0/
> > 882cc4da-dede-11e7-9180-078a62063ab1,vfioid=igd
> >
> > and in target vm, use qemu paramter
> > -device
> > vfio-pci,sysfsdev=/sys/bus/pci/devices/0000:00:02.0/
> > 5ac1fb20-2bbf-4842-bb7e-36c58c3be9cd,vfioid=igd
>
> Why wouldn't we just use the id= (DeviceState.id) value instead of
> adding yet another one? I can't imagine anyone, especially libvirt,
> wants to deal with a vfio specific id for a device.
>
hi Alex
You are right! DeviceState.id can be used here. Thanks for your suggestion.
> > Signed-off-by: Zhao Yan <yan.y.zhao@intel.com>
> > ---
> > hw/vfio/pci.c | 8 +++++++-
> > include/hw/vfio/vfio-common.h | 1 +
> > memory.c | 4 ++++
> > 3 files changed, 12 insertions(+), 1 deletion(-)
> >
> > diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
> > index c0cb1ec289..7bc2ed0752 100644
> > --- a/hw/vfio/pci.c
> > +++ b/hw/vfio/pci.c
> > @@ -2533,7 +2533,12 @@ static void vfio_populate_device(VFIOPCIDevice *vdev, Error **errp)
> > }
> >
> > for (i = VFIO_PCI_BAR0_REGION_INDEX; i < VFIO_PCI_ROM_REGION_INDEX; i++) {
> > - char *name = g_strdup_printf("%s BAR %d", vbasedev->name, i);
> > + char *name;
> > + if (vbasedev->vfioid) {
> > + name = g_strdup_printf("%s BAR %d", vbasedev->vfioid, i);
> > + } else {
> > + name = g_strdup_printf("%s BAR %d", vbasedev->name, i);
> > + }
> >
> > ret = vfio_region_setup(OBJECT(vdev), vbasedev,
> > &vdev->bars[i].region, i, name);
> > @@ -3180,6 +3185,7 @@ static void vfio_instance_init(Object *obj)
> > static Property vfio_pci_dev_properties[] = {
> > DEFINE_PROP_PCI_HOST_DEVADDR("host", VFIOPCIDevice, host),
> > DEFINE_PROP_STRING("sysfsdev", VFIOPCIDevice, vbasedev.sysfsdev),
> > + DEFINE_PROP_STRING("vfioid", VFIOPCIDevice, vbasedev.vfioid),
> > DEFINE_PROP_ON_OFF_AUTO("display", VFIOPCIDevice,
> > display, ON_OFF_AUTO_OFF),
> > DEFINE_PROP_UINT32("x-intx-mmap-timeout-ms", VFIOPCIDevice,
> > diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h
> > index 1b434d02f6..84bab94f52 100644
> > --- a/include/hw/vfio/vfio-common.h
> > +++ b/include/hw/vfio/vfio-common.h
> > @@ -108,6 +108,7 @@ typedef struct VFIODevice {
> > struct VFIOGroup *group;
> > char *sysfsdev;
> > char *name;
> > + char *vfioid;
> > DeviceState *dev;
> > int fd;
> > int type;
> > diff --git a/memory.c b/memory.c
> > index d14c6dec1d..dbb29fa989 100644
> > --- a/memory.c
> > +++ b/memory.c
> > @@ -1588,6 +1588,7 @@ void memory_region_init_ram_ptr(MemoryRegion *mr,
> > uint64_t size,
> > void *ptr)
> > {
> > + DeviceState *owner_dev;
> > memory_region_init(mr, owner, name, size);
> > mr->ram = true;
> > mr->terminates = true;
> > @@ -1597,6 +1598,9 @@ void memory_region_init_ram_ptr(MemoryRegion *mr,
> > /* qemu_ram_alloc_from_ptr cannot fail with ptr != NULL. */
> > assert(ptr != NULL);
> > mr->ram_block = qemu_ram_alloc_from_ptr(size, ptr, mr, &error_fatal);
> > +
> > + owner_dev = DEVICE(owner);
> > + vmstate_register_ram(mr, owner_dev);
>
> Where does the corresponding vmstate_unregister_ram() call occur when
> unplugged? Thanks,
>
sorry, I just updated my qemu code base and found that in migration/ram.c
now it will not save/restore ramblocks who do not call
vmstate_regitser_ram().
therefore, the vmstate_register_ram() may not be necessary for memory
region mapped to device resources, as it's better to save/restore that part
of memory from vendor driver side.
So, do you think it's ok to just call qemu_ram_set_idstr() to set idstr for
ramblocks of mmaped region?
Thanks
Yan
> Alex
>
> > }
> >
> > void memory_region_init_ram_device_ptr(MemoryRegion *mr,
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Qemu-devel] [PATCH] vfio: assign idstr for VFIO's mmaped regions for migration
2019-01-10 1:19 ` Zhao Yan
@ 2019-02-20 11:17 ` Gonglei (Arei)
0 siblings, 0 replies; 5+ messages in thread
From: Gonglei (Arei) @ 2019-02-20 11:17 UTC (permalink / raw)
To: Zhao Yan, Alex Williamson; +Cc: pbonzini, qemu-devel
> -----Original Message-----
> From: Qemu-devel
> [mailto:qemu-devel-bounces+arei.gonglei=huawei.com@nongnu.org] On
> Behalf Of Zhao Yan
> Sent: Thursday, January 10, 2019 9:19 AM
> To: Alex Williamson <alex.williamson@redhat.com>
> Cc: pbonzini@redhat.com; qemu-devel@nongnu.org
> Subject: Re: [Qemu-devel] [PATCH] vfio: assign idstr for VFIO's mmaped regions
> for migration
>
> On Tue, Jan 08, 2019 at 10:09:11AM -0700, Alex Williamson wrote:
> > On Tue, 8 Jan 2019 01:03:48 -0500
> > Zhao Yan <yan.y.zhao@intel.com> wrote:
> >
> > > if multiple regions in vfio are mmaped, their corresponding ramblocks
> > > are like below, i.e. their idstrs are "".
> > >
> > > (qemu) info ramblock
> > > Block Name PSize Offset Used
> Total
> > > pc.ram 4 KiB 0x0000000000000000 0x0000000020000000
> 0x0000000020000000
> > > 4 KiB 0x0000000021100000 0x0000000020000000
> 0x0000000020000000
> > > 4 KiB 0x0000000020900000 0x0000000000800000
> 0x0000000000800000
> > > 4 KiB 0x0000000020240000 0x0000000000687000
> 0x0000000000687000
> > > 4 KiB 0x00000000200c0000 0x0000000000178000
> 0x0000000000178000
> > > pc.bios 4 KiB 0x0000000020000000 0x0000000000040000
> 0x0000000000040000
> > > pc.rom 4 KiB 0x0000000020040000 0x0000000000020000
> 0x0000000000020000
> > >
> > > This is because ramblocks' idstr are assigned by calling
> > > vmstate_register_ram(), but memory region of type ram device ptr does
> not
> > > call vmstate_register_ram().
> > > vfio_region_mmap
> > > |->memory_region_init_ram_device_ptr
> > > |-> memory_region_init_ram_ptr
> > >
> > > Without empty idstrs will cause problem to snapshot copying during
> > > migration, because it uses ramblocks' idstr to identify ramblocks.
> > > ram_save_setup {
> > > …
> > > RAMBLOCK_FOREACH(block) {
> > > qemu_put_byte(f, strlen(block->idstr));
> > > qemu_put_buffer(f, (uint8_t *)block->idstr,strlen(block->idstr));
> > > qemu_put_be64(f, block->used_length);
> > > }
> > > …
> > > }
> > > ram_load() {
> > > block = qemu_ram_block_by_name(id);
> > > if (block) {
> > > if (length != block->used_length) {
> > > qemu_ram_resize(block, length, &local_err);
> > > }
> > > ….
> > > }
> > > }
> > >
> > > Therefore, in this patch,
> > > vmstate_register_ram() is called for memory region of type ram ptr,
> > > also a unique vfioid is assigned to vfio devices across source
> > > and target vms.
> > > e.g. in source vm, use qemu parameter
> > > -device
> > > vfio-pci,sysfsdev=/sys/bus/pci/devices/0000:00:02.0/
> > > 882cc4da-dede-11e7-9180-078a62063ab1,vfioid=igd
> > >
> > > and in target vm, use qemu paramter
> > > -device
> > > vfio-pci,sysfsdev=/sys/bus/pci/devices/0000:00:02.0/
> > > 5ac1fb20-2bbf-4842-bb7e-36c58c3be9cd,vfioid=igd
> >
> > Why wouldn't we just use the id= (DeviceState.id) value instead of
> > adding yet another one? I can't imagine anyone, especially libvirt,
> > wants to deal with a vfio specific id for a device.
> >
> hi Alex
> You are right! DeviceState.id can be used here. Thanks for your suggestion.
>
Then Libvirt and/or Nova need to keep the device id unchanged.
>
> > > Signed-off-by: Zhao Yan <yan.y.zhao@intel.com>
> > > ---
> > > hw/vfio/pci.c | 8 +++++++-
> > > include/hw/vfio/vfio-common.h | 1 +
> > > memory.c | 4 ++++
> > > 3 files changed, 12 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
> > > index c0cb1ec289..7bc2ed0752 100644
> > > --- a/hw/vfio/pci.c
> > > +++ b/hw/vfio/pci.c
> > > @@ -2533,7 +2533,12 @@ static void vfio_populate_device(VFIOPCIDevice
> *vdev, Error **errp)
> > > }
> > >
> > > for (i = VFIO_PCI_BAR0_REGION_INDEX; i <
> VFIO_PCI_ROM_REGION_INDEX; i++) {
> > > - char *name = g_strdup_printf("%s BAR %d", vbasedev->name, i);
> > > + char *name;
> > > + if (vbasedev->vfioid) {
> > > + name = g_strdup_printf("%s BAR %d", vbasedev->vfioid, i);
> > > + } else {
> > > + name = g_strdup_printf("%s BAR %d", vbasedev->name, i);
> > > + }
> > >
> > > ret = vfio_region_setup(OBJECT(vdev), vbasedev,
> > > &vdev->bars[i].region, i, name);
> > > @@ -3180,6 +3185,7 @@ static void vfio_instance_init(Object *obj)
> > > static Property vfio_pci_dev_properties[] = {
> > > DEFINE_PROP_PCI_HOST_DEVADDR("host", VFIOPCIDevice, host),
> > > DEFINE_PROP_STRING("sysfsdev", VFIOPCIDevice,
> vbasedev.sysfsdev),
> > > + DEFINE_PROP_STRING("vfioid", VFIOPCIDevice, vbasedev.vfioid),
> > > DEFINE_PROP_ON_OFF_AUTO("display", VFIOPCIDevice,
> > > display, ON_OFF_AUTO_OFF),
> > > DEFINE_PROP_UINT32("x-intx-mmap-timeout-ms", VFIOPCIDevice,
> > > diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h
> > > index 1b434d02f6..84bab94f52 100644
> > > --- a/include/hw/vfio/vfio-common.h
> > > +++ b/include/hw/vfio/vfio-common.h
> > > @@ -108,6 +108,7 @@ typedef struct VFIODevice {
> > > struct VFIOGroup *group;
> > > char *sysfsdev;
> > > char *name;
> > > + char *vfioid;
> > > DeviceState *dev;
> > > int fd;
> > > int type;
> > > diff --git a/memory.c b/memory.c
> > > index d14c6dec1d..dbb29fa989 100644
> > > --- a/memory.c
> > > +++ b/memory.c
> > > @@ -1588,6 +1588,7 @@ void
> memory_region_init_ram_ptr(MemoryRegion *mr,
> > > uint64_t size,
> > > void *ptr)
> > > {
> > > + DeviceState *owner_dev;
> > > memory_region_init(mr, owner, name, size);
> > > mr->ram = true;
> > > mr->terminates = true;
> > > @@ -1597,6 +1598,9 @@ void
> memory_region_init_ram_ptr(MemoryRegion *mr,
> > > /* qemu_ram_alloc_from_ptr cannot fail with ptr != NULL. */
> > > assert(ptr != NULL);
> > > mr->ram_block = qemu_ram_alloc_from_ptr(size, ptr, mr,
> &error_fatal);
> > > +
> > > + owner_dev = DEVICE(owner);
> > > + vmstate_register_ram(mr, owner_dev);
> >
> > Where does the corresponding vmstate_unregister_ram() call occur when
> > unplugged? Thanks,
> >
> sorry, I just updated my qemu code base and found that in migration/ram.c
> now it will not save/restore ramblocks who do not call
> vmstate_regitser_ram().
> therefore, the vmstate_register_ram() may not be necessary for memory
> region mapped to device resources, as it's better to save/restore that part
> of memory from vendor driver side.
> So, do you think it's ok to just call qemu_ram_set_idstr() to set idstr for
> ramblocks of mmaped region?
>
> Thanks
> Yan
>
Why not invoking vmstate_register_ram() in vfio_region_mmap and
Invoking vmstate_unregister_ram() in vfio_region_exit?
Regards,
-Gonglei
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2019-02-20 11:18 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-01-08 6:03 [Qemu-devel] [PATCH] vfio: assign idstr for VFIO's mmaped regions for migration Zhao Yan
2019-01-08 6:03 ` Zhao Yan
2019-01-08 17:09 ` Alex Williamson
2019-01-10 1:19 ` Zhao Yan
2019-02-20 11:17 ` Gonglei (Arei)
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.