[Qemu-devel] qemu memory manage question

[Qemu-devel] qemu memory manage question

[jack.chen]

hello,I have some questions about memory allocation in qemu for
virtual machine.I found when configure_accelerator function was called
,memory slots  were registered to KVM,but at that time address_space
have not been initialized and ram have not been allocated,it is really
confused me,Thanks a lot!!

Re: [Qemu-devel] qemu memory manage question

[Peter Xu]

On Mon, Apr 17, 2017 at 06:09:11PM +0800, jack.chen wrote:
> hello,I have some questions about memory allocation in qemu for
> virtual machine.I found when configure_accelerator function was called
> ,memory slots  were registered to KVM,but at that time address_space
> have not been initialized and ram have not been allocated,it is really
> confused me,Thanks a lot!!

Here's how I understand it...

configure_accelerator() does not register memory slots in KVM.
Instead, it registers memory listeners. See
kvm_memory_listener_register(), especially:

    kml->listener.region_add = kvm_region_add;

That's the hook function to be called when there are new memory region
added to the system.

Further, when RAM is initialzed, it'll modify the address space layout
of system_memory, and the registered listener of KVM (kvm_region_add)
will be invoked, it'll further sync with kvm. It should be in the
following path if you break at kvm_region_add in gdb:

#0  0x00005555557ba13a in kvm_region_add (listener=0x5555568330c0, section=0x7fffffffd310) at /root/git/qemu/kvm-all.c:859
#1  0x00005555557c1910 in address_space_update_topology_pass (as=0x55555629e240 <address_space_memory>, old_view=0x5555567a7090, new_view=0x5555568d3460, adding=true) at /root/git/qemu/memory.c:871
#2  0x00005555557c19f3 in address_space_update_topology (as=0x55555629e240 <address_space_memory>) at /root/git/qemu/memory.c:886
#3  0x00005555557c1b41 in memory_region_transaction_commit () at /root/git/qemu/memory.c:922
#4  0x00005555557c4bfd in memory_region_update_container_subregions (subregion=0x5555568d2fc0) at /root/git/qemu/memory.c:2075
#5  0x00005555557c4c64 in memory_region_add_subregion_common (mr=0x5555567a5830, offset=0, subregion=0x5555568d2fc0) at /root/git/qemu/memory.c:2085
#6  0x00005555557c4ca0 in memory_region_add_subregion (mr=0x5555567a5830, offset=0, subregion=0x5555568d2fc0) at /root/git/qemu/memory.c:2093
#7  0x000055555583fd68 in pc_memory_init (pcms=0x5555567a4100, system_memory=0x5555567a5830, rom_memory=0x5555568d21a0, ram_memory=0x7fffffffd550) at /root/git/qemu/hw/i386/pc.c:1383
#8  0x0000555555847363 in pc_q35_init (machine=0x5555567a4100) at /root/git/qemu/hw/i386/pc_q35.c:147
#9  0x0000555555847cac in pc_init_v2_9 (machine=0x5555567a4100) at /root/git/qemu/hw/i386/pc_q35.c:310
#10 0x00005555558f7cf8 in main (argc=11, argv=0x7fffffffda78, envp=0x7fffffffdad8) at /root/git/qemu/vl.c:4557

Hope this helps. Thanks.

-- 
Peter Xu

Re: [Qemu-devel] qemu memory manage question

[jack.chen]

Thanks,from the path you have list to me,it can be  well explained,but
according to the source code,in the end of
kvm_init,kvm_memory_listener and  kvm_io_listener were registered by
memory_listener_register(),and  in the end of
memory_listener_register(),listener_add_address_space() was called for
each address_space,so the listener->region_add was executed then.I do
not know what mistake I have made,can you explain it to me ?? thank
you very much!

2017-04-17 18:26 GMT+08:00 Peter Xu <peterx@redhat.com>:
> On Mon, Apr 17, 2017 at 06:09:11PM +0800, jack.chen wrote:
>> hello,I have some questions about memory allocation in qemu for
>> virtual machine.I found when configure_accelerator function was called
>> ,memory slots  were registered to KVM,but at that time address_space
>> have not been initialized and ram have not been allocated,it is really
>> confused me,Thanks a lot!!
>
> Here's how I understand it...
>
> configure_accelerator() does not register memory slots in KVM.
> Instead, it registers memory listeners. See
> kvm_memory_listener_register(), especially:
>
>     kml->listener.region_add = kvm_region_add;
>
> That's the hook function to be called when there are new memory region
> added to the system.
>
> Further, when RAM is initialzed, it'll modify the address space layout
> of system_memory, and the registered listener of KVM (kvm_region_add)
> will be invoked, it'll further sync with kvm. It should be in the
> following path if you break at kvm_region_add in gdb:
>
> #0  0x00005555557ba13a in kvm_region_add (listener=0x5555568330c0, section=0x7fffffffd310) at /root/git/qemu/kvm-all.c:859
> #1  0x00005555557c1910 in address_space_update_topology_pass (as=0x55555629e240 <address_space_memory>, old_view=0x5555567a7090, new_view=0x5555568d3460, adding=true) at /root/git/qemu/memory.c:871
> #2  0x00005555557c19f3 in address_space_update_topology (as=0x55555629e240 <address_space_memory>) at /root/git/qemu/memory.c:886
> #3  0x00005555557c1b41 in memory_region_transaction_commit () at /root/git/qemu/memory.c:922
> #4  0x00005555557c4bfd in memory_region_update_container_subregions (subregion=0x5555568d2fc0) at /root/git/qemu/memory.c:2075
> #5  0x00005555557c4c64 in memory_region_add_subregion_common (mr=0x5555567a5830, offset=0, subregion=0x5555568d2fc0) at /root/git/qemu/memory.c:2085
> #6  0x00005555557c4ca0 in memory_region_add_subregion (mr=0x5555567a5830, offset=0, subregion=0x5555568d2fc0) at /root/git/qemu/memory.c:2093
> #7  0x000055555583fd68 in pc_memory_init (pcms=0x5555567a4100, system_memory=0x5555567a5830, rom_memory=0x5555568d21a0, ram_memory=0x7fffffffd550) at /root/git/qemu/hw/i386/pc.c:1383
> #8  0x0000555555847363 in pc_q35_init (machine=0x5555567a4100) at /root/git/qemu/hw/i386/pc_q35.c:147
> #9  0x0000555555847cac in pc_init_v2_9 (machine=0x5555567a4100) at /root/git/qemu/hw/i386/pc_q35.c:310
> #10 0x00005555558f7cf8 in main (argc=11, argv=0x7fffffffda78, envp=0x7fffffffdad8) at /root/git/qemu/vl.c:4557
>
> Hope this helps. Thanks.
>
> --
> Peter Xu

Re: [Qemu-devel] qemu memory manage question

[李强]

> -----Original Message-----
> From: Qemu-devel
> [mailto:qemu-devel-bounces+liqiang6-s=360.cn@nongnu.org] On Behalf Of
> jack.chen
> Sent: Monday, April 17, 2017 6:56 PM
> To: Peter Xu
> Cc: qemu
> Subject: Re: [Qemu-devel] qemu memory manage question
> 
> Thanks,from the path you have list to me,it can be  well explained,but
> according to the source code,in the end of kvm_init,kvm_memory_listener and
> kvm_io_listener were registered by memory_listener_register(),and  in the
> end of
> memory_listener_register(),listener_add_address_space() was called for each
> address_space,so the listener->region_add was executed then.I do not know
> what mistake I have made,can you explain it to me ?? thank you very much!
> 

They are callbacks.
Every change of memory topology will call these listeners, add subregion(Peter's example),
modify the property of memory, create address space for example. 

Thanks.

--
Li Qiang /the Gear Team, Qihoo 360 Inc


> 2017-04-17 18:26 GMT+08:00 Peter Xu <peterx@redhat.com>:
> > On Mon, Apr 17, 2017 at 06:09:11PM +0800, jack.chen wrote:
> >> hello,I have some questions about memory allocation in qemu for
> >> virtual machine.I found when configure_accelerator function was
> >> called ,memory slots  were registered to KVM,but at that time
> >> address_space have not been initialized and ram have not been
> >> allocated,it is really confused me,Thanks a lot!!
> >
> > Here's how I understand it...
> >
> > configure_accelerator() does not register memory slots in KVM.
> > Instead, it registers memory listeners. See
> > kvm_memory_listener_register(), especially:
> >
> >     kml->listener.region_add = kvm_region_add;
> >
> > That's the hook function to be called when there are new memory region
> > added to the system.
> >
> > Further, when RAM is initialzed, it'll modify the address space layout
> > of system_memory, and the registered listener of KVM (kvm_region_add)
> > will be invoked, it'll further sync with kvm. It should be in the
> > following path if you break at kvm_region_add in gdb:
> >
> > #0  0x00005555557ba13a in kvm_region_add (listener=0x5555568330c0,
> > section=0x7fffffffd310) at /root/git/qemu/kvm-all.c:859
> > #1  0x00005555557c1910 in address_space_update_topology_pass
> > (as=0x55555629e240 <address_space_memory>,
> old_view=0x5555567a7090,
> > new_view=0x5555568d3460, adding=true) at /root/git/qemu/memory.c:871
> > #2  0x00005555557c19f3 in address_space_update_topology
> > (as=0x55555629e240 <address_space_memory>) at
> > /root/git/qemu/memory.c:886
> > #3  0x00005555557c1b41 in memory_region_transaction_commit () at
> > /root/git/qemu/memory.c:922
> > #4  0x00005555557c4bfd in memory_region_update_container_subregions
> > (subregion=0x5555568d2fc0) at /root/git/qemu/memory.c:2075
> > #5  0x00005555557c4c64 in memory_region_add_subregion_common
> > (mr=0x5555567a5830, offset=0, subregion=0x5555568d2fc0) at
> > /root/git/qemu/memory.c:2085
> > #6  0x00005555557c4ca0 in memory_region_add_subregion
> > (mr=0x5555567a5830, offset=0, subregion=0x5555568d2fc0) at
> > /root/git/qemu/memory.c:2093
> > #7  0x000055555583fd68 in pc_memory_init (pcms=0x5555567a4100,
> > system_memory=0x5555567a5830, rom_memory=0x5555568d21a0,
> > ram_memory=0x7fffffffd550) at /root/git/qemu/hw/i386/pc.c:1383
> > #8  0x0000555555847363 in pc_q35_init (machine=0x5555567a4100) at
> > /root/git/qemu/hw/i386/pc_q35.c:147
> > #9  0x0000555555847cac in pc_init_v2_9 (machine=0x5555567a4100) at
> > /root/git/qemu/hw/i386/pc_q35.c:310
> > #10 0x00005555558f7cf8 in main (argc=11, argv=0x7fffffffda78,
> > envp=0x7fffffffdad8) at /root/git/qemu/vl.c:4557
> >
> > Hope this helps. Thanks.
> >
> > --
> > Peter Xu

Re: [Qemu-devel] qemu memory manage question

[Peter Xu]

On Mon, Apr 17, 2017 at 06:55:40PM +0800, jack.chen wrote:
> Thanks,from the path you have list to me,it can be  well explained,but
> according to the source code,in the end of
> kvm_init,kvm_memory_listener and  kvm_io_listener were registered by
> memory_listener_register(),and  in the end of
> memory_listener_register(),listener_add_address_space() was called for
> each address_space,so the listener->region_add was executed then.I do
> not know what mistake I have made,can you explain it to me ?? thank
> you very much!

Firstly, there are two address spaces for each system:

AddressSpace address_space_io;
AddressSpace address_space_memory;

The one you mentioned (kvm_io_listener) should be the listener for
address_space_io, not address_space_memory, while for RAM, it's
located on address_space_memory space.

Secondly, when calling memory_listener_register() in kvm_init(),
region_add() is actually not called, since FOR_EACH_FLAT_RANGE() in
listener_add_address_space() is actually looping nothing since at that
time the address_space_memory may not ever been rendered at all
(rendering should be triggered by memory_region_transaction_commit()
-> address_space_update_topology()).

Thanks,

-- 
Peter Xu

Re: [Qemu-devel] qemu memory manage question

[jack.chen]

Thanks very much!!

2017-04-17 19:19 GMT+08:00 李强 <liqiang6-s@360.cn>:
>
>
>> -----Original Message-----
>> From: Qemu-devel
>> [mailto:qemu-devel-bounces+liqiang6-s=360.cn@nongnu.org] On Behalf Of
>> jack.chen
>> Sent: Monday, April 17, 2017 6:56 PM
>> To: Peter Xu
>> Cc: qemu
>> Subject: Re: [Qemu-devel] qemu memory manage question
>>
>> Thanks,from the path you have list to me,it can be  well explained,but
>> according to the source code,in the end of kvm_init,kvm_memory_listener and
>> kvm_io_listener were registered by memory_listener_register(),and  in the
>> end of
>> memory_listener_register(),listener_add_address_space() was called for each
>> address_space,so the listener->region_add was executed then.I do not know
>> what mistake I have made,can you explain it to me ?? thank you very much!
>>
>
> They are callbacks.
> Every change of memory topology will call these listeners, add subregion(Peter's example),
> modify the property of memory, create address space for example.
>
> Thanks.
>
> --
> Li Qiang /the Gear Team, Qihoo 360 Inc
>
>
>> 2017-04-17 18:26 GMT+08:00 Peter Xu <peterx@redhat.com>:
>> > On Mon, Apr 17, 2017 at 06:09:11PM +0800, jack.chen wrote:
>> >> hello,I have some questions about memory allocation in qemu for
>> >> virtual machine.I found when configure_accelerator function was
>> >> called ,memory slots  were registered to KVM,but at that time
>> >> address_space have not been initialized and ram have not been
>> >> allocated,it is really confused me,Thanks a lot!!
>> >
>> > Here's how I understand it...
>> >
>> > configure_accelerator() does not register memory slots in KVM.
>> > Instead, it registers memory listeners. See
>> > kvm_memory_listener_register(), especially:
>> >
>> >     kml->listener.region_add = kvm_region_add;
>> >
>> > That's the hook function to be called when there are new memory region
>> > added to the system.
>> >
>> > Further, when RAM is initialzed, it'll modify the address space layout
>> > of system_memory, and the registered listener of KVM (kvm_region_add)
>> > will be invoked, it'll further sync with kvm. It should be in the
>> > following path if you break at kvm_region_add in gdb:
>> >
>> > #0  0x00005555557ba13a in kvm_region_add (listener=0x5555568330c0,
>> > section=0x7fffffffd310) at /root/git/qemu/kvm-all.c:859
>> > #1  0x00005555557c1910 in address_space_update_topology_pass
>> > (as=0x55555629e240 <address_space_memory>,
>> old_view=0x5555567a7090,
>> > new_view=0x5555568d3460, adding=true) at /root/git/qemu/memory.c:871
>> > #2  0x00005555557c19f3 in address_space_update_topology
>> > (as=0x55555629e240 <address_space_memory>) at
>> > /root/git/qemu/memory.c:886
>> > #3  0x00005555557c1b41 in memory_region_transaction_commit () at
>> > /root/git/qemu/memory.c:922
>> > #4  0x00005555557c4bfd in memory_region_update_container_subregions
>> > (subregion=0x5555568d2fc0) at /root/git/qemu/memory.c:2075
>> > #5  0x00005555557c4c64 in memory_region_add_subregion_common
>> > (mr=0x5555567a5830, offset=0, subregion=0x5555568d2fc0) at
>> > /root/git/qemu/memory.c:2085
>> > #6  0x00005555557c4ca0 in memory_region_add_subregion
>> > (mr=0x5555567a5830, offset=0, subregion=0x5555568d2fc0) at
>> > /root/git/qemu/memory.c:2093
>> > #7  0x000055555583fd68 in pc_memory_init (pcms=0x5555567a4100,
>> > system_memory=0x5555567a5830, rom_memory=0x5555568d21a0,
>> > ram_memory=0x7fffffffd550) at /root/git/qemu/hw/i386/pc.c:1383
>> > #8  0x0000555555847363 in pc_q35_init (machine=0x5555567a4100) at
>> > /root/git/qemu/hw/i386/pc_q35.c:147
>> > #9  0x0000555555847cac in pc_init_v2_9 (machine=0x5555567a4100) at
>> > /root/git/qemu/hw/i386/pc_q35.c:310
>> > #10 0x00005555558f7cf8 in main (argc=11, argv=0x7fffffffda78,
>> > envp=0x7fffffffdad8) at /root/git/qemu/vl.c:4557
>> >
>> > Hope this helps. Thanks.
>> >
>> > --
>> > Peter Xu
>

Re: [Qemu-devel] qemu memory manage question

[jack.chen]

Hello, now I am confused about some structures in qemu related to
memory management,they are
MemoryRegion、AddressSpace、FlatView、FlatRange、MemoryRegionSection、RAMList、RAMBlock、KVMSlot、kvm_userspace_memory_region,who
can tell me the concrete connection among these structures.
thanks a lot!

2017-04-18 10:25 GMT+08:00 jack.chen <zhunxun@gmail.com>:
> Thanks very much!!
>
> 2017-04-17 19:19 GMT+08:00 李强 <liqiang6-s@360.cn>:
>>
>>
>>> -----Original Message-----
>>> From: Qemu-devel
>>> [mailto:qemu-devel-bounces+liqiang6-s=360.cn@nongnu.org] On Behalf Of
>>> jack.chen
>>> Sent: Monday, April 17, 2017 6:56 PM
>>> To: Peter Xu
>>> Cc: qemu
>>> Subject: Re: [Qemu-devel] qemu memory manage question
>>>
>>> Thanks,from the path you have list to me,it can be  well explained,but
>>> according to the source code,in the end of kvm_init,kvm_memory_listener and
>>> kvm_io_listener were registered by memory_listener_register(),and  in the
>>> end of
>>> memory_listener_register(),listener_add_address_space() was called for each
>>> address_space,so the listener->region_add was executed then.I do not know
>>> what mistake I have made,can you explain it to me ?? thank you very much!
>>>
>>
>> They are callbacks.
>> Every change of memory topology will call these listeners, add subregion(Peter's example),
>> modify the property of memory, create address space for example.
>>
>> Thanks.
>>
>> --
>> Li Qiang /the Gear Team, Qihoo 360 Inc
>>
>>
>>> 2017-04-17 18:26 GMT+08:00 Peter Xu <peterx@redhat.com>:
>>> > On Mon, Apr 17, 2017 at 06:09:11PM +0800, jack.chen wrote:
>>> >> hello,I have some questions about memory allocation in qemu for
>>> >> virtual machine.I found when configure_accelerator function was
>>> >> called ,memory slots  were registered to KVM,but at that time
>>> >> address_space have not been initialized and ram have not been
>>> >> allocated,it is really confused me,Thanks a lot!!
>>> >
>>> > Here's how I understand it...
>>> >
>>> > configure_accelerator() does not register memory slots in KVM.
>>> > Instead, it registers memory listeners. See
>>> > kvm_memory_listener_register(), especially:
>>> >
>>> >     kml->listener.region_add = kvm_region_add;
>>> >
>>> > That's the hook function to be called when there are new memory region
>>> > added to the system.
>>> >
>>> > Further, when RAM is initialzed, it'll modify the address space layout
>>> > of system_memory, and the registered listener of KVM (kvm_region_add)
>>> > will be invoked, it'll further sync with kvm. It should be in the
>>> > following path if you break at kvm_region_add in gdb:
>>> >
>>> > #0  0x00005555557ba13a in kvm_region_add (listener=0x5555568330c0,
>>> > section=0x7fffffffd310) at /root/git/qemu/kvm-all.c:859
>>> > #1  0x00005555557c1910 in address_space_update_topology_pass
>>> > (as=0x55555629e240 <address_space_memory>,
>>> old_view=0x5555567a7090,
>>> > new_view=0x5555568d3460, adding=true) at /root/git/qemu/memory.c:871
>>> > #2  0x00005555557c19f3 in address_space_update_topology
>>> > (as=0x55555629e240 <address_space_memory>) at
>>> > /root/git/qemu/memory.c:886
>>> > #3  0x00005555557c1b41 in memory_region_transaction_commit () at
>>> > /root/git/qemu/memory.c:922
>>> > #4  0x00005555557c4bfd in memory_region_update_container_subregions
>>> > (subregion=0x5555568d2fc0) at /root/git/qemu/memory.c:2075
>>> > #5  0x00005555557c4c64 in memory_region_add_subregion_common
>>> > (mr=0x5555567a5830, offset=0, subregion=0x5555568d2fc0) at
>>> > /root/git/qemu/memory.c:2085
>>> > #6  0x00005555557c4ca0 in memory_region_add_subregion
>>> > (mr=0x5555567a5830, offset=0, subregion=0x5555568d2fc0) at
>>> > /root/git/qemu/memory.c:2093
>>> > #7  0x000055555583fd68 in pc_memory_init (pcms=0x5555567a4100,
>>> > system_memory=0x5555567a5830, rom_memory=0x5555568d21a0,
>>> > ram_memory=0x7fffffffd550) at /root/git/qemu/hw/i386/pc.c:1383
>>> > #8  0x0000555555847363 in pc_q35_init (machine=0x5555567a4100) at
>>> > /root/git/qemu/hw/i386/pc_q35.c:147
>>> > #9  0x0000555555847cac in pc_init_v2_9 (machine=0x5555567a4100) at
>>> > /root/git/qemu/hw/i386/pc_q35.c:310
>>> > #10 0x00005555558f7cf8 in main (argc=11, argv=0x7fffffffda78,
>>> > envp=0x7fffffffdad8) at /root/git/qemu/vl.c:4557
>>> >
>>> > Hope this helps. Thanks.
>>> >
>>> > --
>>> > Peter Xu
>>