Re: [PATCH 3/3] selftests: kvm: Add support for customized slot0 memory size

Re: [PATCH 3/3] selftests: kvm: Add support for customized slot0 memory size

[Andrew Jones]

On Wed, Jun 09, 2021 at 07:38:16AM +0800, Zhenzhong Duan wrote:
> Current implementatiion only create VM with slot0 memory size
> of fixed 512 pages plus page tables and per-cpu memory.
> 
> Add a new parameter slot0_mem_pages to vm_create_with_vcpus() and
> some comments to clarify the meaning of slot0_mem_pages and
> extra_mem_pages.
> 
> Signed-off-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
> ---
>  .../testing/selftests/kvm/include/kvm_util.h  |  7 +--
>  .../selftests/kvm/kvm_page_table_test.c       |  2 +-
>  tools/testing/selftests/kvm/lib/kvm_util.c    | 47 +++++++++++++++----
>  .../selftests/kvm/lib/perf_test_util.c        |  2 +-
>  4 files changed, 44 insertions(+), 14 deletions(-)
> 
> diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
> index fcd8e3855111..8bcadfa8cea5 100644
> --- a/tools/testing/selftests/kvm/include/kvm_util.h
> +++ b/tools/testing/selftests/kvm/include/kvm_util.h
> @@ -285,10 +285,11 @@ struct kvm_vm *vm_create_default_with_vcpus(uint32_t nr_vcpus, uint64_t extra_me
>  					    uint32_t num_percpu_pages, void *guest_code,
>  					    uint32_t vcpuids[]);
>  
> -/* Like vm_create_default_with_vcpus, but accepts mode as a parameter */
> +/* Like vm_create_default_with_vcpus, but accepts mode and slot0 memory as a parameter */
>  struct kvm_vm *vm_create_with_vcpus(enum vm_guest_mode mode, uint32_t nr_vcpus,
> -				    uint64_t extra_mem_pages, uint32_t num_percpu_pages,
> -				    void *guest_code, uint32_t vcpuids[]);
> +				    uint64_t slot0_mem_pages, uint64_t extra_mem_pages,
> +				    uint32_t num_percpu_pages, void *guest_code,
> +				    uint32_t vcpuids[]);
>  
>  /*
>   * Adds a vCPU with reasonable defaults (e.g. a stack)
> diff --git a/tools/testing/selftests/kvm/kvm_page_table_test.c b/tools/testing/selftests/kvm/kvm_page_table_test.c
> index 1c4753fff19e..82171f17c1d7 100644
> --- a/tools/testing/selftests/kvm/kvm_page_table_test.c
> +++ b/tools/testing/selftests/kvm/kvm_page_table_test.c
> @@ -268,7 +268,7 @@ static struct kvm_vm *pre_init_before_test(enum vm_guest_mode mode, void *arg)
>  
>  	/* Create a VM with enough guest pages */
>  	guest_num_pages = test_mem_size / guest_page_size;
> -	vm = vm_create_with_vcpus(mode, nr_vcpus,
> +	vm = vm_create_with_vcpus(mode, nr_vcpus, DEFAULT_GUEST_PHY_PAGES,
>  				  guest_num_pages, 0, guest_code, NULL);
>  
>  	/* Align down GPA of the testing memslot */
> diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c
> index 63418df921f0..d86422a119fd 100644
> --- a/tools/testing/selftests/kvm/lib/kvm_util.c
> +++ b/tools/testing/selftests/kvm/lib/kvm_util.c
> @@ -308,21 +308,50 @@ struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)
>  	return vm;
>  }
>  
> +/*
> + * VM Create with customized parameters
> + *
> + * Input Args:
> + *   mode - VM Mode (e.g. VM_MODE_P52V48_4K)
> + *   nr_vcpus - VCPU count
> + *   slot0_mem_pages - Slot0 physical memory size
> + *   extra_mem_pages - Non-slot0 physical memory total size
> + *   num_percpu_pages - Per-cpu physical memory pages
> + *   guest_code - Guest entry point
> + *   vcpuids - VCPU IDs
> + *
> + * Output Args: None
> + *
> + * Return:
> + *   Pointer to opaque structure that describes the created VM.
> + *
> + * Creates a VM with the mode specified by mode (e.g. VM_MODE_P52V48_4K),
> + * with customized slot0 memory size, at least 512 pages currently.
> + * extra_mem_pages is only used to calculate the maximum page table size,
> + * no real memory allocation for non-slot0 memory in this function.
> + */
>  struct kvm_vm *vm_create_with_vcpus(enum vm_guest_mode mode, uint32_t nr_vcpus,
> -				    uint64_t extra_mem_pages, uint32_t num_percpu_pages,
> -				    void *guest_code, uint32_t vcpuids[])
> +				    uint64_t slot0_mem_pages, uint64_t extra_mem_pages,
> +				    uint32_t num_percpu_pages, void *guest_code,
> +				    uint32_t vcpuids[])
>  {
> +	uint64_t vcpu_pages, extra_pg_pages, pages;
> +	struct kvm_vm *vm;
> +	int i;
> +
> +	/* Force slot0 memory size not small than DEFAULT_GUEST_PHY_PAGES */
> +	if (slot0_mem_pages < DEFAULT_GUEST_PHY_PAGES)
> +		slot0_mem_pages = DEFAULT_GUEST_PHY_PAGES;
> +
>  	/* The maximum page table size for a memory region will be when the
>  	 * smallest pages are used. Considering each page contains x page
>  	 * table descriptors, the total extra size for page tables (for extra
>  	 * N pages) will be: N/x+N/x^2+N/x^3+... which is definitely smaller
>  	 * than N/x*2.
>  	 */
> -	uint64_t vcpu_pages = (DEFAULT_STACK_PGS + num_percpu_pages) * nr_vcpus;
> -	uint64_t extra_pg_pages = (extra_mem_pages + vcpu_pages) / PTES_PER_MIN_PAGE * 2;
> -	uint64_t pages = DEFAULT_GUEST_PHY_PAGES + vcpu_pages + extra_pg_pages;
> -	struct kvm_vm *vm;
> -	int i;
> +	vcpu_pages = (DEFAULT_STACK_PGS + num_percpu_pages) * nr_vcpus;
> +	extra_pg_pages = (slot0_mem_pages + extra_mem_pages + vcpu_pages) / PTES_PER_MIN_PAGE * 2;

We weren't including the slot0 memory (which was DEFAULT_GUEST_PHY_PAGES)
in the extra_pg_pages calculation before, since they weren't considered
"extra". It's probably a good idea to do this calculation for the caller,
but it's making a change not called out in the commit message.

> +	pages = slot0_mem_pages + vcpu_pages + extra_pg_pages;
>  
>  	TEST_ASSERT(nr_vcpus <= kvm_check_cap(KVM_CAP_MAX_VCPUS),
>  		    "nr_vcpus = %d too large for host, max-vcpus = %d",
> @@ -354,8 +383,8 @@ struct kvm_vm *vm_create_default_with_vcpus(uint32_t nr_vcpus, uint64_t extra_me
>  					    uint32_t num_percpu_pages, void *guest_code,
>  					    uint32_t vcpuids[])
>  {
> -	return vm_create_with_vcpus(VM_MODE_DEFAULT, nr_vcpus, extra_mem_pages,
> -				    num_percpu_pages, guest_code, vcpuids);
> +	return vm_create_with_vcpus(VM_MODE_DEFAULT, nr_vcpus, DEFAULT_GUEST_PHY_PAGES,
> +				    extra_mem_pages, num_percpu_pages, guest_code, vcpuids);
>  }
>  
>  struct kvm_vm *vm_create_default(uint32_t vcpuid, uint64_t extra_mem_pages,
> diff --git a/tools/testing/selftests/kvm/lib/perf_test_util.c b/tools/testing/selftests/kvm/lib/perf_test_util.c
> index abf381800a59..7397ca299835 100644
> --- a/tools/testing/selftests/kvm/lib/perf_test_util.c
> +++ b/tools/testing/selftests/kvm/lib/perf_test_util.c
> @@ -69,7 +69,7 @@ struct kvm_vm *perf_test_create_vm(enum vm_guest_mode mode, int vcpus,
>  	TEST_ASSERT(vcpu_memory_bytes % perf_test_args.guest_page_size == 0,
>  		    "Guest memory size is not guest page size aligned.");
>  
> -	vm = vm_create_with_vcpus(mode, vcpus,
> +	vm = vm_create_with_vcpus(mode, vcpus, DEFAULT_GUEST_PHY_PAGES,
>  				  (vcpus * vcpu_memory_bytes) / perf_test_args.guest_page_size,
>  				  0, guest_code, NULL);
>  
> -- 
> 2.25.1
>

Besides the extra_pg_pages comment

Reviewed-by: Andrew Jones <drjones@redhat.com>

Thanks,
drew 

Re: [PATCH 1/3] Revert "selftests: kvm: make allocation of extra memory take effect"

[Andrew Jones]

On Wed, Jun 09, 2021 at 07:38:14AM +0800, Zhenzhong Duan wrote:
> This reverts commit 39fe2fc96694164723846fccf6caa42c3aee6ec4.
> 
> Parameter extra_mem_pages in vm_create_default() is used to calculate
> the page table size for all the memory chunks.
> Real memory allocation for non-slot0 memory happens by extra call of
> vm_userspace_mem_region_add() outside of vm_create_default().
> 
> The reverted commit changed above meaning of extra_mem_pages as extra
> slot0 memory size. This way made the page table size calculations
> open coded in separate test.
> 
> Link: https://lkml.org/lkml/2021/6/3/551
> Signed-off-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
> ---
>  tools/testing/selftests/kvm/lib/kvm_util.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c
> index 28e528c19d28..63418df921f0 100644
> --- a/tools/testing/selftests/kvm/lib/kvm_util.c
> +++ b/tools/testing/selftests/kvm/lib/kvm_util.c
> @@ -320,7 +320,7 @@ struct kvm_vm *vm_create_with_vcpus(enum vm_guest_mode mode, uint32_t nr_vcpus,
>  	 */
>  	uint64_t vcpu_pages = (DEFAULT_STACK_PGS + num_percpu_pages) * nr_vcpus;
>  	uint64_t extra_pg_pages = (extra_mem_pages + vcpu_pages) / PTES_PER_MIN_PAGE * 2;
> -	uint64_t pages = DEFAULT_GUEST_PHY_PAGES + extra_mem_pages + vcpu_pages + extra_pg_pages;
> +	uint64_t pages = DEFAULT_GUEST_PHY_PAGES + vcpu_pages + extra_pg_pages;
>  	struct kvm_vm *vm;
>  	int i;
>  
> -- 
> 2.25.1
>

Reviewed-by: Andrew Jones <drjones@redhat.com>

Re: [PATCH 2/3] Revert "selftests: kvm: fix overlapping addresses in memslot_perf_test"

[Andrew Jones]

On Wed, Jun 09, 2021 at 07:38:15AM +0800, Zhenzhong Duan wrote:
> This reverts commit 000ac42953395a4f0a63d5db640c5e4c88a548c5.
> 
> As (39fe2fc96694 "selftests: kvm: make allocation of extra memory take
> effect") is reverted, this relevant commit should be reverted too.
> 
> Signed-off-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
> ---
>  tools/testing/selftests/kvm/memslot_perf_test.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/kvm/memslot_perf_test.c b/tools/testing/selftests/kvm/memslot_perf_test.c
> index 9307f25d8130..11239652d805 100644
> --- a/tools/testing/selftests/kvm/memslot_perf_test.c
> +++ b/tools/testing/selftests/kvm/memslot_perf_test.c
> @@ -267,7 +267,7 @@ static bool prepare_vm(struct vm_data *data, int nslots, uint64_t *maxslots,
>  	data->hva_slots = malloc(sizeof(*data->hva_slots) * data->nslots);
>  	TEST_ASSERT(data->hva_slots, "malloc() fail");
>  
> -	data->vm = vm_create_default(VCPU_ID, 1024, guest_code);
> +	data->vm = vm_create_default(VCPU_ID, mempages, guest_code);
>  
>  	pr_info_v("Adding slots 1..%i, each slot with %"PRIu64" pages + %"PRIu64" extra pages last\n",
>  		max_mem_slots - 1, data->pages_per_slot, rempages);
> -- 
> 2.25.1
>

Reviewed-by: Andrew Jones <drjones@redhat.com>

Re: [PATCH 0/3] Restore extra_mem_pages and add slot0_mem_pages

[Maciej S. Szmigiero]

On 09.06.2021 01:38, Zhenzhong Duan wrote:
> (39fe2fc96694 "selftests: kvm: make allocation of extra memory take effect")
> changed the meaning of extra_mem_pages and treated it as slot0 memory size.
> 
> In fact extra_mem_pages is used for non-slot0 memory size, there is no custom
> slot0 memory size support. See discuss in https://urldefense.com/v3/__https://lkml.org/lkml/2021/6/3/551__;!!GqivPVa7Brio!K2FcwkE2nzlPAWgBHh6o6jtaWe66RMlfkb-b_8mAtVa5d8ez_sArupY-EqIquuCj2sorww$
> for more details.
> 
> This patchset restores extra_mem_pages's original meaning and adds support for
> custom slot0 memory with a new parameter slot0_mem_pages.
> 
> Run below command, all 39 tests passed.
> # make -C tools/testing/selftests/ TARGETS=kvm run_tests
> 
> Zhenzhong Duan (3):
>    Revert "selftests: kvm: make allocation of extra memory take effect"
>    Revert "selftests: kvm: fix overlapping addresses in
>      memslot_perf_test"
>    selftests: kvm: Add support for customized slot0 memory size
> 
>   .../testing/selftests/kvm/include/kvm_util.h  |  7 +--
>   .../selftests/kvm/kvm_page_table_test.c       |  2 +-
>   tools/testing/selftests/kvm/lib/kvm_util.c    | 47 +++++++++++++++----
>   .../selftests/kvm/lib/perf_test_util.c        |  2 +-
>   .../testing/selftests/kvm/memslot_perf_test.c |  2 +-
>   5 files changed, 45 insertions(+), 15 deletions(-)
> 

Looks good to me, thanks!

For the whole series:
Reviewed-by: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>

BTW: It looks like there was something wrong with the clock (or time zone
setup) of the machine this series was posted from since the "Date:"
headers on these four messages say they were sent Jun 8, 23:38 UTC
(while the time right now is Jun 8, 16:45 UTC).

Maciej

Re: [PATCH 0/3] Restore extra_mem_pages and add slot0_mem_pages

[Paolo Bonzini]

On 09/06/21 01:38, Zhenzhong Duan wrote:
> (39fe2fc96694 "selftests: kvm: make allocation of extra memory take effect")
> changed the meaning of extra_mem_pages and treated it as slot0 memory size.
> 
> In fact extra_mem_pages is used for non-slot0 memory size, there is no custom
> slot0 memory size support. See discuss in https://lkml.org/lkml/2021/6/3/551
> for more details.
> 
> This patchset restores extra_mem_pages's original meaning and adds support for
> custom slot0 memory with a new parameter slot0_mem_pages.

Because the two reverts are so small, I squashed everything in a single 
patch with the following message:

     Until commit 39fe2fc96694 ("selftests: kvm: make allocation of extra
     memory take effect", 2021-05-27), parameter extra_mem_pages was used
     only to calculate the page table size for all the memory chunks,
     because real memory allocation happened with calls of
     vm_userspace_mem_region_add() after vm_create_default().

     Commit 39fe2fc96694 however changed the meaning of extra_mem_pages to
     the size of memory slot 0.  This makes the memory allocation more
     flexible, but makes it harder to account for the number of
     pages needed for the page tables.  For example, memslot_perf_test
     has a small amount of memory in slot 0 but a lot in other slots,
     and adding that memory twice (both in slot 0 and with later
     calls to vm_userspace_mem_region_add()) causes an error that
     was fixed in commit 000ac4295339 ("selftests: kvm: fix overlapping
     addresses in memslot_perf_test", 2021-05-29)

     Since both uses are sensible, add a new parameter slot0_mem_pages
     to vm_create_with_vcpus() and some comments to clarify the meaning of
     slot0_mem_pages and extra_mem_pages.  With this change,
     memslot_perf_test can go back to passing the number of memory
     pages as extra_mem_pages.

Paolo

> Run below command, all 39 tests passed.
> # make -C tools/testing/selftests/ TARGETS=kvm run_tests
> 
> Zhenzhong Duan (3):
>    Revert "selftests: kvm: make allocation of extra memory take effect"
>    Revert "selftests: kvm: fix overlapping addresses in
>      memslot_perf_test"
>    selftests: kvm: Add support for customized slot0 memory size
> 
>   .../testing/selftests/kvm/include/kvm_util.h  |  7 +--
>   .../selftests/kvm/kvm_page_table_test.c       |  2 +-
>   tools/testing/selftests/kvm/lib/kvm_util.c    | 47 +++++++++++++++----
>   .../selftests/kvm/lib/perf_test_util.c        |  2 +-
>   .../testing/selftests/kvm/memslot_perf_test.c |  2 +-
>   5 files changed, 45 insertions(+), 15 deletions(-)
> 

[PATCH 0/3] Restore extra_mem_pages and add slot0_mem_pages

[Zhenzhong Duan]

(39fe2fc96694 "selftests: kvm: make allocation of extra memory take effect")
changed the meaning of extra_mem_pages and treated it as slot0 memory size.

In fact extra_mem_pages is used for non-slot0 memory size, there is no custom
slot0 memory size support. See discuss in https://lkml.org/lkml/2021/6/3/551
for more details.

This patchset restores extra_mem_pages's original meaning and adds support for
custom slot0 memory with a new parameter slot0_mem_pages.

Run below command, all 39 tests passed.
# make -C tools/testing/selftests/ TARGETS=kvm run_tests

Zhenzhong Duan (3):
  Revert "selftests: kvm: make allocation of extra memory take effect"
  Revert "selftests: kvm: fix overlapping addresses in
    memslot_perf_test"
  selftests: kvm: Add support for customized slot0 memory size

 .../testing/selftests/kvm/include/kvm_util.h  |  7 +--
 .../selftests/kvm/kvm_page_table_test.c       |  2 +-
 tools/testing/selftests/kvm/lib/kvm_util.c    | 47 +++++++++++++++----
 .../selftests/kvm/lib/perf_test_util.c        |  2 +-
 .../testing/selftests/kvm/memslot_perf_test.c |  2 +-
 5 files changed, 45 insertions(+), 15 deletions(-)

-- 
2.25.1

[PATCH 1/3] Revert "selftests: kvm: make allocation of extra memory take effect"

[Zhenzhong Duan]

This reverts commit 39fe2fc96694164723846fccf6caa42c3aee6ec4.

Parameter extra_mem_pages in vm_create_default() is used to calculate
the page table size for all the memory chunks.
Real memory allocation for non-slot0 memory happens by extra call of
vm_userspace_mem_region_add() outside of vm_create_default().

The reverted commit changed above meaning of extra_mem_pages as extra
slot0 memory size. This way made the page table size calculations
open coded in separate test.

Link: https://lkml.org/lkml/2021/6/3/551
Signed-off-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
---
 tools/testing/selftests/kvm/lib/kvm_util.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c
index 28e528c19d28..63418df921f0 100644
--- a/tools/testing/selftests/kvm/lib/kvm_util.c
+++ b/tools/testing/selftests/kvm/lib/kvm_util.c
@@ -320,7 +320,7 @@ struct kvm_vm *vm_create_with_vcpus(enum vm_guest_mode mode, uint32_t nr_vcpus,
 	 */
 	uint64_t vcpu_pages = (DEFAULT_STACK_PGS + num_percpu_pages) * nr_vcpus;
 	uint64_t extra_pg_pages = (extra_mem_pages + vcpu_pages) / PTES_PER_MIN_PAGE * 2;
-	uint64_t pages = DEFAULT_GUEST_PHY_PAGES + extra_mem_pages + vcpu_pages + extra_pg_pages;
+	uint64_t pages = DEFAULT_GUEST_PHY_PAGES + vcpu_pages + extra_pg_pages;
 	struct kvm_vm *vm;
 	int i;
 
-- 
2.25.1

[PATCH 2/3] Revert "selftests: kvm: fix overlapping addresses in memslot_perf_test"

[Zhenzhong Duan]

This reverts commit 000ac42953395a4f0a63d5db640c5e4c88a548c5.

As (39fe2fc96694 "selftests: kvm: make allocation of extra memory take
effect") is reverted, this relevant commit should be reverted too.

Signed-off-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
---
 tools/testing/selftests/kvm/memslot_perf_test.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/kvm/memslot_perf_test.c b/tools/testing/selftests/kvm/memslot_perf_test.c
index 9307f25d8130..11239652d805 100644
--- a/tools/testing/selftests/kvm/memslot_perf_test.c
+++ b/tools/testing/selftests/kvm/memslot_perf_test.c
@@ -267,7 +267,7 @@ static bool prepare_vm(struct vm_data *data, int nslots, uint64_t *maxslots,
 	data->hva_slots = malloc(sizeof(*data->hva_slots) * data->nslots);
 	TEST_ASSERT(data->hva_slots, "malloc() fail");
 
-	data->vm = vm_create_default(VCPU_ID, 1024, guest_code);
+	data->vm = vm_create_default(VCPU_ID, mempages, guest_code);
 
 	pr_info_v("Adding slots 1..%i, each slot with %"PRIu64" pages + %"PRIu64" extra pages last\n",
 		max_mem_slots - 1, data->pages_per_slot, rempages);
-- 
2.25.1

[PATCH 3/3] selftests: kvm: Add support for customized slot0 memory size

[Zhenzhong Duan]

Current implementatiion only create VM with slot0 memory size
of fixed 512 pages plus page tables and per-cpu memory.

Add a new parameter slot0_mem_pages to vm_create_with_vcpus() and
some comments to clarify the meaning of slot0_mem_pages and
extra_mem_pages.

Signed-off-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
---
 .../testing/selftests/kvm/include/kvm_util.h  |  7 +--
 .../selftests/kvm/kvm_page_table_test.c       |  2 +-
 tools/testing/selftests/kvm/lib/kvm_util.c    | 47 +++++++++++++++----
 .../selftests/kvm/lib/perf_test_util.c        |  2 +-
 4 files changed, 44 insertions(+), 14 deletions(-)

diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
index fcd8e3855111..8bcadfa8cea5 100644
--- a/tools/testing/selftests/kvm/include/kvm_util.h
+++ b/tools/testing/selftests/kvm/include/kvm_util.h
@@ -285,10 +285,11 @@ struct kvm_vm *vm_create_default_with_vcpus(uint32_t nr_vcpus, uint64_t extra_me
 					    uint32_t num_percpu_pages, void *guest_code,
 					    uint32_t vcpuids[]);
 
-/* Like vm_create_default_with_vcpus, but accepts mode as a parameter */
+/* Like vm_create_default_with_vcpus, but accepts mode and slot0 memory as a parameter */
 struct kvm_vm *vm_create_with_vcpus(enum vm_guest_mode mode, uint32_t nr_vcpus,
-				    uint64_t extra_mem_pages, uint32_t num_percpu_pages,
-				    void *guest_code, uint32_t vcpuids[]);
+				    uint64_t slot0_mem_pages, uint64_t extra_mem_pages,
+				    uint32_t num_percpu_pages, void *guest_code,
+				    uint32_t vcpuids[]);
 
 /*
  * Adds a vCPU with reasonable defaults (e.g. a stack)
diff --git a/tools/testing/selftests/kvm/kvm_page_table_test.c b/tools/testing/selftests/kvm/kvm_page_table_test.c
index 1c4753fff19e..82171f17c1d7 100644
--- a/tools/testing/selftests/kvm/kvm_page_table_test.c
+++ b/tools/testing/selftests/kvm/kvm_page_table_test.c
@@ -268,7 +268,7 @@ static struct kvm_vm *pre_init_before_test(enum vm_guest_mode mode, void *arg)
 
 	/* Create a VM with enough guest pages */
 	guest_num_pages = test_mem_size / guest_page_size;
-	vm = vm_create_with_vcpus(mode, nr_vcpus,
+	vm = vm_create_with_vcpus(mode, nr_vcpus, DEFAULT_GUEST_PHY_PAGES,
 				  guest_num_pages, 0, guest_code, NULL);
 
 	/* Align down GPA of the testing memslot */
diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c
index 63418df921f0..d86422a119fd 100644
--- a/tools/testing/selftests/kvm/lib/kvm_util.c
+++ b/tools/testing/selftests/kvm/lib/kvm_util.c
@@ -308,21 +308,50 @@ struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)
 	return vm;
 }
 
+/*
+ * VM Create with customized parameters
+ *
+ * Input Args:
+ *   mode - VM Mode (e.g. VM_MODE_P52V48_4K)
+ *   nr_vcpus - VCPU count
+ *   slot0_mem_pages - Slot0 physical memory size
+ *   extra_mem_pages - Non-slot0 physical memory total size
+ *   num_percpu_pages - Per-cpu physical memory pages
+ *   guest_code - Guest entry point
+ *   vcpuids - VCPU IDs
+ *
+ * Output Args: None
+ *
+ * Return:
+ *   Pointer to opaque structure that describes the created VM.
+ *
+ * Creates a VM with the mode specified by mode (e.g. VM_MODE_P52V48_4K),
+ * with customized slot0 memory size, at least 512 pages currently.
+ * extra_mem_pages is only used to calculate the maximum page table size,
+ * no real memory allocation for non-slot0 memory in this function.
+ */
 struct kvm_vm *vm_create_with_vcpus(enum vm_guest_mode mode, uint32_t nr_vcpus,
-				    uint64_t extra_mem_pages, uint32_t num_percpu_pages,
-				    void *guest_code, uint32_t vcpuids[])
+				    uint64_t slot0_mem_pages, uint64_t extra_mem_pages,
+				    uint32_t num_percpu_pages, void *guest_code,
+				    uint32_t vcpuids[])
 {
+	uint64_t vcpu_pages, extra_pg_pages, pages;
+	struct kvm_vm *vm;
+	int i;
+
+	/* Force slot0 memory size not small than DEFAULT_GUEST_PHY_PAGES */
+	if (slot0_mem_pages < DEFAULT_GUEST_PHY_PAGES)
+		slot0_mem_pages = DEFAULT_GUEST_PHY_PAGES;
+
 	/* The maximum page table size for a memory region will be when the
 	 * smallest pages are used. Considering each page contains x page
 	 * table descriptors, the total extra size for page tables (for extra
 	 * N pages) will be: N/x+N/x^2+N/x^3+... which is definitely smaller
 	 * than N/x*2.
 	 */
-	uint64_t vcpu_pages = (DEFAULT_STACK_PGS + num_percpu_pages) * nr_vcpus;
-	uint64_t extra_pg_pages = (extra_mem_pages + vcpu_pages) / PTES_PER_MIN_PAGE * 2;
-	uint64_t pages = DEFAULT_GUEST_PHY_PAGES + vcpu_pages + extra_pg_pages;
-	struct kvm_vm *vm;
-	int i;
+	vcpu_pages = (DEFAULT_STACK_PGS + num_percpu_pages) * nr_vcpus;
+	extra_pg_pages = (slot0_mem_pages + extra_mem_pages + vcpu_pages) / PTES_PER_MIN_PAGE * 2;
+	pages = slot0_mem_pages + vcpu_pages + extra_pg_pages;
 
 	TEST_ASSERT(nr_vcpus <= kvm_check_cap(KVM_CAP_MAX_VCPUS),
 		    "nr_vcpus = %d too large for host, max-vcpus = %d",
@@ -354,8 +383,8 @@ struct kvm_vm *vm_create_default_with_vcpus(uint32_t nr_vcpus, uint64_t extra_me
 					    uint32_t num_percpu_pages, void *guest_code,
 					    uint32_t vcpuids[])
 {
-	return vm_create_with_vcpus(VM_MODE_DEFAULT, nr_vcpus, extra_mem_pages,
-				    num_percpu_pages, guest_code, vcpuids);
+	return vm_create_with_vcpus(VM_MODE_DEFAULT, nr_vcpus, DEFAULT_GUEST_PHY_PAGES,
+				    extra_mem_pages, num_percpu_pages, guest_code, vcpuids);
 }
 
 struct kvm_vm *vm_create_default(uint32_t vcpuid, uint64_t extra_mem_pages,
diff --git a/tools/testing/selftests/kvm/lib/perf_test_util.c b/tools/testing/selftests/kvm/lib/perf_test_util.c
index abf381800a59..7397ca299835 100644
--- a/tools/testing/selftests/kvm/lib/perf_test_util.c
+++ b/tools/testing/selftests/kvm/lib/perf_test_util.c
@@ -69,7 +69,7 @@ struct kvm_vm *perf_test_create_vm(enum vm_guest_mode mode, int vcpus,
 	TEST_ASSERT(vcpu_memory_bytes % perf_test_args.guest_page_size == 0,
 		    "Guest memory size is not guest page size aligned.");
 
-	vm = vm_create_with_vcpus(mode, vcpus,
+	vm = vm_create_with_vcpus(mode, vcpus, DEFAULT_GUEST_PHY_PAGES,
 				  (vcpus * vcpu_memory_bytes) / perf_test_args.guest_page_size,
 				  0, guest_code, NULL);
 
-- 
2.25.1

RE: [PATCH 0/3] Restore extra_mem_pages and add slot0_mem_pages

[Duan, Zhenzhong]

> -----Original Message-----
> From: Paolo Bonzini <pbonzini@redhat.com>
> Sent: Wednesday, June 9, 2021 1:30 AM
> To: Duan, Zhenzhong <zhenzhong.duan@intel.com>; linux-
> kernel@vger.kernel.org
> Cc: linux-kselftest@vger.kernel.org; kvm@vger.kernel.org;
> maciej.szmigiero@oracle.com; drjones@redhat.com; shuah@kernel.org
> Subject: Re: [PATCH 0/3] Restore extra_mem_pages and add
> slot0_mem_pages
> 
> On 09/06/21 01:38, Zhenzhong Duan wrote:
> > (39fe2fc96694 "selftests: kvm: make allocation of extra memory take
> > effect") changed the meaning of extra_mem_pages and treated it as slot0
> memory size.
> >
> > In fact extra_mem_pages is used for non-slot0 memory size, there is no
> > custom
> > slot0 memory size support. See discuss in
> > https://lkml.org/lkml/2021/6/3/551
> > for more details.
> >
> > This patchset restores extra_mem_pages's original meaning and adds
> > support for custom slot0 memory with a new parameter slot0_mem_pages.
> 
> Because the two reverts are so small, I squashed everything in a single patch
> with the following message:
> 
>      Until commit 39fe2fc96694 ("selftests: kvm: make allocation of extra
>      memory take effect", 2021-05-27), parameter extra_mem_pages was used
>      only to calculate the page table size for all the memory chunks,
>      because real memory allocation happened with calls of
>      vm_userspace_mem_region_add() after vm_create_default().
> 
>      Commit 39fe2fc96694 however changed the meaning of
> extra_mem_pages to
>      the size of memory slot 0.  This makes the memory allocation more
>      flexible, but makes it harder to account for the number of
>      pages needed for the page tables.  For example, memslot_perf_test
>      has a small amount of memory in slot 0 but a lot in other slots,
>      and adding that memory twice (both in slot 0 and with later
>      calls to vm_userspace_mem_region_add()) causes an error that
>      was fixed in commit 000ac4295339 ("selftests: kvm: fix overlapping
>      addresses in memslot_perf_test", 2021-05-29)
> 
>      Since both uses are sensible, add a new parameter slot0_mem_pages
>      to vm_create_with_vcpus() and some comments to clarify the meaning of
>      slot0_mem_pages and extra_mem_pages.  With this change,
>      memslot_perf_test can go back to passing the number of memory
>      pages as extra_mem_pages.
> 

This looks more clear, thanks for doing that.

Regards
Zhenzhong