Re: [PATCH v6 00/10] KVM: selftests: some improvement and a new test for kvm page table

[Paolo Bonzini <pbonzini@redhat.com>]

On 06/04/21 05:05, wangyanan (Y) wrote:
> Kindly ping...
> 
> Hi Paolo,
> Will this series be picked up soon, or is there any other work for me to 
> do?
> 
> Regards,
> Yanan
> 
> 
> On 2021/3/30 16:08, Yanan Wang wrote:
>> Hi,
>> This v6 series can mainly include two parts.
>> Rebased on kvm queue branch: 
>> https://git.kernel.org/pub/scm/virt/kvm/kvm.git/log/?h=queue
>>
>> In the first part, all the known hugetlb backing src types specified
>> with different hugepage sizes are listed, so that we can specify use
>> of hugetlb source of the exact granularity that we want, instead of
>> the system default ones. And as all the known hugetlb page sizes are
>> listed, it's appropriate for all architectures. Besides, a helper that
>> can get granularity of different backing src types(anonumous/thp/hugetlb)
>> is added, so that we can use the accurate backing src granularity for
>> kinds of alignment or guest memory accessing of vcpus.
>>
>> In the second part, a new test is added:
>> This test is added to serve as a performance tester and a bug reproducer
>> for kvm page table code (GPA->HPA mappings), it gives guidance for the
>> people trying to make some improvement for kvm. And the following 
>> explains
>> what we can exactly do through this test.
>>
>> The function guest_code() can cover the conditions where a single vcpu or
>> multiple vcpus access guest pages within the same memory region, in three
>> VM stages(before dirty logging, during dirty logging, after dirty 
>> logging).
>> Besides, the backing src memory type(ANONYMOUS/THP/HUGETLB) of the tested
>> memory region can be specified by users, which means normal page mappings
>> or block mappings can be chosen by users to be created in the test.
>>
>> If ANONYMOUS memory is specified, kvm will create normal page mappings
>> for the tested memory region before dirty logging, and update attributes
>> of the page mappings from RO to RW during dirty logging. If THP/HUGETLB
>> memory is specified, kvm will create block mappings for the tested memory
>> region before dirty logging, and split the blcok mappings into normal 
>> page
>> mappings during dirty logging, and coalesce the page mappings back into
>> block mappings after dirty logging is stopped.
>>
>> So in summary, as a performance tester, this test can present the
>> performance of kvm creating/updating normal page mappings, or the
>> performance of kvm creating/splitting/recovering block mappings,
>> through execution time.
>>
>> When we need to coalesce the page mappings back to block mappings after
>> dirty logging is stopped, we have to firstly invalidate *all* the TLB
>> entries for the page mappings right before installation of the block 
>> entry,
>> because a TLB conflict abort error could occur if we can't invalidate the
>> TLB entries fully. We have hit this TLB conflict twice on aarch64 
>> software
>> implementation and fixed it. As this test can imulate process from dirty
>> logging enabled to dirty logging stopped of a VM with block mappings,
>> so it can also reproduce this TLB conflict abort due to inadequate TLB
>> invalidation when coalescing tables.
>>
>> Links about the TLB conflict abort:
>> https://lore.kernel.org/lkml/20201201201034.116760-3-wangyanan55@huawei.com/ 
>>
>>
>> ---
>>
>> Change logs:
>>
>> v5->v6:
>> - Address Andrew Jones's comments for v5 series
>> - Add Andrew Jones's R-b tags in some patches
>> - Rebased on newest kvm/queue tree
>> - v5: 
>> https://lore.kernel.org/lkml/20210323135231.24948-1-wangyanan55@huawei.com/ 
>>
>>
>> v4->v5:
>> - Use synchronization(sem_wait) for time measurement
>> - Add a new patch about TEST_ASSERT(patch 4)
>> - Address Andrew Jones's comments for v4 series
>> - Add Andrew Jones's R-b tags in some patches
>> - v4: 
>> https://lore.kernel.org/lkml/20210302125751.19080-1-wangyanan55@huawei.com/ 
>>
>>
>> v3->v4:
>> - Add a helper to get system default hugetlb page size
>> - Add tags of Reviewed-by of Ben in the patches
>> - v3: 
>> https://lore.kernel.org/lkml/20210301065916.11484-1-wangyanan55@huawei.com/ 
>>
>>
>> v2->v3:
>> - Add tags of Suggested-by, Reviewed-by in the patches
>> - Add a generic micro to get hugetlb page sizes
>> - Some changes for suggestions about v2 series
>> - v2: 
>> https://lore.kernel.org/lkml/20210225055940.18748-1-wangyanan55@huawei.com/ 
>>
>>
>> v1->v2:
>> - Add a patch to sync header files
>> - Add helpers to get granularity of different backing src types
>> - Some changes for suggestions about v1 series
>> - v1: 
>> https://lore.kernel.org/lkml/20210208090841.333724-1-wangyanan55@huawei.com/ 
>>
>>
>> ---
>>
>> Yanan Wang (10):
>>    tools headers: sync headers of asm-generic/hugetlb_encode.h
>>    mm/hugetlb: Add a macro to get HUGETLB page sizes for mmap
>>    KVM: selftests: Use flag CLOCK_MONOTONIC_RAW for timing
>>    KVM: selftests: Print the errno besides error-string in TEST_ASSERT
>>    KVM: selftests: Make a generic helper to get vm guest mode strings
>>    KVM: selftests: Add a helper to get system configured THP page size
>>    KVM: selftests: Add a helper to get system default hugetlb page size
>>    KVM: selftests: List all hugetlb src types specified with page sizes
>>    KVM: selftests: Adapt vm_userspace_mem_region_add to new helpers
>>    KVM: selftests: Add a test for kvm page table code
>>
>>   include/uapi/linux/mman.h                     |   2 +
>>   tools/include/asm-generic/hugetlb_encode.h    |   3 +
>>   tools/include/uapi/linux/mman.h               |   2 +
>>   tools/testing/selftests/kvm/.gitignore        |   1 +
>>   tools/testing/selftests/kvm/Makefile          |   3 +
>>   .../selftests/kvm/demand_paging_test.c        |   8 +-
>>   .../selftests/kvm/dirty_log_perf_test.c       |  14 +-
>>   .../testing/selftests/kvm/include/kvm_util.h  |   4 +-
>>   .../testing/selftests/kvm/include/test_util.h |  21 +-
>>   .../selftests/kvm/kvm_page_table_test.c       | 506 ++++++++++++++++++
>>   tools/testing/selftests/kvm/lib/assert.c      |   4 +-
>>   tools/testing/selftests/kvm/lib/kvm_util.c    |  59 +-
>>   tools/testing/selftests/kvm/lib/test_util.c   | 163 +++++-
>>   tools/testing/selftests/kvm/steal_time.c      |   4 +-
>>   14 files changed, 733 insertions(+), 61 deletions(-)
>>   create mode 100644 tools/testing/selftests/kvm/kvm_page_table_test.c
>>
> 

Queued, thanks.

I moved MAP_HUGE_PAGE_SIZE from the uapi header to test_util.c, because 
I'm not confident in adding a new userspace API without an ack.

Paolo