Re: [RFC PATCH v3 2/2] arm64: tlb: Use the TLBI RANGE feature in arm64

[Zhenyu Ye <yezhenyu2@huawei.com>]

[-- Attachment #1: Type: text/plain, Size: 3038 bytes --]

Hi Catalin,

On 2020/5/18 20:21, Zhenyu Ye wrote:
> I will test the performance of your suggestion and then reply you again
> here.
> 

I have sent the v4 of this series [1], and compared the performance of
these two different implement.  The test code is in the attachment (directly
call the __flush_tlb_range()).

First, I tested the v4 on a machine whose cpus do not support tlb range.
Fortunately, the newly added judgment in loop has very little effect on
performance.  When page nums are 256 (loop 256 times), the impact is less
than 0.5%:

	[page num]	[before change]		[v4 change]
	1		1457			1491
	2		1911			1957
	3		2382			2377
	4		2827			2852
	5		3282			3349
	6		3763			3781
	7		4295			4252
	8		4716			4716
	9		5186			5218
	10		5618			5648
	16		8427			8454
	32		15938			15951
	64		30890			30977
	128		60802			60863
	256		120826			121395
	512		1508			1555

Then I tested them on a FPGA machine whose cpus support the tlb range
feature (this machine is not the same as above).  Below is the test
data when the stride = PTE:

	[page num]	[before change]	[v3 change]	[v4 change]
	1		16051		15094		13524
	2		11366		11270		11146
	3		11582		11536		12171
	4		11694		11199		11101
	5		12138		11506		12267
	6		12290		11214		11105
	7		12400		11448		12002
	8		12837		11225		11097
	9		14791		11529		12140
	10		15461		11218		11087
	16		18233		11192		11094
	32		26983		11224		11079
	64		43840		11237		11092
	128		77754		11247		11098
	256		145514		11223		11089
	512		280932		11197		11111

We can see the v3 and v4 are very similar in this scene, and both
of them performance improved very much compared to current
implementation.  When the page nums are 256, the performance is
improved by more than 10 times.  And the TLBI RANGE instruction
cost less time than classic TLBI in all secenes on this machine,
even if the page num is small. (but this may be different on
different machines)

Everything performs will util now, but I added a new judgment of
stride in the v4:

	if (cpus_have_const_cap(ARM64_HAS_TLBI_RANGE) &&
	    stride == PAGE_SIZE)
		use tlbi range here...

So when the stride != PTE, then there will use the classic tlbi
instruction and flush the tlbs one by one, where the performance
becomes worse than v3:

	[page num]	[before change]	[v3 change]	[v4 change]
	1		14047		11332		11611
	2		11568		11255		11701
	3		11664		11231		11759
	4		12097		11204		12173
	5		12229		11236		12374
	6		12399		11203		12497
	7		12802		11266		12914
	8		14764		17098		14907
	9		15370		17106		15551
	10		16130		17103		16137
	16		19029		17175		19194
	32		27300		17097		27604
	64		44172		17075		44609
	128		77878		17176		78548
	256		145185		12022		146063
	512		279822		12029		279922

And as we can see, "handle the 2MB with a single classic TLBI"
costs the same time as a single TLBI RANGE instruction.  So should
I remove the judgment of stride and only figure which to use based
on cpucaps in the loop?  But if removes the judgment, the logic
will be the same as v3.(both of them only judge cpucaps)

Waiting for your suggestions...

Thanks,
Zhenyu

[-- Attachment #2: test-tlb-range-perf.c --]
[-- Type: text/plain, Size: 2931 bytes --]

#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <asm/tlb.h>
#include <linux/time.h>
#include <asm/current.h>
#include <linux/sched.h>
#include <linux/delay.h>
#include <linux/mm.h>

#define TESTTIMES 10000

void testRangePerf(void);

static int __init test_init(void)
{
    printk("BEGIN TEST\n");

    testRangePerf();

    printk("END TEST\n");
    return 0;
}


static void __exit test_exit(void)
{
    return;
}

void testRangePerf(void)
{
    int i, j;
    struct timespec64 start, end;
    struct task_struct *ts;
    struct vm_area_struct *vma;

    printk("BEGIN testRangePerf\n");

    ts = current;
    vma = ts->mm->mmap;

    printk("vma->start: %lx, vma->end: %lx, ttl = 0, PAGE_SIZE = 0x%lx\n", vma->vm_start, vma->vm_end, PAGE_SIZE);
    for (i = 1; i <= 10; i++) {
        ktime_get_ts64(&start);
        for (j = 0; j < TESTTIMES; j++) {
            __flush_tlb_range(vma, vma->vm_start, vma->vm_start + PAGE_SIZE * i, PAGE_SIZE, false);
        }
        ktime_get_ts64(&end);
        printk("test __flush_tlb_range with %04d pages, used time: %12lld ns\n", i,
               ((end.tv_sec - start.tv_sec) * 1000000000 + end.tv_nsec - start.tv_nsec) / TESTTIMES);
        msleep(100);
    }

    for (i = 16; i <= 512; i+=i) {
        ktime_get_ts64(&start);
        for (j = 0; j < TESTTIMES; j++) {
            __flush_tlb_range(vma, vma->vm_start, vma->vm_start + PAGE_SIZE * i, PAGE_SIZE, false);
        }
        ktime_get_ts64(&end);
        printk("test __flush_tlb_range with %04d pages, used time: %12lld ns\n", i,
               ((end.tv_sec - start.tv_sec) * 1000000000 + end.tv_nsec - start.tv_nsec) / TESTTIMES);
        msleep(100);
    }

    printk("vma->start: %lx, vma->end: %lx, ttl = 0, PAGE_SIZE = 0x%lx\n", vma->vm_start, vma->vm_end, PMD_SIZE);
    for (i = 1; i <= 10; i++) {
        ktime_get_ts64(&start);
        for (j = 0; j < TESTTIMES; j++) {
            __flush_tlb_range(vma, vma->vm_start, vma->vm_start + PMD_SIZE * i, PMD_SIZE, false);
        }
        ktime_get_ts64(&end);
        printk("test __flush_tlb_range with %04d pages, used time: %12lld ns\n", i,
               ((end.tv_sec - start.tv_sec) * 1000000000 + end.tv_nsec - start.tv_nsec) / TESTTIMES);
        msleep(100);
    }

    for (i = 16; i <= 512; i+=i) {
        ktime_get_ts64(&start);
        for (j = 0; j < TESTTIMES; j++) {
            __flush_tlb_range(vma, vma->vm_start, vma->vm_start + PMD_SIZE * i, PMD_SIZE, false);
        }
        ktime_get_ts64(&end);
        printk("test __flush_tlb_range with %04d pages, used time: %12lld ns\n", i,
               ((end.tv_sec - start.tv_sec) * 1000000000 + end.tv_nsec - start.tv_nsec) / TESTTIMES);
        msleep(100);
    }
}

module_init(test_init)
module_exit(test_exit)

MODULE_LICENSE("Dual BSD/GPL"); 
MODULE_AUTHOR("Eillon");
MODULE_DESCRIPTION("do TTL test");
MODULE_VERSION("1.0");

[-- Attachment #3: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel