On 27 Mar 2019, at 10:05, Dave Hansen wrote:

> On 3/27/19 10:00 AM, Zi Yan wrote:
>> I ask this because I observe that migrating a list of pages can
>> achieve higher throughput compared to migrating individual page.
>> For example, migrating 512 4KB pages can achieve ~750MB/s
>> throughput, whereas migrating one 4KB page might only achieve
>> ~40MB/s throughput. The experiments were done on a two-socket
>> machine with two Xeon E5-2650 v3 @ 2.30GHz across the QPI link.
>
> What kind of migration?
>
> If you're talking about doing sys_migrate_pages() one page at a time,
> that's a world away from doing something inside of the kernel one page
> at a time.

For 40MB/s vs 750MB/s, they were using sys_migrate_pages(). Sorry about
the confusion there. As I measure only the migrate_pages() in the kernel,
the throughput becomes:
migrating 4KB page: 0.312GB/s vs migrating 512 4KB pages: 0.854GB/s.
They are still >2x difference.

Furthermore, if we only consider the migrate_page_copy() in mm/migrate.c,
which only calls copy_highpage() and migrate_page_states(), the throughput
becomes:
migrating 4KB page: 1.385GB/s vs migrating 512 4KB pages: 1.983GB/s.
The gap is smaller, but migrating 512 4KB pages still achieves 40% more
throughput.

Do these numbers make sense to you?

--
Best Regards,
Yan Zi