From: Zi Yan <ziy@nvidia.com>
To: "Kirill A. Shutemov" <kirill@shutemov.name>
Cc: <akpm@linux-foundation.org>,
Andrea Arcangeli <aarcange@redhat.com>,
"Yang Shi" <yang.shi@linux.alibaba.com>, <linux-mm@kvack.org>,
<linux-kernel@vger.kernel.org>,
"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Subject: Re: [PATCHv2 1/8] khugepaged: Add self test
Date: Wed, 8 Apr 2020 11:53:52 -0400 [thread overview]
Message-ID: <C66E1309-2069-495B-BACD-7F3282C6EC7D@nvidia.com> (raw)
In-Reply-To: <20200408142145.workbzv2n3p27y2w@box>
[-- Attachment #1: Type: text/plain, Size: 10248 bytes --]
On 8 Apr 2020, at 10:21, Kirill A. Shutemov wrote:
> External email: Use caution opening links or attachments
>
>
> On Mon, Apr 06, 2020 at 02:50:32PM -0400, Zi Yan wrote:
>> khugepaged-58 [001] .... 9913.990380: mm_khugepaged_scan_pmd: mm=00000000283d31fc, scan_pfn=0x2ae4bd, writable=0, referenced=512, none_or_zero=0, status=no_writable_page, unmapped=0
>
> Looks like all failures due to lack of writable ptes. That's very strange
> because we write to the page on fill.
>
> You've mentioned that you run it in VM. I wounder if it can be a
> virtualizaiton artefact. I run tests under KVM and they are fine. What is
> your virtualization setup?
My qemu cmd: “qemu-system-x86_64 -kernel ~/repos/linux/arch/x86/boot/bzImage -hda ~/qemu-image/vm.qcow2 -append "root=/dev/sda1 rw console=ttyS0" -pidfile vm.pid -net user,hostfwd=tcp::11022-:22 -net nic -numa node,nodeid=0 -numa node,nodeid=1 -m 20g -smp 8 -cpu host -enable-kvm -nographic”
QEMU version is 4.2.0 (Debian 1:4.2-3)
The patches are applied on top of commit a10c9c710f9ecea87b9f4bbb837467893b4bef01 from Linus’s tree.
In addition, I tested it again on a bare metal, then all tests passed except “Collapse with max_ptes_swap pages swapped out”, which failed most of time but succeeded occasionally.
When it fails, the tracing info is (seems like the same PTE not writable issue):
# sudo ./trace.sh
Save THP and khugepaged settings... OK
Adjust settings... OK
Swapout 64 of 512 pages... OK
Collapse with max_ptes_swap pages swapped out.... Fail
Restore THP and khugepaged settings... OK
# tracer: nop
#
# entries-in-buffer/entries-written: 42/42 #P:48
#
# _-----=> irqs-off
# / _----=> need-resched
# | / _---=> hardirq/softirq
# || / _--=> preempt-depth
# ||| / delay
# TASK-PID CPU# |||| TIMESTAMP FUNCTION
# | | | |||| | |
khugepaged-265 [002] ...1 1007.308366: mm_collapse_huge_page_swapin: mm=000000007c6cc479, swapped_in=64, referenced=448, ret=1
khugepaged-265 [002] ...2 1007.308403: mm_collapse_huge_page_isolate: scan_pfn=0x1faa6ee, none_or_zero=0, referenced=60, writable=0, status=not_suitable_page_count
khugepaged-265 [002] ...1 1007.308404: mm_collapse_huge_page: mm=000000007c6cc479, isolated=0, status=failed
khugepaged-265 [002] ...1 1007.308405: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1fafa45, writable=1, referenced=448, none_or_zero=0, status=succeeded, unmapped=64
khugepaged-265 [002] ...1 1007.328249: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.344263: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.360263: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.376241: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.392237: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.408284: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.424302: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.440284: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.456285: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.472280: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.488282: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.504264: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.520293: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.536282: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.552278: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.568302: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.584290: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.600300: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.616292: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.632290: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.648304: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.664305: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.680289: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.696294: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.712294: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.728309: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.744291: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.760288: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.776312: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.792288: mm_khugepaged_scan_pmd: mm=000000007c6cc479, scan_pfn=0x1faa6ee, writable=0, referenced=60, none_or_zero=0, status=not_suitable_page_count, unmapped=0
khugepaged-265 [002] ...1 1007.811482: mm_khugepaged_scan_pmd: mm=000000006aaca29d, scan_pfn=0xffffffffffffffff, writable=0, referenced=0, none_or_zero=0, status=pmd_null, unmapped=0
khugepaged-265 [002] ...1 1007.811482: mm_khugepaged_scan_pmd: mm=000000006aaca29d, scan_pfn=0xffffffffffffffff, writable=0, referenced=0, none_or_zero=0, status=pmd_null, unmapped=0
khugepaged-265 [002] ...1 1007.811483: mm_khugepaged_scan_pmd: mm=000000006aaca29d, scan_pfn=0xffffffffffffffff, writable=0, referenced=0, none_or_zero=0, status=pmd_null, unmapped=0
khugepaged-265 [002] ...1 1007.811483: mm_khugepaged_scan_pmd: mm=000000006aaca29d, scan_pfn=0xffffffffffffffff, writable=0, referenced=0, none_or_zero=0, status=pmd_null, unmapped=0
khugepaged-265 [002] ...1 1007.811483: mm_khugepaged_scan_pmd: mm=000000006aaca29d, scan_pfn=0xffffffffffffffff, writable=0, referenced=0, none_or_zero=0, status=pmd_null, unmapped=0
khugepaged-265 [002] ...1 1007.811484: mm_khugepaged_scan_pmd: mm=000000006aaca29d, scan_pfn=0xffffffffffffffff, writable=0, referenced=0, none_or_zero=0, status=pmd_null, unmapped=0
khugepaged-265 [002] ...1 1007.811502: mm_khugepaged_scan_pmd: mm=000000005268d9a7, scan_pfn=0xffffffffffffffff, writable=0, referenced=0, none_or_zero=0, status=pmd_null, unmapped=0
khugepaged-265 [002] ...1 1007.811502: mm_khugepaged_scan_pmd: mm=000000005268d9a7, scan_pfn=0xffffffffffffffff, writable=0, referenced=0, none_or_zero=0, status=pmd_null, unmapped=0
—
Best Regards,
Yan Zi
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 854 bytes --]
next prev parent reply other threads:[~2020-04-08 15:54 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-04-03 11:29 [PATCHv2 0/8] thp/khugepaged improvements and CoW semantics Kirill A. Shutemov
2020-04-03 11:29 ` [PATCHv2 1/8] khugepaged: Add self test Kirill A. Shutemov
2020-04-06 14:59 ` Zi Yan
2020-04-06 15:20 ` Kirill A. Shutemov
2020-04-06 18:50 ` Zi Yan
2020-04-08 14:21 ` Kirill A. Shutemov
2020-04-08 15:53 ` Zi Yan [this message]
2020-04-10 11:47 ` Kirill A. Shutemov
2020-04-10 14:36 ` Zi Yan
2020-04-10 14:58 ` Kirill A. Shutemov
2020-04-10 15:03 ` Zi Yan
2020-04-06 18:53 ` Ralph Campbell
2020-04-03 11:29 ` [PATCHv2 2/8] khugepaged: Do not stop collapse if less than half PTEs are referenced Kirill A. Shutemov
2020-04-06 18:13 ` Yang Shi
2020-04-06 19:53 ` Ralph Campbell
2020-04-09 13:34 ` Kirill A. Shutemov
2020-04-03 11:29 ` [PATCHv2 3/8] khugepaged: Drain all LRU caches before scanning pages Kirill A. Shutemov
2020-04-06 18:15 ` Yang Shi
2020-04-03 11:29 ` [PATCHv2 4/8] khugepaged: Drain LRU add pagevec after swapin Kirill A. Shutemov
2020-04-06 13:11 ` Zi Yan
2020-04-06 18:29 ` Yang Shi
2020-04-08 13:05 ` Kirill A. Shutemov
2020-04-08 18:42 ` Yang Shi
2020-04-03 11:29 ` [PATCHv2 5/8] khugepaged: Allow to callapse a page shared across fork Kirill A. Shutemov
2020-04-06 20:15 ` Ralph Campbell
2020-04-06 20:50 ` Yang Shi
2020-04-08 13:10 ` Kirill A. Shutemov
2020-04-08 18:51 ` Yang Shi
2020-04-10 0:03 ` Yang Shi
2020-04-10 15:56 ` Kirill A. Shutemov
2020-04-06 21:30 ` John Hubbard
2020-04-10 15:55 ` Kirill A. Shutemov
2020-04-10 20:59 ` John Hubbard
2020-04-13 9:42 ` Kirill A. Shutemov
2020-04-03 11:29 ` [PATCHv2 6/8] khugepaged: Allow to collapse PTE-mapped compound pages Kirill A. Shutemov
2020-04-06 21:29 ` Yang Shi
2020-04-08 13:29 ` Kirill A. Shutemov
2020-04-08 18:57 ` Yang Shi
2020-04-09 13:47 ` Kirill A. Shutemov
2020-04-03 11:29 ` [PATCHv2 7/8] thp: Change CoW semantics for anon-THP Kirill A. Shutemov
2020-04-07 7:57 ` [thp] db001b7115: vm-scalability.median 8.9% improvement kernel test robot
2020-04-03 11:29 ` [PATCHv2 8/8] khugepaged: Introduce 'max_ptes_shared' tunable Kirill A. Shutemov
2020-04-06 13:17 ` Zi Yan
2020-04-05 23:40 ` [PATCHv2 0/8] thp/khugepaged improvements and CoW semantics William Kucharski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=C66E1309-2069-495B-BACD-7F3282C6EC7D@nvidia.com \
--to=ziy@nvidia.com \
--cc=aarcange@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=kirill.shutemov@linux.intel.com \
--cc=kirill@shutemov.name \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=yang.shi@linux.alibaba.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).