* [LTP] [PATCH] hugeshmat04: don't attach at specific address @ 2021-09-21 16:45 Jan Stancek 2021-09-22 6:34 ` Li Wang 2021-09-23 8:50 ` [LTP] [PATCH v2] hugeshmat04: try to find unmapped range for test Jan Stancek 0 siblings, 2 replies; 8+ messages in thread From: Jan Stancek @ 2021-09-21 16:45 UTC (permalink / raw) To: ltp, liwan The test intermittently fails on ppc64le, when heap happens to overlap with segment that the test is trying to attach at 1GB boundary. Let the kernel find suitable address. Signed-off-by: Jan Stancek <jstancek@redhat.com> --- testcases/kernel/mem/hugetlb/hugeshmat/hugeshmat04.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/testcases/kernel/mem/hugetlb/hugeshmat/hugeshmat04.c b/testcases/kernel/mem/hugetlb/hugeshmat/hugeshmat04.c index e9bb9fbf7b4b..75f61de066d7 100644 --- a/testcases/kernel/mem/hugetlb/hugeshmat/hugeshmat04.c +++ b/testcases/kernel/mem/hugetlb/hugeshmat/hugeshmat04.c @@ -22,7 +22,6 @@ #include "hugetlb.h" #define SIZE (1024 * 1024 * 1024) -#define BOUNDARY (1024 * 1024 * 1024) static long huge_free; static long huge_free2; @@ -54,7 +53,7 @@ static void shared_hugepage(void) if (shmid < 0) tst_brk(TBROK | TERRNO, "shmget"); - buf = shmat(shmid, (void *)BOUNDARY, SHM_RND | 0777); + buf = shmat(shmid, NULL, 0777); if (buf == (void *)-1) { shmctl(shmid, IPC_RMID, NULL); tst_brk(TBROK | TERRNO, "shmat"); -- 2.27.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [LTP] [PATCH] hugeshmat04: don't attach at specific address 2021-09-21 16:45 [LTP] [PATCH] hugeshmat04: don't attach at specific address Jan Stancek @ 2021-09-22 6:34 ` Li Wang 2021-09-22 14:29 ` Cyril Hrubis 2021-09-23 8:50 ` [LTP] [PATCH v2] hugeshmat04: try to find unmapped range for test Jan Stancek 1 sibling, 1 reply; 8+ messages in thread From: Li Wang @ 2021-09-22 6:34 UTC (permalink / raw) To: Jan Stancek; +Cc: LTP List [-- Attachment #1.1: Type: text/plain, Size: 500 bytes --] On Wed, Sep 22, 2021 at 12:46 AM Jan Stancek <jstancek@redhat.com> wrote: > The test intermittently fails on ppc64le, when heap > happens to overlap with segment that the test is trying > to attach at 1GB boundary. > > Let the kernel find suitable address. > I'm not sure why the original way uses fixed address, but it should be fine to cancel that limitation. Reviewed-by: Li Wang <liwang@redhat.com> Or, maybe we can consider doing range_is_mapped check before using it. -- Regards, Li Wang [-- Attachment #1.2: Type: text/html, Size: 1474 bytes --] [-- Attachment #2: Type: text/plain, Size: 60 bytes --] -- Mailing list info: https://lists.linux.it/listinfo/ltp ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [LTP] [PATCH] hugeshmat04: don't attach at specific address 2021-09-22 6:34 ` Li Wang @ 2021-09-22 14:29 ` Cyril Hrubis 2021-09-23 6:49 ` Li Wang 0 siblings, 1 reply; 8+ messages in thread From: Cyril Hrubis @ 2021-09-22 14:29 UTC (permalink / raw) To: Li Wang; +Cc: LTP List Hi! > > The test intermittently fails on ppc64le, when heap > > happens to overlap with segment that the test is trying > > to attach at 1GB boundary. > > > > Let the kernel find suitable address. > > > > I'm not sure why the original way uses fixed address, but > it should be fine to cancel that limitation. It seems to be regression test for: commit c5c99429fa57dcf6e05203ebe3676db1ec646793 Author: Larry Woodman <lwoodman@redhat.com> Date: Thu Jan 24 05:49:25 2008 -0800 fix hugepages leak due to pagetable page sharing The shared page table code for hugetlb memory on x86 and x86_64 is causing a leak. When a user of hugepages exits using this code the system leaks some of the hugepages. ------------------------------------------------------- Part of /proc/meminfo just before database startup: HugePages_Total: 5500 HugePages_Free: 5500 HugePages_Rsvd: 0 Hugepagesize: 2048 kB Just before shutdown: HugePages_Total: 5500 HugePages_Free: 4475 HugePages_Rsvd: 0 Hugepagesize: 2048 kB After shutdown: HugePages_Total: 5500 HugePages_Free: 4988 HugePages_Rsvd: 0 Hugepagesize: 2048 kB ---------------------------------------------------------- The problem occurs durring a fork, in copy_hugetlb_page_range(). It locates the dst_pte using huge_pte_alloc(). Since huge_pte_alloc() calls huge_pmd_share() it will share the pmd page if can, yet the main loop in copy_hugetlb_page_range() does a get_page() on every hugepage. This is a violation of the shared hugepmd pagetable protocol and creates additional referenced to the hugepages causing a leak when the unmap of the VMA occurs. We can skip the entire replication of the ptes when the hugepage pagetables are shared. The attached patch skips copying the ptes and the get_page() calls if the hugetlbpage pagetable is shared. As far as I can tell there is no reason to allocate the memory on a fixed address to trigger this bug but it should be easy enough to check if the test fails on a kernel that has the two lines this patch adds commented before and after the change it test. Also can we pretty please add the commit hash to the test metadata? -- Cyril Hrubis chrubis@suse.cz -- Mailing list info: https://lists.linux.it/listinfo/ltp ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [LTP] [PATCH] hugeshmat04: don't attach at specific address 2021-09-22 14:29 ` Cyril Hrubis @ 2021-09-23 6:49 ` Li Wang 0 siblings, 0 replies; 8+ messages in thread From: Li Wang @ 2021-09-23 6:49 UTC (permalink / raw) To: Cyril Hrubis; +Cc: LTP List [-- Attachment #1.1: Type: text/plain, Size: 761 bytes --] > It seems to be regression test for: > > commit c5c99429fa57dcf6e05203ebe3676db1ec646793 > Author: Larry Woodman <lwoodman@redhat.com> > Date: Thu Jan 24 05:49:25 2008 -0800 > > fix hugepages leak due to pagetable page sharing > > > As far as I can tell there is no reason to allocate the memory on a > fixed address to trigger this bug but it should be easy enough to check > if the test fails on a kernel that has the two lines this patch adds > commented before and after the change it test. > That commit was merged in v2.6.24, it too old kernel and changes a lot in the two lines, I guess it is not easy to test a kernel with revert that patch. > > Also can we pretty please add the commit hash to the test metadata? > +1 -- Regards, Li Wang [-- Attachment #1.2: Type: text/html, Size: 1882 bytes --] [-- Attachment #2: Type: text/plain, Size: 60 bytes --] -- Mailing list info: https://lists.linux.it/listinfo/ltp ^ permalink raw reply [flat|nested] 8+ messages in thread
* [LTP] [PATCH v2] hugeshmat04: try to find unmapped range for test 2021-09-21 16:45 [LTP] [PATCH] hugeshmat04: don't attach at specific address Jan Stancek 2021-09-22 6:34 ` Li Wang @ 2021-09-23 8:50 ` Jan Stancek 2021-09-23 9:04 ` Li Wang 2021-09-23 9:07 ` Cyril Hrubis 1 sibling, 2 replies; 8+ messages in thread From: Jan Stancek @ 2021-09-23 8:50 UTC (permalink / raw) To: ltp, liwan The test intermittently fails on ppc64le, when heap happens to overlap with segment that the test is trying to attach at 1GB boundary. Try to find unmapped range. In my testing, address above 1GB appears to be needed for reproducer to work. Also add kernel commit that fixed it to metadata. Signed-off-by: Jan Stancek <jstancek@redhat.com> --- .../kernel/mem/hugetlb/hugeshmat/hugeshmat04.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/testcases/kernel/mem/hugetlb/hugeshmat/hugeshmat04.c b/testcases/kernel/mem/hugetlb/hugeshmat/hugeshmat04.c index e9bb9fbf7b4b..b76da93a197e 100644 --- a/testcases/kernel/mem/hugetlb/hugeshmat/hugeshmat04.c +++ b/testcases/kernel/mem/hugetlb/hugeshmat/hugeshmat04.c @@ -23,6 +23,7 @@ #define SIZE (1024 * 1024 * 1024) #define BOUNDARY (1024 * 1024 * 1024) +#define BOUNDARY_MAX (3U * 1024 * 1024 * 1024) static long huge_free; static long huge_free2; @@ -49,12 +50,20 @@ static void shared_hugepage(void) int status, shmid; size_t size = (size_t)SIZE; void *buf; + unsigned long boundary = BOUNDARY; shmid = shmget(IPC_PRIVATE, size, SHM_HUGETLB | IPC_CREAT | 0777); if (shmid < 0) tst_brk(TBROK | TERRNO, "shmget"); - buf = shmat(shmid, (void *)BOUNDARY, SHM_RND | 0777); + while (boundary <= BOUNDARY_MAX + && range_is_mapped(boundary, boundary+SIZE)) + boundary += 128*1024*1024; + if (boundary > BOUNDARY_MAX) + tst_brk(TCONF, "failed to find free unmapped range"); + + tst_res(TINFO, "attaching at 0x%lx", boundary); + buf = shmat(shmid, (void *)boundary, SHM_RND | 0777); if (buf == (void *)-1) { shmctl(shmid, IPC_RMID, NULL); tst_brk(TBROK | TERRNO, "shmat"); @@ -104,6 +113,10 @@ static void cleanup(void) } static struct tst_test test = { + .tags = (const struct tst_tag[]) { + {"linux-git", "c5c99429fa57"}, + {} + }, .needs_root = 1, .forks_child = 1, .needs_tmpdir = 1, -- 2.27.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [LTP] [PATCH v2] hugeshmat04: try to find unmapped range for test 2021-09-23 8:50 ` [LTP] [PATCH v2] hugeshmat04: try to find unmapped range for test Jan Stancek @ 2021-09-23 9:04 ` Li Wang 2021-09-23 9:07 ` Cyril Hrubis 1 sibling, 0 replies; 8+ messages in thread From: Li Wang @ 2021-09-23 9:04 UTC (permalink / raw) To: Jan Stancek; +Cc: LTP List [-- Attachment #1.1: Type: text/plain, Size: 498 bytes --] On Thu, Sep 23, 2021 at 4:50 PM Jan Stancek <jstancek@redhat.com> wrote: > The test intermittently fails on ppc64le, when heap > happens to overlap with segment that the test is trying > to attach at 1GB boundary. > > Try to find unmapped range. In my testing, address above 1GB appears > to be needed for reproducer to work. Also add kernel commit that fixed > it to metadata. > > Signed-off-by: Jan Stancek <jstancek@redhat.com> > Reviewed-by: Li Wang <liwang@redhat.com> -- Regards, Li Wang [-- Attachment #1.2: Type: text/html, Size: 1185 bytes --] [-- Attachment #2: Type: text/plain, Size: 60 bytes --] -- Mailing list info: https://lists.linux.it/listinfo/ltp ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [LTP] [PATCH v2] hugeshmat04: try to find unmapped range for test 2021-09-23 8:50 ` [LTP] [PATCH v2] hugeshmat04: try to find unmapped range for test Jan Stancek 2021-09-23 9:04 ` Li Wang @ 2021-09-23 9:07 ` Cyril Hrubis 2021-09-24 3:15 ` Li Wang 1 sibling, 1 reply; 8+ messages in thread From: Cyril Hrubis @ 2021-09-23 9:07 UTC (permalink / raw) To: Jan Stancek; +Cc: liwan, ltp Hi! Great work. Reviewed-by: Cyril Hrubis <chrubis@suse.cz> -- Cyril Hrubis chrubis@suse.cz -- Mailing list info: https://lists.linux.it/listinfo/ltp ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [LTP] [PATCH v2] hugeshmat04: try to find unmapped range for test 2021-09-23 9:07 ` Cyril Hrubis @ 2021-09-24 3:15 ` Li Wang 0 siblings, 0 replies; 8+ messages in thread From: Li Wang @ 2021-09-24 3:15 UTC (permalink / raw) To: Cyril Hrubis; +Cc: LTP List [-- Attachment #1.1: Type: text/plain, Size: 270 bytes --] On Thu, Sep 23, 2021 at 5:07 PM Cyril Hrubis <chrubis@suse.cz> wrote: > Hi! > Great work. > > Reviewed-by: Cyril Hrubis <chrubis@suse.cz> I'm going to start a round of test for the latest branch before the release. So this patch merged. Thanks! -- Regards, Li Wang [-- Attachment #1.2: Type: text/html, Size: 946 bytes --] [-- Attachment #2: Type: text/plain, Size: 60 bytes --] -- Mailing list info: https://lists.linux.it/listinfo/ltp ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-09-24 3:15 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-09-21 16:45 [LTP] [PATCH] hugeshmat04: don't attach at specific address Jan Stancek 2021-09-22 6:34 ` Li Wang 2021-09-22 14:29 ` Cyril Hrubis 2021-09-23 6:49 ` Li Wang 2021-09-23 8:50 ` [LTP] [PATCH v2] hugeshmat04: try to find unmapped range for test Jan Stancek 2021-09-23 9:04 ` Li Wang 2021-09-23 9:07 ` Cyril Hrubis 2021-09-24 3:15 ` Li Wang
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.