From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 445AD6453 for ; Tue, 22 Mar 2022 21:41:57 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0D905C340EC; Tue, 22 Mar 2022 21:41:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1647985317; bh=tvkOCvNvu86WaZ7JJiDd5vDGRJ4/ZlmTN54lTdhKS5g=; h=Date:To:From:In-Reply-To:Subject:From; b=0lYBmuN9qqJ/SfjJeLRfxlYUpYI75Ko2Iarel0UBf6JAP05S+6lXtdkgONuDyGr38 EVU06pRUEq+0iP1ye1jgGp8UNz7m1vPh31xyPKN4ybUF7u3Rb8w9tsvP410HJV+uK0 tsY1gOHp82fp5SQ8pg2gqr5bEBzcXKjgffA2I1tw= Date: Tue, 22 Mar 2022 14:41:56 -0700 To: ziy@nvidia.com,rientjes@google.com,peterx@redhat.com,mike.kravetz@oracle.com,lars.persson@axis.com,kirill.shutemov@linux.intel.com,fam.zheng@bytedance.com,duanxiongchun@bytedance.com,axelrasmussen@google.com,songmuchun@bytedance.com,akpm@linux-foundation.org,patches@lists.linux.dev,linux-mm@kvack.org,mm-commits@vger.kernel.org,torvalds@linux-foundation.org,akpm@linux-foundation.org From: Andrew Morton In-Reply-To: <20220322143803.04a5e59a07e48284f196a2f9@linux-foundation.org> Subject: [patch 067/227] mm: fix missing cache flush for all tail pages of compound page Message-Id: <20220322214157.0D905C340EC@smtp.kernel.org> Precedence: bulk X-Mailing-List: patches@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: From: Muchun Song Subject: mm: fix missing cache flush for all tail pages of compound page The D-cache maintenance inside move_to_new_page() only consider one page, there is still D-cache maintenance issue for tail pages of compound page (e.g. THP or HugeTLB). THP migration is only enabled on x86_64, ARM64 and powerpc, while powerpc and arm64 need to maintain the consistency between I-Cache and D-Cache, which depends on flush_dcache_page() to maintain the consistency between I-Cache and D-Cache. But there is no issues on arm64 and powerpc since they already considers the compound page cache flushing in their icache flush function. HugeTLB migration is enabled on arm, arm64, mips, parisc, powerpc, riscv, s390 and sh, while arm has handled the compound page cache flush in flush_dcache_page(), but most others do not. In theory, the issue exists on many architectures. Fix this by not using flush_dcache_folio() since it is not backportable. Link: https://lkml.kernel.org/r/20220210123058.79206-3-songmuchun@bytedance.com Fixes: 290408d4a250 ("hugetlb: hugepage migration core") Signed-off-by: Muchun Song Reviewed-by: Zi Yan Cc: Axel Rasmussen Cc: David Rientjes Cc: Fam Zheng Cc: Kirill A. Shutemov Cc: Lars Persson Cc: Mike Kravetz Cc: Peter Xu Cc: Xiongchun Duan Signed-off-by: Andrew Morton --- mm/migrate.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) --- a/mm/migrate.c~mm-fix-missing-cache-flush-for-all-tail-pages-of-compound-page +++ a/mm/migrate.c @@ -916,9 +916,12 @@ static int move_to_new_page(struct page if (!PageMappingFlags(page)) page->mapping = NULL; - if (likely(!is_zone_device_page(newpage))) - flush_dcache_page(newpage); + if (likely(!is_zone_device_page(newpage))) { + int i, nr = compound_nr(newpage); + for (i = 0; i < nr; i++) + flush_dcache_page(newpage + i); + } } out: return rc; _ From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDD6AC433EF for ; Tue, 22 Mar 2022 21:42:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236075AbiCVVnq (ORCPT ); Tue, 22 Mar 2022 17:43:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50512 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236124AbiCVVnf (ORCPT ); Tue, 22 Mar 2022 17:43:35 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 16DC05F27F for ; Tue, 22 Mar 2022 14:41:58 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A7BAC6101E for ; Tue, 22 Mar 2022 21:41:57 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0D905C340EC; Tue, 22 Mar 2022 21:41:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1647985317; bh=tvkOCvNvu86WaZ7JJiDd5vDGRJ4/ZlmTN54lTdhKS5g=; h=Date:To:From:In-Reply-To:Subject:From; b=0lYBmuN9qqJ/SfjJeLRfxlYUpYI75Ko2Iarel0UBf6JAP05S+6lXtdkgONuDyGr38 EVU06pRUEq+0iP1ye1jgGp8UNz7m1vPh31xyPKN4ybUF7u3Rb8w9tsvP410HJV+uK0 tsY1gOHp82fp5SQ8pg2gqr5bEBzcXKjgffA2I1tw= Date: Tue, 22 Mar 2022 14:41:56 -0700 To: ziy@nvidia.com, rientjes@google.com, peterx@redhat.com, mike.kravetz@oracle.com, lars.persson@axis.com, kirill.shutemov@linux.intel.com, fam.zheng@bytedance.com, duanxiongchun@bytedance.com, axelrasmussen@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org, patches@lists.linux.dev, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, akpm@linux-foundation.org From: Andrew Morton In-Reply-To: <20220322143803.04a5e59a07e48284f196a2f9@linux-foundation.org> Subject: [patch 067/227] mm: fix missing cache flush for all tail pages of compound page Message-Id: <20220322214157.0D905C340EC@smtp.kernel.org> Precedence: bulk Reply-To: linux-kernel@vger.kernel.org List-ID: X-Mailing-List: mm-commits@vger.kernel.org From: Muchun Song Subject: mm: fix missing cache flush for all tail pages of compound page The D-cache maintenance inside move_to_new_page() only consider one page, there is still D-cache maintenance issue for tail pages of compound page (e.g. THP or HugeTLB). THP migration is only enabled on x86_64, ARM64 and powerpc, while powerpc and arm64 need to maintain the consistency between I-Cache and D-Cache, which depends on flush_dcache_page() to maintain the consistency between I-Cache and D-Cache. But there is no issues on arm64 and powerpc since they already considers the compound page cache flushing in their icache flush function. HugeTLB migration is enabled on arm, arm64, mips, parisc, powerpc, riscv, s390 and sh, while arm has handled the compound page cache flush in flush_dcache_page(), but most others do not. In theory, the issue exists on many architectures. Fix this by not using flush_dcache_folio() since it is not backportable. Link: https://lkml.kernel.org/r/20220210123058.79206-3-songmuchun@bytedance.com Fixes: 290408d4a250 ("hugetlb: hugepage migration core") Signed-off-by: Muchun Song Reviewed-by: Zi Yan Cc: Axel Rasmussen Cc: David Rientjes Cc: Fam Zheng Cc: Kirill A. Shutemov Cc: Lars Persson Cc: Mike Kravetz Cc: Peter Xu Cc: Xiongchun Duan Signed-off-by: Andrew Morton --- mm/migrate.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) --- a/mm/migrate.c~mm-fix-missing-cache-flush-for-all-tail-pages-of-compound-page +++ a/mm/migrate.c @@ -916,9 +916,12 @@ static int move_to_new_page(struct page if (!PageMappingFlags(page)) page->mapping = NULL; - if (likely(!is_zone_device_page(newpage))) - flush_dcache_page(newpage); + if (likely(!is_zone_device_page(newpage))) { + int i, nr = compound_nr(newpage); + for (i = 0; i < nr; i++) + flush_dcache_page(newpage + i); + } } out: return rc; _