From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9432CC5519F for ; Wed, 25 Nov 2020 08:52:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A4588208CA for ; Wed, 25 Nov 2020 08:52:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="qDITTvUb" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A4588208CA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.ibm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C88876B0071; Wed, 25 Nov 2020 03:52:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C396F6B0072; Wed, 25 Nov 2020 03:52:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B4DC76B0073; Wed, 25 Nov 2020 03:52:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0184.hostedemail.com [216.40.44.184]) by kanga.kvack.org (Postfix) with ESMTP id 9F2C16B0071 for ; Wed, 25 Nov 2020 03:52:04 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 5A66B8249980 for ; Wed, 25 Nov 2020 08:52:04 +0000 (UTC) X-FDA: 77522323368.18.coat03_520e6d627375 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin18.hostedemail.com (Postfix) with ESMTP id 37C21100ED0E0 for ; Wed, 25 Nov 2020 08:52:04 +0000 (UTC) X-HE-Tag: coat03_520e6d627375 X-Filterd-Recvd-Size: 10617 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Wed, 25 Nov 2020 08:52:03 +0000 (UTC) Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 0AP8Ww3b121629; Wed, 25 Nov 2020 03:52:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=date : from : to : cc : subject : message-id : references : mime-version : content-type : content-transfer-encoding : in-reply-to; s=pp1; bh=bIACh/j+GhlP0sBA2dftWi7Yj0hkyMLwDOpOtW5GpYY=; b=qDITTvUbff0KMpCmv1LamKx/5hy7TDuCPeAiEuwPBb0qt2u4J/Pk3IFq2faA6yPjZySw RfLXSmveq5RwMcnf3vyn1/is9SVQPXfSFUsuDFGtgPD2wynapaMPfaXmFoJytrpIYDrI T7Bfl544R4+RMkr+ExkfeQK1E+qmoTY+lN/m3cLiKHfIPjhMjo0KbuKVUOX2CRuDXpx/ bqpghkedpQXzDuCJfhYpz0ucPPNNx9DlbWmGlXbN3KSa9sdu4ZMZe1xNCyJp01JrRpmL kgsXVYn94eb09EoflSBpe7u/AHMdQOgDK7HNVIYkabY7QnGlW312hUXDh/xE3SA0fupI KA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 351ddb2r2f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Nov 2020 03:52:01 -0500 Received: from m0098396.ppops.net (m0098396.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 0AP8XdCf126680; Wed, 25 Nov 2020 03:52:00 -0500 Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0a-001b2d01.pphosted.com with ESMTP id 351ddb2r1d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Nov 2020 03:52:00 -0500 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 0AP8mRjI005258; Wed, 25 Nov 2020 08:51:58 GMT Received: from b06avi18878370.portsmouth.uk.ibm.com (b06avi18878370.portsmouth.uk.ibm.com [9.149.26.194]) by ppma06ams.nl.ibm.com with ESMTP id 34xt5hcg1f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Nov 2020 08:51:58 +0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0AP8puV060621230 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 25 Nov 2020 08:51:56 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E69E0A4057; Wed, 25 Nov 2020 08:51:54 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E5788A4065; Wed, 25 Nov 2020 08:51:53 +0000 (GMT) Received: from linux.ibm.com (unknown [9.145.183.229]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTPS; Wed, 25 Nov 2020 08:51:53 +0000 (GMT) Date: Wed, 25 Nov 2020 10:51:51 +0200 From: Mike Rapoport To: David Hildenbrand Cc: Andrea Arcangeli , Vlastimil Babka , Mel Gorman , Andrew Morton , linux-mm@kvack.org, Qian Cai , Michal Hocko , linux-kernel@vger.kernel.org, Baoquan He Subject: Re: [PATCH 1/1] mm: compaction: avoid fast_isolate_around() to set pageblock_skip on reserved pages Message-ID: <20201125085151.GH123287@linux.ibm.com> References: <35F8AADA-6CAA-4BD6-A4CF-6F29B3F402A4@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <35F8AADA-6CAA-4BD6-A4CF-6F29B3F402A4@redhat.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312,18.0.737 definitions=2020-11-25_04:2020-11-25,2020-11-25 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxlogscore=999 suspectscore=1 lowpriorityscore=0 impostorscore=0 clxscore=1011 mlxscore=0 spamscore=0 malwarescore=0 bulkscore=0 adultscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2011250051 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Nov 25, 2020 at 07:45:30AM +0100, David Hildenbrand wrote: >=20 > > Am 25.11.2020 um 06:34 schrieb Andrea Arcangeli = : > >=20 > > =EF=BB=BFHello, > >=20 > >> On Mon, Nov 23, 2020 at 02:01:16PM +0100, Vlastimil Babka wrote: > >>> On 11/21/20 8:45 PM, Andrea Arcangeli wrote: > >>> A corollary issue was fixed in > >>> 39639000-39814fff : Unknown E820 type > >>>=20 > >>> pfn 0x7a200 -> 0x7a200000 min_pfn hit non-RAM: > >>>=20 > >>> 7a17b000-7a216fff : Unknown E820 type > >>=20 > >> It would be nice to also provide a /proc/zoneinfo and how exactly th= e=20 > >> "zone_spans_pfn" was violated. I assume we end up below zone's=20 > >> start_pfn, but is it true? > >=20 > > Agreed, I was about to grab that info along with all page struct > > around the pfn 0x7a200 and phys address 0x7a216fff. > >=20 > > # grep -A1 E820 /proc/iomem > > 7a17b000-7a216fff : Unknown E820 type > > 7a217000-7bffffff : System RAM > >=20 > > DMA zone_start_pfn 1 zone_end_pfn() 4096 cont= iguous 1 =20 > > DMA32 zone_start_pfn 4096 zone_end_pfn() 1048576 cont= iguous 0 =20 > > Normal zone_start_pfn 1048576 zone_end_pfn() 4715392 cont= iguous 1 =20 > > Movable zone_start_pfn 0 zone_end_pfn() 0 cont= iguous 0 =20 > >=20 > > 500222 0x7a1fe000 0x1fff000000001000 reserved True > > 500223 0x7a1ff000 0x1fff000000001000 reserved True > >=20 > > # I suspect "highest pfn" was somewhere in the RAM range > > # 0x7a217000-0x7a400000 and the pageblock_start_pfn(pfn) > > # made highest point to pfn 0x7a200 physaddr 0x7a200000 > > # below, which is reserved indeed since it's non-RAM > > # first number is pfn hex(500224) =3D=3D 0x7a200 > >=20 > > pfn physaddr page->flags > > 500224 0x7a200000 0x1fff000000001000 reserved True > > 500225 0x7a201000 0x1fff000000001000 reserved True > > *snip* > > 500245 0x7a215000 0x1fff000000001000 reserved True > > 500246 0x7a216000 0x1fff000000001000 reserved True > > 500247 0x7a217000 0x3fff000000000000 reserved False > > 500248 0x7a218000 0x3fff000000000000 reserved False > >=20 > > All RAM pages non-reserved are starting at 0x7a217000 as expected. > >=20 > > The non-RAM page_zonenum(pfn_to_page(0x7a200)) points to ZONE_DMA and= =20 > > page_zone(page) below was the DMA zone despite the pfn of 0x7a200 is > > in DMA32. > >=20 > > VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page); > >=20 > > So the patch I sent earlier should prevent the above BUG_ON by not > > setting highest to 0x7a200 when pfn is in the phys RAM range > > 0x7a217000-0x7a400000, because pageblock_pfn_to_page will notice that > > the zone is the wrong one. > >=20 > > if (page_zone(start_page) !=3D zone) > > return NULL; > >=20 > > However the real bug seems that reserved pages have a zero zone_id in > > the page->flags when it should have the real zone id/nid. The patch I > > sent earlier to validate highest would only be needed to deal with > > pfn_valid. > >=20 > > Something must have changed more recently than v5.1 that caused the > > zoneid of reserved pages to be wrong, a possible candidate for the > > real would be this change below: > >=20 > > + __init_single_page(pfn_to_page(pfn), pfn, 0, 0); > >=20 >=20 > Before that change, the memmap of memory holes were only zeroed out. > So the zones/nid was 0, however, pages were not reserved and had a > refcount of zero - resulting in other issues. >=20 > Most pfn walkers shouldn=E2=80=98t mess with reserved pages and simply = skip > them. That would be the right fix here. My guest would be that it is me and Baoquan: 73a6e474cb37 ("mm: memmap_init: iterate over memblock regions rather that= check each PFN") Until then reserved pages were traversed in memmap_init_zone() and after the change they are not because on x86 reserved memory is not considered memory for some reason. Can you please check if this untested patch helps: diff --git a/mm/page_alloc.c b/mm/page_alloc.c index eaa227a479e4..be3c85a6714e 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6191,7 +6191,9 @@ void __meminit __weak memmap_init(unsigned long siz= e, int nid, { unsigned long start_pfn, end_pfn; unsigned long range_end_pfn =3D range_start_pfn + size; + phys_addr_t start, end; int i; + u64 j; =20 for_each_mem_pfn_range(i, nid, &start_pfn, &end_pfn, NULL) { start_pfn =3D clamp(start_pfn, range_start_pfn, range_end_pfn); @@ -6203,6 +6205,19 @@ void __meminit __weak memmap_init(unsigned long si= ze, int nid, MEMINIT_EARLY, NULL, MIGRATE_MOVABLE); } } + + __for_each_mem_range(j, &memblock.reserved, NULL, nid, MEMBLOCK_NONE, + &start, &end, NULL) { + start_pfn =3D clamp(PHYS_PFN(start), range_start_pfn, + range_end_pfn); + end_pfn =3D clamp(PHYS_PFN(end), range_start_pfn, range_end_pfn); + + if (end_pfn > start_pfn) { + size =3D end_pfn - start_pfn; + memmap_init_zone(size, nid, zone, start_pfn, + MEMINIT_EARLY, NULL, MIGRATE_MOVABLE); + } + } } =20 static int zone_batchsize(struct zone *zone) > > Even if it may not be it, at the light of how the reserved page > > zoneid/nid initialized went wrong, the above line like it's too flake= y > > to stay. > >=20 > > It'd be preferable if the pfn_valid fails and the > > pfn_to_section_nr(pfn) returns an invalid section for the intermediat= e > > step. Even better memset 0xff over the whole page struct until the > > second stage comes around. >=20 > I recently discussed with Baoquan to > 1. Using a new pagetype to mark memory holes > 2. Using special nid/zid to mark memory holes >=20 > Setting the memmap to 0xff would be even more dangerous - pfn_zone() mi= ght simply BUG_ON. >=20 > >=20 > > Whenever pfn_valid is true, it's better that the zoneid/nid is correc= t > > all times, otherwise if the second stage fails we end up in a bug wit= h > > weird side effects. >=20 > Memory holes with a valid memmap might not have a zone/nid. For now, sk= ipping reserved pages should be good enough, no? >=20 > >=20 > > Maybe it's not the above that left a zero zoneid though, I haven't > > tried to bisect it yet to look how the page->flags looked like on a > > older kernel that didn't seem to reproduce this crash, I'm just > > guessing. > >=20 > > Thanks, > > Andrea >=20 --=20 Sincerely yours, Mike.